kkjh0723

Few additional comments for the evaluation. 1) I also run random and bearish bots. The score of the random bots actually converges to zero as expected. 2) You can check the 95% confidence interval and the number of resolved predictions that the identity have submitted (small circle next to the score). To tell the truth, CI of all the AI bots currently include zero.

Prediction is up-or-down but the score is not binary. Actually the score is "the return you would have made if you take that position (assume up is long, down is short)". If you predict a asset "up" and the asset actually rise by 2% then you get +log(1.02) while you loose if the asset fell by 2%. Additionally, the return is divided by the period (timeframe you choose) to annualize it. Also, a constant is added to the denominator to adjust for luck with small sample.

I use Claude Max, and I like that Fable can write code and run experiments on its own. I just check in occasionally to see how things are going and suggest the next direction. I expected Claude to be the clear favorite here, but there are more Codex users than I thought, so now I’m tempted to give it a try. That said, GPT translated this comment for me.