Comment Comment by croddin There is still plenty of room for growth on the ARC-AGI benchmarks. ARC-AGI 2 is still <5% for o3-pro and ARC-AGI 1 is only at 59% for o3-pro-high:"ARC-AGI-1: * Low: 44%, $1.64/task * Medium: 57%, $3.18/task * High: 59%, $4.16/taskARC-AGI-2: * All reasoning efforts: <5%, $4-7/taskTakeaways: * o3-pro in line with o3 performance * o3's new price sets the ARC-AGI-1 Frontier"- https://x.com/arcprize/status/1932535378080395332 parent croddin 1 year ago
"ARC-AGI-1: * Low: 44%, $1.64/task * Medium: 57%, $3.18/task * High: 59%, $4.16/task
ARC-AGI-2: * All reasoning efforts: <5%, $4-7/task
Takeaways: * o3-pro in line with o3 performance * o3's new price sets the ARC-AGI-1 Frontier"
- https://x.com/arcprize/status/1932535378080395332