wyre
Joined 7 years ago
Joined 7 years ago
however I still get the same issues with sol getting more apologetic and is more likely to stop working when i tell it is wrong, instead of fixing course and trudging ahead.
OAI is not simply prompting "Prove exponential time hypothesis false. Make no mistakes" for the same reasons you can't one-shot "Write a fully standards compliant internet browser from scratch"
AAv5.0: 25% Knowledge work: AA-Briefcase v1.1 - 15% AutomationBench-AA - 5% GDP.pdf - 5% (-5%) 25% Coding: FrontierCode 2.0 - 15% (new) Terminal-Bench4.0 - 10% 25% Science: Terminal-Bench Science .1- 15% (new) CritPt v2 (hard subset) - 10% (new version) 25% General: AA-Omniscience v1.1 - 15% HLE-Diamond - 10%
AAv4.3: 30% Agents: AA-Briefcase v1.1 - 15% GDPval-AA v2 - 10% AutomationBench-AA - 5% 20% Coding: Terminal-Bench4.0 - 10% SciCode - 10% 30% General: AA-Omniscience (Accuracy) - 10% AA-Omniscience (Non-hallucination) - 5% GDP.pdf - 10% AA-LCR v1.1 - 5% 20% Scientific Reasoning: HLE - 10% CritPt 10%
tldr: gained Terminal-Bench Science and FrontierCode, dropped SciCode, GDPval-AA and AA-LCR. Lowered weight of GDP.pdf.
No statistics on tokens usage or wall time?
What is it about this MBP that makes it so much more preferable to an MBP with M3 Pro/Max?
Talk to me about anything: [email protected]