When you're working on something difficult that requires a model to reason intelligently, lower level and strongly typed languages often outperform on the same problems [0]. We have a few hypotheses about why, with a moderately high correlation between performance and token density of the output program -- i.e. more token dense languages are more difficult for programs to reason about.
Most models come up with the least effective solutions when writing Python.
Most models come up with the least effective solutions when writing Python.
[0] https://gertlabs.com/rankings