Comment

Comment by w4

Because of benchmarking LLMs have also been pushed towards fluency in Python, and related frameworks like Django and Flask. For example, SWE-Bench Verified is nearly 50% Django framework PR tasks: https://epoch.ai/blog/what-skills-does-swe-bench-verified-ev...

It will be interesting to see how durable these biases are as labs work towards developing more capable small models that are less reliant on memorized information. My naive instinct is that these biases will be less salient over time as context windows improve and models become increasingly capable of processing documentation as a part of their code writing loop, but also that, in the absence of instruction to the contrary, the models will favor working with these tools as a default for quite some time.

parent w4 1 year ago

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton