The variance mostly comes down to prompt craft and context management.
People who get consistently good results have usually internalized a few things: (1) being explicit about constraints and output format, (2) providing relevant context without noise, (3) matching the model to the task (reasoning-heavy vs creative vs code), and (4) iterating on the prompt when something fails rather than assuming the model is broken.
I've seen the same person get wildly different results depending on whether they ask "write code to do X" vs "I need a function that takes A, returns B, handles edge case C, and should be optimized for D. Here's the existing code it needs to integrate with: [context]."
The gap between those two approaches can be a 10x difference in usefulness. Most of the "LLMs are useless" crowd and the "LLMs are magic" crowd are just working with very different prompt habits.
People who get consistently good results have usually internalized a few things: (1) being explicit about constraints and output format, (2) providing relevant context without noise, (3) matching the model to the task (reasoning-heavy vs creative vs code), and (4) iterating on the prompt when something fails rather than assuming the model is broken.
I've seen the same person get wildly different results depending on whether they ask "write code to do X" vs "I need a function that takes A, returns B, handles edge case C, and should be optimized for D. Here's the existing code it needs to integrate with: [context]."
The gap between those two approaches can be a 10x difference in usefulness. Most of the "LLMs are useless" crowd and the "LLMs are magic" crowd are just working with very different prompt habits.