Context is the most challenging bit. FWIW, the codebases I'm working on are still small enough to where I rarely need to include more than 12 files into context. And I find as I make the context bigger beyond that, results degrade significantly.
So I don't know how this would go in a much larger codebase.
What floored him was simply how much of my programming I was doing with an LLM / how little I write line-by-line (vs edit line-by-line).
If you're really curious, I recorded some work for a friend. The first video has terrible audio, unfortunately. This second one I think gives a very realistic demonstration – you'll see the model struggle a bit at the beginning:
So I don't know how this would go in a much larger codebase.
What floored him was simply how much of my programming I was doing with an LLM / how little I write line-by-line (vs edit line-by-line).
If you're really curious, I recorded some work for a friend. The first video has terrible audio, unfortunately. This second one I think gives a very realistic demonstration – you'll see the model struggle a bit at the beginning:
https://www.loom.com/share/20d967be827141578c64074735eb84a8