bonigv
Joined 12 years ago
Karma
3
Submissions
25
Member since
Sep 15
Recent submissions
Comment · 4 months ago
Comment · 4 months ago
You are absolutely right about what we feel intuitively - LSPs should beat the shit out of the competition. But surprisingly it did not. Across 10 different LSP servers, across 5 different levels of prompt complexity it did not. Mind you, I painstakingly warmed up the LSP servers that needed it warmed. Some liked it cold and it fared equally non impressively. The pattern I saw was, LLMs (sonnet w.6 with cc) was very clever to use whatever it had to get to a verifiable answer. It could do it just with bash for sure. But as the prompt complexity grew the cost also rose.
Treesitter is sitting in a sweet spot here. a vrainy LLM can find the shortest path with high quality with treesitter and a few bash calls.
Comment · 4 months ago
oh! I thought this was like bash w/o verbosity. That is a tool I would love to see. bash4llm . May be llike Gandhi said - I should be the change you want to see!
Comment · 4 months ago
Bruno, if we are operating the agent (assuming coding-agent ) in an environment where the build and test tools as well the source code lives, why would we need an additional step to supply context? Wouldn't even the most basic of agents eb able to operate on those tools and build the discovery themselves?
Comment · 4 months ago
Agree. The challenge is that same model/provider can use dramatically different numbers for thefirst two items. Maintainability is an invariant of quality of the system. Measuring that is a good metric for sure.
Comment · 4 months ago
Agree that LOC and Story points are not metrics that needs tracking. What is happening is that same model and provider can use a very different experience in terms of [Turns, Tokens, Time and to certain extent the results]. This is expected and to some extent acceptable. In this scenario how will we write a test case? What is success? What moves the needle to say that a commit made the software perform 20% better?
Comment · 4 months ago
This is a fascinating thought. I agree. It is one thing to let it write a prompt. Prompting, Generating, Judging and going till it is satiated is gluttony.
Every week I ship 4-5 sprints' worth of work that improves features, simplifies code, adds maintainability while delivering value to my end users without feeling I butchered a software hand-engineered over 10 years. This is one product. Mentioning it as an example because it got built over the years starting 2010. It has ~7000 paying customers and works in the B2B space. The B2B SaaS space is brutal since the defendable value should be delivered with margins two levels deep. I wouldn't be able to do it w/o coding agents.
BUT I will never do it with LOOP ENGINEERING. Not in 2026 and not in 2036. Software Engineering as such is not going to be much different in 2036 than it is now.
What stops me is the accountability that lies with me in delivered software. Setting lofty goals, a few guardrails and then letting harness and agent rip it overnight can (and have) created slop.
I did not find LLMs systematically generate poor code. What is poor? Poor in traditional handcraft coding days is defined by
- Poor performance - Verbosity - Complexity - Hardcoding - Poor maintainability
These are guardrails that the models understand. However it cannot understand all these upfront - when building new. It always needs a retrospection cycle and a token budget to refactor.
This is tech debt which we all carried and solved. It is just at an exponential scale because - well, the generation is happening at a super human scale.
So, either build the harness with the tech debt coverage cycles or take on ourselves to add that cycle triggered by human judgement.
A few things that are working for me and I trust are:
Forge CLI - the harness I use with a built-in engineering philosophy for everyday work [https://github.com/Entelligentsia/forge-cli]
A few patterns that I have known to trust for building software that can be maintained - expressed as skills [https://github.com/Entelligentsia/skillforge/tree/main/desig...]
A few patterns for the new probabilistic constructs we build with in 2026 LLM Patterns [https://github.com/Entelligentsia/skillforge/tree/main/desig...] Harness Patterns [https://github.com/Entelligentsia/skillforge/tree/main/harne...]
The harness gives me a process. Patterns give me architectural guardrails as I could enforce (with a prayer). But still read code. We gotta.
Last month microsoft published an agent reference implementation where the Coding Agent delegated the exploration to a local agent that used a local light LLM. There was an arXiv paper , a github repo and a custom tarined model. Later in the month all these resources got pulled from the internet without any explanation.
So I tested the setup all over again to understand better. The model was poor and unreliable. The repo was duct taped at the best. The tests it ran were very poor quality explorations on small codebases.
But the architecture genuinely looked interesting. So I ported it to Grove as a fresh rewrite. Ran it on real large codebases.