How do you view harness engineering as an organic development that emerges from its use within a specific domain? Basically the meta-loop that allows an agent to tailor its harness to improve outcomes based on performance feedback. I use Pi a lot and I'm very interested in "self-assembling software".
One concrete example might be maintaining a conventions document per-project that covers how to name things semantically from a list of nouns and verbs. The idea is that LLMs are often not very globally aware, but it's important to maintain coherence across a code base in order for it to scale (in size and over time). Sometimes an LLM might call the same concept a Materialization, sometimes a Projection, and its not useful if its using two terms interchangeably without purpose.
Basically, how are you maintaining coherence when there isn't a human steering the code beyond providing requirements and validation directives?
One concrete example might be maintaining a conventions document per-project that covers how to name things semantically from a list of nouns and verbs. The idea is that LLMs are often not very globally aware, but it's important to maintain coherence across a code base in order for it to scale (in size and over time). Sometimes an LLM might call the same concept a Materialization, sometimes a Projection, and its not useful if its using two terms interchangeably without purpose.
Basically, how are you maintaining coherence when there isn't a human steering the code beyond providing requirements and validation directives?
I see you have relevant context in the repo like https://github.com/lopopolo/harness-engineering/tree/trunk/d... but I'm curious what exists beyond context. Do you use any tooling to steer this type of thing more consistently?