codegladiator

    - Only Use Attempto Controlled English
    - Do not make any sentence in negative phrasing

All instructions are vague unless its code. But you can also give llm "code" and expect vague outcomes if you ask it to emulate what the runtime would look like.

like if someone says "is this car reliable", i dont expect an infallible car. but if the cars third gear is 'you know sometimes it doesnt work', i wouldnt take that car out of city.

if dsl gets large, the number of potential interactions your dsl allow will grow exponentially (unless you are building an one dimensional action layer). and there will be semantic issues unless your dsl is "clear and intuitive", also comprehensive enough to accomodate your ongoing changes, else every change is now 2 changes.

so you now you need a comprehensive manual for your agent which needs to be sent in every /completion request.

semantics are defined by the converter/compiler/interpreter, and that is the process which is going to consume the said json/yaml. if the json/yaml is going to be consumed by any process then the semantics are inherently defined, so by your definition all jsons/yamls are in their own a "language" (or they are not being consumed at all), which just defeats the purpose of calling it a language at that point.

you are probably talking of some coding harness which looked up the existing code base and then made it and not something like first prompt to llm "write xyz testcase in my systems company dsl in tcl from early 2000".

> LLMs can write it fine

sure, coding by examples is fine (it goes back to "system prompt describing the language"). but the claim we are arguing is reliability. did the llm generate 100% test case or code after reading your existing codebase, likely not as you mention it "almost acceptably on first sight".

i would probably be in denial if was suggesting llms not good at finding and fitting patterns.