Comment by nasretdinov

Unless something dramatic changes and it'll become cheap to "constantly re-educate" the models to learn the new fundamental facts about the system (ideally with a small lag of maybe a few days like with humans) I see no reason to think that LLMs can have a profound understanding of how business systems work of a remotely comparable quality to humans. Just adding the context about the system already reduces it's capacity significantly according to what I can tell. Not having large context helps somewhat at a cost of having to explain lots of task-relevant things to LLM over and over again...