Anecdotally, for small tools and projects, it can crank out functional tools that would take me days or weeks to remember how to do manually. Every time I ask LLM to write something, it also writes tests and validates its code against known values. I additionally spot-check the LLM's code against my own known values in prod, and have had to correct it on some functions. The code I write is mainly analysis of logs and configurations, and they seem well trained on the products that I am using.
Testing. It's all about having unit tests, and known outcomes that the LLM can check against every time it makes changes. For my work, that is good enough. For safety-critical systems I would demand more rigor in the testing and the human review and creation of tests.
Testing. It's all about having unit tests, and known outcomes that the LLM can check against every time it makes changes. For my work, that is good enough. For safety-critical systems I would demand more rigor in the testing and the human review and creation of tests.