AstroBen
Joined 1 year ago
Joined 1 year ago
Here[0] we have benchmarks for AI writing tests for code that already has the correct behavior. Best case scenario is that they're at 67% with Fable, and that's with the highest quality specification we could possibly have (code). English is far more ambiguous
Can you break down for me exactly how the unit and integration tests are made? Do you ask the AI what to assert?
Black-box e2e testing isn't suitable, or practical, for a lot of applications. Unit and integration tests generally require you to understand the code you want tests for, to know what to actually assert
Are you just trusting the AI to do it all?
Now I'm here staring at AI with the reality that if it makes my skills irrelevant I'll have no fallback, and I'll be at an age where reskilling is extremely hard and I'm still nowhere close to retiring.
Personally I have my own mail server and use smtp2go for sending which handles the deliverability issue. I'm not sure it's worth it going this way but I found it fun and its been 0 maintenance
English isn't a good programming language if you want to trust that the output will be as you want it to be. High level code has no room for ambiguity - that's why we use it. It's a concise, very explicit language to tell the compiler exactly what to do.
Dealing with an LLM is a fundamentally different thing. The reason it's able to speed us up is because it makes assumptions for us and reads between the lines. The only world where we trust the output is one where it can literally read our minds. That's not coming any time soon