I've worked on a LLM-based visual validation framework for mobile layouts of different websites. The results are pretty flaky, even with Claude models. I've actually seen better performance with older ones like Meta Llama 4 or Amazon's Nova (the solution was running on AWS Bedrock).
I've since refactored it to first do some deterministic checks using CSS, to determine visual glitches like overlaps. Only after do I use LLMs to figure out if text is broken, loading spinners still display, or other such things where models are better.
For deterministic tasks like pixel-by-pixel comparison, I don't think LLMs are good enough (for now, at least).
I've since refactored it to first do some deterministic checks using CSS, to determine visual glitches like overlaps. Only after do I use LLMs to figure out if text is broken, loading spinners still display, or other such things where models are better.
For deterministic tasks like pixel-by-pixel comparison, I don't think LLMs are good enough (for now, at least).