rebeccajae
Joined 2 years ago
Karma
20
Submissions
7
Member since
Sep 19
Joined 2 years ago
It felt like it was commenting on the diff sometimes instead of what the code was doing.
My experience running Gemma 4 locally has been similar: after maybe one or two tool calls it starts making tool calls however it feels like. Just yesterday, I watched it redefine a tool like read_file(start, end) into read_file(start, number_of_bytes) and it refused to even consider that it was wrong.
One of these orgs had basically all of the noisy pages under control through automated remediation and a culture that pushed towards aggressive automation of pages that had the same root causes. Almost everything I was paged for was some novel or uncommon fault condition. Developing an intuition about how the system worked (well, more specifically, how the system failed) took a lot longer because most things I had to look at were some edge case failure.