devashish86

The key constraint was keeping the quality bar high.

If I lower the quality bar and _only_ care about a working solution, this works great. And I dare not look at the implementation details if this is the outcome I care about.

But if I want a working solution _and_ have it done the right way, then I need to be in the loop at key points. Newer models, unfortunately, have made this problem worse. I will inevitably come back to some form of bad decisions, deviations or just slop. Left unattended, it snowballs into a pile of mess. At this point, I decide if I spend time fixing it or switch my mental model to not care about the quality.

The problem is worse when there's a team involved here. Fragmented tooling , prompts, harness, model causes more divergence. Especially in quality. Senior engineers become bottleneck as the quality enforcers.

I do like the revert rate metric that Posthog tracks. This can help incrementally build the tooling to a point it can be trusted and scales.

fwiw, this is my high level process.

1. i keep comprehensive notes _while_ i'm experimenting(like checkpoints). this is a mix of commits and a append-only changelog file

2. if some part of what i've done seems like i can share with public, i create an outline of it. small paragraph with a few points

3. then i ask the model to merge the two to generate a post. i have my own style guide but ofcourse, the model idiosyncrasies will always creep in.

The tool_choice="auto" failure on Qwen3-Next isn't a parser issue — the model reasons inside <think>, decides, and never emits the tool call. No error, just empty tool_calls. The fix was swapping the backbone from Thinking to Instruct, not tuning any parser flag.

The "load the bigger model first, size the smaller against actual residency" playbook generalizes to anything with shared CUDA framework overhead. The ~5 GiB framework floor shows up even at small gpu_memory_utilization values — plan against actuals, not targets.