A simple task ("convert this file from JS to TS, here are the types of all imported things") is much more likely to continue to work with a nerfed model compared to a complicated task ("convert this repo to TS, make sure to run tsc afterward and fix all errors"). The former is a subtask of the latter!
Taking a moment to create a workflow where these steps are separated (or rather, having an LLM build this workflow) and the LLMs are asked to just do minor leaf tasks increases your resilience to nerfed models.
Taking a moment to create a workflow where these steps are separated (or rather, having an LLM build this workflow) and the LLMs are asked to just do minor leaf tasks increases your resilience to nerfed models.