That second "Mind the Gap" paper seems to be doing something very much like that, though their "conformative decoding" (CD) seems very simple: they make both the RLHFed and the base model produce a probability distribution for a given context, then apply top-p truncation to the RLHFed probabilities as one normally does, and then they look up what logprobs the base model assigned to those same tokens, and sample according to the linear combination of the two models' logprobs. This seems to work a moderate amount for them.
Looking at the cited-by I found a paper that evaluated a bunch of such methods by LLM-judged novelty, https://arxiv.org/abs/2606.12790 . CD is weirdly weak according to these results, they get better changes with other methods - but all of the others are less technical and basically amount to asking the model to try harder at being original. I wonder whether CD would work for a task like programming...
Looking at the cited-by I found a paper that evaluated a bunch of such methods by LLM-judged novelty, https://arxiv.org/abs/2606.12790 . CD is weirdly weak according to these results, they get better changes with other methods - but all of the others are less technical and basically amount to asking the model to try harder at being original. I wonder whether CD would work for a task like programming...