Comment by Der_Einzige

You can take any Mistral 7B model or fine-tune and use this adapter to unalign it.

https://huggingface.co/Hellisotherpeople/toxic-dpo-v2-mistra...