This is interesting and important work, thank you!
Question - has your interp group looked at any of Anthropic’s neuralese-to-words tech? I’d be curious to see thinking traces (as in actual weights thinking not the output thinking) from the open weights models and your finetune; seems like it could make good followup research or possibly be a tighter path for evaluating censorship, since it directly evals off weights mid-inference.
Question - has your interp group looked at any of Anthropic’s neuralese-to-words tech? I’d be curious to see thinking traces (as in actual weights thinking not the output thinking) from the open weights models and your finetune; seems like it could make good followup research or possibly be a tighter path for evaluating censorship, since it directly evals off weights mid-inference.