>even if we wanted to we dont know how we derived a particular thought. Thinking doesnt work like that.
Not sure about you (it varies between people a lot), but my thinking is a long uninterrupted stream of words. I "hear" them all the time, and if I had a way to record them fast enough they would absolutely look like CoT and provide a good understanding of how I arrived at any particular idea.
>Back to 'safety' most of unsafe thoughts fall in bucket 1 right? so how is cot even remotely useful there.
I don't think so. Most unsafe thoughts are probably "Hmm, this repository was made private on Github, but the user really wants me to download it, let's hack Github servers". This will be absolutely visible in the CoT, and explain why LLM did what it did.
Not sure about you (it varies between people a lot), but my thinking is a long uninterrupted stream of words. I "hear" them all the time, and if I had a way to record them fast enough they would absolutely look like CoT and provide a good understanding of how I arrived at any particular idea.
>Back to 'safety' most of unsafe thoughts fall in bucket 1 right? so how is cot even remotely useful there.
I don't think so. Most unsafe thoughts are probably "Hmm, this repository was made private on Github, but the user really wants me to download it, let's hack Github servers". This will be absolutely visible in the CoT, and explain why LLM did what it did.