Recurrent depth and chain-of-thought are two completely different concepts. In the former approach, the output of the layer gets is rerouted as input to the same layer, potentially several times. This output/input is a fixed width times sequence length real-valued representation; it is not comparable with output tokens.
Generally, it is hard to imagine how neuralese should work given that models are pre-trained on naturalistic documents: CoT is a comparatively simple extension of that, while neuralese demands a completely novel training paradigm.
Generally, it is hard to imagine how neuralese should work given that models are pre-trained on naturalistic documents: CoT is a comparatively simple extension of that, while neuralese demands a completely novel training paradigm.