I’m sorry, perhaps I’m behind the times here, but do frontier models currently use continuous (arbitrary embedding) thinking tokens? I know a bunch of papers on the topic came out late last year but to my knowledge have not been successfully implemented into the SOTA yet.
I know the biggest/best open models don’t use continuous thinking tokens, and while the big labs keep things proprietary, I don’t think we have evidence yet that they are.
If I’m wrong please point me in that direction, I’d love to read up on it. Using continuous embeddings in CoT has always seemed obvious to me but doesn’t seem to be as simple as that.
I know the biggest/best open models don’t use continuous thinking tokens, and while the big labs keep things proprietary, I don’t think we have evidence yet that they are.
If I’m wrong please point me in that direction, I’d love to read up on it. Using continuous embeddings in CoT has always seemed obvious to me but doesn’t seem to be as simple as that.