diyer22
Joined 8 years ago
Joined 8 years ago
On Feb 6, the just-concluded AIME 2026 I, Step 3.5 Flash take first place. Step 3.5 Flash was released on Feb 1, making cheating impossible.
More detials: https://static.stepfun.com/blog/step-3.5-flash/
https://x.com/diyerxx/status/1978531040068321766
Getting started on Twitter is so tough—engaging with my posts would really help me out a lot!
And it's recommended to combine it with an autoregressive model (GPT) for more powerful modeling capabilities.
In DDN, the GT is only used to calculate the loss and guide sampling; it never becomes an input to the model.
- The DDN single-shot generator architecture is more efficient than diffusion.
- DDN is fully end-to-end differentiable, allowing for more efficient optimization when integrated with discriminative models or reinforcement learning.
- Moreover, DDN inherently avoids mode collapse.
These points are all mentioned in the blog: https://github.com/Discrete-Distribution-Networks/Discrete-D...
1. DDN doesn’t need to be invertible. 2. Its latent is discrete, not continuous. 3. As far as I know, flow keeps input and output the same size so it can compute log|detJ|. DDN’s latent is 1-D and discrete, so that condition fails. 4. To me, “hierarchical many-shot generation + split-and-prune” is simpler and more general than “invertible design + log|detJ|.” 5. Your design seems to have abandoned the characteristics of DDN. (ZSCG, 1D tree latent, lossy compression)
The two designs start from different premises and are built differently. Your proposal would change so much that whatever came out wouldn’t be DDN any more.
And, their work is far more polished; I’ve only put together a quick GPT+DDN proof-of-concept.
Thank you for sharing.
I never even ran an ablation that disabled the stem features; I assume the network would still train without them, but since the previous layer has already computed the features, it would be wasteful not to reuse them. Retaining the stem features also lets DDN adopt the more efficient single-shot-generator architecture.
Another deeper reason is that, unlike diffusion models, DDN does not need the Markov-chain property between adjacent layers.