Comment by Bolwin

Very similar to Kimi K3 it seems, but fewer active params (80b vs 104b). I don't think they mentioned what form of attention they use