mungoman2

Will be interesting to hear an update after two months. I think two weeks is too little to conclude that it works.

My experience and what I’m concerned about is that eventually more time is spent refactoring to enable new features than writing the features themselves. In my experience this becomes worse with time as the models are reluctant to remove behavior, so the code base will grow to support some version of the old behavior together with the new behavior.

Ohh this is really cool.

So one could pack all of common state, every question, every answer in the same prefill, using attention mask to only let them attend to their logical parent.

Then additionally do position encoding for token based on their logical position rather than physical.

Then the diffusion step also applies an attention mask to prevent bidirectional attention between answers.

Yes that seems sensible for isolating questions/answers.

> 6. Calculate only 64 possible label scores—not the whole vocabulary.

This I don’t understand though, could you expand this please?

This is very interesting! Seems like a promising direction.

I wonder though if it supports the same claims as Jev: answers are not impacted by other answers to the same questions, nor the existance of other questions? It seems by sharing KV cache all questions will be visible. And I think the diffusion causes the answers to attend to eachother?

Also I think without fine-tuning we can not say that the probabilities it output are actually probabilities. Maybe fine tuning using Brier scoring would do the trick?

Maybe some kind of hierarchical structure of the KV cache can make questions independent, and with smaller diffusion canvas’ generated in batch can be a way to make answers generate independently?

Now this is why I come here!

Amazing thing, will replicate.

This could be interesting but too vague for me to understand what you mean. Could it be explained more clearly, or link some blog entry?

Well, was the contract terminated?

Saying no will sometimes have consequences, and the counterpart is sometimes unreasonable.

Similarly curious about this. The intuition I extracted:

Let’s say we have 10 requests, where 9 of them take 1 second to complete but one that takes 100 seconds. The average time to complete a request is about 10 seconds, but if you experience the requests in series, at any given time you’re much more likely to sit and wait in one of those 100 second requests.

So if you imagine a long series of requests from this distribution and place yourself randomly in the series, the average time to completion is just a bit less than 50 seconds.

This is what is meant by t-weighted, that events with a large t take a larger place.

In this case it could be solved by not letting the LLM consume the transaction message. Effectively the same as preventing user supplied info going to the first argument of printf().

The transaction in question can remain opaque to the LLM and a %transaction.message% string is resolved in the layer between the LLM and the user.

Curious about this take, how do you mean?

I understand the point of distorted facts, but what I’m not sure how things are improved by basically having no trust in any facts?

This looks very interesting. Possible to get those rates without exotic hardware.

But I have to say that the comparison is not really fair. Comparison is done with a 2 B model vs frontier models that are likely 100s of times larger. Also taalas with their 15000 tok/s inference are suspiciously missing from the comparison.

We need to see the comparison with this framework and useful models, which at present seems to mean ~30 B.

Wow, this is really fascinating. And it reads like the intro of a sci-fi short.

This looks very cool, but it's not clear what it means.

I wonder if it is captivating simply because it syncs cool graphics to audio, like those Winamp visualization filters in the old days.

Very interesting! Will definitely try this.

Feedback: I think it would be beneficial to clarify that there is a trial at the top of the page. Currently we have download and buy buttons next to each other, which I assumed meant that the software can be downloaded first, but will require a key you get from ”Buy”.

The images show Switch 1 controllers. Since the Switch 2 is already out, does this imply that the tracking doesn’t work on Switch 2 controllers?

I don’t like the 3 PC activation thing. It implies that the software needs to call home and will stop working if the company goes under. Or, at least, will not be possible to install anew.