hiimshort

Just a passerby, but wanted to say thanks for your work. Ory services are a delight and I was excited to see them spring up years ago and even more excited to see them continue to be developed and put to good use!

I have been wondering about the use of diffusion techniques for text generation, it is nice to see Google release a model that, seemingly, validates some thoughts I had.

Most folks I have seen experimenting with AI are either using a paid service or running high-grade hardware (even if consumer-level). The best I have in my current repertoire is a 5700XT and am not able to upgrade from that yet. The limitation, though, has at least also given some more significant insights into the shortcomings of current models.

Model sizes have gotten quite large and coherence seems to mostly have scaled with the density of a model, leaving the smaller models useful for only smaller tasks. Context size is also extremely important from my experiments with long-running dialogues and agent sessions, but a smaller GPU simply cannot fit a decent model and enough context at the same time. I do wonder if diffusion techniques will allow for a rebalancing of this density-to-coherence connection, letting smaller models produce chunks of coherent text even if limited by context. From my viewpoint it seems it will. Mixed tool call + response outputs also have the potential to be better.

Speed is also another problem I, and everyone else, has had with modern LLMs. The nature of cycling around the input with a new additional output each time is time consuming. On an older GPU with no AI-specific hardware it is an eternity! Being able to at least track 0-100% progress state would be an improvement from the current solution. At the moment one must simply wait for the LLM to decide to stop (or hit the max number of inference tokens). I am hopeful that, even on lower-end GPUs, a diffusion model will perform slightly better.

This does now beg several questions. If we are processing noise, where does the noise come from? Is there a good source of noise for LLMs/text specifically? Is the entire block sized beforehand or is it possible to have variable length in responses?

Lix is excellent. It is already faster (parsing), safer (better defaults, removed footguns), and easier to use (better errors, etc) than Nix. If anyone wants to get started using Nix then I highlight recommend you install Lix from the link in the parent comment.

For those unfamiliar with the game I would highly recommend it if you are interested in CRPG games with excellent writing. There is a lot of text in this game, but with the most recent version of the game most of it is voice acted. Many lines will stick with you later. It's rare to not be taken by something in the game, as expansive as it is.

For a more general description of the game: you are a detective, you must solve the case, and your fractured psyche will not let you do it alone.

The same is true for me on Safari. I've tried with and without AdGuard and the experience on plenty of sites that I just want to get a quick tidbit from is awfully painful.

Pikuma is excellent! I have enjoyed going through the videos on the YouTube channel to help understand some 3d graphics rendering techniques. Everything that I have seen has been as easy to digest as this post. Highly recommend checking out the rest of the catalogue if this kind of content is interesting to you.

As someone who grew up with iPhones coming out around the time I was in middle school and apps producing noises like this were used as pranks in class I have to ask that people don't do this. The sound is painful.

This is against Sony's TOS and can result in a ban. In fact, this is exactly what happened to a Chinese user who tried to do that.