mxmlnkn

I have been imagining this since GPT-3. I still only know of very few games making use of LLMs. There are add-ons for Skyrim, and web search points me to NVIDIA's ACE autonomous game characters framework and games such as InZOI, PUBG's ACE-powered AI teammate, Dead Meat (unreleased interrogation game), AI Dungeon, Suck Up!, Vaudeville, and more. It seems there are a few more than I would have thought, but the smaller ones seem to have some mixed reviews. It would be interesting to check out how these games resolve the reasons against LLMs in games I can think of:

Cost: LLMs fitting into local GPUs, in addition to the memory required for game textures, are too constrained for the quality, latency, and concurrency required. Connecting to the cloud would be too expensive, and even if it would work, it would indeed be the death of local offline gaming, and it would give a valid reason for a subscription to keep the servers running as opposed to a buy-once play-forever game business model.

Unreliability: You don't want to miss a major quest because the LLM did not decide to hand it out. It might be workable with some proper "agentic harness", with some hardcoded world lore, long-term memories, goals, midterm memories, some deterministic actions, and a short-term free-form loop that gets cleared every now and then. For OpenClaw instances, it would be the SOUL.md, but it would have to be read-only. Some logic might have to be hardcoded outside of LLMs, e.g., to start a new narrative branch powered by a new character memory when a keyword has been mentioned or a quest item has been given. Similar to agentic frameworks, it would have tool calls such as `offer_quest`, `give_reward`, etc.

These ideas seem to converge onto the Westworld mechanics of backstories, cornerstone memories, reveries, narratives, and loops, at least as far as I remember them from the series. Thinking about it, given enough investment, I feel like the amusement park of Westworld should already be somewhat possible to implement with the recent advances in LLMs and robotics.

In the end, I am not sure how much sense it makes for games. Maybe it would suffice to only make companion characters LLM-enhanced while NPCs stay pre-programmed in order to let the game stay deterministic. Games should simplify real life in order to increase enjoyability. Making all characters backed by LLMs might decrease enjoyability because of forgetfulness or confabulations inconsistent with the game world and actions. It would need to be heavily constrained and trained or prompted to say "I don't know about that.". It might also be questionable how much needs to be dynamic. E.g., when thinking about the NPCs in The Witcher, which are talking about random stuff, maybe you only need to pregenerate much more random stuff for them to talk about so that you don't notice the repetition, but there might be no need to have this completely dynamic and burn through GPU power for every action. The question would be what to cache and what really needs to be dynamic.

This sounds really cool. My intuition was that the selected experts might change heavily for each token, resulting in slow SSD loads for each token. This seems to be wrong. Did you create some statistics on how often the experts need to be changed? What is the longest token run without any expert change? What does such a token run look like? In which cases do experts change frequently?

Why not simply have both? This does not have to be an either-or decision. Have a default repository with vetted extensions, but leave the option to install from other sources open.

> 10. Code is cheap, but maintenance, support, and security aren’t.

I also keep circling around this point. So many software repositories in the AI space seem to follow a publish and forget pattern. If you simply can show that you have the patience to maintain a project, ideally with manual intervention instead of a fully autonomous AI, then you already have an outstanding project.

The title seems misleading.

These are not errors. They are simply warnings about extended attributes being ignored when extracting files, which seems completely fine to me, and creating the tar without those extended attributes has exactly the same outcome, but throws away the metadata at archive time instead of extraction time.

Furthermore, this is not an Apple/macOS issue. The tool used is bsdtar, so it would also affect all BSD-variants that default to bsdtar/libarchive, and those systems also have extended attributes, e.g., for SELinux, which would get added to the TAR.

> Apparently the internal state is only 32kB

Exactly. And often this state is either highly compressible or non-compressible but only sparsely used. The latter can then be made compressible by replacing the unused bytes with zeros.

Ratarmount uses indexed_gzip, and when parallelization makes sense, it also uses rapidgzip. Rapidgzip implements the sparsity analysis to increase compressibility and then simply uses the gztool index format, i.e., compresses each 32 KiB using gzip itself, with unused bytes replaced with zeros where possible.

indexed_gzip, gztool, and rapidgzip all support seeking in gzip streams, but all have some trade-offs, e.g., rapidgzip is parallelized but will have much higher memory usage because of that than indexed_gzip or gztool. It might be possible to compile either of these to WebAssembly if there is demand.

I like the idea of bubblewrap, but my pain point is that it is work to set it up correctly with bind mounts and forwarding necessary environment variables to make the program actually work usefully. Could you share your pip bwrap configuration? It sounds useful.

> The obvious one outside of KV caches as mentioned above is vector databases. Any RAG pipeline that stores embedding vectors for retrieval benefits from the same compression. TurboQuant reduces indexing time to “virtually zero” on vector search tasks and outperforms product quantisation and RabbiQ on recall benchmarks using GloVe vectors.

This part sounds especially cool. I did not think about this application when reading the other articles about TurboQuant. It would be cool to have access to this performance optimization for local RAG.

After 2 minutes at 150 kHashes on mobile, I finally see the first pixel of the progress bar filling up. Seems like it will take hours or a day to finish. Some estimate would have been nice.

I never understood why the recommended replacement for ß is ss. It is a ligature of sz (similar to & being a ligature of et) and is even pronounced ess-zet. The only logical replacement would have been sz, and it would have avoided the clash of Masse (mass) and Maße (measurements). Then again, it only affects whether the vowel before it is pronounced short or long, and there are better ways to encode that in written language in the first place.

On the other hand, fonts can be an expression of your personality. Shouldn't it be preferable to centrally enable overriding fonts instead of forcing every site designer not to use custom fonts to express themselves? Theoretically, it is easier to remove formatting than it is to add it. Therefore, this functionality should be part of the browser, not the website. Firefox has this as an option: "Allow pages to choose their own fonts, instead of your selections above".

Personally, I quite like the site's design and its font. My gripe often is light gray text on a darker gray background. The bad readability that so many newer sites seem to prefer makes me question my eyes or my monitor capabilities. Reader mode in Firefox is also often very helpful.

I can confirm usual compression ratios of 10-20 for JSON. For example, wikidata-20220103.json.gz is quite fun to work with. It is 109 GB, which decompresses to 1.4 TB, and even the non-compressed index for random access with indexed_gzip is 11 GiB. The compressed random access index format, which gztool supports, would be 1.4 GB (compression ratio 8). And rapidgzip even supports the compressed gztool format with further file size reduction by doing a sparsity analysis of required seek point data and setting all unnecessary bytes to 0 to increase compressibility. The resulting index is only 536 MiB.

The trick for the mix of JSON with binary is a good reminder. That's how the ASAR file archive format works. That could indeed be usable for what I was working on: a new file format for random seek indexes. Although the gztool index format seems to suffice for now.