langs

I don't get it. Why benchmark the latency instead of recall/precision? Optimizing for millisecond-level latency is meaningless in the context of LLM calls. Accuracy is the tool's greatest value, yet there is no testing for it?

Simply feeding the Slay the Spire game state directly to an LLM is currently insufficient to achieve stable deck-building or consistent combat performance.

I categorize the game's actions into three types: combat, deck-building, and other interactions.

For combat, MCTS is used. Yes, it's similar to the approach used in Go AI.

For deck-building, a sparse graph search is used: the goal is to rapidly identify winning deck templates within the graph structure.

Only the remaining aspects are delegated to the LLM to make reasoned decisions.

Yes, it uses modern technology to tackle an old problem: retrieval.

I am working on the same thing right now. However, unlike storing conversations in an external retrieval system, I use a local LLM to store the conversation's KV cache and perform retrieval directly on that cache. The method involves running a prefill pass and, after obtaining the attention scores, filtering for the corpus segments that received attention.

This aligns with the "zero tokens" approach described in this paper. :)

I tested it on the LoCoMo used in this paper, and also LongMemEval, both achieved SOTA results.

Pi’s minimalism reveals a simple truth: the LLM is the core of any agent harness. Consequently, much of current harness tuning will become redundant(or even a hindrance) with next-gen models.

> Myth 2: Writing Code Is the Bottleneck

Writing code is indeed the bottleneck for same resource constrained companies.

Rapid code development creates more opportunities for trial and error, providing companies with more information for decision making, that previously might have been addressed by meetings.

Of course, this might bring other problems, but it might not right to generally speaking that writing code is not a bottleneck.

You need to optimize the KVCache part(save to disk to save compute) to achieve this goal.

So, is this a competition? To see whose model can be jailbroken the most times and incite the highest level of public alarm?

"mandatory safety testing" is an impractical ideal, it's not workable in real world. Like any technology, LLMs are dual-use tools capable of both beneficial and malicious applications—a fundamental reality that human intent cannot change.