oli5679

A simple way to score 50%ish is to take a sequence of text and press the left button if even length, right button if odd.

This is what I also use for rock, paper scissors (mod 3). You can also use second hand of a watch for random samples, e.g splitting 0-30 vs. 31-60.

You need a set of Evals, which catch enough of the mistakes your llm makes following your baseline prompt, so you have an equal or lower error rate than humans.

This can be determined by offline benchmarks if you build a system that takes small sequences of actions, and requires live ab test for long sequences of actions.

The more your human ops team work from documented standard operating procedure, rather than tacit knowledge, the less you need Evals except to capture edge cases

I build ai systems for ops automation and don’t understand the author’s pessimism.

I agree with that Evals are a scarce commodity rn. A business needs to define what good looks like. This is a laborious, and sometimes politically controversial, process.

Given good Evals, frontier llms are a magical tool that can automate tasks and do them more accurately than human ops teams.

Exactly which llm to use depends on the mixture of speed, cost and quality of the output.

Jev makes a claim to expand some regions of the Pareto frontier. I look forward to testing if this is true.

There are many areas of work we can’t automate rn. We cannot create good Evals either because time horizons are too long, or it’s too difficult to create good Evals.

That doesn’t mean there’s anything wrong with building good ai engineering systems in areas where it works magically.

I wonder if you could pursue an antitrust cases in eu against Claude code refusing to use fable to work on some deep learning problems and downgrading to an inferior model.

Would this ever be viewed as an abuse of dominant market position, aiming to disadvantage potential entrants.

Would llms be more robust to this prompt injection if the tags used in fine tuning are sanitised from user input?

E.g. map <think> -> THINK <user> -> USER <tool> -> TOOL

If they learn something specific in the chat finetuning stage, this might show LLM its user input text not these tag references.

This ties directly into the superposition theory.

It is believed dense models cram many features into shared weights, making circuits hard to interpret.

Sparsity reduces that pressure by giving features more isolated space, so individual neurons are more likely to represent a single, interpretable concept.