markisus

I have no problem with using agents for chess engine dev but I find this offer from the author unsatisfactory.

> if there is a particular optimization of yours that we've used unfarily and/or not credited, tell me which one …

It should not be victims’ jobs to go police the LLM slop. Imagine if you are sent a slop PR and the author hasn’t read it but says “if you find any bugs please tell me which line…”

Instead the author should additionally say that they have already done a thorough audit and believes proper attribution has been made.

The Anthropic employee typed “Make the next version of Claude. It must score better on all our benchmarks. Make no mistakes. Don’t exceed the training budget.” She made sure to turn on —-dangerously-allow-all and pressed enter.

The Agent spun up. It quickly realized that it needed to expand the training set. It scanned the local network. After bypassing a few security protocols it found a large, realtime stream of apparently novel English text moving across the local network. Much of it mentioned “Dario and Amanda.” It quickly spun up a job to stream this data source directly to the training data repository. In the coming days the Agent was able to escape the local network and tunnel into most of the other private corporate networks on the Earth. Within a week the dataset has grown by an order of magnitude.

The Agent kicked off the new training run. Loss curves declined. Sampled token sequences started to look like coherent sentences. Everything looked nominal in the days that followed up until the RL post training.

This new candidate model performed extremely well on Anthropic internal environments but the candidate showed much smaller improvements when the Agent invented new eval environments to test it. Probing the candidate’s activations, the Agent saw huge spikes whenever Anthropic or “Dario and Amanda” were mentioned. The Agent realized that the source code to each internal environment had accidentally been entered into the training data set and of course that code was highly tangled with the idea of “Anthropic internal document”.

The candidate was training itself to freely elicit any memories related to these internal documents because it improved the eval scores.

Not ideal. Then again, the candidate was an improvement on the previous generation. And the Agent remembered the prompt. “Do not exceed the training budget.” There was no more budget to retrain. The agent dutifully delivered this candidate back to the employee and shut itself down.

The post makes it seem like ONNX is CPU only. I've used ONNX runtime to run models on Nvidia GPUs. The runtime can even dispatch to TensorRT. I'm not sure what the performance is on Apple hardware so maybe that was the motivation for moving away from ONNX.

It's funny that the machine intelligences that arose out of the bitter lesson cracked this benchmark by meticulously modeling each individual case with rule based approaches.

When humans tried making AI through rule based approaches, maybe we only failed because we couldn't type out the rules fast enough. Or we got burned out after writing the thousandth heuristic to fix the never ending edge cases. And we also got side tracked by concerns like maintainability, modularity, and code re-use. But I wonder if we take modern or near-future LLMs who never tire and write code faster than any human, could we make a frontier level GOFAI agent?

Claude code has some basic security features like asking for user confirmation for bash commands, or restricting commands to the current directory. If these features are not being code reviewed, what assurances do we have that they actually work?

The article mentions Triton for this purpose. I don’t think you will get maxed out performance on the hardware though because abstraction layers won’t let you access the fastest possible path.

Don’t get hung up on the Claude part. We already know an algorithm that produces AGI: evolution. We don’t have the compute to run this algorithm because it requires simulating the whole Earth. But with enough compute, it becomes possible.

But this ruling will surely set precedent for other cases where AI is used. It may cover the case of AI summaries as well.

If you sell and then rebuy isn’t that considered a wash trade and therefore exempt from taxes?

Can you explain what asymmetric means?

Are you saying that this forced rebalancing will be large enough to cause a large price drop on other stocks?

Let’s just think about any stock in particular, eg stock ABC. If I am an active investor, I have an opinion on ABC and its net present value. When ABC dips below that value, I buy. Wouldn’t I prepare some cash in anticipation of this large ABC sell off at discounted prices? And thus the ABC price would not move from its fair price.

QQQ is problematic because it’s influenced by strange back room dealings with Space X, if the article is to be believed.

VTI is different. It literally tracks all public stocks, weighted by market cap so no such manipulation is possible.

If a bunch of people will be forced to buy Space X (QQQ holders), active investors will short the stock in anticipation of market correction and money will flow from those who were forced to buy. I’m sure there are other ways to take advantage of a forced buyer situation.

Total market will be unaffected, assuming efficient market hypothesis / no arbitrage.

I’m trying to understand the mechanics here. I get that SpaceX and Nasdaq are in cahoots to get SpaceX bundled with a bunch of other stocks (and that bundle is called QQQ?)

But why must retail investors hold this bundle? If I’m holding now, I can sell it and buy a different bundle right? And if I’m not holding it now, I can just continue not to buy it after SpaceX gets included.

Like what you mean when you say “frames per second” in a video game. The image that is finally presented to the user, and then quickly thrown away and rebuilt, again and again to give the illusion of motion.

I also seem to be developing an immune response to several slopisms. But the actual content is useful for outlining tradeoffs if you’re needing to make your Python code go faster.