muricula

I'm not sure it's so profound at all. The Intel vector ISA has always been filled with instructions with odd semantics and inscrutable names. They probably made a mistake in some deep corner case in their rounding implementation and had a choice between releasing the new instruction with the wrong semantics and pulling the "chicken bit" making the instruction inaccessible for this generation of CPUs, better luck next time.

Price is confounded by VC subsidies, economies of scale, and inference optimizations. I think a more interesting chart would be ARC AGI vs forwards pass flops or ARC AGI vs training tokens. Of course we don't have those numbers for the closed source models or even some of the open weight ones.

I'm no expert but it seems like a descendent of LSTMs. There's a series of papers which show how to reformulate attention as RNNs which arrives at linear attention. Then they add a decay term to get mamba2. Then they add modified the decay term as like a scale to apply both to the existing state and the new update to get delta net. Then they added a gate matrix on the output to get gated delta net. Then Kimi Linear Attention seems to be gated delta net with a more expressive gate. The Gated DeltaNet paper recaptilulates this evolution decently well. But yeah, it feels like they're starting with the same lego blocks and assembling them in similar shapes to accomplish similar but slightly distinct modules.

Same Armv8 ISA. And it's the same ISA Android Linux has run on for over a decade.

So you don't care if injustice happens to other people, only if it could happen to you?

The host kernel and probably the host VMM can see guest memory, so I wouldn't use it for that.

Python has that property when you don't bring C extensions into the conversation. Data races exist, but can never cause memory corruption due to the GIL.

Usually people are talking about race conditions. When you say contrived you're thinking races conditions are difficult to win and unrealistic but attackers who have a lot of money on the line spend the time to win all sorts of wild race conditions consistently.

I've played with something similar with my M1 using Apple's MLX framework. The problem is I'm compute bound. I've never managed to get my M1 Max's GPU to process more than ~7.8k tokens per second at bf16 precision, so to train a 112M parameter model on ~20 billion tokens I'd need to run the model training for ~30 days.

One solution is to reduce the scope of the problem -- you can train on a smaller less diverse dataset such as TinyStories which is a collection of 1 billion tokens of chatGPT generated children's stories. After about 40 hours, less than one weekend, you'll have a model which can generate mostly grammatical children's stories.

If you have a newer mac and/or an ultra chip you'll have more and faster GPU cores, and might be able to train on FineWeb or a similar, larger and more diverse dataset.

Is there a whitepaper or ISA manual change describing the feature?

I could be mistaken, but I think the mac studio comes with either an M3 ultra or an M4 max, and the ipad comes with an M4 chip. I think they decided not to make an ultra for the M4 generation, but don't take my word for it.

I think the audio quality gives this recording character. What could be more cyberpunk than hearing the quirky artifacts resulting from ripping an obsolete recording medium?