valine

I always thought if Apple did touch on the macbook it would come with a redesign to merge macOS with touch elements from the iPad. As it stands this feels like more of an afterthought.

If there’s one killer feature though, it’s iOS device mirroring and the iOS simulators. I’ve instinctively reached up to touch the iOS simulator so many times.

I wonder how they do the glowing apple logo with no backlight.

Astra is the first chat model with really strong spatial reasoning. Gemini is nowhere close. Hard to say what google has going on internally, but if they have an astra like model I doubt they’ve had it for very long.

The bitter lesson is finally coming for the self-driving cars. The vision stack, 3D maps, lane selection grammar, occupancy networks, it’s maybe all about to give way to a single GPT looking at camera feeds and predicting the next steering wheel adjustment.

It’s mostly a latency problem at this point. The models are too big to run locally, but given that open-weight models like Qwen already exist, an open-weight, low latency equivalent to Astra can’t be too far out.

> The two modified satellites will test hardware planned for Starlink V3 and will attempt to scan Starship’s heat shield and transmit imagery down to operators to test methods of analyzing Starship’s heat shield readiness for return to launch site on future missions

Hope we get to see those images. Would be awesome to see a 3rd person view of starship in space.

Apple silicon has been an unmitigated success so it makes sense they’d go with Ternus. On a related note Apple needs to add Ternus to their spell check dictionary

Yup exactly, in principle it helps with both inference speed by reducing memory bandwidth usage and also reduces the memory footprint of your kvcache.

So let’s start with a really simple decoder transformer with a single layer and single attention head, and train it to predict the next token in a sequence of text. To predict the next token you need a few things: a query for the very last token in the sequence, and a key and value for every prior token. You take your query and compute a dot product with every prior key (two large vectors in, scaler attention score out). That scaler attention score first goes through softmax, and then becomes the weight you use to compute a weighted average of your values, new value goes through the mlp, mlp output is projected into the logits from which you sample your next token (that’s the general idea at least skipped a few steps).

The last query in the sequence will be new for every new token you predict, but the set of prior keys and values stay the same, ie keys and values are reusable. The key value cache gets bigger and bigger for each new token you add to the sequence, and that’s where compression comes in. You have to store the keys and values in vram, and you’d like to keep the size down by not storing the raw uncompressed tensors. To make this work well your compression needs two things: it needs to be fast so that you can compress and decompress on the fly, and it needs to play well with softmax attention. Prior attempts at compression usually suck at one or the other, either the speed to decompress is too slow and your token/s takes a hit, or you lose important precision and the model output quality suffers. The claim in the paper is that they’ve made progress on both.

The allegations of fraud made by the people invading my home and terrorizing my friends? I’ll take those with a grain of salt.

And for the record, I’m not afraid of ICE, never said I was. ICE is racially profiling people, arresting them without cause, and deporting them without due process. I happen to be white, so that doesn’t apply in my case. It’s also coincidentally the same reason I feel safe posting a comment like this online. Free speech is being chilled in communities that ice targets, and I feel responsibility to relay what I’m hearing.

ICE also just gunned down a US citizen in the street, and that should scare everyone.

I lived my whole life in the twin cities and have a lot of friends, US citizens, who are too scared to go out to eat right now because of the ICE raids. If that wasn’t the point it is certainly the effect. I applaud Walz and Frey, and I will be ranking Frey first next time he’s up for reelection. Something tells me though he will be on to bigger things than mayor of Minneapolis.

> About the second point, I've been under the impression that because LLMs are trained on average code, they infer that the bugs and architectural flaws are desirable

This is really only true about base models that haven’t undergone post training. The big difference between ChatGPT and GPT3 was OpenAI’s instruct fine tuning. Out of the box, language models behave how you describe. Ask them a question and half the time they generate a list of questions instead of an answer. The primary goal of post training is to coerce the model into a state in which it’s more likely to output things as if it were a helpful assistant. The simplest version is text at the start of your context window like: “the following is code was written by a meticulous senior engineer”. After a prompt like that the most likely next tokens will never be the models imitation of a sloppy code. Instruct fine tuning does the same thing but as permanent modifications to the weights of the model.

Humans don’t learn to write messy complex code. Messy, complex code is the default, writing clean code takes skill.

You’re assuming the LLM produces extra complexity because it’s mimicking human code. I think it’s more likely that LLMs output complex code because it requires less thought and planning, and LLMs are still bad at planning.

Image generation and image input are two totally different things. This is about feeding text into LLMs as images, it has nothing to do with image generation.

Both Claude and GPT5 can single shot this type of game. The score counter looks exactly like the type of thing Claude spits out.