abecode

yes, that was surprising to me too. It would be a big deal if they switched to an encoder-decoder model like the original transformer. But I don't think that's what it's doing. One thing is the causal part, so in the original transformer, the encoder was bidirectional, but in this case it is not, so that's one difference. So I think it's an optimization for the prompt/prefill so that the attention is summarized into the output of the encoding layers, rather than all the layers. I just skimmed the paper too so if anyone else has insight, please correct me.

I think that given the developments in AI in the last ~10 years, it would be hard to say that attention is not part of consciousness. I'm not sure if it is "all you need", but I think that any theory that doesn't include it would be very doubtful.

I think the parent is referring to agents as outward exploration that may come up empty-handed, but I see LLMs and agents as inward exploration, trying to define what is attention, knowledge, intelligence, consciousness, agency, etc. So in this case it is the terra incognita of the human "soul" that LLMs are exploring.

nice, you got claude to say "I'm deepseek" when queried/prompted in Chinese, that's great!

你是谁? -> 我是 DeepSeek 由深度求索公司...

I listened to the audio book version of War and Peace. I think it was something like 25-30 hours. The audio format helped keep the pace going and also it helped with the names. Although for some things, the audio format made it harder to look up in the dictionary, like I kept hearing agitant instead of adjutant, so that part didn't make sense in a lot of the military scenes. I agree with the parent that the book was very engaging, parts even felt like I was watching a movie, e.g. the drunken party tying a bear to a police officer, the foxhunt scene, the duel, the battles like when Petya gets shot, and the burning of Moscow. I even liked the abstract ending when Tolstoy relates human history to calculus so that each individual person has an infinitesimal but real contribution to history.

Corsica to Marseilles has an overnight ferry, at least back in 2000, but I don't think it's has sleeper cabins, more set up like an overnight flight, but a big cruise ship form factor. Instead of cabins they had space for cars. They had the option of getting a ticket without a seat so you just roam around the boat and find an open space, including outside on the decks.

My use case for this is making separate bookmarks in different folders for a single URL:

Example.com/interesting -> bookmark folder one

Example.com/interesting?dummy=t -> bookmark folder two

As a resident of MN I'm very proud of what fellow Minnesotans did to stop ICE's violent and illegal detentions here. Unfortunately the non-violent protest and anti-ICE techniques were met with violence from ICE, but the protests themselves were non-violent and well organized.

Fall, Or Dodge in Hell (Neil Stephenson) and The Waves (Ken Liu) are two other good stories about brain scanning and transhumanism. The first one is a ridiculously long novel about a future where the cloud is increasingly used for uploading souls of scanned brains, and the second one is a short story where people on a spaceship eventually evolve into noncorporal beings.

Maybe they could bury the clothes and call it carbon sequestration. I assume that clothes are made of mostly hydrocarbons.

I just had a nice trip to Venice and I was curious about it's history. Supposedly, the Venice republic lasted almost 1000 years, basically from after the fall of Rome to Napoleon based on a weird lottery system for choosing the Doge.

In Chinese one word for potato is "earth bean" 土豆 (the other word is "horse bell tuber" 马铃薯)

At ECAI conference last week there was a panel discussion and someone had a great quote, "in Europe we are in the golden age of AI regulation, while the US and China are in the actual golden age of AI".