geraneum

Are they trying to push more users towards Chinese models?

> Most parliamentary democracies assume you are electing a person with the knowledge/hope the party leader will be sitting in the big chair at the end.

In Germany the focus is usually on the parties and not the figurehead for over 70 years now.

edit: for the election

The “more features != better” is still a valid principle IMO, and AI doesn’t help with that. Probably is making it worse.

> I'm fine if some of the stuff is having scheduled or unscheduled maintenance.

You might be, and your customers may not! And your competitor could be more than happy to offer a more reliable service.

> Don't take it too seriously.

I didn’t, hence the question.

> How would they have both agreed to use the cache though

I think it’s safe to assume that extracting information using timing in cache is already known in training set, since we have the likes of Specter.

Maybe they visited the website and the existing questions were enough? These stats seems to be questions asked. Maybe that’s the reason.

It’s so cute that some people imagine the armed forces and police were on the other side or are on their side. If you’re average Joe, the police in a tyrannical state are never on your “side”.

Do you foresee that AI will be able to count Rs in strawberry by then?

Jokes aside, by then we're gonna have StarTrek Replicators. Exciting!

So when they release those model, we all will be solving millennium prize problems?

But people do unpaid painting, horseback riding, calligraphy, etc. which are outdated because they love it. Why is it so unfortunate for you?

> And unfortunately, that works pretty well.

Why unfortunately?

There’s no such thing as a read/write head in transformers. You can use it as a metaphor but that’s a leaky metaphor, hence all these misunderstandings.

> In fact it has N read / write heads equal to the size of the context

And what is the asymptotic notation when computation scales linearly with the size of context?

> Either way, the simple truth is that you can train a transformer system

Which still doesn’t mean they have a read/write head or have O(1) complexity for token computation with “infinite context”.

> If the volume of work goes up, its cost goes down. Supply and demand.

What?! Good to know. When contractor comes to do the kitchen, I will make them do the garden as well and will reduce the money I’d pay them, because supply and demand!

> Unless you think you should pay more now for more content

We do already! You realize you’re paying by the token? More token -> bigger bill!

> Same idea for that transformer

It’s not the same for transformers, especially not with the theoretical infinite context. It simply doesn’t process tokens in O(1) in your given setup.

In practice, with frontier LLMs, it’s of course more complicated, and also they don’t reveal how they do lookups, but still the theoretical limit applies regardless.

Do you mind elaborating or perhaps providing some reference as to why you think transformers should be similar to a Turing machine on this specific criteria?