asjir

Exactly, they put a lot of money into engineering and it does give results

Just keeping it up to date with competitors is much cheaper, by copying better ones like Qwen did with Claude. Also a bunch of research is trickling into open source / arxiv so catching up should continue becoming cheaper at least as a fraction of training from scratch

maybe you can preselect good ideas, build up guidelines describing most common pitfalls, extrapolate from ideas you already vetted etc and run on autopilot on a safe-ish subset

But what if it's a bubble driven by speculation?

It wouldn't pay off.

Starting a futures exchange on RAM chips, on the other hand...

Nvidia's parakeet dropped recently with better performance and 0.6B params, so the rate of progress here looks good, probably next year (or mby the year after) they'll be running no probs

Not really answering your question, but: One completely imo unnecessary category of sloppy software is electron apps. It's totally ridiculous how little resources are put into alternatives like tauri given how most dekstop apps run on electron, and we know how bad it is.

I read this comment first and thought: "oh come on, how bad can it be". So then seeing this really being so bad(-ly overdesigned) was quite amusing.

I'm not disagreeing per se, but in my experience my tests are so disposable they're never worth investing that much engineering effort in past getting them to pass

The author makes an argument that at least looks like people choose YT over Nebula because YT is free. I, for example, already pay for Nebula, so I can watch it for free, but I still go to YT.

IMO it might be just a product problem. I opened nebula and:

* The same video had a better title on YT that was actually less clickbaity and more informative - assumedly because of YT algorithm for optimization

* Nebula auto set quality to 480p compared to 1080p in YT - if I wasn't tech-savvy I'd assume it's just worse quality.

* The loading times when you seek to part that's not loaded yet are 10x longer

* I missed comments

The recommendation algorithm is weaker too, I can't tell to what extent this is due to YouTube having simply more data and to what extent it's weaker engineering.

If they do some decarbonization real estate SaaS and need connections with both regulators or whoever issues certificates and REITs etc, then it makes sense. I don't think they need a lot of software for that.

To expand upon the other comment: Indexing and multiplying with one-hot embeddings are equivalent.

IF N is vocab size and L is sequence length, you'd need to create a NxL matrix, and multiply it with the embedding matrix. But since your NxL matrix will be sparse with only a single 1 per column, it'd make sense to represent it internally as just one number per column, representing the index at which 1 is. At which point if you defined new multiplication by this matrix, it would basically just index with this number.

And just like you write a special forward pass, you can write a special backward pass so that backpropagation would reach it.

I'd be more concerned about the size used being 70b (deepseek r1 has 671b) which makes catching up with SOTA kinda more difficult to begin with.

In a 20-person office you'll pass by like 2 people on the way with the card, so it's not a problem. If the office was bigger, and it was a problem, you could just designate a spot for the card next to the pantry.

Ahh, I see, thank you (and other commenters)

I was a bit surprised by the change in tone in a way I probably wouldn't've been if I'd read chronologically.

Why not start with Colour of Magic? For context, I started with it, then Light Fantastic and I'm finishing Sourcery now.