phamilton

Re: arrays are pointers.

One of my favorite things to show just how bare this is in C is to show array access commutativity.

  char c = {1,2,3}

  c[1] == *(c + 1)
  
  *(c + 1) == *(1 + c)
  
  c[1] == 1[c]

C is wonderfully simple at times.

> new core data substrate

The AWS Aurora white paper was released almost 10 years go. Aurora was announced in 2014 and the paper released in 2017.

(Aurora is built on S3)

Simple math:

A training set of 15 trillion tokens is 10 trillion words.

A penny a word is cheaper than the cheapest beginner freelance writer.

That makes a training set of 10 trillion words cost $100B.

Lots of assumptions there for sure, but we're certainly in the ballpark you are describing.

Any graphql library would work. I like async-graphql except compilation is a bear due to all the macros. We wired LISTEN up to subscriptions using tokio mpmc channels. It was a great moment of "fearless concurrency", it all just worked.

Just sharing a data point and experience.

We had a lot of success with LISTEN/NOTIFY when we paired it with a Rust graphql subscription broker. 10s of thousands of subscriptions, but only 3 or 4 LISTEN connections (one for each host). All changes would be pushed out to all hosts, who would each manage the actual user subscriptions and choose what to actually publish.

This worked super well. In general, moving from hundreds of Ruby or Node hosts to just a few Rust hosts just allows so many simplifications and things that "don't scale" to actually work quite well.

Just a set of things too minor to move off of it but annoying enough to not want to start with it.

My list:

No `explain (analyze,buffers)`. Instant DDL has some warts (e.g. fk, metadata locks). Query planning bugs (actually... query planning in general is disappointing). Exiting the repl doesn't stop queries. Implicit type casting. Replication lag from large DDL (e.g. creating an index). Lack of two phase DDL (creating constraints NOT VALID and then VALIDATE later). Lack of extensions (e.g. pg_vector). No safe access to inspect buffer cache. AWS Aurora seems to only add shiny new things to Postgres. And more.

Again, none of this is quite enough to migrate off of it for an established system, but certainly enough to avoid it on a new project.

I've always known how to do zero downtime migrations. The question has always been "Is the engineering cost worth it?"

> It only makes backwards compatible deployments easier insofar you're able to do the overhead for splitting the change with less effort then before.

Yes. That reduces the cost of implementing them.

To be clear, I'm not talking about "Split my db migration and my code that depends on the new table". I'm talking about things like "Set up dual writes between an old database schema and a new database schema with a thorough test suite and do shadow reads against both datasets in prod to do differential testing". That's nontrivial engineering effort that would definitely warrant a discussion in the past. Today we just do it. It's fast and lowers risk.

> Old things are new again.

This is exactly how I've felt. I've read some many old papers and books and found great techniques that are even more applicable than ever.

> perfect refactor-proof testing contracts are still usually pretty hard to design

Not sure about hard, but definitely rare and we as an industry are under-skilled in these areas.

We have decades of research and tools for testing and verification of software. Property tests, dependent types, formal verification and proof, etc. The paths have been there, we have just collectively prioritized other things.

It requires an intentional shift in how we design and build. That shift is the harder part.

That's not at all what I meant.

I mean "We can't build X because our code structure makes that difficult" has an opportunity cost of the value of X.

I don't think the future of dev work is being a bureaucrat. I've done more rigorous engineering the last two years than I did previously. I'm more confident in the things I shop and they were built in a fraction of the time. It's a bright future for software engineering.

I'm not talking about time. I'm talking about safety. The amount of times I've seen "I refactored it, but I'm not confident enough to take it to prod" is significant. Being able to go faster but still not ship it is the huge opportunity cost.

The cost of restructuring has also gone down.

The cost of shoring up behavior with tests ahead of a restructure has gone down because of AI.

The cost of implementing a zero downtime migration has gone down because of AI.

A big part of the rust hype has been the low cost of restructuring within an application, even before AI. And now even more so.

The opportunity cost of not being able to safely restructure has gone up substantially.

This is the number one thing I optimize for now: the ability to quickly and safely change significant parts of the code and product.

MTP on a MoE is hit or miss. If you're bottlenecked on memory, MTP can increase the number of active experts (like any batch processing would), which can eat away gains from it.

Generation is basically just memory bandwidth math.

Each token has to read all the active weights. I think that's around 40B parameters active. At a 4-bit quant that's 20GB. With 100GB/s (replace with whatever your bandwidth is) and you get 5 tokens per second.

My favorite lens on SQLite is that it is actually two things:

1. A robust durability implementation 2. A library of high performance data structure and algorithms

The fact this it's SQL is nice, but those two attributes are what make it great.

For example, I'm implement an in-process event log that I want to be durable. I started simple, but soon saw some edge cases and instead of playing whackamole I just swapped to using sqlite as an ordered kv store that gives me ACID.

Another example: ingesting multiple inter related datasets. Instead of a dozen hash maps in memory, I load them up into sqlite (no persistence) and then slice and dice as I need to.

It's a super useful tool.