krapht

I know NVIDIA is dominating the market, but it's not like AMD/Intel/Google don't have credible alternatives. Hipify/OneAPI/MLIR(Jax etc) can be used today, it's just not as ergonomic as CUDA.

Should mention Jax in title, otherwise people won't know why they should use this instead of torch.

spinning up a venv with duckdb: 5 minutes

using sql everywhere; never having to remember dataframe syntax: timeless

Yep. Nobody has repealed Medicare, for example

His example of a web crawler stinks. I can get behind the philosophy, but his specific web crawler example at a minimum needs to retry, respect robots.txt, and rate limit per domain. So no, xargs + curl is a dumb example.

Which is why I've never quite understood the appeal of portable SIMD libraries for performance-critical code. If I'm explicitly writing SIMD rather than relying on the auto-vectorizer, it's usually because I want access to the particular capabilities of the target ISA.

For many problems, choosing the right instruction or instruction sequence makes a large difference. Portable SIMD abstractions necessarily expose some common semantic layer, but SIMD ISAs don't actually have equivalent capabilities. Instructions like pshufb, for example, enable algorithmic tricks that don't necessarily have an equally efficient analogue on another architecture.

If maximum performance matters, I generally want intrinsics and architecture-specific implementations; if portability matters more, I'd rather move further up the abstraction stack and use something designed to target multiple architectures, such as ISPC. There are certainly cases where portable SIMD gets close enough to optimal, but I don't think there's a compiler or abstraction that can express every useful SIMD idiom and lower it equally efficiently across fundamentally different ISAs.

One I spent a few months working on was pathfinding for trucks. The goal is to find dominant solutions over {shortest time, lowest cost (tolls + fuel), avg road speed variance - traffic sensitivity} and then return 3-4 routes that are equal distance from each other in this dimensional space for users to pick from.

As you say, the most useful things happen in low-dimensional spaces.

social media is a completely different market though - since there are massive returns to scale, it's incredibly hard for a new entrant to break in.

model training and inference is the opposite. people switch LLMs like people change clothes in the morning. there are popular services (openrouter) that make moving as easy as changing a model string.

Is Gemini really doomed? I'm still bullish on Google: 1) they have more free cash flow and capital than God due to the ads business 2) they have data - intent from web searches, youtube videos, google books and music 3) they have dedicated inference hardware

for all these reasons, is being 6 months behind the frontier actually a structural, long term disadvantage? some day the pace of improvement will slow, and google will vacuum up the market. they'll be able to compete with open-weight models just on pure cost advantage from their vertical integration

I think that's backwards. Anything that changes the performance of a model deserves a minor version bump. A new model has to be qualified before being pushed to production; but we don't get the choice here, just cross your fingers there are no regressions at all on all possible tasks the model might be asked to do.

News to me. What do you call mixture-of-experts and model distillation?

ironically, any frontier LLM will easily generate a tutorial at any detail you like explaining what these are.

if don't have time for that, just know that these are technical parameters that affect how likely it is an llm will produce the same result after being asked the same question.

It's more rare, and definitely less lucrative, but there are opportunities in software engineering for non-profits, open-source, and smaller non-tech companies. If you've made your blood money and don't need top n% pay, it's something worth checking out.

Slight tangent: a long time ago when I was into this hobby, Oshpark was usually the go-to instead of JLCPCB. What happened to them? Macrofab too if anyone's used them.