smj-edison

“The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise.” — Edsger Dijkstra

I've thought a lot about this quote, and the best abstractions I use are absolutely this. I love array indexing as an abstraction, but in the back of my head I have pointer + index * size. I like having the kernel do memory mapping, but I know there's a buddy allocator and TLB cache there.

Agreed on the limits, they're quite frustrating. I've still been preferring Argon to Opus 5.5 though, because even though Argon gets details wrong, I find that it's much more creative and willing to explore ideas. In my experience Opus tends to be rather condescending and won't make room for less orthodox ideas to develop. For what I'm doing, I'd much rather have good ideas that I implement mostly by hand rather than regressing to the mean and have the LLM write the code.

One other thought, I think another aspect of biological sciences is just how much effort it takes to do research. Hypotheses are a dime a dozen, but actually verifying these is quite difficult. It takes a lot of patience, time, attention to details, and even then it often fails to replicate.

I don't think LLMs will just magically be able to connect all the dots, because "garbage in, garbage out" still applies. And there's just too many invisible intermediate states, there is literally just not enough information to fully constrain a model. Hence my interest in scaling computational chemistry to get these intermediates :)

Computational biochemistry is what I'm currently pursuing, but I'm pretty new to the field so take everything I say with a grain of salt: Biochemistry is a really really really tricky problem, for a couple reasons:

1. Biology has a very nasty habit of not cleanly abstracting. It's not like we can treat every protein as a "diffuse mass" without losing a ton of important interactions. This is unlike something like CFD where you can treat a mass of water molecules with certain constants. Think of something like cancer: a small mutation to a DNA strand can cascade into a tumor. You have to deal with the mutation happening at an atomic level while scaling to a whole organism (yes I know there's models that can approximate this, but they can't capture all the interactions).

2. This brings us to modelling at the atomic level. To model a system of atoms with the Schrödinger equation, it naively scales O(6^n) per particle. Note that means every nucleus and electron, so if you have say 10 carbon atoms and 22 hydrogen you're already at 10^127 operations. We can cut corners a bit and end up with something like CCSD which scales O(n^7), or cut more corners and scale O(n^3) for DFT. DFT is pretty much the last method where bond forming and breaking is modelled, so after more corner cutting you lose losing reactions, which are important for understanding why something is happening. Most protein folding simulations (moreso prior to alphafold) run a very crude approximation that models atoms as balls on springs (things like AMBER and CHARMM), which obviously can't handle reactions.

3. If this wasn't bad enough, you also can only move through time at about 1 femtosecond (10^-15), so it takes a ton of simulation steps to get even a nanosecond in. Forget millisecond or second lengths where things start to happen.

So, while the math of the physics exists, computationally it's practically intractible. There's some exciting work with something called Machine Learning Interatomic Potentials (MLIP) which approximate DFT using a neural network which are about 1000x faster than DFT, and iirc scale much more linearly (look at things like MACE and orbmol if you're curious). So we're making progress, but we've gone from so ridiculously impossible to pretty much impossible for large systems.

I have a couple ideas about developing custom microchips to make this faster, which is why I'm pursuing this, but there's still a massive way to go!

The problem with lifetimes is they become a object tree, where Rc and co let you represent an object graph (well, a DAG at least). So when your problem resembles a tree, it's great, but some problems don't like being forced into that shape.

You don't have to deal with Rust's aliasing rules for &mut and &.

Unless you use cgroups or turn off overcommit :) I'm also hoping to use the core interpreter on memory-limited devices like an ESP-32.

Oh that's exciting! I knew about the allocator work, but I hadn't heard about error reporting for failed allocations, or that it was about to be stabilized.

I'm one of those people who's adopting Zig :) But I don't think it's best for every project. For context, I've used Rust before for probably two years writing a realtime audio synthesis engine, so I'm fairly familiar with Rust vs Zig for handling low level details.

The biggest reason I use Zig is it's a very explicit language. The creators made a very intentional decision to avoid too many "high level" designs. This doesn't mean there's no capabilities for abstraction (comptime is great for that), but when you see array indexing, you can think "ptr + index * size with bounds check". There's lots of other things like that where the language does exactly one thing, and that thing is a low level operation.

This is terrible when you want to create high level abstractions that hide details from the programmer, but it's what I need when doing realtime audio synthesis or what I'm doing now which is writing an interpreter. I know exactly what allocates, I know what calls IO and can block (both operations explicitly take in an allocator or IO parameter), no data structures have private fields so I can always poke around at the insides. I know what types of errors a function returns, and creating errors is cheap with Zig's error union design.

So I don't use Zig because I think it's the safer language, I know it has sharp edges because of the number of times I've caused a panic on a poisoned pointer or use-after-free. But because it gives me such a transparent view into what is happening I find it liberating.

I think this is fair the majority of time, but I'd like to mention Zig, since Zig has a convention of all allocations being fallible and handled. It's a pain at first to handle error.OutOfMemory at each allocating site, but I feel like I'm much more conscious of where allocation can fail and how to gracefully handle it. I've also gotten a lot better at transactions since pretty much every operation has failure points now.

In fact I've written a whole interpreter that can recover from OOM by raising a recoverable exception to the user. It's really only because Zig made recovering idiomatic, and I'm not sure I could've done it in another language (maybe Rust but I'd have to rewrite large parts of stdlib to both return an error and take a custom allocator).

> Despite the uncertainty, there are some clear messages we can send to the next generation of mathematicians. The first is that we stand with you. ...Second, mathematics is as important today as it ever was, and we need you. AI must not replace our collective ability to reason and deliberate, and mathematics remains a core capacity for doing so. We cannot imagine a world in which mathematics does not play an important part in our lives.

This is a really uplifting message. I'm currently in school pursuing a degree in applied math, and I've definitely felt unsettled with the recent advancements with automated proving. So it's nice to know that there's still a place for learning mathematics, and that we can find a new way forward.

I remember reading about their setup and it blew my mind. They have MEMS mirrors that can rotate to redirect one of thousands of fiber lines into any other fiber line by using other optics. That way you can take in like 1000 lines and optically connect it to any of the other 1000 lines.