musebox35

> Now I can't get an interview and the only employment option came through a contracting referral from my network.

The more senior you are, the more you will need to rely on network referrals. Most HR processes are designed around filling average positions and filter for it especially when the number of applicants are high. I switched jobs around 2024 and all the interviews I got was through referrals or directly reaching out to the founders. I got zero callbacks from applications to HR.

Also five years ago, we had a once in a generation pandemic that forced all retailers to go online at global level, pushed remote work to be the norm and the tech workers suddenly had access to a growing global job market. That was the exception not the norm.

I think such progress by agents is not a sign of broad generalization but of broad coverage. We have exposure to a subset of deeper scientific subfields and thus can only generate certain attacks to solve a particular problem. Since it is not clear which combination will lead to a solution beforehand it is nontrivial to look at a problem and fill our knowledge gaps. LLMs on the other hand have broad coverage and can generate hypothesis on a wide combination of subfields. With Lean an agentic loop can test these to sift the weak ones. In a way the problems solvable with this setup is also solvable by a human who happens to know the right subfields. These problems are likely to require an esoteric combination so nobody could solve them before. I really am not sure whether all generalization is like this or we can leap and create novelties beyond what an llm can generate. That I guess is the tough question that we need to answer to understand the boundaries of intelligence.

I agree that it is a bit weird, kind of coding in natural language but also not exactly like that. Maybe as the agents stabilize and there are proper new tools for this new process we can relax a bit and it will become the new normal. For now it feels like surfing over an ever changing seascape, exciting but tiring at the same time.

> Some things you need to understand-others, not so much.

This is the core of the matter though, knowing what you need to understand and what you can ignore is the actual programmer's skill. It requires you to have a clear mental picture of both what you are trying to build and what the underlying machine will do when you are finished.

You need to understand the abstractions, but also where they leak, when they won't match reality, and how. This is why knowing computer architecture and assembly helps you to optimize your code even if you are coding in a high level language.

The problem with coding agents is that they are tuned to work on all contexts so they always fill an underspecified request by optimizing the average case and often without stating all the assumptions that they make. So you still need to understand what you specified and what got filled in automagically by the agent. My experience is that they (even the paid frontier models) are poor judges of the most important assumptions they make, which will might be corrected by a prompt or a tool output in which case it is fine. Otherwise it will be ignored and steer the model into a weird loop. It then tries to fix things but can not do so since its mental model is totally broken now.

Do not get me wrong, I am so happy to let the agent handle tool building (especially those that involve a web UI) and fill in the CLI command line argument parser. But every time I trust the agent by relying on it to drive the mental model of what we are doing, I got seriously bitten. Well, maybe that should not be surprise me, but I can understand the confusion of less experienced programmers and non-coders. It must really be frustrating to be able to build so much, but also not to be able to fix seemingly small issues.

It was Nvidia’s competitors’ job to ensure this never happened but hardware companies rarely value the software stack as much as they should have. AMD screwed up several times to build a similar tool and Intel did not manage to create a proper programming model for their vector instruction sets despite having some promising internal efforts. Cuda allowed AlexNet to usher the deep learning era, written by university researchers. The flash attention implementation allowed quadratic attention to have reasonable memory requirements again written by a university professor and used by everyone. If I were to complain about Nvidia, I would complain about them abandoning the personal computing track for datacenter profits, definitely not about pushing Cuda.

This is one question that is not just for kids but for all "gpu poor" adults as well. One concept that I found useful is "toy models": small compute models that replicate a certain, not very well explained phenomenon that the large models have. Of course, you can't have a toy model for everything, some behaviors are only visible at scale. But, there is a bunch of things like in-context learning that you can play with without burning a ton of compute.

The best analogy is strip mining (big labs) vs cave exploration (solitary/small teams). I think this is how science progresses at the boundaries by smart/curious/hardworking individuals because depth is a requisite for finding the right questions and then the answer. It is not for everyone and it does not always work. But you learn a ton even if it doesn't pan out to be a big breakthrough.

A more apt analogy is the diminishing understanding of assembly language and machine instruction sets. I love programming and something got lost in both transitions, first to high level languages and now to coding agents. I am quite sure some contexts will require programming by hand for a long while as there are still people who are writing amd64/avx512/sass by hand. I hope in time we will find new ways to enjoy solving problems with this new programming modality. By now, I am pretty sure it is here to stay and evolve from its current crude form.

That also surprised me, the diffusion gemma is actually a bunch of architectural changes to make discrete diffusion practical and an sft + distill of the Gemma 4. One cool trick they do is to take advantage of the entropy of the estimated probability distribution to adaptively adjust the required number of diffusion steps for inference. Image diffusion only yields the score function, a gradient instead of the probabilities, so the same trick does not apply.

This is one of the dilemmas that I am trying to wrap my head around. I love optimizing software pipelines, which often boils down to figuring out the operational constraints that the compiler and the generic libraries can’t assume. Then I exploit these to squeeze out performance. But in a world I can start from scratch and code a domain specific solution from line zero in a matter of hours/days, I do not need general libraries as much as I used to. On one hand the code won’t be as well tested as a good general library. On the other hand, it also won’t have a plethora of library bugs that are there because the code is generic and opaque. One counter argument is that things are never static and you can’t have specific code for too long. A counter to that is that you can then change the code to be specific to the new reality at very low cost. This is the mental loop I ride constantly. Disclaimer: My circumstances are definitely not general, I am not writing code that is truly large scale.

I would add that it is not just capacity, but also negotiation ability. With scale comes the ability to negotiate better prices than everyone else. Even if you can find capacity for your smallish user base, your inference cost can not match these companies unless you have a technical advantage for your inference cases. Squeezing the hardware requires request batching and caching which are far easier at scale and sustained user activity.

The biggest advantage of tpus is the high bandwidth fiber optic interconnect between them that allows distributed computing on pods with thousands of tpus and the co-design of cooling systems that go with their racks. I do not think that we will see personal tpus any time soon.

Sadly no. Despite the name, the experts are not routed per concept or topic but per token. So for the same sentence you might activate multiple experts for different tokens. What it solves is the distributed training and inference problem. As long as each expert fits a single gpu, coordinating the model evaluation is much easier and it is faster. It does not buy as much for running on a single device though still less costly than a dense version.

dev variants are usually cfg distilled which means that directly finetuning isn’t as effective. In the past, for the flux2 klein models,they released base versions that are not distilled. So it will probably be a while before you can fully take advantage of the open weights.

Thanks for posting this. This is the tinkerer mentality. It is not for everyone, but certain things can only be learned in that way. It is the best antidote to AI paranoia. There is much that does not transfer between frontier models and local ones. There is that. But you can not tinker as much as you can with the former.