megadragon9
Joined 12 years ago
Karma
73
Submissions
50
Member since
Mar 13
Getting back on the right track.
View on Hacker NewsRecent submissions
Comment · 9 days ago
This reminded me of a question I've been thinking about lately. 6 months ago, I was thinking this Forward Deployed Engineer (FDE) is a temporary thing until companies build enough expertise and this expertise fully permeate the job market. But now I have a weird feeling that this FDE job category is here to stay in the long-term. There seems to be some parallels to the system integration folks back in the 2000s when internet adoption just kicked off. OpenAI and other AI companies invested pretty heavily in their FDE efforts, that's along side their token business. Perhaps a lower risk & profitable strategy than starting companies themselves?
Comment · 1 month ago
this makes me wonder, does having kids change the list of things on your list? and why?
Comment · 2 months ago
I think it's more about the scar tissue (a.k.a. intuition) when building LLMs from scratch than whether it's transferable to job search. Maybe the person will decide they don't like LLMs and not develop that into a career, or maybe they become a researcher in another field because that's a better way to solve problems inherent in LLMs.
Comment · 2 months ago
I did something similar but with smart home devices. I use the homebridge interface to connect my smart home devices to Apple's homekit protocol. Some homebridge plugins for my devices were outdated and no longer maintained, so I asked Codex/Claude to help me create a patch of it as a local fork, so my smart home devices can still run without problems.
It does feel magical when these agents can debug in the real-world, like turning on/off my living room lights and using another living room camera to take a snapshot of the living room to see whether it worked or not.
Comment · 2 months ago
i created something similar a while back. Inspired by micrograd for showing the connection between math/calculus and code, then building along the way to a full NumPy deep learning library that I pretrained GPT-2 124M model with it. One way to learn is to trace through the PRs merged to the repo in chronological order.
The philosophy is the same: "What I cannot create, I do not understand." https://github.com/workofart/ml-by-hand
Comment · 2 months ago
I did something similar. It started off as a "self-improving agent" project, inspired by autoresearch, then later on I reframed it as "harness training" (discrete program search) borrowing the mental model from ML training.
I "trained" the harness on a subset of Terminal-Bench 2.0 tasks while keeping the LLM (local Qwen3.6-35B A3B) frozen. Making LLM inference and the task environments fully deterministic was necessary for clean credit assignment. I learned this the hard way after spending the initial 1 month on experiment noise.
My final results showed that on the full 89-task Terminal-Bench 2.0 suite, the trained harness matched or beat the official Terminus 2 harness for four LLMs that it never collaborated with during training (e.g. GPT-OSS-120B score increased from 18.7% to 36%, while using 55% fewer input tokens per solve). A harness trained only on SWE-bench improved Terminal-Bench scores too. Here's the write-up: https://www.henrypan.com/blog/2026-07-18-harness-training/
I packaged the training loop as a PyTorch-style framework. https://github.com/workofart/harness-training
Comment · 3 months ago
I worked on this project (https://github.com/workofart/harness-training) for the past few months to reframe "Agent-driven Self-improving Harness" to "Harness Training".
The idea is simple, the harness is trained once with a frozen task LLM against a given task environment. Then you can then swap out the task LLM to any model and evaluate the "frozen trained harness" with any task LLM on any new task environment.
Since this was a general problem, I took the chance to create a general PyTorch-like training framework. Right now, you can train with any OpenAI-compatible API for interfacing with the task LLM and train against Terminal-Bench or SWE-Bench tasks, but you can easily extend it to support any task environments.
I wrote a blog post (https://www.henrypan.com/blog/2026-07-18-harness-training) on this journey, including (but not limited to):
- results from using this harness training framework to improve general capabilities across many task LLMs to beat Terminal Bench 2.0 (Terminus Harness) and also transfer learnings towards better task-solving abilities in unseen task environments (e.g. harness trained on SWE-Bench tasks solving Terminal Bench tasks)
- how this framework is built
- learnings on what was missing in my initial version of the project (hint: determinism)
Comment · 4 months ago
Interesting project. Do you think manual memory management help understand computational graph lifecycle better, or does it distract from backprop itself?
btw, I went down the micrograd path with numpy-primitives all the way to building a PyTorch clone that can pre-train and post-train LLMs (https://github.com/workofart/ml-by-hand). My learning focus was on the math/calculus <-> high-level APIs, instead of efficiency. I'm glad to see more people tackling this problem from different angles.