aseg

Joined 4 years ago

Karma
163
Submissions
18
Member since
Nov 9
Postdoc @ Caltech working on self-evolving agents for scientific discovery. Reach out at atharvas.github.io
View on Hacker News

Recent submissions

Comment · 9 months ago
Gordo and Bruce are pioneers in the gliding world. One of their coolest flights that shows their creative flight planning shows up in their 3000km flight in the Sierra Nevada's, and the build up to it.

Some basics: The major challenge in flying gliders is the inherent stochasticity in the weather system. Think of it as a contextual bandit problem with high variance w.r.t local weather (i.e. Even the best planning cannot help if the weather doesn't comply). We have some observability due to forecasting tools (skysight.io) and any policy must have affordances for pilot skill and a margin of safety. A good pilot (or 'policy') starts with multiple plans, quickly modifies to plans to suit the environment, and can seamlessly switch between plans. The primary "reward signals" are duration of flight, distance covered, and (in competitions) hitting certain waypoints.

Previous WR's for longest flight were mostly in the Andes or Alps. You want to be in a mountain range to utilize either the [ridge lift](https://en.wikipedia.org/wiki/Orographic_lift) of a mountain face or [mountain wave](https://en.wikipedia.org/wiki/Lee_wave), ideally in a polar region during the summer to maximize the daylight hours so you can fly under VFR for longer.

However, while the Sierra Nevada's have great mountain wave and ridge lift, the number of daylight hours is not ``competitive''. Their main innovation was in acclimatizing themselves with using night vision goggles for long duration in a glider. There's an article on this [here](https://magazine.weglide.org/gliding-at-night-breaking-the-3...) which describes the acclimatizing flights and the 3k km flight in great detail. It doesn't get official recognition because the FAI requires the flight to be done in daylight, but still an extremely cool flight!

Comment · 1 year ago
Comment · 1 year ago
Comment · 1 year ago
In gliding, tow upsets are pretty common and, in rare cases, can be fatal. An out-of-position glider out can _easily_ and very quickly overcome the tow planes elevator authority (ability to pitch up or down) which leads to accidents like this. This video does a good job explaining explaining the root causes and potential dangers (https://youtu.be/5cpqFzhM9dY?si=J7GxP1dI9Xopy3xu). Also read the comments from testimonials from other glider pilots.

This is my biggest concern with this concept as well. Towing things is challenging because the tow plane's center of gravity can change drastically depending on the forces on the glider it is towing -- if the glider deploys its spoilers / crabs in a crosswind / gets in your wake turbulence you're not going to be able to predict how it changes your CG (and your control authority) without training or experience. Also, with gliders, the tow plane is traveling at around 60MPH to 90MPH, with a decision window of 2-3 seconds. Commercial planes travel at ~500 MPH... The concept seems like a hard sell to the pilot unions. I bet they've thought about this though.

Comment · 1 year ago
This is my research area. I just finished reviewing six NeurIPS papers (myself, no LLM involved) on LLM Agents for discovery and generation and I'm finding that evaluating LLM agents on raw performance for a task isn't as insightful anymore -- every paper is claiming state of the art 10x performance boost by {insert random acronym that devolves into combinatorial search}. Rather the true test for such algorithms is whether the empirical scaling curves for these algorithms are more computationally amenable than an existing baseline search algorithm (like CoT).

Three motivating points:

- GEPA / evolutionary agents are performing a zero-th order (no gradient) optimization in a combinatorial space. Their loss curves are VERY noisy and stochastic. If we run such agents multiple times, the performance variance is extremely high -- and in some cases cancels out the gains from single experiment. However, obtaining the error bounds is hard because the API costs are pretty restrictive.

- The problem we face with test time scaling is not that prompt engineering is ineffective/less effective than fine-tuning. It is that fine-tuning _reliably_ increases performance for a model for any subset of tasks and the scaling curves for performance per additional data token are well understood.

- Test time optimization techniques work well on in-distribution problems (e.g. generate and debug this Python code) but fail pretty badly on even slightly out of distribution problems (e.g. generate and debug this Julia code). Compare this to gradient search -- it wouldve been so fascinating and confusing if SGD failed to optimize a CNN image classifier on COCO but worked very well on ImageNet.

How do people feel about this? Does this line up with your viewpoints?

Comment · 1 year ago
Assume you have data for Hooke's law (a spreadsheet with F, x, and other variables) and you want AlphaEvolve to give you the equation ``F = -C_1*x``.

Let's say the model hallucinates in two directions:

1. "There is a trigonometric relationship between variable F and x". It expresses this as ``F = -C_1*sin(x)``. You fit the constant C_1 w.r.t the dataset, execute the program, and your best fit has a high error. You can discard the program.

2. "There is an inverse linear relationship between variable F and x". Now it expresses this as ``F = -C_1*x``. You fit the constant C_1 w.r.t the dataset, execute the program, and your best fit has extremely low error. You now know for sure that you're on the right track.

Comment · 1 year ago
Comment · 1 year ago
Finally—something directly relevant to my research (https://trishullab.github.io/lasr-web/). Below are my take‑aways from the blog post, plus a little “reading between the lines.”

- One lesson DeepMind drew from AlphaCode, AlphaTensor, and AlphaChip is that large‑scale pre‑training, combined with carefully chosen inductive biases, enables models to solve specialized problems at—or above—human performance.

- These systems still require curated datasets and experts who can hand‑design task‑specific pipelines.

- Conceptually, this work is an improved version of FunSearch (https://github.com/google-deepmind/funsearch/).

- In broad terms, FunSearch (and AlphaEvolve) follow three core design principles:

    - Off‑the‑shelf LLMs can both generate code and recall domain knowledge. The “knowledge retrieval” stage may hallucinate, but—because the knowledge is expressed as code—we can execute it and validate the result against a custom evaluation function.

    - Gradient descent is not an option for discrete code; a zeroth‑order optimizer—specifically evolutionary search—is required.

    - During evolution we bias toward (1) _succinct_ programs and (2) _novel_ programs. Succinctness is approximated by program length; novelty is encouraged via a MAP‑Elites–style “novelty bias,” yielding a three‑dimensional Pareto frontier whose axes are _performance, simplicity,_ and _novelty_ (see e.g. OE‑Dreamer: (https://claireaoi.github.io/OE-Dreamer/).
Pros

- Any general‑purpose foundation model can be coupled with evolutionary search.

- A domain expert merely supplies a Python evaluation function (with a docstring explaining domain‑specific details). Most scientists I've talked with - astronomers, seismologists, neuroscientists, etc. - already maintain such evaluation functions for their own code.

- The output is an interpretable program; even if it overfits or ignores a corner case, it often provides valuable insight into the regimes where it succeeds.

Cons

- Evolutionary search is compute‑heavy and LLM calls are slow unless heavily optimized. In my projects we need ≈ 60 k LLM calls per iteration to support a reasonable number of islands and populations. In equation discovery we offset cost by making ~99 % of mutations purely random; every extra 1 % of LLM‑generated mutations yields roughly a 10 % increase in high‑performing programs across the population.

- Evaluation functions typically undergo many refinement cycles; without careful curation the search may converge to a useless program that exploits loopholes in the metric.

Additional heuristics make the search practical. If your evaluator is slow, overlap it with LLM calls. To foster diversity, try dissimilar training: run models trained on different data subsets and let them compete. Interestingly, a smaller model (e.g., Llama-3 8 B) often outperforms a larger one (Llama‑3 70 B) simply because it emits shorter programs.

Comment · 1 year ago
Comment · 1 year ago
If you're looking to run a .exe file, there are a couple of hypervisors in the market (sometimes found on the high seas). I've tried these on a couple of obscure .exe files:

- Parallels

- VMWare fusion

- Apple's gamekit

Parallels has had the highest coverage, but they've locked down Parallels pretty well, and there is no way to outright own a copy of Parallels (monthly subscription). I think the gamekit project is the most exciting one of these three though!

Comment · 4 years ago
You might find (Cramer, 1985) interesting! IIRC they go into exactly this problem. However, I can't find an open pdf to link to unfortunately. I'll edit this list tomorrow morning in case I misjudged (Cramer, 1985) or if I find a link..

Here are two hand-wavy arguments that may not be 100% correct:

* Structured Bias: A symbolic regression term allows you -- the scientist -- to control exactly what sort of expressions you expect to see. If I'm looking at data coming from a spring, I expect to see a lot of dampened sinusoids and little quantum physics. SR gives you control over the "programming language" while parametric regression will only allow you to change the number of parameters (not useful in this context).

* Generality: A regression term guarantees the best fit parametric equation as long as you have a comprehensive sample of your data range. A symbolic expression (most of the time) extrapolates beyond the provided data range. In fact, this is one of the constraints in the main proof in the paper (f* should generalize)! Basically: If I only have data for sin(x) from 0 to \pi, PR will find the best fit but there is no guarantee that the best fit will also work in the range \pi to 2\pi.

I want to stress that these aren't established facts and each of these pros actually introduces a lot of cons in the process (what if you introduce incorrect structured bias / what if the "general/simple" solution is actually a little imprecise... Like Newton's laws vs Einstein's theory)! This just means that there is plenty of exciting work to be done!

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton