chabons

One of the core claims made in the paper is "To inspect any latent reasoning span, one can easily recover its semantics by mapping each latent step produced by HybridLink to its nearest token embedding, as in Eq. 5". They show some charts to support this.

In doing this however, you lose meaning where the model is expressing combinations of tokens in a vector, and your logging/monitoring outputs just the closest one. You would need another model going through and outputting the full meaning being described (with more than one output token per reasoning token)

I wonder if their observability claims will hold up on a model trained from scratch to reason in latent space.

Sure, but even heavily discounted Nvidia chips won’t be competitive for inference if they’re worse on perf/W.

As opposed to closed-source models? Benchmarks for GPT Sol wouldn’t be particularly meaningful, as no one else can run the benchmark, and we don’t know what the exact model specs are.

Picking the best open source models is really the best they can do.

They didn't even attempt to make a clean-room implementation, they just had the LLM clone it for them with access to the source.

> "[…] and these interlopers are the real geniuses […]

This wasn’t the framing I was going for. My point was that industry boundaries are fluid and expand/contract as demand dictates. You’re correct that incentives are not all positive (pay, work life balance, perks), other industries contracting might force people to find work elsewhere.

All that said, I don’t think these people have “odd backgrounds”. I work in a math-heavy domain, so these backgrounds make as much sense as a traditional CS background, and I think these folks are just as likely to be retained in a crunch.

“There is a population of qualified workers […]”

In my experience, this is not true. Demand for software engineers has been so high, and pay so high as a result, that it’s pulling in workers from adjacent industries. The total software-qualified workforce is larger than the set currently working in software, and people with transferrable skills move in and out of software as incentives dictate.

A number of my current and former coworkers are from math and physics backgrounds (CFD, energy, etc…). These are folks that before might have stayed in academia, or ended up in aerospace, defense, or other engineering fields.

If everyone over hired, demand drops, and companies drop pay as a result, I’m sure we’ll see some folks in software with transferrable skills move to other industries.

Where in academia can one get a Billion (with a b) dollars to research something?

Developers are users too. If users cared enough not to use cross-platform software because it didn’t look native, then it wouldn’t get built.

"rather hard" is really underselling things. Without considering generator->motor losses, 250W is a lot of power, even for a larger (>80kg/176lbs) rider. For basically anyone who doesn't train on the bike, this is an unsustainable effort.

Even a floor of 100W would rule out smaller/less athletic riders.

I'm missing something. What change in demographics are we talking about, and why would that influence math results?

Garmin has a market cap of ~40B USD. Inflation adjusted, that's where NVIDIA was <10yr ago. It's not missing resources to do the work IMO, but it's missing the market dominance to singlehanded set the standard for the industry.

For me, it's that the ergonomics are straightforward, and everything works out of the box. If I find myself on a new machine, just installing fish gives me an ergonomic setup without having to install too many additional tools or mess with configuration.

I don't have statistics, but given the student visa -> H-1B pipeline changes, it would seem there are a number of H-1B holders who are educated in US colleges (either at the undergrad or graduate level). This indicates that the problem is not entirely a training gap.