learndeeply

Hilarious name for an open source project released by a government lab.

> For me, it was simply a gut feeling. I’ve been talking to founders and doing deep dives into technology companies for decades. It’s been my entire professional life as a writer. And because of that experience, there must be a pattern-matching algorithm churning away somewhere in my subconscious. I don’t know how I know, I just do. SBF is a winner.

This is a fantastic insight into journalists. They need to fit their story into a narrative. If you match their stereotype, then they'll spin whatever tale they can imagine to convince themselves that it's really true. The level of self-deception that's needed is difficult to comprehend for me.

Is it common to have the same number of employees as there are companies funded at a VC? Is each one assigned a company?

Brilliant way to get thousands of tech people to look at your business card.

Edit: This was updated, it used to point to his real Github username. Not that there's anything wrong with that! It's clever marketing.

Aside from the project itself, the production value of this video is crazy good.

JAX is a DSL on top of XLA, instead of writing Python. Example: a JAX for loop looks like this:

   def summ(i, v): return i + v
   x = jax.lax.fori_loop(0, 100, summ, 5)

A for loop in TinyGrad or PyTorch looks like regular Python:

   x = 5
   for i in range(0, 100):
      x += 1

By the way, PyTorch also has JIT.

I can't think of anything that neural nets can't beat, except small tabular data with boosted decision trees. Can you give some examples?

The code is very easy to read. Doesn't seem like there's data/model parallelism support for training, which will be important for real-world use.

> And honestly, their hardware is pretty bad for ML given you can pick up a used 3090 for $600.

How do you know it's bad? Do you have benchmarks?

Just checked, used 3090s are going for $700-900 on eBay.

No, it's meant for taking something like a CSV file and deciding if each row matches a specific category. A common example, have a CSV, columns corresponding to different features of flowers (e.g. number of petals, size of petals) and the output is the type of flower.

My mistake, I was thinking of gpt-neox. Thanks for the correction.

> Tensorflow serving is barely controllable, and requires insane tuning to make it perform the same as pytorch

What are you using for serving PyTorch models?