swimwiththebeat

> Tensordyne is expected to accelerate AI inference with a logarithmic number system that leans on a property of logarithms: The log of A times B equals the log of A plus the log of B. So, storing numbers as their exponents lets the chip add where it would otherwise multiply. That matters in silicon because multiplier circuits draw more power and use more die area than adders do. Tensordyne says its rack-scale hardware, called Napier, can produce up to 1,300 tokens per second per user, and can do so while using less than a tenth as much power as comparable Nvidia hardware.

Did not know about this cool trick about storing numbers as exponents! Is there a name for this technique? Wouldn’t there be overhead in converting back and forth between the exponent and the number?

This is really cool, great way to reinforce our understanding of model architectures! But how is the author confirming that these model architecture implementations are correct though? I don't see any details in the README.md.

I definitely experienced this in training for my marathon and half Ironman (which I had stupidly signed up for in consecutive months). I didn't follow my training plan properly and tried to cram all the volume in the last month or two and it didn't end well. I was able to make it through the half Ironman through pure adrenaline, but I laid in bed for the following two weeks and barely moved. I had no motivation to do anything whatsoever.

It turns out that I was deficient in plenty of vitamins: D, B12, iron. I'm sure these were all symptoms of OTS. I didn't feel the same in any of my workouts until maybe 4 months later. Even then, it took very gradual build-up after those 4 months to get back to where I was. Taking vitamin supplements helped, sure, but the true cure was just ramping down the exercise a lot and getting proper sleep over months.

I empathize with these athletes because exercise is addicting; it feels almost morally wrong to take days off. There's also always someone better than you. But as I got older, I realized that recovery is just as important (if not, more important) than the actual work. In fact, that's where all the gains are made.

I found a few helpful pointers with structuring my workouts now is to:

* Increase training volume <10% W/W. It gives your body time to adapt and also helps with preventing injury.

* Mixing in recovery weeks is also really helpful with helping your body adapt.

* There's no way around doing a lot of volume when training for endurance events, but keeping your HR lower the whole time is a good way to make sure you don't overtrain and can show up the next day.

Your blog always has such informative visualizations that make these concepts easy to digest! Which tools do you use to create them?

So is this a new technique of doing computations within existing DRAM to overcome the memory wall issue of modern computing?

I just finished this book. I thought it was a great read, capturing most of the problems (with product, sales, marketing, competition) Nvidia faced in its life and how exactly the team solved them each and every time. It also goes into how the management, org structure, and culture have empowered its engineers to find creative solutions to staying alive in the brutal graphic chips industry and continuously discover and exploit new market opportunities.

I learned a lot about how important not just superior technology, but better operations, marketing, sales, and culture are all critical to a successful business.

The only con of this book is that it skips over some parts of nvidia’s history like the short-lived crypto boom, failed acquisition of ARM, etc. It’s still just a minor flaw in an otherwise great book though.

Is this open-source?

She definitely knows, she’s just trying to avoid any chance of future litigation by feigning ignorance. Makes sense since OpenAI’s been getting a lot of bad press for using copyright data in training their models.

As someone who wants to learn and explore more areas of computer science and programming, this outline and centralized list of resources is incredibly helpful! Thanks so much.

Thanks for taking the time to answer the questions and link those resources, really appreciate it and the work your team did!

Thanks for replying, that's a perspective I didn't consider. The capability to "talk to your data" just seems so enticing as a solution that I was tunnel-visioned into that UX. If I'm understanding correctly, what you're suggesting is more of a SQL assistant to help people write the correct SQL queries instead of writing the entire SQL query from scratch to answer a generic natural-language question?