- The NeurIPS Mirage paper vs Jason/Yi's Emergent Abilities paper
- The Efficiency Misnomer - reminder on Arvind Narayanan's recent callout that "active params" isn't actually the same thing as lower cost, and also not the same thing as faster inference
- Echoing Founders Fund's skepticism that Open Source AI can have any real lasting impact
Special interest topics we touched on:
- why encoder-decoder is not actually that different than decoder-only architecture (Reka is notably enc-dec vs GPT which is dec-only)
- why the "Noam architecture" is All You Need
- The chaotic vs stable periods of spinning up GPU clusters ( per this great post I submitted https://news.ycombinator.com/item?id=39609997 )
- The NeurIPS Mirage paper vs Jason/Yi's Emergent Abilities paper
- The Efficiency Misnomer - reminder on Arvind Narayanan's recent callout that "active params" isn't actually the same thing as lower cost, and also not the same thing as faster inference
- Echoing Founders Fund's skepticism that Open Source AI can have any real lasting impact
please AMA! (Yi will see this)