cepth

But, I don’t think it’s really debatable that AMD is closing the relative gap, given the ROCm ecosystem didn’t exist until at all relatively recently. I’m guessing the very credible list of software partners now at least trying ROCm (https://www.amd.com/en/corporate/events/advancing-ai.html#ec...) are not committing time + resources to an ecosystem that they see as hopeless.

---

Final thoughts:

A) It was completely rational for AMD to focus on devoting the vast majority of R&D spend to its CPUs (particularly server/EPYC), particularly after the success of Zen. From the day that Lisa Su took over (Oct 8, 2014), the stock is up 50x+ (even more earlier in 2024), not that share price is reflective of value in the short term. AMD revenue for calendar year 2014 was $5.5B, operating income negative 155 million. Revenue for 2023 was $22.68B, operating income $401 million. Operating income was substantially higher in 2022 ($1.2B) and 2021 ($3.6B), but AMD has poured that money into R&D spending (https://www.statista.com/statistics/267873/amds-expenditure-...), as well as the Xilinx acquisition.

B) It was completely rational for NVIDIA to build out CUDA, as a way to make it possible to do what they initially called "scientific computing" and eventually "GPU-accelerated computing". There's also the reality that Jensen, the consummate hype man, had to sell investors a growth story. The reality is that gaming will always be a relatively niche market. Cloud gaming (GeForce Now) never matched up to revenue expectations.

C) It’s difficult for me to identify any obvious “points of divergence” that in an alternate history would’ve led to better outcomes with AMD. Without the benefit of “future knowledge”, at what point should AMD have ramped up ROCm investment? Given, as I noted above, in the months before ChatGPT went viral, Jensen’s GTC keynote gave only a tiny mention to LLMs.

D) If anything, the company that missed out was Intel. Beyond floundering on the transition from 14nm to 10nm (allowing TSMC and thus AMD to surpass them), Intel wasted its CPU-monopoly years and the associated profits. Projects like Larrabee (https://www.anandtech.com/show/3738/intel-kills-larrabee-gpu...) and Xe (doomed in part by internal turf wars) (https://www.tomshardware.com/news/intel-axes-xe-hp-gpus-for-...) were killed off. R&D spending was actually comparable to the amount spent on share buybacks in 2011 (14.1B in buybacks vs 8.3B in R&D spending), 2014 (10.7B vs 11.1B), 2018 (10.8B vs 13.B), 2019 (13.5B vs 13.3B) and 2020 (14.1B vs 13.55B). (See https://www.intc.com/stock-info/dividends-and-buybacks and https://www.macrotrends.net/stocks/charts/INTC/intel/researc...).

And, for nuclear modeling (be it weapons or civilian nuclear), do you know if AMD engineers can get involved? (https://insidehpc.com/2023/02/frontier-pushes-boundaries-86-... this article claims "86% of nodes" were used on at least one modeling run, which I imagine is among the larger jobs)

> It is and has been miles beyond the competition and that's clearly all you need. Nvidia has > 90% market share and is worth ~10x AMD. 17 years of focus and investment (30% of their R&D spend is software) when your competitors are wandering all over the place in fits and starts will do that.

No dispute here that NVIDIA is the market leader today, deservedly so. NVIDIA to its credit has invested in CUDA for many years, even when it wasn't clear there was an immediate ROI.

But, I bristle at the narrative fallacy that it was some divine inspiration and/or careful planning (“focus”) that made CUDA the perfect backbone for deep learning.

In 2018, NVIDIA was chasing crypto mining, and felt the need to underplay (i.e., lie) to investors about how large that segment was (https://wccftech.com/nvidia-sued-cryptocurrency-mining-reven...). As late as 2022, NVIDIA was diverting wafer supply from consumer, professional, and datacenter GPUs to produce crippled "LHR" mining cards.

Jensen has at various points pumped (during GTC and other high profile events):

- Ray tracing (2018) (https://www.youtube.com/watch?v=95nphvtVf34)

- More ray tracing (2019) (https://youtu.be/Z2XlNfCtxwI)

- "Omniverse" (2020) https://youtu.be/o_XeGyg2NIo?list=PLZHnYvH1qtOYOfzAj7JZFwqta...)

- Blockchain, NFTs, and the metaverse (2021) (https://cointelegraph.com/news/nvidia-ceo-we-re-on-the-cusp-...) (https://blockonomi.com/nvidiz-ceo-talks-crypto-nfts-metavers...)

- ETH (2021) (https://markets.businessinsider.com/currencies/news/nvidia-c...)

- "Omniverse"/digital twins (2022) (https://www.youtube.com/watch?v=PWcNlRI00jo)

- Autonomous vehicles (2022) (https://www.youtube.com/watch?v=PWcNlRI00jo)

Most of these predictions about use cases have not panned out at all. The last GTC keynote prior to the "ChatGPT moment" took place just 2 months before the general availability of ChatGPT. And, if you click through to the video, you'll see that LLMs got under 7 minutes of time at the very end of a 90 minute keynote. Clearly, Jensen + NVIDIA leadership had no idea that LLMs would get the kind of mainstream adoption/hype that they have.

On the business side, it hasn't exactly always been a smooth ride for NVIDIA either. In Q2 2022 (again right before the "ChatGPT moment"), the company missed earnings estimates by 18%(!) due to inventory writedowns (https://www.pcworld.com/article/828754/nvidia-preannounces-l...).

The end markets that Jensen forecasts/predicts on quarterly earnings calls (I’ve listened to nearly every one for the last decade) are comically disconnected from what ends up happening.

It's a running joke among buy-side firms that there'll always be an opportunity to buy the NVDA dip, given the volatility of the company's performance + stock.

NVIDIA's "to the moon" run as a company is due in large part to factors outside of its design or control. Of course, how large is up for debate.

If/when it turns out that most generative products can't turn a profit, and NVIDIA revenues decline as a result, it wouldn't be fair to place the blame for the collapse of those end markets at NVIDIA’s feet. Similarly, the fact that LLMs and generative AI turned out to be hit use cases has little to do with NVIDIA's decisions.

AMD is a company that was on death’s door until just a few years ago (2017). It made one of the most incredible corporate comebacks in the history of capitalism on the back of its CPUs, and is now dipping its toes into GPUs again.

NVIDIA had a near-monopoly on non-console gaming. It parlayed that into a dominant software stack.

It’s possible to admire both, without papering over the less appealing aspects of each’s history.

> Depends on what you mean by "real DL workloads". Vanilla torch? Yes. Then start looking at flash attention, triton, xformers, and production inference workloads...

As I mentioned above, this is a chicken-and-egg phenomenon with the developer ecosystem. I don't think we really disagree.

CUDA is an "easy enough" GPGPU backbone that due to incumbency and the lack of real competition from AMD and Intel for a decade led to the flourishing of a developer ecosystem.

Tri Dao (sensibly) decided to write his original Flash Attention paper with an NVIDIA focus, for all the reasons you and I have mentioned. Install base size, ease of use of ROCm vs CUDA, availability of hardware on-prem & in the cloud, etc.

Let's not forget that Xformers is a Meta project, and that non-A100 workloads (i.e., GPUs without 8.0 compute capability) were not officially supported by Meta for the first year of Xformers (https://github.com/huggingface/diffusers/issues/2234) (https://github.com/facebookresearch/xformers/issues/517#issu...). This is the developer ecosystem at work.

AMD right now is forced to put in the lion's share of the work to get a sliver of software parity. It took years to get mainline PyTorch and Tensorflow support for ROCm. The lack of a ROCm developer community (hello chicken and egg), means that AMD ends up being responsbile for first-party implementations of most of the hot new ideas coming from research.

Flash Attention for ROCm does exist (https://github.com/ROCm/flash-attention) (https://llm-tracker.info/howto/AMD-GPUs#flash-attention-2), albeit only on a subset of cards.

Triton added (initial) support for ROCm relatively recently (https://github.com/triton-lang/triton/pull/1983).

Production-scale LLM inference is now entirely possible with ROCm, via first-party support for vLLM (https://rocm.blogs.amd.com/artificial-intelligence/vllm/READ...) (https://community.amd.com/t5/instinct-accelerators/competiti...).

> Compute capability is why code targeting a given lineage of hardware just works. You can target 8.0 (for example) and as long as your hardware is 8.0 it will run on anything with Nvidia stamped on it from laptop to Jetson to datacenter and the higher-level software doesn't know the difference (less VRAM, which is what it is).

This in theory is the case. But, even as an owner of multiple generations of NVIDIA hardware, I find myself occasionally tripped up.

Case in point:

RAPIDS (https://rapids.ai/) is one of the great non-deep learning success stories to come out of CUDA, a child of the “accelerated computing” push that predates the company’s LLM efforts. The GIS and spatial libraries are incredible.

Yet, I was puzzled when earlier this year I updated cuSpatial to the newest available version (24.02) (https://github.com/rapidsai/cuspatial/releases/tag/v24.02.00) via my package manager (Mamba/Conda), and started seeing pretty vanilla functions start breaking on my Pascal card. Logs indicated I needed a Volta card (7.0 CC or newer). They must've reimplemented certain functions altogether.

There’s nothing in the release notes that indicates this bump in minimum CC. The consumer-facing page for RAPIDS (https://rapids.ai/) has a mention under requirements.

So I’m led to wonder, did the RAPIDS devs themselves not realize that certain dependencies experienced a bump in CC?

- Frontier (deployment begun in 2021) (https://en.wikipedia.org/wiki/Frontier_(supercomputer)) - used at Oak Ridge for modeling nuclear reactors, materials science, biology, etc.

- El Capitan (2023) (https://en.wikipedia.org/wiki/El_Capitan_(supercomputer)) - Livermore national lab

AMD GPUs are pretty well represented on the TOP500 list (https://top500.org/lists/top500/list/2024/06/), which tends to feature computers used by major national-level labs for scientific research. AMD CPUs are even moreso represented.

* HPC tends to focus exclusively on FP64 computation, since rounding errors in that kind of use-case are a much bigger deal than in DL (see for example https://hal.science/hal-02486753/document). NVIDIA innovations like TensorFloat, mixed precision, custom silicon (e.g., the "transformer engine") are of limited interest to HPC customers. It's no surprise that AMD didn't pursue similar R&D, given who they were selling GPUs to.

* People tend to forget that less than a decade ago, AMD as a company had a few quarters of cash left before the company would've been bankrupt. When Lisa Su took over as CEO in 2014, AMD market share for all CPUs was 23.4% (even lower in the more lucrative datacenter market). This would bottom out at 17.8% in 2016 (https://www.trefis.com/data/companies/AMD,.INTC/no-login-req...).

AMD's "Zen moment" didn't arrive until March 2017. And it wasn't until Zen 2 (July 2019), that major datacenter customers began to adopt AMD CPUs again.

* In interviews with key AMD figures like Mark Papermaster and Forrest Norrod, they've mentioned how in the years leading up to the Zen release, all other R&D was slashed to the bone. You can see (https://www.statista.com/statistics/267873/amds-expenditure-...) that AMD R&D spending didn't surpass its previous peak (on a nominal dollar, not even inflation-adjusted, basis) until 2020.

There was barely enough money to fund the CPUs that would stop the company from going bankrupt, much less fund GPU hardware and software development.

* By the time AMD could afford to spend on GPU development, CUDA was the entrenched leader. CUDA was first released in 2003(!), ROCm not until 2016. AMD is playing from behind, and had to make various concessions. The ROCm API is designed around CUDA API verbs/nouns. AMD funded ZLUDA, intended to be a "translation layer" so that CUDA programs can run as a drop-in on ROCm.

* There's a chicken-and-egg problem here.

1) There's only one major cloud (Azure) that has ready access to AMD's datacenter-grade GPUs (the Instinct series).

2) I suspect a substantial portion of their datacenter revenue still comes from traditional HPC customers, who have no need for the ROCm stack.

3) The lack of a ROCm developer ecosystem means that development and bug fixes come much slower than they would for CUDA. For example, the mainline TensorFlow release was broken on ROCm for a while (you had to install the nightly release).

4) But, things are improving (slowly). ROCm 6 works substantially better than ROCm 5 did for me. PyTorch and TensorFlow benchmark suites will run.

Trust me, I share the frustration around the semi-broken state that ROCm is in for deep learning applications. As an owner of various NVIDIA GPUs (from consumer laptop/desktop cards to datacenter accelerators), in 90% of cases things just work on CUDA.

On ROCm, as of today it definitely doesn't "just work". I put together a guide for Framework laptop owners to get ROCm working on the AMD GPU that ships as an optional add-in (https://community.frame.work/t/installing-rocm-hiplib-on-ubu...). This took a lot of head banging, and the parsing of obscure blogs and Github issues.

TL;DR, if you consider where AMD GPUs were just a few years ago, things are much better now. But, it still takes too much effort for the average developer to get started on ROCm today.

Few (if any) US representatives or senators at the federal level are actually writing out legislation themselves. They hire staffers (see the "Personal staff" section https://en.wikipedia.org/wiki/Congressional_staff), sometimes lift language straight from lobbyist proposals (aka "model legislation" https://en.wikipedia.org/wiki/Model_act, which is widely used at the state level), or defer to committee staffers (who are subject matter experts) to do the heavy lifting. For example, Lina Khan served from 2019-2020 as counsel to the House's "Subcommittee on Antitrust, Commercial, and Administrative Law", and you can see her fingerprints all over the written work that the committee produced. The framing, and sometimes direct language, of committee report sections are clearly lifted from her legal academia work.

This is comparable to the fact that US federal judge at all levels (including SCOTUS) lean on their clerks to write the first drafts of their opinions, and serve primarily as editors of the final text.

---

In regards to the empirical claim about the backgrounds of lawmakers, see page 8 of this report from the Congressional Research Service (https://crsreports.congress.gov/product/pdf/R/R46705). They say that 144 House members (32.7% of the total), and 50 senators (50%) hold law degrees. While I think you may have been using a bit of hyperbole, it is worth pointing out that there are not enough lawyers in Congress for "ALL of the democrats" to be lawyers.

In terms of occupation (page 3), 85 reps and 28 senators were previously educators; 14 reps and 4 senators were physicians; etc.

Yes there are plenty of law degree holders, but it's also worth considering what law-related job they held. Per the CRS report, 29 reps and 9 senators were previously prosecutors, and 1 rep and 6 senators were previously attorney generals. It's unclear to me why a career in the criminal side of our legal system would have much bearing on how someone drafts laws affecting taxation, provision of government services, etc.

There's also the fact that many law degree holders practiced law for not long at all before winning elected office, or had more substantial "chapters" of their life not related to their degree. Take Jason Crow (https://en.wikipedia.org/wiki/Jason_Crow). He spent as much time as an Army Ranger as he did as a lawyer. One could easily construct a narrative that Mr. Crow, who has complained often about the bureaucracy of accessing veterans' healthcare, should be allergic to red tape and bureaucracy. But with the crude taxonomy of "he has a law degree", the other parts of his life would be overlooked.

---

IMO, a big part of the bloated and inhumane parts of the bureaucracy have to do with the outgrowth of "administrative law" and "rulemaking" (https://en.wikipedia.org/wiki/Administrative_law; https://en.wikipedia.org/wiki/United_States_administrative_l...; https://www.everycrsreport.com/reports/RL32240.html). Once a bill has been signed into law, the rulemaking process begins. These are where the actual details of a new law are hashed out. A bill may designate $X in funding for a program. Which contractors receive those contracts, hours of service, the amount of paperwork required, etc. are all handled at the administrative level. And it's certainly the case that for 99.9% of citizens, no one is submitting public comments during this period, and the input of ordinary people is often lacking.

So when the IRS makes a free-file options for taxes difficult to use, in large part due to Intuit's lobbying (https://www.propublica.org/article/inside-turbotax-20-year-f...), this is not the result of a carve-out or giveaway spelled out in actual legislation's text. It's the result of an actor exploiting the opacity of the rulemaking and administrative law practices.

Chess.com uses Glicko with an initial rating of 1200 (https://support.chess.com/article/210-how-do-ratings-work-on...), while Lichess uses Glicko-2, and sets their initial rating to 1500 (https://lichess.org/page/rating-systems).

Just anecdotally, I’m an ~1800 rated player on chess.com, and ~2050 on Lichess. Percentile rank wise, I’m in the top 3.5% on chess.com, and closer to top 10% on Lichess.

I think this matches the general perception that Chess.com has many more casual players. This is likely a function of all the cross promotion they’ve done to grow the game, especially on Twitch. Neither site is “good” or “bad”, but my friends who play casually seem more likely to play on Chess.com.

Some relevant quotes:

> For example, each Formula 1 team is only allowed to use 25 teraflops (trillions of floating point operations per second) of double precision (64-bit) computing power for simulating car aerodynamics.

> Oddly, the F1 regulations also stipulate that only CPUs can be used, not GPUs, and that teams must explicitly prove whether they're using AVX instructions or not. Without AVX, the FIA rates a single Sandy Bridge or Ivy Bridge CPU core at 4 flops; with AVX, each core is rated at 8 flops. Every team has to submit the exact specifications of their compute cluster to the FIA at the start of the season, and then a logfile after every eight weeks of ongoing testing.

> Everest says that every team has its own on-premises hardware setup and that no one has yet moved to the cloud. There's no technical reason why the cloud can't be used for car aerodynamics simulations—and F1 teams are investigating such a possibility—but the aforementioned stringent CPU stipulations currently make it impossible. The result is that most F1 teams use a somewhat hybridised setup, with a local Linux cluster outputting aerodynamics data that informs the manufacturing of physical components, the details of which are kept in the cloud.

> Wind tunnel usage is similarly restricted: F1 teams are only allowed 25 hours of "wind on" time per week to test new chassis designs. 10 years ago, in 2007, it was very different, says Everest: "There was no restriction on teraflops, no restriction on wind tunnel hours," continues Everest. "We had three shifts running the wind tunnel 24/7. It got to the point where a lot of teams were talking about building a second wind tunnel; Williams built a second tunnel.

With the new cost cap in F1 (https://www.autoweek.com/racing/formula-1/a35293542/f1-budge...) (which notably excludes driver salaries), it would be interesting to know how much these on-prem clusters cost to operate.

Trevor Milton’s experience prior to starting Nikola was selling home security systems. Lucid’s leadership team features various Tesla, Audi, Ford, VW, etc. veterans. (See page 9: https://www.lucidmotors.com/files/lucid-investor-deck-februa...)

Lucid has finished building a factory that can produce roughly 30k cars per year, expandable to 400k.

They’ve given rides in their launch vehicle to various auto journalists (https://youtu.be/gqSN2QNgO5k).

Their battery pack technology is a component of the Formula E drivetrain system (https://lucidmotors.com/media-room/atieva-powers-season-6-fo...).

This isn’t a “Nikola rolling a non functional truck down a hill” situation. They have a working product.

The car could suck, the company could be overvalued. But I think hard to compare Nikola to Lucid.

EDIT:

I should also add for comparison, that at the time that Tesla IPOed in 2010, it was a 1.7B market cap company. Only ~2450 Roadsters (their only car at the time) would be sold in total. By November 29, 2010 Tesla had not yet sold 1400 cars (https://www.tesla.com/blog/race-champions-2010-motorsport-go...).

Tesla's Fremont factory was opened in October 2010. In other words, when the company went public on June 29th, 2010 you would have been buying into a car company without a factory.

The first Model S wasn't delivered until June 2012 (https://www.tesla.com/blog/tesla-motors-begin-customer-deliv...).

Not to say that Lucid will or won't ever reach Tesla's heights, but assigning a 12B valuation to the company isn't loony. The SPAC price though is a different story.

EDIT 2:

There are some fun short videos of Lucid CEO Peter Rawlinson in the workshop from his Tesla days (https://www.youtube.com/watch?v=TrbOLHW8Pec, https://www.youtube.com/watch?v=8YxHp2ot61Y, https://www.youtube.com/watch?v=NGKqPYvtqXE). It's pretty awe inspiring to see where Tesla and the global EV industry as a whole was in 2011 vs today.

A post about Putin and the CCP exploiting US domestic political turmoil: https://news.ycombinator.com/item?id=25830075

A book recommendation by a longtime China-skeptical writer: https://news.ycombinator.com/item?id=25843422 (https://en.wikipedia.org/wiki/Bill_Gertz)

My personal preference is for a HN community where dissenting views don’t lead to assigning ulterior motives (in this case being paid to shill for the CCP) to our debating “opponents”.

E.g. in NYC (https://www1.nyc.gov/html/dot/html/motorist/sizewt.shtml), the max weight for any vehicle is 80,000 lbs, likely much less if you’re the length of a typical garbage truck.

Can you really go electric? A 2016 bullish Quartz article (https://qz.com/749622/the-economics-of-electric-garbage-truc...) says the typical garbage truck travels 130 miles a day. Unclear what the additional weight of a capable battery pack would be, even after accounting for the saved weight by removing what must be a pretty hefty combustion engine. It’s certainly interesting that the company (Wrightspeed) profiled in the article seems to be doing more than just garbage trucks now. Their Route 1000 powertrain/platform only quotes 24 miles of pure EV range (https://www.wrightspeed.com/the-route-powertrain), so it seems like there are certainly trade offs between range and weight regulations.

I imagine once long and medium haul trucks make it to market (perhaps Tesla or Volvo), maybe we can say we’ve reached the energy densities in packs that make hauling around weight equivalent to an 18 wheeler possible at a reasonable cost.

Other reasons that maybe no one has done this already:

* Garbage trucks cost $250k+ (https://bigtruckrental.com/front-loader-garbage-truck-rental...). I’m guessing there are regulatory requirements around collision safety, and maybe longevity requirements that become a factor.

* Given that a brand new sleeper semi goes for 50% less (https://youngtrucks.com/new-trucks/2020-volvo-vnl64t860-860-...), there are probably significant costs not typical of a combustion engine vehicle. Maybe those front loading bins require powerful pneumatics, or those on-vehicle compactors need to be able to exert tremendous amounts of force, and so on and so on.

* How large is the market for garbage trucks? Some press release claimed that the global garbage truck market in 2019 was ~$22 billion (https://www.globenewswire.com/news-release/2020/07/29/206925...). If we use a unit price of $250k (probably on the low end), that would mean only ~88000 total units sold globally. Of course, I’m sure different countries may have larger and smaller trucks, but you get the gist.

* The customers for these trucks are likely to be municipal governments and a handful of private waste management companies (https://craft.co/waste-management/competitors). I think it’d be a huge risk to build out a plant to put together a garbage truck, have maybe a few hundred plausible decisions makers to pitch on the product, and potentially risk burning hundreds of millions in capital between labor, regulatory certification, endurance testing, battery pack development, power train development, etc.