dgfl
Joined 3 years ago
Karma
257
Submissions
123
Member since
Feb 9
Recent submissions
Comment · 1 month ago
Comment · 1 month ago
IEDM is more or less the most prestigious conference on semiconductor devices. Hard to get a more reliable source than a TSCM-affiliated paper from IEDM.
Comment · 2 months ago
I wouldn't really say SRAM-fast. Maybe DRAM-fast? At least in write speed. But the thing is, you don't need high write speed for this application. And read speed should be the same between flash and FeFETs.
Btw, I've seen some roadmaps in which foundries were heading towards integrating ferroelectrics and floating gates to accelerate switching. The two technologies are not mutually exclusive really, so I think we might see something in the relatively near future. The crazy memory market may help!
Comment · 2 months ago
The vanguard chiplets even steps down to 30M… but they do claim to have a “Mead” design [1] designed to run GPT-3 in a single chip.
3D NAND flash can indeed routinely store hundreds of GB per die, so that’s proven. The question is about all the peripheral circuitry needed. Each attention block would need its own KV cache (i.e. SRAM or DRAM somewhere), plus DAC/ADC inputs and outputs, unless they figure out a way to keep it analog all the way (really cool but unlikely).
I think this field is very interesting, at least from a technology point of view. Whether it works out or not will sadly be a matter of economics more than physics I fear.
Comment · 2 months ago
Temperature is a relatively trivial issue because it can either be compensated or the chip simply kept at a constant temperature (they’re low power chips anyway, throttling could be skipped to simplify the design). The ADCs and DACs are indeed the main thing though. The whole analog compute game is about making those power efficient and scaled enough that the math still works out in your favor. The demos do work though, this company is far from the only one; see [1] or [2] for example.
Flash NAND can routinely be bought with 4 bits per cell, perhaps even 5 soon (QLC and PLC drives). Since it has been proven to store 4 digital bits at production scale, I’m willing to bet an analog architecture running an LLM should be able to yield the performance analog of a 4-8 bit quantized model. Where in that 4-8 range is pretty crucial, but it depends on the specific design.
[1] https://www.nature.com/articles/s41928-023-01010-1 [2] https://www.nature.com/articles/s41586-022-04992-8
Comment · 2 months ago
Joke’s on us, all of their pages are LLM pages! LLM generated, that is.
Btw, can guarantee that they are not ready to demonstrate that yet. They’re using 2D FLASH with 30M weights per die [1], so to get to 1T they will need… 33,333 dies. Interesting scaling problem to say the least
Comment · 2 months ago
This page seems pretty in depth: https://www.mythic.ai/supply-chainmanufacturing
They are using GlobalFoundries’ 28nm node for the floating gate transistors, afaiu, which they then bond onto a TSCM 5nm digital I/O wafer.
Analog computation’s principles are sound. It’s mostly doing matrix vector multiplications though. The rest is digital.
Comment · 2 months ago
They will keep a live leaderboard at https://huggingface.co/spaces/tabularisai/YapBench
Here is the top 10 right now:
| Rank | Model | YapIndex | YapTax$ |
| ---: | ---------------------------------------- | ---------: | ------: |
| 1 | openai/gpt-5.6-sol (reasoning) | 18.5 ±4.8 | 0.51 |
| 2 | openai/gpt-5.6-sol | 19.2 ±4.4 | 0.51 |
| 3 | openai/gpt-3.5-turbo | 22.7 ±4.8 | 0.02 |
| 4 | openai/gpt-5.6-luna | 27.8 ±9.8 | 0.15 |
| 5 | openai/gpt-5.4 (reasoning) | 40.7 ±10.7 | — |
| 6 | openai/gpt-5.4 | 40.7 ±9.0 | — |
| 7 | moonshotai/kimi-k2-0905 | 44.7 ±4.8 | 0.05 |
| 8 | mistralai/mistral-small-2603 (reasoning) | 46.2 ±31.5 | 0.03 |
| 9 | openai/gpt-4 | 51.2 ±20.6 | 1.39 |
| 10 | openai/gpt-5.3-codex | 64.8 ±9.9 | — |
| Rank | Model | YapIndex | YapTax$ |
| ---: | --------------------------------------- | ----------: | ------: |
| 23 | anthropic/claude-opus-4.5 | 97.0 ±28.9 | 1.52 |
| 25 | anthropic/claude-opus-4.5 (reasoning) | 99.2 ±29.3 | 1.44 |
| 42 | anthropic/claude-3.5-sonnet | 199.7 ±24.5 | 2.53 |
| 57 | anthropic/claude-sonnet-4.5 (reasoning) | 278.7 ±41.7 | 1.65 |
| 61 | anthropic/claude-sonnet-4.5 | 285.0 ±39.4 | 1.63 |
| 71 | anthropic/claude-opus-4.6 | 330.3 ±59.1 | — |
| 72 | anthropic/claude-haiku-4.5 | 333.2 ±26.7 | 0.64 |
| 73 | anthropic/claude-haiku-4.5 (reasoning) | 335.2 ±26.8 | 0.65 |
| 76 | anthropic/claude-opus-4.6 (reasoning) | 342.3 ±30.5 | — |
| 87 | anthropic/claude-3.5-haiku | 401.2 ±25.4 | 0.59 |
| 88 | anthropic/claude-sonnet-4.6 (reasoning) | 422.8 ±23.3 | — |
| 89 | anthropic/claude-sonnet-4.6 | 425.0 ±26.3 | — |
Comment · 2 months ago
That one is more specific, but "vomit" captures the feeling of Opus 5's writing very well for me. I don't know if it's the watermarking, but every single language idiosyncrasy that Opus 4.x (x > 5) had has been pushed up to 11 on Opus 5. Plus we got nouns verbing and seams seaming.
It's really unusable for anything other than code. And I have to remove its incomprehensible comments 50% of the time before committing anyway. After interacting with it, "slop vomit" is truly the most fitting description. I have to admit I have lost my temper and spontaneously referred to its output as vomit more than once. Seems like I'm not the only one.
Comment · 3 months ago
It can also go the other way. It outputs a paragraph so full of telegraphic technobabble that it’s just unreadable, inventing new jargon every other sentence, using terminology invented in its thinking stream and never explained. And when it combines the two issues and it outputs 4 pages of technobabble, it gets just exhausting to read. Paradoxically, given that 2 years ago everyone was using LLMs to summarize web pages, their ability to condense text (including their own thought stream) is still lackluster.
I dream of some post-processing diffusion optimizer just condensing all of their slop away. Until then, I just tell it to _avoid_ telegraphic and newspaper-title speech and to output one or two paragraphs of prose only.
Comment · 3 months ago
This is a great related watch if you have some time to kill: https://youtu.be/KnUFH5GX_fI
Comment · 3 months ago
I realized how awful of an OS modern android is when I bought an iPad. I later fully switched to iOS and never looked back.
Point is, sometimes people just have different experiences from you.
Comment · 3 months ago
This is intentional repetition. More specifically, anaphora [1]. You may not personally like it, but it is a rhetorical device used to emphasize a point. This one also comes with a nice progression: storms, storms, heat, heat, farmland, groceries.
This substack article also comes with additional graphs, a much better story flow (data is progressively introduced and explained before reaching the final plot), and was posted 2 days before the OP. I agree with GP that it is significantly superior to the OP (which is likely AI slop). Thanks for posting it.
Comment · 3 months ago
Not at all. The model could (and sometimes should) burn all the money it wants, and then produce a single line of actual production code. Only some things, e.g. full rewrites, have clear cost - LoC scaling.
For my usage, I would very much prefer if those $/task were being spent in thinking and experimenting, and the actual output would be as short and maintainable as possible. “maintainability” is a vague target of course, but it’s at least somewhat correlated with code size.
I wonder if this is a global study. Cynically, it feels like a way to get a few more billions of tokens of free alignment training material. They explicitly say that they will “use the results to inform how we develop our models and products”.