lelag
Joined 12 years ago
Joined 12 years ago
It's seems you need lot's of ram and vram. Reading the issues on github[1], it does not seem many others have had success in using this effectively:
- someone with a 96 Gb VRAM RTX 6000 Pro had cuda oom issues
- someone somehow made it work on a RTX 4090 somehow, but RTF processing time was 12...
- someone with a RTX 5090 managed to use it, but with clips no longer than 20s
It seems utility of the model for hobbyist with consumer grade cards will be low.
[1]: https://github.com/facebookresearch/sam-audio/issues/24
I'll let myself out now.
> People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours, about what an oven would use in a little over one second, or a high-efficiency lightbulb would use in a couple of minutes. It also uses about 0.000085 gallons of water; roughly one fifteenth of a teaspoon.
https://www.softwareheritage.org/wp-content/uploads/2019/07/...
The web frontend is written in TypeScript/React.
While they offer digital downloads on the eShop, their pricing actively discourages it.
Case in point: I just bought my kid a new first-party Switch game. Physical copy on Amazon was ~25% cheaper than the identical digital version on Nintendo's own eShop. Even my 9-year-old noted how illogical it seems, the physical version requires manufacturing, shipping, retail markup, yet costs significantly less than the digital bits that have near-zero marginal cost.
It strongly suggests Nintendo wants the physical retail channel to thrive, or values the perceived permanence/resale value of cartridges.
This context makes the Switch 2 "gamekey" cartridges (physical auth token, digital download) fit their pattern of valuing a physical artifact and retail presence, even if the data delivery shifts.
One beam could be "This is a way to solv-". With no obvious "good" next token. Another beam could be "This way is solv-". With "ing" as the obvious next token.
It will select the best beam for the output.
That said, I think the most impressive achievement is the English translation of the French novel. Writing an original constrained novel is hard enough, but translating one means you can’t just steer the story wherever you like. You have to preserve the plot, tone, and themes of the original, all while respecting a completely different set of linguistic limitations. That’s a remarkable balancing act.
What I will add is that constrained generation is supported by the major inference engine like llama.cpp, vllm and the likes, so what you are describing is actually trivial on locally hosted models, you just have to provide a regex that prevent them to use the letter 'e' in the output.
DeepSeek R1 hosting is out of reach for most, but it being open is a game changer if you are a building a business that needs the SoTA capabilities of such a large model, not because you will necessarily host it yourself, but because you can't be locked out of using it.
If you build your business on top of OpenAI, and they decide they don't like you, they can shut you down. If you use an open model like R1, you always have the option to self host even if it can be costly, and not be at the mercy of a third party being able to just kill your business by shutting down your access to their service.
At first, write clean code with functions and don’t obsess over call overhead. Once it works, profile, then optimize where it actually matters.
Premature optimization, etc.