1R053
Joined 6 years ago
Joined 6 years ago
https://de.wikipedia.org/wiki/Wegzugsbesteuerung
Essentially, it assumes you sell your assets at market value and taxes the difference to your expenses for it.
Not yet sure how to really do it, but one concept I like from NI plugins is that you have multiple keyboard zones: one zone is for notes, others are e.g. for patterns or styles. Imagine a guitar where one zone is for the chord type and tone, another for the striking pattern...
The challenge here is probably the resonance algo for multiple systems based on multiple notes... Maybe the piano concept would be handy here... imagine instead of having 3 strings like on the piano the instrument to be one system for each key... that excite each other via air or direct resonance points... the systems should be automatically tuned based on one reference system (e.g. using automatic string length or tension scaling)
Anyway, amazing work and having it on GPU allows this really to scale.
Also, if there is no answer yet on the web the AI may also not know it. Then these questions should still end up on SO.
I might add, that SO also could build their own chat / research UI. It would need to have some benefit over others, but I guess the community aspect of it alone would suffice...
as in https://en.wikipedia.org/wiki/The_Innovator%27s_Dilemma
Obviously, you need to destroy the right things...
Once you have something with 192 GB it gets interesting. You could probably have 7 at FP8 per GPU. At FP16 it probably only would fit 3 per card, requiring 9 again.
I'd say for the current memory layout of cards they missed a little bit the sweet spot. With slightly smaller models or one expert less one should be able to run it on 8 H100s at FP8 or 2 B100s at FP8 or even on 4 B100s at FP16 if I calculated correctly.
They use
- 16 experts, of which one is activated per token
- 1 shared expert that is always active
in summary that makes around 52B active parameters per token instead of the 405B of LLama3.1.