eurekin
Joined 14 years ago
Joined 14 years ago
I was promised a PC at around 11 years. I think for months (or even more than a year) I fantasized about, what I could do with it. Read some books, that I had access to. Understood practically nothing, but that didn't stop me from dreaming about the bright future. The book might have been "Amiga E programming reference" and it had two chapters dedicated to the blitter and sprite programming. I reread those chapters many times, trying to imagine how to make a simple game with it.
The promised PC came, but it was a work decommissioned 486 at the time, when everyone else was upgrading to Pentiums 200 to play the latest Lara Croft. My sister was bummed out that no software from friends worked on it.
I somehow was able to make a very simple arkanoid game anyway, in C and OpenGL software renderer. Pretty sure, that the main reason I managed to do anything, was all the time spent expecting the PC and imagining possibilities.
Where's that figure comning from? Last time I checked (could be the 3.6 Qwen 27b) single token needed 32kb
- It's just an Independent Security Researcher.
- So that's it? You will do no action?
- Correct
I have no idea, why in that context, the number of attention heads isn't mentioned. Models have a limited set of them and obviously, a model can focus at N max things at a time, which has to put an upper bound of long context support in some way. There's just more things to lose focus to (or, mismanage the limited attention heads resources - per token)
Self hosting, SBCs, AI/Vision/LLMs