ThunderSizzle
Joined 5 years ago
Joined 5 years ago
My stack uses a local llamacpp ROCM build, modified by https://github.com/stew675/llama-cpp-rdna-boosts
The build uses my machine's ROCM and I just link it instead of embedded all that in an image, like the published ones do.
Then deploying it via docker compose to my local machine. The models I want to run are stored in a docker volume, so it's easy to switch the model while keeping the same build.
The biggest negative for me is that I don't think it's enough to do parallelization through llamacpp (I'd prefer to work on one thing faster than two things slowly). I haven't gotten vllm working yet. Also, once you have one, you constantly start thinking about a second card...
However, that also means AI will be even more lobotomized since the cloud providers will panic over being liable
Sadly, the r9700 has gone up in price, but it's still the cheapest route. I use It on an old x99 Linux desktop and it achieves 50 tps decode and 1000+ tps prefill.
Why trust Google of they aren't acting in your interest?
Now, moving a steam account into a trust might be a breach, if they care. Even using a trust to buy a game might also be a violation of some form, perhaps, buried in the toc.
But the trust itself is a legal entity and it can own things. Just because the people behind the trust changes doesn't mean it was a transfer on the public side of the trust.
No one gets pulled over for turning light violations for safety reasons, but that's beside the point. I just hate how many laws we have that are just selectively enforced.
I force all work to be split up into chunks, properly investigated, spiked, and planned out, and then implemented via TDD (with only system-boundary mocking permitted). It's not perfect, but I've seen regression bugs very much cut so far.
Beyond that, if it's a GUI app, I also require the TDD cycle to require GUI tests with screenshot evidence, and proper assert statements.
In other words, my email can be signed in from anywhere on purpose. It's a feature. The chances of me forgetting my password is zero, because I can back it up 7 way to Sunday.
A passkey seems to miss these basic accessibility issues. For a random website, that's fine, but for the service all of websites use to auth me, access is more important than locking it down to one device.
One domain, one account.
Seems like simplify or just erradicting the tax code would be more just at this point.