mikeravkine

The Internet is 90% mobile users that can't hover.

Thank you for your efforts on behalf of the GPU poor!

It's getting tougher to use older, cheaper GPUs (Pascal/Maxwell) with modern quantization schemes so anything you can do to keep kernels compatible with SM52 and SM61 would be greatly appreciated.

Ubuntu 18 went EOL on May 31, 2023 so about 8 months ago. What CentOS are you running that's still in support and was affected by this change? I had to upgrade all my systems last year after the EOL as most python packages dropped 3.8 support.

The challenge to Mastodon isn't adoption, its absense is a symptom of a deeper issue: it's a Very Bad Idea to trust random server operators with both your data and it's sovereignty.

You can take your data to another server only if the original server is up. When the admin forgets to pay the bill and ISP nukes the box and the backups admin said they had don't exist and the coadmin turns out to have no access at all, you lose everything. Ask me how I know.

Self hosting in theory should fix this but it doesn't: it's a resource intensive afterthought because the developers only really care about their Big Instance.

I just wanted to say thank you, aider (with the new unified diff format) is the first AI tool that has actually changed the way I work.

The razor doesn't apply to the police.

Hetzner can and will terminate your account and delete all your data on a whim, never telling you why. There is nothing at all you can do to prove your usage was legitimate. I get that they must deal with a lot of abuse but I wouldn't use them for anything mission critical ever.

In terms of economics specifically, in what way have things "gotten better"? Purchasing power has collapsed, the poor sentiment here seems the correct sentiment to me.

There is no difference in download time for 20mb vs 50mb in a datacenter, but the amount of time lost to alpine being "quirky" is incalculable.

If anyone is looking for a more reasonably cost effective solution, Hetzner has 16 vCPU/32GB RAM ARM VMs for $24E/mo that will run 34b Q4 GGUF at around 4 tok/sec. It's not very fast, but it is very cheap.

Where do I get one of these tech writing destroying AIs that will both understand how to use my product and produce clear, concise guides for doing so without burdening the existing team?

This is not about intelligence, it's about agency. Text completion generators don't have agency no matter how big they get.

Hetzner offers incredibly cheap ARM machines in the Falkenstein DC, for 25Eur a month you can snag the top of the line with 16 vCPU and 32GB RAM.

If your usecase fits inside that 32GB (no 70B models, sadly) the price to performance of a GGUF Q4KM is really attractive on this setup.

Pascals := is concise and intuitively obvious, so obviously it never caught on.

1997 era internet would have scared the shit out of you.