whs

Now I wonder when will people find how Carmack found 0x5F3759DF

It's the same thing. Seems like the major version bump is to advertise WSLc, which is like Apple containers - an SDK for launching Linux containers as part of a Windows app.

I wonder if API is affected by this issue, especially Claude on public clouds? Would that means the subsidized rate just means they use cheaper quantized models and it's not comparable to API spending.

Tell that to AWS Lambda and Google Compute Engine?

Seems clear from their website?

1. Their API server provide an attestation JWT. This JWT is signed by Google's private key. 2. The attestation has details on the running container. I suppose the container host is a Google-provided distro and Google's signer will verify that the OS is theirs and up-to-date. 3. They could've proxy the attestation. To prove this is not the case, the field eat_nonce include the TLS certificate fingerprint, which should match the API server you're connecting to. I suppose you will need to pull their container and verify from the source that the container itself generate the private key, it never leaves the container, and the container has no way to run arbitrary code such as SSH or vulnerabilities.

I had one.

The only two reasons it was worth it is my company paid for it, and you can get a lounge space and some swag at reinvent. I never get to use it professionally

I've been looking for similar distributed computing style LLM, and I found AI Horde and a few other smaller efforts like one from Aphrodite people and distributed training from Nous Research.

AI Horde seems to be the biggest of them all. Their API speaks KoboldCPP text completion (not even chat completion). It seems that the community (or at least the active people) strongly prefer it this way because the API exposes more tunables than chat completions, which for roleplay use seems to result in better result. I don't know what else you can use AI Horde for anyway since all other use cases likely will require tool use. Just this week I was set out to improve their OpenAI bridge to support chat templates and response parsing. We'll see if I could get it deployed officially then you might be able to use it to code, although you'll have to use RP models.

I think Horde do have a lot more abuse prevention. Workers needs to have 1 week of cumulative uptime to be considered trusted to prevent brigading - users can opt into trusted workers only. Running a worker give you kudos which is required for >512 max tokens generations and also free requests get bumped to last.

I wonder how security is done in this engine, since it's accepting input from anyone. llama.cpp's RPC layer seems to says that you shouldn't run it in public (I assume because it is lower level and may result in RCE on your GPU)

Sounds like the same problem as condom. Nobody want to buy the "mini" size.

Most people don't directly call Linux syscalls though but go through glibc. It might even be unavoidable if you want to ship desktop apps as the library will use it. If it's that easy there wouldn't be Python's manylinux, flatpak base packages or Steam Linux runtime

My company is heavy on Cursor and I still ask them to provide me GitHub Copilot, for the sole reason that Cursor is probably the reason Microsoft had to implement technical enforcement of their TOS on proprietary plugins. Previously, you could use PyLance on VSCodium but now those plugins do not work outside VSCode anymore.

If Cursor (and every other commercial VSCode forks) didn't use MS extension store in the beginning and violate the TOS these might not have happened.

I think AWS billing is quite complicated that they probably don't even know what did you get charged specifically for this machine.

You might have leftover reserve instance that applies, which make the listed price inaccurate. That reservation might even be in a different AWS account in the same organization that you don't have access to. That reservation might not even be there between the time you quote and the time you actually launch it if someone/something did launch before you.

Your organization might also have discounts. I believe some discount may also be very confidential. For example, my reseller policy is that the customer must not be able to see AWS Billing in the organization root account as supposedly the price in that console are the price AWS charged the reseller, while we pay listed price minus any discount we negotiated ourselves.

Finally, I suppose they don't want to have prices shown in multiple places as they will need to update it when prices changes. Doesn't want to risk forgetting one place and getting sued for it. You can see that AWS documentations often do not want to mention the price at all, even if that price is currently free.

Chinese clouds kinda make this simple by making reservation part of the buying machine itself - you have to mark that particular machine as monthly/yearly committed when you start it (or convert it later). The complicated part is recycling instances - if you delete a server before its reservation ends it ends up in a recycle bin that you need to look before making new reservations.

I wish that is an option. Nowadays many non competitives games that you play with friends you trust still use EAC (yet accept non-kernel mode operation on Linux). I suppose other than VAC you can't buy a usermode anticheat middleware now.

The analogy I like to use when people say "I paid" is that you can't pay for a buffet then get all the food take-home for free.