dkdcdev

why?

see also “ Fair Benchmarking Considered Difficult: Common Pitfalls In Database Performance Testing” by the DuckDB folks with a classic Figure 1

you can also put that REST API behind a self-documenting CLI. that’s how I setup my agents to talk to each other

The idea that an LLM can discern intent on any given prompt is farcical. I might be researching nukes to commit an atrocity, or to prevent one. I might be asking about laundering money to commit a crime, or to prevent one. I might be researching the Nazis because I want to commit a genocide, or I want to read up so I know how to prevent one. Same with cybersecurity. Same with anything.

In my opinion, these companies should put their effort elsewhere. Obviously if all someone is doing on their platform is looking up how to build a nuke, where to buy uranium, the best city to explode it in, etc. please report them to the authorities. If someone is clearly just using LLMs to write hate speech they go post on the internet, ban them. And so on.

This cat & mouse game trying to have LLMs police inquiries is ridiculous to me.

they also already do this…

Anthropic and OpenAI license to the public clouds. Google reportedly licenses to Apple. licensing to Fortune 100 companies running on their own infra is an obvious next step

it is a race to the bottom and I’m not sure the labs win that race. we’ll see!

at their scale they could also just run a large on-premise or rented (basically still cloud, but cheaper) GPU cluster and run through that. fixed costs, even license a SOTA model’s weights if you’d like

I believe the canonical example is Databricks on round L