manwithopinions

Profitable on a per token basis is meaningless. Ed Zitron doesn’t argue that it is impossible to offer profitable inference, he argues that the business as it stands today is deeply unprofitable and only getting worse because it isn’t a high-margin inference business.

Play out the most likely pessimist’s scenario: LLMs are useful but frontier models are overkill so businesses just use dirt cheap open weight models on their own hardware and/or they rent hardware instead of paying per token. Then what for OpenAI and Anthropic?

OpenAI’s business collapses if customers are happy with an LLM that costs $0.10 per million tokens even if it only costs OpenAI $0.05 in inference per million tokens. The insane bonkers claim from Garry Tan that in 2 years we will be using 90,000x as many tokens as today is… well, obviously not true.

The fixed costs that OpenAI and Anthropic have created need inference demand far beyond what is plausible.

edit: and hand waving away the vast losses of companies like OpenAI because of “training” is ridiculous. Anthropic are spending a billion dollars per month to rent additional capacity from xAI for inference, not training. The models don’t need to get better: if there is a case for LLMs to change business forever, GPT 5.4 is just as capable of achieving it as GPT 5.5.

Peter Skaliy, Head of Legal at Corgi, same person who sent the cease and desist to people on Twitter, is filing a suit against me for defamation because of that Substack post.

"Corgi builds insurance structures that allow us to best serve the needs of our customers. For technology companies, operating a technology liability line through a Risk Retention Group is not unusual, improper, or exotic; it is a standard insurance structure for specialty liability risks where similarly situated businesses benefit from tailored underwriting, specialized coverage, and risk alignment. The suggestion that Corgi customers are unknowingly taking on “balance sheet risk,” member-assessment risk, or responsibility for unrelated insureds’ liabilities is false.

Your draft’s statement that “Corgi will help you share that risk,” combined with the question whether customers understand the risk of other companies in the group, does not merely describe RRGs in the abstract. It falsely implies that Corgi leaves customers exposed to open ended financial liability for other insureds. That implication is defamatory and false. RRGs are regulated insurance carriers subject to financial, reserve, governance, and regulatory requirements. They are not informal risk sharing clubs where policyholders unknowingly become responsible for each other’s balance sheets.

The RRG structure unique to Corgi. Major insurance groups use different insurer structures for different classes of risk because different risks are best served by different structures. Berkshire Hathaway, which the draft itself invokes, has affiliated insurance operations involving Risk Retention Groups in specialty liability markets, including medical and legal professional liability. That underscores the point: RRGs are a widely recognized insurance structure for specialty liability lines, and allow insurers to provide more tailored coverage options rather than issuing a generic policy.

The draft’s statement that Corgi “innovated with AI in a regulated industry by cutting corners” is also false and defamatory. Corgi raised millions pre-revenue and spent nearly two years building and obtaining regulatory approvals for its insurance operations, including approvals and requirements relating to reserves, pricing, liquidity, governance, and compliance. That is the opposite of “cutting corners.” Any allegation that Corgi used AI to evade regulatory approval, underwriting standards, reserve requirements, pricing controls, liquidity controls, or other compliance obligations is false.

Any article suggesting otherwise, including by implying that Corgi misleads customers, conceals the RRG structure, exposes policyholders to undisclosed balance-sheet risk, or uses RRGs and AI to evade proper underwriting or regulatory obligations, is false and highly damaging.

To be clear, if you publish these false statements or defamatory implications, Corgi will sue you personally and will pursue all available claims and remedies against you and any other responsible parties. Corgi has enforced its rights before and will do so again. You should not mistake this for an abstract legal reservation."

Skimmed it and based on my experience in fintech, it looks good, accurately represents the real world. I guess there’s still a chance it is AI generated but it doesn’t seem like vacuous slop, it has substance!

My understanding is a normal insurance tech startup would be acting as a broker.

A rare but sensible insurance tech startup would use external underwriters and reinsurance and provide insolvency protection.

Corgi doesn’t have any external underwriters, doesn’t have any insolvency protection, doesn’t have any reinsurance.

I think they’re bad on all 3 points, not just the underwriting?

If you’re a business that deals in documents from external customers / partners, you use a data room like DocSend (by Dropbox) to share and receive documents with access management, analytics, auditing etc.

Papermark is an open source alternative to DocSend. Papermark is very popular, as it is a much more cost effective alternative to DocSend — self-host or hosted.

Corgi is a YC backed insurance startup that sells insurance to other YC startups. Nico is a founder. Recently they raised $100m at a ~$3bn valuation. They’re one of the darlings of YC right now, endless fawning over them.

Since insurance underwriting involves lots of documents, Corgi were paying Dropbox thousands of dollars per month for DocSend. For some reason, Corgi ostensibly formed a team of 12 to build their own DocSend alternative, called Dataroom. And Corgi decided to make it into a SaaS product, pitched as a cheaper DocSend from just $10/month, in an already crowded space.

Papermark noticed immediately that Corgi’s Dataroom used a lot of identical language and structure that Papermark’s open source product does. Papermark assumed that Corgi had taken Papermark’s work without attribution. Corgi have denied it, claiming it is just a coincidence that there are word for word matches between the products.

Another YC startup, Delve, got caught doing what Corgi are accused of (and much more) which led to their removal from YC.

They’re not really, it’s just the YC hype cycle. The business is selling insurance to other YC startups with some AI flair. They’re not even the first YC startup to do this, a previous YC insurance startup was acquired a few years ago for ~$1bn. So, they’re worth 3x the exit of the exact same company… because of what, AI? The fact that they’re cloning other software to release SaaS products is extremely bearish. Why are they wasting their time on this? A wildly successful $3bn startup would not spend their precious resources by launching a $10/m document sending SaaS. They’ll be doing down rounds soon enough. Could you imagine Paul Graham encouraging this?

> Their margins on inference are >90% today for tokens they sell (plans are hard to count, but still profitable).

That doesn’t make any sense, it doesn’t add up. Have you seen how much money they’re raising and burning? We know that training does not cost tens of billions.

Brockman said OpenAI expects to spend $50 billion on compute this year. OpenAI’s revenue run rate is less than $50 billion for this year! For 90% margins to be possible on inference, you are suggesting that less than $5 billion of that compute spend is inference and over $45 billion of that compute is training.

Anthropic have been desperately trying to juggle capacity by shaping user behavior through peak time usage limits because they are struggling with capacity for inference.

Plan based usage is widely acknowledged to be subsidized, you are probably the only person on earth suggesting that plans are profitable.

So why are they losing so much money?

Money is made on the subset of inference that is charged at cost + margin via their APIs. API usage is so high because customers are still finding their feet, trying to understand how to measure the value they get from their spend, erring on the side of spend.

Yes, in a world of unmeasured value and tokenmaxxing, inference is profitable on SOTA models because all capacity is being consumed at all times, driving down marginal costs, but what about a world in which capacity isn’t constrained? There are still huge fixed costs.

Even the most optimistic leaks with the current high prices put the margin on API token inference at around 50%. How can SOTA models ever come close to competing on price? Price always matters. Offering the best model with the most brand recognition does not exempt OpenAI from the basic rules of business.

Historically, software has been such a successful business because the margins are incredible, 95%+ in many cases, driven by direct measurable value to customers that dwarfs the cost. A 50% margin at a time when your customers are falling over themselves to spend as much money as they can is not a good sign, it is a very bad sign, it leaves no room to ever achieve traditional technology margins, and inevitably leads to very weak margins.

Inference needs to become an order of magnitude cheaper than the value it delivers to ever have a chance of delivering on this wildly profitable vision. The cheap model providers have a much better chance of achieving that.

Outside of coding, almost every business case for AI doesn’t need above human intelligence, it doesn’t even need human intelligence, or half a human intelligence, a business can extract a lot of value from a machine that has a fraction of a human’s intelligence. Most human work does not use our intelligence, it is rote, a monkey could do it, and that’s where AI will be used most. Who is going to pay $10 per million tokens when they could pay $0.10 to get the same outcomes?