Comment by k9294

I'm surprised more people aren't talking about the cache hit price: $0.003 per million tokens. I have a feeling that the price of 1 million tokens transmitted over the internet is more expensive than cache hit. Are we close to making the chat completion API obsolete because the cost of context transfer over network is going to dominate the task total cost?

Here's the same token usage priced at different rates: a real long-running coding task, medium codebase, 447 turns.

  Input           1,026,957
  Output          164,667
  Cache read      36,554,368

GPT-6-astra

  Type      Rate     Cost  Share
  Input   10.000   10.270    19%
  Output  50.000    8.233    15%
  Cache    1.000   36.554    66%
  Total            55.057   100%

DeepSeek v4.1 Flash, $0.003 cache hit

  Type      Rate     Cost  Share
  Input    0.300    0.308    50%
  Output   1.200    0.198    32%
  Cache    0.003    0.110    18%
  Total             0.615   100%

DeepSeek v4.1 Flash, $0.006 cache hit

  Type      Rate     Cost  Share
  Input    0.300    0.308    42%
  Output   1.200    0.198    27%
  Cache    0.006    0.219    30%
  Total             0.725   100%

Hypothetical: same DeepSeek input/output rates, but cache priced so it accounts for 66% of the bill.

  Type      Rate     Cost  Share
  Input    0.300    0.308    21%
  Output   1.200    0.198    13%
  Cache    0.027    0.982    66%
  Total             1.487   100%

This cache it improvement makes the model x2-x2.5 more efficient on a long horizon tasks in terms of cost.