zhoutong
Joined 16 years ago
Karma
1,727
Submissions
274
Member since
Dec 20
Recent submissions
Comment · 2 months ago
There’s also input tokens. For many agentic use cases input/output ratio can be 2:1 or even 4:1. Non-quantised DeepSeek V4 Flash costs $0.14 / $0.28 on most inference providers with ZDR.
When self-hosting the model, cache hits are basically free. RAM cache also helps with hit rate (prefixes may be around for hours instead of minutes).
To be clear this project doesn’t aim to achieve the best inference economics per token. MI300X doesn’t have native MXFP4 so it’s not even the right platform for the model. That’s why very few deployment recipes are available.
It’s interesting to me because MI300X is quite accessible to a small team with budget for just 1-2 GPUs. DeepSeek V4 Flash otherwise wouldn’t even fit on 2x H100s.
We can run several coding agents during the day and batch inference jobs overnight and serve the entire team with guaranteed privacy, without compromising precision or speed.
In fact we found that many inference providers are quantising the weights or even KV cache, and due to the low prices they serve at massive batches, resulting in unstable throughput. I ran GSM8K as a quick validation test and this deployment is “better” than the OpenRouter endpoint in a statistically significant way (I wouldn’t name the provider here). I will run some follow up benchmarks and update the repo when I find some time.
Comment · 2 months ago
It’s available on demand from a few cloud providers. Seems like the cheapest is AMD Developer Cloud (https://www.amd.com/en/developer/resources/cloud-access/amd-...) powered by Digital Ocean at $1.99/hour.
Edit: Now I think about it, this might be the cheapest way to run the DeepSeek V4 Flash 0731 on a dedicated inference server at original weights. I haven’t run mixed load benchmarks but I guess it’s possible to generate $3-$4 worth of tokens per hour and still maintain a usable per-user throughput.
Comment · 3 years ago
Because First Citizens Bank acquired all of the deposits and loans but none of the securities, presumably the only way for the FDIC to complete the deal is to pay First Citizens Bank the difference in cash, which is roughly $63.5 billion (napkin maths: the $119B in deposits are assumed one-to-one, and $72B loans are acquired at $16.5B discount, resulting in a cash outlay of -$63.5B for the acquirer for a bundle of net assets worth -$47B on paper).
If depositors start withdrawing money from the new bank, they at least have access to this amount of extra liquidity from the acquisition.
Comment · 3 years ago
Upon further reading I found that the Viability Event (which is also a Write-down Event) is probably the applicable one in this case:
> (b) customary measures to improve CSG’s capital adequacy being at the time inadequate or unfeasible, CSG has received an irrevocable commitment of extraordinary support from the Public Sector (beyond customary transactions and arrangements in the ordinary course) that has, or imminently will have, the effect of improving CSG’s capital adequacy and without which, in the determination of the Regulator, CSG would have become insolvent, bankrupt, unable to pay a material part of its debts as they fall due or unable to carry on its business.
Clearly this deal required non-customary, extraordinary support from the Public Sector, so if the regulator determines that without such a transaction CSG would have liquidity issues, then this event would occur.
However I think my point stands that the terms of these CS AT1 notes should be understood as materially different from similar securities. For example, this note from ING (https://www.ing.com/MediaEditPage/XS2122174415-ING-Groep-N.V...) has a single Trigger Event that will cause mandatory conversion into ordinary shares, and a more general provision for Statutory Loss Absorption which may be a conversion or a write-down. The terms of CS AT1 notes do not seem to provide for any automatic or discretionary conversion, only automatic write-down.
I completely agree that in situations like this, the regulators have a lot of discretion on these bail-in securities, but I consider this "automatic permanent write-down" feature to be of a materially higher risk than "automatic mandatory conversion" variant because it could be a difference between getting back something (or everything) vs nothing. What the regulators do are, by definition, not "automatic", and an automatic write-down should be a much lower hurdle than an explicit regulatory action.
Comment · 3 years ago
I was initially surprised about this because AT1 notes are supposed to rank higher than equity. It seems that almost no one saw this coming (CS AT1 bonds traded higher this weekend before the write-down announcement), and traders presumed that bondholders should be made whole if equity holders get something.
However then I looked at the information memorandum of these AT1 bonds (e.g. https://www.credit-suisse.com/media/assets/about-us/docs/inv...). Credit Suisse titled their issues as "Perpetual Tier 1 Contingent Write-down Capital Notes". Note that it's "contingent write-down" rather than the more typical "contingent convertible". The IM also doesn't contain an explicit conversion price or conditions.
Almost everyone would call this a "CoCo bond", even though its terms are exceedingly clear -- if CET1 falls below 7%, a Contingency Event, which is a Write-down Event, occurs, and "the full principal amount of the Notes will automatically and permanently be written-down to zero on the Write-down Date." In other IM issued by other banks I've seen, usually such event is followed by a mandatory conversion to ordinary shares rather than an immediate write-down. I wonder if this nuance was fully considered and priced in the trading of such instruments.
Comment · 3 years ago
Effectively there are two banks (Silicon Valley Bridge Bank, N.A. and Signature Bridge Bank, N.A.) with de facto unlimited FDIC insurance, as there's explicit guarantee for all existing and new deposits.
Contact: https://www.ryanzhou.com/contact/