ahzhou
Joined 7 years ago
Karma
303
Submissions
90
Member since
Nov 8
Recent submissions
Comment · 8 months ago
Fast as in time-to-market
Comment · 1 year ago
VC vs bootstrap is usually based on company TAM. There are certainly high growth bootstrapped businesses.
Comment · 1 year ago
Not common in Silicon Valley, but much more common in the rest of the country.
There’s an archetype for bootstrapped tech businesses:
- highly vertical specific
- couple hundred million TAM
- founder started the business in their 30s and is now in their 40s
Comment · 1 year ago
It’s a tensor stored in GPU memory to improve inference throughput. Check out the PagedAttention (which introduces vLLM) paper for how most systems implement it nowadays.
Comment · 1 year ago
They slightly restructure their MoE [1], but I think the main difference is that other big models (e.g Llama 504B) are dense and have higher FLOP requirements. MoE should represent a ~5x improvement. FP8 should be about a ~2x improvement.
We don’t know how much of a speed improvement GRPO represents. They didn’t say how many GPU hours went into to RLing DeepSeek-r1 and we don’t have a o1 numbers to compare.
There’s definitely lots of misinformation spreading though. The $5.5m number refers to Deepseek-v3, not Deepseek-r1. I don't want to take away from HighFlyer's accomplishment, though. I think a lot of these innovations were forced to work around H800 networking limitations, and it's impressive what they've done.
Comment · 1 year ago
You can easily do a fermi estimate based on the information given. They are comparing GPU hours.
Comment · 1 year ago
I might be missing something, but DeepSeek’s recipe is right there in plain sight. Most of the cost efficiency of DeepSeek v3 seem to be attributable to MoE and FP8 training. DeepSeek R1s improvements are from GRPO-based RL.
Interesting to note - we have no idea how much R1 cost to train. To speculate - maybe DeepSeek’s release made an upcoming Llama release moot in comparison.
Comment · 1 year ago
LLMs are inherently bad at this due to tokenization, scaling, and lack of training on the task. Anthropic’s computer use feature has a specialized model for pixel-counting:
> Training Claude to count pixels accurately was critical. Without this skill, the model finds it difficult to give mouse commands. [1]
For a VLM trained on identifying bounding boxes, check out PaliGemma [2]
You may also be able to get the computer use API to draw bounding boxes if the costs make sense.
That said, I think the correct solution is likely to use a non-VLM to draw bounding boxes. Depends on the dataset and problem.
1. https://www.anthropic.com/news/developing-computer-use 2. https://huggingface.co/blog/paligemma
Comment · 2 years ago
Author: @fandzomga
Username: fsndz
Why try to funnel us to your paywalled article?
Comment · 2 years ago
Conditionally yes. There are many libraries that cannot be tree shaken for various reasons. Libraries typically need to stick to a subset of full JS to ensure that the code can be statically analyzed.
Comment · 2 years ago
GraphQL is very powerful when combined with Relay. It’s useless extra bloat if you just use it like REST.
The difference between the two technologies is that LangChain was developed and funded before anyone know what to do with LLMs and GraphQL was internal tooling using to solve a real problem at Meta.
In a lot of ways, LangChain is a poor abstraction because the layer it’s abstracting was (and still is) in it’s infancy.
Comment · 2 years ago
To be fair, LLM-based chatbots are much better about this because you don't need to discover the magic incantation to talk to a human. It's a trade-off because that same property introduces the possibility of hallucination.
Comment · 2 years ago
It depends on the business, but the kind of metrics you are talking about are measured and taken seriously. People have absolutely gotten fired for CS quality KPI drops.
Comment · 2 years ago
While it may not happen for you, “too lazy to look it up” is the vast majority of CS requests.
My understanding from talking to a couple of CS execs is that these have been a slam dunk in terms of ROI because CS agents don’t need to handle type C requests. I expect we’ll only see more as time goes on.
Comment · 2 years ago
> AppStore would be dead on arrival
Certainly not. PMF was already established via the jailbreaking scene and Installer.app / Cydia. Millions of people went through the annoying processing of jailbreaking their phone to get apps.