nserrino
Joined 11 years ago
Karma
393
Submissions
56
Member since
Dec 25
Recent submissions
Comment · 3 months ago
Voice is one of the most latency-sensitive modalities in AI. Moonshine is doing awesome stuff
Comment · 1 year ago
Hey, thanks for the thoughtful comments. A lot of big claims have been made in this area so skepticism is the right default reaction. tl;dr: agree that we should provide the kernels and benchmark suite so this can be evaluated by others, will follow up with that.
A few clarifications:
1. Baselines - We didn't compare to torch.compile because as of PyTorch 2.7, torch.compile doesn't support the MPS backend, and we ran into some issues on many of the problems when using it. GitHub issue: https://github.com/pytorch/pytorch/issues/150121. Once it's supported, it will be the obvious baseline.
2. Methodology - We followed KernelBench’s protocol to establish a baseline on Metal, adding more correctness checks. Warmup and synchronization were done. We recognize the limitations here and are expanding the validation suite.
3. Optimizations - Right now most of the optimizations are fusions, but there is some use of Metal-specific primitives/optimizations. We expect as we make the supervisor more sophisticated, the novelty of the optimized kernels will also increase.
Overall the goal here is to get some % of the benefit of a human expert in kernel engineering, without developer effort. Compiler-based optimizations are great, but hand-tuned implementations are still common for performance-critical models. The hope is that we can automate some of that process.
Comment · 1 year ago
PyTorch is the baseline because that's what people prototype in, and the most common reference point. The aim here is to show that you can start from prototype code and automatically produce lower-level kernels (in this case Metal) that are more usable in real deployments, without additional work from the developer. Frontier models are capable at generating efficient Metal kernels automatically/immediately, and will only get better. We expect to see significant improvements as we refine the approach, but it's enough to show this seems to be a tractable problem for AI.
Comment · 1 year ago
They are just incredible live ... Flood was the soundtrack of my childhood. It's great to see how many fans on HN they have. If you have a chance to check them out in concert, do it!
Comment · 1 year ago
What kind of models are you deploying and what type of problems are you having with deploying them?
Comment · 2 years ago
I've mostly switched over to Waymo, but had to take Ubers twice in the last month:
* The first one, the driver made multiple racist remarks about different groups he observed as we drove.
* The second one, the driver talked at length about UFOs and how they are real, for the entire 50 minute drive.
Most drivers are totally normal and don't do things like that, but the tail end of negative experiences can be quite bad. Dirty cars, loud radios, body odor, and unsafe driving are all relatively common with human drivers. A Lyft driver I was riding with a few years back almost ran over a man in a wheelchair who had the right of way.
Wait times are also more reliable so far with Waymo. It's not uncommon for an Uber/Lyft driver to accept a ride but then not drive toward you for 5+ minutes. Waymo has the advantage of predictability - both in terms of arrival time and overall travel time (whereas there is variance among human drivers).
Sometimes I've had Waymos get stuck, but it usually resolves within 10-15 seconds.
Given how smooth, predictable, and safe Waymos are, I don't see a strong reason to risk a negative experience with a human driver (beyond ideological reasons). However, I hope another strong provider comes on the market soon to give them some competition.
Comment · 4 years ago
The stories we often see about burnout in tech are interesting. There’s a lot about burning out of programming etc, the hours. Of course, long hours will eventually burn you out in most any profession. Less commonly discussed, it seems like most people I know actually burn out from bureaucracy. It’s far more exhausting and demotivating to not make forward progress and waste time in meetings all day. Of course, it should go without saying that some meetings are necessary.