nravic
Joined 9 years ago
Karma
53
Submissions
45
Member since
May 1
[ my public key: https://keybase.io/nravic; my proof: https://keybase.io/nravic/sigs/-W7C_1M4aYXWPyuN8wQdlxtuDKdPESuSpqvyZRykcTc ]
View on Hacker NewsRecent submissions
Comment · 1 year ago
I think even easier in fact - what's happening behind the scenes w/ an LLM is far more opaque
Comment · 1 year ago
This is super interesting! We do something similar I think by taking a checkpoint after model initialization. I'm curious what you think about our approach, here's some benchmarks: https://docs.cedana.ai/articles/performance-of-cedanas-gpu-i...
We do some on-the-fly optimizations as well (like compiling into CUDA graphs or fusing together calls) which ends up resulting (for some inference engines) faster token throughput too.
Comment · 2 years ago
How were you handling GPU state w/ pytorch? We added some custom code around CRIU to enable GPU checkpointing fwiw: https://docs.cedana.ai/setup/gpu-checkpointing/
Comment · 2 years ago
We do this using CRIU right now! https://github.com/cedana/cedana
In fact one of our customer's use cases is exactly what you describe, allowing users to "hibernate" container workspaces.
Comment · 2 years ago
I'm probably the cofounder of the guy you spoke with! Here's our repo: https://github.com/cedana/cedana
Comment · 3 years ago
what kind of pannier bags do you use to haul groceries? All the ones I've used have been too small for my purposes.
Comment · 3 years ago
IIRC (and in my experience) KL divergence doesn't account for double counting. Wrote a paper where I ended up having to use a custom metric instead:
https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=4...
Comment · 3 years ago
This is so cool! Out of curiosity, could you walk through your decisionmaking around using Nix and how it fits in with Rivet?
Comment · 3 years ago
how often are these batch jobs run? I'm curious to know what the absolute maximum sync frequency can be.
Comment · 3 years ago
Self plug: run llama.cpp as an inference server on a spot instance anywhere: https://cedana.readthedocs.io/en/latest/examples.html#runnin...
Comment · 3 years ago
GrapheneOS uses WebUSB to flash your phone.
Pretty wild first experience doing it for me - I'm used to the hassle of sideloading a bootloader and then flashing.
Comment · 4 years ago
Hey! I'd love to learn more about the position - the hook @ jpl email on your website is bouncing though. Do you mind sending me contact details so we can chat?
Thanks :)
Comment · 5 years ago
Do people out of college even do Leetcode anymore? Surely there's a better use of time once you're in a paying job.