arilotter

remove the "change" button for areas until you have it lol i was trying so hard to find startups in rio kkkk

My company was using Iroh for a production distributed ML training system & we LOVED it. The team was incredibly responsive even before we hooked up with an enterprise support contract, they're incredibly knowledgeable and the library itself worked amazingly. ++ to this lib. would use again over libp2p anytime.

This title is moderately clickbait-y and comes with a subtle implication that Rust might be getting removed from the kernel. IMO it should be changed to "Rust in the kernel is no longer experimental"

Hi - Psyche developer here. Happy to answer any/all questions about the project :)

iroh's stuff is great but their local peer discovery can't work in a browser, since it uses an mdns-like protocol to do it

big ++ for iroh - using it for a project at work, it's evolving really nicely, has reliable holepunching, and the team is super responsive and happy to dig thru gigs of logs to find a bug :)

you could absolutely use DisTrO for federated learning. The DeMo optimizer on its own doesn't solve the adverserial aspects of training on local-only data, nor does it solve tensor parallelism across devices, so you're still limited to only what fits on your local GPUs, but it does enable distributed data parallelism over the internet at a bandwidth orders of magnitude lower than before.

This specific model is only trained on 100 billion tokens, so it's not SOTA by any means, but we've got designs on larger training runs later :)

> Are you planning to eventually do a SETI@home style thing where anyone can join?

That's one of the goals of the stuff we're working on right now, I personally hope we can make it work!

> What was Durk Kingma's involvement in DisTrO?

As a co-author on the paper, we brought him in to bounce some ideas off of & have him validate DeMo's design and implementation to ensure we weren't hallucinating these results.

Yo! I'm on the team at Nous that built this tech, happy to answer any questions :)

we're excited to drop the paper & code and let you run the code for yourself to see otherwise !! coming in the next couple months :)

We're excited about the potential and want to find other folks also excited about it that are interested in working for/with us to build things on the foundations of DisTrO! Plus also it's so cool and mind boggling to us that we wanted to share the hype a little bit, it was hard not being able to tell anyone we were working on it