alexgartrell
Joined 17 years ago
Karma
2,937
Submissions
583
Member since
Mar 25
Recent submissions
Comment · 22 days ago
new account, obvious troll
Comment · 2 months ago
https://thinkingmachines.ai/tinker/ https://github.com/thinking-machines-lab/tinker-cookbook
(As a TML person, I'm obviously biased, but I couldn't resist because of "tinkering").
TBF it's hard to imagine a real architecture change that wouldn't require a ton of compute, but you could certainly fine tune and play with different recipes, loss functions, etc. And Claude can carry you a lot of the way through doing this.
One fun task is to invent a tool and then train a small model to use it. You could export that small model and run it locally for free forever to do your thing. I think this is what a lot of Software Engineering will look like later.
There are a lot of other high level abstractions here to look at. Prime Intellect has one.
The other thing to play with is self-hosting small models, but IMO most of the interesting stuff is actually related to multi-gpu or multi-node inference so there's not necessarily a ton to learn here.
Comment · 7 months ago
The linked article says they decided to do CD in 2016 fwiw so that's not inconsistent with what I said.
You reduced the number of patches a lot and also pushed very hard to get us to 3.0 after we sat on 2.6.38 ~forever. Which was very appreciated, btw. We built the whole plan going forward based on this work.
I'm not arguing that anyone should be nice to anyone or not (it's a waste of breath when it comes to Linux). I'm just saying that the benchmarking was thorough and that contemporary 2014 hardware could zero pages fast.
Comment · 7 months ago
For the peanut gallery more: I worked with both of these guys at Meta on this.
The "servers are only on for a few hours" thing was like never true so I have no idea where that claim is coming from. The web performance test took more than a few hours to run alone and we had way more aggressive soaks for other workloads.
My recollection was that "write zeroes" just became a cheaper operation between '12 and '14.
A fun fact to distract from the awkwardness: a lot of the kernel work done in the early days was exceedingly scrappy. The port mapping stuff for memcached UDP before SO_REUSEPORT for example. FB binaries couldn't even run on vanilla linux a lot of the time. Over the next several years we put a TON of effort in getting as close to mainline as possible and now Meta is one of the biggest drivers of Linux development.
Comment · 1 year ago
I did something similar a long time ago https://github.com/facebookresearch/py2bpf
It was definitely a toy, I transliterated from python bytecode (a stack based vm) into bpf. I also wrote the full code gen stack myself (bpf was simpler back then)
But using llvm and not marrying things to cpython implementation makes this approach way better
Comment · 1 year ago
Not sure it’s relevant in the cloths these guys take
Comment · 1 year ago
The cloud business model is to use scale and customer ownership to crush hardware margins to dust. They’re also building their own accelerators to try to cut Nvidia out altogether.
Comment · 1 year ago
I’d imagine that these clouds are probably being incentivized to participate
Comment · 1 year ago
I don’t think pivot_root is necessary for something like this, but a new mount namespace will definitely help avoid creating a mess on accident
Comment · 1 year ago
More low effort posts please!
Comment · 1 year ago
Sharing a queue itself is not new https://www.kernel.org/doc/html/v5.8/networking/packet_mmap.... and https://docs.kernel.org/next/userspace-api/perf_ring_buffer.... are two examples.
Issues with io_uring security mostly stemmed from an old architecture and just the fact that there's a ton of surface area.
Comment · 2 years ago
The thing that we need in order for your dream to become a reality is excellent user space frameworks, so I encourage you (and anyone else) to go build one or (better) find one you like and contribute.
Comment · 2 years ago
> File IO is perhaps the best example of this (at least on Linux). To handle such cases, languages must provide some sort of alternative strategy such as performing the work in a dedicated pool of OS threads.
AIO has existed for a long time. A lot longer than io_uring.
I think the thing that the author misses here is that the majority of IO that happens is actually interrupt driven in the first place, so async io is always going to be the more efficient approach.
The author also misses that scheduling threads efficiently from a kernel context is really hard. Async io also confers a benefit in terms of “data scheduling.” This is more relevant for workloads like memcached.
Comment · 2 years ago
Yeah that’s it. Probably just wasn’t supported in the rust age library when I used it. Will double check.
Comment · 2 years ago
Age is great. I used the rust crate to write an ftp server that encrypts the files before they hit disk (specific use case is having a drop box for my network scanner) and I love the simplicity and composability it provides.
One feature request: it would be awesome to have paraphrase encryption for age private keys.
Computers are cool