logotype

I'm developing an inference engine and need access to the latest Neural Accelerator features, and therefore need to keep up to date with the Apple releases. M1 through M5 Max 128GB. As a broke bootstrapped un-funded startup founder this is extremely tough to afford given the price increases and all.

> "There is nothing happening between requests to the inference engine." This is true for vLLM/SGLang/etc but LayerScale is making use of idle GPUs. A new novel inference engine for agentic workloads. Our advanced scheduler enables double the workload on the same number of GPUs (e.g. prioritization of on-demand queries and background processing of batch workloads simultaneously). Let me know if you want to try it out!

Hi HN. I'm an independent researcher and I've been working (completely self-funded) on my inference engine since January 2025. It's built from scratch and uses highly optimized fused kernels and I've been testing it against vLLM and SGLang. Thank you, Victor

MTurk existed since forever, and somehow flew under the radar for all these years. Very few developers I know didn't even know it existed.

Damn. I just bought a maxed out MacBook Pro M5 Max 128GB 8TB, still waiting for it to be delivered. I could get 256GB RAM M5 Ultra 1TB for roughly the same price, and it's double the memory bandwidth. Which one would you recommend? I do plan to run local LLMs.

Both can be true at the same time.

With native support for sessions in an inference engine we can make use of idle GPUs... doing work!

I bought a server with 768GB RAM a couple of years ago, for cheap. Today just the RAM itself is worth more than I paid for the entire server. DDR4... Same thing with SSD, two years ago you could buy 4TB SSDs for like 250 USD, and today its more than $700. Madness!

Fastest tool calling on the planet!

Attended Dreamhack in the late 90s. Fun times! I was always AMAZED by the demoscene.