jtbaker
Joined 6 years ago
Joined 6 years ago
just wish their containerization / linux compat wasn't so nerfed. Just moved my homelab's inference engine over to a GB10 from a Mac MTP setup to have first class k8s support, as well as open up access to the broader CUDA ecosystem (cuOpt etc.).
> “I’m thinking about banning Claude Code at Shopify until they change their mind and read AGENTS.md and .agents/skills etc.,” Lütke posted Tuesday on X.
In general, I'm a big believer in doing more with fewer resources, within reason, and think having local setups really helps me be mindful with what's happening under the hood with these systems and managing context efficiently to get high quality results.
I (ok, Qwen 3.7 27B) wrote some benchmarking code to compare throughput, and niquests seems to be substantially more performant for traffic that doesn't need to traverse the public web, e.g. intra cluster service to service communication:
https://gist.github.com/jtbaker/61061d27949ef48ac31e85ff28c2...
uv run scripts/bench_http.py --count 1000 Benchmarking 1000 requests to http://localhost:8000/health (concurrency=16, timeout=30.0s) Date: 2026-08-28T10:18:34.480111
Running httpx... Running httpx2... Running niquests...
Metric httpx httpx2 niquests ------------------------------------------------------- mean 17.8ms 4.6ms 3.3ms min 3.2ms 1.5ms 1.2ms max 632.4ms 11.1ms 10.8ms p50 15.9ms 4.2ms 3.0ms p95 32.4ms 7.4ms 5.6ms p99 52.1ms 8.5ms 8.4ms total 1.15s 315.5ms 239.7ms throughput 869.2 req/s 3170.0 req/s 4171.1 req/s
Winner: niquests (4.8x faster by wall clock)