Comment

Comment by ndr_

If you require consistency for research, I wouldn't recommend relying on hosted inference without validating the provider/inference stack very carefully.

For my contribution to OpenAI's gpt-oss red-teaming competition (published as arXiv:2510.01259), I initially used OpenRouter with DeepInfra and Together AI. The results were much too noisy to draw reliable conclusions from, and markedly different behavior through AWS Bedrock was the last straw. I ended up renting GPUs through vast.ai and running the model myself with vLLM.

parent ndr_ 1 month ago

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton