fzysingularity
Joined 13 years ago
Karma
219
Submissions
170
Member since
Jan 3
computer-vision, ml and all things meta.
View on Hacker NewsRecent submissions
Comment · 2 days ago
It doesn’t seem like they have monopoly here given how big Uber Eats is?
Comment · 17 days ago
Yeah, it’s surprising that this is the runtime model they went with as opposed to a hybrid edge/cloud model. I’d love to see a system 1 on edge, system 2 on cloud offering. They’re likely the best positioned to do this.
Comment · 1 month ago
VLM Run (https://vlm.run) | 1x Founding Infrastructure Engineer | In-Person (Bay Area) | Full-time
We’re building the inference platform for visual intelligence. We’re a deeply technical team of AI / computer-vision engineers (20+ years combined, MIT/CMU/NC State PhDs) who’ve shipped production ML infra across autonomous driving and LLMs.
We launched the VLM Run Gateway (https://vlm.run/gateway) in September, a unified API that serves open-weight VLMs, embodied VLAs, ViTs, served across a fleet of GPUs and clouds. That's exactly the infrastructure problem this role will own. We're already scaling to serve 100s of thousands of requests per day, so if this sounds exciting to you, read on.
Email us at [email protected] with your GitHub profile, papers, ML projects you’ve recently shipped (100+ GH stars only) - especially with Ray, k8s, GPUs, serverless. No AI text please, keep it short, shorter emails are more likely to get a response. No remote work, must be in the Bay Area (specify in email subject).
Comment · 1 month ago
When you say Jev is optimized for it, I get that there’s no need for unnecessary decodes in an autoregressive fashion.
But both LLMs and Jev-like models would need to prefill, the only optimization Jev does differently is the decode which can be emulated by reading off logprobs.
We don’t know the param size of Jev, to determine the most comparable model, but if I had to guess it’s sub-100B.
Comment · 1 month ago
I don’t know of one definitive resource but “constrained decoding”, regex/grammar-based decoding, json-schema decoding will give you a bunch of hits. Look up vLLM, SGLang and outlines’ implementations for more technical details.
This looks pretty decent: https://www.aidancooper.co.uk/constrained-decoding/
Comment · 1 month ago
FWIW you’re absolutely correct on how most developers are unaware of this as they’re mostly operating on the API level and not aware of server side (vLLM/SGLang) capabilities.
The aspect that I like the most is the typesafe API that introduces new probabilistic concepts that are more sound than json schema and constrained decoding with quasi-confidence scores. Developers were asking LLMs to also emit confidences which made absolutely no sense whatsoever.
Comment · 1 month ago
Mostly just 3d world models though, they're not the same as multimodal models.
Comment · 1 month ago
We just did exactly this - added TypeSafe-compatible API (incl. websocket support) for various VLMs. Latency right now is <250ms, but will be able to get it to <150ms (p95).
Take a look at a snake demo with streaming image inputs: https://x.com/spillai/status/2103630735425089957
Comment · 1 month ago
Is the LiDAR sufficiently accurate at those distances?
Comment · 1 month ago
Curious about this: do you expect that Claude science will also scoop interesting directions looking at anonymous usage, to publish posts like this before the original author (similar to the Navier Stokes debacle)?
Comment · 1 month ago
Am I missing something here:
p(y = next thinking+decision token | x = question) != p(y = next decision token | x = question)
The former is what LLMs are trained for, the latter is what Jev was likely trained on (likely used thinking alignment as an auxiliary loss, but not explicitly included in the probability calibration).
Comment · 1 month ago
Very cool to see the openness here, and likely more like this will come from smaller startups where they win users on transparency.