seldo

For agentic use cases, where you might need several round-trips to the LLM to reflect on a query, improve a result, etc., getting fast inference means you can do more round-trips while still responding in reasonable time. So basically any LLM use-case is improved by having greater speed available IMO.

We are planning to move our blog off of Medium (we've been busy!), but this post is public so you can actually just click through the nag screen if you see one.

Retrieval-Augmented Generation, where you ask an LLM to answer a question by giving it some context information that you have retrieved from your own data rather than just the data it was trained on.

LlamaIndex lets you attach metadata to Nodes which are basically chunks, although that fact is poorly documented! Will fix.

This really isn't news, folks. It happens every week. I was just grumpy this morning.

We're working on a full post-mortem now. Until then we don't want to give out misleading/partial information.

Absofuckinglutely. It's being done as we speak.

I think you should probably re-examine your priorities in life, but that's just me. You do you.

Yarn does not run a mirror of the registry. registry.yarnpkg.com is a pass-through domain to the npm registry. It allows them to collect stats about yarn usage but is not a mirror.

L1 visas are easier for the company to get, because there is no cap on the number issued. The employee on an L1 is just as qualified as an H1, but less free -- they cannot switch jobs, and if they get fired they must leave the country within 15 (!) days.