yoeven

DefaultModel is a new inference service that hosts one model: the single best open source model, picked by the community.

Open source models are catching up to their closed source counterparts, but inference for them is scattered across general GPU providers. Those providers constantly swap models, and pricing, reliability and support differ from one to the next. You don't get the consistency of a provider like OpenAI.

Large labs get around this by aggregating their own GPU resources, pointing most of the capacity at their latest model, and migrating users there when a new one ships. We want the same for open source models.

So we're taking a different approach. We host one model, the best one, voted by you, the community. We host it for a quarter, and every quarter you vote to keep it or replace it based on the latest releases.

- All GPU capacity goes into one deployment instead of a catalog

- Serving is tuned for that one model, not averaged across fifty

- Performance stays consistent across every request, with no degradation in quality

- When a better model wins the vote, you migrate by changing one model ID instead of rewriting an integration

thanks!

1. yeah, we've optimized significantly for latency, give the API a try and you'll see it's pretty fast.

2. more than you think, we use a wide range of providers internally depending on the context needed. Exa could be great in some for example people search but is pretty bad at getting the latest content of the page, Parallel is better there. Apify is great for getting google ranked pages. Perplexity is great at financial data which Exa is pretty bad at. Context7 (coming soon) great at code, way better than exa for code. Depending on the use case, different indexes do better. we're also seeing companies with access to prosperity data like Valyu.

Really cool! LLMs tho, it's hard to measure accuracy and attribution in real time like confidence scores and bounding boxes which is great for KYC apps. Try Interfaze for LLM OCR, get's you the metadata and output you want.

Yup does really depend on the use case.

We see two types: workflows & agents.

Workflows are the most common, there's a pipeline like processing loan documents before data gets loaded to the next step or translating user comments before being stored in the database.

Agents are where you have a chat based system or a brain of sorts that calls many tools to achieve a user goal. The model doing this is a lot better at non deterministic task which then delegates to Interfaze for specific deterministic actions like OCR, Web extract then consumes that data. That's the article you referenced :)

Thanks for the feedback!

We're working a lot more on speed in the coming few weeks :) More GPUs and more optimizations.

Our has been focus on quality of output first and we'll make optimizations as we grow :)

The lite models are great for simple use cases but won't don well in more complex OCR use cases.

It can, you could try prompting the model to use object detection vision and text extraction, we realized when we purely extract text it does amazing at word/sentence level bounds since the text acts as the anchor. However, when you treat it as a object detection problem, it sees that chunk of text as a segment allowing you the extract it as one column bound. Give that a try.

Not directly, LAMs tend to be focused a lot on tool calling or trained for a set of specific action for example in the robotics field. Good tool calling might be a good by product of Interfaze but wasn't specifically trained for that use case.

The focus has been for deterministic outputs that require high accuracy. In situations where there is "one right answer"