Hi HN, we built TOD, a model and API for jevlike LLM pipelines.
we have people using it for, routing a ticket, classifying intent, deciding whether something is urgent, or rating severity.
Architecture
- 12B Gemma fine-tuned to score options. since it is gemma based it has multimodality built in
- When there are >120 labels, we have a 150M retriever that narrows them to a shortlist (recall@16 is 0.9–1.0 on our evals), and the 12B model scores that shortlist.
- Context goes up to 49k tokens, with up to 8 images per request.
we have people using it for, routing a ticket, classifying intent, deciding whether something is urgent, or rating severity.
Architecture - 12B Gemma fine-tuned to score options. since it is gemma based it has multimodality built in - When there are >120 labels, we have a 150M retriever that narrows them to a shortlist (recall@16 is 0.9–1.0 on our evals), and the 12B model scores that shortlist. - Context goes up to 49k tokens, with up to 8 images per request.
we have open sourced it,
model: https://huggingface.co/parsecai/tod API: https://parseclab.ai/tod
We are working on getting the RL environments to finetune for custom use cases.
would love to hear your opinions on this.