Given the recent popularity for Jev, I also gov the simplicity behind this system, and I understood there is a big opportunity for the opensource community to create and maintain an alternative with even better powers. Then I made Typed-lm.
Typed-lm used regular LMs with Parallel Evaluation via KV-Cache Broadcasting, reaching subsecond results inference times for typed structures.
Also, it is able to train and fine tune your own models, focusing the Jev types structure, and allows providing a supply file to preffil it as a RAG.
Ideia is allowing you to create your own decision inside your servers, with privacy, and fully oriented to your business need, with Jo vendor lock-in.
Please, let me know your feedback, and add any comments and issues you'd like on GitHub.
Where is also a tutorial page:
https://neurono-ml.github.io/typed-lm/
Typed-lm used regular LMs with Parallel Evaluation via KV-Cache Broadcasting, reaching subsecond results inference times for typed structures.
Also, it is able to train and fine tune your own models, focusing the Jev types structure, and allows providing a supply file to preffil it as a RAG.
Ideia is allowing you to create your own decision inside your servers, with privacy, and fully oriented to your business need, with Jo vendor lock-in.
Please, let me know your feedback, and add any comments and issues you'd like on GitHub.
Where is also a tutorial page:
https://neurono-ml.github.io/typed-lm/