Comment Comment by polvi A new spec, TPX, which is meant to enable building SaaS where the user brings their own inference:https://tokenpony.dev/It has a few potentially interesting side effects:- enterprises can plug their preferred LLM into the SaaS they use.- self hosters can extend their rigs into the web.- simple SaaS products can offer LLM integrations without their entire pricing model being upended by LLM hosting and infrastructure.- we can all live like Ender in Xenocide and have a personal AI that has context on everything we are doing.- have a no op provider for people who are anti llm. parent polvi 2 months ago
https://tokenpony.dev/
It has a few potentially interesting side effects:
- enterprises can plug their preferred LLM into the SaaS they use.
- self hosters can extend their rigs into the web.
- simple SaaS products can offer LLM integrations without their entire pricing model being upended by LLM hosting and infrastructure.
- we can all live like Ender in Xenocide and have a personal AI that has context on everything we are doing.
- have a no op provider for people who are anti llm.