vkkhare

updated the link

Isn't the inference cost of running these models at scale challenging? Currently it feels like small LLMs (1B-4B) are able to perform well for simpler agentic workfows. There are definitely some constraints but surely much easier than to pay for big clusters on cloud running for these tasks. I believe it distributes the cost more uniformly

I would love to talk more about it and understand your take. Is there a way I can reach out to you? I have been reading the content in web 3 and form my own opinion on the merits/demerits.

I was thinking along ML. How analytics and ML happens on user data which is stored on data silos. For example, twitter has data about all our posts and interactions and uses that data to build recommendation models which it monetizes as to who comes on top.

In decentralized world, everyone can build their models, host them and train them on user data. But since, I wouldn't want my data to be used for any other purpose, I don't want them to make a copy of that data. So they send their models (after taking my consent) to my device, train it locally and send the aggregated model back.