thelastbender12
Joined 9 years ago
Joined 9 years ago
Why does not justify charging a fraction of your spend on the LLM platform? This is pretty much how every service business operates.
I still think what drives languages to continuously make changes is the focus on developer UX, or at least the intent to make it better. So, PLs with more developers will always keep evolving.
Programming languages which do get used are always in flux, for good reason - python is still undergoing major changes (free-threading, immutability, and others), and I'm grateful for it.
An addon makes it seem like an afterthought, which I'm certain you are not going for! But still making is as seamless as possible would be great. For ex, response time for Claude in Cursor is much better than even the Claude web app for me.
I'd love it if you lean into pooled model usage, rather than it being an addon. IMO it is the biggest win for Cursor usage - a reasonable num of LLM calls per month, so I never have to do token math or fiddle with api keys. Of course, it is available as a feature already (I'm gonna try Continue) but the difference in response time b/w Cursor and Github copilot (who don't seem to care) is drastic.
IMO a hindrance to this was lack of built-in fixed-size list array support in the Arrow format, until recently. Some implementations/clients supported it, while others didn't. Else, it could have been used as the default storage format for numpy arrays, torch tensors, too.
(You could always store arrays as variable length list arrays with fixed strides and handle the conversion).
Though I am assuming building durability at a language-level means the whole program state must be serializable, which sounds tricky. Curious if you could share more?
To wit, if I am doing a high school geometry proof, I come up with a sequence of steps. If the proof is correct, each step follows logically from the one before it.
However, when I go from step 2 to step 3, there are multiple options for step-3 I could have chose. Is it so different from a "most-likely-prediction" an LLM makes? I suppose the difference is humans can filter out logically-incorrect steps, or prune chains-of-steps that won't lead to the actual theorem quicker. But an LLM predictor coupled with a verifier doesn't feel that different from it.
https://ananis.in