redmalang
Joined 9 years ago
Karma
43
Submissions
24
Member since
Jun 27
say hello: [email protected]
View on Hacker NewsRecent submissions
Comment · 4 days ago
vaguely related - I want to make it easy for the llms to deploy and manage derp,tailgate etc with the hard requirement that you can continue to use the tailscale clients (very thankful to tailscale for making this so easy!):
Comment · 13 days ago
we use it to assign cost/project level analytics which is actually quite useful!
Comment · 13 days ago
For a lot of large scale day to day work, deepseek / zai models (the latter comes with a subscription) can be very cost effective and very competitive actually.
Another thing of note is that when coding at 'scale' to my surprise over time with memory aggregation across users/sessions these models can be better than the evals would have you believe because they seem to be differently/less (?) RL'ed (aka actually listen to instructions), which results in them being more fastidious about writing memories and benefiting from them, especially over time. These are of course mostly subjective opinions but they were surprising, and not what we were expecting.
Relevant: the proxy we wrote and use: https://yapp.latchlabs.dev is partially open sourced: still working on releasing more of our memories stuff.
Comment · 1 month ago
For the past several (months now!) I have been slowly working to open source a proxy we built for pi internally.
Its been super helpful for us, helping us centralise session logging, hook and model insights as people work on stuff. There is a bunch of other interesting things (such as runtime model evals) we now adding. If this would be of interest -- open source of course -- please drop me a note here, it will incentivise me to finally extricate it from our broader system:
Slopsite here: https://piproxy.latchlabs.dev
Comment · 3 months ago
This is in fact what we do (with higher order abstractions now built on top of this). This builds self evolving interactive knowledge base and puts it into a QMD searchable index. The indexer is already open source: https://github.com/jibs/duffel
Comment · 3 months ago
Aside 2:
Anecdotally we found that Pi performs more or less on par with native harnesses at lower cost on decently specified prompts. It is also phenomenal at context cacheing especially on Deepseek models (its hard to precisely attribute credit here are my understanding is this is a DS speciality).
But it fails much worse on poorly drafted prompts. I'm generalising but native harnesses seem to be better kind of flailing along on those.
Comment · 3 months ago
Yeah.
Comment · 3 months ago
We have an internal proxy (that I've been meaning to open source for ages) that routes all llm usage at our company, which allows us to see data in realtime. Its been fascinating how rapidly Pi has been adopted. Moreover since its pretty hackable, we've been able to automatically aggregate context from pi sessions, which has resulted in Pi efficacy being higher as more people use it, putting in place a interesting virtuous loop.
I didn't expect this outcome: for whatever reason I assumed proprietary harnesses fine tuned to work with a companies' models would work better?
ps/random aside: there is something slightly off about Pi's edit command, we are planning to investigate this further and patch this as we have quite a few session traces now..
Comment · 4 months ago
We have switched approx 80% of our work to deepseek, and it works great. Our setup is a bit unconventional though, we upload all cot / sessions to shared storage and generate centralised project level context. We've found this is helpful in directing and working with these slightly less sota models and getting great value for ai spend.
I'm planning to open source all this infra soon, hopefully useful for others too.
Comment · 4 months ago
Try llama.cpp it seems to be a lot more performant and a lot more hackable.
Also I'm surprised how substantial the impact of some of the inference configs (beyond just temp) can have, though this is much more model specific.
Comment · 7 months ago
i've found llama.cpp (as i understand it, ollama now uses their own version of this) to work much better in practice, faster and much more flexible.
Comment · 1 year ago
Anybody aware of anything like this for mandarin ?