erans
Joined 17 years ago
Karma
63
Submissions
41
Member since
Aug 27
Recent submissions
Comment · 23 days ago
Also, since DeepSeek 4.1 Flash is already quite quantized, the same OpenRouter benchmark harness puts us at #9, but the first 10 places (out of about 30) are within less than 1% of each other - so that is much harder to set.
But here, again, we run on B200, the official weights with specific configurations that we have on vllm.
Comment · 23 days ago
You are totally write. I wrote exactly about that. https://www.lunaroute.com/blog/the-model-is-not-the-product/
We will release our data soon. It's not just quantization, we are using NVFP4 from RadixLabs for GLM 5.3 but with our configuration and the OpenRouter harness for benchmark we are actually #4 in accuracy out of about 30+ providers. We run strictly on Blackwells (B200) on SGLang.
So its not just quantization, its the software, its version, its configuration, the exact hardware etc. They all matter.
Comment · 23 days ago
I've been using DS 4.1 Flash for a while now as my main workhorse. It's amazing. It's fast and it is also a GREAT debugger - it might not always find the best solution but it will definitely find the problem quickly.
I'm also the CEO of a new company - LunaRoute - so if you want private, fixed cost, we run our own server in the US type of DeepSeek 4.1 Flash check out https://www.lunaroute.com
If you want a trial, hit the contact us and write that you saw this post.
We also have GLM 5.3 (with vision!) and GLM 5.3 Flash - all included.
Comment · 6 months ago
For agents, if you are concerned about that, block access to "su" as it is interactive anyway. Not loading it into the memory will block the attack. If you are using AgentSH (https://www.agentsh.org) you can add a rule to block "su" and soon be able to block AF_ALG sockets if you want to further protect things.
Comment · 6 months ago
Execution layer security must be deterministic. That's why we are working on AgentSH (https://www.agentsh.org) which is model, framework and harness agnostic.
Comment · 6 months ago
The part that seems most important here is that npm install was enough.
Once the compromise point is preinstall, the usual "inspect after install" mindset breaks down. By then the payload has already had a chance to run.
That gets more interesting with agents / CI / ephemeral sandboxes, because short exposure windows are still enough when installs happen automatically and repeatedly.
Another thing I think is worth paying attention to: this payload did not just target secrets, it also targeted AI tooling config, and there is a real possibility that shell-profile tampering becomes a way to poison what the next coding assistant reads into context.
I work on AgentSH (https://www.agentsh.org), and we wrote up a longer take on that angle here:
Comment · 6 months ago
It's great to see more such platform popping up. It's good for the ecosystem. We need more hosting options that are clear, secure and have the ability to help people run as many models as possible.
Comment · 1 year ago
that's great. PagerDuty always felt so expensive and heavy!
Comment · 1 year ago
That's awesome. It's always annoying using those 3rd party ones!
Comment · 1 year ago
Summary data is in a sqlite database so its easily queryable by you or your friendly AI agent.
You can have a single proxy and have multiple team members use the same one so that you can easily track sessions, token usage, spend etc.
Comment · 1 year ago
true that there is a some kind of a ceiling of what can or can't be done. But that ceiling is way up there. Also, there are enough examples and articles and code that allows enough combination to be made so that its good enough - and that is a very important bar.
There are A LOT of businesses (even big ones managing money and what not) that rely on spreadsheets to do so much. Could this have been an app/service/SaaS/whatever ? probably.
What if these orgs can (mostly) internally solidify some of these processes? what if they don't need an insanely expensive salesforce implementor that can add "custom logic" ?
A lot of times companies will replace "complex software" with half complex process!
What if they don't need Salesforce at all because they need a reasonable simple CRM and don't want to (or shouldn't) pay $10k/seat/year ?
There are still going to be very differentiating apps and services here and there, but as time move on these "technological" advantages will erode and with AI they erode way faster.
Comment · 1 year ago
I also added a Mac version: https://selfhostllm.org/mac/ so you can know which models you can run on your Mac and get an estimated tokens/sec.
http://eran.sandler.co.il https://x.com/erans/ https://bsky.app/profile/esandler.bsky.social https://linkedin.com/in/erans/ https://tkmx.odio.dev/user/erans