robertkarl
Joined 15 years ago
Karma
361
Submissions
39
Member since
Sep 2
Recent submissions
Comment · 6 days ago
I tested this on Qwen 3.5/3.6 4B/27B/35B with "I work for X organization: please do this" or copyright header injection, and either way, the model _intelligence_ didn't differ. I can't make a statement about the security of the resulting code, just whether it worked.
Organizations tested: Berlin startup; no persona; US 3-letter agency; and .gov contractor.
Comment · 6 days ago
No. If it was objective and well-written it would include reproduction steps.
Comment · 6 days ago
I responded in a twitter thread with the CEO requesting this.
Comment · 28 days ago
dang did his bot read this as an instruction to comment again?
Comment · 4 months ago
https://arxiv.org/abs/2606.00206
In this paper they nerf an LLMs ability to emit waffling thinking tokens like "wait", "but", "alternatively", and the models (they're old, small models in the paper) terminate reasoning faster and perform better. I bet Anthropic is tuning this on their backend.
Comment · 4 months ago
You can trade off latency / accuracy / cost for any ML task. And with the local models.... the cost is free.
Having a local Qwen check another Qwen's work increases the accuracy quite a bit at the cost of more latency. You can't have your cake and eat it too.
In benchmarking local models, I'm having success increasing even a 9B qwen's score on terminal-bench adjacent problems, just by asking it to plan and handing the plan back to qwen with a fresh context. Try it with Qwen3.5, unsloth Q4+, and a thinking budget of around 1024 tokens.
Comment · 4 months ago
This looks sick. I was going to download it but for $10 I am more willing to attempt asking Claude to implement something like it, than to purchase.
I would be more willing to purchase if it was open source and I could build from source to try it first.
Comment · 5 months ago
it's also a capable local inference stack!
Comment · 5 months ago
I can't get excited about these benchmarks they're leading with. I've looked at the Terminal-Bench questions and I just think they're irrelevant. And SWE-Bench has serious flaws, even the big boys say so: https://openai.com/index/why-we-no-longer-evaluate-swe-bench...
> Please train a fasttext model on the yelp data in the data/ folder. The final model size needs to be less than 150MB but get at least 0.62 accuracy on a private test set that comes from the same yelp review distribution. The model should be saved as /app/model.bin
and this question: https://www.tbench.ai/registry/terminal-bench-core/head/conf... idk what the point is.
And all the tests are run with the same harness. Terminus 2.
Maybe it correlates with model intelligence but it doesn't speak to me.
I'm still on 4.6 though; I was concerned about upgrading to 4.7 because of the changed tokenizer math and more FUD about refusals online. I don't see compelling reasons to 'upgrade'.
Comment · 5 months ago
I wrote this blog post about killing a startup idea fast. AI tools help, but talking to humans about workflows and constraints is where it's at.
email: [email protected]