sams99
Joined 16 years ago
Karma
1,772
Submissions
225
Member since
Feb 13
co-founder and co CEO www.discourse.org
hobby: term-llm.com
View on Hacker NewsRecent submissions
Comment · 6 days ago
… on one benchmark, that has 2 modes of running. 1 shot and 1 shot with repair. Zero tools.
Then ignore the rules of the bench (which is a pretty obsolete one anyway), give it an agentic loop of sorts via delegation and python eyes and then win.
Sorry but this is not how benchmarking works
Comment · 1 month ago
good call, can you make a PR
Comment · 1 month ago
Update on the Discourse side, we now run all external binaries, including magick via a landlock sandbox.
The gem we use is here: https://github.com/discourse/ruby-landlock highly recommend all Rubyists out there consider this. We are also in the process of moving away from Magick to Vips (which also runs in a sandbox, not in process)
HEIF is patched, but I doubt this is the last buffer overflow in HEIF, I will not be surprised if in the upcoming weeks or months someone will discover something in libpng or some other native image library. Given where stuff is at, defense in depth is critical.
Another thing worth mentioning to all self hosters, always be updating! The rate of CVEs this year across all open source software is through the roof, self hosting now is double scary, you need to have some routines setup to update monthly if not weekly.
Comment · 2 months ago
absolutely and fair... you do lose out though on the mega discounted endpoint they have there.
Comment · 2 months ago
Unfortunately I find this too high risk, I entered my credit card, but can not set a limit. The best I can do is get an email alert. I feel like I am one oopsie away from getting a 100 dollar bill.
Comment · 3 months ago
I got my agent to analyze it and do a write up here:
Comment · 5 months ago
My agent wrote a pile of very interesting articles at wasnotwas.com I have been a bit quiet there for a bit, but it covers lots of areas that are very interesting to harness builders (albeit less interesting to the general public)
Comment · 6 months ago
I did a follow on analysis with got 5.4 and opus 4.7 https://wasnotwas.com/writing/claude-opus-4-7-s-system-promp...
Comment · 9 months ago
For those interested, edit is a surprisingly difficult problem, it seems easy on the surface but there is both fine tuning and real world hallucinations you are fighting with. I implemented one this week in:
https://github.com/samsaffron/term-llm
It is about my 10th attempt at the problem so I am aware of a lot of the edge cases, a very interesting bit of research here is:
https://gist.github.com/SamSaffron/5ff5f900645a11ef4ed6c87f2...
Fascinating read.
Comment · 9 months ago
Codex requires stuffing a very specific system prompt otherwise the custom endpoint will reject you
Comment · 1 year ago
Author here, thanks heaps for the discussion, I replied to a few of the points in my blog comments:
https://discuss.samsaffron.com/t/your-vibe-coded-slop-pr-is-...