nilsherzig
Joined 4 years ago
Joined 4 years ago
Also, even before reading the comments, I was pleasantly surprised that everything on that page worked and nothing bothered me. There is a good amount of attention to detail (switching of the rounding on codeblocks when switching to light mode for example, avoiding a weird corner on mobile). Nothing moves around on load etc. I know this is a low bar, but it's gotten pretty rare, even for pages linked on here.
I'm trying to say that his 100k lines might actually be spent on attention to detail and aren't just slop.
PTQ1_0 has no optimized MMQ-Path in their llama-cpp fork, try running PTQ2_0 (needs a bit more vram, but is about 2x faster on my 6700 XT)
https://gist.github.com/nilsherzig/b8266d001c5c01bdb3d81d209...
Here is a detailed explanation from Pangram how their scoring works https://www.pangram.com/research/how-it-works
as an extension so you can continue to comment and use hn as usual
Im trying to get something cheaper to work, Pangram has some nice docs on how to build something like their service https://github.com/pangramlabs/EditLens, they even have training data online.
The harm a missclassification carries is much lower for a hn post than a masters thesis, so we might be fine with a worse model.
OPs solution does more than keyword filtering. But its still about filtering out "content about ai" not filtering out "ai written content".
Instead of filtering keywords this ranks the content using Pangram. But it's also pretty effective at filtering content about AI. Turns out that a lot of LLM tooling projects have LLM written project descriptions/blog posts.
It's amazing to see how many of the "AI hurts my brain" posts are apparently written by an LLM.