thulle
Joined 5 years ago
Joined 5 years ago
Sighted testing and different audio levels are other common sources of perceived vinyl superiority.
Many people enjoy the ritual of grabbing a vinyl and setting it up for playback, having physical access to coverart and maybe lyrics, which are entirely valid reasons to prefer vinyl, but that isn't audio quality.
> This approach enables Heretic to work completely automatically. Heretic finds high-quality abliteration parameters by co-minimizing the number of refusals and the KL divergence from the original model. This results in a decensored model that retains as much of the original model's intelligence as possible. Using Heretic does not require an understanding of transformer internals. In fact, anyone who knows how to run a command-line program can use Heretic to decensor language models.
Abliteration seems to be what Heretic does?
> Now these "abliterated" models all suffer from catastrophic breakage because they are not as simple anymore.
I'm not knowledgeable about each step in the process of making these abliterated models, but some more popular ones with steps after Heretic, seem to improve on the benchmarks tried of the base model:
arc/c arc/e boolq hswag obkqa piqa wino
Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF [instruct mode]
mxfp8 0.711,0.879,0.910,0.790,0.514,0.823,0.763
mxfp4 0.701,0.873,0.909,0.786,0.488,0.813,0.759
Qwen3.6-27B-Instruct: [base, non heretic]
mxfp8 0.647,0.803,0.910,0.773,0.450,0.806,0.742
https://huggingface.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-...I'm not seeing any similar benchmarks of the HauhauCS models, at least the ones I checked, so I assumed the opinion is based on your own trials, but then you argue in favour Heretic. Is the based on pre-Heretic abliteration techniques? Which might then not be appliable to this "Proprietary weight-level abliteration developed by the dealignai research team."?
A comment points out that they aren't randomized:
> The values used were chosen such that it takes a large number of bit flips to change from allowed to denied. Using random values doesn't really protect against this attack.
Nitter: I get all 8 posts in the thread in 18 requests, 207 kB, 169 kB transferred.
X: I only get the first post, 128 requests, 11.45 MB, 2.11 MB transferred.
Assuming they're not selling the 4090 at a loss and being generous that the whole cost is the chip cost, that makes for a $1600 chip. Scaled to the H100 we get $2140. That's a pretty minuscule fraction of a $35k card.
That doesn't say anything about the fraction of the cost on the finished product though? Those numbers just say it could be something like 2% vs 6% of the cost of the card.
Output generated in 101.74 seconds (0.98 tokens/s, 100 tokens, context 82, seed 532878022)
Output generated in 515.46 seconds (0.99 tokens/s, 511 tokens, context 27, seed 660997525)
Checking nvidia-smi it stalls at ~130W (out of ~470 W max) power usage, ~25% GPU usage and ~10% memory bandwidth usage. There's fairly much traffic on the pci-bus though, and the python process is stable at 100% usage of one core. GPU possibly limited by some thing handled in python? Pausing the GPU-accelerated video-decoding of a twitch stream it get a surprisingly large boost:
Output generated in 380.42 seconds (1.34 tokens/s, 511 tokens, context 26, seed 648992918)
They'd need 20 months to manage to notify users?
Holding off on 2.2 seems recommended, and if you're keeping critical data on OpenZFS it might be a good idea to give the issues a glance. The 2nd one might have the same underlying solution as an issue that has given me system freezes when closing in on ~90% pool usage using 2.1 on top of LUKS.
https://news.ycombinator.com/item?id=49552520