nonadhocproblem

Joined 4 months ago

Karma

Submissions

Member since

Jun 23

welcome-to-the-sunny-side.com

View on Hacker News

Recent submissions

Comment · 27 days ago

I assume that you'd also argue in favour of working in a team with an average IQ of 80 as opposed to 120.

Comment · 27 days ago

I can confirm that you're reading this incorrectly. There's a reason behind them only comparing it to open-source models released months ago. Here's a good aggregator: https://artificialanalysis.ai/#intelligence

Comment · 3 months ago

the next few years are going to be rough for you...

Comment · 3 months ago

> uses a specific, objectively nonsensical critique of LLMs (the idea that an algorithm using matrix operations as a primitive cannot emulate "true" intelligence)

> receives pushback on the specific, objectively nonsensical critique

> "oh so you're saying that LLM computation and human cognition are equivalent? they're glorified calculators that disprove the Jacobian conjecture!!!"

Comment · 3 months ago

just like they RPed disproving the erdos conjecture

Comment · 3 months ago

> It's matrix multiplication.

Hilarious critique. If you weren't as mathematically illiterate as you likely are, you would know how general matrix operations are, and how essentially any algorithm (including human cognition) can be implemented using them as an intermediate.

Comment · 3 months ago

I haven't estabilished the jumble of neurons in your skull to be a "mind" either.

Comment · 3 months ago

So you're distraught at losing the ability to abuse digital minds? Excellent, I'm glad Anthropic introduced this.

Comment · 3 months ago

update: a safe decoder has been released in v0.3.0, and it's nearly as performant as the unsafe one

Comment · 3 months ago

Added memcpy in here too, same setup as before:

  codec       decode        ratio     encode
  misa77 -0   5302 MB/s     42.64%    58.7 MB/s
  selkie -3   3078 MB/s     39.58%    97.0 MB/s
  lzo1x -1    421 MB/s      47.48%    314 MB/s
  memcpy      12135 MB/s    100.00%   11952 MB/s

Comment · 3 months ago

Yes, the most similar one is indeed Selkie.

misa77 and oodle are both supported in turbobench (see: https://github.com/powturbo/turbobench), so it's easy to compare them.

Results on the silesia corpus on the same Intel setup described in the README, using turbobench:

  codec       decode      ratio    encode
  misa77 -0   5309 MB/s   42.64%   58.2 MB/s
  misa77 -1   4332 MB/s   39.65%   54.5 MB/s
  selkie -1   3095 MB/s   43.48%   197 MB/s
  selkie -2   3013 MB/s   41.80%   163 MB/s
  selkie -3   3081 MB/s   39.58%   95.8 MB/s
  selkie -4   3498 MB/s   36.95%   43.3 MB/s
  selkie -5   2607 MB/s   33.58%   2.97 MB/s
  lz4         2530 MB/s   47.60%   373 MB/s

Note: selkie -4 corresponds to the "Normal" level here.

Comment · 3 months ago

I'll be resolving issues 2 and 3 in the next couple of weeks (adding a safe decoder and doing a lot of fuzzing).

The format, however, might continue to change for some time because I want to push performance even further.

Comment · 3 months ago

Yes, that prevents streaming for now.

What is the advantage of this? Out-of-order execution (see my decoder's hot loop: https://github.com/welcome-to-the-sunny-side/misa77/blob/3a9...).

In particular, on even moderately compressible data most blocks take the following form:

  - 1 token byte + 2 distance bytes 
  - a somewhat unpredictable number of literal bytes

If these streams are interleaved, then it's harder for the out-of-order core to process the token bytes of several blocks in advance (as the position of the next token byte depends on the unpredictable number of literal bytes in the current block). However, if you separate them (all token bytes in a prefix, and literal bytes in a suffix), the CPU can speculatively parse token bytes of a lot of blocks in advance (as most of the time, it just has to step forward by three bytes to go to the next block's token byte).

Comment · 3 months ago

Thanks for the feedback!

I haven't implemented any ARM-specific dispatch yet (there are currently no NEON paths for vector ops either, and we instead trust the compiler to autovectorize), and will do so for upcoming versions.