marci

Joined 9 years ago

Karma

517

Submissions

347

Member since

Oct 10

View on Hacker News

Recent submissions

Comment · 7 days ago

I wish his llms included sanakirja* in the comparisons, which has some (human made) speed claims*.

* https://docs.rs/sanakirja-core

* https://pijul.org/posts/2021-02-06-rethinking-sanakirja#benc...

Comment · 1 month ago

After looking at freely available SVGs of pelicans and bicycles, I have a hard time imagining what they could be using to game this.

Comment · 2 months ago

delta.chat is email

https://delta.chat/en/help#can-i-use-a-classic-email-address...

Comment · 2 months ago

Then maybe what you want is email?

Sprinkled with a little bit of https://delta.chat ?

And potentially a dash of iroh https://news.ycombinator.com/item?id=48542480 https://www.iroh.computer/solutions/delta-chat ?

Comment · 2 months ago

Nothing to understand. Straight up hallucination. I could have sworn I read that they used a novel architecture where the model is dense but you could select specific layers or something at inference. reread the announcement: just said MoE. Corrected my brain's weights so thanks.

https://machinelearning.apple.com/research/introducing-third...

Comment · 2 months ago

Seems like what Apple's going for with afm3. Their latest model that will be embedded in macOS 27 is a quantized dense 20B that only select between 1 to 4B at inference, based on the prompt, not token by token. If only they could make a 100B or 400B dense that selects ~5 to 15B...

Comment · 3 months ago

https://xcancel.com/Alibaba_Qwen/status/2084100707423289643#...

Comment · 3 months ago

Makes me wonnder... how much compute/storage there's in all the satellites currently in LEO combined.

Comment · 4 months ago

This was a preview release. They haven't finish training. The Pro contains more knowledge but it probably takes longer training than flash for the smarts to kick in.

Comment · 4 months ago

  "That’s where EMO comes in.

  We show that EMO – a 1B-active, 14B-total-parameter (8-expert active, 128-expert total) MoE trained on 1 trillion tokens – supports selective expert use: for a given task or domain, we can use only a small subset of experts (just 12.5% of total experts) while retaining near full-model performance."

https://allenai.org/blog/emo

Comment · 4 months ago

But with Apple's AFM 3 architecture, we might end up with huge SOTA adjacent on devices with limited RAM.

They use a technique where you only load between 1B and 4B of a 20B dense model for an entire prompt run, not token by token like a MoE, and use mostly the low power ANE instead of GPU cores.

Now, imagine if/when they scale up to 100B or more? On a chip using 2W?

Comment · 4 months ago

Don't worry. Most people spend most of their compute time on a phone, where you're ability to filter ads is way more enshitified.

Comment · 5 months ago

I wonder where's the line between using a font and copyright/trademark infringement.

Comment · 5 months ago

"That’s where EMO comes in.

We show that EMO – a 1B-active, 14B-total-parameter (8-expert active, 128-expert total) MoE trained on 1 trillion tokens – supports selective expert use: for a given task or domain, we can use only a small subset of experts (just 12.5% of total experts) while retaining near full-model performance."

https://allenai.org/blog/emo

Comment · 5 months ago

Did they modify their post? I can't see who claimed that consumer hardware will be able to build most things?