ActorNightly

Joined 7 years ago

Karma
2,289
Submissions
2,846
Member since
Apr 8
View on Hacker News

Recent submissions

Comment · 3 days ago
Be interesting to see the efficiency metrics on this compared to high aspect ratio planes.
Comment · 3 days ago
Unless you have a general interest in self sufficiency, spending time practicing this is going to be a waste of time given its risk of actually happening. Thats where the llm come in. The most likely scenario isn't some crazy invasion like in CoD games, but more likely temporary dissarray, and even that is fairly unlikely considering how many things are automated now.
Comment · 3 days ago
I mean, if you are in a scenario where its everyone for themselves, weapon knowledge is going to be absolutely essential.
Comment · 3 days ago
Qwen3.8-27b can be ran at 100 tok/sec average https://github.com/L-Forster/open-jet/tree/master/megakernel.

Gemma4:31b, which is the better model with more complex architecture (and thus can't be compiled into a megakernel), is gonna be way slower in prefill, and ~50% slower than a 3090 even with oMLX. You can't get away from much slower shared ram vs vram, and less parallelism compared to 3090.

The other advantage of an eGPU setup, you can simply add/swap performance. Dual 4070s give you 24b of ram and you can still get about 80% of the performance of a single 3090, and actually you get faster prefill due to more parallelism.

So in terms of money/usefulness, Mac always loses.

Comment · 3 days ago
> My guess is, up until the last five years that was almost never done.

It was done plenty. Just with extensions to python like numpy and opencv. There is no reason to translate everything versus just the critical pieces.

Comment · 3 days ago
>Qwen 3.8 27B Q4 with 64k, 3090: ~40tok/sec

Try more like 70-130 tok/sec. https://github.com/L-Forster/open-jet/tree/master/megakernel.

And also Qwen models famously chase benchmarks. They are good for writing code, but fall behind in other areas. Gemma4:31B is 6gb larger and does a much better job across all areas. https://www.reddit.com/r/LocalLLaMA/comments/1vyzopv/gemma4_...

Go ahead and run Gemma4 on your M1, and you will find that its unusable at 12 tok/sec.

And this is also not touching prefill, which is the biggest thing that you need fast, and Apple sucks in this regard compared to nvidia cards. With gfx cards, you can throw big codebases at the local models and it will happily crunch through them in very little time.

Sure, spending 1.5k on a Macbook enables you to run a local model, and if you really have a good reason for running local (like running heretic/abliterated models) for tasks you don't mind waiting for, it works. The difference is, spend 1k more and let you never have to pay for tokens, and you can run inference all day long without your computer thermal throttling, like your Macbook.

And I don't need to make assumptions when you say idiotic stuff like "We're getting close to local AI being really useful", like you have any idea what you are talking about. On par with being an Apple fan though.

Comment · 4 days ago
Man you Apple Fans really live under a rock.

Mini PC with Occulink, EGPU dock, 750w power supply and a refurbished 3090 is 2500. It can run Gemma4:31b at 1000+ tok/sec prefill, ~40 tok/sec generation (which is slow, but a fully connected model is way better for all tasks)

if you want more specialized partial activation models like Qwen, you can get 100+ tok/sec generation.

This beats anything that Apple has by a long shot. You don't need to run large models for usefulness. Large models can get things write the first time, but thats their only advantage. Running small models that are good enough like gemma4:31b fast allows you to do faster iterations.

And if you really want, you can also do multi GPU setups if you want slightly slower model loading with usb4/thunderbolt ports.

TLDR; local ai has already been useful for quite some time if you arent brainwashed into believing that Macs are the best for running local inference.

Comment · 4 days ago
I mean, I have had great success with prototyping stuff in python, then letting an llm translate it to C. My 3090 with Gemma4:31b does this very well with a few guidelines.

Dunno how much investment is needed into language optimization anymore.

Comment · 5 days ago
Nobody is giving up. We are just going to start on the path of being what Russia is now, while China takes over as the world superpower.
Comment · 5 days ago
Has nothing to do with oligarchy, has more to do with that SpaceX is providing valuable services, the disruption of which can be worse than punishing those in charge.
Comment · 5 days ago
Once you understand what attention actually is, its pretty easy to adapt it how you need it.

https://news.ycombinator.com/threads?id=ActorNightly&next=49...

TLDR its not so much about the transformer, as much as expanding dimentionality. You can chose any which way to do so.

Comment · 5 days ago
>All the clocks running out.

Hate to burst your bubble, but behind Trump there are quite a few people that are ready to take his place, with no real chance of voting ever being able to stop them. Don't get your hopes up that voting blue matters. Everyone had the chance in 2024, half the eligible population did not vote. Now we get to live with those consequences.

US is done for.

Comment · 5 days ago
While Im not going to dissuade people from voting, everyone should be prepared for the following reality: true and fair elections in US very like to be a thing of the past.

There are 3 big factors in play

* 2020 attempted coup gave the Republicans valuable information on what to do this time around. Basically, the game is to put US into legal limbo and remain in power. They know nobody on the democrat side is going to think about using armed forces, and they are not going to use armed forces to keep their public image and keep economy from panicking, so they are going to just declare some emergency and keep power under the pretense of fixing that emergency. Reminder that Supreme Court is on his side.

* Trump has shown that he has no issue with having a private army operating with extrajudicial authority in public. Now imagine with what he is ok doing with a non-public army. You should fully expect people going against him to get "arrested" under some made up charges.

* Even with Trump gone, there are smarter, younger, non senile conservatives ready to take his place and grab power and do the same things, with way less mistakes. Vance is right up there with him.

As such, even if everyone gets out and votes blue, in the end its all pointless. Trump can get impeached, convicted, and either Supreme Court says nah, or he just won't leave. In 2028, he (or Vance) will declare election fraud again and do it successfully this time to remain in power.

And even if somehow Dems win the house and presidency, and manage to pack supreme court to take control of all the branches, the thing is the will be left with a insanely shitty situation to clean up that will result in economic downturn, and the idiots in US will vote republican again in 2032.

Comment · 8 days ago
Nobody really paid that much for software before LLMs, most of the web was moving to a subscription model with API pricing anyways.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton