Comment by Zylokloto

You can already run it locally its just not the same.

It is still slow, a lot slower than what you are used to with claude and co.

And as soon as you increase context size, your memory requirements jump.

Then when it runs for 30 minutes for something claude needs 5, your device will get hot.

And even a used 3090 is apparently now between 1-2k.