trucks-refinish

I would give it significantly more context, if you don't have enough room you can quantize the kv cache with some flags that I don't remember ottomh.

I would go ahead and give it at least 64k but the more the better really!

I run the kv cache at q8 for keys and q4 for values, but going below q8 is generally not recommended.

yes thank you!

rsync unfortunately doesn't do relinking at all, so I can't use it as a generic replacement for cp.

In rust you cannot mutate anything that is being read basically. The borrow checker enforces this.

So an iterator takes an immutable reference to the vector and mutation requires a mut ref, and you can't have both at the same time.

I'm able to run qwen 3.6 35b a3b with my rtx 2060 super which also has 8gb of vram. It's a very capable model, not as smart as a frontier model but it is still very useful.

I use an 6bit quant and get around 20 tokens per second.

yeah and I'm a Navy seal with over 300 confirmed kills etc.

git-shell is actually made for this exact situation. it does allow uploading git repos I think so you can't open it to the public, maybe there's a way to configure it though to not allow that.

For learning? You can use llms to help you with stuff and still learn new things in the process. Its really surprising that so many people don't understand this.

Are people really out there just mass copy pasting llm code without even trying to understand it! lol

Yes I highly suggest working in a container or setting up selinux. Selinux is especially powerful for this kind of thing but also requires a lot of advanced knowledge and skills to setup and maintain sadly.

yikes that in a minimal configuration costs ~700 euros or whatever the currency is by default.

You could get a much more powerful system for a lot less.