testerius
Joined 2 months ago
Joined 2 months ago
I tried llama.cpp with offloading "all what could be" to my GPU (GTX 750 Ti 2 GB). I could run local Qwen2.5-Coder quite fast (I do not remember but something like 25-30 t/s). However this model generates bad code (I tried Python and JavaScript). So, for my setup it is better to chat with free ChatGPT and do copying-pasting between web chat and code editor.
Is it even worth for trying something on my GPU? What do you think?
And yes, I use Claude Code (Opus 5.5) and Codex in my job but for personal stuff I would prefer trying free solutions (it is the reason I tried local LLM). I do not want to buy subs, at leats for now.
I know and understand it. However I see a lot of people being sad and angry because of LLMs/AI agents. I would like to add to this website section about LLMs/AI. You can use them or not at all if you want. I guess nobody forces you to use "magic" AI in your hobby projects.
And ihmo programming, math, doing music or anything else still can be hobby - with our without AI. Do not be worry, you do not have to use AI for your hobby stuff. You can still code, create from scratch crappy personal frameworks/libs and so on.
I only want for times when local LLMs/AI are powerful and available on average hardware, so I could use it for my personal stuff without monthly payments. I wish doing so many things on my own.
Imagine: you are a boss of a company and own a powerful tool which can give you a lot of advantages. In short: it allows you to earn more money than competing companies. Would you share this tool to everybody, so that means competing companies could destroy your business or you keep the tool in secret and slowly build empire?
I do not know if OpenAI has powerful LLMs/AI things but I can be sure they do not share everything what they have. I think the only way they could share hidden "powerful tool" is situation where men in black visit CEO and talk with him/her about co-op. Of course CEO is "happy" because he/she can not refuse if you know what I mean. But still hidden powerful tool is hidden but this time another "more important guys" have access to it.