Greenpants

Reminds me of letting an LLM generate a strong password in the first place. Very practical, very quick, no terminal commands needed! I'm sure some have done exactly this. But then you realise your password is always shared with your LLM provider, and it's not so random after all (unless it used tools).

Alternatively, what I tend to do after receiving generated code is a lot of asking "why?".

I've learned things I wouldn't otherwise have learned because I hadn't considered using the tools the LLM recommends. It's also a way to eliminate some hallucinating, given that critical questions are posed as unbiased as possible. For that, I also like to open a new chat with a different model and asking open-ended questions about a recommended tool I don't know much about, to double-check that the original LLM was likely correct in its recommendation in the first place.

Agent-related files should've been dot-files at least to minimize human error. In fact, I'd been discussing: shouldn't agent instructions exist on a per-developer basis, and thus be an IDE setting for each individual, removed from the repository itself? Agent instructions don't add content to the source code, after all. It's like additional READMEs, but we already have a README.

The way I see it, it's the incessant stimuli you get through these apps. There's just no point where the screen stays quiet, stimulus free. Moving elements capture our attention naturally, especially when the entire screen keeps moving. The endless scroll element just makes sure that the stimulus keeps getting renewed the moment you're done with it.

Instead, a scroll should give you a break before heading into the next video. I'm willing to bet this would help severely with addiction. People are then forced to reconsider whether they actually want to play the next video. "Done" should not always lead to "here's the next stimulus". That's what's addictive. The brain isn't made to break out of that loop easily.

And those installed programs could have vulnerabilities that just a non-root user account could still take advantage of. Perhaps not likely for an LLM to do, but more so if you let them loose on the internet and they end up coming across prompt injection that instructs exactly that :)

Though I'm in the camp "people should really know to sandbox by now and be careful", I'd say we should also be mindful of how far from everyone has deep knowledge of the systems and tools they use. This behaviour of a tool is just malicious. You have to take into account the human factor, of how people likely end up using a system. And in this case, the consequences of exfiltrating so many secrets this way are really quite unacceptable.

I specifically chose a Mac Studio 128GB as my home server that's also running LLMs to be always online, in part due to the minimal idle power consumption and mostly fan-less operation. It's definitely expensive, especially nowadays, but I can still recommend Mac Minis as a cheaper alternative for someone to just get started with an affordable, always-on home server that won't annoy any housemates. I think both are in some sweet spot in terms of value for money, depending on what you're looking for in a home server. If image or video generation is your thing, look further though, definitely look into a proper GPU then. Macs are quite slow at that. They're just great at MoE LLMs because it's mostly a matter of (V)RAM size.

Up to the point where consumers notice and decide to stop using these models because of it.

Might be why we're already rarely seeing models output an "I don't know".

That's precisely why my agent use is IDE-agnostic: I run Pi in any terminal. Often use it with the terminal inside VSCodium, though sometimes in a terminal outside an IDE if I don't expect to edit any files myself (e.g. for small one-shot projects).

I'm afraid I'd have to agree. That is, unless you have 512GB+ RAM sitting on a shelf and run the much larger SOTA-comparable local models.

I'm glad you're asking. I already started writing a blog post on how to best make use of local models. I'll share it as soon as I have a complete enough list. If anyone else reading this would like to chime in with their tips & tricks, let us know!

For the time being, off the top of my head, I'd say:

- Prompt Engineering tips & tricks apply here (like being complete in the relevant context you provide in your question, and the specific task(s) the agent should do like reasoning, modifying one file, or trying to fix a complex task all at once (not recommended)).

- If you already know which files the agent should look into, mention them to save time and potentially context.

- In my personal workflow, I write down lots of atomic TODOs needed to solve a problem. As I write it down, I'll notice assumptions I'm making, or the fact that the TODO could still be decomposed further into (atomic) subtasks.

- It's best to get a feeling yourself for how Qwen handles your repository. I noticed if I don't specify an architecture for development, it'll make quick & dirty fixes. If I don't tell it to remove debug statements, it won't. This is what was meant with "be precise" – Claude Opus might think for you and act in your best interest. Smaller Qwen models will just do what you ask them to, and no more. They have design knowledge, but you have to explicitly ask them to "activate" that part of their knowledge.

I get that it's a deal breaker to some; it definitely requires patience.

In Pi, /new is my best friend and most-used command for sure. For simple tasks (I decompose complex ones anyway since I don't trust small local LLMs to do this for me), the model doesn't need much context, given that I'm proficient in my codebase myself: "I'd like Feature X. Look into files 1, 2 and 3 to make your edits."

The other upside of running local LLMs is that there's no cloud provider to suddenly charge more for the same, or even less, model use.

It's personal, but I prefer CapEx over OpEx for this. If you can purchase a device upfront that runs a decent local LLM, you get the peace of mind that your setup won't suddenly change over time and can only get better.

Some of the work I do, I do for an (EU) organisation that doesn't have clear rules or guidelines on the use of AI yet. Though I have seen colleague-developers blatantly putting source code into external Claude-like models, I stay true to my principles and don't. I know for certain that everything that I run through my local, offline Pi Container Sandbox cannot leave the machine, and thus can't result in a data breach. I do this for the peace of mind.

I do (unscientifically) experiment whenever a new capable local LLM (<=130b) releases with a license that permits commercial use. As for knowing my models require more work than Opus, I don't mind still having to puzzle on getting the architecture right. In any case, it forces me to stay in the loop of what's being built, which is a good thing.