nicoty

Joined 8 years ago

Karma

Submissions

Member since

Apr 13

Recent submissions

I have a solution based on Nix that can be used to generate reproducible container images: https://github.com/nothingnesses/agent-images . It lets you customise which agents, harnesses, or any other packages you want included in the VM and it uses `agent-box` for sandboxing.

Comment · 3 months ago

I've been thinking a lot about this recently. In my case: how do I get an agent to see the important parts of the current plan and get it to stick to it without deviating, especially as loops get into longer and longer cycles and compactions erase prior context?

I think the problem with just encoding a whole plan in a single markdown file is that it gets polluted really quickly (agents can stop adhering to instructions to keep it clean and conform to a specific structure), which makes it harder for agents to see which parts of the plan they should focus on. As such, I've reached a similar conclusion about giving the agent access to CLI tools to help them deal with this. To try to mitigate this, I've been getting Claude to develop for me a CLI tool that:

1. Scaffolds reusable, structured plan templates and a reusable workflow that structures how to tackle the plan, step by step.

2. Validates that the plan files still conform to the correct, parseable structure.

3. Parses and evaluates the plan files, to determine what the current state is and what the next valid transitions and states are according to the workflow, like a state machine, and outputs instructions and reminders for agents as to what they should do at each step of the workflow.

So far, I've been dogfooding the tool and it seems promising: I can leave Claude running for longer and it doesn't drift as much. However I haven't ran any benchmarks yet and I'm still not entirely happy with the state of the codebase ( https://github.com/nothingnesses/agent-scaffold ), so take this with a grain of salt.

That said, I'm also bullish on using agents with formal methods and proofs. Type checkers and compile-time checking in general are great because they surface errors early and with great specificity. So if you can encode your specifications with, e.g. dependent types, you can use the type-checker as a way to steer the agent when it goes wrong and gets off-track.

Comment · 3 months ago

I've codified mine into a reusable workflow https://github.com/nothingnesses/agent-scaffold . To be honest, this isn't fool-proof though, since the agents can simply choose to ignore them, so I also like to pair this with deterministic linting and compile time checking.

For a Rust project, I created macros that output compiler errors when documentation and tests are not in a shape I want them to be, like missing function invocations or assertions, which forces the agent to address them, where otherwise they would've just worked around them by adding stupid trivial assertions like `assert_eq!(true, true)`.

That still isn't fool-proof either, but it helps minimise those instances. I'm bullish on the idea of integrating formal methods and model-checking with AI. I think that combo feels like a promising avenue for constraining the stochastic side of AI-generated code with something closer to deterministic verification. Provided you can write correct specs of course!

Comment · 4 months ago

I've contributed to https://github.com/0xferrous/agent-box which allows you to bind-mount git repositories into containers that agents operate in, preventing the agents from accessing files that aren't bind-mounted. Your usual .gitignore can then be used to also ignore files within the repo to be bind-mounted, which prevents agents from accessing them at all, essentially working as a sandbox.

I also maintain https://github.com/nothingnesses/agent-images which allows you to use Nix to reproducibly spin up OCI container images containing agents and any other tools you need for development and use these with agent-box.

I use both at the moment to work on some personal projects with agents, where I set up multiple separate git worktrees for the agents to work in, preventing them from accessing anything outside of the worktrees and from trampling over each other's work.

Comment · 4 months ago

I also maintain https://github.com/nothingnesses/agent-images which allows you to use Nix to reproducibly spin up OCI containers containing agents and any other tools you need and use these with agent-box.

Comment · 4 months ago

I don't now how the MacOS equivalent compares, but Linux/Windows has KDE Connect that I use for that https://kdeconnect.kde.org/

Comment · 5 months ago

Out of curiosity, what do you think is wrong with monomorphization-based polymorphism? The other alternatives I'm aware of are 1. type-erasure via v-table based dynamic dispatch (which Rust also has in the form of the `dyn` keyword), which has performance and memory-allocation overhead and 2. macros, which Rust also has and, if used for polymorphism, would essentially be like compile-time monomorphization, but clunkier.

Maybe I'm missing something though and there are other alternatives done differently in other languages?

Comment · 6 months ago

https://github.com/carp-lang/Carp might be of interest. It's a statically typed lisp.

Comment · 6 months ago

How does this compare to https://academy.fpblock.com/blog/quickcheck-hedgehog-validit... ? As far as I understand, Validity also has free generators and shrinking for types by having them implement various typeclasses that represent invariants and also has pre-made combinators to test properties with.

Comment · 7 months ago

How would they do that if they don't use git for version control? Does GitHub allow other forms of version control other than git?

Comment · 9 months ago

Filesystems like zfs, btrfs and bcachefs have snapshot creation and rollbacks as features.

Comment · 9 months ago

I know I'll sound like a rube but somehow it rubs me the wrong way that the rich and powerful are spending billions trying to establish multiplanetary civilisations despite the fact that we still have plenty of unfixed problems here at home that also deserve attention and resources, if not more so.

Comment · 11 months ago

Like how capsaicin makes food feel hot even when it isn't?

Comment · 1 year ago

Thanks for the spoilers

Comment · 1 year ago

I implemented an iterative, stack-based DFS iterator in JS last year for a project that I didn't end up using it on. Maybe someone else can find some use of it: https://gist.github.com/nothingnesses/5f974a43a2da5d1d8a6b9c...