calpaterson

Git is supported in the spec, though the tool doesn't yet handle it (soon! git is nice as you get a log "for free" which helps agents understand more about the memories).

As for sqlite in git: yes you can gitignore it, you could use git-lfs, you could use an out of band database. It would be great if there were another format more amenable to git to store vectors in. I looked at csv closely, but I was worried about float formatting/representational bugs. There is a gap in the market for a format here.

Comment · 2 months ago

Thanks for pointing out recfiles. I was not aware of it and will investigate whether we can reuse anything. That said Markdown with YAML frontmatter is not exactly my own invention!

Comment · 2 months ago

I find that semantic search is substantially better than keyword search even for small corpuses. Being able to find "related" material that doesn't match the keyword is a big advance over traditional full text search.

Comment · 2 months ago

What I propose is only a very slightly more formal version of what you describe.

Just to start with: memoryfields are possible to use in a server/client system. That was a key aim and I do already use them over Amazon S3 (though not always).

I started, like you did, with a personal library of prompts. But the issue is that as your library of little pieces of prompts increases a) you get tired of constantly editing them yourself b) you have no easy way to export and share them with others c) it's frustrating that the agent doesn't "automatically" find your little bit of prompt on X even when clearly it is relevant - hence sem search.

I think a lot of people are still using the "personal library of bits of prompt" model. It is ok. But I wanted to propose an minimal, interchangeable standard for sharing them. So the idea of being an institution and having a shared memoryfield: that's something I want as well!

The spec, feedback greatly welcome: