I encountered this with trying to have LLMs populate facts about electoral campaigns. Like when a candidate drops out, when endorsements happen, but also if a candidate is un-endorsed or drops and rejoins. It also needed to handle if any of these facts were incorrect.
I settled on a knowledge graph in Postgres and downloading/storing the source documents so it could iterate on past results without more scraping or network calls.
This blog post helped me understand security analysis in this context! A lot of the important systems around malware analysis or large scale system security (the parts people really care about) clicked for me. So thanks for writing it.
Anyways I hope we can find some pattern to converge on with this wrt "facts management" since I feel this is currently something a lot of people and LLMs are struggling with. In practice current LLMs working with episodic memory feels similar to a grandparent with dementia scrawling things down in notebooks, crossing things out, and getting very confused.
I settled on a knowledge graph in Postgres and downloading/storing the source documents so it could iterate on past results without more scraping or network calls.
This blog post helped me understand security analysis in this context! A lot of the important systems around malware analysis or large scale system security (the parts people really care about) clicked for me. So thanks for writing it.
Anyways I hope we can find some pattern to converge on with this wrt "facts management" since I feel this is currently something a lot of people and LLMs are struggling with. In practice current LLMs working with episodic memory feels similar to a grandparent with dementia scrawling things down in notebooks, crossing things out, and getting very confused.