Comment by codekisser

I develop AI girlfriends. I've struggled a lot with achieving natural-feeling recall. I've gone through a few iterations, but I'm currently experimenting with a knowledge graph/vector hybrid that uses an LLM to extract facts to build the graph. Both the performance and $ cost really hurt, but it truly does breathe life into AI characters. I've seen a lot of the commercial products using the latest and most expensive models in the cloud to extract facts. Instead, I fine-tuned a local model on a gpt4-generated dataset, and it works surprisingly well. It will miss some connections but in practice I don't think it will be too noticeable.