I am curious if you tried something like langchain[1] it would solve the problem of not remembering previous conversations. In one of the examples people have used an entire company notion database on top of GPT-3 to answer questions specifically from that database.
PS: not tried myself.
[1] https://github.com/hwchase17/langchain