Comment by ubertoop

> I just keep everything in memory in native data structures and flush to disk periodically;

Can you expand on what you mean by this? I am interpreting that to mean that you are actually writing to a file and reading from a file rather than using SQL?