Putting reverse indexes in S3 alongside large aggregate log objects has been an effective pattern for more than a decade. With objects in cheap access modes and indexes in modes that are cheaper to access it becomes far more cost effective than most options (cheap S3 clones aside).
The earliest code I have to do this looks to have been created in 2012.
Separation of compute and storage is such a fabulous pattern.
The earliest code I have to do this looks to have been created in 2012.
Separation of compute and storage is such a fabulous pattern.