Comment by HereBePandas

They explicitly address this in page 11 of the report. Basically perfect recall for up to 1M tokens; way better than GPT-4.