Combined with HFT CppCon talk that emphasizes the impact of node-based STL containers you may benefit from using some optimized hash maps on top of vectors, if you aim at top-performance in runtime. Also, abseil has some flat sets/maps implementation. All of this could be a nice addition if you experiment with performance (I figured that from std::pmr use in your project).
I also recall the discussion here related to fast associative containers, like unordered map/set. This is the implementation - https://github.com/martinus/unordered_dense
Combined with HFT CppCon talk that emphasizes the impact of node-based STL containers you may benefit from using some optimized hash maps on top of vectors, if you aim at top-performance in runtime. Also, abseil has some flat sets/maps implementation. All of this could be a nice addition if you experiment with performance (I figured that from std::pmr use in your project).