Efficiently handling large text files requires extra special care.
~2x memory looks like a naive implementation of just allocating an std::string from heap for each line. Due to heap fragmentation and various overhead it would quickly blow up.
~1x memory looks like just reading the entire file into RAM (that would still be slow).
A truly efficient implementation would never need to load the entire original file in RAM. It would just need to remember the binary offset of each line in a way that combines random access and reasonably fast insertion/deletion (e.g. K-fold trees). You can even keep everything beyond the top-level directory in an temporary on-disk file, so your RAM usage could be less than 1MB with nearly instant performance.
The efficient implementation is tricky, error-prone, and involves handling a solid amount of corner cases, which is beyond the amount of hassle a typical hobbyist developer is willing to go through.
~2x memory looks like a naive implementation of just allocating an std::string from heap for each line. Due to heap fragmentation and various overhead it would quickly blow up.
~1x memory looks like just reading the entire file into RAM (that would still be slow).
A truly efficient implementation would never need to load the entire original file in RAM. It would just need to remember the binary offset of each line in a way that combines random access and reasonably fast insertion/deletion (e.g. K-fold trees). You can even keep everything beyond the top-level directory in an temporary on-disk file, so your RAM usage could be less than 1MB with nearly instant performance.
The efficient implementation is tricky, error-prone, and involves handling a solid amount of corner cases, which is beyond the amount of hassle a typical hobbyist developer is willing to go through.