1. Small read buffers. No reason to sequentially read and parse gigabytes only 4kb at a time.
2. parseDelimitedFrom created a new CodedInputStream on every message, which has its own internal buffer; that's why you don't see a buffered stream wrapper in the examples. Every iteration of the loop is allocating fresh 4kb byte[]s.
3. The nio protobuf code creates wrappers for the allocated ByteBuffer on every iteration of the loop.
But the real sin with the protobuf code is serializing the same city names over and over, reading parsing and hashing. Making a header with the city string mapped to an integer would dramatically shrink the file and speed up parsing. If that was done, your cost would essentially be the cost of decoding varints.
1. Small read buffers. No reason to sequentially read and parse gigabytes only 4kb at a time.
2. parseDelimitedFrom created a new CodedInputStream on every message, which has its own internal buffer; that's why you don't see a buffered stream wrapper in the examples. Every iteration of the loop is allocating fresh 4kb byte[]s.
3. The nio protobuf code creates wrappers for the allocated ByteBuffer on every iteration of the loop.
But the real sin with the protobuf code is serializing the same city names over and over, reading parsing and hashing. Making a header with the city string mapped to an integer would dramatically shrink the file and speed up parsing. If that was done, your cost would essentially be the cost of decoding varints.