Yes, but the compression process can be split into two components:
1. modelling (you convert data to symbols and predict probabilities of seeing each symbol)
2. entropy coding (convert symbols + probabilities to minimum number of bits)
The second stage is relatively simple, and basically solved. We went from Huffman's approximation to Arithmetic Coding which is theoretically ideal, and now got ANS which is close to ideal and pretty fast too.
The modelling part is indeed a completely open problem, and at this point as much art as science.
1. modelling (you convert data to symbols and predict probabilities of seeing each symbol)
2. entropy coding (convert symbols + probabilities to minimum number of bits)
The second stage is relatively simple, and basically solved. We went from Huffman's approximation to Arithmetic Coding which is theoretically ideal, and now got ANS which is close to ideal and pretty fast too.
The modelling part is indeed a completely open problem, and at this point as much art as science.