The scary thing about this is that many people have images that look like child porn at home: Family photos of naked grandchildren playing at the sea or in the mud, as well as sexting between consenting teenagers. There is no way automatic classification can distinguish these from real child porn, because they look almost identical and thus many private pictures will be shared with the government. This not only infringes the privacy of those people in the most drastic way possible, but might also lead to lawsuits that can tear families apart (think your father being accused of having CP only for it to turn out months later it was an old picture of yourself). The article mentions that 40% of CP investigations in Germany are started against minors (implied: through sexting), which shows that this concern is not purely theoretical at all.
This was not an issue with the old approach of comparing checksums against public CP images, but it will increasingly be if we use more AI algorithms for this.
EDIT: Probably Apples neural hashes that are currently in the news won't be vulnerable to this since they trained specifically to only detect changes in color/cropping/rotation. But we can't know for sure since the white paper is not that detailed and "naked kid on beach" photos all look really similar usually.
This was not an issue with the old approach of comparing checksums against public CP images, but it will increasingly be if we use more AI algorithms for this.
EDIT: Probably Apples neural hashes that are currently in the news won't be vulnerable to this since they trained specifically to only detect changes in color/cropping/rotation. But we can't know for sure since the white paper is not that detailed and "naked kid on beach" photos all look really similar usually.