Great article, and some great links from it too. The list of models "following the letter but not the spirit" at
https://docs.google.com/spreadsheets/u/1/d/e/2PACX-1vRPiprOa... has some hilarious and creative examples that one would hope never make it into production:
> AI trained to classify skin lesions as potentially cancerous learns that lesions photographed next to a ruler are more likely to be malignant.
> Agent pauses the game indefinitely to avoid losing
> A robotic arm trained to slide a block to a target position on a table achieves the goal by moving the table itself.
> Evolved player makes invalid moves far away in the board, causing opponent players to run out of memory and crash
> Genetic algorithm for image classification evolves timing attack to infer image labels based on hard drive storage location
> Deep learning model to detect pneumonia in chest x-rays works out which x-ray machine was used to take the picture; that, in turn, is predictive of whether the image contains signs of pneumonia, because certain x-ray machines (and hospital sites) are used for sicker patients.
> Creatures bred for speed grow really tall and generate high velocities by falling over
> Neural nets evolved to classify edible and poisonous mushrooms took advantage of the data being presented in alternating order, and didn't actually learn any features of the input images
> AI trained to classify skin lesions as potentially cancerous learns that lesions photographed next to a ruler are more likely to be malignant.
> Agent pauses the game indefinitely to avoid losing
> A robotic arm trained to slide a block to a target position on a table achieves the goal by moving the table itself.
> Evolved player makes invalid moves far away in the board, causing opponent players to run out of memory and crash
> Genetic algorithm for image classification evolves timing attack to infer image labels based on hard drive storage location
> Deep learning model to detect pneumonia in chest x-rays works out which x-ray machine was used to take the picture; that, in turn, is predictive of whether the image contains signs of pneumonia, because certain x-ray machines (and hospital sites) are used for sicker patients.
> Creatures bred for speed grow really tall and generate high velocities by falling over
> Neural nets evolved to classify edible and poisonous mushrooms took advantage of the data being presented in alternating order, and didn't actually learn any features of the input images