So there's some arguing over terminology, and then the main technical point seems to be that you can reverse-engineer a training dataset from the "virtual machine" built by training a neural network.
The decompilation process doesn't just use the neural network though, if I understand correctly it also uses logs from the final training epoch that include error and weight update data. Does this somehow smuggle the training dataset back into the VM? To me, if you're making a statement about the nature of existing ML systems, the statement "reconstruct the source dataset from the cryptic matrices that constitute the software executed by them" would imply that this is possible from trained networks alone.
The decompilation process doesn't just use the neural network though, if I understand correctly it also uses logs from the final training epoch that include error and weight update data. Does this somehow smuggle the training dataset back into the VM? To me, if you're making a statement about the nature of existing ML systems, the statement "reconstruct the source dataset from the cryptic matrices that constitute the software executed by them" would imply that this is possible from trained networks alone.