I like the analogy to de-noising autoencoders; that's a good way of thinking about this.
> In the limit, you might even have a neural network generate increasingly harder types of grammatical corruptions, with the goal of "fooling" the corrector network.
Very interesting. I wonder how many constraints would need to be added to the corruptor model to ensure the corrupted sentence retains the same meaning as the original. Somewhat related to that, I've thought that a more basic curriculum learning setup could be deployed quite effectively here, and am hoping to try that out soon.
> In the limit, you might even have a neural network generate increasingly harder types of grammatical corruptions, with the goal of "fooling" the corrector network.
Very interesting. I wonder how many constraints would need to be added to the corruptor model to ensure the corrupted sentence retains the same meaning as the original. Somewhat related to that, I've thought that a more basic curriculum learning setup could be deployed quite effectively here, and am hoping to try that out soon.