I was thinking the same thing, but is SSD to GPU faster than RAM to GPU? In many (not all) cases you buy a tonne of RAM and load your entire dataset into memory once and then iterate over it as necessary.
You also lose the flexibility of doing any sort of data modification or augmentation. One domain where your data usually doesn't fit in RAM is image recognition, but often you want to do things like apply random flips, crops and change hues before training to make the neural net less sensitive to those changes, which you can't really do with this.
You also lose the flexibility of doing any sort of data modification or augmentation. One domain where your data usually doesn't fit in RAM is image recognition, but often you want to do things like apply random flips, crops and change hues before training to make the neural net less sensitive to those changes, which you can't really do with this.