Running Windows is perfectly fine; the major libraries for GPU-accelerated autodiff and networks (CUDNN with Pytorch or Tensorflow) have great support nowadays. It's the AMD GPU that remains essentially useless, as of 2019. If you want to get into the game, I'd recommend buying a middle-of-the-road NVIDIA GPU like the RTX2060.
For toying with autodiff and basic CNNs, CPU works just fine by the way...
For toying with autodiff and basic CNNs, CPU works just fine by the way...