A month or so ago a paper about a deep learning model using a cpu beating the best (4000$ or so) nvidia gpu was on hn, the trick was a quick hashing technique similar to nearest neighbour, so the algorithm keep only the most active neurons. I think the gpu price was about 4000$ and the computer was 60% of that based on some ball park estimates from the comments in HN.
Edited: This is the paper:https://news.ycombinator.com/item?id=22502131
Perhaps both techniques can be used together, since the paper take advantage of sparcity to beat gpu when computation are on a sparse subset.