When I used to build these scrapers for people, I would usually pretend to be a browser. This normally meant changing the UA and making the headers look like a read browser. Obviously more advanced techniques of bot detection technique would fail.
Failing that I would use Chrome / Phantom JS or similar to browse the page in a real headless browser.
Failing that I would use Chrome / Phantom JS or similar to browse the page in a real headless browser.