In uni I built a simple web scraper in JavaScript. It just ran in the browser. It would fetch a page, extract the links, then fetch those pages.
You could watch it run in realtime and it would build out a tree of nested links as it crawled. It was a lot of fun to watch! (More fun than CLI based crawlers for sure.)
The only issue I had was not being able to fetch pages from the front end due to CORS, so I just added a "proxy" to my server in 1 line of PHP. There, now it's secure? ;)
You could watch it run in realtime and it would build out a tree of nested links as it crawled. It was a lot of fun to watch! (More fun than CLI based crawlers for sure.)
The only issue I had was not being able to fetch pages from the front end due to CORS, so I just added a "proxy" to my server in 1 line of PHP. There, now it's secure? ;)