May-06-2018, 06:35 AM
I wanted to set up a daily web-scraper for this site. The scraper does about 100 requests/hour every day, however the site seems to be blocking all bots. This is the url: 'https://www.nvsos.gov/' and it is protected by some company called "Distil Networks". I couldn't find much online about a way around, so I am asking here. If there is a way around this, any help is appreciated.