Took a quick look at the pages you provided.
The first page has a query form which can be partially filled and then searched on
I would use selenium for this both pages:
selenium can automate this process. I don't use scrappy, so can't say if it's capable or not.
I would suggest the following quick tutorials (on this forum)
Web-Scraping part-1
Web-scraping part-2
The first page has a query form which can be partially filled and then searched on
I would use selenium for this both pages:
- 1st url because of the query form, from which you need to at lease select the date, and select the Troop from map.
I noticed that the date range is limited (by pull-down) to last few days, you may be able to work around this.
- 2nd url because you must verify that you have read the conditions
once this has been done, you will be directed to the search page which contains a form that must be filled out.
selenium can automate this process. I don't use scrappy, so can't say if it's capable or not.
I would suggest the following quick tutorials (on this forum)
Web-Scraping part-1
Web-scraping part-2