Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Link implementation
#5
Took a quick look at the pages you provided.
The first page has a query form which can be partially filled and then searched on

I would use selenium for this both pages:
  1. 1st url because of the query form, from which you need to at lease select the date, and select the Troop from map.
    I noticed that the date range is limited (by pull-down) to last few days, you may be able to work around this.

  2. 2nd url because you must verify that you have read the conditions
    once this has been done, you will be directed to the search page which contains a form that must be filled out.

selenium can automate this process. I don't use scrappy, so can't say if it's capable or not.

I would suggest the following quick tutorials (on this forum)

Web-Scraping part-1
Web-scraping part-2
Reply


Messages In This Thread
Link implementation - by JonWayn - Aug-30-2022, 03:38 PM
RE: Link implementation - by Larz60+ - Aug-30-2022, 07:49 PM
RE: Link implementation - by JonWayn - Aug-30-2022, 09:28 PM
RE: Link implementation - by Larz60+ - Aug-31-2022, 05:50 AM
RE: Link implementation - by Larz60+ - Aug-31-2022, 12:17 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  get link and link text from table metulburr 5 6,472 Jun-13-2019, 07:50 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020