Aug-30-2022, 03:38 PM
Total newbie here. For starters in my scraping coding, I am faced with 2 websites that implement their links differently. In one case, I can return all the links in a table element and even close the window and be just fine because the href of each link will take me to its destination anyhow, but with the other website, those links will only take me back to the disclaimer page. What are the steps for involved in circumventing this behavior if any, or best practices in scraping sites like this? I am using scrapy and selenium