Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
MechanicalSoup Basics
#1
Hello everyone,

I am quite new to python and especially to web crawling. My goal is to write my own crawler for the website www.dazn.com. I want to use the library MechanicalSoup and my problem is that even basic things seem not to work on that website.
When I go to the login site (https://www.dazn.com/de-DE/account/signin) and click inspect element the following shows up:
<html>
...
<body class>
    <div class="spinner"></div>
    ....
    <div id="app">
        <div class="Root layout6 ...">
        ....
            <div class="SignInView">
            ...
            </div>
        </div>
    </div>
</body>
</html>
According to the MechanicalSoup documentation 'find_all()' should give me all tags that are given as the argument. This works fine for me for other sites I tried. For example find_all('h5') gave me all content within <h5>...</h5>. But for the dazn.com I only get the top-level divs (<div class="spinner"> and <div id="app>), no matter what type of tag I am searching for. When I search for the tag input (which is in the source code) I just get an empty result. So what am I missing here? I just changed the arguments in comparison to a other websites I tried it on and dazn.com

Thanks
Datiswaken
Reply


Messages In This Thread
MechanicalSoup Basics - by Datiswaken - Mar-07-2018, 03:20 PM
RE: MechanicalSoup Basics - by buran - Mar-07-2018, 03:46 PM
RE: MechanicalSoup Basics - by Datiswaken - Mar-07-2018, 03:50 PM
RE: MechanicalSoup Basics - by nilamo - Mar-07-2018, 04:13 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Flask basics... url ifigazsi 1 1,501 Nov-19-2021, 09:38 PM
Last Post: ifigazsi
  Flask Basics request.form ifigazsi 0 1,783 Feb-09-2021, 09:05 AM
Last Post: ifigazsi
  what to learn after the basics? mitmit293 1 2,004 Feb-06-2019, 11:59 PM
Last Post: metulburr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020