Apr-21-2020, 07:43 PM
I actually never use it.
Usually the only sites that I scrape are open enough that's it's not needed.
I have run across enough instances where others use it that it was in the back of my mind, therefore the post.
requests documentation here: https://requests.readthedocs.io/en/maste...entication
shows that it should be used as I posted.
I only capture the return (in my variable response) because I usually use status_code, but also other parts of the returned structure.
Usually the only sites that I scrape are open enough that's it's not needed.
I have run across enough instances where others use it that it was in the back of my mind, therefore the post.
requests documentation here: https://requests.readthedocs.io/en/maste...entication
shows that it should be used as I posted.
I only capture the return (in my variable response) because I usually use status_code, but also other parts of the returned structure.