Python Forum

Full Version: Need advice how to scrape a Chinese webpage
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
I need to scrape a Chinese webpage on a website that needs logging in + verification by mobile.

I have a username and password. I can login and verify the login by SMS code that is sent.
It's OK, if the code is semi-automated - just not sure how I can get the login or session token passed to the Chinese website?

The page has images that only load after you scroll down - you need to scroll down progressively to get the entire content.

Any advice on where I need to start?

Thanks.
Anyone?
I was thinking, maybe it's possible for Python to display the login page and allow the user to enter username and password + the SMS notification confirmation?
Look at Selenium for this.