Returning None value while parsing - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html) +--- Thread: Returning None value while parsing (/thread-27735.html) |
Returning None value while parsing - Calli - Jun-19-2020 Quote:This is the Html element which is present in the website that i want to parseThis is to see if i am getting the right element from the page but i don't think i am getting the right element from the page. from bs4 import BeautifulSoup import requests def stock(): request = requests.get("https://global.bittrex.com/home/markets") content = request.content soup = BeautifulSoup(content, "html.parser") print(soup.prettify()) if __name__ == '__main__': stock() Output is uploaded on pastebin https://pastebin.com/VvrLaBNR from bs4 import BeautifulSoup import requests request = requests.get("https://global.bittrex.com/home/markets") content = request.content soup = BeautifulSoup(content, "html.parser") green_element = soup.find("span",{"class":"color-green"}) red_element = soup.find("span",{"class":"color-red"}) print('Green: ',green_element) #print the total number of green coins like "Green : 125" print('Red: ',red_element) RE: Returning None value while parsing - Larz60+ - Jun-19-2020 I can't access the webpage, but I would suggest that you save request.content to a file and examine the html to make sure the tags you are looking for exist on the unprocessed page. My guess is that there is some JavaScript that needs to execute before the 'color' tags are visible. If this is the case, you will need selenium to execute the JavaScript for you before you scan the results with BeautifulSoup. Note: if you examine the page from your browser, the tags will be there, (if they exist at all), because the browser has already run the JavaScript. There's a tutorial on this forum that you can run through in a small amount of time, it will show you how to use selenium. web scraping part 1 web scraping part 2 RE: Returning None value while parsing - Calli - Jun-19-2020 I think your guess is as good as mine i am thinking the same thing that i need to execute some javascript before i can scan the website. Because right now i am only getting this https://pastebin.com/VvrLaBNR if you can't access pastebin here is the alternative https://throwbin.io/XXj6Bm6 |