Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Returning None value while parsing
#1
Quote:This is the Html element which is present in the website that i want to parse
"""
<span data-bind="text: title">BTC Markets</span>

<td class="order-type">

<td class="order-type">
<span data-bind="text: displayChange(), css: {'color-green': change() > 0, 'color-red': change() < 0, '': change() == 0}" class="color-green">0.8</span>
<span data-bind="css: { 'icon-arrow': change() > 0, 'icon-arrow-down': change() < 0, '': change() == 0 }" class="icon-arrow"></span>


<span data-bind="text: displayChange(), css: {'color-green': change() > 0, 'color-red': change() < 0, '': change() == 0}" class="color-red">-7.5</span>

</td>
"""
I only need to check how many of the elements are in red-color and green-color
So first of all i parse the website into HTML
Than i call the span tag which i want to read (here i think i have done something wrong since it doesn't return anything except the value None)
We can identify the red color because it has a -7.5 and the green color has no -
My objective is to find out how many are in green and red and give me the total of red and a total of green
Please help your brother out Wall
This is to see if i am getting the right element from the page but i don't think i am getting the right element from the page.
from bs4 import BeautifulSoup
import requests

def stock():
    request = requests.get("https://global.bittrex.com/home/markets")
    content = request.content
    soup = BeautifulSoup(content, "html.parser")
    print(soup.prettify())
if __name__ == '__main__':
    stock()

Output is uploaded on pastebin
https://pastebin.com/VvrLaBNR
from bs4 import BeautifulSoup
import requests


request = requests.get("https://global.bittrex.com/home/markets")
content = request.content
soup = BeautifulSoup(content, "html.parser")
green_element = soup.find("span",{"class":"color-green"})
red_element = soup.find("span",{"class":"color-red"})

print('Green: ',green_element) #print the total number of green coins like "Green : 125"
print('Red: ',red_element)
Reply
#2
I can't access the webpage, but I would suggest that you save request.content to a file and examine the html to make sure the tags you are looking for exist on the unprocessed page.
My guess is that there is some JavaScript that needs to execute before the 'color' tags are visible.
If this is the case, you will need selenium to execute the JavaScript for you before you scan the results with BeautifulSoup.

Note: if you examine the page from your browser, the tags will be there, (if they exist at all), because the browser has already run the JavaScript.

There's a tutorial on this forum that you can run through in a small amount of time, it will show you how to use selenium.
web scraping part 1
web scraping part 2
Reply
#3
I think your guess is as good as mine i am thinking the same thing that i need to execute some javascript before i can scan the website. Because right now i am only getting this https://pastebin.com/VvrLaBNR
if you can't access pastebin here is the alternative https://throwbin.io/XXj6Bm6
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020