Mar-07-2020, 02:56 PM
Hello Everyone.
I am trying to scrape a website using python. I want to scrape text inside a span, which is inside a div. but the span is thrown automatically there, after every 5 minutes. so I want the script to run forever, and scrap every time the value of span, when it appears.
My code is working fine locally when I have the HTML file in my local machine. but when I try to scrape from the website, it is not working.
Where could be the error?
More details here. Details here
I am trying to scrape a website using python. I want to scrape text inside a span, which is inside a div. but the span is thrown automatically there, after every 5 minutes. so I want the script to run forever, and scrap every time the value of span, when it appears.
My code is working fine locally when I have the HTML file in my local machine. but when I try to scrape from the website, it is not working.
Where could be the error?
More details here. Details here
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
from urllib.request import urlopenfrom bs4 import BeautifulSoupfrom bs4 import BeautifulSoup as soup # HTML data structurefrom urllib.request import urlopen as uReq # Web clientimport reimport timeimport pyperclipwhile True: uClient = uReq(page_url) page_soup = soup(uClient.read(), "html.parser") div = page_soup.find('div',{'class':'content'}) if div: span = div.find('span') if not span: continue numbers = span.get_text() match = re.search('\d{5,}', numbers) card = match.group(0) print(card) time.sleep(2) |