For Research - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: General Coding Help (https://python-forum.io/forum-8.html) +--- Thread: For Research (/thread-24264.html) |
For Research - AgileAVS - Feb-06-2020 Hello to any one who is reading this, I am actually trying to scrape data from the internet regarding stock prices and number of shares traded to develop an algorithm for my academic research. I have already gotten the permission from the website to scrape the data, however, I am fairly new to Python and despite trying to run codes for two sleepless weeks, nothing seems to work. Besides, the other problems are that the website : https://www.sharesansar.com/today-share-price itself runs after scraping data from the main stock exchange website. And the other problem is that there is a use of multiple dates (calendar), so how do I get the data from the calendar. I only need the symbol of the company, closing price, volume and previous closing price from the table. I WOULD OWE YOU GUYS ALOT FOR YOUR HELP :). Besides, I did not post the codes that I written so far, because none of them seem to work, still here is one: import urllib.request, urllib.parse, urllib.error import re import requests import xml.etree.ElementTree as ET from bs4 import BeautifulSoup fhand = urllib.request.urlopen('https://www.sharesansar.com/today-share-price') html = fhand soup = BeautifulSoup(html, 'lxml') #for article in soup.find_all('table', class_='table table-condensed table-hover'): #data = article.tr #print(data.text) for article in soup.find_all('tr', class_='unique'): data=article.text #print(data) for td_tag1 in data: td_tag1=soup.find_all('td', class_='alnright').text td_tag2=soup.select('y:nth-child(0)') print(td_tag1) #print(headline.text) #td_tag1=soup.find_all('td', class_='alnright') #td_tag2=soup.select('td_tag1:nth-child(0)') #print(td_tag2) RE: For Research - Larz60+ - Feb-06-2020 Please fix your tags, I'd supply more info if I could read your post you need selenium to do this. see: web scraping part 1 web scraping part 2 Also, you can download complete data sets here: http://www.eoddata.com/ archive goes back 20 years on much of the data. |