Aug-17-2020, 10:10 AM
(Aug-17-2020, 03:35 AM)DustinKlent Wrote: How do I go about going into the link to extract the stock ticker symbol?You can open link same way as did with first url.
Could now also make function for open url to avoid the repeat code.
import requests from bs4 import BeautifulSoup # Get info from main site url = 'https://mattrode.com/blog/robinhood-collections-list/' headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36'} response = requests.get(url, headers=headers) soup = BeautifulSoup(response.content, 'lxml') ol_tag = soup.find('ol') li_tag = ol_tag.find('li') link = li_tag.a.get('href') #print(link) response = requests.get(link, headers=headers) new_soup = BeautifulSoup(response.content, 'lxml')Test.
>>> new_soup.select_one('div.col-13 > section > div > table > tbody > tr:nth-child(1) > td:nth-child(2) > a') <a class="rh-hyperlink qD5a4psv-CV7GnWdHxLvn AaXTyP3x99eRIDW0ExfYP" href="/stocks/CPRX" rel=""><div><span>CPRX</span></div></a> >>> new_soup.select_one('div.col-13 > section > div > table > tbody > tr:nth-child(1) > td:nth-child(2) > a').text 'CPRX' >>> new_soup.select_one('div.col-13 > section > div > table > tbody > tr:nth-child(1) > td:nth-child(3) > a').text '$3.43' >>> new_soup.select_one('div.col-13 > section > div > table > tbody > tr:nth-child(1) > td:nth-child(4) > a').text '0.59%' >>> new_soup.select_one('div.col-13 > section > div > table > tbody > tr:nth-child(1) > td:nth-child(5) > a').text '359.02M'