Python Forum
Scrap a value from website - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/Forum-Python-Coding)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/Forum-Web-Scraping-Web-Development)
+--- Thread: Scrap a value from website (/Thread-Scrap-a-value-from-website)



Scrap a value from website - harsush - Aug-29-2019

Iam trying scrap trade date value -- below is what i have written, but not getting the value. Can someone help on this.

#!/usr/bin/env python
import urllib2, json
ua = { 'User-Agent' : 'Mozilla/5.0' }
request = urllib2.Request('http://www.cmegroup.com/CmeWS/mvc/TimeandSales/300/G/N8?timeSlot=16&entryDate=20180522&pageNumber=1&_=1527157308610', headers=ua)
response = urllib2.urlopen(request)
data = json.loads(response.read())
response.close()
print data['tradeDate']



RE: Scrap a value from website - snippsat - Aug-29-2019

Use Requests,and stop using Python 2.
import requests

url = 'https://www.cmegroup.com/CmeWS/mvc/TimeandSales/300/G/N8?timeSlot=16&entryDate=20180522&pageNumber=1&_=1527157308610'
response = requests.get(url).json()
Output:
>>> response {'entries': [], 'productDescription': 'Corn Futures Jul 2028 Globex', 'props': {'pageNumber': 1, 'pageSize': 100, 'pageTotal': 0}, 'tradeDate': '-'} >>> response['tradeDate'] '-'