Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scrap a value from website
#1
Iam trying scrap trade date value -- below is what i have written, but not getting the value. Can someone help on this.

#!/usr/bin/env python
import urllib2, json
ua = { 'User-Agent' : 'Mozilla/5.0' }
request = urllib2.Request('http://www.cmegroup.com/CmeWS/mvc/TimeandSales/300/G/N8?timeSlot=16&entryDate=20180522&pageNumber=1&_=1527157308610', headers=ua)
response = urllib2.urlopen(request)
data = json.loads(response.read())
response.close()
print data['tradeDate']
Reply
#2
Use Requests,and stop using Python 2.
import requests

url = 'https://www.cmegroup.com/CmeWS/mvc/TimeandSales/300/G/N8?timeSlot=16&entryDate=20180522&pageNumber=1&_=1527157308610'
response = requests.get(url).json()
Output:
>>> response {'entries': [], 'productDescription': 'Corn Futures Jul 2028 Globex', 'props': {'pageNumber': 1, 'pageSize': 100, 'pageTotal': 0}, 'tradeDate': '-'} >>> response['tradeDate'] '-'
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Web scrap --Need help Lizardpython 4 954 Oct-01-2023, 11:37 AM
Last Post: Lizardpython
  I tried every way to scrap morningstar financials data without success so far sparkt 2 8,172 Oct-20-2020, 05:43 PM
Last Post: sparkt
  Web scrap multiple pages anilacem_302 3 3,784 Jul-01-2020, 07:50 PM
Last Post: mlieqo
  Need logic on how to scrap 100K URLs goodmind 2 2,571 Jun-29-2020, 09:53 AM
Last Post: goodmind
  Scrap a dynamic span hefaz 0 2,660 Mar-07-2020, 02:56 PM
Last Post: hefaz
  scrap by defining 3 functions zarize 0 1,834 Feb-18-2020, 03:55 PM
Last Post: zarize
  Skipping anti-scrap zarize 0 1,854 Jan-17-2020, 11:51 AM
Last Post: zarize
  Cannot get selenium to scrap past the first two pages newbie_programmer 0 4,134 Dec-12-2019, 06:19 AM
Last Post: newbie_programmer
  Scrap data from not standarized page? zarize 4 3,244 Nov-25-2019, 10:25 AM
Last Post: zarize
  page impossible to scrap? :O zarize 2 3,883 Oct-03-2019, 02:44 PM
Last Post: zarize

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020