Large JSON file

[mr_byte31]

Hi All,

I build a program to read a JSON file from internet.
When the file is large , the python program hangs and I have to shut it down then run it again and it hangs again.
The programs works well with small JSON files.

import urllib.request, json

myURL = "https://query1.finance.yahoo.com/v8/finance/chart/AAPL?symbol=INTC&period1=0&period2=9999999999&interval=1d"
with urllib.request.urlopen(myURL) as url:
    data = json.loads(url.read().decode())
    print(data)

any idea how to overcome this issue?

--Update.
I found that the program run file with the print(data)
it seems printing is causing a problem that I don't know !

[DeaD_EyE]

I guess you had some network issues. Try it again.

Later you may want to save the data.
Requesting data over and over again when testing, takes too much time.
Some providers limit their APIs with x requests per minute.

from urllib.request import urlopen
import json
import sys
import pickle

myURL = "https://query1.finance.yahoo.com/v8/finance/chart/AAPL?symbol=INTC&period1=0&period2=9999999999&interval=1d"
timeout = 5
try:
    with urlopen(myURL, timeout=timeout) as request:
        data = request.read().decode()
except Exception as e:
    print('Unable to request data from internet')
    sys.exit(1)

try:
    data = json.loads(data)
except Exception as e:
    print("Can't parse json string")
    sys.exit(1)
else:
    print('Finished loading json')


with open('finance.yahoo.com.pickle', 'wb') as fd:
    pickle.dump(data, fd)

with open('finance.yahoo.com.json', 'w') as fd:
    json.dump(data, fd)

On my side it works. I get the data and the data can parsed with json.loads.

[mr_byte31]

I think the problem is in printing data !

I modified your code a little bit

else:
    print('Finished loading json') 
    print(data)

The program freeze after I added print(data)