Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Exporting Json events to a text file
#1
Hi, I was given a Python script from a vendor to pull down events from their API. The script works but I need to save these events to a file and not just view them. I'm not that familiar with Python but have a general understanding of programming. When I run the code below, it generates my text file but it only saves the last event pulled from the API and oddly saves two entries for it in the text file. In the code, I see a while loop and for loop and I think my json.dump command is inside both loops and therefore creating two entries in the text file. I've tried moving it outside each loop but not dice.

Any ideas would be appreciated!

    while True:
        data = export_events(token, event_type, batch_size, start_date, end_date, client_id, client_secret)

        total_events = total_events + len(data)
        if data is None:
            break
        elif len(data) == 0:
            break
        else:
		        for event in data:
    			    with open('jsonfile.txt','w') as outfile:
				                   json.dump(data,outfile, indent=2, sort_keys=True)  
			        print json.dumps(data, indent=2, sort_keys=True)


Quote
#2
you open and close the output file for each event in 'w' mode. That means you overwrite what is already in the file. second you dump the event twice - both on lines 12 and 13 and that is why you have two times the same event.
You need to open the file at the beginning. Also note that (assuming all events come as one json) you don't need to loop. Just dump the whole data json at once.
Also note that you cannot add to a json file. i.e. if you want to add new downloaded events to previously downloaded events you need to read the file content in memory, add what you downloaded this time and then dump everything in the file.
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Quote
#3
Thanks for the quick response. I don't believe the data comes all at once. The only lines I added to the code were the 'with open...' and 'json.dump...' lines. The others were already there so I think the while loop is to read from all the data and just print it to the screen. The script when running will display all the data but not save it.
Quote
#4
(Aug-21-2018, 02:08 PM)uklipse Wrote: he others were already there so I think the while loop is to read from all the data and just print it to the screen
That exactly means all events come as one json. They loop over events in order to print them on separate lines. Otherwise thej will print everything on one line.
try
while True:
    data = export_events(token, event_type, batch_size, start_date, end_date, client_id, client_secret)
    if data:
        with open('events.json','w') as outfile:
            json.dump(data,outfile, indent=2, sort_keys=True)
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Exporting Stock Fundamental Data to a CSV file with yahoo_fin DustinKlent 0 146 Sep-13-2020, 12:01 AM
Last Post: DustinKlent
  saving data from text file to CSV file in python having delimiter as space K11 1 138 Sep-11-2020, 06:28 AM
Last Post: bowlofred
  How to convert what appears to be a JSON file to CSV NewBeie 4 222 Aug-28-2020, 04:45 PM
Last Post: Larz60+
  live updated json file Nyanpasu 2 255 Aug-22-2020, 04:41 PM
Last Post: Nyanpasu
  Web Form to Python Script to Text File to zip file to web wfsteadman 1 292 Aug-09-2020, 02:12 PM
Last Post: snippsat
  Python code for exporting table using Selenium gj31980 4 321 Aug-04-2020, 01:29 AM
Last Post: gj31980
  How to open MIDI-file and get events in a list? philipbergwerf 2 331 Aug-01-2020, 10:58 AM
Last Post: philipbergwerf
  Convert Excel file to Text file marvel_plato 6 429 Jul-17-2020, 01:45 PM
Last Post: marvel_plato
  empty json file error mcmxl22 1 359 Jun-17-2020, 10:20 AM
Last Post: buran
  saving a dictionary as json file vinay_py 6 317 Jun-06-2020, 05:07 PM
Last Post: vinay_py

Forum Jump:


Users browsing this thread: 1 Guest(s)