Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Exporting Json events to a text file
#1
Hi, I was given a Python script from a vendor to pull down events from their API. The script works but I need to save these events to a file and not just view them. I'm not that familiar with Python but have a general understanding of programming. When I run the code below, it generates my text file but it only saves the last event pulled from the API and oddly saves two entries for it in the text file. In the code, I see a while loop and for loop and I think my json.dump command is inside both loops and therefore creating two entries in the text file. I've tried moving it outside each loop but not dice.

Any ideas would be appreciated!

    while True:
        data = export_events(token, event_type, batch_size, start_date, end_date, client_id, client_secret)

        total_events = total_events + len(data)
        if data is None:
            break
        elif len(data) == 0:
            break
        else:
		        for event in data:
    			    with open('jsonfile.txt','w') as outfile:
				                   json.dump(data,outfile, indent=2, sort_keys=True)  
			        print json.dumps(data, indent=2, sort_keys=True)


Quote
#2
you open and close the output file for each event in 'w' mode. That means you overwrite what is already in the file. second you dump the event twice - both on lines 12 and 13 and that is why you have two times the same event.
You need to open the file at the beginning. Also note that (assuming all events come as one json) you don't need to loop. Just dump the whole data json at once.
Also note that you cannot add to a json file. i.e. if you want to add new downloaded events to previously downloaded events you need to read the file content in memory, add what you downloaded this time and then dump everything in the file.
Quote
#3
Thanks for the quick response. I don't believe the data comes all at once. The only lines I added to the code were the 'with open...' and 'json.dump...' lines. The others were already there so I think the while loop is to read from all the data and just print it to the screen. The script when running will display all the data but not save it.
Quote
#4
(Aug-21-2018, 02:08 PM)uklipse Wrote: he others were already there so I think the while loop is to read from all the data and just print it to the screen
That exactly means all events come as one json. They loop over events in order to print them on separate lines. Otherwise thej will print everything on one line.
try
while True:
    data = export_events(token, event_type, batch_size, start_date, end_date, client_id, client_secret)
    if data:
        with open('events.json','w') as outfile:
            json.dump(data,outfile, indent=2, sort_keys=True)
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Python Parameter inside Json file treated as String dhiliptcs 0 136 Dec-10-2019, 07:28 PM
Last Post: dhiliptcs
  Output to a json file problem Netcode 3 140 Nov-22-2019, 01:44 AM
Last Post: Skaperen
  Exporting list with dictionary to Excel veromi22 0 141 Oct-15-2019, 12:54 AM
Last Post: veromi22
  from Json Time Serie file to python dictionnary Reims 1 172 Sep-11-2019, 08:17 AM
Last Post: DeaD_EyE
  convert a json file to a python dictionnary of array Reims 2 194 Sep-10-2019, 01:08 PM
Last Post: Reims
  Exporting to Excel Nirmal 5 334 Aug-08-2019, 12:22 PM
Last Post: Nirmal
  Trying to make column based file from text file scor1pion 7 374 Jul-16-2019, 02:43 PM
Last Post: scor1pion
  Load JSON file data into mongodb using pymongo klllmmm 1 1,692 Jun-28-2019, 12:47 AM
Last Post: klllmmm
  How best to format text in a simple text file? Pedroski55 2 635 Jun-14-2019, 06:07 AM
Last Post: noisefloor
  Print a JSON file mrobie 5 521 May-31-2019, 09:38 PM
Last Post: Yoriz

Forum Jump:


Users browsing this thread: 1 Guest(s)