Python Forum
Exporting Json events to a text file
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Exporting Json events to a text file
#1
Hi, I was given a Python script from a vendor to pull down events from their API. The script works but I need to save these events to a file and not just view them. I'm not that familiar with Python but have a general understanding of programming. When I run the code below, it generates my text file but it only saves the last event pulled from the API and oddly saves two entries for it in the text file. In the code, I see a while loop and for loop and I think my json.dump command is inside both loops and therefore creating two entries in the text file. I've tried moving it outside each loop but not dice.

Any ideas would be appreciated!

    while True:
        data = export_events(token, event_type, batch_size, start_date, end_date, client_id, client_secret)

        total_events = total_events + len(data)
        if data is None:
            break
        elif len(data) == 0:
            break
        else:
		        for event in data:
    			    with open('jsonfile.txt','w') as outfile:
				                   json.dump(data,outfile, indent=2, sort_keys=True)  
			        print json.dumps(data, indent=2, sort_keys=True)
Reply
#2
you open and close the output file for each event in 'w' mode. That means you overwrite what is already in the file. second you dump the event twice - both on lines 12 and 13 and that is why you have two times the same event.
You need to open the file at the beginning. Also note that (assuming all events come as one json) you don't need to loop. Just dump the whole data json at once.
Also note that you cannot add to a json file. i.e. if you want to add new downloaded events to previously downloaded events you need to read the file content in memory, add what you downloaded this time and then dump everything in the file.
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#3
Thanks for the quick response. I don't believe the data comes all at once. The only lines I added to the code were the 'with open...' and 'json.dump...' lines. The others were already there so I think the while loop is to read from all the data and just print it to the screen. The script when running will display all the data but not save it.
Reply
#4
(Aug-21-2018, 02:08 PM)uklipse Wrote: he others were already there so I think the while loop is to read from all the data and just print it to the screen
That exactly means all events come as one json. They loop over events in order to print them on separate lines. Otherwise thej will print everything on one line.
try
while True:
    data = export_events(token, event_type, batch_size, start_date, end_date, client_id, client_secret)
    if data:
        with open('events.json','w') as outfile:
            json.dump(data,outfile, indent=2, sort_keys=True)
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  encrypt data in json file help jacksfrustration 1 33 2 hours ago
Last Post: deanhystad
  parse json field from csv file lebossejames 4 668 Nov-14-2023, 11:34 PM
Last Post: snippsat
  Python Script to convert Json to CSV file chvsnarayana 8 2,344 Apr-26-2023, 10:31 PM
Last Post: DeaD_EyE
  Loop through json file and reset values [SOLVED] AlphaInc 2 1,961 Apr-06-2023, 11:15 AM
Last Post: AlphaInc
  Converting a json file to a dataframe with rows and columns eyavuz21 13 4,168 Jan-29-2023, 03:59 PM
Last Post: eyavuz21
Thumbs Up Need to compare the Excel file name with a directory text file. veeran1991 1 1,064 Dec-15-2022, 04:32 PM
Last Post: Larz60+
  validate large json file with millions of records in batches herobpv 3 1,221 Dec-10-2022, 10:36 PM
Last Post: bowlofred
  Exporting Stock Fundamental Data to a CSV file with yahoo_fin DustinKlent 2 4,637 Aug-01-2022, 06:08 PM
Last Post: paulyan
  Writing to json file ebolisa 1 970 Jul-17-2022, 04:51 PM
Last Post: deanhystad
  Trying to parse only 3 key values from json file cubangt 8 3,338 Jul-16-2022, 02:05 PM
Last Post: deanhystad

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020