Python Forum
Code review for S3 object creation
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Code review for S3 object creation
#1
Hello!

I have developed a python3.6 program to create a csv object in S3 from a JSON payload. As this is my first python program, it would really help me if I someone could help me with the review.

Thank you.

           # Main module. Contains the event handler to be triggered from LAMBDA
           
           import json
           import boto3 # Needed to work with Amazon S3
           import botocore
           
           import re # regular expression module
           
           MYS3BUCKET = 'xyzbucket'
           FILE_ENCODING = 'utf-8'           
           CONN_S3 = boto3.resource('s3')
           		   
           # Event Handler function triggered from Lambda
           def lambda_handler(event,content):
               try:
                 # create an object for source
                 Mysource = Source(event)
           
                 # convert the source object into csv
                 csv_record = Mysource.create_csvrecord()
           
                 # store in s3
                 message = Mysource.write_to_s3(MYS3BUCKET,csv_record)
               except:
                 raise
               else:
                 return message
           
           class Source:
               def __init__(self,event):
                   self.__event = event
           
               def set_eventdata(self,event):
                   self.__event = event
           
               def get_eventdata(self,event):
                   return self.__event
           
               # function to convert dictionay input to csv
               def create_csvrecord(self):
                   try:
                       count = 0
                       csvrecord = ''
           
                       # extract the source object from JSON event
                       json_data = json.loads(json.dumps(self.__event['data']['object']['source']))
                       for key in json_data.keys():
                           if count != len(json_data) - 1:
                               csvrecord = csvrecord + re.sub(r'^None$','',str(json_data[key])) + ',' # replace None with null values
                           else:
                               csvrecord = csvrecord + re.sub(r'^None$','',str(json_data[key])) # do not append comma at end of record
           
                           count +=1
                   except:
                       raise
                   else:
                       return csvrecord
           
               # function to store the data on Amazon S3
               def write_to_s3(self,bucketname,data):
                   try:
                       # create bucket
                       self.create_bucket(bucketname)
           
                       objectname = json.dumps(self.__event['id']).strip('\"') + '.csv'
           
                       s3object = CONN_S3.Object(bucketname,objectname)
                       s3object.put(Body=bytes(data,FILE_ENCODING))
                   except:
                       print ('Exception occurred inside write_to_s3 function!')
                       raise
                   else:
                       return 'Successfully stored on S3.'
           
               # function to create a S3 bucket if not exists
               def create_bucket(self,bucketname):
                   try:
                       exists = True
                       
                       try:
                           CONN_S3.meta.client.head_bucket(Bucket=bucketname)
                       except botocore.exceptions.ClientError as e:
                           error_code = int(e.response['Error']['Code'])
                           if error_code == 404:
                               exists = False
                       
                       if not exists:
                           CONN_S3.create_bucket(Bucket=bucketname)
                   except:
                       print ('Exception occurred inside create_bucket function!')
                       raise
                   else:
                       return 
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Python best library for Excel reports & review of existing code MasterOfDestr 4 496 Feb-14-2024, 03:39 PM
Last Post: MasterOfDestr
  Review my code: convert a HTTP date header to a datetime object stevendaprano 1 1,903 Dec-17-2022, 12:24 AM
Last Post: snippsat
  dynamic object creation using python gary 7 1,198 Oct-15-2022, 01:35 PM
Last Post: Larz60+
  List Creation and Position of Continue Statement In Regular Expression Code new_coder_231013 3 1,602 Jun-15-2022, 12:00 PM
Last Post: new_coder_231013
  This is an Object - Code interpretation JoeDainton123 2 1,768 Jun-16-2021, 08:11 PM
Last Post: snippsat
  Is there a library for recursive object creation using config objects johsmi96 0 1,823 May-03-2021, 08:09 PM
Last Post: johsmi96
  Where is the error with this db creation code & the 'conn' variable? pcsailor 6 3,394 Nov-11-2018, 10:25 AM
Last Post: pcsailor
  The number of object makes code slower? fig0 1 2,467 Jan-25-2018, 11:16 PM
Last Post: Gribouillis
  Python-Function object creation studnik2 2 3,212 Jul-28-2017, 08:34 PM
Last Post: DeaD_EyE
  Code Review? PappaBear 9 7,076 Apr-18-2017, 10:43 PM
Last Post: metulburr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020