Python Forum
help me with python script for AWS S3
Thread Rating:
  • 1 Vote(s) - 4 Average
  • 1
  • 2
  • 3
  • 4
  • 5
help me with python script for AWS S3
#1
In detail: when a file is uploaded via FTP to an FTP server in the one region, the data providers will move the file, causing an i_notify event. cron should handle this event and kick off a script that will upload this file to the S_3 bucket in the other region... Upload this file to the bucket using the accelerated endpoint and using multi part uploads.
Reply
#2
You forgot one detail... what is your question? Post the code you've written so far as well as the full Traceback if your code fails.
If it ain't broke, I just haven't gotten to it yet.
OS: Windows 10, openSuse 42.3, freeBSD 11, Raspian "Stretch"
Python 3.6.5, IDE: PyCharm 2018 Community Edition
Reply
#3
i want to write a script that will be called by incron to upload files to the S3 bucket in the "other region" when the data files are moved. In detail: when a file is uploaded via FTP to an FTP server in the Ireland region, the file will move , causing an inotify event. Incron should handle this event and kick off a script that will upload this file to the S3 bucket in the us-east-1 bucket. Upload this file to the bucket using the accelerated endpoint  and using multipart uploads...
i wrote this much code for this..but i dont have any idea about incron and script is also not completed ... plz guide me through this.
i am posting screenshot of my code in attachment..and here too 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
import os, sys
 
import boto3
 
client = boto3.client('s3')
 
def accelarationBucket():
 
    response = client.put_bucket_accelerate_configuration(
 
     Bucket=' mybucketname',
 
     AccelerateConfiguration={'Status': 'Enabled'}
 
    )
 
def creatMultipartUpload():    
 
    response = client.create_multipart_upload(
 
     Bucket='twc-ecmwf-data-eu-west-1',
 
     key='first'
 
    )
 
 
    response = client.complete_multipart_upload(
 
     Bucket='twc-ecmwf-data-eu-west-1',
 
     Key='first',
 
     UploadId=''
 
    )
 
def rsyncBucket():
 
    boto-rsync s3://twc-ecmwf-data-eu-west-1  s3://twc-ecmwf-data-us-east-1 
 
accelarationBucket()
 
creatMultipartUpload()
rsyncBucket()
İmage
Reply
#4
Could you elaborate on your question? You're more likely to get a reply that why, since it's harder for would-be answerers to try to figure out the question by reading your code and seeing what it does/doesn't do relative to your description of what you ultimately want.
Reply
#5
(Oct-20-2016, 10:48 AM)DDPYTHON Wrote: In detail: when a file is uploaded via FTP to an FTP server in the one region, the data providers will move the file, causing an i_notify event. cron should handle this event and kick off a script that will upload this file to the S_3 bucket in the other region... Upload this file to the bucket using the accelerated endpoint and using multi part uploads.

That sounds like it was copy-pasted from a homework assignment.
Reply
#6
Merged threads, keep that in mind while trying to read the flow above!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Is there a *.bat DOS batch script to *.py Python Script converter? pstein 3 8,212 Jun-29-2023, 11:57 AM
Last Post: gologica
  How to kill a bash script running as root from a python script? jc_lafleur 4 8,096 Jun-26-2020, 10:50 PM
Last Post: jc_lafleur
  crontab on RHEL7 not calling python script wrapped in shell script benthomson 1 3,014 May-28-2020, 05:27 PM
Last Post: micseydel
  Package python script which has different libraries as a single executable or script tej7gandhi 1 3,274 May-11-2019, 08:12 PM
Last Post: keames
  Check Python version from inside script? Run Pythons script in v2 compatibility mode? pstein 2 10,731 Jul-07-2017, 08:59 AM
Last Post: snippsat
  Cant pass corect variables to python script in bash script neradp 3 7,312 Nov-05-2016, 01:26 AM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020