Posts: 11
Threads: 1
Joined: Nov 2019
(Dec-06-2019, 12:16 PM)Malt Wrote: I can suggest you a logic, you can have a flag and make the flag to turn True when all the expected sub folders are available. Till then it has to keep checking the folders availability. Here we are having a check, when any folder creation fails then it will keep on waiting and checking endlessly.
To avoid that you can have a timer, either timer should expire or your condition should meet. After meeting either of the case, your next step which is copying the sub folders to another location should take place.
Thanks mr. malt
can you write a script for that? It will be very useful for me to learn Mr.malt
Thanks in advance....
Posts: 101
Threads: 5
Joined: Jul 2019
I'm suggesting code here to check the folder existence and I'm making counter to avoid infinite loop. You can tweak the logic according to your need. And after this you have to write code for copying the complete folders from source to your required destination.
def checking_folder():
folder_found = False
count = 0
folders_to_check = ['a','b','c','d']
while not folder_found and count < 50:
print("Checking folders existence")
folder_path = os.path.join("your path","")
for folder in folders_to_check:
if os.path.exists(folder_path+folder):
print(f"Folder is identified: {folder}")
folder_found = True
else:
print(f"This folder is missing: {folder}")
folder_found = False
count += 1
break
Posts: 1
Threads: 0
Joined: Dec 2019
Hi
Please try below :-
this code will move newly file from source to destination folder!!
import time
import os
import shutil
SECONDS_IN_DAY = 120 #inseconds
src = "C:\\Users\Ramesh.kumar\Desktop\SourceData"
dst = "C:\\Users\Ramesh.kumar\Desktop\newDataIdentified"
Data ="C:\\Users\Ramesh.kumar\Desktop\MoveToLinux"
now = time.time()
before = now - SECONDS_IN_DAY
print("time interval is:",before)
def last_mod_time(fname):
return os.path.getmtime(fname)
for fname in os.listdir(src):
src_fname = os.path.join(src, fname)
data_fname = os.path.join(Data, fname)
if last_mod_time(src_fname) > before:
dst_fname = os.path.join(dst, fname)
print("-------------------------------------------------------------------------------")
print("data is going to from A TO B")
print("-------------------------------------------------------------------------------")
shutil.copy(src_fname, dst_fname)
shutil.move(dst_fname,data_fname)
print("-------------------------------------------------------------------------------")
print("new data has been moved")
print("-------------------------------------------------------------------------------") Thanks
HadoopHelp
Posts: 101
Threads: 5
Joined: Jul 2019
If my understanding is correct then the parent folder is getting created everyday with today's date as folder name and there won't be any more parent folder per day.
If that is the case, we can directly check the source place whether the parent directory for today's date is created and if that is created then look for sub folders availability. If the sub folders are also available then check in the destination path for the latest folder existence(ideally, it shouldn't be there).
In case if the latest parent folder is not available in destination then copy the parent folder from source to destination. If the latest parent folder is already there in the destination then skip the copying part (In destination ,if it is needed to check sub folders, add the check as well)
So, the skeleton will be like,
Quote:if <today's folder is available in source>
-> Wait for the sub folders creation
-> Check whether all the sub folders are available after stipulated time period
-> Check the destination path for latest parent folder
if <latest parent folder is not available in destination>
Copy the folders from source to destination
Check for sub folders content after copying (if needed)
elif <latest parent folder is already available in the destination>
Skip the copying part
Check for sub folders content (if needed)
else <when today's parent folder is not available in the source location>
do force exit from the script
Posts: 11
Threads: 1
Joined: Nov 2019
(Dec-06-2019, 12:16 PM)Malt Wrote: I can suggest you a logic, you can have a flag and make the flag to turn True when all the expected sub folders are available. Till then it has to keep checking the folders availability. Here we are having a check, when any folder creation fails then it will keep on waiting and checking endlessly.
To avoid that you can have a timer, either timer should expire or your condition should meet. After meeting either of the case, your next step which is copying the sub folders to another location should take place.
Thank u malt.
Can u write this script for me which will be useful.
Thanks in advance.
Posts: 11
Threads: 1
Joined: Nov 2019
Dec-25-2019, 06:31 AM
(This post was last modified: Dec-25-2019, 07:27 AM by buran.)
(Dec-24-2019, 05:08 AM)sampathkumar Wrote: (Dec-06-2019, 12:16 PM)Malt Wrote: I can suggest you a logic, you can have a flag and make the flag to turn True when all the expected sub folders are available. Till then it has to keep checking the folders availability. Here we are having a check, when any folder creation fails then it will keep on waiting and checking endlessly.
To avoid that you can have a timer, either timer should expire or your condition should meet. After meeting either of the case, your next step which is copying the sub folders to another location should take place.
Thank u malt.
Can u write this script for me which will be useful.
Thanks in advance.
This is my updated code:
import os, sys
import os.path
import glob
import shutil
import time, datetime
import logging
from datetime import date, datetime, timedelta
today = date.today()
parent_dir = "/home/user1/incoming/"
folder = today.strftime("%Y-%m-%d")
directory_path = os.path.join(parent_dir, folder)
print(directory_path)
print(" ")
def checking_folder():
folder_found = False
count = 0
folders_to_check = ['folder1','folder2','folder3','folder4']
while not folder_found and count < 50:
print("Checking folders existence")
folder_path = os.path.join("parent_dir","folder")
for folder in folders_to_check:
if os.path.exists(folder_path+folder):
print(f"Folder is identified: {folder}")
folder_found = True
else:
print(f"This folder is missing: {folder}")
folder_found = False
count += 1
break
---------------------------------------
Iam getting the following error:
File "date15.py", line 28
print(f"Folder is identified: {folder}")
^
SyntaxError: invalid syntax
Pls help me.
Thanks in advance
Posts: 11
Threads: 1
Joined: Nov 2019
Dec-25-2019, 12:26 PM
(This post was last modified: Dec-25-2019, 12:26 PM by sampathkumar.)
import os, sys
import os.path
import glob
import shutil
import time, datetime
import logging
from datetime import date, datetime, timedelta
today = date.today()
parent_dir = "/home/user1/incoming/"
folder = today.strftime("%Y-%m-%d")
directory_path = os.path.join(parent_dir, folder)
print(directory_path)
print(" ")
def checking_folder():
folder_found = False
count = 0
folders_to_check = ['folder1','folder2','folder3','folder4']
while not folder_found and count < 50:
print("Checking folders existence")
folder_path = os.path.join("parent_dir","folder")
for folder in folders_to_check:
if os.path.exists(folder_path+folder):
print(f"Folder is identified: {folder}")
folder_found = True
else:
print(f"This folder is missing: {folder}")
folder_found = False
count += 1
break Iam getting the following error:
File "date15.py", line 28
print(f"Folder is identified: {folder}")
^
SyntaxError: invalid syntax
Pls help me.
Thanks in advance
Posts: 11
Threads: 1
Joined: Nov 2019
(Dec-24-2019, 05:08 AM)sampathkumar Wrote: (Dec-06-2019, 12:16 PM)Malt Wrote: I can suggest you a logic, you can have a flag and make the flag to turn True when all the expected sub folders are available. Till then it has to keep checking the folders availability. Here we are having a check, when any folder creation fails then it will keep on waiting and checking endlessly.
To avoid that you can have a timer, either timer should expire or your condition should meet. After meeting either of the case, your next step which is copying the sub folders to another location should take place.
Thank u malt.
Can u write this script for me which will be useful.
Thanks in advance.
import os, sys
import os.path
import glob
import shutil
import time, datetime
import logging
from datetime import date, datetime, timedelta
parent_dir = "/home/user1/incoming/"
today = date.today()
folder1 = today.strftime("%Y-%m-%d")
directory_path = os.path.join(parent_dir, folder1)
def checking_folder():
folder_found = False
count = 0
folders_to_check = ['folder1','folder2','folder3','folder4']
while not folder_found and count < 50:
print("Checking folders existence")
folder_path = os.path.join(parent_dir, folder1)
print(folder_path)
for folder in folders_to_check:
if os.path.exists(folder_path + folder):
print(f"Folder is identified: {folder}")
folder_found = True
else:
print(f"This folder is missing: {folder}")
folder_found = False
count += 1
break
if os.path.exists(directory_path):
print("Folder found")
checking_folder()
else:
print("Folder doesn't exists") Dear mr. Malt
While running this code iam getting the following error msg:
File "final.py", line 25
print(f"Folder is identified: {folder}")
^
SyntaxError: invalid syntax
Pls help me correct this problem.....
Thanks in advance....
Posts: 101
Threads: 5
Joined: Jul 2019
Can you please let us know which version of Python you are actually using?
Posts: 11
Threads: 1
Joined: Nov 2019
(Mar-27-2020, 06:45 PM)Malt Wrote: Can you please let us know which version of Python you are actually using?
Thank u malt. i have used older version. now have upgraded to latest. its working now.
|