Python Forum

Full Version: Shutil problem in file sorting script
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hey everyone,

I'm currently using shutil to sort a large number of files based on information contained in the filename.
Everything is working fine except for the fact that once the files are sorted in the appropriate folder/sub-folder/sub-folder
architecture, I'm unable to delete them. Windows tells me that all the sorted files are still being used in another process.

I've closed all the python script, restarted the computer but nothing seems to help me so far.

Have you ever experienced the same problem?

Thanks for the help!

my code here
Shutil does not contain a method for deleting files.
You can delete directories.
so a work around is to move to a temp directory, and then delete that.
see: Deleting Files and Directories

You can also use the remove command:

import os
os.remove('Sample.pdf')
Thanks for the fast reply. I probably didn't express myself correctly.
Everything in the script runs fine. The only problem is that when I access the copied files, I am unable to delete them manually.
Windows tells me that the files are being used in another process =( I closed the computer, looked for hidden files but nothing is working
so far.

Here is the code. (I'm new to Python (less than a week) so it may be quite chaotic, I apologize in advance)

I hope someone already run into the same issue. I'm using Python 3.6 on windows 10 Professional.

import os
import json
import time
import shutil
import os.path
from datetime import datetime
import subprocess

Tfiles=[]

#We load a dictionary that contains the serial number of the file as a key and the filepath to this file as the corresponding string
#The Tree is created in another script that runs every 30mins
with open('Tree.json', 'r') as fp:
   Tree = json.load(fp)
   
PDF_type= [ A HUGE LIST OF THE DIFFERENT PDF DOCUMENTS']
Matching_Directory = [ A HUGE LIST OF MATCHING FILEPATH FOR EACH TYPE OF DOCUMENTS ]

#We input the filepath of the folder where the PDFs are stored
directory_in_str = "C:\\Users\\***\\Desktop\\Reception PDF files"
directory = os.fsencode(directory_in_str)

#We get the list of folders
for file in os.listdir(directory):
   filename = os.fsdecode(file)
   #We make sure that only pdf are sorted
   if filename.endswith(".pdf"):
       #We save the filepath and the valuable information
       origine = (os.path.join(directory_in_str, filename))
       Temporary_WO = filename[:7]
       Identification = filename [8:-4]
       #_____________________________________________________________
       #We make sure the picked up information is in accordance with the approved nomanclature
       Char_test=0;
       Numb_test=0;
       for char in Temporary_WO:
           if char.isalpha()==True:
               Char_test+=1
           elif char.isdigit()==True:
               Numb_test+=1
       #_____________________________________________________________        
       #If the folder successfuly undergo the test
       if Char_test==2 and Numb_test==5:
           WO = Temporary_WO
           #We compare the Pdf type with the list 'Pdf_type'. If there is a match, we take the corresponding filepath in 'Matching_Directory'
           for i in range(0,len(PDF_type)):
               if Identification == PDF_type[i]:
                   #We verify that the serial number of the file already exists in the Tree dictionary (meaning that a folder for this project already exists on the server)
                   if WO in dict(Tree):
                       Start_Filepath = str(Tree[WO])
                       End_Filepath = str(Matching_Directory[i])
                       Arrival_Filepath = os.path.abspath(Start_Filepath +'\\'+ End_Filepath)
                       Test_Filepath = os.path.abspath(Arrival_Filepath +'\\'+ filename)
                       #We make sure that a file with the same name doesnt exist (dont want to delete an existing file)
                       for j in range(1,5):
                           if os.path.isfile(Test_Filepath):
                               Test_Filepath = os.path.abspath(Arrival_Filepath +'\\'+ WO +'_'+ Identification + str(j) +'.pdf')
                               os.rename(origine,(directory_in_str + '\\'+ WO +'_'+ Identification + str(j) +'.pdf'))
                               origine = os.path.abspath(directory_in_str + '\\'+ WO +'_'+ Identification + str(j) +'.pdf')
                           else:
                               for k in range(0,21,4): # Ajusted to the network speed
                                   #Copying the pdf
                                   shutil.copy(origine,Arrival_Filepath)
                                   time.sleep(k)

                                   #Making sure the copied file is there (Would be nice to add a second condition to make sure that the pdf can be read)
                                   if os.path.isfile(Test_Filepath):
                                       os.remove(origine)
                                       #Creation of the log
                                       Tfiles.append(origine)
                                       Tfiles.append(Arrival_Filepath)
                                       break
                               break
                               #else:
                                   #Error_Copy_Failed=()

FORMAT = '%d/%m/%Y at %H:%M:%S'
save_path = 'C:\\Users\\***\\Desktop\\ALPHA TESTING FOLDER'
name_of_file = 'Transfer Log'
completeName = os.path.join(save_path, name_of_file+'.txt')    
file1 = open(completeName,'w')
toFile = ('The program ran for the last time on: %s \n\n' % (datetime.now().strftime(FORMAT)))
toFile1 = ('''Here is the list of the different files and the corresponding filepath where they have been transfered. Have a good day!\n
#________________________________________________________________________________________\n\n''')
file1.write(toFile)
file1.write(toFile1)
for lines in Tfiles:
   toFile2 = ('%s \n' % lines)
   file1.write(toFile2)
toFile3 = ('''________________________________________________________________________________________
#\nEnd of the list.''')
file1.write(toFile3)
file1.close()      
Again, thanks for your time!
And if you have any suggestions on ways to improve my coding, I'll be glad to learn a few tricks from the experienced coders out there. =)

After running some tests, the problem seems to be that my program or another program keeps the copied files opened.

The complete Windows error message is: This action cannot be completed because the file is open in another program. Close the file and try again.

I've successfully deleted the files by using brute force (Clicking on my mouse a thousand time until the file is deleted).

I'm looking in the computer's process but everything is closed. I also added a quit() on the last line of the program but it still
doesn't work.

Do you have any clue?

Thanks again for the help.
you really need to learn how to use functions.
This code is painful to read.
if you still have the program running, the file handle may not have been completely released.
one way to assure this is done is to use with rather than manually opening and closing (as you do with the json file)
My guess is that files are not getting properly closed.