Python Forum
Shutil problem in file sorting script
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Shutil problem in file sorting script
#1
Hey everyone,

I'm currently using shutil to sort a large number of files based on information contained in the filename.
Everything is working fine except for the fact that once the files are sorted in the appropriate folder/sub-folder/sub-folder
architecture, I'm unable to delete them. Windows tells me that all the sorted files are still being used in another process.

I've closed all the python script, restarted the computer but nothing seems to help me so far.

Have you ever experienced the same problem?

Thanks for the help!

my code here
Reply
#2
Shutil does not contain a method for deleting files.
You can delete directories.
so a work around is to move to a temp directory, and then delete that.
see: Deleting Files and Directories

You can also use the remove command:

import os
os.remove('Sample.pdf')
Reply
#3
Thanks for the fast reply. I probably didn't express myself correctly.
Everything in the script runs fine. The only problem is that when I access the copied files, I am unable to delete them manually.
Windows tells me that the files are being used in another process =( I closed the computer, looked for hidden files but nothing is working
so far.

Here is the code. (I'm new to Python (less than a week) so it may be quite chaotic, I apologize in advance)

I hope someone already run into the same issue. I'm using Python 3.6 on windows 10 Professional.

import os
import json
import time
import shutil
import os.path
from datetime import datetime
import subprocess

Tfiles=[]

#We load a dictionary that contains the serial number of the file as a key and the filepath to this file as the corresponding string
#The Tree is created in another script that runs every 30mins
with open('Tree.json', 'r') as fp:
   Tree = json.load(fp)
   
PDF_type= [ A HUGE LIST OF THE DIFFERENT PDF DOCUMENTS']
Matching_Directory = [ A HUGE LIST OF MATCHING FILEPATH FOR EACH TYPE OF DOCUMENTS ]

#We input the filepath of the folder where the PDFs are stored
directory_in_str = "C:\\Users\\***\\Desktop\\Reception PDF files"
directory = os.fsencode(directory_in_str)

#We get the list of folders
for file in os.listdir(directory):
   filename = os.fsdecode(file)
   #We make sure that only pdf are sorted
   if filename.endswith(".pdf"):
       #We save the filepath and the valuable information
       origine = (os.path.join(directory_in_str, filename))
       Temporary_WO = filename[:7]
       Identification = filename [8:-4]
       #_____________________________________________________________
       #We make sure the picked up information is in accordance with the approved nomanclature
       Char_test=0;
       Numb_test=0;
       for char in Temporary_WO:
           if char.isalpha()==True:
               Char_test+=1
           elif char.isdigit()==True:
               Numb_test+=1
       #_____________________________________________________________        
       #If the folder successfuly undergo the test
       if Char_test==2 and Numb_test==5:
           WO = Temporary_WO
           #We compare the Pdf type with the list 'Pdf_type'. If there is a match, we take the corresponding filepath in 'Matching_Directory'
           for i in range(0,len(PDF_type)):
               if Identification == PDF_type[i]:
                   #We verify that the serial number of the file already exists in the Tree dictionary (meaning that a folder for this project already exists on the server)
                   if WO in dict(Tree):
                       Start_Filepath = str(Tree[WO])
                       End_Filepath = str(Matching_Directory[i])
                       Arrival_Filepath = os.path.abspath(Start_Filepath +'\\'+ End_Filepath)
                       Test_Filepath = os.path.abspath(Arrival_Filepath +'\\'+ filename)
                       #We make sure that a file with the same name doesnt exist (dont want to delete an existing file)
                       for j in range(1,5):
                           if os.path.isfile(Test_Filepath):
                               Test_Filepath = os.path.abspath(Arrival_Filepath +'\\'+ WO +'_'+ Identification + str(j) +'.pdf')
                               os.rename(origine,(directory_in_str + '\\'+ WO +'_'+ Identification + str(j) +'.pdf'))
                               origine = os.path.abspath(directory_in_str + '\\'+ WO +'_'+ Identification + str(j) +'.pdf')
                           else:
                               for k in range(0,21,4): # Ajusted to the network speed
                                   #Copying the pdf
                                   shutil.copy(origine,Arrival_Filepath)
                                   time.sleep(k)

                                   #Making sure the copied file is there (Would be nice to add a second condition to make sure that the pdf can be read)
                                   if os.path.isfile(Test_Filepath):
                                       os.remove(origine)
                                       #Creation of the log
                                       Tfiles.append(origine)
                                       Tfiles.append(Arrival_Filepath)
                                       break
                               break
                               #else:
                                   #Error_Copy_Failed=()

FORMAT = '%d/%m/%Y at %H:%M:%S'
save_path = 'C:\\Users\\***\\Desktop\\ALPHA TESTING FOLDER'
name_of_file = 'Transfer Log'
completeName = os.path.join(save_path, name_of_file+'.txt')    
file1 = open(completeName,'w')
toFile = ('The program ran for the last time on: %s \n\n' % (datetime.now().strftime(FORMAT)))
toFile1 = ('''Here is the list of the different files and the corresponding filepath where they have been transfered. Have a good day!\n
#________________________________________________________________________________________\n\n''')
file1.write(toFile)
file1.write(toFile1)
for lines in Tfiles:
   toFile2 = ('%s \n' % lines)
   file1.write(toFile2)
toFile3 = ('''________________________________________________________________________________________
#\nEnd of the list.''')
file1.write(toFile3)
file1.close()      
Again, thanks for your time!
And if you have any suggestions on ways to improve my coding, I'll be glad to learn a few tricks from the experienced coders out there. =)

After running some tests, the problem seems to be that my program or another program keeps the copied files opened.

The complete Windows error message is: This action cannot be completed because the file is open in another program. Close the file and try again.

I've successfully deleted the files by using brute force (Clicking on my mouse a thousand time until the file is deleted).

I'm looking in the computer's process but everything is closed. I also added a quit() on the last line of the program but it still
doesn't work.

Do you have any clue?

Thanks again for the help.
Reply
#4
you really need to learn how to use functions.
This code is painful to read.
if you still have the program running, the file handle may not have been completely released.
one way to assure this is done is to use with rather than manually opening and closing (as you do with the json file)
My guess is that files are not getting properly closed.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  use of shutil.copytree with ENOTDIR exception yan 2 919 Nov-29-2023, 03:02 PM
Last Post: yan
  shutil.move make data corrupt kucingkembar 0 805 Feb-01-2023, 01:30 PM
Last Post: kucingkembar
  List Sorting Problem ZZTurn 5 1,350 Sep-22-2022, 11:23 PM
Last Post: ZZTurn
  Merge htm files with shutil library (TypeError: 'module' object is not callable) Melcu54 5 1,608 Aug-28-2022, 07:11 AM
Last Post: Melcu54
  Script stop work after 3 actioins - PLEASE WHERE IS THE PROBLEM? rondon442 0 1,568 Sep-27-2021, 05:40 PM
Last Post: rondon442
  Problem executing a script on a remote host tester_V 3 2,483 Sep-26-2021, 04:25 AM
Last Post: tester_V
  File sorting by user-chosen category Bachelar 0 1,538 Aug-28-2021, 08:14 AM
Last Post: Bachelar
  extra slashes in the network path for shutil.copy tester_V 3 3,789 Jun-02-2021, 07:57 AM
Last Post: supuflounder
  problem with sphinx and file directory in script kiyoshi7 0 2,296 Mar-11-2021, 03:52 PM
Last Post: kiyoshi7
  Sorting problem deanhystad 2 1,578 Feb-17-2021, 10:04 PM
Last Post: deanhystad

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020