Errors using --processes parameter - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: General Coding Help (https://python-forum.io/forum-8.html) +--- Thread: Errors using --processes parameter (/thread-28017.html) |
Errors using --processes parameter - sonhospa - Jul-01-2020 Hello all, I'm new to Python and not even a "real" programmer, so I apologize if any of my questions show the lack of expertise ;-) I'm trying to create "bags" (a defined structure) using a Python script that's been provided. It's called "bagit.py" and can be found on Github (https://github.com/edsu/bagit/blob/master/bagit.py). The script works very well, but considering the amount of data to be packed it runs for many hours. In a helpfile (readme.rst) I found: Quote:Since calculating checksums can take a while when creating a bag, you may want to calculate them in parallel if you are on a multicore machine. You can do that with the --processes option: Unfortunately following this approach leads to multiple errors (I don't know where to attach the screenshot, but I have one) and NO result at all. As a probable solution I changed every occurance of "processes=1" into "processes=4" in the script, but that didn't help... just different error messages resulting. Would one of you probably be able to guide me to the correct use (or syntax), please? Thank you ever so much! Michael RE: Errors using --processes parameter - buran - Jul-01-2020 first of all - show the exact command you are using, e.g. note thar --process is a CLI option, there is no equal sign like you show. second - copy paste the error in error tags (see BBcode help for more info.). RE: Errors using --processes parameter - Gribouillis - Jul-01-2020 sonhospa Wrote:Unfortunately following this approach leads to multiple errors (I don't know where to attach the screenshot, but I have one) and NO result at all.These error messages that you have are the starting point to solving the problem. Find a way to add them to a post. You could perhaps redirect the command's output to a file and then copy and paste that file's content. Use error tags to post. RE: Errors using --processes parameter - sonhospa - Jul-01-2020 Hello Buran and Gribouillis, thank you for your help. I hope adding a "New Reply" is the right way here, as I couldn't see an individual reply button. Buran, I used the command line exactly as provided in the quote above (just my own path instead of the sample path). Only after this attempt failed, as an alternative experiment, I changed the Python script (bagit.py) where the equal sign is part of several Python commands, and that's where I changed "processes=1" into "processes=4". You could see that in the link I provided to Github. Now the errors: a) First attempt before changing the script: "bagit.py --processes 4 path\to\bag" This is only a small part, because the error messages run endlessly (i.e. hours!). I interrupted the process after a few seconds and have a file of 7000 lines already.b) Second attempt: Changes to the script (at 4 positions) from "processes=1" to "processes=4" and rename it. Command: "bagitprocesses4.py path\to\bag" BUT: After all the files were moved to a particular 'data' direcory correctly and writing the first text files correctly, there's only one process used. The relevant part of the output is (first 6,000 lines and all the following lines left out, I marked the message):Quote:2020-07-01 15:08:16,775 - INFO - Moving reel_08.006042.jpg to E:\Alpha-Omega\Alpha-Omega Bilder für Tests\KAMERADSCHAFT_08_dF_Fertig_jpg\tmprkg_lbc4\reel_08.006042.jpg So it seems the attempt to change didn't result in errors like using the CLI command, but also didn't change the number of processes and was therefore obsolete. Is that information helping? |