Recursive Call Going Elsewhere?

[Oliver]

I have some python code outside of a function that walks a directory tree. When the code detects a zip file (which could be nested), it calls a method called "ProcessZipFile(filePath)".

Inside that recursive function, if another zip file is found, it calls itself to process.

Here's the recursive function:

# process zip files recursively if necessary
def processZipFile(filePath):
    file = zipfile.ZipFile(filePath, "r")
    for name in file.namelist():
        data = file.read(name)
        print(name)  # of file
        if (str(name).endswith(".zip")):
            processZipFile(name)

The strange thing is that when the code hits the lower processZipFile(name) line of code in the method above, instead of going back to the top and recursively processing, the code goes to the original call in the code that called this function.

main code below (not in a function):
.
.
.

  if (filePath.endswith("zip")):
            processZipFile(filePath)

I'm still new to Python, but I don't see what's going on. Why isn't the recursive call going back to the top of the function?

The recursive method does process the first zip file correctly.

Hope this posting is clear.

Thanks in advance,

[Oliver]

Update: The problem appears to be that python can't readily process nested zip files. I found code online that will do 2 levels, but I'm having issues making that same code recursive for nested zip files beyond 2 levels. I think I may need to extract files to a temp directory and always process the same way rather than try to get Python to figure it out using limited, non-recursive, code like this:

def processZipFile(filePath):
    global totalMatches
    try:
        with zipfile.ZipFile(filePath, 'r') as zfile:
            for name in zfile.namelist():
                if re.search(r'\.zip$', name) != None:
                    zfiledata = io.BytesIO(zfile.read(name))
                    with zipfile.ZipFile(zfiledata) as zfile2:
                        for name2 in zfile2.namelist():
                            if (not str(name2).endswith("zip")):
                                print(name2)
                            else:
                                read_file()
                else:
                    read_file()
    except IOError:
        print("IO Error in reading zip file data.")

[snippsat]

I found code online that will do 2 levels, but I'm having issues making that same code recursive for nested zip files beyond 2 levels.

Both os.walk() and glob(recursive=True) do a recursive search of whole file tree and fast after PEP 471
Example:

import os

path = r'E:\1py_div\click'
for root, dirs, files in os.walk(path):
    for file in files:
        if file.endswith(('.zip', '.also_me')):
            print(file)
            # To get absolute path that may be needed in a function call
            # print(os.path.join(root, file))
            # Eg ProcessZipFile(os.path.join(root, file))

[DeaD_EyE]

This snippet works:

def list_zip(file, filename=None):
    zf = zipfile.ZipFile(file, 'r')
    for cf in zf.filelist:
        print(zf.filename or filename, cf.filename, sep=': ')
        if cf.filename.lower().endswith('.zip'):
            with zf.open(cf.filename, 'r') as new_zf:
                list_zip(io.BytesIO(new_zf.read()), cf.filename)

The argument filename is only used to hand over the recursive call the current name of the open zip file.
The error-handling is your task :-P

[Oliver]

This snippet works:

def list_zip(file, filename=None):
    zf = zipfile.ZipFile(file, 'r')
    for cf in zf.filelist:
        print(zf.filename or filename, cf.filename, sep=': ')
        if cf.filename.lower().endswith('.zip'):
            with zf.open(cf.filename, 'r') as new_zf:
                list_zip(io.BytesIO(new_zf.read()), cf.filename)

The argument filename is only used to hand over the recursive call the current name of the open zip file.
The error-handling is your task :-P

SO COOL!!!!!

OK, that's just fricken amazing! :)

Did you have this code mostly figured out or I'm assuming you've worked with nested zip files before?

I was finally able to get my version working with recursively extracting, then processing that tree, but yours is so much cleaner.

I'm new to Python, but still humbled by your amazing reply and code.

Thanks very much.

- O

[DeaD_EyE]

Did you have this code mostly figured out or I'm assuming you've worked with nested zip files before?

No, but I think i needed one time the content from an archive as a fileobject.

The rest is experience. When you know which functions returns which types, it's very helpful. Also the knowledge about duck-typing helps. For example a fileobject is something, that implements the methods of a fileobject. If you have for example a string and want to convert it to a usable fileobject: fd = io.StringIO('Hello World'). Mostly all parts from the python stdlib sticks good together. This is the cause why I love this language.

[Oliver]

Did you have this code mostly figured out or I'm assuming you've worked with nested zip files before?

No, but I think i needed one time the content from an archive as a fileobject.

The rest is experience. When you know which functions returns which types, it's very helpful. Also the knowledge about duck-typing helps. For example a fileobject is something, that implements the methods of a fileobject. If you have for example a string and want to convert it to a usable fileobject: fd = io.StringIO('Hello World'). Mostly all parts from the python stdlib sticks good together. This is the cause why I love this language.

Excellent info again. :)

Thanks very much.