Python Forum
Using list comprehension with 'yield' in function - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Using list comprehension with 'yield' in function (/thread-39722.html)

Using list comprehension with 'yield' in function - tester_V - Apr-02-2023

Greetings!
I'm trying to speed up my script and I thought I'll use List Comprehensions in a Function.
 from pathlib import Path
some_dir = 'C:\\Python27'

def scan_dir(some_dir) :
    dirs = [str(sbd) for sbd in Path(some_dir).iterdir() if sbd.is_dir()]
    yield dirs

sub_dirs=scan_dir(some_dir)
for i in sub_dirs :
    print(f" Subdirectories -> {i}")
I was expecting and output like this:
                   Subdirectories -> C:\Python27\DLLs
                   Subdirectories -> C:\Python27\Doc
                   Subdirectories -> C:\Python27\include
                   Subdirectories -> C:\Python27\Lib
                   Subdirectories -> C:\Python27\libs
                   Subdirectories -> C:\Python27\Scripts
                   Subdirectories -> C:\Python27\tcl
                   Subdirectories -> C:\Python27\Tools
Instead I got this:
                   Subdirectories -> ['C:\\Python27\\DLLs', 'C:\\Python27\\Doc', 'C:\\Python27\\include', 'C:\\Python27\\Lib', 'C:\\Python27\\libs', 'C:\\Python27\\Scripts', 'C:\\Python27\\tcl', 'C:\\Python27\\Tools']
I'm clearly doing something wrong, I just do not see what it is...
Thank you in advance!

RE: Using list comprehension with 'yield' in function - Gribouillis - Apr-02-2023

(Apr-02-2023, 05:25 AM)tester_V Wrote: I'm clearly doing something wrong
Your generator is generating a single item which is the whole list of subdirectories.

You could return the whole generator
def scan_dir(dir):
    return (str(x) for x in dir.iterdir() if x.is_dir())
or
import os
def scan_dir(dir):
    for name in next(os.walk(dir))[1]:
        yield os.path.join(dir, name)
or
import os
def scan_dir(dir):
    return (x.path for x in os.scandir(dir) if x.is_dir())
I think the last one is better.

RE: Using list comprehension with 'yield' in function - tester_V - Apr-02-2023

So, it is not possible to use "yield" with the list comprehension. Right?
Thank you!

RE: Using list comprehension with 'yield' in function - Gribouillis - Apr-02-2023

(Apr-02-2023, 06:31 AM)tester_V Wrote: So, it is not possible to use "yield" with the list comprehension.
Yes you could use yield from
def func():
    yield from range(3)
    yield from (x**2 for x in range(2, 7))


RE: Using list comprehension with 'yield' in function - snippsat - Apr-02-2023

(Apr-02-2023, 05:25 AM)tester_V Wrote: I'm trying to speed up my script and I thought I'll use List Comprehensions in a Function.
It will not speed things,and the list comprehensions will create the list in memory unnecessary.
The point with yield is that it keep it it a lazy iterators do not store the whole content of data in the memory.
So then could write it like this.
from pathlib import Path

def scan_dir(some_dir):
    for sbd in some_dir.iterdir():
        if sbd.is_dir:
            yield sbd

if __name__ == '__main__':
    some_dir = Path('C:\\Python27')
    for i in scan_dir(some_dir):
        print(f" Subdirectories -> {i}")
Output:
Subdirectories -> C:\Python27\DLLs
Subdirectories -> C:\Python27\Doc
Subdirectories -> C:\Python27\include
.....


RE: Using list comprehension with 'yield' in function - tester_V - Apr-02-2023

Thank you!