Python Forum
Using list comprehension with 'yield' in function
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Using list comprehension with 'yield' in function
#1
Greetings!
I'm trying to speed up my script and I thought I'll use List Comprehensions in a Function.
 from pathlib import Path
some_dir = 'C:\\Python27'

def scan_dir(some_dir) :
    dirs = [str(sbd) for sbd in Path(some_dir).iterdir() if sbd.is_dir()]
    yield dirs

sub_dirs=scan_dir(some_dir)
for i in sub_dirs :
    print(f" Subdirectories -> {i}") 
I was expecting and output like this:
                   Subdirectories -> C:\Python27\DLLs
                   Subdirectories -> C:\Python27\Doc
                   Subdirectories -> C:\Python27\include
                   Subdirectories -> C:\Python27\Lib
                   Subdirectories -> C:\Python27\libs
                   Subdirectories -> C:\Python27\Scripts
                   Subdirectories -> C:\Python27\tcl
                   Subdirectories -> C:\Python27\Tools
Instead I got this:
                   Subdirectories -> ['C:\\Python27\\DLLs', 'C:\\Python27\\Doc', 'C:\\Python27\\include', 'C:\\Python27\\Lib', 'C:\\Python27\\libs', 'C:\\Python27\\Scripts', 'C:\\Python27\\tcl', 'C:\\Python27\\Tools']
I'm clearly doing something wrong, I just do not see what it is...
Thank you in advance!
Reply
#2
(Apr-02-2023, 05:25 AM)tester_V Wrote: I'm clearly doing something wrong
Your generator is generating a single item which is the whole list of subdirectories.

You could return the whole generator
def scan_dir(dir):
    return (str(x) for x in dir.iterdir() if x.is_dir())
or
import os
def scan_dir(dir):
    for name in next(os.walk(dir))[1]:
        yield os.path.join(dir, name)
or
import os
def scan_dir(dir):
    return (x.path for x in os.scandir(dir) if x.is_dir())
I think the last one is better.
tester_V likes this post
Reply
#3
So, it is not possible to use "yield" with the list comprehension. Right?
Thank you!
Reply
#4
(Apr-02-2023, 06:31 AM)tester_V Wrote: So, it is not possible to use "yield" with the list comprehension.
Yes you could use yield from
def func():
    yield from range(3)
    yield from (x**2 for x in range(2, 7))
tester_V likes this post
Reply
#5
(Apr-02-2023, 05:25 AM)tester_V Wrote: I'm trying to speed up my script and I thought I'll use List Comprehensions in a Function.
It will not speed things,and the list comprehensions will create the list in memory unnecessary.
The point with yield is that it keep it it a lazy iterators do not store the whole content of data in the memory.
So then could write it like this.
from pathlib import Path

def scan_dir(some_dir):
    for sbd in some_dir.iterdir():
        if sbd.is_dir:
            yield sbd

if __name__ == '__main__':
    some_dir = Path('C:\\Python27')
    for i in scan_dir(some_dir):
        print(f" Subdirectories -> {i}")
Output:
Subdirectories -> C:\Python27\DLLs Subdirectories -> C:\Python27\Doc Subdirectories -> C:\Python27\include .....
buran and tester_V like this post
Reply
#6
Thank you!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  yield from akbarza 4 298 Apr-19-2024, 07:58 PM
Last Post: DeaD_EyE
  List Comprehension Issue johnywhy 5 545 Jan-14-2024, 07:58 AM
Last Post: Pedroski55
  yield usage as statement or expression akbarza 5 818 Oct-23-2023, 11:43 AM
Last Post: Gribouillis
Question mypy unable to analyse types of tuple elements in a list comprehension tomciodev 1 491 Oct-17-2023, 09:46 AM
Last Post: tomciodev
  list comprehension 3lnyn0 4 1,426 Jul-12-2022, 09:49 AM
Last Post: DeaD_EyE
  List comprehension used differently coder_sw99 3 1,739 Oct-03-2021, 04:12 PM
Last Post: coder_sw99
  How to invoke a function with return statement in list comprehension? maiya 4 2,865 Jul-17-2021, 04:30 PM
Last Post: maiya
  List comprehension and Lambda cametan 2 2,253 Jun-08-2021, 08:29 AM
Last Post: cametan
  What is the difference between a generator and a list comprehension? Pedroski55 2 2,230 Jan-02-2021, 04:24 AM
Last Post: Pedroski55
  For Loop with List Comprehension muzikman 25 6,698 Dec-18-2020, 10:45 PM
Last Post: muzikman

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020