Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
generator runout
#1
this can get annoying:
Output:
>>> regions_fns = (f'{pre}{my}{aws}{ec2}region{s}{ext}' \ ... for my in ('my_','my','') \ ... for aws in ('aws_','aws','') \ ... for ec2 in ('ec2_','ec2','') \ ... for pre in ('','.','.config/') \ ... for s in ('s','') \ ... for ext in ('.list','.txt','.conf','') \ ... ) >>> len([x for x in regions_fns]) 648 >>> len([x for x in regions_fns]) 0 >>>
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
(Oct-31-2019, 01:24 AM)Skaperen Wrote: this can get annoying:
Not sure if there is question here, but generators are exhausted after you iterate over them once.

Also why not use itertools.product to shorten the comprehension?
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#3
(Oct-31-2019, 01:24 AM)Skaperen Wrote: this can get annoying

I wouldn't call it 'annoying' i would call it 'silent'. There is no error raised when using exhausted iterator or generator only strange behavior and missing values.

There are many ways to mitigate this situation. One of them is to define function which returns new generator/iterator every time called.
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Reply
#4
it's not an error. it's the way things a supposed to be. it was just annoying to me because i thought it was better to use a generator like this wherever i could. today i ran into an issue because i needed to count the number of iterations a generator had and i didn't think about it being exhausted. i just changed it to a list.

show me what my line-wrapped comprehension would look like with itertools.product. i think the way i have now (changed to be a list) looks clear and easy to read. my imagination of itertools.product is not like that.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#5
(Oct-31-2019, 08:11 AM)Skaperen Wrote: how me what my line-wrapped comprehension would look like with itertools.product.
from itertools import product
my_product = product(('my_','my',''), ('aws_','aws',''), ('ec2_','ec2',''),
                     ('','.','.config/'), ('s',''), ('.list','.txt','.conf',''))
regions_fns = (f'{pre}{my}{aws}{ec2}region{s}{ext}' for my, aws, ec2, pre, s, ext in my_product)
# or with str.format()
# regions_fns = ('{3}{0}{1}{2}region{4}{5}'.format(*item) for item in my_product)

at least don't use line continuation char \. following is valid code
regions_fns = (f'{pre}{my}{aws}{ec2}region{s}{ext}'
               for my in ('my_','my','')
               for aws in ('aws_','aws','')
               for ec2 in ('ec2_','ec2','')
               for pre in ('','.','.config/')
               for s in ('s','')
               for ext in ('.list','.txt','.conf',''))
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#6
from itertools import tee

iterator1, iterator2 = tee(your_generator())
More efficient is to save the result in a list or tuple.
The function tee keeps track of yielded elements.
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply
#7
(Oct-31-2019, 08:36 AM)buran Wrote: at least don't use line continuation char \.

i don't always remember when the line continuation char is needed. i use it when unsure, to be sure the code works. i know ... i should just try it and see.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020