Python Forum
Unable to use random.choice(list) in async method
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Unable to use random.choice(list) in async method
#1
I need to pull a random proxy from a list in an async method but the code exits the method as soon as it hits the line of code:
proxy = random.choice(proxy_list)
Here's the full method and the next line (the "print" line) never executes.
async def download_site(session, url):
    proxy_list = [
    (754, '38.39.205.220:80'),
    (681, '38.39.204.100:80'),
    (682, '38.39.204.101:80'),
    (678, '38.39.204.94:80')
    ]
    proxy = random.choice(proxy_list)
    print(proxy[1])
    async with session.get(url, proxy="http://" + proxy[1]) as response:
        print("Read {0} from {1}".format(response.content_length, url))
Can someone please share a work-around or a fix for this?
Thanks.
Reply
#2
But how are you calling that function? Unless you have something await on it, it won't be scheduled to execute. Can you post the full code? How do you know that is the line that is causing problems? Do you have a traceback?

(It seems to run that section for me when I call it via asyncio.run())
Reply
#3
This works

import asyncio
import random as rnd

async def download_site():
    proxy_list = [
        (754, '38.39.205.220:80'),
        (681, '38.39.204.100:80'),
        (682, '38.39.204.101:80'),
        (678, '38.39.204.94:80')
        ]
    await asyncio.sleep(1)
    proxy = rnd.choice(proxy_list)
    print(proxy)

asyncio.run(download_site())
Output:
(682, '38.39.204.101:80')
Reply
#4
Thanks @bowlofred and @menator01. This is a small app and I should have just posted all the code the first time.

I added the: await asyncio.sleep(1)
and it still did not work. Please see the code below and THANK YOU very much.

Line 34 does not execute.
Also, once I get this working, I would like to move the list out of this method can get the proxy from the method "get_proxy():

import asyncio
import time
import aiohttp

# This code taken from:
# https://realpython.com/python-concurrency/#what-is-parallelism

# Info for adding headers for the proxy
# https://docs.aiohttp.org/en/stable/client_advanced.html

# Metod to provide random proxies
def get_proxy(self):
    proxy_list = [
    (754, '38.39.205.220:80'),
    (681, '38.39.204.100:80'),
    (682, '38.39.204.101:80'),
    (678, '38.39.204.94:80')
    ]
    proxy = random.choice(proxy_list)
    print(proxy[1])
    return proxy

async def download_site(session, url):
    #For test, use this rather than calling get_proxy
    proxy_list = [
    (754, '38.39.205.220:80'),
    (681, '38.39.204.100:80'),
    (682, '38.39.204.101:80'),
    (678, '38.39.204.94:80')
    ]
    await asyncio.sleep(1)
    proxy = random.choice(proxy_list)
    # This line does not execute
    print(proxy[1])
    async with session.get(url, proxy="http://" + proxy[1]) as response:
        print("Read {0} from {1}".format(response.content_length, url))


async def download_all_sites(sites):
    async with aiohttp.ClientSession() as session:
        tasks = []
        for url in sites:
            task = asyncio.ensure_future(download_site(session, url))
            tasks.append(task)
        await asyncio.gather(*tasks, return_exceptions=True)

if __name__ == "__main__":
    sites = [
        "https://www.jython.org",
       # "http://olympus.realpython.org/dice",
    ] #* 80
    start_time = time.time()
    asyncio.get_event_loop().run_until_complete(download_all_sites(sites))
    duration = time.time() - start_time
    print(f"Downloaded {len(sites)} sites in {duration} seconds")
Note: I modified the main method to return a list of one item rather than 160 to make it easier to step through and debug.
Reply
#5
thanks all
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  About list and method .... Fernando_7obink 3 516 Dec-22-2020, 09:15 AM
Last Post: Fernando_7obink
  Help with a random.randint choice in Python booponion 5 629 Oct-23-2020, 05:13 PM
Last Post: deanhystad
  Async requests lukee 0 466 Oct-06-2020, 04:40 AM
Last Post: lukee
  trying to input a variable using random.choice python63 9 1,007 Aug-13-2020, 05:37 PM
Last Post: python63
  Random Choice Operations Souls99 6 839 Jul-31-2020, 10:37 PM
Last Post: Souls99
  random.choice() takes two positional arguments, but three were given. ShakeyPakey 5 3,063 May-31-2020, 03:13 PM
Last Post: deanhystad
  List of n random elements medatib531 5 1,050 Mar-07-2020, 07:09 PM
Last Post: jefsummers
  output a list of random numbers 'x' columns wide adityavpratap 4 1,014 Jan-13-2020, 05:32 PM
Last Post: perfringo
  Finding MINIMUM number in a random list is not working Mona 5 1,026 Nov-18-2019, 07:27 PM
Last Post: ThomasL
  print all method and property of list object engmoh 4 1,077 Oct-26-2019, 05:33 PM
Last Post: engmoh

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020