Python Forum
Async / Await usage with asyncio to retrieve urls in parallel
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Async / Await usage with asyncio to retrieve urls in parallel
#6
Just a few small changes. I would personally leave the collection of all the results in the get_results function:
def get_results(urls):

    async def gather_coro():
        return await asyncio.gather(*(get_url_result(url) for url in urls))

    url_results = asyncio.run(gather_coro())
    return dict(url_results)
And then acquiring the single result to get_url_result coroutine:

async def get_url_result(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as resp:
            return resp.url, await resp.text()
Reply


Messages In This Thread
RE: Async / Await usage with asyncio to retrieve urls in parallel - by mlieqo - Sep-20-2020, 08:18 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  get data from 2 async functions korenron 0 1,246 Sep-22-2021, 08:39 AM
Last Post: korenron
  Async requests lukee 0 1,524 Oct-06-2020, 04:40 AM
Last Post: lukee
  Using Python to search through a list of urls jeremy 4 2,909 Dec-18-2019, 11:52 AM
Last Post: Malt
  Async IO writing to two Different Tables Help TiagoV 0 2,652 Oct-09-2019, 04:45 AM
Last Post: TiagoV
  Urls in a file to be executed pyseeker 2 2,073 Sep-09-2019, 03:38 PM
Last Post: pyseeker
  user validation for opening urls Ashley 6 2,746 Jul-08-2019, 09:08 PM
Last Post: metulburr
  Issues with async and yielding from API GSerum 1 2,158 Dec-18-2018, 08:37 PM
Last Post: nilamo
  async question on raspberry pi baukeplugge 2 61,228 Nov-07-2018, 07:58 PM
Last Post: baukeplugge
  How I can limit quantity of parallel executable tasks in asyncio? AlekseyPython 1 2,451 Oct-24-2018, 10:22 AM
Last Post: AlekseyPython
  Looping URLs breaks them PythonStudent 2 2,953 Apr-21-2018, 02:54 PM
Last Post: PythonStudent

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020