Python Forum
Async / Await usage with asyncio to retrieve urls in parallel
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Async / Await usage with asyncio to retrieve urls in parallel
#1
I'm looking for transform the following code into parallel execution :

urls = ['http://example1.org', 'http://example2.org', '...']

def getResults(urls):
  results = {}
  for url in urls:
    results[url] = getResult(url)
  return results

def getResult(url):
  return get(url).json()
Here is what I've tried:

urls = ['http://example1.org', 'http://example2.org', '...']

def getResults(urls):
  return asyncio.gather((getResult(url)) for url in urls)

async def getResult(url):
    return await get(url).json()
Am I on the right track ? What's the correct way to use async/await with python 3?

Thanks
Reply


Messages In This Thread
Async / Await usage with asyncio to retrieve urls in parallel - by lukee - Sep-18-2020, 10:06 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  get data from 2 async functions korenron 0 1,231 Sep-22-2021, 08:39 AM
Last Post: korenron
  Async requests lukee 0 1,509 Oct-06-2020, 04:40 AM
Last Post: lukee
  Using Python to search through a list of urls jeremy 4 2,877 Dec-18-2019, 11:52 AM
Last Post: Malt
  Async IO writing to two Different Tables Help TiagoV 0 2,642 Oct-09-2019, 04:45 AM
Last Post: TiagoV
  Urls in a file to be executed pyseeker 2 2,045 Sep-09-2019, 03:38 PM
Last Post: pyseeker
  user validation for opening urls Ashley 6 2,729 Jul-08-2019, 09:08 PM
Last Post: metulburr
  Issues with async and yielding from API GSerum 1 2,142 Dec-18-2018, 08:37 PM
Last Post: nilamo
  async question on raspberry pi baukeplugge 2 56,436 Nov-07-2018, 07:58 PM
Last Post: baukeplugge
  How I can limit quantity of parallel executable tasks in asyncio? AlekseyPython 1 2,433 Oct-24-2018, 10:22 AM
Last Post: AlekseyPython
  Looping URLs breaks them PythonStudent 2 2,929 Apr-21-2018, 02:54 PM
Last Post: PythonStudent

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020