Python Forum
Async / Await usage with asyncio to retrieve urls in parallel
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Async / Await usage with asyncio to retrieve urls in parallel
#4
1.) Yes it's possible, you can just call the asyncio.run in getResults but you will have to move the asyncio.gather to a new coroutine

2.) Let's say your new getResult function will look something like this:
async def get_url_result(url):
    response = await asynchronous_get_request(url)  # not an actual working code
    return response
then response will be a some kind of Response object which will have url attribute and the actual response that you can then use to update your dictionary with

3.) requests is a blocking library, which would make the whole point of using asyncio in your case pointless. There is actually no "thread split", asyncio runs only in one thread and it runs concurrently and not in parallel. What happens when you use non-blocking call in this case, is that when a request is made, while it waits for response, it gives back control to the event loop, so other coroutines that are scheduled on it, can be run. There is a nice explanation on real python -> https://realpython.com/async-io-python/

Regarding an example for aiohttp, there is a really simple one in the link I sent you:
async with aiohttp.ClientSession() as session:
    async with session.get('http://httpbin.org/get') as resp:
        print(resp.status)
        print(await resp.text())
It would be good if you'll try to do it on your own, post your experiments and I'll help
Reply


Messages In This Thread
RE: Async / Await usage with asyncio to retrieve urls in parallel - by mlieqo - Sep-20-2020, 06:35 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  get data from 2 async functions korenron 0 1,250 Sep-22-2021, 08:39 AM
Last Post: korenron
  Async requests lukee 0 1,527 Oct-06-2020, 04:40 AM
Last Post: lukee
  Using Python to search through a list of urls jeremy 4 2,915 Dec-18-2019, 11:52 AM
Last Post: Malt
  Async IO writing to two Different Tables Help TiagoV 0 2,657 Oct-09-2019, 04:45 AM
Last Post: TiagoV
  Urls in a file to be executed pyseeker 2 2,077 Sep-09-2019, 03:38 PM
Last Post: pyseeker
  user validation for opening urls Ashley 6 2,754 Jul-08-2019, 09:08 PM
Last Post: metulburr
  Issues with async and yielding from API GSerum 1 2,166 Dec-18-2018, 08:37 PM
Last Post: nilamo
  async question on raspberry pi baukeplugge 2 62,289 Nov-07-2018, 07:58 PM
Last Post: baukeplugge
  How I can limit quantity of parallel executable tasks in asyncio? AlekseyPython 1 2,458 Oct-24-2018, 10:22 AM
Last Post: AlekseyPython
  Looping URLs breaks them PythonStudent 2 2,963 Apr-21-2018, 02:54 PM
Last Post: PythonStudent

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020