Python Forum
Running A Loop Until You See A Particular Result
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Running A Loop Until You See A Particular Result
#1
Hi guys,

I've been learning about rotating proxies and have found myself a little stuck and after many many hours have passed, I thought it was time to reach out for some assistance. Smile

In a nutshell, I've got a list of proxies and I want pick a random one from a list for each request. If the random list finds a proxy that works, the code below works correctly. If it tries to use a proxy that doesn't work, I get:

Error:
ConnectTimeout: HTTPSConnectionPool(host='hostname.com', port=443): Max retries exceeded with url: /google.com
I understand that the error is the proxy not working(I tested it with several working proxies to verify the problem), so what I'm trying to do, is to run a loop to find a random proxy from my list each time a request is made.

So I've got:
from bs4 import BeautifulSoup
import requests
import random

url = ‘testurl.com’
proxy_list = ['173.68.59.131:3128','64.124.38.139:8080','69.197.181.202:3128']

proxies = random.choice(proxy_list)
response = requests.get(url, headers=headers, proxies={'https': proxies}, timeout=3)
if response.status_code == 200:
    print(response.status_code)
elif response.status_code != 200:
    proxies = random.choice(proxy_list)
    response = requests.get(url, headers=headers, proxies={'https': proxies}, timeout=3)
(At the moment, the code is simply printing a response code of 200 if it's successful, but I'll be changing that later to get html information.)

But anyway, my goal of the above code is to grab a random proxy from the list, test it to check if it works and if it does, do the request. Alteratively if it doesn't, keep randomly looping through the proxy list until it can find a working proxy- and then go ahead and complete the request.

Can anyone please enlighten me how this can be done?

Thanks a lot.
Reply


Messages In This Thread
Running A Loop Until You See A Particular Result - by knight2000 - Sep-01-2021, 06:23 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  help RuntimeError: no running event loop marpaslight 5 3,746 Oct-18-2022, 10:04 PM
Last Post: marpaslight
  code running for more than an hour now, yet didn't get any result, what should I do? aiden 2 1,508 Apr-06-2022, 03:41 PM
Last Post: Gribouillis
  bleak library RuntimeError: This event loop is already running alice93 3 4,102 Sep-30-2021, 08:06 AM
Last Post: alice93
  loop running indefinitely shantanu97 6 2,591 Sep-29-2021, 08:03 PM
Last Post: deanhystad
  Running loop at specific frequency mdsousa 3 5,940 Apr-21-2021, 11:22 AM
Last Post: jefsummers
  Noob Alert! Wrong result using loop and if statemnent GJG 7 2,883 Dec-19-2020, 05:18 PM
Last Post: buran
  RuntimeError: This event loop is already running newbie2019 2 6,958 Sep-30-2020, 06:59 PM
Last Post: forest44
  Running function from parent module which has a loop in it. ta2909i 1 2,691 Nov-18-2019, 07:04 PM
Last Post: Gribouillis
  How to add coroutine to a running event loop? AlekseyPython 1 8,160 Mar-21-2019, 06:04 PM
Last Post: nilamo
  action on MQTT while long loop is running runboy 4 6,105 Oct-05-2018, 11:57 PM
Last Post: runboy

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020