Python Forum
Why I am not getting the Redirected codes using requests?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Why I am not getting the Redirected codes using requests?
#1
Hi, thank you for spending your valuable time to help me. I have tried the following code to get the dead, redirected, alive URLs. But I am not getting the redirected URL response code, example
I am sure about 302 code status from the browsers network view but not getting by the code. I even have specified instead of a range but did not work. My code records status code 200 instead of 302! Please help.

import csv
import requests

def check_url_status(url):
"""Checks the status of a URL and returns a descriptive status string."""
try:
response = requests.get(url, timeout=5)
status_code = response.status_code
if status_code in range(200, 299):
return f"Alive ({status_code})"
elif status_code == 300:
return f"Multiple Choices Redirected ({status_code})"
elif status_code == 301:
return f"Moved Permanently Redirected ({status_code})"
elif status_code == 302:
return f"Temporarily Moved Redirected ({status_code})"
elif status_code in range(303, 400):
return f"Redirected ({status_code})"
else:
return f"Not Found ({status_code})"
except Exception as e:
return f"Error: {e}"

# Read URLs from CSV file
urls = []
with open("urls.csv", "r") as csvfile:
reader = csv.reader(csvfile)
for row in reader:
urls.append(row[0])

# Check each URL and write results to new file
with open("url_status.csv", "w", newline="") as outfile:
writer = csv.writer(outfile)
writer.writerow(["URL", "Status"])
for url in urls:
status = check_url_status(url)
writer.writerow([url, status])

print("URL status successfully written to url_status.csv!")
Larz60+ write Dec-21-2023, 06:00 PM:
Please post all code, output and errors (it it's entirety) between their respective tags. Refer to BBCode help topic on how to post. Use the "Preview Post" button to make sure the code is presented as you expect before hitting the "Post Reply/Thread" button.
Larz60+ write Dec-21-2023, 05:59 PM:
removed spam link
Reply
#2
Thank you all. I have found the reason! By default I see the requests.get follows the redirected URLs and takes the status code 200! That is why I did not get the 300 codes!! I have disabled the redirect following by "allow_redirects=False"

So, the line number 6 of code was
response = requests.get(url, timeout=5)
and I changed to
response = requests.get(url, timeout=5,allow_redirects=False)
Larz60+ write Dec-21-2023, 06:02 PM:
removed spam link and frivolous reply to own post
Reply
#3
Please keep two things in mind in the future.

1. Post code in code tags (Python Button).
2. Full quotes are pointless.
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020