Python Forum
[SOLVED] [requests] Why 404?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
[SOLVED] [requests] Why 404?
#1
Hello,

I'm learning how to use the "requests" module to download and parse web pages with Python3.

For some reason, I get a 404 when looping through URLs read from a file, while it works if I set the URL manually:

import requests

#Open list of URLs
file1 = open('urls.txt', 'r') 
Lines = file1.readlines() 
for line in Lines: 
	#URL looks OK
	print(line)

	#OK line = "https://www.google.com"
	response = requests.get(line)

	#print(response.text)
	#Says 404
	print(response.status_code)
file1.close()
Any idea what it could be?

Thank you.
Reply
#2
readlines() separates the data by newline, but it doesn't remove the newline. All your lines (except possibly the final one) will have a trailing newline that is not part of the url. The newline is in the print you're doing, but isn't obvious.
Reply
#3
Thanks much for the info.

Working (and simpler) code:

Lines = open("urls.txt").read().splitlines()
for line in Lines: 
	response = requests.get(line)
	print(response.text)
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020