urllib.error.HTTPError: HTTP Error 404: Not Found - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html) +--- Thread: urllib.error.HTTPError: HTTP Error 404: Not Found (/thread-24424.html) |
urllib.error.HTTPError: HTTP Error 404: Not Found - ckkkkk - Feb-13-2020 This is the first time i am using the Google API key and the code is suppose to open up the browser when there is a new video uploaded from the channel. But am getting the http error. Did some searches and some forum actually recommended using the user agent, but no luck making it works. Here is the code import urllib3, json import urllib.request from selenium import webdriver import time import requests def look_for_new_video(): headers = {} headers['User-Agent'] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36" api_key = "KEY_REMOVED" channel_id = "UCWr0mx597DnSGLFk1WfvSkQ" base_video_url = 'https://www.youtube.com/watch?v=' base_search_url = 'https://wwww.googleapis.com/youtube/v3/search?' url = base_search_url + 'key={}&channelId={}&part=snippet,id&order=date&maxResults=1'.format(api_key, channel_id) req = urllib.request.Request(url, headers = headers) print (req) #####Error starts here inp = urllib.request.urlopen(url) print (inp) resp = json.loads(inp) responese = requests.get(url) print (responese) look_for_new_video() vidID = resp['items'][0]['id']['videoId'] video_exists = False with open('videoid.json', 'r') as json_file: data = json.load(json_file) if data['videoId'] != vidID: driver = webdriver.Chrome() driver.get(base_video_url + vidID) video_exists = True if video_exists: with open('videoid.json', 'w') as json_file: data = {'videoId' : vidID} json.dump(data, json_file) try: while True: look_for_new_video() time.sleep(10) except KeyboardInterrupt: print ("Stop")Here are the error messages I did managed to find out which line caused the error but i couldn't really solve it even after surfing through the internet. So any help would be appreciated. Thanks in advance.
RE: urllib.error.HTTPError: HTTP Error 404: Not Found - jim2007 - Feb-13-2020 Line 14, you've got 4 Ws in the url: 'https://wwww.googleapis.com/youtube/v3/search?' RE: urllib.error.HTTPError: HTTP Error 404: Not Found - snippsat - Feb-13-2020 Drop all other modules and use only Requests as you have in import. Call in line 15 is wrong snippet,id&order can not have , .Example getting subscribers count for a YouTube channel. import requests api_key = 'xxxxxxxxxxxxxxxxx' channel_id = 'UC5kS0l76kC0xOzMPtOmSFGw' response = requests.get(f'https://www.googleapis.com/youtube/v3/channels?part=statistics&id={channel_id}&key={api_key}')Test usage: >>> response <Response [200]> >>> j = response.json() >>> j['items'][0]['statistics']['subscriberCount'] '273000'Possible fix for your call. import requests api_key = 'xxxxxxxxxxxxxxxx' channel_id = 'UC5kS0l76kC0xOzMPtOmSFGw' response = requests.get(f'https://www.googleapis.com/youtube/v3/channels?part=snippet&order=date&maxResults=1&id={channel_id}&key={api_key}') json_data = response.json() # Do call on returned json_data RE: urllib.error.HTTPError: HTTP Error 404: Not Found - ckkkkk - Mar-03-2020 (Feb-13-2020, 06:03 PM)jim2007 Wrote: Line 14, you've got 4 Ws in the url: 'https://wwww.googleapis.com/youtube/v3/search?' Omg, Thank you!Also, how did i missed on that... (Feb-13-2020, 07:11 PM)snippsat Wrote: Possible fix for your call. Thank you, it works!!. After that, i tried to make changes to my original code it works too but only for a few minutes. hdr = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3', 'Accept-Encoding': 'none', 'Accept-Language': 'en-US,en;q=0.8', 'Connection': 'keep-alive'} base_video_url = 'https://www.youtube.com/watch?v=' base_search_url = 'https://www.googleapis.com/youtube/v3/search?' url = base_search_url + 'key={}&channelId={}&part=snippet&order=date&maxResults=1'.format(api_key, channel_id) req = urllib.request.Request(url, headers = hdr) inp = urllib.request.urlopen(req).read().decode('utf-8') resp = json.loads(inp) with open('videoid.json', 'w') as txt: json.dump(resp, txt) txt.close() vidID = resp['items'][0]['id']['videoId'] video_exists = False with open('videoid.json', 'r') as json_file: data = json.load(json_file) if data['items'][0]['id']['videoId'] != vidID: driver = webdriver.Chrome() driver.get(base_video_url + vidID) video_exists = True if video_exists: with open('videoid.json', 'w+') as json_file: data = {'videoId' : vidID} json.dump(data, json_file)Then this error keep popping up ever since, even with user agent added.
RE: urllib.error.HTTPError: HTTP Error 404: Not Found - snippsat - Mar-03-2020 (Mar-03-2020, 06:13 AM)ckkkkk Wrote: Then this error keep popping up ever since, even with user agent added.The error message do not match the code you have posted. I tested code i posted again it still work. You still use urllib as it raise the 403 error,i did advice to drop urllib also do not use it as the Requests code work.
|