Python Forum

Full Version: urllib.error.HTTPError: HTTP Error 404: Not Found
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
This is the first time i am using the Google API key and the code is suppose to open up the browser when there is a new video uploaded from the channel. But am getting the http error. Did some searches and some forum actually recommended using the user agent, but no luck making it works.

Here is the code
import urllib3, json
import urllib.request
from selenium import webdriver
import time
import requests

def look_for_new_video():
    headers = {}
    headers['User-Agent'] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"
    api_key = "KEY_REMOVED"
    channel_id = "UCWr0mx597DnSGLFk1WfvSkQ"
    
    base_video_url = 'https://www.youtube.com/watch?v='
    base_search_url = 'https://wwww.googleapis.com/youtube/v3/search?'
    url = base_search_url + 'key={}&channelId={}&part=snippet,id&order=date&maxResults=1'.format(api_key, channel_id)
    req = urllib.request.Request(url, headers = headers)
    print (req)
#####Error starts here
    inp = urllib.request.urlopen(url)
    print (inp)
    resp = json.loads(inp)
    responese = requests.get(url)
    print (responese)
    
look_for_new_video()
    vidID = resp['items'][0]['id']['videoId']

    video_exists = False
    with open('videoid.json', 'r') as json_file:
        data = json.load(json_file)
        if data['videoId'] != vidID:
            driver = webdriver.Chrome()
            driver.get(base_video_url + vidID)
            video_exists = True

        if video_exists:
            with open('videoid.json', 'w') as json_file:
                data = {'videoId' : vidID}
                json.dump(data, json_file)

try:
    while True:
        look_for_new_video()
        time.sleep(10)
except KeyboardInterrupt:
    print ("Stop")
Here are the error messages
Error:
Traceback (most recent call last): File "C:\Users\tyu\video.py", line 24, in <module> look_for_new_video() File "C:\Users\tyu\video.py", line 18, in look_for_new_video inp = urllib.request.urlopen(url) File "c:\users\tyu\appdata\local\programs\python\python37\lib\urllib\request.py", line 222, in urlopen return opener.open(url, data, timeout) File "c:\users\tyu\appdata\local\programs\python\python37\lib\urllib\request.py", line 531, in open response = meth(req, response) File "c:\users\tyu\appdata\local\programs\python\python37\lib\urllib\request.py", line 641, in http_response 'http', request, response, code, msg, hdrs) File "c:\users\tyu\appdata\local\programs\python\python37\lib\urllib\request.py", line 569, in error return self._call_chain(*args) File "c:\users\tyu\appdata\local\programs\python\python37\lib\urllib\request.py", line 503, in _call_chain result = func(*args) File "c:\users\tyu\appdata\local\programs\python\python37\lib\urllib\request.py", line 649, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found
I did managed to find out which line caused the error but i couldn't really solve it even after surfing through the internet. So any help would be appreciated. Thanks in advance.
Line 14, you've got 4 Ws in the url: 'https://wwww.googleapis.com/youtube/v3/search?'
Drop all other modules and use only Requests as you have in import.
Call in line 15 is wrong snippet,id&order can not have ,.
Example getting subscribers count for a YouTube channel.
import requests

api_key = 'xxxxxxxxxxxxxxxxx'
channel_id = 'UC5kS0l76kC0xOzMPtOmSFGw'
response = requests.get(f'https://www.googleapis.com/youtube/v3/channels?part=statistics&id={channel_id}&key={api_key}')
Test usage:
>>> response
<Response [200]>
>>> j = response.json()
>>> j['items'][0]['statistics']['subscriberCount']
'273000'
Possible fix for your call.
import requests

api_key = 'xxxxxxxxxxxxxxxx'
channel_id = 'UC5kS0l76kC0xOzMPtOmSFGw'
response = requests.get(f'https://www.googleapis.com/youtube/v3/channels?part=snippet&order=date&maxResults=1&id={channel_id}&key={api_key}')
json_data = response.json()
# Do call on returned json_data
(Feb-13-2020, 06:03 PM)jim2007 Wrote: [ -> ]Line 14, you've got 4 Ws in the url: 'https://wwww.googleapis.com/youtube/v3/search?'

Omg, Thank you!Also, how did i missed on that...

(Feb-13-2020, 07:11 PM)snippsat Wrote: [ -> ]Possible fix for your call.

Thank you, it works!!. After that, i tried to make changes to my original code it works too but only for a few minutes.
    hdr = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36',
           'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
           'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3',
           'Accept-Encoding': 'none',
           'Accept-Language': 'en-US,en;q=0.8',
           'Connection': 'keep-alive'}
    
    base_video_url = 'https://www.youtube.com/watch?v='
    base_search_url = 'https://www.googleapis.com/youtube/v3/search?'
    url = base_search_url + 'key={}&channelId={}&part=snippet&order=date&maxResults=1'.format(api_key, channel_id)
    req = urllib.request.Request(url, headers = hdr)
    inp = urllib.request.urlopen(req).read().decode('utf-8')
    resp = json.loads(inp)
    with open('videoid.json', 'w') as txt:
        json.dump(resp, txt)
    txt.close()
    vidID = resp['items'][0]['id']['videoId']
    video_exists = False
    with open('videoid.json', 'r') as json_file:
        data = json.load(json_file)
        if data['items'][0]['id']['videoId'] != vidID:
            driver = webdriver.Chrome()
            driver.get(base_video_url + vidID)
            video_exists = True

        if video_exists:
            with open('videoid.json', 'w+') as json_file:
                data = {'videoId' : vidID}
                json.dump(data, json_file)
Then this error keep popping up ever since, even with user agent added.
Error:
Traceback (most recent call last): File "C:\Users\yee\video.py", line 46, in <module> look_for_new_video() File "C:\Users\yee\video.py", line 25, in look_for_new_video inp = urllib.request.urlopen(req).read().decode('utf-8') File "c:\users\yee\appdata\local\programs\python\python37\lib\urllib\request.py", line 222, in urlopen return opener.open(url, data, timeout) File "c:\users\yee\appdata\local\programs\python\python37\lib\urllib\request.py", line 531, in open response = meth(req, response) File "c:\users\yee\appdata\local\programs\python\python37\lib\urllib\request.py", line 641, in http_response 'http', request, response, code, msg, hdrs) File "c:\users\yee\appdata\local\programs\python\python37\lib\urllib\request.py", line 569, in error return self._call_chain(*args) File "c:\users\yee\appdata\local\programs\python\python37\lib\urllib\request.py", line 503, in _call_chain result = func(*args) File "c:\users\yee\appdata\local\programs\python\python37\lib\urllib\request.py", line 649, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden
(Mar-03-2020, 06:13 AM)ckkkkk Wrote: [ -> ]Then this error keep popping up ever since, even with user agent added.
The error message do not match the code you have posted.
I tested code i posted again it still work.
You still use urllib as it raise the 403 error,i did advice to drop urllib also do not use it as the Requests code work.