Python Forum
urllib.error.HTTPError: HTTP Error 404: Not Found
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
urllib.error.HTTPError: HTTP Error 404: Not Found
#1
This is the first time i am using the Google API key and the code is suppose to open up the browser when there is a new video uploaded from the channel. But am getting the http error. Did some searches and some forum actually recommended using the user agent, but no luck making it works.

Here is the code
import urllib3, json
import urllib.request
from selenium import webdriver
import time
import requests

def look_for_new_video():
    headers = {}
    headers['User-Agent'] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"
    api_key = "KEY_REMOVED"
    channel_id = "UCWr0mx597DnSGLFk1WfvSkQ"
    
    base_video_url = 'https://www.youtube.com/watch?v='
    base_search_url = 'https://wwww.googleapis.com/youtube/v3/search?'
    url = base_search_url + 'key={}&channelId={}&part=snippet,id&order=date&maxResults=1'.format(api_key, channel_id)
    req = urllib.request.Request(url, headers = headers)
    print (req)
#####Error starts here
    inp = urllib.request.urlopen(url)
    print (inp)
    resp = json.loads(inp)
    responese = requests.get(url)
    print (responese)
    
look_for_new_video()
    vidID = resp['items'][0]['id']['videoId']

    video_exists = False
    with open('videoid.json', 'r') as json_file:
        data = json.load(json_file)
        if data['videoId'] != vidID:
            driver = webdriver.Chrome()
            driver.get(base_video_url + vidID)
            video_exists = True

        if video_exists:
            with open('videoid.json', 'w') as json_file:
                data = {'videoId' : vidID}
                json.dump(data, json_file)

try:
    while True:
        look_for_new_video()
        time.sleep(10)
except KeyboardInterrupt:
    print ("Stop")
Here are the error messages
Error:
Traceback (most recent call last): File "C:\Users\tyu\video.py", line 24, in <module> look_for_new_video() File "C:\Users\tyu\video.py", line 18, in look_for_new_video inp = urllib.request.urlopen(url) File "c:\users\tyu\appdata\local\programs\python\python37\lib\urllib\request.py", line 222, in urlopen return opener.open(url, data, timeout) File "c:\users\tyu\appdata\local\programs\python\python37\lib\urllib\request.py", line 531, in open response = meth(req, response) File "c:\users\tyu\appdata\local\programs\python\python37\lib\urllib\request.py", line 641, in http_response 'http', request, response, code, msg, hdrs) File "c:\users\tyu\appdata\local\programs\python\python37\lib\urllib\request.py", line 569, in error return self._call_chain(*args) File "c:\users\tyu\appdata\local\programs\python\python37\lib\urllib\request.py", line 503, in _call_chain result = func(*args) File "c:\users\tyu\appdata\local\programs\python\python37\lib\urllib\request.py", line 649, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found
I did managed to find out which line caused the error but i couldn't really solve it even after surfing through the internet. So any help would be appreciated. Thanks in advance.
Reply
#2
Line 14, you've got 4 Ws in the url: 'https://wwww.googleapis.com/youtube/v3/search?'
There is no passion to be found playing small - in settling for a life that is less than the one you are capable of living.
Reply
#3
Drop all other modules and use only Requests as you have in import.
Call in line 15 is wrong snippet,id&order can not have ,.
Example getting subscribers count for a YouTube channel.
import requests

api_key = 'xxxxxxxxxxxxxxxxx'
channel_id = 'UC5kS0l76kC0xOzMPtOmSFGw'
response = requests.get(f'https://www.googleapis.com/youtube/v3/channels?part=statistics&id={channel_id}&key={api_key}')
Test usage:
>>> response
<Response [200]>
>>> j = response.json()
>>> j['items'][0]['statistics']['subscriberCount']
'273000'
Possible fix for your call.
import requests

api_key = 'xxxxxxxxxxxxxxxx'
channel_id = 'UC5kS0l76kC0xOzMPtOmSFGw'
response = requests.get(f'https://www.googleapis.com/youtube/v3/channels?part=snippet&order=date&maxResults=1&id={channel_id}&key={api_key}')
json_data = response.json()
# Do call on returned json_data
Reply
#4
(Feb-13-2020, 06:03 PM)jim2007 Wrote: Line 14, you've got 4 Ws in the url: 'https://wwww.googleapis.com/youtube/v3/search?'

Omg, Thank you!Also, how did i missed on that...

(Feb-13-2020, 07:11 PM)snippsat Wrote: Possible fix for your call.

Thank you, it works!!. After that, i tried to make changes to my original code it works too but only for a few minutes.
    hdr = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36',
           'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
           'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3',
           'Accept-Encoding': 'none',
           'Accept-Language': 'en-US,en;q=0.8',
           'Connection': 'keep-alive'}
    
    base_video_url = 'https://www.youtube.com/watch?v='
    base_search_url = 'https://www.googleapis.com/youtube/v3/search?'
    url = base_search_url + 'key={}&channelId={}&part=snippet&order=date&maxResults=1'.format(api_key, channel_id)
    req = urllib.request.Request(url, headers = hdr)
    inp = urllib.request.urlopen(req).read().decode('utf-8')
    resp = json.loads(inp)
    with open('videoid.json', 'w') as txt:
        json.dump(resp, txt)
    txt.close()
    vidID = resp['items'][0]['id']['videoId']
    video_exists = False
    with open('videoid.json', 'r') as json_file:
        data = json.load(json_file)
        if data['items'][0]['id']['videoId'] != vidID:
            driver = webdriver.Chrome()
            driver.get(base_video_url + vidID)
            video_exists = True

        if video_exists:
            with open('videoid.json', 'w+') as json_file:
                data = {'videoId' : vidID}
                json.dump(data, json_file)
Then this error keep popping up ever since, even with user agent added.
Error:
Traceback (most recent call last): File "C:\Users\yee\video.py", line 46, in <module> look_for_new_video() File "C:\Users\yee\video.py", line 25, in look_for_new_video inp = urllib.request.urlopen(req).read().decode('utf-8') File "c:\users\yee\appdata\local\programs\python\python37\lib\urllib\request.py", line 222, in urlopen return opener.open(url, data, timeout) File "c:\users\yee\appdata\local\programs\python\python37\lib\urllib\request.py", line 531, in open response = meth(req, response) File "c:\users\yee\appdata\local\programs\python\python37\lib\urllib\request.py", line 641, in http_response 'http', request, response, code, msg, hdrs) File "c:\users\yee\appdata\local\programs\python\python37\lib\urllib\request.py", line 569, in error return self._call_chain(*args) File "c:\users\yee\appdata\local\programs\python\python37\lib\urllib\request.py", line 503, in _call_chain result = func(*args) File "c:\users\yee\appdata\local\programs\python\python37\lib\urllib\request.py", line 649, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden
Reply
#5
(Mar-03-2020, 06:13 AM)ckkkkk Wrote: Then this error keep popping up ever since, even with user agent added.
The error message do not match the code you have posted.
I tested code i posted again it still work.
You still use urllib as it raise the 403 error,i did advice to drop urllib also do not use it as the Requests code work.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  What is the error of not being able to pull data in this code? i didn't see an error? TestPerson 2 1,183 Sep-30-2022, 02:36 PM
Last Post: DeaD_EyE
  Getting from <td> tag by using urllib,Beautifulsoup KuroBuster 2 2,027 Aug-20-2021, 07:53 AM
Last Post: KuroBuster
  HTTP 404 error with Session Pool Clives 0 1,285 Jun-17-2021, 06:45 PM
Last Post: Clives
  error HTTP Error 403: Forbidden local_bit 1 2,781 Nov-14-2020, 11:34 AM
Last Post: ndc85430
  Beginner: urllib error tomfry 7 6,466 May-03-2020, 04:35 AM
Last Post: Larz60+
  python beginner HTTP Error 500 leofcastro 0 2,134 Jan-24-2020, 04:37 PM
Last Post: leofcastro
  HTTP error 404 Karin 4 4,656 May-31-2019, 02:23 PM
Last Post: snippsat
  How to check HTTP error 500 and bypass SriMekala 3 10,375 May-04-2019, 02:07 PM
Last Post: snippsat
  SSLCertVerificationError using urllib (urlopen) FalseFact 1 5,834 Mar-31-2019, 08:34 AM
Last Post: snippsat
  Error: module 'urllib' has no attribute 'urlopen' mitmit293 2 14,952 Jan-29-2019, 02:32 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020