Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
why I can't scrape a website?
#3
Thanks for the response, Metulburr. I really appreciate it. I could scrape the page.
I will take a look into Selenium and tutorials you have mentioned to get better.

I have one more question.

I typed in 'vr' in a search box from https://map.naver.com/ but the address doesn't get changed. I was expecting longer address instead of just https://map.naver.com/

So I clicked onto one of the names of the result and ctrl+shift+i > network tab > searched for a request including arcade information which is added to my code below.

My question is how I can scrape this page? I used the header but I've got an unexpected result.

import requests
from bs4 import BeautifulSoup

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36"
}
        

url = 'https://map.naver.com/search2/local.nhn?sm=hty&searchCoord=127.1302174%3B37.4119829&isFirstSearch=true&query=vr&menu=location&mpx=02135620%3A37.4119829%2C127.1302174%3AZ11%3A0.0195464%2C0.0085920'
req = requests.get(url, headers=headers)
bs = BeautifulSoup(req.text, 'html.parser')
print(bs)
Error:
{ "error": { "code": "HMAC_AUTH_FAILED", "msg": "Hmac Authentication has failed.", "extraInfo": null } }
Reply


Messages In This Thread
why I can't scrape a website? - by kmkim319 - Sep-18-2019, 03:46 PM
RE: why I can't scrape a website? - by metulburr - Sep-18-2019, 06:03 PM
RE: why I can't scrape a website? - by kmkim319 - Sep-20-2019, 06:07 PM
RE: why I can't scrape a website? - by metulburr - Sep-20-2019, 06:38 PM
RE: why I can't scrape a website? - by snippsat - Sep-20-2019, 07:41 PM
RE: why I can't scrape a website? - by kmkim319 - Sep-25-2019, 03:49 AM
RE: why I can't scrape a website? - by metulburr - Sep-25-2019, 11:31 AM
RE: why I can't scrape a website? - by kmkim319 - Sep-27-2019, 03:14 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Unable to Scrape Website muhamdasim 2 2,627 Dec-27-2021, 07:49 PM
Last Post: JohnRaven
  how to scrape a website from a keyword list greenpine 2 2,394 Dec-04-2020, 03:50 PM
Last Post: greenpine
  scrape data 1 go to next page scrape data 2 and so on alkaline3 6 5,214 Mar-13-2020, 07:59 PM
Last Post: alkaline3
  Read url from CSV and Scrape website Prince_Bhatia 3 10,277 Jan-08-2020, 09:08 AM
Last Post: binaryanimal
  How do i scrape website whose page changes using javsacript _dopostback function and Prince_Bhatia 1 7,247 Aug-06-2018, 09:45 AM
Last Post: wavic
  Scrape A tags from a website Prince_Bhatia 1 4,232 Oct-15-2017, 12:56 AM
Last Post: metulburr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020