Python Forum
web scraping for new additions/modifed website?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
web scraping for new additions/modifed website?
#4
(Apr-14-2022, 09:06 AM)kingoman123 Wrote: - Have some experience with java, but not with python, yet...
If want doing stuff like this most learn web-scraping in Python eg look at Web-Scraping part-1
kingoman123 Wrote Wrote:What if my task is as following:
The price is generated bye JavaScript,so most use other tool like Selenium.
To give example as the site is not easy as also most click on a couple of buttons then it generate the price.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
import time

#--| Setup
options = Options()
#options.add_argument("--headless")
ser = Service(r"C:\cmder\bin\chromedriver.exe")
browser = webdriver.Chrome(service=ser, options=options)
#--| Parse or automation
browser.get("https://www.ditur.dk/dissing-mk9-black-friday-limited-edition-d1456?linktoken=xFXOjSAR3FTGeEVN")
accept_button = browser.find_element(By.CSS_SELECTOR, 'button.coi-banner__accept')
accept_button.click()
close_button = browser.find_element(By.CSS_SELECTOR, 'div.fancybox-overlay.fancybox-overlay-fixed.ajaxcart-modal--overlay > div > div > a')
close_button.click()
time.sleep(5)
price = browser.find_element(By.CSS_SELECTOR, 'div.jataktilbud--price')
print(price.text)
Output:
799 kr
Look at this Thread for setup in Selenium v4.
Reply


Messages In This Thread
RE: web scraping for new additions/modifed website? - by snippsat - Apr-14-2022, 12:39 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Scraping lender data from Ren Ren Dai website using Python. I will pay for that 200$ Hafedh_2021 1 2,817 May-18-2021, 08:41 PM
Last Post: snippsat
  Scraping all website text using Python MKMKMKMK 1 2,156 Nov-26-2020, 10:35 PM
Last Post: Larz60+
  Scraping a Website (HELP) LearnPython2 1 1,821 May-08-2020, 03:20 PM
Last Post: Larz60+
  scraping from a website that hides source code PIWI_Protein 1 2,036 Mar-27-2020, 05:08 PM
Last Post: Larz60+
  Scraping not moving to the next pages in a website jithin123 0 2,029 Mar-23-2020, 06:10 PM
Last Post: jithin123
  Random Loss of Control of Website When Scraping bmccollum 0 1,576 Aug-30-2019, 04:04 AM
Last Post: bmccollum
  MaxRetryError while scraping a website multiple times kawasso 6 17,735 Aug-29-2019, 05:25 PM
Last Post: kawasso
  scraping multiple pages of a website. Blue Dog 14 22,691 Jun-21-2018, 09:03 PM
Last Post: Blue Dog
  Scraping number in % from website santax 3 4,577 Mar-19-2017, 12:22 PM
Last Post: santax

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020