Python Forum

headers = {'User-Agent': 'IDK IF THIS IS SMART TO SHARE SO I'M NOT SHARING IT:)}
gourl = requests.get('https://www.jdsports.nl/product/wit-jordan-air-1-low-heren/132976_jdsportsnl/', headers=headers)
page = bs4.BeautifulSoup(gourl.text, 'html.parser')
page.select('#productSizeStock .btn-default:nth-child(1)')

I want to click the button '#productSizeStock .btn-default:nth-child(1)'. But if it's not possible to click the button, I would like to receive a error.

But if I run this piece of code with a random text on the last line (page.select('...'), it doesn't give an error, even though the thing I'm asking requests to select, doesnt exist. Is this an error in my piece of code or is that just a characteristic of requests?

Thanks in advance! :)

you can do it using selenium, but the XPath seems to be wrong:

from selenium import webdriver
from selenium.webdriver.common.by import By
from bs4 import BeautifulSoup
import time


class PushButton:
    def __init__(self):
        pass        

    def start_browser(self):
        caps = webdriver.DesiredCapabilities().FIREFOX
        caps["marionette"] = True
        return webdriver.Firefox(capabilities=caps)

    def stop_browser(self, browser):
        browser.close()
    
    def process_page(self):
        browser = None

        url = 'https://www.jdsports.nl/product/wit-jordan-air-1-low-heren/132976_jdsportsnl/'
        browser = self.start_browser()
        browser.get(url)
        time.sleep(2)

        browser.find_element(By.XPATH, '#productSizeStock .btn-default:nth-child(1)').click()
        self.stop_browser(browser)
        

if __name__ == '__main__':
    pb = PushButton()
    pb.process_page()

Yes I know it's possible with selenium. But i believe requests is a bit faster, am I right? And speed is key in the thing I want to program:)

Quote:Yes I know it's possible with selenium. But i believe requests is a bit faster,

True, but how do you click a button with requests?
I sometimes use selenium to get past the javaScript, then switch to BeautifulSoup

(Jan-23-2020, 04:14 AM)Larz60+ Wrote: [ -> ]

Quote:Yes I know it's possible with selenium. But i believe requests is a bit faster,

True, but how do you click a button with requests? I sometimes use selenium to get past the javaScript, then switch to BeautifulSoup

Ohh well yeah, that's possible aswell. I used bs4 and then: page.select -> I believe this will select a button

I often favor selenium now over other methods as I dont have to rewrite scrapers if it contains javascript. I have never noticed a slow down opposed to requests if you write it correctly without bottlenecks

(Jan-24-2020, 02:06 PM)metulburr Wrote: [ -> ]I often favor selenium now over other methods as I dont have to rewrite scrapers if it contains javascript. I have never noticed a slow down opposed to requests if you write it correctly without bottlenecks

ahh okay. I'm sorry, but what do you mean with 'bottlenecks'?