How to run multiple threads with selenium? - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html) +--- Thread: How to run multiple threads with selenium? (/thread-24360.html) |
How to run multiple threads with selenium? - julio2000 - Feb-10-2020 Hey, So I've created a piece of code where my script goes to a given product url and it checks (about every second) if the product is in stock. As soon as it's instock it gives me a notification. Else it tries again. So my goal is to let 1 instance of my program, try the process described above for multiple product url's at the same time (so not after each other). So I want multiple threads that each checks if the product they are on is instock (so every thread is on a different product). How do I do this? Thanks in advance, Julius RE: How to run multiple threads with selenium? - snippsat - Feb-11-2020 schedule or APScheduler will work fine for this. You should try yourself,as i have lot of code with selenium can do a test. import schedule import time import threading from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.common.keys import Keys from bs4 import BeautifulSoup def check_stock(): options = Options() options.add_argument("--headless") options.add_argument('--disable-gpu') options.add_argument('--log-level=3') browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe', options=options) #--| Parse or automation browser.get('https://www.morningstar.com/stocks/XOSL/XXL/quote.html') time.sleep(1) soup = BeautifulSoup(browser.page_source, 'lxml') bid_size = soup.select('div.dp-value.price-down.ng-binding.ng-scope') price_sales = soup.select('li:nth-child(9) > div > div.dp-value.ng-binding') print(price_sales[0].text.strip()) def check_size(): options = Options() options.add_argument("--headless") options.add_argument('--disable-gpu') options.add_argument('--log-level=3') browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe', options=options) #--| Parse or automation browser.get('https://www.morningstar.com/stocks/XOSL/XXL/quote.html') time.sleep(2) soup = BeautifulSoup(browser.page_source, 'lxml') bid_size = soup.select('div.dp-value.price-down.ng-binding.ng-scope') price_sales = soup.select('li:nth-child(9) > div > div.dp-value.ng-binding') print(bid_size[0].text.strip()) def run_threaded(job_func): job_thread = threading.Thread(target=job_func) job_thread.start() schedule.every(60).seconds.do(run_threaded, check_stock) schedule.every(20).seconds.do(run_threaded, check_size) while True: schedule.run_pending() time.sleep(1)So calling same site at with different time interval. The bid size more often(20-sec) that sale price(60-sec).
|