selenium & weblinks help - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html) +--- Thread: selenium & weblinks help (/thread-30375.html) |
selenium & weblinks help - gr3yali3n - Oct-18-2020 i am playing around learning how selenium is supposed to work , i think that this is a really cool module. but i am struggling with web elements a little bit. when it comes to links.. i do not under stand why when i copy the css path , the css selector or even the xpath what i paste into the idle isnt working.. i obviously have misunderstood something about element selectors (at least when it comes to links.), i can find the search bar of google , duckduckgo and bing but when i try the links to the web sites nada.., if some one could help out. this is the last thing i was messing with just a simple open webpage but i cant even get the link to work. i guess i should also add that i am using firefox. import pyautogui from selenium import webdriver import time browser = webdriver.Firefox() browser.get('https://www.duckduckgo.com') search_bar = browser.find_element_by_css_selector('#search_form_input_homepage') search_bar.click() search_bar.send_keys('liveleak') search_bar.submit() LL = browser.find_element_by_css_selector("#r1-0 > div:nth-child(1) > div:nth-child(2) > div:nth-child(1) > a:nth-child(2) > span:nth-child(1) ") LL.click() time.sleep(5) browser.close()and i have also tried a try and except line.. from selenium import webdriver import time browser = webdriver.Firefox() browser.get('https://www.duckduckgo.com') search_bar = browser.find_element_by_css_selector('#search_form_input_homepage') search_bar.click() search_bar.send_keys('liveleak') search_bar.submit() try: LL = browser.find_element_by_css_selector("#r1-0 > div:nth-child(1) > div:nth-child(2) > div:nth-child(1) > a:nth-child(2) ") LL.click() except: ll = browser.find_element_by_css_selector("#r1-0 > div:nth-child(1) > div:nth-child(2) > div:nth-child(1) > a:nth-child(2) > span:nth-child(1)") ll.click() finally: print('neither of those worked') time.sleep(5) browser.close() RE: selenium & weblinks help - snippsat - Oct-18-2020 find_element_by_css_selector() return a list.Don't need a .click() on search_bar.Also add delay before trying to get content in link time.sleep a fast first way when testing,or look at Waits browser.get('https://duckduckgo.com') search_bar = browser.find_elements_by_css_selector('#search_form_input_homepage')[0] #search_bar.click() search_bar.send_keys('liveleak') search_bar.submit() time.sleep(3) first_link = browser.find_elements_by_css_selector('#r1-0 > div:nth-child(1) > h2:nth-child(1) > a:nth-child(1)') print(first_link[0].text)
RE: selenium & weblinks help - gr3yali3n - Oct-18-2020 (Oct-18-2020, 01:04 PM)snippsat Wrote: thank you , i was getting frustrated with this i kept going over and over the html link , i thought i was about to have to start taking html and css classes : ) just to figure this out. RE: selenium & weblinks help - gr3yali3n - Oct-21-2020 also is there a easy way to locate and compare the elements like i've seen with firefox firepath/firebug?that looked like it would be less of a pain, i am pretty much guessing which is the correct element at this point. i mean i have a good idea but i have to run the program a few times until i get the right line in there.. RE: selenium & weblinks help - snippsat - Oct-21-2020 (Oct-21-2020, 05:50 AM)gr3yali3n Wrote: also is there a easy way to locate and compare the elements like i've seen with firefox firepath/firebug?Both in Chrome and FireFox can copy CSS selector or XPath. Right click inspect,the choose a tag and right click copy, Selector or XPath. Sometime do i use ChroPath. RE: selenium & weblinks help - gr3yali3n - Nov-05-2020 just learned that you can compare elements in the devtools console by typing $$("input") this will return all of the input tags so you need to refine this further, by adding brackets [] $$("input[id='ipt2']") and the id of the element you are wanting to compare. there is away to do this with xpath also $x("") but this gets even more complicated and honestly i am still trying to figure it all out , i just want to add this to my own dumb comment from earlier, i went out and found an answer. so if anyone faces this later on i hope this helps or gives some idea on which direction to head in. |