Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Selenium TypeError
#1
Hi,my code give TypeError.Can you help me?
from selenium import webdriver
import random
import time
browser= webdriver.Firefox()

url="https://eksisozluk.com/mustafa-kemal-ataturk--34712?p="
pagecount=1
entries=[]
entrycount=1
while pagecount<=10:
    ramdonpage=random.randint(1,1290)
    newurl=url+str(ramdonpage)
    browser.get(newurl)

    elements=browser.Find_Elements_By_Css_Selector(".content")
    for element in elements:
        entries.append(element.text)
    time.sleep(3)
    pagecount+=1

for entry in entries:
    print("*************************************")
    print(entry)
,

Error:
Traceback (most recent call last): File "...", line 16, in <module> for element in elements: TypeError: 'NoneType' object is not iterable
Reply
#2
There is problem with your setup here @charset.
Find_Elements_By_Css_Selector it should not be capitalized,will give AttributeError.
Here is more modern way to set up,here use Firefox an these setup option may be required,
it can also depend on version of driver and Selenium which should be the newest.
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.common.keys import Keys
import time, random

#--| Setup
options = Options()
#options.add_argument("--headless")
caps = webdriver.DesiredCapabilities().FIREFOX
caps["marionette"] = True
browser = webdriver.Firefox(options=options, capabilities=caps, executable_path=r"geckodriver.exe")
#--| Parse or automation
rand_page = random.randint(1, 5)
url = f"https://eksisozluk.com/mustafa-kemal-ataturk--34712?p={rand_page}"
browser.get(url)
time.sleep(2)
content = browser.find_elements_by_css_selector('.content')
print(content)
Work in test,content will return a list of Selenium objects.
So it call .text on it,will see content.
>>> content[0].text
'(bkz: ubermensch)'
>>> content[1].text
'(bkz: deha)\n(bkz: dusunce)\n(bkz: gorus)\n(bkz: zeka)\n(bkz: basari)'
>>> content[2].text
("kutsal kehanetlerde adi gecen '19 yy da ortadogudan bir adam cikacak ve tum "
 "dunyaya kafa tutacak' dedigi , yuce insan, 4 kusak ileriyi gorebilen, esi "
 'bulunmaz bir yasam insani, arif, yol gosterici')
Reply
#3
Also there is no need to use Selenium here,should be used when needed if not it's just extra overhead and more resources used.
Need to set user agent then it work.
from bs4 import BeautifulSoup
import requests
import random

rand_page = random.randint(1, 5)
url = f'https://eksisozluk.com/mustafa-kemal-ataturk--34712?p={rand_page}'
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, 'lxml')
p_tag = soup.select_one('.content')
print(p_tag.text.strip())
Output:
bu adam olmasaydı ne ırkımız kalırdı... ne dinimiz... nede adımız...entry'yi kötüleyenler için, elinde de o mouse'un tuşu yerine, işgal güçlerinin sana verdiği kürek, çekiç olurdu onunla çalışırdın angut !
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Error in Selenium: CRITICAL:root:Selenium module is not installed...Exiting program. AcszE 1 3,587 Nov-03-2017, 08:41 PM
Last Post: metulburr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020