Python Forum
Create Dictionary List (From a webpage dropdown) for Comparison to a CSV File
Thread Rating:
  • 1 Vote(s) - 3 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Create Dictionary List (From a webpage dropdown) for Comparison to a CSV File
#1
Hi, i have the below code working - which will output a list to a csv file...  but, I want to store the list as a dictionary so that I can then read a locally stored csv file and create a second dictionary, then match each item (row) to the best match of the list I am creating below...(using fuzzywuzzy) can anyone help me out?

hope that makes sense... if not, my short answer is how do I make the list that is being generated below as a dictionary instead of a cs

elements = driver.find_elements_by_xpath("//ul[@id='AssociationList_listbox']/li")
driver.elements = elements
outputfile = open('C:\\Users\\bgutt\\desktop\\Output.csv', 'wb')
writer = csv.writer(outputfile)
with outputfile as output:
     for element in elements:
          data = str(element.text)
          writer.writerow([data])
Moderator zivoni: please use code tags for future posts
Reply
#2
I dont know what do you want to use as the keys in your dictionary, but equivalent of your csv would be probably list containing your "rows"

elements = driver.find_elements_by_xpath("//ul[@id='AssociationList_listbox']/li")
elements_list = []
for element in elements:
      elements_list.append(str(element.text))
If you had some unique (and hashable) key for each element, you could just change elements_list to a dictionary and instead of appending use elements_dict[key] = str(element.text)
Reply
#3
Hi, thanks for the reply. I am using this list to compare to a saved csv, which has 3 "columns", then using fuzzywuzzy to select the best match for each item per line, in column 2, of the saved csv.

With that match, ill be selecting the appropriate drop down item, from the list (or dictionary) we just created.

Will your suggestion work for that? Im very new to python so I appreciate you taking the time to help!
Reply
#4
I dont know exactly what your data is, but elements_list should contain all text of all items from elements, in same order. I think that it could work for you, but you need to try it ...
Reply
#5
Okay, that works.  --- So, below is what I have so far:

Now, I need to:

-Create a list from a csv located here: "C:\users\bgutt\desktop\input.csv" which has 3 columns (File Number, Association, Address) - Create a dictionary, or a list from the 2nd column, lets call that "Association list"

-Compare each item of the "association list" to the elements_list using fuzzywuzzy - and select the item from the elements_list that has the highest percent confidence


# -*- coding: utf-8 -*-
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoAlertPresentException
from itertools import izip
import unittest, csv, StringIO
class Here(unittest.TestCase):
    def setUp(self):
        self.driver = webdriver.Firefox()
        self.driver.implicitly_wait(30)
        self.base_url = "my base url"
        self.verificationErrors = []
        self.accept_next_alert = True
    
    def test_here(self):
        driver = self.driver
        driver.get("mywebsite")
        self.assertEqual("", driver.title)
        driver.find_element_by_id("MainContent_Login1_ErlLogin_UserName").clear()
        driver.find_element_by_id("MainContent_Login1_ErlLogin_UserName").send_keys("mylogin")
        driver.find_element_by_id("MainContent_Login1_ErlLogin_Password").clear()
        driver.find_element_by_id("MainContent_Login1_ErlLogin_Password").send_keys("mypassword")
        driver.find_element_by_id("MainContent_Login1_ErlLogin_LoginButton").click()
        driver.find_element_by_css_selector("span.k-input").click()
        elements = driver.find_elements_by_xpath("//ul[@id='AssociationList_listbox']/li")
        driver.elements = elements
        elements_list = []
        with outputfile as output:
            for element in elements:
                data = str(element.text)
                elements_list.append(str(element.text))
            print elements_list
    def is_element_present(self, how, what):
        try: self.driver.find_element(by=how, value=what)
        except NoSuchElementException as e: return False
        return True
    
    def is_alert_present(self):
        try: self.driver.switch_to_alert()
        except NoAlertPresentException as e: return False
        return True
    
    def close_alert_and_get_its_text(self):
        try:
            alert = self.driver.switch_to_alert()
            alert_text = alert.text
            if self.accept_next_alert:
                alert.accept()
            else:
                alert.dismiss()
            return alert_text
        finally: self.accept_next_alert = True
    
    def tearDown(self):
        self.driver.quit()
        self.assertEqual([], self.verificationErrors)

if __name__ == "__main__":
    unittest.main()
Reply
#6
alright, so I have now got a dictionary list from the csv (will call it lookupdictionary for reference)... and a list from the webpage dropdown (will call it matchlist for reference)
________________________________________________________________________________________________
the lookupdictionary is in this format:

{'135': {'creditor': 'MY ASSOCIATION1', 'propert_add': '1111 PROPERTY DR'}, '134': {'creditor': 'CREDITOR2', 'propert_add': '2222 PROPERTY WAY'},  ### ETC.}

This lookupdictionary will be 2-3,000 items
________________________________________________________________________________________________
The matchlist is in this format:

['MY ASSOCIATION1', "MYASSOCIATION2", 'MYASSOCIATION3', 'MYASSOCIATON4', ### ETC.]

The matchlist is around 150 items

* Note that some of these are surrounded by " and some are surrounded by '

____________________________________________________________________________________________

I want to use fuzzywuzzy to loop through each item in the lookupdictionary and search for the "Creditor" Name

                        {'135': {'creditor': 'MY ASSOCIATION1', 'propert_add': '1111 PROPERTY DR'},

and then return the top match from the matchlist

_______________________________________________________________________________________________

Can anyone help me with this last piece??  below is the current code:

# -*- coding: utf-8 -*-
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoAlertPresentException
from itertools import izip
import unittest, csv, StringIO
from fuzzywuzzy import process

class Here(unittest.TestCase):
    def setUp(self):
        self.driver = webdriver.Chrome()
        self.driver.implicitly_wait(30)
        self.base_url = "http://mybaseurl/"
        self.verificationErrors = []
        self.accept_next_alert = True
    
    def test_here(self):
        driver = self.driver
        driver.get("https://mybaseurl/etc.")
        self.assertEqual("", driver.title)
        driver.find_element_by_id("MainContent_Login1_ErlLogin_UserName").clear()
        driver.find_element_by_id("MainContent_Login1_ErlLogin_UserName").send_keys("My Username")
        driver.find_element_by_id("MainContent_Login1_ErlLogin_Password").clear()
        driver.find_element_by_id("MainContent_Login1_ErlLogin_Password").send_keys("My Password")
        driver.find_element_by_id("MainContent_Login1_ErlLogin_LoginButton").click()
        self.driver.implicitly_wait(45)
        driver.find_element_by_css_selector("span.k-input").click()
        elements = driver.find_elements_by_xpath("//ul[@id='AssociationList_listbox']/li")
        driver.elements = elements
        elements_list = []
        outputfile = open('temp.csv', 'wb')
        with outputfile as output:
            for element in elements:
                elements_list.append(str(element.text))
            print elements_list

        reader = csv.DictReader(open('c:\\users\\bgutt\\desktop\\input.csv'))
        
        input_dict = {}
        for row in reader:
            key = row.pop('ourfile')
            if key in input_dict:
                pass
            input_dict[key] = row
        print input_dict
        
    ######################IM HERE##############################

    def is_element_present(self, how, what):
        try: self.driver.find_element(by=how, value=what)
        except NoSuchElementException as e: return False
        return True
    
    def is_alert_present(self):
        try: self.driver.switch_to_alert()
        except NoAlertPresentException as e: return False
        return True
    
    def close_alert_and_get_its_text(self):
        try:
            alert = self.driver.switch_to_alert()
            alert_text = alert.text
            if self.accept_next_alert:
                alert.accept()
            else:
                alert.dismiss()
            return alert_text
        finally: self.accept_next_alert = True
    
    def tearDown(self):
        self.driver.quit()
        self.assertEqual([], self.verificationErrors)

if __name__ == "__main__":
    unittest.main()
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Generating dynamic dropdown list test 1 1,428 Aug-30-2023, 08:00 AM
Last Post: blessinguvula
  Python Selenium (Dropdown-) data Robin_at_Cantelli 2 6,158 Dec-29-2021, 03:16 AM
Last Post: ondreweil
  Selenium Python for Dropdown not working gj31980 1 2,612 Oct-27-2020, 02:02 AM
Last Post: gj31980
  Dropdown interact moisesfelipee 0 1,659 May-04-2020, 01:11 AM
Last Post: moisesfelipee
  While loop skips multiple dropdown menu options and then works as intended newbie_programmer 1 2,842 Dec-23-2019, 10:26 PM
Last Post: keuninkske
  Create .exe file for Python flask website. vintysaw 4 18,957 Nov-18-2019, 07:56 AM
Last Post: tonycstech
  How can i scrape dropdown value ? caca 0 2,945 Nov-03-2019, 11:24 PM
Last Post: caca
  Python Flask Dependent Dropdown Anfaa 3 18,368 Oct-24-2018, 09:35 PM
Last Post: nilamo
  Click dropdown menu option with Selenium PyChrome AcszE 3 5,870 Oct-26-2017, 10:07 PM
Last Post: metulburr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020