Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Loop a list and write to .xlsx
#1
This is a part of my scraper code.

There is a .txt file with some keywords (data.txt). The code is loop through all the keywords to find them on a page. The end of the code I would like to save all the results in the export.xlsx with all the keywords from data.txt

The code run fine but the save method doesn't work as i wanted.

In every loop the .xlsx file is cleared and only the new results appears in it.

Can You help me in this case? Thank You!

with open(r'C:\Users\MyName\Documents\fbm\data.txt') as f:
    for line in f:

        str1 = 'first_part_of_url'
        str2 = 'second_part_of_url'
        str3 = 'third_part_of_url'
        url = str1 + str2 + line + str3
        driver.get(url)
        time.sleep(3)

        ids =driver.find_elements_by_xpath("//a[contains(@href,'urlthing/')]")

        n=1
        links=()


        for ii in ids [ :n]:
            links+=(ii.get_attribute('href'),)

        datalist= []

        for link in links:
            driver.get(link)
            time.sleep(1)

            Keyword = driver.find_element_by_xpath('//*[@id="data"]/some_div/span').text

            df_text = [Keyword,link,line]
            datalist.append((Keyword,link,line))
            ColName = ['Keyword','link','line']

            df = pd.DataFrame(datalist, columns =['Keyword','link','line'])
            df.to_excel(r'C:\Users\MyName\Documents\fbm\export.xlsx', index= True, encoding= 'utf-8')
Reply


Messages In This Thread
Loop a list and write to .xlsx - by urobee - Apr-11-2020, 03:41 PM
RE: Loop a list and write to .xlsx - by Larz60+ - Apr-11-2020, 04:02 PM
RE: Loop a list and write to .xlsx - by urobee - Apr-11-2020, 04:24 PM
RE: Loop a list and write to .xlsx - by Larz60+ - Apr-11-2020, 09:07 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Trying to write a code to get a long list of unknown URLs hkynefin 2 2,558 Sep-16-2018, 11:44 AM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020