Apr-11-2020, 03:41 PM
This is a part of my scraper code.
There is a .txt file with some keywords (data.txt). The code is loop through all the keywords to find them on a page. The end of the code I would like to save all the results in the export.xlsx with all the keywords from data.txt
The code run fine but the save method doesn't work as i wanted.
In every loop the .xlsx file is cleared and only the new results appears in it.
Can You help me in this case? Thank You!
There is a .txt file with some keywords (data.txt). The code is loop through all the keywords to find them on a page. The end of the code I would like to save all the results in the export.xlsx with all the keywords from data.txt
The code run fine but the save method doesn't work as i wanted.
In every loop the .xlsx file is cleared and only the new results appears in it.
Can You help me in this case? Thank You!
with open(r'C:\Users\MyName\Documents\fbm\data.txt') as f: for line in f: str1 = 'first_part_of_url' str2 = 'second_part_of_url' str3 = 'third_part_of_url' url = str1 + str2 + line + str3 driver.get(url) time.sleep(3) ids =driver.find_elements_by_xpath("//a[contains(@href,'urlthing/')]") n=1 links=() for ii in ids [ :n]: links+=(ii.get_attribute('href'),) datalist= [] for link in links: driver.get(link) time.sleep(1) Keyword = driver.find_element_by_xpath('//*[@id="data"]/some_div/span').text df_text = [Keyword,link,line] datalist.append((Keyword,link,line)) ColName = ['Keyword','link','line'] df = pd.DataFrame(datalist, columns =['Keyword','link','line']) df.to_excel(r'C:\Users\MyName\Documents\fbm\export.xlsx', index= True, encoding= 'utf-8')