scraping webiste using pandas and googletranslation dont compile on the terminal - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: General Coding Help (https://python-forum.io/forum-8.html) +--- Thread: scraping webiste using pandas and googletranslation dont compile on the terminal (/thread-39550.html) |
scraping webiste using pandas and googletranslation dont compile on the terminal - gcozba2023 - Mar-06-2023 Hi Team I need some help with my script, i am trying to scrap the website but when i try to use data frame to translate the text from the website its not running but its crashing. Here is my terminal output below... // Code to execute this.. from googletrans import Translator from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions import pandas as pd import urllib.request as urllib2 import time chrome_options = webdriver.ChromeOptions() chrome_options.add_argument('--headless') chrome_options.add_argument("window-size=1920,1080") driver = webdriver.Chrome(options=chrome_options) url = "https://www.classcentral.com/collection/top-free-online-courses" driver.get(url) translator = Translator() try: while True: # wait until button is clickable WebDriverWait(driver, 1).until( expected_conditions.element_to_be_clickable((By.XPATH, "//button[@data-name='LOAD_MORE']")) ).click() time.sleep(0.5) except Exception as e: pass all_courses = driver.find_element(by=By.CLASS_NAME, value='catalog-grid__results') courses = all_courses.find_elements(by=By.CSS_SELECTOR, value='[class="color-charcoal course-name"]') df = pd.DataFrame([[course.text, course.get_attribute('href')] for course in courses], columns=['Title (eng)', 'Link']) df['Title (hin)'] = df['Title (eng)'].apply(lambda x: translator.translate(x, dest='hi').text) print(df)// output from the terminal
RE: scraping webiste using pandas and googletranslation dont compile on the terminal - snippsat - Mar-06-2023 The problem is with googletrans Do this. pip uninstall googletransThen. pip install googletrans==3.1.0a0Test of your code. df['Title (hin)'] = df['Title (eng)'].apply(lambda x: translator.translate(x, dest='hi').text)No error and now is translated to Hindi. >>> df['Title (hin)'] 0 मेडिकल पैरासिटोलॉजी | मेडिकल पैरासिटोलॉजी 1 मेडिकल रिसर्च को समझना: आपका फेसबुक फ्रेंड गलत है 2 पायथन में इंटरएक्टिव प्रोग्रामिंग का परिचय (भा... 3 पर्वत 101 4 सभी के लिए क्वांटम यांत्रिकी 5 डिमेंशिया को समझना 6 आधुनिक और समकालीन अमेरिकी कविता ("मॉडपो") 7 सीखना कैसे सीखें: कठिन विषयों में महारत हासिल ... 8 एक दिमागी जीवन बनाए रखना 9 मल्टीपल स्केलेरोसिस (एमएस) को समझना 10 असामान्य ज्ञान शिक्षण 11 विपणन अभिनव उत्पादों और सेवाओं 12 एक पेशेवर की तरह सीखें: किसी भी चीज़ में बेहतर... 13 जीव विज्ञान का परिचय - जीवन का रहस्य 14 भलाई और शिखर प्रदर्शन के लिए सचेतनता Name: Title (hin), dtype: object RE: scraping webiste using pandas and googletranslation dont compile on the terminal - gcozba2023 - Mar-06-2023 (Mar-06-2023, 07:23 PM)snippsat Wrote: The problem is with googletrans Thank you so much, you are lifesaver, came to the right forum. Its now running as way expected with no issues at all. |