Python Forum
change source from csv data to gsheet
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
change source from csv data to gsheet
#1
Hey Guys,

I have tweaked a code which scrapes data from google search results and place them into a csv file called data.
I already have made an api connection with gsheet except I can't get it working. It says "TypeError: open() takes 2 positional arguments but 3 were given"

I don't know if the commands I have used are correct.

I will post the original code and what I have changed so far.

Original:
 
from urllib.parse import urlencode, urlparse, parse_qs

from lxml.html import fromstring
from requests import get
import csv

def scrape_run():
    with open('searches.txt') as searches:
        for search in searches:
           userQuery = search
           raw = get("https://www.google.com/search?q=" + userQuery).text
           page = fromstring(raw)
           links = page.cssselect('.r a')
           csvfile = 'data.csv'
           for row in links:
               raw_url = row.get('href')
               title = row.text_content()
               if raw_url.startswith("/url?"):
                   url = parse_qs(urlparse(raw_url).query)['q']
                   csvRow = [userQuery, url[0], title]
                   with open(csvfile, 'a') as data:
                       writer = csv.writer(data)
                       writer.writerow(csvRow)

scrape_run()
My edit:
import gspread
from oauth2client.service_account import ServiceAccountCredentials

scope = ['https://spreadsheets.google.com/feeds', 'https://www.googleapis.com/auth/drive']

credentials = ServiceAccountCredentials.from_json_keyfile_name('Api koppeling Python-67433e13bb00.json', scope)

gc = gspread.authorize(credentials)

wks = gc.open('Python koppeling').sheet1

from urllib.parse import urlencode, urlparse, parse_qs

from lxml.html import fromstring
from requests import get

def scrape_run():
    with open('searches.txt', encoding='utf-8') as searches:
        for search in searches:
            userQuery = search
            raw = get("https://www.google.com/search?q=" + userQuery).text
            page = fromstring(raw)
            links = page.cssselect('.r a')
            csvfile = wks
            for row in links:
                raw_url = row.get('href')
                title = row.text_content()
                if raw_url.startswith("/url?"):
                    url = parse_qs(urlparse(raw_url).query)['q']
                    wks.append_row = [userQuery, url[0], title]
                    with gc.open(csvfile, 'a') as data:
                        writer = gspread.writer(data)
                        writer.writerow(csvRow)
scrape_run()
Can you help me what I am doing wrong? :)

Thanks for your help and effort!

Tummerke
Reply
#2
Probably this line is wrong:
with gc.open(csvfile, 'a') as data:
I think it should be without: "gc."
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Help with to check an Input list data with a data read from an external source sacharyya 3 317 Mar-09-2024, 12:33 PM
Last Post: Pedroski55
Question Change elements of array based on position of input data Cola_Reb 6 2,060 May-13-2022, 12:57 PM
Last Post: Cola_Reb
  Showing data change korenron 10 2,454 Mar-20-2022, 01:50 PM
Last Post: korenron
  Load the data from multiple source files to one table amy83 2 2,521 Apr-27-2021, 12:33 AM
Last Post: Pedroski55
  Change of mass hexa data kosteloos 0 1,716 Aug-12-2019, 10:04 AM
Last Post: kosteloos
  binary data in source code form Skaperen 4 4,774 Jun-02-2017, 02:21 AM
Last Post: Skaperen

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020