Python Forum
Loop a list of usernames from a xlsx file
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Loop a list of usernames from a xlsx file
#1
G'day All.

I'm working on a .py that will scrape tweets from twitter and write them to a csv file: Full disclosure, i got the majority of the script from (nocodewebscraping)
However I have modified some of the code to drill down a bit further and get some specific data that i'm chasing.

The part that I am stuck on is getting the .py to loop through an xlsx file with a list of user names, as i need to scrape approx 400 names a week. 

Effectively, what I need/want it to do is this 

load.work_book('ClientList') - Use data from Cell A1, (User1) in search_name query then post (User1) as Title in (ScrapingFile) Cell A1, then post last 20 tweets in cells A2:A21 underneath Title, (then)
return to Client List Cell A2, (User2) in search_name query then post (User2) as Title in (ScrapingFile) Cell A22, then post last 20 tweets in cells A23:A44 underneath Title - rinse/repeat.

I will freely admit, I've spent the better part of 2 days scratching my noggin, and have searched git/stack/reddit/google/here and have not had much luck, perhaps it is my wording or misunderstanding of something basic, but i would love some assistance.

I am using Atom.Io, Windows 7, and CSV, Tweepy & Openpyxl 

Thanks in advance!

import tweepy 
import csv
import openpyxl

    consumer_key = Secret
    consumer_secret = Secret
    access_key = Secret
    access_secret = Secret

# This is where i "think" i use the "FOR" loop, but freely admit to being lost
    def get_all_tweets(screen_name):

    auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
    auth.set_access_token(access_key, access_secret)
    api = tweepy.API(auth)
    #Limit tweets to last 20 *modifiable*
    for tweet in tweepy.Cursor(api.search, q="google", rpp=1, count=20, result_type="recent", include_entities=True, lang="en") .items(20):


    alltweets = []

#First instance of screen_name being repeated, however i assume i simply change screen_name= to whatever the for loop is up on line 11
    new_tweets = api.user_timeline(screen_name = screen_name,count=200)


    alltweets.extend(new_tweets)


    oldest = alltweets[-1].id - 1


    while len(new_tweets) > 0:
    print "getting tweets before %s" % (oldest)

#Second instance of screen_name being repeated, however i assume i simply change screen_name= to whatever the for loop is up on line 11
    new_tweets = api.user_timeline(screen_name = screen_name,count=200,max_id=oldest)

    alltweets.extend(new_tweets)


    oldest = alltweets[-1].id - 1

    print "...%s tweets downloaded so far" % (len(alltweets))

# Hashtag modifiers to drill further into tweets content
    outtweets = [[tweet.id_str, tweet.created_at, tweet.text.encode("utf-8")]for tweet in alltweets if '#firsttag' if 'secondtag' in tweet.text.encode("utf-8")]

#I want to create a CSV file, that will dump all records into ScrapingFile.csv once it has looped all names in ClientList.xlsx
#format will be name from Clientlist CellA1, then 20 tweets then name from ClientList Cell A2 (rinse repeat)
    with open('%s_tweets.csv' % ScrapingFile, 'wb') as f:
    writer = csv.writer(f)
    writer.writerow(["id","created_at","text","retweet_count","favorite_count"])
    writer.writerows(outtweets)

    pass


    if __name__ == '__main__':

#final instance of screen_name being repeated, however i assume i simply change screen_name= to whatever the for loop is up on line 11
    get_all_tweets("screen_name")
Reply
#2
http://www.pythonexcel.com/openpyxl.php
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  python Read each xlsx file and write it into csv with pipe delimiter mg24 4 1,308 Nov-09-2023, 10:56 AM
Last Post: mg24
  Convert legacy print file to XLSX file davidm 1 1,768 Oct-17-2021, 05:08 AM
Last Post: davidm
  guys please help me , pycharm is not able to identify my xlsx file CrazyGreenYT7 1 1,971 Jun-13-2021, 02:22 PM
Last Post: Larz60+
  Python3 doesn't populate xlsx file with openpyxl Auldyin75 2 2,489 Feb-16-2021, 12:00 PM
Last Post: Auldyin75
  Appending to list of list in For loop nico_mnbl 2 2,319 Sep-25-2020, 04:09 PM
Last Post: nico_mnbl
  Append list into list within a for loop rama27 2 2,308 Jul-21-2020, 04:49 AM
Last Post: deanhystad
  Call a .xlsx file outside a class criscferr 2 1,823 Apr-24-2020, 04:23 PM
Last Post: criscferr
  Writing list as a file, then reading that file as a list Zoastria_Balnala 3 2,551 Oct-17-2019, 07:54 PM
Last Post: Zoastria_Balnala
  I need help using Python to generate usernames and passwords with excel documents Jannejannesson 3 3,947 May-08-2019, 02:30 PM
Last Post: Jannejannesson
  Getting error while loading excel(.xlsx) file using openpyxl module shubhamjainj 1 8,899 Mar-01-2019, 01:05 PM
Last Post: buran

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020