Python Forum
Thread Rating:
  • 1 Vote(s) - 1 Average
  • 1
  • 2
  • 3
  • 4
  • 5
new to this
#1
when i try to run this python script i get this message in CMD, could anyone help me fix these issues please ?

Quote:Traceback (most recent call last):
File "C:\Users\Dan Coombes\Downloads\getfollowers.py", line 48, in <module>
getFollowers(csvFile)
File "C:\Users\Dan Coombes\Downloads\getfollowers.py", line 32, in getFollowers
csvWriter.writerow([user.screen_name, user.followers_count, user.location])
File "C:\Users\Dan Coombes\AppData\Local\Programs\Python\Python37-32\lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u0130' in position 14: character maps to <undefined>

Here is the script
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
import tweepy, time, csv
 
consumerKey = "HIDDEN"
consumerSecretKey = "HIDDEN"
accessToken = "HIDDEN"
accessTokenSecret = "HIDDEN"
 
filePath = "results.csv"
 
csvHeader = ['Username', 'Followers', 'Location']
userFound = False
user = None
 
def authenticate():
 
    auth = tweepy.OAuthHandler(consumerKey, consumerSecretKey)
    auth.set_access_token(accessToken, accessTokenSecret)
 
    return tweepy.API(auth, wait_on_rate_limit=True)
 
def getFollowers(csvFile):
    csvWriter = csv.writer(csvFile)
    csvWriter.writerow(csvHeader)
 
    users = tweepy.Cursor(api.followers, screen_name=username).items()
 
    while True:
        try:
            user = next(users)
        except StopIteration:
            break
        csvWriter.writerow([user.screen_name, user.followers_count, user.location])
        csvFile.flush()
 
api = authenticate()
 
while userFound == False:
    username = input("Enter twitter username: ")
 
    try:
        user = api.get_user(username)
        userFound = True
    except Exception as e:
        print("User not found - try again")
        continue
 
with open('results.csv', 'w') as csvFile:
    getFollowers(csvFile)
Reply
#2
Try:
1
2
with open('results.csv', 'w', encoding='utf-8') as csvFile:
    getFollowers(csvFile)
Reply
#3
(Jan-08-2019, 02:45 PM)snippsat Wrote: Try:
1
2
with open('results.csv', 'w', encoding='utf-8') as csvFile:
    getFollowers(csvFile)

thank you, the script seems to be working, letting it run via CMD. are you good with twitter scripts ?

is there anyway to add a filter that user scraped doesnt have 2000+ followers dont exract ?
Reply
#4
(Jan-08-2019, 02:56 PM)danbryn16 Wrote: are you good with twitter scripts ?
I would say no,as i only use Twitter as news source and never post there.
But it's Python and network(web HTTP) and there i have some knowledge.
(Jan-08-2019, 02:56 PM)danbryn16 Wrote: is there anyway to add a filter that user scraped doesnt have 2000+ followers dont exract ?
It's depend on what tweepy API has build in.
If i take a quick look at doc.
1
2
3
4
5
6
7
8
9
10
11
12
13
# In this example, the handler is time.sleep(15 * 60),
# but you can of course handle it in any way you want.
 
def limit_handled(cursor):
    while True:
        try:
            yield cursor.next()
        except tweepy.RateLimitError:
            time.sleep(15 * 60)
 
for follower in limit_handled(tweepy.Cursor(api.followers).items()):
    if follower.friends_count < 2000:
       # Do scraping here
Reply
#5
(Jan-08-2019, 03:55 PM)snippsat Wrote:
(Jan-08-2019, 02:56 PM)danbryn16 Wrote: are you good with twitter scripts ?
I would say no,as i only use Twitter as news source and never post there.
But it's Python and network(web HTTP) and there i have some knowledge.
(Jan-08-2019, 02:56 PM)danbryn16 Wrote: is there anyway to add a filter that user scraped doesnt have 2000+ followers dont exract ?
It's depend on what tweepy API has build in.
If i take a quick look at doc.
1
2
3
4
5
6
7
8
9
10
11
12
13
# In this example, the handler is time.sleep(15 * 60),
# but you can of course handle it in any way you want.
 
def limit_handled(cursor):
    while True:
        try:
            yield cursor.next()
        except tweepy.RateLimitError:
            time.sleep(15 * 60)
 
for follower in limit_handled(tweepy.Cursor(api.followers).items()):
    if follower.friends_count < 2000:
       # Do scraping here

Sorry, the code you gave me im unsure where i would add this
Reply
#6
(Jan-08-2019, 05:47 PM)danbryn16 Wrote: Sorry, the code you gave me im unsure where i would add this
I not sure what you want,it can also be problem if you start with this and have limit knowledge about Python.
I can try to give some hint with untested code,as i have not done any authentication process to use tweetpy.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
import tweepy, time, csv
 
consumerKey = "HIDDEN"
consumerSecretKey = "HIDDEN"
accessToken = "HIDDEN"
accessTokenSecret = "HIDDEN"
filePath = "results.csv"
csvHeader = ['Username', 'Followers', 'Location']
userFound = False
user = None
 
def authenticate():
    auth = tweepy.OAuthHandler(consumerKey, consumerSecretKey)
    auth.set_access_token(accessToken, accessTokenSecret)
    return tweepy.API(auth, wait_on_rate_limit=True)
 
def limit_handled(cursor):
    while True:
        try:
            yield cursor.next()
        except tweepy.RateLimitError:
            time.sleep(15 * 60)
 
def getFollowers(csvFile=None):
    #csvWriter = csv.writer(csvFile)
    #csvWriter.writerow(csvHeader)
    #users = tweepy.Cursor(api.followers, screen_name=username).items()
    for user in limit_handled(tweepy.Cursor(api.followers).items()):
        if user.friends_count < 2000:
            print(user.followers_count) # Do different test print before write anything to file
 
api = authenticate()
while userFound == False:
    username = input("Enter twitter username: ")
    try:
        user = api.get_user(username)
        userFound = True
    except Exception as e:
        print("User not found - try again")
        continue
 
'''
with open('results.csv', 'w', encoding='utf-8') as csvFile:
    getFollowers(csvFile)'''
 
getFollowers()
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020