+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: snscrape (/thread-39575.html)
snscrape - pythonical - Mar-11-2023
I'm new to Python. I am trying the snscrape method to scan tweets and save them in tabular format. The code I found from a site works. Here my code:
import pandas as pd
from tqdm.notebook import tqdm
import snscrape.modules.twitter as sntwitter
scraper = sntwitter.TwitterSearchScraper("#yağma")
tweets = []
n_tweets = 50
for i, tweet in tqdm(enumerate(scraper.get_items()), total = n_tweets):
data=[tweet.date, tweet.id, tweet.content, tweet.user.username, tweet.likeCount, tweet.retweetCount]
tweets.append(data)
if i>n_tweets:
break
tweet_df = pd.DataFrame(
tweets, columns=["date", "id", "content", "username", "like_count", "retweet_count"]
)
tweet_dfBut there are some problems:- 1- When I download tweets in xls format, the content of the tweets is limited to 40 characters, I can't see all the content
2- I can't set language and region for tweets
3- When there are too many lines, I separate them with three dots and I can only download a limited number of lines. For example:
- 1.line: tweet date- content
2.line tweet date2-content2
...... (i cannot see or download these lines)
1001.line tweetdate1001-content1001
- 1.line: tweet date- content