Twitter scraping exclude some data

[Robbert]

Hello everyone! Im new here and im also new to python. Im eager to learn python because the possibilities are immense. Currently im working on a twitter streaming code, which I pasted in the code section below.
Im wondering how I should exclude data from the streamer?
1. For instance, i want to check wether the 'status' or 'location' fields are not null.
2. I would like to exclude some fields. For instance, 'retweets'.

If someone could explain how I'm supposed to program [1] en [2] then I would be very happy :)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

from tweepy import Stream from tweepy import OAuthHandler from tweepy.streaming import StreamListener import json   # consumer key, consumer secret, access token, access secret. consumer_key = "xxx" consumer_secret = "xxx" access_token = "xxxx" access_token_secret = "xxxx"     class StdOutlistener(StreamListener):     def on_data(self, data):         json_data = json.loads(data)         print (json_data)           # Open json text file to save the tweets         with open('tweets.json', 'a') as tf:             tf.write(data)         return True       def on_error(self, status):           print(status)     auth = OAuthHandler(consumer_key, consumer_secret) auth.set_access_token(access_token, access_token_secret)   twitterStream = Stream(auth, StdOutlistener()) twitterStream.filter(track=["Test"])

[Robbert]

Can anyone help me?
Is my question unclear>?

[nilamo]

Are those fields part of the json response you're receiving?

[Robbert]

Are those fields part of the json response you're receiving?

Yes. The json format contains all the data that is available.
In this tutorial: http://adilmoujahid.com/posts/2014/07/tw...analytics/ an overview is given of the data and Json output when no filters are applied. Literally everything is passing through and I would like to know whether it is possible to skip some fields. For instance; im not interesse in the fact that someone does have 20 followers or something.

[nilamo]

Here's a direct link to the StreamListener class from the tweepy module: https://github.com/tweepy/tweepy/blob/v3...ing.py#L30

You're currently using on_data(), which fires off for every single message.  Have you tried using one of the more specific ones, like on_status()?

[Robbert]

Here's a direct link to the StreamListener class from the tweepy module: https://github.com/tweepy/tweepy/blob/v3...ing.py#L30

You're currently using on_data(), which fires off for every single message.  Have you tried using one of the more specific ones, like on_status()?

Thanks for your reply and suggestion. I will have at the webpage you mentioned.
No, i haven't tried on_status which would probably be better. But i have no idea how to use on_status in this particular script.

Do you perhaps have a link for that to?

[nilamo]

You currently use on_data.  replace the word "data" with "status", and it should run.