Need some help with Pytrends

[cmg15326]

Hey! So I did set up Jupyter Lab and followed some tutorials on how to request data from Google Trends. Somehow I get an Error and I don't really know what's causing this I believe I did everything I need to in the correct order, but somewhere I did something wrong for sure!

import pandas as pd
from pytrends.request import TrendReq

# Set up the TrendReq object
pytrend = TrendReq()

# Define the keywords and category
keywords = ['baby', 'car', 'beanie']
category = 18 # Shopping category in the USA

# Get the historical interest data for the past 90 days
pytrend.build_payload(kw_list=keywords, cat=category, timeframe='today 90-d', geo='US')
interest_over_time_df = pytrend.interest_over_time()

# Filter the results based on the given conditions
filtered_df = interest_over_time_df[(interest_over_time_df >= 50).sum(axis=1) >= 2] # Filter for 2 or more keywords with values >= 50
filtered_df = filtered_df[filtered_df.iloc[:, -1] >= 35] # Filter for the last date value >= 35

print(filtered_df)

[img]

Hope for some help, and thank you for your time and help in advance!

With kind regards
Chris

[snippsat]

Post code and not image.
Look at Common API parameters.

Daily: 'now #-d' where # is the number of days from that date to pull data for

For example: 'now 7-d' would get data from the last week
Works for 1, 7 days only!

So if want 90-days use months.

timeframe = 'today 3-m'

[cmg15326]

Thanks a lot for your time and answer! I edited the code like that:

import pandas as pd
from pytrends.request import TrendReq
import time

# Set up the TrendReq object
pytrend = TrendReq()

# Define the keywords and category
keywords = ['baby', 'car', 'baby shower', 'beanie', 'stroller', 'diaper', 'pacifier', 'crib', 'onesie', 'bottle', 'baby shower', 'baby food', 'rattle', 'playpen', 'changing table', 'baby carrier', 'swaddle', 'high chair', 'teething toy', 'breast pump', 'baby monitor', 'mobile', 'baby swing', 'nursery', 'burp cloth', 'baby gate', 'baby lotion', 'bath tub', 'baby clothes', 'baby hat', 'baby bib', 'baby book', 'baby oil', 'baby shampoo', 'baby blanket', 'baby shoes', 'baby socks', 'baby towel', 'baby wipes', 'binky', 'boppy', 'booster seat', 'bouncy seat', 'breastfeeding', 'car seat', 'changing pad', 'cot', 'cradle', 'formula', 'infant car seat', 'infant formula', 'nursing pillow', 'onesies']
category = 18 # Shopping category in the USA

# Split keywords into subsets of five
keyword_subsets = [keywords[i:i+5] for i in range(0, len(keywords), 5)]

# Get the historical interest data for the past 90 days for each subset
for subset in keyword_subsets:
    try:
        pytrend.build_payload(kw_list=subset, cat=category, timeframe='today 3-m', geo='US')
        interest_over_time_df = pytrend.interest_over_time()

        # Filter the results based on the given conditions
        filtered_terms = []
        for col in interest_over_time_df.columns:
            if (interest_over_time_df[col] >= 50).sum() / len(interest_over_time_df) >= 0.5 and interest_over_time_df[col].iloc[-1] >= 35:
                filtered_terms.append(col)

        # Print the filtered terms for the current subset
        print(filtered_terms)
        time.sleep(1) # wait for 1 second before making the next request
    except Exception as e:
        print(f"Error occurred while processing subset {subset}: {e}")

And it works!

The actual output is:

Output:
['baby', 'car']
['bottle']
['baby shower']
[]
['mobile']
['baby clothes']
[]
['baby shoes']
[]
['formula']
['onesies']

BUT I assume the empty brackets should contain one of these keywords as well. Also, it seems that sometimes the results differ (seems it won't get all the keywords all the time and that seems kinda strange to me)


And for whatever reason right now when I want to test it again this is the result:

Output:
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]

Post code and not image.
Look at Common API parameters.

Daily: 'now #-d' where # is the number of days from that date to pull data for

For example: 'now 7-d' would get data from the last week
Works for 1, 7 days only!

So if want 90-days use months.

timeframe = 'today 3-m'

Larz60+ write Apr-30-2023, 11:20 PM:
This moderation took place after snippsat's request for adding code tags in next post:
Please post all code, output and errors (it it's entirety) between their respective tags. Refer to BBCode help topic on how to post. Use the "Preview Post" button to make sure the code is presented as you expect before hitting the "Post Reply/Thread" button. Fixed for you this time. Please use BBCode tags on future posts.

[snippsat]

Use Code tags..
The way you filter to lists is a uncommon and not the normally way to filter in a Pandas DataFrame.
To take something that you started with.

import pandas as pd
from pytrends.request import TrendReq

# Set up the TrendReq object
pytrend = TrendReq()

# Define the keywords and category
keywords = ['baby', 'car', 'beanie']
category = 18 # Shopping category in the USA

# Get the historical interest data for the past 90 days
pytrend.build_payload(kw_list=keywords, cat=category, timeframe='today 3-m', geo='US')
interest_over_time_df = pytrend.interest_over_time()

# So i fixed this
filtered_df = interest_over_time_df[(interest_over_time_df >= 50).sum(axis=1) >= 2] 
last_date = filtered_df.iloc[-1]
filtered_terms = last_date[last_date >= 35]
filtered_terms

Output:
baby    93
car     62
Name: 2023-04-26 00:00:00, dtype: object

So here get outputs to pandas Series.

Another way is to Filter and get a DataFrame back as result.
So eg baby values over 80 and beaine less or equal than 10.

import pandas as pd
from pytrends.request import TrendReq

# Set up the TrendReq object
pytrend = TrendReq()

# Define the keywords and category
keywords = ['baby', 'car', 'beanie']
category = 18 # Shopping category in the USA

# Get the historical interest data for the past 90 days
pytrend.build_payload(kw_list=keywords, cat=category, timeframe='today 3-m', geo='US')
df = pytrend.interest_over_time()
df1 = df.loc[(df['baby'] > 80) & (df['beanie'] <= 10)]
df1.reset_index()

Output:
	date	       baby     car     beanie  isPartial
0	2023-04-22	88	74	10	False
1	2023-04-24	86	63	9	False
2	2023-04-25	84	63	8	False
3	2023-04-26	93	62	7	False

Also as tips when you use a regular Python loop in Pandas then most likely doing something not optimal or just a wrong way to work with a DataFrame.

[cmg15326]

Cant thank you enough, I would be totally lost without you! I try to make this work with the help of ChatGPT but that's often a bad address if you don't know a lot in the area you try to accomplish something.

The output looks way different!

I assume the numbers behind the keywords baby 79 car 63 show the average score from Google Trends for this Keyword within the past 90 days?

The only "issue" I have is that your filter does the following: It filters for keywords that have an interest score of 50 or more for at least 2 days and a score of 35+ on the last day, right?

The filter for the score of 35+ on the last day is perfect!

But the other must be like that: Filters for keywords that have an interest score of 50 or more for at least 50% of the time (or more). So I thought I could simply change the 2 days to 45 days or more, but changing that value results in an IndexError.

import pandas as pd
from pytrends.request import TrendReq
 
# Set up the TrendReq object
pytrend = TrendReq()
 
# Define the keywords and category
keywords = ['baby', 'car', 'beanie', 'baby shower', 'mobile']
category = 18 # Shopping category in the USA
 
# Get the historical interest data for the past 90 days
pytrend.build_payload(kw_list=keywords, cat=category, timeframe='today 3-m', geo='US')
interest_over_time_df = pytrend.interest_over_time()
 
# So i fixed this
filtered_df = interest_over_time_df[(interest_over_time_df >= 50).sum(axis=1) >= 2] 
last_date = filtered_df.iloc[-1]
filtered_terms = last_date[last_date >= 35]
filtered_terms

Output:
baby    79
car     63
Name: 2023-04-27 00:00:00, dtype: object

Also, when I add keywords like I did in the code (sorry for not using the [python] before I always tried clicking the "code" icon but that didnt do anything haha) it won't show the other two ('baby shower' and 'mobile') but they meet the criteria of the filters.