Website Search Assistance Needed

[CaptianNewb]

Hi,
I decided to combine my love for good tequila and being a nerd and am trying to read the site below for a particular bottle I like that doesn't come in too often. I don't have time to drive regularly to the store, so I thought I'd put code to it and have it send a chime when the bottle is found/able to be added to a cart.

I'm having trouble in reading a piece of the webpage and looking for assistance. I think I'm very close. The urls are below. The difference between these websites is the Add to Cart button vs the Out of Stock button and the box that says pickup. One says In stock while the other says Out of stock online and in store. I'm looking for help reading that particular information and if it says Add to Cart or In stock, then send me a chime.

https://www.totalwine.com/spirits/tequil...rules=true

https://www.totalwine.com/spirits/tequil...rules=true

My code is below but it's not reading the loop properly. Any assistance would be greatly appreciated. I think I'm close but there's an issue with the following portion of the code:

if soup.find(string="add to cart"):

Something about the above search isn't correct. I was hoping someone could right click the 3-amigos link above, look at the source, and correct my if soup.find string.

from bs4 import BeautifulSoup
import os
import requests

headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36', 'referer': 'https://www.google.com'}
nowdatetime.datetime.now()

print('============================================================================')

url1 = 'https://www.totalwine.com/spirits/tequila/reposado/clase-azul-reposado-tequila/p/218990175?s=101&igrules=true'

response1 = requests.get(url1, headers=headers)

if response1.status_code == 200:
    soup = BeautifulSoup(response1.content, 'html.parser')
       if soup.find(string="add to cart"):
           print(now, "They have it")
           os.system('afplay /System/Library/Sounds/Blow.aiff')
       else:
           print(now, "They don't have it")

else:
print('Error accessing web page. Error:', response1.status_code)

Any guidance or corrections to my code would be greatly appreciated! Thank you!

[snippsat]

There are some errors,as headers User-Agent are wrong,so will never reach the site(get 403).
Code also have indentation errors,and find will not work.
Here some help.

import requests
from bs4 import BeautifulSoup

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36'}
url = 'https://www.totalwine.com/spirits/tequila/reposado/clase-azul-reposado-tequila/p/218990175?s=101&igrules=true'
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, 'lxml')
#print(soup.find('title').text)
stock = soup.select_one('#AT-atc-upsell')

So now have found the CSS selector #AT-atc-upsell on site,this button tag will contain all data needed for this task.

>>> stock.get('aria-label')
'Out of stock for Clase Azul Reposado Tequila 1.75L'
>>> stock.get('data-price')
'439.99'
>>> stock.text
'Out of stock'
>>> 
>>> # All info in this tag
>>> stock.attrs
{'aria-busy': 'false',
 'aria-disabled': 'true',
 'aria-label': 'Out of stock for Clase Azul Reposado Tequila 1.75L',
 'atc': 'product_Add to cart',
 'class': ['Buttonstyled__StyledButton-shared-packages__sc-1171nsp-0',
           'eoBufg',
           'button__1KODjift',
           'addToCartButton__XptyfZW1'],
 'data-at': 'product-addtocart-disabled',
 'data-department': 'Spirits',
 'data-price': '439.99',
 'data-product-name': 'Clase Azul Reposado Tequila',
 'data-qty': '1',
 'data-sku': '218990175-1',
 'id': 'AT-atc-upsell',
 'type': 'button',
 'variant': 'transactional'}

[CaptianNewb]

Thank you for correcting my header information and the indentation. I think the indentation somehow changed during copy/paste. Thank you.

I'm very new to Python, so I have a few questions for you..

After

stock = soup.select_one('#AT-atc-upsell')

Wouldn't I need an if statement similar to below?

if soup.find(string="add to cart"):
print(now, "They have it")
os.system('afplay /System/Library/Sounds/Blow.aiff')

Also, do I need to do anything with the large block of code that you provided, or is that from the website? Sorry, but I'm new and still learning this! It's a tough task for me, but seems very easy for you. Thank you for your kind help!

[snippsat]

Wouldn't I need an if statement similar to below?

Yes,try not to code all,but give hint so you can try yourself.
Ok to give a example.

import requests
from bs4 import BeautifulSoup

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36'}
url = 'https://www.totalwine.com/spirits/tequila/blancosilver/3-amigos-blanco-tequila/p/106918750?s=101&igrules=true'
# url = 'https://www.totalwine.com/spirits/tequila/reposado/clase-azul-reposado-tequila/p/218990175?s=101&igrules=true'
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, 'lxml')
stock = soup.select_one('#AT-atc-upsell')
if stock.text == 'Add to cart':
    print('They have it in stock \N{Bottle with Popping Cork}')
    print(f'The price of {stock.get("data-product-name")} is ${stock.get("data-price")}')
else:
    print("They don't have it")

Output:
They have it in stock 🍾
The price of 3 Amigos Blanco Tequila is $27.99

So now i test with a site that have it in stock,if test with original site will get don't have it.
Also take out info about label and price.

Also look into CSS selectors,BS support it with select and select_one.
This can make it simpler to get info about right tag,i copy Selector #AT-atc-upsell from browser(right click inspect over tag--> right click --> copy Selector).

[CaptianNewb]

Wow! This is sooo cool!!! Thank you! I was close but didn't know how to interpret that text. Thank you!!!

[CaptianNewb]

I just tried running your code above, and am receiving the following error:

line 10, in <module>
if stock.text == 'Add to cart':
AttributeError: 'NoneType' object has no attribute 'text'

Any idea why this might be happening?

[snippsat]

I just tried running your code above, and am receiving the following error:

line 10, in <module>
if stock.text == 'Add to cart':
AttributeError: 'NoneType' object has no attribute 'text'

Any idea why this might be happening?

Try to copy my code,and don't write anything yourself.
Here i run code in Python 3.11 and 3.10 from command line,and i change parser to html.parser if you not have lxml(pip install lxml).

G:\div_code\scrape
λ python -V
Python 3.11.3

# Run code in Python 3.11
G:\div_code\scrape
λ python stock.py
They have it in stock 🍾
The price of 3 Amigos Blanco Tequila is $27.99

# Run code in 3.10
G:\div_code\scrape
λ py -3.10 stock.py
They have it in stock 🍾
The price of 3 Amigos Blanco Tequila is $27.99

# The code run over is this(using Rich-CLI to print the code on command line)
G:\div_code\scrape
λ rich stock.py

import requests
from bs4 import BeautifulSoup

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36'}
url = 'https://www.totalwine.com/spirits/tequila/blancosilver/3-amigos-blanco-tequila/p/106918750?s=101&igrules=true'
# url = 'https://www.totalwine.com/spirits/tequila/reposado/clase-azul-reposado-tequila/p/218990175?s=101&igrules=true'
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, 'html.parser')
stock = soup.select_one('#AT-atc-upsell')
if stock.text == 'Add to cart':
    print('They have it in stock \N{Bottle with Popping Cork}')
    print(f'The price of {stock.get("data-product-name")} is ${stock.get("data-price")}')
else:
    print("They don't have it")

[CaptianNewb]

It's crazy, but I'm getting the same error using your code, no changes:

if stock.text == 'Add to cart':
AttributeError: 'NoneType' object has no attribute 'text'


I'm running python3.9.6

[snippsat]

I'm running python3.9.6

I can run it in 3.9 so that's not the problem,or here is a NoteBook press run button.

G:\div_code\scrape
λ py -3.9 stock.py
They have it in stock 🍾
The price of 3 Amigos Blanco Tequila is $27.99

G:\div_code\scrape
λ py -3.9 -V
Python 3.9.5

You most test what you get back eg add print(response)(200) line 8,same with soup line 9.

Test if other code work eg this.

import requests
from bs4 import BeautifulSoup

url = 'https://edition.cnn.com/'
response = requests.get(url)
soup = BeautifulSoup(response.content, 'html.parser')
print(soup.find('title').text)

Output:
Breaking News, Latest News and Videos | CNN

[CaptianNewb]

You're awesome for helping me so much!

When I run

import requests
from bs4 import BeautifulSoup
 
url = 'https://edition.cnn.com/'
response = requests.get(url)
soup = BeautifulSoup(response.content, 'html.parser')
print(soup.find('title').text)

I get: Breaking News, Latest News and Videos | CNN
<Response [200]>

So that works