Python Forum
No Internet connection when running a Python script
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
No Internet connection when running a Python script
#1
Hello. I have the follwoing Python code that checks a website for change. This script always gives me an error "Error checking website". What am I doing wrong?

import requests
import os
from bs4 import BeautifulSoup
import time
import logging
import smtplib as smtp

URL_TO_MONITOR = "https://www.yahoo.com/" #change this to the URL you want to monitor
DELAY_TIME = 15 # seconds

def process_html(string):
soup = BeautifulSoup(string, features="lxml")

    # make the html look good
    soup.prettify()
    
    # remove script tags
    for s in soup.select('script'):
        s.extract()
    
    # remove meta tags 
    for s in soup.select('meta'):
        s.extract()
    
    # convert to a string, remove '\r', and return
    return str(soup).replace('\r', '')

def webpage_was_changed():
"""Returns true if the webpage was changed, otherwise false."""
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36',
'Pragma': 'no-cache', 'Cache-Control': 'no-cache'}
response = requests.get(URL_TO_MONITOR, headers=headers)

    # create the previous_content.txt if it doesn't exist
    if not os.path.exists("previous_content.txt"):
        open("previous_content.txt", 'w+').close()
    
    filehandle = open("previous_content.txt", 'r')
    previous_response_html = filehandle.read() 
    filehandle.close()
    
    processed_response_html = process_html(response.text)
    
    if processed_response_html == previous_response_html:
        return False
    else:
        filehandle = open("previous_content.txt", 'w')
        filehandle.write(processed_response_html)
        filehandle.close()
        return True

def main():
log = logging.getLogger(__name__)
logging.basicConfig(level=os.environ.get("LOGLEVEL", "INFO"), format='%(asctime)s %(message)s')
log.info("Running Website Monitor")
while True:
try:
if webpage_was_changed():
log.info("WEBPAGE WAS CHANGED.")

                print("The website was changed")
            else:
                log.info("Webpage was not changed.")
        except:
            log.info("Error checking website.")
        time.sleep(DELAY_TIME)

if __name__ == "__main__":
main()
Reply


Messages In This Thread
No Internet connection when running a Python script - by basil_555 - Mar-10-2024, 09:18 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  I don't know what is wrong (Python and SQL connection) shereen 3 519 Apr-01-2024, 08:56 AM
Last Post: Pedroski55
Question Running Python script through Task Scheduler? Winfried 8 939 Mar-10-2024, 07:24 PM
Last Post: Winfried
  Connection LTspice-Python with PyLTSpice bartel90 0 470 Feb-05-2024, 11:46 AM
Last Post: bartel90
  Virtual Env changing mysql connection string in python Fredesetes 0 464 Dec-20-2023, 04:06 PM
Last Post: Fredesetes
  connection python and SQL dawid294 4 842 Dec-12-2023, 08:22 AM
Last Post: Pedroski55
  Help Running Python Script in Mac OS emojistickers 0 417 Nov-20-2023, 01:58 PM
Last Post: emojistickers
  Trying to make a board with turtle, nothing happens when running script Quascia 3 824 Nov-01-2023, 03:11 PM
Last Post: deanhystad
  Is there a *.bat DOS batch script to *.py Python Script converter? pstein 3 3,731 Jun-29-2023, 11:57 AM
Last Post: gologica
  Python script running under windows over nssm.exe JaroslavZ 0 809 May-12-2023, 09:22 AM
Last Post: JaroslavZ
  Networking Issues - Python GUI client and server connection always freezes Veritas_Vos_Liberabit24 0 809 Mar-21-2023, 03:18 AM
Last Post: Veritas_Vos_Liberabit24

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020