Python Forum

Hi All,

I have written a python script (myfile.py) which scrapes the product related data from an e-commerce site and store in mysql db.

Now I want to schedule this script to refresh once a week.

I have installed the APScheduler for this work but need your help to implement this-
https://apscheduler.readthedocs.io/en/la...guide.html

Can anyone please share his knowledge?

you should know the drill already - what have you tried, post code and ask questions, etc...

Read first the provided examples: https://github.com/agronholm/apscheduler...schedulers

https://github.com/agronholm/apscheduler...kground.py

Yes, I have seen the examples.
But I am not sure where to use my python script (myfile.py) in a scheduler.

following is the content of my python script-

def get_soup(url):
soup = None
try:
    response = requests.get(url)
    if response.status_code == 200:
        html = response.content
        soup = BeautifulSoup(html, "html.parser")
    return soup

def get_category_urls(url):
soup = get_soup(url)
cat_urls = []
try:
    categories = soup.find('div', attrs={'id': 'menu_oc'})
    if categories is not None:
        for c in categories.findAll('a'):
            if c['href'] is not None:
                cat_urls.append(c['href'])
    return cat_urls

def get_product_urls(url):
 soup = get_soup(url)
 prod_urls = []
 if soup.find('div', attrs={'class': 'pagination'}):
     for link in soup.select('div.links a'):
         if link.string.isdecimal():  # dump next and last links
             prod_urls.append(link['href'])
 print("Found following product urls::", prod_urls)
 return prod_urls

if __name__ == '__main__':
category_urls = get_category_urls(URL)
product_urls = get_product_urls(URL)
#TODO upload in db

Now I have created a scheduler-refresh.py with following content

import schedule
import time
def job():
    //how to call myfile.py here?
    print("refreshing...")
schedule.every().week.at("10:30").do(job)
while 1:
    schedule.run_pending()
    time.sleep(1)

Here I don;t know how to call myfile.py. Can you help me?