Sep-28-2018, 09:02 PM
I have a Python script that runs daily on an Amazon Web Services instance (Windows) daily to scrape data from a website.
Most of the time the scrape runs successfully without issue and sends a command to shut down the instance once the scrape is complete.
I have several error catching statements throughout the script to try to identify possible failure points & either handle them or just shut down the instance if it's a major error. It's a fairly beefy AWS instance & I of course don't want the instance to sit the running & costing my employer $$$.
However, occasionally the Python script will error out unexpectedly & the window will just sit there without exiting gracefully & the charges unnecessarily continue mounting.
Any experience from u guys on how you handle this sort of thing as to make SURE your AWS instances aren't sitting there running for nothing if your Python script has failed at an unexpected spot & your AWS instance is otherwise sitting there left to run & rack up charges?
Thanks!
Most of the time the scrape runs successfully without issue and sends a command to shut down the instance once the scrape is complete.
I have several error catching statements throughout the script to try to identify possible failure points & either handle them or just shut down the instance if it's a major error. It's a fairly beefy AWS instance & I of course don't want the instance to sit the running & costing my employer $$$.
However, occasionally the Python script will error out unexpectedly & the window will just sit there without exiting gracefully & the charges unnecessarily continue mounting.
Any experience from u guys on how you handle this sort of thing as to make SURE your AWS instances aren't sitting there running for nothing if your Python script has failed at an unexpected spot & your AWS instance is otherwise sitting there left to run & rack up charges?
Thanks!