Python Forum

I am new to web-scraping and deployment of spiders.

1) Basically, I am in the midst of creating a spider to scrape some items and download Excel files from a website

2) I want to schedule the spider on Heroku to send an email with the downloaded files. How do I do so for free?

From https://github.com/yashashreesuresh/amaz...ce_tracker, I know we can use smtplib to let the spider send email.

However, for the downloaded file, where will the file be located in if we deploy the spider from Heroku?

Thank you

First, are you sure that this is something you should be doing on Heroku? I mean, do their terms of use say you shouldn't?

Note that the file system in your app's dyno is ephemeral - see this for more info. If you want more permanent storage, look at a database or some cloud storage service like Amazon S3, Google Cloud Storage, etc.

(Mar-24-2022, 06:31 AM)ndc85430 Wrote: [ -> ]First, are you sure that this is something you should be doing on Heroku? I mean, do their terms of use say you shouldn't?

Note that the file system in your app's dyno is ephemeral - see this for more info. If you want more permanent storage, look at a database or some cloud storage service like Amazon S3, Google Cloud Storage, etc.

Hi

Thanks for the reply. I know the file system is ephemeral. So if the spider click some button on the website to trigger a file download, where will the file be saved? I know I can create code in the spider to send an email using SMTP_SSL, for example, but I will need the file path to attach it to the email.

Thanks