How to schedule independent processes

[vaeVictis]

Hi all.

I'm coding a little routine whose job I am going to explain in a very simplified fashion.

I have a list of "hours" and "tags", on a txt file.
At "some hour" I need to check a web page and see whether the "some tag" is present or not.
The search for the "some tag" starts at "some hour".
If the "some tag" is found, the search finishes. It also finishes after a timeout I could impose.
(Obviously, when I find it I must do some stuff with that "some tag")

I am using selenium for retrieving information and have not problem with this aspect.

The problem (to me) is: there might be more than one search which start at the same hour.
I could not use sched module "priority" because the several search must start in parallel (respecting obviously the treshold of cores number).
I can not risk that a search lasts long and the second one in priority starts when the first finishes.

I could not figure out how to schedule several different processes I read from a file.

Any help would be appreciated.

[Larz60+]

There are many scheduling packages available (also cron if on linux), see: https://pypi.org/search/?q=scheduler

[vaeVictis]

Thanks for your quick answer :)

The problem is not with the scheduling module but with the process module.

Or maybe I misunderstood and you are suggesting to use a scheduling mudule which directly implement a way to schedule really independent jobs? Is this your point, Larz60+?

[Larz60+]

Or maybe I misunderstood and you are suggesting to use a scheduling mudule which directly implement a way to schedule really independent jobs?

Yes

[vaeVictis]

Ok, thanks.
I'll take into account your suggestion. I did not know all that usefull stuff.

By the way, I'm considering if it would be better to rethink the problem, make a single list with all the "some tag", iterate over it, and eventually pop out the found element.

I'll keep you updated.

[Larz60+]

rethink the problem

Always a good idea, even after it's working well!