+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Getting remote addresses from database (/thread-3292.html)
Getting remote addresses from database - desudesu - May-11-2017
Hi, im looking to make a program that connects to 20+ remote machines and runs some commands on them and also downloads some files from them.
Because the number of remote hosts is high and will increase in future, i was thinking i would make a simple database (mysql) that has id, description and ip_add field, the program would then get the ip's that it should connect to from the database. I would also make a simple .php site for adding and removing remote hosts from the database.
Does this kind of setup make any sense? Or is there some other solution that anyone can think off?
Im just starting to learn python so im not sure if what i have planned makes any sense.
For the actual connecting to remote hosts i was thinking of using paramiko.
(no coding done yet, still on planning phase, sorry)
Thanks
RE: Getting remote addresses from database - wavic - May-11-2017
I think this is what you need: https://github.com/0rpc/zerorpc-python
RE: Getting remote addresses from database - snippsat - May-11-2017
(May-11-2017, 05:02 PM)desudesu Wrote: desudesuDoes this kind of setup make any sense?Yes,but not the php site
Look into Fabric(port to 3) or Ansible.
Ansible is a very powerful,so if want to connect to many remote host i can be worth looking into.
Eg Strategies
Quote:All hosts will run each task before any host starts the next task, using the number of forks (default 5) to parallelize.
Quote:A second strategy ships with ansible free, which allows each host to run until the end of the play as fast as it can.:
RE: Getting remote addresses from database - desudesu - May-12-2017
Hi, read up on ansible a little, bit, it seems its mainly used in running simple tasks on servers, not sure if it can do the complex things i would need it to do.
What i have is a empty server, what i want to make it do is, connect to remote clients that collects different kinds of data non-stop 24/7, every 12-24 hours the server should connect to these remote hosts and download new data files to the server, also somekind of check should be implemented so data already on server isnt download and if there is data missing from previous days (from last 10days for example, no internet connection perhaps) that data should be downloaded aswell.
So in short:
1. Connect to 20+ remote hosts daily (if connection not avaible, notify admin and possibly try again later for that specific remost host)
2. Check data on remotehosts (different filetypes, data on different directories on most remote hosts)
3. Check if new data is intact (no 0byte files, if 0byte files, admin should be notified, since theres a problem on remote host)
4. Download new and missing data (from days when connection to remote host wasnt available)
5. when all files downloaded, push them to a fileserver to be analyzed. END
6. Display analyzed data on website (this we have already worked out and is in use now)
This is a VERY VERY(!) simplified discription off the things this new server has to do.
Currently we have 30+ scripts running from several different machines, that do the basic data transfers and someone does the data integrity verifications manually everyday. The idea is the automate all the work, so the only thing left to do manually is to check a simple output file that says what has been done and what hasnt.