run a health check script on cloud server through paramiko

[amritjsr]

I have a requirement of log into client server ( it's a cloud-server behind Bastion Hosts, we lg in through ssh tunnel )
Now i have to run health check script (multiple checks need to be done as root) there and generate report,
for that i decided to use, paramiko module for it. and i wrote the below python script for this purpose ...
But the problem is it's not giving desire output, what i strongly believe that there is a speed or something mismatch in between command execution and output from terminal .... i just need help how to fix it ....

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

import os import paramiko from time import sleep paramiko.util.log_to_file('/tmp/paramiko.log', level=5) paramiko.util.load_host_keys(os.path.expanduser('~/.ssh/known_hosts')) host = 'W.X.Y.Z' port = 22 username = 'myUserID' password = 'passw0rd' ssh_client = paramiko.SSHClient() ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) user_ssh_cfg_file = os.path.expanduser("~/.ssh/config") userSshConfig = paramiko.SSHConfig() userSshConfig.parse(open(user_ssh_cfg_file)) user_config = userSshConfig.lookup(host) ssh_client.connect(host, username=user_config['user'], password=password, sock=paramiko.ProxyCommand(user_config['proxycommand']), timeout=10) conn = ssh_client.get_transport() session = conn.open_session() session.settimeout(5) user_shell = ssh_client.invoke_shell() print("STOP-1 => ",user_shell.recv_ready()) while not user_shell.recv_ready() and not user_shell.exit_status_ready() and not user_shell.event_ready :     sleep(0.25); print("STOP-2") user_shell.send('pbrun policy -u root\n') # root_shell = user_shell while not user_shell.recv_ready() and not user_shell.exit_status_ready()  and not user_shell.event_ready :     sleep(0.25); print("STOP-3") user_shell.send( password + '\n') while not user_shell.recv_ready() and not user_shell.exit_status_ready()  and not user_shell.event_ready :     sleep(0.25); print("STOP-3") stdout = user_shell.makefile('rb') stdin = user_shell.makefile_stdin('wb') print("Printing  ==>> ", stdout.channel.recv(1000000)) user_shell.send('ls /tmp/*' + '\n') while not user_shell.recv_ready() and not user_shell.exit_status_ready()  and not user_shell.event_ready :     sleep(0.25); print("STOP-4") output = stdout.channel.recv(1000000).decode('ascii').split('\n') access_log_files = [] for I in range(1, len(output) - 1 ):     access_log_files.append(output[I].strip('\r')) print("STOP-5 => ", access_log_files) ssh_client.close()

Output:
STOP-1 =>  True
Printing  ==>>  b'Last login: Mon Jul 20 14:40:14 2020 from cloud-bastion-01.bastion1.myCorporation.com\r\r\nAuthorized uses only. All activity may be monitored and reported.\r\n'
STOP-5 =>  []
Exception ignored in: <function BufferedFile.__del__ at 0x7f43cb452050>
Traceback (most recent call last):
  File "/home/amritdas/vPy37BastionHost/lib/python3.7/site-packages/paramiko/file.py", line 66, in __del__
  File "/home/amritdas/vPy37BastionHost/lib/python3.7/site-packages/paramiko/channel.py", line 1392, in close
  File "/home/amritdas/vPy37BastionHost/lib/python3.7/site-packages/paramiko/channel.py", line 991, in shutdown_write
  File "/home/amritdas/vPy37BastionHost/lib/python3.7/site-packages/paramiko/channel.py", line 963, in shutdown
  File "/home/amritdas/vPy37BastionHost/lib/python3.7/site-packages/paramiko/channel.py", line 1246, in _send_eof
  File "/home/amritdas/vPy37BastionHost/lib/python3.7/site-packages/paramiko/message.py", line 232, in add_int
TypeError: 'NoneType' object is not callable

[Larz60+]

But the problem is it's not giving desire output

What does this mean?
what is the 'desired output' supposed to look like?
That doesn't look like the complete error traceback.
Please post complete and unaltered error traceback, and what desired output should look like ... Use error tags for traceback

[amritjsr]

The desire output means, it's not able execute the given command properly. There are few things to mention ...
1. It's able to log into the server
2. It's able to invoke_shell ( i can see from other ssh console on same server)
3. if i leave the root part as of now it's may/may-not able to execute the given command but i am unable to capture the output.
What possibly, i am thinking i am not able to figure out the command prompt ( may be script is running fast and tty is slow ) and send the command and gather the output ...

Yes, above mentioned output is the whole one, not truncated at all.

---

Here i want to mention one thing ... if i keep adding sleep for couple of seconds in between then we are getting desire out but along with error too .... as below ...

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49

import os import paramiko from time import sleep import pdb paramiko.util.log_to_file('/tmp/paramiko.log', level=5) paramiko.util.load_host_keys(os.path.expanduser('~/.ssh/known_hosts')) host = 'W.X.Y.Z' port = 22 username = 'myUserID' password = 'passw0rd' ssh_client = paramiko.SSHClient() ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) user_ssh_cfg_file = os.path.expanduser("~/.ssh/config") userSshConfig = paramiko.SSHConfig() userSshConfig.parse(open(user_ssh_cfg_file)) user_config = userSshConfig.lookup(host) ssh_client.connect(host, username=user_config['user'], password=password, sock=paramiko.ProxyCommand(user_config['proxycommand']), timeout=10) sleep(5) user_shell = ssh_client.invoke_shell() sleep(5) print("STOP-1 => ",user_shell.recv_ready()) while not user_shell.recv_ready() and not user_shell.exit_status_ready() and not user_shell.event_ready :     sleep(0.25); print("STOP-2") user_shell.send('pbrun policy -u root\n') sleep(5) # root_shell = user_shell while not user_shell.recv_ready() and not user_shell.exit_status_ready()  and not user_shell.event_ready :     sleep(0.25); print("STOP-3") user_shell.send( password + '\n') sleep(5) while not user_shell.recv_ready() and not user_shell.exit_status_ready()  and not user_shell.event_ready :     sleep(0.25); print("STOP-3") stdout = user_shell.makefile('rb') stdin = user_shell.makefile_stdin('wb') sleep(5) print("Printing  ==>> ", stdout.channel.recv(1000000)) sleep(2) user_shell.send('ls /tmp/*' + '\n') sleep(5) while not user_shell.recv_ready() and not user_shell.exit_status_ready()  and not user_shell.event_ready :     sleep(0.25); print("STOP-4") output = stdout.channel.recv(1000000).decode('ascii').split('\n')   access_log_files = [] for I in range(1, len(output) - 1 ):     access_log_files.append(output[I].strip('\r'))   print("STOP-5 => ", access_log_files) ssh_client.close()

Error:
myUserID@docker-manager:~/app-devops-scripts/app-scripts$ python3 cloud_sftp_paramiko.py
STOP-1 =>  True
Printing  ==>>  b'Last login: Mon Jul 20 17:29:48 2020 from cloud-bastion-01.bastion1.myCorporation.com\r\r\nAuthorized uses only. All activity may be monitored and reported.\r\n[user@cloud-server ~]$ pbrun policy-root -u root\r\nSuccessful login using -u option.  If you want to allow X traffic please use -x option.\r\nUsage: /usr/local/bin/pbrun [policy] -x [target user]\r\n\r\nPassword: \r\n\r\nsu from myUserID to root at Mon Jul 20 17:44:45 UTC 2020\r\nLast login: Mon Jul 20 11:11:33 UTC 2020 on pts/2\r\n[root@could-server ~]# '
STOP-5 =>  ['/tmp/access.log', '/tmp/access.log00053', '/tmp/access.log00054', '/tmp/access.log00055', '/tmp/access.log00056', '/tmp/access.log00057', '/tmp/access.log00058', '/tmp/access.log00059', '/tmp/access.log00060', '/tmp/access.log00061']
Exception ignored in: <function BufferedFile.__del__ at 0x7f7009ffa0e0>
Traceback (most recent call last):
  File "/home/myUserID/vPy37BastionHost/lib/python3.7/site-packages/paramiko/file.py", line 66, in __del__
  File "/home/myUserID/vPy37BastionHost/lib/python3.7/site-packages/paramiko/channel.py", line 1392, in close
  File "/home/myUserID/vPy37BastionHost/lib/python3.7/site-packages/paramiko/channel.py", line 991, in shutdown_write
  File "/home/myUserID/vPy37BastionHost/lib/python3.7/site-packages/paramiko/channel.py", line 963, in shutdown
  File "/home/myUserID/vPy37BastionHost/lib/python3.7/site-packages/paramiko/channel.py", line 1246, in _send_eof
  File "/home/myUserID/vPy37BastionHost/lib/python3.7/site-packages/paramiko/message.py", line 232, in add_int
TypeError: 'NoneType' object is not callable
myUserID@docker-manager:~/app-devops-scripts/app-scripts$ 

[Larz60+]

it prints Stop-5
the error seems to be generated by close command. Does server automatically close, thus creating subsequent close?

[amritjsr]

Thanks for replying, I am not very much worried about that last error which is possibly generated by close statement, I am bit worried about use of sleep statement, which will make my script very long running, how do I synchronize with remote shell output and my input and gather output timely