I recently decided to use paramiko to develop a remote command execution tool.
It was very easy to setup initially and ran much faster than my existing pexpect implementation but it had a problem with sudo commands because they required the password to be provided as input.
I solved the problem by using a pseudo-terminal and by creating my own ChannelFile objects for stdin and stdout/stderr. The solution should be general enough to handle any case that requires simple input but it is not as flexible as pexpect. I hope that you find it useful.
Current Version
This is similar to the older version below but it handles more than 64KiB by eliminating the stdout buffer, the stdin buffer and using recv polling. It even does a silly check for input. That check could be expanded to make it behave more like pexpect but that is for another day.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 |
#!/usr/bin/env python ''' This class allows you to run commands on a remote host and provide input if necessary. VERSION 1.2 ''' import paramiko import logging import socket import time import datetime # ================================================================ # class MySSH # ================================================================ class MySSH: ''' Create an SSH connection to a server and execute commands. Here is a typical usage: ssh = MySSH() ssh.connect('host', 'user', 'password', port=22) if ssh.connected() is False: sys.exit('Connection failed') # Run a command that does not require input. status, output = ssh.run('uname -a') print 'status = %d' % (status) print 'output (%d):' % (len(output)) print '%s' % (output) # Run a command that does requires input. status, output = ssh.run('sudo uname -a', 'sudo-password') print 'status = %d' % (status) print 'output (%d):' % (len(output)) print '%s' % (output) ''' def __init__(self, compress=True, verbose=False): ''' Setup the initial verbosity level and the logger. @param compress Enable/disable compression. @param verbose Enable/disable verbose messages. ''' self.ssh = None self.transport = None self.compress = compress self.bufsize = 65536 # Setup the logger self.logger = logging.getLogger('MySSH') self.set_verbosity(verbose) fmt = '%(asctime)s MySSH:%(funcName)s:%(lineno)d %(message)s' format = logging.Formatter(fmt) handler = logging.StreamHandler() handler.setFormatter(format) self.logger.addHandler(handler) self.info = self.logger.info def __del__(self): if self.transport is not None: self.transport.close() self.transport = None def connect(self, hostname, username, password, port=22): ''' Connect to the host. @param hostname The hostname. @param username The username. @param password The password. @param port The port (default=22). @returns True if the connection succeeded or false otherwise. ''' self.info('connecting %s@%s:%d' % (username, hostname, port)) self.hostname = hostname self.username = username self.port = port self.ssh = paramiko.SSHClient() self.ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) try: self.ssh.connect(hostname=hostname, port=port, username=username, password=password) self.transport = self.ssh.get_transport() self.transport.use_compression(self.compress) self.info('succeeded: %s@%s:%d' % (username, hostname, port)) except socket.error as e: self.transport = None self.info('failed: %s@%s:%d: %s' % (username, hostname, port, str(e))) except paramiko.BadAuthenticationType as e: self.transport = None self.info('failed: %s@%s:%d: %s' % (username, hostname, port, str(e))) return self.transport is not None def run(self, cmd, input_data=None, timeout=10): ''' Run a command with optional input data. Here is an example that shows how to run commands with no input: ssh = MySSH() ssh.connect('host', 'user', 'password') status, output = ssh.run('uname -a') status, output = ssh.run('uptime') Here is an example that shows how to run commands that require input: ssh = MySSH() ssh.connect('host', 'user', 'password') status, output = ssh.run('sudo uname -a', '<sudo-password>') @param cmd The command to run. @param input_data The input data (default is None). @param timeout The timeout in seconds (default is 10 seconds). @returns The status and the output (stdout and stderr combined). ''' self.info('running command: (%d) %s' % (timeout, cmd)) if self.transport is None: self.info('no connection to %s@%s:%s' % (str(self.username), str(self.hostname), str(self.port))) return -1, 'ERROR: connection not established\n' # Fix the input data. input_data = self._run_fix_input_data(input_data) # Initialize the session. self.info('initializing the session') session = self.transport.open_session() session.set_combine_stderr(True) session.get_pty() session.exec_command(cmd) output = self._run_poll(session, timeout, input_data) status = session.recv_exit_status() self.info('output size %d' % (len(output))) self.info('status %d' % (status)) return status, output def connected(self): ''' Am I connected to a host? @returns True if connected or false otherwise. ''' return self.transport is not None def set_verbosity(self, verbose): ''' Turn verbose messages on or off. @param verbose Enable/disable verbose messages. ''' if verbose > 0: self.logger.setLevel(logging.INFO) else: self.logger.setLevel(logging.ERROR) def _run_fix_input_data(self, input_data): ''' Fix the input data supplied by the user for a command. @param input_data The input data (default is None). @returns the fixed input data. ''' if input_data is not None: if len(input_data) > 0: if '\\n' in input_data: # Convert \n in the input into new lines. lines = input_data.split('\\n') input_data = '\n'.join(lines) return input_data.split('\n') return [] def _run_send_input(self, session, stdin, input_data): ''' Send the input data. @param session The session. @param stdin The stdin stream for the session. @param input_data The input data (default is None). ''' if input_data is not None: self.info('session.exit_status_ready() %s' % str(session.exit_status_ready())) self.info('stdin.channel.closed %s' % str(stdin.channel.closed)) if stdin.channel.closed is False: self.info('sending input data') stdin.write(input_data) def _run_poll(self, session, timeout, input_data): ''' Poll until the command completes. @param session The session. @param timeout The timeout in seconds. @param input_data The input data. @returns the output ''' interval = 0.1 maxseconds = timeout maxcount = maxseconds / interval # Poll until completion or timeout # Note that we cannot directly use the stdout file descriptor # because it stalls at 64K bytes (65536). input_idx = 0 timeout_flag = False self.info('polling (%d, %d)' % (maxseconds, maxcount)) start = datetime.datetime.now() start_secs = time.mktime(start.timetuple()) output = '' session.setblocking(0) while True: if session.recv_ready(): data = session.recv(self.bufsize) output += data self.info('read %d bytes, total %d' % (len(data), len(output))) if session.send_ready(): # We received a potential prompt. # In the future this could be made to work more like # pexpect with pattern matching. if input_idx < len(input_data): data = input_data[input_idx] + '\n' input_idx += 1 self.info('sending input data %d' % (len(data))) session.send(data) self.info('session.exit_status_ready() = %s' % (str(session.exit_status_ready()))) if session.exit_status_ready(): break # Timeout check now = datetime.datetime.now() now_secs = time.mktime(now.timetuple()) et_secs = now_secs - start_secs self.info('timeout check %d %d' % (et_secs, maxseconds)) if et_secs > maxseconds: self.info('polling finished - timeout') timeout_flag = True break time.sleep(0.200) self.info('polling loop ended') if session.recv_ready(): data = session.recv(self.bufsize) output += data self.info('read %d bytes, total %d' % (len(data), len(output))) self.info('polling finished - %d output bytes' % (len(output))) if timeout_flag: self.info('appending timeout message') output += '\nERROR: timeout after %d seconds\n' % (timeout) session.close() return output # ================================================================ # MAIN # ================================================================ if __name__ == '__main__': import sys # Access variables. hostname = 'hostname' port = 22 username = 'username' password = 'password' sudo_password = password # assume that it is the same password # Create the SSH connection ssh = MySSH() ssh.set_verbosity(False) ssh.connect(hostname=hostname, username=username, password=password, port=port) if ssh.connected() is False: print 'ERROR: connection failed.' sys.exit(1) def run_cmd(cmd, indata=None): ''' Run a command with optional input. @param cmd The command to execute. @param indata The input data. @returns The command exit status and output. Stdout and stderr are combined. ''' print print '=' * 64 print 'command: %s' % (cmd) status, output = ssh.run(cmd, indata) print 'status : %d' % (status) print 'output : %d bytes' % (len(output)) print '=' * 64 print '%s' % (output) run_cmd('uname -a') run_cmd('sudo ls -ltrh /var/log | tail', sudo_password) # sudo command |
Older Version
This version will not handle more than 64KiB of output. I am not sure why.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 |
#!/usr/bin/env python ''' This class allows you to run commands on a remote host and provide input if necessary. ''' import paramiko import logging import socket import time # ================================================================ # class MySSH # ================================================================ class MySSH: ''' Create an SSH connection to a server and execute commands. Here is a typical usage: ssh = MySSH() ssh.connect('host', 'user', 'password', port=22) if ssh.connected() is False: sys.exit('Connection failed') # Run a command that does not require input. status, output = ssh.run('uname -a') print 'status = %d' % (status) print 'output (%d):' % (len(output)) print '%s' % (output) # Run a command that does requires input. status, output = ssh.run('sudo uname -a', 'sudo-password') print 'status = %d' % (status) print 'output (%d):' % (len(output)) print '%s' % (output) ''' def __init__(self, compress=True, verbose=False): ''' Setup the initial verbosity level and the logger. @param compress Enable/disable compression. @param verbose Enable/disable verbose messages. ''' self.ssh = None self.transport = None self.compress = compress # Setup the logger self.logger = logging.getLogger('MySSH') self.set_verbosity(verbose) fmt = '%(asctime)s MySSH:%(funcName)s:%(lineno)d %(message)s' format = logging.Formatter(fmt) handler = logging.StreamHandler() handler.setFormatter(format) self.logger.addHandler(handler) self.info = self.logger.info def __del__(self): if self.transport is not None: self.transport.close() self.transport = None def connect(self, hostname, username, password, port=22): ''' Connect to the host. @param hostname The hostname. @param username The username. @param password The password. @param port The port (default=22). @returns True if the connection succeeded or false otherwise. ''' self.info('connecting %s@%s:%d' % (username, hostname, port)) self.hostname = hostname self.username = username self.port = port self.ssh = paramiko.SSHClient() self.ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) try: self.ssh.connect(hostname=hostname, port=port, username=username, password=password) self.transport = self.ssh.get_transport() self.transport.use_compression(self.compress) self.info('succeeded: %s@%s:%d' % (username, hostname, port)) except socket.error: self.transport = None self.info('failed: %s@%s:%d: %s' % (username, hostname, port, str(e))) except paramiko.BadAuthenticationType as e: self.transport = None self.info('failed: %s@%s:%d: %s' % (username, hostname, port, str(e))) return self.transport is not None def run(self, cmd, input_data=None, timeout=10): ''' Run a command with optional input data. Here is an example that shows how to run commands with no input: ssh = MySSH() ssh.connect('host', 'user', 'password') status, output = ssh.run('uname -a') status, output = ssh.run('uptime') Here is an example that shows how to run commands that require input: ssh = MySSH() ssh.connect('host', 'user', 'password') status, output = ssh.run('sudo uname -a', '<sudo-password>') @param cmd The command to run. @param input_data The input data (default is None). @param timeout The timeout in seconds (default is 10 seconds). @returns The status and the output (stdout and stderr combined). ''' self.info('running command: (%d) %s' % (timeout, cmd)) if self.transport is None: self.info('no connection to %s@%s:%s' % (str(self.username), str(self.hostname), str(self.port))) return -1, 'ERROR: connection not established\n' # Fix the input data. input_data = self._run_fix_input_data(input_data) # Initialize the session. self.info('initializing the session') session = self.transport.open_session() session.set_combine_stderr(True) session.get_pty() session.exec_command(cmd) stdin = session.makefile('wb', -1) stdout = session.makefile('rb', -1) self._run_send_input(stdout, stdin, input_data) output = self._run_poll(stdout, timeout, input_data) status = stdout.channel.recv_exit_status() self.info('output size %d' % (len(output))) self.info('status %d' % (status)) return status, output def connected(self): ''' Am I connected to a host? @returns True if connected or false otherwise. ''' return self.transport is not None def set_verbosity(self, verbose): ''' Turn verbose messages on or off. @param verbose Enable/disable verbose messages. ''' if verbose is True: self.logger.setLevel(logging.INFO) else: self.logger.setLevel(logging.ERROR) def _run_fix_input_data(self, input_data): ''' Fix the input data supplied by the user for a command. @param input_data The input data (default is None). @returns the fixed input data. ''' if input_data is not None: if len(input_data) > 0: if '\\n' in input_data: # Convert \n in the input into new lines. lines = input_data.split('\\n') input_data = '\n'.join(lines) if input_data[-1] != '\n': input_data += '\n' return input_data def _run_send_input(self, stdout, stdin, input_data): ''' Send the input data. @param stdout The stdout stream for the session. @param stdin The stdin stream for the session. @param input_data The input data (default is None). ''' if stdout.channel.closed is False: if input_data is not None: self.info('sending input data') stdin.write(input_data) def _run_poll(self, stdout, timeout, input_data): ''' Poll until the command completes. @param timeout The timeout in seconds. @param input_data The input data. @returns the output ''' interval = 0.1 maxseconds = timeout maxcount = maxseconds / interval # Poll until completion or timeout self.info('polling (%d, %d)' % (maxseconds, maxcount)) count = 0 while stdout.channel.closed is False and count <= maxcount: count += 1 time.sleep(interval) # Polling finished. if stdout.channel.closed is False: # Some sort of error occurred, assume a timeout. self.info('polling finished - timeout') stdout.channel.close() output = stdout.read() output += '\nERROR: timeout after %d seconds\n' % (timeout) else: output = stdout.read() self.info('polling finished - %d output bytes' % (len(output))) if input_data is not None: # Strip out the input data. output = output[len(input_data):] self.info('stripped %d input bytes' % (len(input_data))) return output # ================================================================ # MAIN # ================================================================ if __name__ == '__main__': import sys # Access variables. hostname = 'hostname' port = 22 username = 'username' password = 'password' sudo_password = password # assume that it is the same password # Create the SSH connection ssh = MySSH() ssh.set_verbosity(False) ssh.connect(hostname=hostname, username=username, password=password, port=port) if ssh.connected() is False: print 'ERROR: connection failed.' sys.exit(1) def run_cmd(cmd, indata=None): ''' Run a command with optional input. @param cmd The command to execute. @param indata The input data. @returns The command exit status and output. Stdout and stderr are combined. ''' print print '=' * 64 print 'command: %s' % (cmd) status, output = ssh.run(cmd, indata) print 'status : %d' % (status) print 'output : %d bytes' % (len(output)) print '=' * 64 print '%s' % (output) run_cmd('uname -a') run_cmd('sudo ls -ltrh /var/log | tail', sudo_password) # sudo command |
Updated to a newer version that correctly supports more than 64KiB of output.
Updated the current version again to correct a blocking problem when input was expected. Also fixed a timeout problem.
Thanks jlinoff for the valuable code.
Srikanth
Excelent !
Hello Joe!
Thanks a lot for you code.
I was debugging my code for the last two days untill I ran into your page…
My code was missing the .get_pty() call
Anyhow, thanks a lot for your help.
I tried to run your script from a windows machine but it does not seem to work when using commands that require interactivity.
It is not clear from your description what is happening. How does it fail? Are you using a cygwin environment? Which version of windows are you using?
Hi Joe!
Great post! I am new to development and have been spending considerable time understanding the instantiation of classes and functions in your script to tune it for my needs.
Had a couple questions:
session.send(data) on line 242, how is that working? I understand it pushes the sudo password but couldn’t find a method “send” that was executing this?
How is _run_send_input being used? I don’t see it referenced anywhere?
What I am doing with this:
I am trying to tune this and combine it with peexpect potentially to login to a Linux System that is connected to a Fortinet FW CLI and issue certain commands recording the output
Feel free to suggest best ways to do this and very much appreciate the write up and reading of my questions above!
Hi Joe,
Thanks for the script. I used your script it works fine but at the last I get timeout error. Even if i set timeout to 100 it wait till that time and gives timeout error.
following is the output of the script:
Resolving deltas: 98% (8714/8889)
Resolving deltas: 99% (8806/8889)
Resolving deltas: 100% (8889/8889)
Resolving deltas: 100% (8889/8889), done.
ERROR: timeout after 100 seconds
I am tryign to clone a repo on a remote machine. AFter the above output it check for the connection and then print done but the script is not waiting for that line and it always timeout. I increased the timeout to very high value but then also it will give same error. this is not time -out error but its failing to read the last lines and giving this error.
please sugges.
thanks,
Madhusudan
Hey,
I solved the issue by setting timeout flag to False. 🙂
If this code can read multiple inputs from user?
I am not sure that i understand the question. Can you provide an example?