Problem
I’d like to create a function that will run a shell command and return the result as a string, whether it’s an error or a success message. I simply want to achieve the same result as if I had used the command line.
What would be an example of code that accomplishes this?
For example:
def run_command(cmd):
# ??????
print run_command('mysqladmin create test -uroot -pmysqladmin12')
# Should output something like:
# mysqladmin: CREATE DATABASE failed; error: 'Can't create database 'test'; database exists'
Asked by Silver Light
Solution #1
The simplest solution in all officially supported versions of Python is to utilize the subprocess.check output function:
>>> subprocess.check_output(['ls', '-l'])
b'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
check output executes a single program that only accepts parameters. 1 It prints the result to stdout and then returns it. Skip ahead to the run or Popen parts if you need to write input to stdin. See the remark on shell=True at the end of this answer if you want to run complicated shell commands.
The check output function is compatible with all versions of Python that are officially supported. However, a more flexible method is available in more current versions.
The new run function is recommended by the official documentation for most jobs if you’re using Python 3.5+ and don’t need backwards compatibility. For the subprocess module, it provides a very generic, high-level API. Pass the subprocess to capture a program’s output. To the stdout keyword argument, add the PIPE flag. Then, on the returned CompletedProcess object, access the stdout attribute:
>>> import subprocess
>>> result = subprocess.run(['ls', '-l'], stdout=subprocess.PIPE)
>>> result.stdout
b'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
Because the return value is a bytes object, you’ll need to decode it to get a valid string. Assuming the calling process delivers a UTF-8-encoded string, the following code is generated:
>>> result.stdout.decode('utf-8')
'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
If desired, everything of above can be condensed into a one-liner:
>>> subprocess.run(['ls', '-l'], stdout=subprocess.PIPE).stdout.decode('utf-8')
'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
You can supply a bytes object to the input keyword argument to pass input to the process’s stdin:
>>> cmd = ['awk', 'length($0) > 5']
>>> ip = 'foo\nfoofoo\n'.encode('utf-8')
>>> result = subprocess.run(cmd, stdout=subprocess.PIPE, input=ip)
>>> result.stdout.decode('utf-8')
'foofoo\n'
Passing stderr=subprocess.PIPE (capture to result.stderr) or stderr=subprocess.STDOUT (capture to result.stderr) or stderr=subprocess.STDOUT (capture to result.stderr) or stderr=subprocess.STDOUT (capture to (capture to result.stdout along with regular output). If you pass check=True to run, it will throw an exception if the process returns a nonzero exit code. (Alternatively, look at the returncode attribute of the result above.) You can also run more complex shell commands by passing shell=True, as mentioned at the conclusion of this answer, if security is not an issue.
The above is simplified even more in later Python versions. The above one-liner can be spelled as follows in Python 3.7+:
>>> subprocess.run(['ls', '-l'], capture_output=True, text=True).stdout
'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
When compared to the traditional manner of doing things, using run this way adds a small amount of complexity. However, you can now accomplish practically everything you need with just the run function.
You can use the check output function, which is briefly discussed above, if you’re using an earlier version of Python or require some backwards compatibility. Since Python 2.7, it has been available.
subprocess.check_output(*popenargs, **kwargs)
It accepts the same arguments as Popen (see below) and returns a string containing the output of the program. A more extensive usage example may be seen at the start of this response. Check output in Python 3.5+ is identical to running run with check=True and stdout=PIPE and only returning the stdout attribute.
You can use the stderr=subprocess parameter. STDOUT is used to make sure that any error messages are included in the output. You can also run more complex shell commands by passing shell=True, as mentioned at the conclusion of this answer, if security is not an issue.
Check output will not work if you need to pipe from stderr or pass input to the process. In that scenario, see the Popen examples below.
You’ll have to work directly with Popen objects, which encapsulate the low-level API for subprocesses, if you need deep backwards compatibility or more complex capabilities than check output or run give.
The Popen constructor takes either a single command with no arguments or a list with a command as the first item and any number of arguments as separate items in the list. shlex.split can aid in the parsing of strings into properly formed lists. Popen objects also accept a host of different arguments for process IO management and low-level configuration.
Communicate is nearly always the preferred mechanism for sending input and capturing output. As an example:
output = subprocess.Popen(["mycmd", "myarg"],
stdout=subprocess.PIPE).communicate()[0]
Or
>>> import subprocess
>>> p = subprocess.Popen(['ls', '-a'], stdout=subprocess.PIPE,
... stderr=subprocess.PIPE)
>>> out, err = p.communicate()
>>> print out
.
..
foo
You may also give data to the process via stdin if you specify stdin=PIPE in communicate:
>>> cmd = ['awk', 'length($0) > 5']
>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
... stderr=subprocess.PIPE,
... stdin=subprocess.PIPE)
>>> out, err = p.communicate('foo\nfoofoo\n')
>>> print out
foofoo
On some platforms, you may need to set stdout, stderr, and stdin all to PIPE (or DEVNULL) to get communicate to work at all, according to Aaron Hall’s response.
In some rare instances, elaborate, real-time output capturing may be required. Vartec’s response indicates a path forward, however if not handled carefully, methods other than communicate are prone to deadlocks.
When security is not an issue, you can run more complicated shell commands by giving shell=True to any of the following functions.
The shell=True parameter is used to run shell commands.
Each call to run, check output, or the Popen constructor normally runs a single program. That implies you won’t be able to use any sophisticated bash-style pipes. If you want to run complex shell commands, you can pass shell=True, which all three functions support. For example:
>>> subprocess.check_output('cat books/* | wc', shell=True, text=True)
' 1299377 17005208 101299376\n'
This, however, creates security problems. If you’re doing anything more complicated than light scripting, you might be better off starting each process individually and using the result from each as an input to the next.
run(cmd, [stdout=etc...], input=other_output)
Or
Popen(cmd, [stdout=etc...]).communicate(other_output)
The desire to join pipes directly is strong; resist it. Otherwise, you’ll most likely run into deadlocks or be forced to perform stuff like this.
Answered by senderle
Solution #2
This is a lot easier, but it only works with Python2.7 and Unix (including Cygwin).
import commands
print commands.getstatusoutput('wc -l file')
It gives you a tuple with (return value, output) in it.
Instead, utilize the subprocess module for a solution that works in both Python 2 and Python 3:
from subprocess import Popen, PIPE
output = Popen(["date"],stdout=PIPE)
response = output.communicate()
print response
Answered by byte_array
Solution #3
Something like that:
def runProcess(exe):
p = subprocess.Popen(exe, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while(True):
# returns None while subprocess is running
retcode = p.poll()
line = p.stdout.readline()
yield line
if retcode is not None:
break
Note that I’m forwarding stderr to stdout; this may or may not be what you want, but I’d like to see error messages as well.
This function returns each line as it arrives (usually, you’d have to wait for the subprocess to finish to get the entire output).
In your scenario, the usage would be as follows:
for line in runProcess('mysqladmin create test -uroot -pmysqladmin12'.split()):
print line,
Answered by vartec
Solution #4
I had the same issue, however I worked out a pretty simple solution:
import subprocess
output = subprocess.getoutput("ls -l")
print(output)
I hope it becomes useful.
As a subprocess, this solution is Python3 specific. In Python2, getoutput() does not work.
Answered by azhar22k
Solution #5
Vartec’s response doesn’t read all of the lines, so I wrote one that does.
def run_command(command):
p = subprocess.Popen(command,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
return iter(p.stdout.readline, b'')
The allowed answer’s usage is the same:
command = 'mysqladmin create test -uroot -pmysqladmin12'.split()
for line in run_command(command):
print(line)
Answered by Max Ekman
Post is based on https://stackoverflow.com/questions/4760215/running-shell-command-and-capturing-the-output