Python Subprocess Explained: run, Capture Output, Timeouts and Safety
Python subprocess runs other programs from your code: git, ffmpeg, a shell script or another Python. Learn subprocess.run with captured output, error checking, timeouts, input, environment variables and live streaming with Popen, plus the shell injection we reproduced and the Windows encoding surprise, with every example run on Python 3.13.
- Python subprocess work should start with
subprocess.run([...], capture_output=True, text=True, check=True). - Without
check=True, a program that exited with code 3 raised no error at all. - With
shell=True, the inputreport.txt & echo INJECTEDran a second command; passing a list of arguments kept it as one harmless string. timeout=1stopped a 5-second job after 1.01 seconds withTimeoutExpired.
Sooner or later every Python script needs to run another program: call git to read a commit hash, convert a video with ffmpeg, run a database backup tool, or start another Python script in isolation. The standard library’s Python subprocess module does this safely and portably, and since Python 3.5 one function, subprocess.run, covers almost every case.
We ran every example on Python 3.13.6 on Windows, using sys.executable (the current Python interpreter) as the child program so the examples work on any operating system. Output is copied from the run.
subprocess.run basics
import subprocess, sys
r = subprocess.run([sys.executable, "-c", "print('hello')"])
r.returncode # 0
Pass the command as a list: the program first, then each argument as a separate item. run starts the program, waits for it to finish and returns a CompletedProcess object with the args, returncode, and (if captured) stdout and stderr. A return code of 0 conventionally means success; anything else is an error code chosen by the program.
Capturing output
r = subprocess.run(
[sys.executable, "-c", "print('hello'); import sys; print('warn', file=sys.stderr)"],
capture_output=True, text=True)
type(r).__name__, r.returncode, r.stdout, r.stderr
# ('CompletedProcess', 0, 'hello\n', 'warn\n')
capture_output=True collects standard output and standard error separately instead of printing them to your terminal. text=True decodes the bytes into strings. Note the trailing newline: use r.stdout.strip() or r.stdout.splitlines() before comparing. For a one-liner that returns only stdout, subprocess.check_output([sys.executable, "--version"], text=True) gave 'Python 3.13.6\n'.
Errors and check=True
r = subprocess.run([sys.executable, "-c", "import sys; sys.exit(3)"])
r.returncode # 3 and no exception
try:
subprocess.run([sys.executable, "-c", "import sys; print('oops', file=sys.stderr); sys.exit(3)"],
capture_output=True, text=True, check=True)
except subprocess.CalledProcessError as e:
e.returncode, e.stderr # (3, 'oops\n')
# str(e): Command [...] returned non-zero exit status 3.
This is the most common subprocess bug: a failed command is silently ignored because nobody checked the return code. check=True raises CalledProcessError on any non-zero exit, carrying the return code and captured output, so your normal exception handling applies. Log e.stderr: it usually contains the real reason for the failure.
Timeouts
try:
subprocess.run([sys.executable, "-c", "import time; time.sleep(5)"], timeout=1)
except subprocess.TimeoutExpired as e:
print(e) # ... timed out after 1 seconds
# raised after 1.01 s
Without a timeout, a hung child (a network call that never returns, a prompt waiting for input) hangs your program forever. With timeout, run kills the child and raises TimeoutExpired; it fired after 1.01 seconds in our test. Grandchild processes started by the child may survive the kill, so for complex trees use process groups or a job manager.
When the program does not exist
subprocess.run(["no-such-program-xyz"])
# FileNotFoundError: The system cannot find the file specified
A missing executable raises FileNotFoundError before anything runs, which is different from the program running and failing. Catch both if a tool may not be installed, or check first with shutil.which("git"), which returns the full path or None.
Sending input
r = subprocess.run([sys.executable, "-c", "import sys; print(sys.stdin.read().upper())"],
input="piped text", capture_output=True, text=True)
r.stdout.strip() # 'PIPED TEXT'
input= writes data to the child’s standard input and closes it, which is how you feed text to filters such as grep, jq or a formatter without temporary files.
Environment variables and the working folder
env = dict(os.environ, GREETING="hi from env")
subprocess.run([sys.executable, "-c", "import os; print(os.environ['GREETING'])"],
capture_output=True, text=True, env=env).stdout.strip()
# 'hi from env'
subprocess.run([sys.executable, "-c", "import os; print(os.path.basename(os.getcwd()))"],
capture_output=True, text=True, cwd=os.path.dirname(sys.executable)).stdout.strip()
# 'Python313'
env= replaces the child’s whole environment, so build it from os.environ and add or override keys; passing only your new key removes PATH and breaks most programs. This is the safe way to hand secrets to a tool without putting them on the command line, as described in our environment variables guide. cwd= runs the child in another folder; a pathlib Path works there too.
shell=True and command injection
user_input = "report.txt & echo INJECTED"
subprocess.run(f"echo {user_input}", shell=True, capture_output=True, text=True).stdout.splitlines()
# ['report.txt ', 'INJECTED'] a second command ran
subprocess.run([sys.executable, "-c", "import sys; print(sys.argv[1:])", user_input],
capture_output=True, text=True).stdout.strip()
# "['report.txt & echo INJECTED']" one harmless argument
With shell=True, the command string is handed to the system shell (cmd.exe on Windows, /bin/sh elsewhere), which interprets &, ;, |, $() and backticks. Our “file name” contained & echo INJECTED, and the shell ran it as a second command. A real attacker would use & del or ; curl ... | sh. With a list and no shell, each item is passed directly to the program as one argument, and special characters mean nothing. Only use shell=True with fixed strings you wrote yourself, and if you must include values on Unix, quote them with shlex.quote.
Tip: to split a command line you copied from documentation into a list, use shlex.split("git log -n 5"), which gives ['git', 'log', '-n', '5'].
Redirecting and merging streams
subprocess.run([...], stdout=subprocess.DEVNULL).stdout # None (output discarded)
r = subprocess.run([sys.executable, "-c", "import sys; print('out'); print('err', file=sys.stderr)"],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
r.stdout.split() # ['err', 'out']
DEVNULL throws output away, and stderr=subprocess.STDOUT merges errors into the output stream, which is handy for logs. Notice the order: “err” came before “out” even though the program printed “out” first. Standard output is buffered when it goes to a pipe while standard error is not, so merged streams can appear out of order. For a Python child, pass -u or set PYTHONUNBUFFERED=1 to keep them in order. You can also pass an open file, for example stdout=open("build.log", "w").
The encoding trap on Windows
subprocess.run([sys.executable, "-c", "print('café')"], capture_output=True).stdout
# b'caf\xe9\r\n'
Without text=True you get raw bytes, and here the bytes were not UTF-8: the child Python wrote “é” as \xe9 using the Windows code page because its output was a pipe, and ended the line with \r\n. Decoding that as UTF-8 would raise an error. When the child’s encoding matters, pass encoding="utf-8" (and, for a Python child, set PYTHONUTF8=1 or PYTHONIOENCODING=utf-8 in its environment) so both sides agree.
Streaming output with Popen
p = subprocess.Popen([sys.executable, "-u", "-c",
"import time\nfor i in range(3):\n print('line', i); time.sleep(0.2)"],
stdout=subprocess.PIPE, text=True)
for line in p.stdout:
print("streamed:", line.strip())
p.wait()
# streamed: line 0 / streamed: line 1 / streamed: line 2, then returncode 0
run waits until the program finishes. For long jobs where you want progress as it happens, or to run several programs at once, use Popen directly and read lines as they arrive. Always wait() (or use it as a context manager) so the process is cleaned up and the return code is collected. If you capture both stdout and stderr with Popen, use communicate() instead of reading them yourself, or a full pipe buffer can deadlock both processes.
Start-up cost
for _ in range(20): subprocess.run([sys.executable, "-c", "pass"])
# 20 Python start-ups: 0.95 s
Each new process costs time; starting Python took about 48 ms each here. Calling a tool once is fine; calling it 100,000 times in a loop is not. Batch the work into one call (most tools accept many files), or use a Python library instead. For parallel Python work, the multiprocessing guide covers worker pools, and for waiting on many subprocesses at once, asyncio has create_subprocess_exec.
Subprocess cheat sheet
| Task | Code |
|---|---|
| Run and fail on error | run(cmd, check=True) |
| Get output as text | run(cmd, capture_output=True, text=True).stdout |
| Limit run time | run(cmd, timeout=30) |
| Feed input | run(cmd, input="data", text=True) |
| Extra environment | run(cmd, env={**os.environ, "KEY": "v"}) |
| Different folder | run(cmd, cwd="path") |
| Discard output | run(cmd, stdout=subprocess.DEVNULL) |
| Live output | Popen(cmd, stdout=PIPE, text=True) |
| Find a program | shutil.which("git") |
Log the command and result of every subprocess call with the logging module; it makes failures in scheduled jobs far easier to diagnose.
Final thoughts
Python subprocess is simple once you follow a few rules. Pass commands as lists, add check=True so failures raise, set a timeout, capture output with text=True and an explicit encoding when it matters, and avoid shell=True with any value you did not write yourself. In our tests those rules caught a silent exit code, stopped a hung job in one second and blocked an injected command.
Frequently asked questions
What is the difference between subprocess.run and Popen?
run starts a program, waits for it and returns the result in one call. Popen starts it and returns immediately, so you can stream output, interact with it or run several programs at the same time.
How do I get the output of a command in Python?
Call subprocess.run with capture_output=True and text=True, then read the stdout attribute of the result. Add check=True so a failing command raises an error instead of returning partial output.
Why is shell=True dangerous?
The command string is interpreted by the system shell, so characters like ampersands and semicolons in user input can start extra commands. Pass a list of arguments without the shell instead.
How do I set a timeout for a subprocess?
Pass timeout with a number of seconds. If the program runs longer, Python kills it and raises subprocess.TimeoutExpired, which you can catch and handle.
Should I use os.system or subprocess?
Use subprocess. os.system always uses the shell, cannot capture output and only returns an exit status, while subprocess gives you output, errors, timeouts and safe argument handling.
Keep learning
Need help with testing, WordPress or SEO?
We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.