src.util.processes module

Utility functions for dealing with subprocesses.

class src.util.processes.ExceptionPropagatingThread(group=None, target=None, name=None, args=(), kwargs=None, *, daemon=None)

Bases: threading.Thread

Class to propagate exceptions raised in a child thread back to the caller thread when the child is join()ed. Adapted from https://stackoverflow.com/a/31614591.

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

join(timeout=None)

Wait until the thread terminates.

This blocks the calling thread until the thread whose join() method is called terminates – either normally or through an unhandled exception or until the optional timeout occurs.

When the timeout argument is present and not None, it should be a floating point number specifying a timeout for the operation in seconds (or fractions thereof). As join() always returns None, you must call is_alive() after join() to decide whether a timeout happened – if the thread is still alive, the join() call timed out.

When the timeout argument is not present or None, the operation will block until the thread terminates.

A thread can be join()ed many times.

join() raises a RuntimeError if an attempt is made to join the current thread as that would cause a deadlock. It is also an error to join() a thread before it has been started and attempts to do so raises the same exception.

src.util.processes.poll_command(command, shell=False, env=None)

Runs a shell command and prints stdout in real-time.

Optional ability to pass a different environment to the subprocess. See documentation for the Python2 subprocess module.

Parameters

• command – list of command + arguments, or the same as a single string. See subprocess syntax. Note this interacts with the shell setting.

• shell (bool, optional) – shell flag, passed to Popen, default False.

• env (dict, optional) – environment variables to set, passed to Popen, default None.

src.util.processes.run_command(command, env=None, cwd=None, timeout=0, dry_run=False, log=<Logger src.util.processes (WARNING)>)

Subprocess wrapper to facilitate running single command without starting a shell.

Note

We hope to save some process overhead by not running the command in a shell, but this means the command can’t use piping, quoting, environment variables, or filename globbing etc.

See documentation for the Python2 subprocess module.

Parameters

• command (list of str) – List of commands to execute

• env (dict, optional) – environment variables to set, passed to Popen, default None.

• cwd (str, optional) – child processes’ working directory, passed to Popen. Default is None, which uses parent processes’ directory.

• timeout (int, optional) – Optionally, kill the command’s subprocess and raise a MDTFCalledProcessError if the command doesn’t finish in timeout seconds.

Returns

list of str containing output that was written to stdout by each command. Note: this is split on newlines after the fact.

Raises

MDTFCalledProcessError – If any commands return with nonzero exit code. Stderr for that command is stored in output attribute.

src.util.processes.run_shell_command(command, env=None, cwd=None, dry_run=False, log=<Logger src.util.processes (WARNING)>)

Subprocess wrapper to facilitate running shell commands.

See documentation for the Python2 subprocess module.

Parameters

• commands (list of str) – List of commands to execute

• env (dict, optional) – environment variables to set, passed to Popen, default None.

• cwd (str, optional) – child processes’ working directory, passed to Popen. Default is None, which uses parent processes’ directory.

Returns

list of str containing output that was written to stdout by each command. Note: this is split on newlines after the fact, so if commands give != 1 lines of output this will not map to the list of commands given.

Raises

MDTFCalledProcessError – If any commands return with nonzero exit code. Stderr for that command is stored in output attribute.