src.cli module

Classes related to customizing the framework’s command line interface and setting option values from user input.

src.cli.canonical_arg_name(str_)

Convert a flag or other specification to a destination variable name. The destination variable name always has underscores, never hyphens, in accordance with PEP8. Eg., “–GNU-style-flag” -> “GNU_style_flag”.

src.cli.plugin_key(plugin_name)

Ignore spaces and underscores in supplied choices for CLI plugins, and make matching of plugin names case-insensititve.

src.cli.word_wrap(str_)

Clean whitespace and produces better word wrapping for multi-line help

and description strings. Explicit paragraph breaks must be encoded as a double newline ( ``

`` ).

src.cli.read_config_files(code_root, file_name, site='')

Utility function to read a pair of configuration files (one for the framework defaults, another optional one for site-specific config.)

Parameters

file_name (str) – Name of file to search for. We search for the file in all subdirectories of _CLIConfigHandler.site_dir() and _CLIConfigHandler.framework_dir(), respectively.

Returns: a tuple of the two files’ contents. First entry is the

site specific file (empty dict if that file isn’t found) and second is the framework file (fatal error; exit immediately if not found.)

src.cli.read_config_file(code_root, file_name, site='')

Return site’s file if present, else the framework’s file.

class src.cli.CustomHelpFormatter(*args, **kwargs)

Bases: argparse.RawDescriptionHelpFormatter, argparse.ArgumentDefaultsHelpFormatter

Modify help text formatter to only display variable placeholder (“metavar”) once, to save space. Taken from https://stackoverflow.com/a/16969505. Also inherit from RawDescriptionHelpFormatter in order to preserve line breaks in description only (https://stackoverflow.com/a/18462760).

_format_action_invocation(action)

_get_help_string(action)

Only print the argument’s default in the help string if it’s defined. Based on https://stackoverflow.com/a/34545549.

class src.cli.RecordDefaultsAction(option_strings, dest, nargs=None, const=None, default=None, type=None, **kwargs)

Bases: argparse.Action

Action that adds a boolean to record if user actually set argument’s value, or if we’re using the default value specified in the parser. From https://stackoverflow.com/a/50936474. This also re-implements the ‘store_true’ and ‘store_false’ actions, in order to give defaults information on boolean flags.

If the user specifies a value for <option>, the __call__() method adds a variable named <option>_is_default_ to the returned argparse.Namespace. This information is used by MDTFArgParser.parse_args() to populate the is_default attribute of MDTFArgParser.

Subclasses of argparse.Action are only called on user-supplied values, not default values. If the call_on_defaults flag is set on a subclass, MDTFArgParser.parse_args() will also call the action on default values.

default_value_suffix = '_is_default_'

call_on_defaults = False

class src.cli.PathAction(option_strings, dest, nargs=None, const=None, default=None, type=None, **kwargs)

Bases: src.cli.RecordDefaultsAction

Action that performs shell environment variable expansion and resolution of relative paths, using src.util.filesystem.resolve_path().

call_on_defaults = True

class src.cli.ClassImportAction(option_strings, dest, nargs=None, const=None, default=None, type=None, **kwargs)

Bases: src.cli.RecordDefaultsAction

Action to import classes on demand. Values are looked up from the ‘cli_plugins.jsonc’ file. Placeholder used to trigger behavior when arguments are parsed.

call_on_defaults = False

class src.cli.PluginArgAction(option_strings, dest, nargs=None, const=None, default=None, type=None, **kwargs)

Bases: src.cli.ClassImportAction

Action to invoke the CLI plugin functionality. Placeholder used to trigger behavior when arguments are parsed.

call_on_defaults = False

class src.cli.CLIArgument(name: str, action: str = None, nargs: str = None, const: Any = None, default: Any = None, type: Any = None, choices: Iterable = None, required: bool = False, help: str = None, metavar: str = None, dest: str = None, is_positional: bool = False, short_name: str = None, hidden: bool = False)

Bases: object

Class holding configuration options for a single argument of an argparse.ArgumentParser, with several custom options to simplify the parsing of CLIs defined in JSON files.

name: str

action: str = None

nargs: str = None

const: Any = None

default: Any = None

type: Any = None

choices: Iterable = None

required: bool = False

help: str = None

metavar: str = None

dest: str = None

arg_flags: list

is_positional: bool = False

short_name: str = None

hidden: bool = False

add(target_p)

Adds the CLI argument to the parser target_p. Wraps add_argument().

Parameters

target_p – Parser object (or parser group, or subparser) to which the argument will be added.

class src.cli.CLIArgumentGroup(title: str, description: str = None, arguments: list = <factory>)

Bases: object

Class holding configuration options for an argparse.ArgumentParser argument group.

title: str

description: str = None

arguments: list

classmethod from_dict(d)

Initialize an instance of this object from a nested dict obtained from reading a JSON file.

add(target_p)

Adds the CLI argument group, as well as all arguments it contains, to the parser target_p. Wraps add_argument_group().

Parameters

target_p – Parser object (or parser group, or subparser) to which the argument group will be added.

class src.cli.CLIParser(prog: str = None, usage: str = None, description: str = None, epilog: str = None, arguments: list = <factory>, argument_groups: list = <factory>)

Bases: object

Class holding configuration options for an instance of argparse.ArgumentParser (or equivalently a subparser or a command plugin).

prog: str = None

usage: str = None

description: str = None

epilog: str = None

arguments: list

argument_groups: list

classmethod from_dict(d)

Initialize an instance of this object from a nested dict obtained from reading a JSON file.

iter_args(filter_class=None)

Iterator over all CLIArgument objects associated with this parser.

configure(target_p)

Configures a parser object by setting top-level attributes and adding all arguments and argument groups.

Parameters

target_p – Parser object (or parser group, or subparser) to configure.

add_plugin_args(preparsed_d)

Revise arguments after we know what plugins are being used. This annotates the help string of the plugin selector argument and configures its choices attribute. It then inserts the plugin-specifc CLI arguments following that argument.

Parameters

preparsed_d – dict of results of the preparsing operation. Keys are the destination strings of the plugin selector arguments (identified by having their action set to PluginArgAction), and values are the values assigned to them by preparsing.

class src.cli.CLICommand(name: str, entry_point: str, help: str = '', cli_file: str = None, cli: dict = None, parser: Field(name=None, type=None, default=<dataclasses._MISSING_TYPE object at 0x7f180ca83310>, default_factory=<dataclasses._MISSING_TYPE object at 0x7f180ca83310>, init=False, repr=True, hash=None, compare=True, metadata=mappingproxy({}), _field_type=None) = None, code_root: dataclasses.InitVar = '')

Bases: object

Class holding configuration options for a subcommand (invoked via a subparser) or a plugin.

name: str

entry_point: str

help: str = ''

cli_file: str = None

cli: dict = None

parser: Field(name=None,type=None,default=<dataclasses._MISSING_TYPE object at 0x7f180ca83310>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f180ca83310>,init=False,repr=True,hash=None,compare=True,metadata=mappingproxy({}),_field_type=None) = None

code_root: dataclasses.InitVar = ''

import_target()

Imports the function or class referred to by the entry_point attribute.

call(*args, **kwargs)

Imports the function or class referred to by the entry_point attribute, and calls it with the passed arguments.

class src.cli.DefaultsFileTypes(value)

Bases: src.util.basic.MDTFEnum

MDTFEnum to distinguish the three different categories of input settings files. In order of precedence:

1. USER: Input settings read from a file supplied by the user.

2. SITE: Settings specific to the given site (--site flag.)

3. GLOBAL: Settings applicable to all sites. The main intended use case

   of this file is to enable the user to configure a default site at install time.

USER = 1

SITE = 2

GLOBAL = 3

class src.cli.CLIConfigManager(*args, **kwargs)

Bases: src.util.basic.Singleton

Singleton to handle search, loading and parsing of configuration files for the CLI and CLI default values. We encapsulate this functionality in its own class, instead of MDTFArgParser or its children, to try to make the code easier to understand (not out of necessity).

Warning

This is intended to be initialized by a calling script before being referenced by the classes in this module.

default_site = 'local'

defaults_filename = 'defaults.jsonc'

subcommands_filename = 'cli_subcommands.jsonc'

plugins_filename = 'cli_plugins.jsonc'

property framework_dir

property sites_dir

property site_dir

read_defaults(def_type, path=None)

Populate one of the entries in self.defaults by reading from the appropriate defaults file.

Parameters

• def_type (DefaultsFileTypes) – Type of defaults file to read.

• path (str, optional) – path of the file. Only used for user-specified defaults.

site_default_text()

read_subcommands()

Populates subcommands and subparser_kwargs attributes with contents of CLI plugin files for the framework and site. Site-specific subcommand definitions override those defined on the framework.

read_plugins()

Populates plugins attribute with contents of CLI plugin files for the framework and site.

get_plugin(plugin_name, choice_of_plugin=None)

Lookup requested CLI plugin from plugins attribute, logging appropriate errors where KeyErrors would be raised.

Parameters

• plugin_name (str) – Name of the plugin selected.

• choice_of_plugin (str, optional) – if provided, the name of the choice of plugin.

Returns: CLICommand object corresponding to the requested

plugin choice if both arguments are given, or a dict of recognized choices if only the first argument is given.

class src.cli.MDTFArgParser(*args, **kwargs)

Bases: argparse.ArgumentParser

Customized argparse.ArgumentParser. Added functionality:

• Configuring the parser from an external file (configure()).

• Customized help text formatting provided by CustomHelpFormatter.

• Recording whether the user specified each argument value, or whether the default was used, via RecordDefaultsAction.

• Better bookkeeping of argument groups, e.g. which arguments belong to which group.

static split_args(argv)

Wrapper for shlex.split().

iter_actions()

Iterator over Action objects associated with all user-defined arguments in parser, as well as those for any subcommands.

_set_is_default(parsed_args)

Populates the is_default attribute based on whether the user explicitly specified a value, or whether a default was used.

Parameters

parsed_args – dict of args and values returned by initial parsing.

_call_actions_on_defaults(namespace)

Subclasses of argparse.Action are only called on user-supplied values, not default values. If the call_on_defaults flag has been set on our custom actions, call the action on default values to do the same parsing for default values that we would’ve done for user input.

_default_argv(parsed_args)

Utility method returning the arguments passed to the parser for lowest-priority defaults in MDTFArgParser.parse_known_args().

parse_known_args(args=None, namespace=None)

Wrapper for parse_known_args() which handles intermediate levels of default settings derived from the user’s settings files. These override defaults defined in the parser itself. The precedence order is:

1. Argument values explictly given by the user on the command line, as recorded in the is_default attribute of MDTFArgParser.

2. Argument values from a file the user gave via the -f flag. (CLIConfigManager.defaults[DefaultsFileTypes.USER]).

3. Argument values specified as the default values in the argument parser, which in turn are set with the following precedence order:

   1. Default values from a site-specfic file (defaults.jsonc), stored in CLIConfigManager.defaults[DefaultsFileTypes.SITE].

   2. Default values from a defaults.jsonc file in the sites/ directory, stored in CLIConfigManager.defaults[DefaultsFileTypes.GLOBAL].

   3. Default values hard-coded in the CLI definition file itself.

Parameters

• args (optional) – String or list of strings to parse. If a single string is passed, it’s split using shlex.split(). If not supplied, the default behavior parses sys.argv().

• namespace (optional) – An object to store the parsed arguments. The default is a new empty argparse.Namespace object.

Returns

Tuple of 1) populated namespace containing parsed arguments and 2) unrecognized arguments, as with argparse.ArgumentParser.parse_known_args().

parse_args(args=None, namespace=None)

Subclassed implementation of parse_args() which wraps parse_known_args().

class src.cli.MDTFArgPreparser

Bases: src.cli.MDTFArgParser

Parser class used to “preparse” plugin selector arguments, to determine which plugins to use. Plugin selector arguments are identified by having their action set to PluginArgAction.

parse_site(argv=None, default_site=None)

Wrapper for parse_known_args() used to determine what site to use.

parse_input_file(argv=None)

parse_plugins(argv=None)

Wrapper for parse_known_args() used to parse the plugin selector arguments.

class src.cli.MDTFTopLevelArgParser(code_root, skip_defaults=False, argv=None)

Bases: src.cli.MDTFArgParser

Class for constructing the command-line interface, parsing the options, and handing off execution to the selected subcommand.

iter_arg_groups(subcommand=None)

iter_group_actions(subcommand=None, group=None)

add_input_file_arg(target_p)

Convenience method to add the flag to pass a user-designated defaults file to the parser target_p (either the top-level parser, or the preparser.)

init_user_defaults()

Set user defaults using values read in from a configuration file in one of two formats.

Parameters

• config_str (str) – contents of the configuration file, either:

• A JSON/JSONC file of key-value pairs. This is parsed using (1.) – parse_json().

• A plain text file containing flags and arguments as they would (2.) – be passed on the command line (except shell expansions are not performed). This is parsed by the MDTFArgParser.parse_args() method of the configured parser.

The format is determined from context. ValueError is raised if the string cannot be parsed.

add_site_arg(target_p)

Convenience method to add the argument flag to select which site-specific code to use, to the parser target_p (either the top-level parser, or the preparser.)

init_site()

We allow site-specific installations to customize the CLI, so before we construct the CLI parser we need to determine what site to use. We do this by running a parser that only looks for the --site flag.

This sets the site attribute and populates the global and site-specific settings dicts.

add_contents(target_p)

Convenience method to fully configure a parser target_p (either the top-level parser, or the preparser), adding subparsers for all subcommands.

setup()

Method to wrap all configuration methods needed to configure the parser before calling parse_arg: reading the defaults files and configuring plugins based on existing values.

configure()

Method that assembles the top-level CLI parser. Options specific to the script are hard-coded here; CLI options for each subcommand are given in jsonc configuration files for each command which are read in here. See associated documentation for MDTFArgParser for information on the configuration file mechanism.

parse_args(args=None, namespace=None)

Wrapper for parse_args() which handles intermediate levels of default settings.

parse_known_args(args=None, namespace=None)

Wrapper for parse_known_args() which handles intermediate levels of default settings.

dispatch(args=None)

Parse args, and call the subcommand that was selected.

class src.cli.MDTFTopLevelSubcommandArgParser(code_root, skip_defaults=False, argv=None)

Bases: src.cli.MDTFTopLevelArgParser

Implement top-level parser with multiple subcommands.

_default_argv(parsed_args)

Utility method returning the arguments passed to the parser for lowest-priority defaults in MDTFArgParser.parse_known_args().

add_contents(target_p)

Convenience method to fully configure a parser target_p (either the top-level parser, or the preparser), adding subparsers for all subcommands.

configure()

Method that assembles the top-level CLI parser. Options specific to the script are hard-coded here; CLI options for each subcommand are given in jsonc configuration files for each command which are read in here. See associated documentation for MDTFArgParser for information on the configuration file mechanism.

dispatch()

Parse args, and call the subcommand that was selected.