src.core module

Common functions and classes used in multiple places in the MDTF code.

class src.core.ObjectStatus(value)

Bases: src.util.basic.MDTFEnum

util.MDTFEnum used to track the status of a MDTFObjectBase:

• NOTSET: the object hasn’t been fully initialized.

• ACTIVE: the object is currently being processed by the framework.

• INACTIVE: the object has been initialized, but isn’t being processed (e.g.,

  alternate VarlistEntrys).

• FAILED: processing of the object has encountered an error, and no further

  work will be done.

NOTSET = 1

ACTIVE = 2

INACTIVE = 3

FAILED = 4

SUCCEEDED = 5

class src.core.MDTFObjectBase(*args, **kwargs)

Bases: object

Base class providing shared functionality for the “object hierarchy”:

• DataSourceBases belonging to a run of the package

  (MDTFFramework);

• Diagnostics (PODs) belonging to a

  DataSourceBase;

• VarlistEntrys (requested model variables) belonging

  to a Diagnostic.

_id: src.util.basic.MDTF_ID = None

name: str = sentinel.Mandatory

_parent: Any = sentinel.Mandatory

status: src.core.ObjectStatus = 1

property _log_name

property full_name

property failed

property active

abstract property _children

Iterable of child objects associated with this object.

iter_children(child_type=None, status=None, status_neq=None)

Generator iterating over child objects associated with this object.

Parameters

• status – None or ObjectStatus, default None. If None, iterates over all child objects, regardless of status. If a ObjectStatus value is passed, only iterates over child objects with that status.

• status_neq – None or ObjectStatus, default None. If set, iterates over child objects which don’t have the given status. If status is set, this setting is ignored.

child_deactivation_handler(child, exc)

child_status_update(exc=None)

_deactivation_log_level = 40

deactivate(exc, level=None)

_abc_impl = <_abc_data object>

class src.core.ConfigTuple(name, backup_filename, contents)

Bases: tuple

Class wrapping general structs used for configuration

_asdict()

Return a new OrderedDict which maps field names to their values.

_field_defaults = {}

_fields = ('name', 'backup_filename', 'contents')

_fields_defaults = {}

classmethod _make(iterable)

Make a new ConfigTuple object from a sequence or iterable

_replace(**kwds)

Return a new ConfigTuple object replacing specified fields with new values

property backup_filename

Alias for field number 1

property contents

Alias for field number 2

property name

Alias for field number 0

class src.core.ConfigManager(*args, **kwargs)

Bases: src.util.basic.Singleton, src.util.basic.NameSpace

backup_config(cli_obj, case_d)

Copy serializable version of parsed settings, in order to write backup config file.

class src.core.PathManager(*args, **kwargs)

Bases: src.util.basic.Singleton, src.util.basic.NameSpace

Singleton holding the root directories for all paths used by the code.

_init_path(key, d, env=None)

model_paths(case, overwrite=False)

pod_paths(pod, case)

class src.core.TempDirManager(*args, **kwargs)

Bases: src.util.basic.Singleton

_prefix = 'MDTF_temp_'

make_tempdir(hash_obj=None)

rm_tempdir(path)

cleanup()

tempdir_cleanup_handler(signum=None, frame=None)

class src.core.TranslatedVarlistEntry(coords: dataclasses.InitVar = None, name: str = sentinel.Mandatory, standard_name: str = sentinel.Mandatory, units: src.units.Units = sentinel.Mandatory, convention: str = sentinel.Mandatory, log: Any = sentinel.Mandatory)

Bases: src.data_model.DMVariable

Class returned by VarlistTranslator.translate(). Marks some attributes inherited from DMVariable as being queryable in data_manager.DataframeQueryDataSourceBase.query_dataset().

convention: str = sentinel.Mandatory

name: str = sentinel.Mandatory

standard_name: str = sentinel.Mandatory

units: src.units.Units = sentinel.Mandatory

scalar_coords: list

log: Any = sentinel.Mandatory

class src.core.FieldlistEntry(coords: dataclasses.InitVar = None, name: str = sentinel.Mandatory, standard_name: str = sentinel.Mandatory, units: src.units.Units = '', scalar_coord_templates: dict = <factory>)

Bases: src.data_model.DMDependentVariable

Class corresponding to an entry in a fieldlist file.

scalar_coord_templates: dict

_ndim_to_axes_set = {1: 'PLACEHOLDER_T_COORD', 2: ('PLACEHOLDER_Y_COORD', 'PLACEHOLDER_X_COORD'), 3: ('PLACEHOLDER_T_COORD', 'PLACEHOLDER_Y_COORD', 'PLACEHOLDER_X_COORD'), 4: ('PLACEHOLDER_T_COORD', 'PLACEHOLDER_Z_COORD', 'PLACEHOLDER_Y_COORD', 'PLACEHOLDER_X_COORD')}

_placeholder_class_dict = {'PLACEHOLDER_COORD': <class 'src.data_model.DMPlaceholderCoordinate'>, 'PLACEHOLDER_T_COORD': <class 'src.data_model.DMPlaceholderTCoordinate'>, 'PLACEHOLDER_X_COORD': <class 'src.data_model.DMPlaceholderXCoordinate'>, 'PLACEHOLDER_Y_COORD': <class 'src.data_model.DMPlaceholderYCoordinate'>, 'PLACEHOLDER_Z_COORD': <class 'src.data_model.DMPlaceholderZCoordinate'>}

classmethod from_struct(dims_d, name, **kwargs)

scalar_name(old_coord, new_coord, log=<Logger src.core (WARNING)>)

Uses one of the scalar_coord_templates to construct the translated variable name for this variable on a scalar coordinate slice (eg. pressure level).

class src.core.Fieldlist(name: str = sentinel.Mandatory, axes: src.util.basic.WormDict = <factory>, axes_lut: src.util.basic.WormDict = <factory>, entries: src.util.basic.WormDict = <factory>, lut: src.util.basic.WormDict = <factory>, env_vars: dict = <factory>)

Bases: object

Class corresponding to a single variable naming convention (single file in data/fieldlist_*.jsonc).

TODO: implement more robust indexing/lookup scheme. standard_name is not a unique identifier, but should include cell_methods, etc. as well as dimensionality.

name: str = sentinel.Mandatory

axes: util.WormDict

axes_lut: util.WormDict

entries: util.WormDict

lut: util.WormDict

env_vars: dict

classmethod from_struct(d)

to_CF(var_or_name)

Returns FieldlistEntry for the variable having the given name in this convention.

to_CF_name(var_or_name)

Like to_CF(), but only return the CF standard name, given the name in this convention.

from_CF(var_or_name, axes_set=None)

Look up FieldlistEntry corresponding to the given standard name, optionally providing an axes_set to resolve ambiguity.

TODO: this is a hacky implementation; FieldlistEntry needs to be expanded with more ways to uniquely identify variable (eg cell methods).

from_CF_name(var_or_name, axes_set=None)

Like from_CF(), but only return the variable’s name in this convention.

translate_coord(coord, log=<Logger src.core (WARNING)>)

Given a DMCoordinate, look up the corresponding translated DMCoordinate in this convention.

translate(var)

Returns TranslatedVarlistEntry instance, with populated coordinate axes. Units of scalar coord slices are translated to the units of the conventions’ coordinates. Includes logic to translate and rename scalar coords/slices, e.g. VarlistEntry for ‘ua’ (intrinsically 4D) @ 500mb could produce a TranslatedVarlistEntry for ‘u500’ (3D slice), depending on naming convention.

class src.core.NoTranslationFieldlist(*args, **kwargs)

Bases: src.util.basic.Singleton

Class which partially implements the Fieldlist interface but does no variable translation. VarlistEntry objects from the POD are passed through to create TranslatedVarlistEntry objects.

to_CF(var_or_name)

to_CF_name(var_or_name)

from_CF(var_or_name, axes_set=None)

from_CF_name(var_or_name, axes_set=None)

translate_coord(coord, log=<Logger src.core (WARNING)>)

translate(var)

Returns TranslatedVarlistEntry instance, populated with contents of input VarlistEntry instance.

Note

We return a copy of the VarlistEntry because logic in DefaultDatasetParser alters the translation based on the file’s actual contents.

class src.core.VariableTranslator(*args, **kwargs)

Bases: src.util.basic.Singleton

Singleton containing information for different variable naming conventions. These are defined in the data/fieldlist_*.jsonc files.

add_convention(d)

get_convention_name(conv_name)

Resolve the naming convention associated with a given Fieldlist object from among a set of possible aliases.

get_convention(conv_name)

Return the Fieldlist object containing the variable name translation logic for a given convention name.

_fieldlist_method(conv_name, method_name, *args, **kwargs)

Wrapper which determines the requested convention and calls the requested method_name on the Fieldlist object for that convention.

to_CF(conv_name, name)

to_CF_name(conv_name, name)

from_CF(conv_name, standard_name, axes_set=None)

from_CF_name(conv_name, standard_name, axes_set=None)

translate_coord(conv_name, coord, log=<Logger src.core (WARNING)>)

translate(conv_name, var)

class src.core.MDTFFramework(*args, **kwargs)

Bases: src.core.MDTFObjectBase

property _children

Iterable of child objects associated with this object.

property full_name

configure(cli_obj, pod_info_tuple, log_config)

Wrapper for all configuration done based on CLI arguments.

_cli_post_parse_hook(cli_obj)

dispatch_classes(cli_obj)

static _populate_from_cli(cli_obj, group_nm, target_d=None)

parse_mdtf_args(cli_obj, pod_info_tuple)

Parse script options returned by the CLI. For greater customizability, most of the functionality is spun out into sub-methods.

parse_flags(cli_obj)

parse_env_vars(cli_obj)

parse_pod_list(pod_list, pod_info_tuple)

set_case_pod_list(case, cli_obj, pod_info_tuple)

parse_case(n, d, cli_obj, pod_info_tuple)

parse_case_list(cli_obj, pod_info_tuple)

verify_paths(config, p)

_abc_impl = <_abc_data object>

_post_parse_hook(cli_obj, config, paths)

_print_config(cli_obj, config, paths)

Log end result of parsing package settings. This is only for the user’s benefit; a machine-readable version which is usable for provenance/reproducibility is saved by the OutputManager as config_save.jsonc.

property failed

Overall success/failure of this run of the framework. Return True if any case or any POD has failed, else return False.

main()

src.core.print_summary(fmwk)