src.util.basic module

Classes and functions that define and operate on basic data structures.

src.util.basic.abstract_attribute(obj=None)

Decorator for abstract attributes in abstract base classes by analogy with abc.abstract_method(). Based on https://stackoverflow.com/a/50381071.

class src.util.basic.MDTFABCMeta(name, bases, namespace, **kwargs)

Bases: abc.ABCMeta

Wrap the metaclass for abstract base classes to enable definition of abstract attributes via abstract_attribute(). Based on https://stackoverflow.com/a/50381071.

Raises

NotImplementedError – If a child class doesn’t define an abstract attribute, by analogy with abc.abstract_method().

__init__(*args, **kwargs)

Initialize self. See help(type(self)) for accurate signature.

mro()

Return a type’s method resolution order.

register(subclass)

Register a virtual subclass of an ABC.

Returns the subclass, to allow usage as a class decorator.

class src.util.basic.Singleton(*args, **kwargs)

Bases: src.util.basic.SingletonMeta

Parent class defining the Singleton pattern. We use this as safer way to pass around global state.

__init__()

Initialize self. See help(type(self)) for accurate signature.

class src.util.basic.MultiMap(*args, **kwargs)

Bases: collections.defaultdict

Extension of the dict class that allows doing dictionary lookups from either keys or values.

Syntax for lookup from keys is unchanged, while lookup from values is done on the inverse() attribute and returns a list of matching keys if more than one match is present. See https://stackoverflow.com/a/21894086.

Example:

>>> d = MultiMap({'key1': 'val', 'key2':'val'})

>>> d['key1']
'val'

>>> d.inverse['val']
['key1', 'key2']

__init__(*args, **kwargs)

Inherited from collections.defaultdict. Construct by passing an ordinary dict.

get_(key)

Re-implement __getitem__ to handle returning possibly multiple items.

to_dict()

Convert to ordinary dict.

inverse()

Construct inverse dict mapping values to keys.

inverse_get_(val)

Construct inverse dict and return keys corresponding to val.

default_factory

Factory for default value called by __missing__().

fromkeys(value=None, /)

Create a new dictionary with keys from iterable and values set to value.

class src.util.basic.WormDict(**kwargs)

Bases: collections.UserDict, dict

Dict which raises exceptions when trying to overwrite or delete an existing entry. “WORM” is an acronym for “write once, read many.”

Raises

WormKeyError – If code attempts to reassign or delete an existing key.

classmethod from_struct(d)

Construct a WormDict from a dict d. Intended to be used for automatic type coercion done on fields of a mdtf_dataclass().

__init__(**kwargs)

Initialize self. See help(type(self)) for accurate signature.

clear() → None. Remove all items from D.

copy() → a shallow copy of D

classmethod fromkeys(iterable, value=None)

Create a new dictionary with keys from iterable and values set to value.

get(k[, d]) → D[k] if k in D, else d. d defaults to None.

items() → a set-like object providing a view on D’s items

keys() → a set-like object providing a view on D’s keys

pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D

update([E, ]**F) → None. Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() → an object providing a view on D’s values

class src.util.basic.ConsistentDict(**kwargs)

Bases: src.util.basic.WormDict

Like WormDict, but we only raise WormKeyError if we try to reassign to a different value.

Raises

WormKeyError – If code attempts to reassign an existing key to a value different than the current one.

__init__(**kwargs)

Initialize self. See help(type(self)) for accurate signature.

clear() → None. Remove all items from D.

copy() → a shallow copy of D

classmethod from_struct(d)

Construct a WormDict from a dict d. Intended to be used for automatic type coercion done on fields of a mdtf_dataclass().

classmethod fromkeys(iterable, value=None)

Create a new dictionary with keys from iterable and values set to value.

get(k[, d]) → D[k] if k in D, else d. d defaults to None.

items() → a set-like object providing a view on D’s items

keys() → a set-like object providing a view on D’s keys

pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D

update([E, ]**F) → None. Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() → an object providing a view on D’s values

class src.util.basic.WormDefaultDict(default_factory=None, *args, **kwargs)

Bases: src.util.basic.WormDict

src.util.basic.WormDict with collections.defaultdict functionality.

__init__(default_factory=None, *args, **kwargs)

Inherited from collections.defaultdict and takes same arguments.

clear() → None. Remove all items from D.

copy() → a shallow copy of D

classmethod from_struct(d)

Construct a WormDict from a dict d. Intended to be used for automatic type coercion done on fields of a mdtf_dataclass().

classmethod fromkeys(iterable, value=None)

Create a new dictionary with keys from iterable and values set to value.

get(k[, d]) → D[k] if k in D, else d. d defaults to None.

items() → a set-like object providing a view on D’s items

keys() → a set-like object providing a view on D’s keys

pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D

update([E, ]**F) → None. Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() → an object providing a view on D’s values

class src.util.basic.NameSpace

Bases: dict

A dictionary that provides attribute-style access.

For example, d[‘key’] = value becomes d.key = value. All methods of dict are supported.

Note

Recursive access (d.key.subkey, as in C-style languages) is not supported.

Implementation is based on https://github.com/Infinidat/munch.

Raises

AttributeError – In cases where dict would raise a KeyError.

toDict()

Recursively converts a NameSpace back into a dictionary.

classmethod fromDict(x)

Recursively transforms a dictionary into a NameSpace via copy. (Note: as dicts are not hashable, they cannot be nested in sets/frozensets.)

__init__(*args, **kwargs)

Initialize self. See help(type(self)) for accurate signature.

fromkeys(value=None, /)

Create a new dictionary with keys from iterable and values set to value.

class src.util.basic.MDTFEnum(value)

Bases: src.util.basic._MDTFEnumMixin, enum.Enum

Customize behavior of Enum:

1. Assign (integer) values automatically to the members of the enumeration.

2. Provide a from_struct() method to simplify instantiating an instance from a string. Intended to be used for automatic type coercion done on fields of a mdtf_dataclass(). To avoid potential confusion with reserved keywords, we use the Python convention that members of the enumeration are all uppercase.

class src.util.basic.MDTFIntEnum(value)

Bases: src.util.basic._MDTFEnumMixin, enum.IntEnum

Customize IntEnum analogous to MDTFEnum.

src.util.basic.sentinel_object_factory(obj_name)

Return a unique singleton object/class (same difference for singletons). For implementation, see python docs.

class src.util.basic.MDTF_ID(id_=None)

Bases: object

Class wrapping UUID, to provide unique ID numbers for members of the object hierarchy (cases, pods, variables, etc.), so that we don’t need to require that objects in these classes have unique names.

__init__(id_=None)

Parameters

id_ (optional) – hard-code an ID instead of generating one.

src.util.basic.is_iterable(obj)

Test if obj is an iterable collection.

Parameters

obj – Object to test.

Returns

True if obj is an iterable collection and not a string.

Return type

bool

src.util.basic.to_iter(obj, coll_type=<class 'list'>)

Cast arbitrary object obj to an iterable collection. If obj is not a collection, returns a one-element list containing obj.

Parameters

• obj – Object to cast to collection.

• coll_type – One of list, set or tuple, default list. Class to cast obj to.

Returns

obj, cast to an iterable collection of type coll_type.

src.util.basic.from_iter(obj)

Inverse of to_iter(). If obj is a single-element iterable collection, return its only element.

src.util.basic.remove_prefix(s1, s2)

If string s1 starts with string s2, return s1 with s2 removed. Otherwise return s1 unmodified.

src.util.basic.remove_suffix(s1, s2)

If string s1 ends with string s2, return s1 with s2 removed. Otherwise return s1 unmodified.

src.util.basic.filter_kwargs(kwarg_dict, function)

Given keyword arguments kwarg_dict, return only those kwargs accepted by function.

Parameters

• kwarg_dict (dict) – Keyword arguments to be passed to function.

• function (function) – Function to be called.

Returns

Subset of key:value entries of kwarg_dict where keys are keyword arguments recognized by function.

Return type

dict

src.util.basic.splice_into_list(list_, splice_d, key_fn=None, log=<Logger>)

Splice sub-lists (values of splice_d) into list list_ after their corresponding entries (keys of slice_d). Example:

>>> splice_into_list(['a','b','c'], {'b': ['b1', 'b2']})
['a', 'b', 'b1', 'b2', 'c']

Parameters

• list_ (list) – Parent list to splice sub-lists into.

• splice_d (dict) – Sub-lists to splice in. Keys are entries in list_ and values are the sub-lists to insert after that entry. Duplicate or missing entries are handled appropriately.

• key_fn (function) – Optional. If supplied, function applied to elements of list_ to compare to keys of splice_d.

Returns

Spliced list_ as described above.

src.util.basic.deserialize_class(name)

Given the name of a currently defined class, return the class itself. This avoids security issues with calling eval(). Based on https://stackoverflow.com/a/11781721.

Parameters

name (str) – name of the class to look up.

Returns

class with the given name, if currently imported.

Raises

ValueError – If class not found in current namespace.