src.cmip6 module

Code to parse CMIP6 controlled vocabularies and elements of the CMIP6 DRS.

Specifications for the above were taken from the planning document http://goo.gl/v1drZl, which doesn’t seem to have a permanent link. The CMIP6 controlled vocabularies (lists of registered MIPs, modeling centers, etc.) are derived from data in the PCMDI/cmip6-cmor-tables repo, which is included as a subtree under /data.

Warning

Functionality here has been added as needed for the project and is incomplete, for example parsing subexperiments is not supported.

class src.cmip6.CMIP6_CVs(*args, **kwargs)[source]

Bases: src.util.basic.Singleton

Interface for looking up information from the CMIP6 CV file.

Note

Lookups are implemented in an ad-hoc way with util.MultiMap; a more robust solution would use sqlite.

_make_cv()[source]

Populate the cv attribute of CMIP6_CVs with the tables read in during __init__().

Do this on-demand rather than in __init__, in case this information isn’t needed for this run of the framework.

is_in_cv(category, items)[source]

Determine if items take values that are valid for the CV category category.

Parameters

  • category (str) – the CV category to use to validate values.

  • items (str or list of str) – Entries whose validity we’d like to check.

Returns: boolean or list of booleans, corresponding to the validity of

the entries in items.

get_lookup(source, dest)[source]

Find the appropriate lookup table to convert values in source (keys) to values in dest (values), generating it if necessary.

Parameters

  • source (str) – the CV category to use for the keys.

  • dest (str) – the CV category to use for the values.

Returns: util.MultiMap providing a dict-like lookup interface,

ie dest_value = d[source_key].

lookup(source_items, source, dest)[source]

Lookup the corresponding dest values for source_items (keys).

Parameters

  • source_items (str or list) – one or more keys

  • source (str) – the CV category that the items in source_items belong to.

  • dest (str) – the CV category we’d like the corresponding values for.

Returns: list of dest values corresponding to each entry in source_items.

lookup_single(source_item, source, dest)[source]

The same as lookup(), but perform lookup for a single source_item, and raise KeyError if the number of values returned is != 1.

table_id_from_freq(frequency)[source]

Specialized lookup to determine which MIP tables use data at the requested frequency.

Should really be handled as a special case of lookup().

Parameters

frequency (CMIP6DateFrequency) – DateFrequency

Returns: list of MIP table table_id names, if any, that use data at

the given frequency.

class src.cmip6.CMIP6DateFrequency(quantity, unit=None)[source]

Bases: src.util.datelabel.DateFrequency

Subclass of src.util.datelabel.DateFrequency to parse data frequency information as encoded in MIP tables, DRS filenames, etc.

Extends DateFrequency in that this records if the data is a climatological average, although this information is not currently used.

Reference: http://goo.gl/v1drZl page 16.

_precision_lookup = {'day': 3, 'fx': 0, 'hr': 5, 'min': 6, 'mo': 2, 'yr': 1}
_regex = re.compile('\n ^\n (?P<quantity>(1|3|6)?)\n (?P<unit>[a-z]*?)\n (?P<avg>(C|CM|Pt)?)\n $\n ', re.VERBOSE)
classmethod _parse_input_string(quantity, unit)[source]
format()[source]
class src.cmip6.CMIP6_VariantLabel(first_arg=None, *args, **kwargs)[source]

Bases: object

Dataclass which represents and parses the CMIP6 DRS variant label identifier string.

References: https://earthsystemcog.org/projects/wip/mip_table_about, although this doesn’t document all cases used in CMIP6. See also note 8 on page 9 of http://goo.gl/v1drZl.

variant_label: str = sentinel.Mandatory
realization_index: int = None
initialization_index: int = None
physics_index: int = None
forcing_index: int = None
_is_regex_dataclass = True
_pattern = {}
classmethod from_string(str_, *args)
class src.cmip6.CMIP6_MIPTable(first_arg=None, *args, **kwargs)[source]

Bases: object

Dataclass which represents and parses MIP table identifier string.

Reference: https://earthsystemcog.org/projects/wip/mip_table_about, although this doesn’t document all cases used in CMIP6.

table_id: str = sentinel.Mandatory
table_prefix: str = ''
table_freq: dataclasses.InitVar = ''
table_suffix: str = ''
table_qualifier: str = ''
frequency: CMIP6DateFrequency
spatial_avg: str
temporal_avg: str
region: str
_is_regex_dataclass = True
_pattern = {}
classmethod from_string(str_, *args)
class src.cmip6.CMIP6_GridLabel(first_arg=None, *args, **kwargs)[source]

Bases: object

Dataclass which represents and parses the CMIP6 DRS grid label identifier string.

Reference: http://goo.gl/v1drZl, note 11 on page 11.

grid_label: str = sentinel.Mandatory
global_mean: dataclasses.InitVar = ''
regrid: str = ''
grid_number: int = 0
region: str = ''
zonal_mean: dataclasses.InitVar = ''
spatial_avg: str
native_grid: bool
_is_regex_dataclass = True
_pattern = {}
classmethod from_string(str_, *args)
class src.cmip6.CMIP6_DRSDirectory(first_arg=None, *args, **kwargs)[source]

Bases: src.cmip6.CMIP6_VariantLabel, src.cmip6.CMIP6_MIPTable, src.cmip6.CMIP6_GridLabel

Dataclass which represents and parses the DRS directory, using regex defined above.

Reference: http://goo.gl/v1drZl, page 17.

Warning

This regex will fail on paths involving subexperiments.

directory: str = sentinel.Mandatory
activity_id: str = ''
institution_id: str = ''
source_id: str = ''
experiment_id: str = ''
variant_label: src.cmip6.CMIP6_VariantLabel = ''
table_id: src.cmip6.CMIP6_MIPTable = ''
grid_label: src.cmip6.CMIP6_GridLabel = ''
version_date: src.util.datelabel.Date = None
_is_regex_dataclass = True
_pattern = {}
classmethod from_string(str_, *args)
class src.cmip6.CMIP6_DRSFilename(first_arg=None, *args, **kwargs)[source]

Bases: src.cmip6.CMIP6_VariantLabel, src.cmip6.CMIP6_MIPTable, src.cmip6.CMIP6_GridLabel

Dataclass which represents and parses the DRS filename, using regex defined above.

Reference: http://goo.gl/v1drZl, page 14-15.

filename: str = sentinel.Mandatory
variable_id: str = ''
table_id: src.cmip6.CMIP6_MIPTable = ''
source_id: str = ''
experiment_id: str = ''
variant_label: src.cmip6.CMIP6_VariantLabel = ''
grid_label: src.cmip6.CMIP6_GridLabel = ''
start_date: src.util.datelabel.Date = None
end_date: src.util.datelabel.Date = None
date_range: util.DateRange
_is_regex_dataclass = True
_pattern = <src.util.dataclass.ChainedRegexPattern object>
classmethod from_string(str_, *args)
class src.cmip6.CMIP6_DRSPath(first_arg=None, *args, **kwargs)[source]

Bases: src.cmip6.CMIP6_DRSDirectory, src.cmip6.CMIP6_DRSFilename

Dataclass which represents and parses a full CMIP6 DRS path.

path: str = sentinel.Mandatory
directory: src.cmip6.CMIP6_DRSDirectory = ''
_is_regex_dataclass = True
_pattern = {}
filename: src.cmip6.CMIP6_DRSFilename = ''
classmethod from_string(str_, *args)