adorn.params module

Params: a glorified dictionary

class adorn.params.Params(params, history='')

Bases: collections.abc.MutableMapping

Represents a parameter dictionary with a history.

Contains other functionality around parameter passing and validation for Adorn. There are currently two benefits of a Params object over a plain dictionary for parameter passing:

1. We handle a few kinds of parameter validation, including making sure that parameters representing discrete choices actually have acceptable values, and making sure no extra parameters are passed.

2. We log all parameter reads, including default values. This gives a more complete specification of the actual parameters used than is given in a JSON file, because those may not specify what default values were used, whereas this will log them.

Parameters

• params (Dict[str, Any]) –

• history (str) –

Return type

None

DEFAULT = <object object>

as_dict(quiet=False, infer_type_and_cast=False)

Sometimes we need to just represent the parameters as a dict.

For instance when we pass them to PyTorch code.

Parameters

• quiet (bool) – Whether to log the parameters before returning them as a dict.

• infer_type_and_cast (bool) – If True, we infer types and cast (e.g. things that look like floats to floats).

Returns

dict containing the content of Params

Return type

Dict[str, Any]

as_flat_dict()

Returns the parameters of a flat dictionary from keys to values.

Nested structure is collapsed with periods.

Return type

Dict[str, Any]

as_ordered_dict()

Returns Ordered Dict of Params from list of partial order preferences.

Return type

collections.OrderedDict

assert_empty(class_name)

Raises a ConfigurationError if self.params is not empty.

We take class_name as an argument so that the error message gives some idea of where an error happened, if there was one. class_name should be the name of the calling class, the one that got extra parameters (if there are any).

Parameters

class_name (str) –

Return type

None

duplicate()

Uses copy.deepcopy() to create a duplicate (but fully distinct)

copy of these Params.

Return type

adorn.params.Params

classmethod from_file(params_file)

Load a Params object from a configuration file.

Parameters

params_file (str) – The path to the configuration file to load.

Returns

content of the file wrapped in a Params

Return type

Params

get(key, default=<object object>)

Performs the functionality associated with dict.get(key).

Also checks for returned dicts and returns a Params object in their place with an updated history.

Parameters

• key (str) –

• default (Any) –

Return type

Any

get_hash()

Returns a hash code representing the current state of this Params object.

We don’t want to implement __hash__ because that has deeper python implications (and this is a mutable object), but this will give you a representation of the current state. We use zlib.adler32 instead of Python’s builtin hash because the random seed for the latter is reset on each new program invocation, as discussed here: https://stackoverflow.com/questions/27954892/deterministic-hashing-in-python-3.

Return type

str

left_merge(rhs)

Inject values from a new config into the current config

Parameters

rhs (Union[Params, Dict[str, Any]]) – configuration, where its values will be injected into the current config

Returns

flattened version of the original config that includes

values from the passed config, rhs

Return type

Dict[str, Any]

pop(key, default=<object object>, keep_as_dict=False)

Performs the functionality associated with dict.pop(key).

Along with checking for returned dictionaries, replacing them with Param objects with an updated history (unless keep_as_dict is True, in which case we leave them as dictionaries). If key is not present in the dictionary, and no default was specified, we raise a ConfigurationError, instead of the typical KeyError.

Parameters

• key (str) –

• default (Any) –

• keep_as_dict (bool) –

Return type

Any

pop_bool(key, default=<object object>)

Performs a pop and coerces to a bool.

Parameters

• key (str) –

• default (Any) –

Return type

Optional[bool]

pop_choice(key, choices, default_to_first_choice=False, allow_class_names=True)

Gets the value of key in the params dictionary,

Ensuring that the value is one of the given choices. Note that this pops the key from params, modifying the dictionary, consistent with how parameters are processed in this codebase.

Parameters

• key (str) – Key to get the value from in the param dictionary

• choices (List[Any]) – A list of valid options for values corresponding to key. For example, if you’re specifying the type of encoder to use for some part of your model, the choices might be the list of encoder classes we know about and can instantiate. If the value we find in the param dictionary is not in choices, we raise a ConfigurationError, because the user specified an invalid value in their parameter file.

• default_to_first_choice (bool) – If this is True, we allow the key to not be present in the parameter dictionary. If the key is not present, we will use the return as the value the first choice in the choices list. If this is False, we raise a ConfigurationError, because specifying the key is required (e.g., you have to specify your model class when running an experiment, but you can feel free to usedefault settings for encoders if you want).

• allow_class_names (bool) – If this is True, then we allow unknown choices that look like fully-qualified class names. This is to allow e.g. specifying a model type as my_library.my_model.MyModel and importing it on the fly. Our check for “looks like” is extremely lenient and consists of checking that the value contains a ‘.’.

Returns

value which was one of the listed choices

Return type

Any

Raises

ConfigurationError – requested a key that DNE

pop_float(key, default=<object object>)

Performs a pop and coerces to a float.

Parameters

• key (str) –

• default (Any) –

Return type

Optional[float]

pop_int(key, default=<object object>)

Performs a pop and coerces to an int.

Parameters

• key (str) –

• default (Any) –

Return type

Optional[int]

to_file(params_file)

Writes params object to disk as a json

Parameters

params_file (str) –

Return type

None

class adorn.params.ParamsEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

Bases: json.encoder.JSONEncoder

Custom Encoder for nested Params

default(obj)

Ensure nested Params are serializable

Parameters

obj – something trying to be serialized to disk

Returns

Encoded version of the input

adorn.params.infer_and_cast(value)

In some cases we’ll be feeding params dicts to functions we don’t own

For example, PyTorch optimizers. In that case we can’t use pop_int or similar to force casts (which means you can’t specify int parameters using environment variables). This function takes something that looks JSON-like and recursively casts things that look like (bool, int, float) to (bool, int, float).

Parameters

value (Any) –

Return type

Any

adorn.params.unflatten(flat_dict)

Given a “flattened” dict with compound keys, e.g.

{"a.b": 0} after unflatten, {"a": {"b": 0}}

Parameters

flat_dict (Dict[str, Any]) –

Return type

Dict[str, Any]