hyperparameter_hunter.keys package

Submodules

hyperparameter_hunter.keys.hashing module

hyperparameter_hunter.keys.hashing.make_hash_sha256(obj, **kwargs)

Create an sha256 hash of the input obj

Parameters

obj: Object

Object for which a hash will be created

**kwargs: Dict

Extra kwargs to supply to key_handler.hash_callable()

Returns

Stringified sha256 hash

hyperparameter_hunter.keys.hashing.to_hashable(obj, **kwargs)

Format the input obj to be hashable

Parameters

obj: Object

Object to convert to a hashable format

**kwargs: Dict

Extra kwargs to supply to key_handler.hash_callable()

Returns

obj: object

Hashable object

hyperparameter_hunter.keys.hashing.hash_callable(obj, ignore_line_comments=True, ignore_first_line=False, ignore_module=False, ignore_name=False, ignore_keywords=False, ignore_source_lines=False)

Prepare callable object for hashing by selecting important characterization properties

Parameters

obj: Callable

Callable to convert to a hashable format. Supports: function, class, functools.partial

ignore_line_comments: Boolean, default=True

If True, any line comments will be stripped from the source code of obj, specifically any lines that start with zero or more whitespaces, followed by an octothorpe (#). This does not apply to comments on the same line as code

ignore_first_line: Boolean, default=False

If True, strip the first line from the callable’s source code, specifically its name and signature. If ignore_name=True, this will be treated as True

ignore_module: Boolean, default=False

If True, ignore the name of the module containing the source code (obj.__module__)

ignore_name: Boolean, default=False

If True, ignore obj.__name__. Note the difference from ignore_first_line, which strips the entire callable signature from the source code. ignore_name does not alter the source code. To ensure thorough ignorance, ignore_first_line=True is recommended

ignore_keywords: Boolean, default=False

If True and obj is a functools.partial, ignore obj.keywords

ignore_source_lines: Boolean, default=False

If True, all source code will be ignored by the hashing function. Ignoring all other kwargs, this means that only obj.__module__, and obj.__name__, (and obj.keywords if obj is partial) will be used for hashing

Returns

Tuple

Hashable properties of the callable object input

hyperparameter_hunter.keys.hashing.is_line_comment(string)

Return True if the given string is a line comment, else False

Parameters

string: Str

The str in which to check for a line comment

Returns

Boolean

hyperparameter_hunter.keys.makers module

This module handles the creation of cross_experiment_key s and hyperparameter_key s for hyperparameter_hunter.environment.Environment, and hyperparameter_hunter.experiments.BaseExperiment, respectively. It also handles the treatment of complex-typed inputs and their storage in the ‘KeyAttributeLookup’ subdirectory. The descendants of hyperparameter_hunter.keys.makers.KeyMaker defined herein are each responsible for the generation and saving of their keys, as well as determining whether such a key already exists

Related

hyperparameter_hunter.environment

This module uses hyperparameter_hunter.keys.makers.CrossExperimentKeyMaker to set hyperparameter_hunter.environment.Environment.cross_experiment_key

hyperparameter_hunter.experiments

This module uses hyperparameter_hunter.keys.makers.HyperparameterKeyMaker to set hyperparameter_hunter.experiments.BaseExperiment.hyperparameter_key

class hyperparameter_hunter.keys.makers.KeyMaker(parameters, **kwargs)

Bases: object

Base class to handle making key hashes and checking for their existence. Additionally, this class handles saving entries for complex-typed parameters, along with their hashes to ensure experiments are reproducible

Parameters

parameters: Dict

All the parameters to be included when creating the key hash. Keys should correspond to parameter names, and values should be the values of the corresponding keys

**kwargs: Dict

Additional arguments

Attributes

parameters: Dict

A deep copy of the given parameters input

key: Str, or None

If a key has been generated for parameters, it is saved here. Else, None

exists: Boolean

True if key is not None, and already exists in tested_keys_dir. Else, False

lookup_dir: Str

The directory in which complex-typed parameter entries will be saved

tested_keys_dir: Str, or None

The directory is which key will be saved if it does not already contain key

Methods

add_complex_type_lookup_entry(self, path, …)	Add lookup entry in lookup_dir for a complex-typed parameter, linking the parameter key, its value, and its hashed_value
does_key_exist(self)	Check if key hash exists among saved keys in the contents of tested_keys_dir
handle_complex_types(self)	Locate complex types in parameters, create hashes for them, add lookup entries linking their original values to their hashes, then update their values in parameters to their hashes to facilitate Description saving
make_key(self)	Set key to an sha256 hash for parameters
save_key(self)	Save the key hash and the parameters used to make it to tested_keys_dir
validate_environment(self)	Check that the currently active Environment is suitable

validate_environment(self)

Check that the currently active Environment is suitable

handle_complex_types(self)

Locate complex types in parameters, create hashes for them, add lookup entries linking their original values to their hashes, then update their values in parameters to their hashes to facilitate Description saving

add_complex_type_lookup_entry(self, path, key, value, hashed_value)

Add lookup entry in lookup_dir for a complex-typed parameter, linking the parameter key, its value, and its hashed_value

Parameters

path: Tuple

The path of keys that leads to key

key: Str

The parameter name

value: *

The value of the parameter key

hashed_value: Str

The hash produced for value

make_key(self)

Set key to an sha256 hash for parameters

abstract property key_type

Str in [“hyperparameter”, “cross_experiment”], denoting the key type being processed

abstract does_key_exist(self) → bool

Check if key hash exists among saved keys in the contents of tested_keys_dir

abstract save_key(self)

Save the key hash and the parameters used to make it to tested_keys_dir

class hyperparameter_hunter.keys.makers.CrossExperimentKeyMaker(parameters, **kwargs)

Bases: hyperparameter_hunter.keys.makers.KeyMaker

A KeyMaker class dedicated to creating cross-experiment keys, which determine when experiments were executed under sufficiently similar conditions to permit proper comparison. Two separate instances of environment.Environment should produce identical cross_experiment_key s if their arguments are the same (or close enough)

Parameters

parameters: Dict

All the parameters to be included when creating the key hash. Keys should correspond to parameter names, and values should be the values of the corresponding keys

**kwargs: Dict

Additional arguments supplied to keys.makers.KeyMaker.__init__()

Methods

add_complex_type_lookup_entry(self, path, …)	Add lookup entry in lookup_dir for a complex-typed parameter, linking the parameter key, its value, and its hashed_value
does_key_exist(self)	Check if a file corresponding to this cross_experiment_key already exists
handle_complex_types(self)	Locate complex types in parameters, create hashes for them, add lookup entries linking their original values to their hashes, then update their values in parameters to their hashes to facilitate Description saving
make_key(self)	Set key to an sha256 hash for parameters
save_key(self)	Create a new file for this cross_experiment_key if exists is False
validate_environment(self)	Check that the currently active Environment is suitable

key_type = 'cross_experiment'

does_key_exist(self)

Check if a file corresponding to this cross_experiment_key already exists

Returns

Boolean

save_key(self)

Create a new file for this cross_experiment_key if exists is False

class hyperparameter_hunter.keys.makers.HyperparameterKeyMaker(parameters, cross_experiment_key, **kwargs)

Bases: hyperparameter_hunter.keys.makers.KeyMaker

A KeyMaker class dedicated to creating hyperparameter keys, which determine when experiments were executed using identical hyperparameters. Two separate instances of experiments.CVExperiment should produce identical hyperparameter_key s if their hyperparameters are the same (or close enough)

Parameters

parameters: Dict

All the parameters to be included when creating the key hash. Keys should correspond to parameter names, and values should be the values of the corresponding keys

cross_experiment_key: Str

The key produced by the active Environment via keys.makers.CrossExperimentKeyMaker, used for determining when a hyperparameter key has already been tested under the same cross-experiment parameters

**kwargs: Dict

Additional arguments supplied to keys.makers.KeyMaker.__init__()

Methods

add_complex_type_lookup_entry(self, path, …)	Add lookup entry in lookup_dir for a complex-typed parameter, linking the parameter key, its value, and its hashed_value
does_key_exist(self)	Check that 1) there is a file for cross_experiment_key.key, 2) the aforementioned file contains the key key, and 3) the value at key is a non-empty list
handle_complex_types(self)	Locate complex types in parameters, create hashes for them, add lookup entries linking their original values to their hashes, then update their values in parameters to their hashes to facilitate Description saving
make_key(self)	Set key to an sha256 hash for parameters
save_key(self)	Create an entry in the dict contained in the file at cross_experiment_key.key, whose key is key, and whose value is an empty list if exists is False
validate_environment(self)	Check that the currently active Environment is suitable

key_type = 'hyperparameter'

does_key_exist(self)

Check that 1) there is a file for cross_experiment_key.key, 2) the aforementioned file contains the key key, and 3) the value at key is a non-empty list

Returns

Boolean

save_key(self)

Create an entry in the dict contained in the file at cross_experiment_key.key, whose key is key, and whose value is an empty list if exists is False

Module contents