hyperparameter_hunter.space package

Submodules

hyperparameter_hunter.space.dimensions module

Defines Dimension classes used for defining hyperparameter search spaces. Rather than hyperparameter_hunter.space.space_core.Space, the subclasses of hyperparameter_hunter.space.dimensions.Dimension are the only tools necessary for a user to define a hyperparameter search space, when used as intended, in conjunction with a concrete descendant of hyperparameter_hunter.optimization.protocol_core.BaseOptPro.

Notes

Many of the tools defined herein (although substantially modified) are based on those provided by the excellent [Scikit-Optimize](https://github.com/scikit-optimize/scikit-optimize) library. See hyperparameter_hunter.optimization.backends.skopt for a copy of SKOpt’s license

class hyperparameter_hunter.space.dimensions.Singleton(name, bases, namespace)

Bases: type

Methods

__call__(*args, **kwargs)

Call self as a function.

mro()

return a type’s method resolution order
class hyperparameter_hunter.space.dimensions.RejectedOptional(*args, **kwargs)

Bases: object

Singleton class to symbolize the rejection of an optional Categorical value

This is used as a sentinel, when the value in Categorical.categories is not used, to be inserted into a FeatureEngineer. If hyperparameter_hunter.feature_engineering.FeatureEngineer.steps contains an instance of RejectedOptional, it is removed from steps

class hyperparameter_hunter.space.dimensions.Log10

Bases: skopt.space.transformers.LogN

Methods

fit

inverse_transform

transform
class hyperparameter_hunter.space.dimensions.Dimension(**kwargs)

Bases: abc.ABC

Abstract base class for hyperparameter search space dimensions

Attributes
id: String

A stringified UUID used to link space dimensions to their locations in a model’s overall hyperparameter structure

transform_: String

Original value passed through the transform kwarg - Because transform() exists

distribution: rv_generic

See documentation of _make_distribution() or distribution()

transformer: Transformer

See documentation of _make_transformer() or transformer()

Methods

distance(a, b)

Calculate distance between two points in the dimension’s bounds

get_params()

Get dict of parameters used to initialize the Dimension, or their defaults

inverse_transform(data_t)

Inverse transform samples from the warped space back to the original space

rvs([n_samples, random_state])

Draw random samples.

transform(data)

Transform samples from the original space into a warped space
prior = None
rvs(n_samples=1, random_state=None)

Draw random samples. Samples are in the original (untransformed) space. They must be transformed before being passed to a model or minimizer via transform()

Parameters
n_samples: Int, default=1

Number of samples to be drawn

random_state: Int, RandomState, or None, default=None

Set random state to something other than None for reproducible results

Returns
List

Randomly drawn samples from the original space

transform(data)

Transform samples from the original space into a warped space

Parameters
data: List

Samples to transform. Should be of shape (<# samples>, size)

Returns
List

Samples transformed into a warped space. Will be of shape (<# samples>, transformed_size)

Notes

Expected to be used to project samples into a suitable space for numerical optimization

inverse_transform(data_t)

Inverse transform samples from the warped space back to the original space

Parameters
data_t: List

Samples to inverse transform. Should be of shape (<# samples>, transformed_size)

Returns
List

Samples transformed back to original space. Will be shape (<# samples>, size)

property distribution

Class used for random sampling of points within the space

Returns
rv_generic

_distribution

Notes

“setter” work for this property is performed by _make_distribution(). The reason for this unconventional behavior is noted in distribution.setter

property transformer

Class used to transform and inverse-transform samples in the space

Returns
Transformer

_transformer

Notes

“setter” work for this property is performed by _make_transformer(). The reason for this unconventional behavior is noted in distribution.setter, which behaves similarly

property size

Size of the original (untransformed) space for the dimension

property transformed_size

Size of the transformed space for the dimension

abstract property bounds

Dimension bounds in the original space

abstract property transformed_bounds

Dimension bounds in the warped space

property name

A name associated with the dimension

Returns
String, tuple, or None

_name

abstract distance(a, b) → numbers.Number

Calculate distance between two points in the dimension’s bounds

abstract get_params() → dict

Get dict of parameters used to initialize the Dimension, or their defaults

class hyperparameter_hunter.space.dimensions.NumericalDimension(low, high, **kwargs)

Bases: hyperparameter_hunter.space.dimensions.Dimension, abc.ABC

Abstract base class for strictly numerical Dimension subclasses

Parameters
low: Number

Lower bound (inclusive)

high: Number

Upper bound (inclusive)

**kwargs: Dict

Additional kwargs passed through from the concrete class to Dimension

Attributes
bounds

Dimension bounds in the original space

distribution

Class used for random sampling of points within the space

name

A name associated with the dimension

prior
size

Size of the original (untransformed) space for the dimension

transformed_bounds

Dimension bounds in the warped space

transformed_size

Size of the transformed space for the dimension

transformer

Class used to transform and inverse-transform samples in the space

Methods

distance(a, b)

Calculate distance between two points in the dimension’s bounds

get_params()

Get dict of parameters used to initialize the Dimension, or their defaults

inverse_transform(data_t)

Inverse transform samples from the warped space back to the original space

rvs([n_samples, random_state])

Draw random samples.

transform(data)

Transform samples from the original space into a warped space
property bounds

Dimension bounds in the original space

Returns
Tuple

Tuple of (low, high). For Real dimensions, the values will be floats. For Integer dimensions, the values will be ints

distance(a, b)

Calculate distance between two points in the dimension’s bounds

Returns
Number

Absolute value of the difference between a and b

class hyperparameter_hunter.space.dimensions.Real(low, high, prior='uniform', transform='identity', name=None)

Bases: hyperparameter_hunter.space.dimensions.NumericalDimension

Search space dimension that can assume any real value in a given range

Parameters
low: Float

Lower bound (inclusive)

high: Float

Upper bound (inclusive)

prior: {“uniform”, “log-uniform”}, default=”uniform”

Distribution to use when sampling random points for this dimension. If “uniform”, points are sampled uniformly between the lower and upper bounds. If “log-uniform”, points are sampled uniformly between log10(lower) and log10(upper)

transform: {“identity”, “normalize”}, default=”identity”

Transformation to apply to the original space. If “identity”, the transformed space is the same as the original space. If “normalize”, the transformed space is scaled between 0 and 1

name: String, tuple, or None, default=None

A name associated with the dimension

Attributes
distribution: rv_generic

See documentation of _make_distribution() or distribution()

transform_: String

Original value passed through the transform kwarg - Because transform() exists

transformer: Transformer

See documentation of _make_transformer() or transformer()

Methods

distance(a, b)

Calculate distance between two points in the dimension’s bounds

get_params()

Get dict of parameters used to initialize the Real, or their defaults

inverse_transform(data_t)

Inverse transform samples from the warped space back to the original space

rvs([n_samples, random_state])

Draw random samples.

transform(data)

Transform samples from the original space into a warped space
inverse_transform(data_t)

Inverse transform samples from the warped space back to the original space

Parameters
data_t: List

Samples to inverse transform. Should be of shape (<# samples>, transformed_size)

Returns
List

Samples transformed back to original space. Will be shape (<# samples>, size)

property transformed_bounds

Dimension bounds in the warped space

Returns
low: Float

0.0 if transform_`="normalize". If :attr:`transform_`="identity" and :attr:`prior`="uniform", then :attr:`low. Else log10(low)

high: Float

1.0 if transform_`="normalize". If :attr:`transform_`="identity" and :attr:`prior`="uniform", then :attr:`high. Else log10(high)

get_params() → dict

Get dict of parameters used to initialize the Real, or their defaults

class hyperparameter_hunter.space.dimensions.Integer(low, high, transform='identity', name=None)

Bases: hyperparameter_hunter.space.dimensions.NumericalDimension

Search space dimension that can assume any integer value in a given range

Parameters
low: Int

Lower bound (inclusive)

high: Int

Upper bound (inclusive)

transform: {“identity”, “normalize”}, default=”identity”

Transformation to apply to the original space. If “identity”, the transformed space is the same as the original space. If “normalize”, the transformed space is scaled between 0 and 1

name: String, tuple, or None, default=None

A name associated with the dimension

Attributes
distribution: rv_generic

See documentation of _make_distribution() or distribution()

transform_: String

Original value passed through the transform kwarg - Because transform() exists

transformer: Transformer

See documentation of _make_transformer() or transformer()

Methods

distance(a, b)

Calculate distance between two points in the dimension’s bounds

get_params()

Get dict of parameters used to initialize the Integer, or their defaults

inverse_transform(data_t)

Inverse transform samples from the warped space back to the original space

rvs([n_samples, random_state])

Draw random samples.

transform(data)

Transform samples from the original space into a warped space
inverse_transform(data_t)

Inverse transform samples from the warped space back to the original space

Parameters
data_t: List

Samples to inverse transform. Should be of shape (<# samples>, transformed_size)

Returns
List

Samples transformed back to original space. Will be shape (<# samples>, size)

property transformed_bounds

Dimension bounds in the warped space

Returns
low: Int

0 if transform_`="normalize", else :attr:`low

high: Int

1 if transform_`="normalize", else :attr:`high

get_params() → dict

Get dict of parameters used to initialize the Integer, or their defaults

class hyperparameter_hunter.space.dimensions.Categorical(categories: list, prior: Optional[list] = None, transform='onehot', optional=False, name=None)

Bases: hyperparameter_hunter.space.dimensions.Dimension

Search space dimension that can assume any categorical value in a given list

Parameters
categories: List

Sequence of possible categories of shape (n_categories,)

prior: List, or None, default=None

If list, prior probabilities for each category of shape (categories,). By default all categories are equally likely

transform: {“onehot”, “identity”}, default=”onehot”

Transformation to apply to the original space. If “identity”, the transformed space is the same as the original space. If “onehot”, the transformed space is a one-hot encoded representation of the original space

optional: Boolean, default=False

Intended for use by FeatureEngineer when optimizing an EngineerStep. Specifically, this enables searching through a space in which an EngineerStep either may or may not be used. This is contrary to Categorical’s usual function of creating a space comprising multiple categories. When optional = True, the space created will represent any of the values in categories either being included in the entire FeatureEngineer process, or being skipped entirely. Internally, a value excluded by optional is represented by a sentinel value that signals it should be removed from the containing list, so optional will not work for choosing between a single value and None, for example

name: String, tuple, or None, default=None

A name associated with the dimension

Attributes
categories: Tuple

Original value passed through the categories kwarg, cast to a tuple. If optional is True, then an instance of RejectedOptional will be appended to categories

distribution: rv_generic

See documentation of _make_distribution() or distribution()

optional: Boolean

Original value passed through the optional kwarg

prior: List, or None

Original value passed through the prior kwarg

prior_actual: List

Calculated prior value, initially equivalent to prior, but then set to a default array if None

transform_: String

Original value passed through the transform kwarg - Because transform() exists

transformer: Transformer

See documentation of _make_transformer() or transformer()

Methods

distance(a, b)

Calculate distance between two points in the dimension’s bounds

get_params()

Get dict of parameters used to initialize the Categorical, or their defaults

inverse_transform(data_t)

Inverse transform samples from the warped space back to the original space

rvs([n_samples, random_state])

Draw random samples.

transform(data)

Transform samples from the original space into a warped space
rvs(n_samples=None, random_state=None)

Draw random samples. Samples are in the original (untransformed) space. They must be transformed before being passed to a model or minimizer via transform()

Parameters
n_samples: Int (optional)

Number of samples to be drawn. If not given, a single sample will be returned

random_state: Int, RandomState, or None, default=None

Set random state to something other than None for reproducible results

Returns
List

Randomly drawn samples from the original space

property transformed_size

Size of the transformed space for the dimension

Returns
Int

  • 1 if transform_ == “identity”

  • 1 if transform_ == “onehot” and length of categories is 1 or 2

  • Length of categories in all other cases

property bounds

Dimension bounds in the original space

Returns
Tuple

categories

property transformed_bounds

Dimension bounds in the warped space

Returns
Tuple, or list

If transformed_size == 1, then a tuple of (0.0, 1.0). Otherwise, returns a list containing transformed_size-many tuples of (0.0, 1.0)

Notes

transformed_size == 1 when the length of categories == 2, so if there are two items in categories, (0.0, 1.0) is returned. If there are three items in categories, [(0.0, 1.0), (0.0, 1.0), (0.0, 1.0)] is returned, and so on.

Because transformed_bounds uses transformed_size, it is affected by transform_. Specifically, the returns described above are for transform_ == “onehot” (default).

Examples

>>> Categorical(["a", "b"]).transformed_bounds
(0.0, 1.0)
>>> Categorical(["a", "b", "c"]).transformed_bounds
[(0.0, 1.0), (0.0, 1.0), (0.0, 1.0)]
>>> Categorical(["a", "b", "c", "d"]).transformed_bounds
[(0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0)]
distance(a, b) → int

Calculate distance between two points in the dimension’s bounds

Parameters
a

First category

b

Second category

Returns
Int

0 if a == b. Else 1 (because categories have no order)

get_params() → dict

Get dict of parameters used to initialize the Categorical, or their defaults

hyperparameter_hunter.space.space_core module

Defines utilities intended for internal use only, most notably hyperparameter_hunter.space.space_core.Space. These tools are used behind the scenes by hyperparameter_hunter.optimization.protocol_core.BaseOptPro to combine instances of dimensions defined in hyperparameter_hunter.space.dimensions into a usable hyperparameter search Space

Related

hyperparameter_hunter.space.dimensions

Defines concrete descendants of hyperparameter_hunter.space.dimensions.Dimension, which are intended for direct use. hyperparameter_hunter.space.space_core.Space is used to combine these Dimension instances

Notes

Many of the tools defined herein (although substantially modified) are based on those provided by the excellent [Scikit-Optimize](https://github.com/scikit-optimize/scikit-optimize) library. See hyperparameter_hunter.optimization.backends.skopt for a copy of SKOpt’s license

hyperparameter_hunter.space.space_core.check_dimension(dimension, transform=None)

Turn a provided dimension description into a dimension object. Checks that the provided dimension falls into one of the supported types, listed below in the description of dimension

Parameters
dimension: Tuple, list, or Dimension

Search space Dimension. May be any of the following: * (lower_bound, upper_bound) tuple (Real or Integer) * (lower_bound, upper_bound, prior) tuple (Real) * List of categories (Categorical) * Dimension instance (Real, Integer or Categorical)

transform: {“identity”, “normalize”, “onehot”} (optional)

  • Categorical dimensions support “onehot” or “identity”. See Categorical documentation for more information

  • Real and Integer dimensions support “identity” or “normalize”. See Real or Integer documentation for more information

Returns
dimension: Dimension

Dimension instance created from the provided dimension description. If dimension is already an instance of Dimension, it is returned unchanged

class hyperparameter_hunter.space.space_core.Space(dimensions)

Bases: object

Initialize a search space from given specifications

Parameters
dimensions: List

List of search space Dimension instances or representatives. Each search dimension may be any of the following: * (lower_bound, upper_bound) tuple (Real or Integer) * (lower_bound, upper_bound, prior) tuple (Real) * List of categories (Categorical) * Dimension instance (Real, Integer or Categorical)

Notes

The upper and lower bounds are inclusive for Integer dimensions

Attributes
bounds

The dimension bounds, in the original space

is_categorical

Whether dimensions contains exclusively Categorical dimensions

is_real

Whether dimensions contains exclusively Real dimensions

n_dims

Dimensionality of the original space

transformed_bounds

The dimension bounds, in the warped space

transformed_n_dims

Dimensionality of the warped space

Methods

distance(point_a, point_b)

Compute distance between two points in this space.

get_by_name(name[, use_location, default])

Retrieve a single dimension by its name

inverse_transform(data_t)

Inverse transform samples from the warped space back to the original space

names([use_location])

Retrieve the names, or locations of all dimensions in the hyperparameter search space

rvs([n_samples, random_state])

Draw random samples.

transform(data)

Transform samples from the original space into a warped space
rvs(n_samples=1, random_state=None)

Draw random samples. Samples are in the original (untransformed) space. They must be transformed before being passed to a model or minimizer via transform()

Parameters
n_samples: Int, default=1

Number of samples to be drawn from the space

random_state: Int, RandomState, or None, default=None

Set random state to something other than None for reproducible results

Returns
List

Randomly drawn samples from the original space. Will be a list of lists, of shape (n_samples, n_dims)

transform(data)

Transform samples from the original space into a warped space

Parameters
data: List

Samples to transform. Should be of shape (<# samples>, n_dims)

Returns
data_t: List

Samples transformed into a warped space. Will be of shape (<# samples>, transformed_n_dims)

Notes

Expected to be used to project samples into a suitable space for numerical optimization

inverse_transform(data_t)

Inverse transform samples from the warped space back to the original space

Parameters
data_t: List

Samples to inverse transform. Should be of shape (<# samples>, transformed_n_dims)

Returns
List

Samples transformed back to the original space. Will be of shape (<# samples>, n_dims)

property n_dims

Dimensionality of the original space

Returns
Int

Length of dimensions

property transformed_n_dims

Dimensionality of the warped space

Returns
Int

Sum of the transformed_size of all dimensions in dimensions

property bounds

The dimension bounds, in the original space

Returns
List

Collection of the bounds of each dimension in dimensions

property transformed_bounds

The dimension bounds, in the warped space

Returns
List

Collection of the transformed_bounds of each dimension in dimensions

property is_real

Whether dimensions contains exclusively Real dimensions

Returns
Boolean

True if all dimensions in dimensions are Real. Else, False

property is_categorical

Whether dimensions contains exclusively Categorical dimensions

Returns
Boolean

True if all dimensions in dimensions are Categorical. Else, False

names(use_location=True)

Retrieve the names, or locations of all dimensions in the hyperparameter search space

Parameters
use_location: Boolean, default=True

If True and a dimension has a non-null attribute called ‘location’, its value will be used instead of ‘name’

Returns
names: List

A list of strings or tuples, in which each value is the name or location of the dimension at that index

get_by_name(name, use_location=True, default=<object object>)

Retrieve a single dimension by its name

Parameters
name: Tuple, or str

Name of the dimension in dimensions to return

use_location: Boolean, default=True

If True and a dimension has a non-null attribute called “location”, its value will be used instead of that dimension’s “name”

default: Any (optional)

If given and name is not found, default will be returned. Otherwise, KeyError will be raised when name is not found

Returns
Dimension

Dimension subclass in dimensions, whose “name” attribute is equal to name

distance(point_a, point_b)

Compute distance between two points in this space. Both point_a and point_b are expected to be of the same length as dimensions, with values corresponding to the Dimension bounds of dimensions

Parameters
point_a: List

First point

point_b: List

Second point

Returns
Number

Distance between point_a and point_b

hyperparameter_hunter.space.space_core.normalize_dimensions(dimensions)

Create a Space where all dimensions are instructed to be normalized to unit range. Note that this doesn’t really return normalized dimensions. It just returns the given dimensions, with each one’s transform set to the appropriate value, so that when each dimension’s transform() is called, the dimensions are actually normalized

Parameters
dimensions: List

List of search space dimensions. Each search dimension can be defined as any of the following: 1) a (lower_bound, upper_bound) tuple (for Real or Integer dimensions). 2) A (lower_bound, upper_bound, “prior”) tuple (for Real dimensions). 3) A list of categories (for Categorical dimensions). 4) An instance of a Dimension object (Real, Integer, or Categorical)

Returns
hyperparameter_hunter.space.Space

Hyperparameter space class instance, in which dimensions have been instructed to be normalized to unit range upon invocation of the transform method

Raises
RuntimeError

If a processed element of dimensions is not one of: Real, Integer, Categorical

Notes

The upper and lower bounds are inclusive for Integer dimensions

Module contents

Defines tools for declaring individual Dimension ranges, as well as a collective Space for managing groups of Dimension instances

Notes

Many of the tools defined herein (although substantially modified) are based on those provided by the excellent [Scikit-Optimize](https://github.com/scikit-optimize/scikit-optimize) library. See hyperparameter_hunter.optimization.backends.skopt for a copy of SKOpt’s license