hyperparameter_hunter.space package¶
Submodules¶
hyperparameter_hunter.space.dimensions module¶
Defines Dimension classes used for defining hyperparameter search spaces. Rather than
hyperparameter_hunter.space.space_core.Space
, the subclasses of
hyperparameter_hunter.space.dimensions.Dimension
are the only tools necessary for a user
to define a hyperparameter search space, when used as intended, in conjunction with a concrete
descendant of hyperparameter_hunter.optimization.protocol_core.BaseOptPro
.
Notes¶
Many of the tools defined herein (although substantially modified) are based on those provided by
the excellent [ScikitOptimize](https://github.com/scikitoptimize/scikitoptimize) library. See
hyperparameter_hunter.optimization.backends.skopt
for a copy of SKOpt’s license

class
hyperparameter_hunter.space.dimensions.
Singleton
¶ Bases:
type
Methods
__call__
(cls, \*args, \*\*kwargs)Call self as a function.
mro
()return a type’s method resolution order

class
hyperparameter_hunter.space.dimensions.
RejectedOptional
¶ Bases:
object
Singleton class to symbolize the rejection of an optional Categorical value
This is used as a sentinel, when the value in Categorical.categories is not used, to be inserted into a
FeatureEngineer
. Ifhyperparameter_hunter.feature_engineering.FeatureEngineer.steps
contains an instance of RejectedOptional, it is removed from steps

class
hyperparameter_hunter.space.dimensions.
Dimension
(**kwargs)¶ Bases:
abc.ABC
Abstract base class for hyperparameter search space dimensions
 Attributes
 id: String
A stringified UUID used to link space dimensions to their locations in a model’s overall hyperparameter structure
 transform_: String
Original value passed through the transform kwarg  Because
transform()
exists distribution: rv_generic
See documentation of
_make_distribution()
ordistribution()
 transformer: Transformer
See documentation of
_make_transformer()
ortransformer()
Methods
distance
(self, a, b)Calculate distance between two points in the dimension’s bounds
get_params
(self)Get dict of parameters used to initialize the Dimension, or their defaults
inverse_transform
(self, data_t)Inverse transform samples from the warped space back to the original space
rvs
(self[, n_samples, random_state])Draw random samples.
transform
(self, data)Transform samples from the original space into a warped space

prior
= None¶

rvs
(self, n_samples=1, random_state=None)¶ Draw random samples. Samples are in the original (untransformed) space. They must be transformed before being passed to a model or minimizer via
transform()
 Parameters
 n_samples: Int, default=1
Number of samples to be drawn
 random_state: Int, RandomState, or None, default=None
Set random state to something other than None for reproducible results
 Returns
 List
Randomly drawn samples from the original space

transform
(self, data)¶ Transform samples from the original space into a warped space
 Parameters
 data: List
Samples to transform. Should be of shape (<# samples>,
size
)
 Returns
 List
Samples transformed into a warped space. Will be of shape (<# samples>,
transformed_size
)
Notes
Expected to be used to project samples into a suitable space for numerical optimization

inverse_transform
(self, data_t)¶ Inverse transform samples from the warped space back to the original space
 Parameters
 data_t: List
Samples to inverse transform. Should be of shape (<# samples>,
transformed_size
)
 Returns
 List
Samples transformed back to original space. Will be shape (<# samples>,
size
)

property
distribution
¶ Class used for random sampling of points within the space
 Returns
 rv_generic
_distribution
Notes
“setter” work for this property is performed by
_make_distribution()
. The reason for this unconventional behavior is noted in distribution.setter

property
transformer
¶ Class used to transform and inversetransform samples in the space
 Returns
 Transformer
_transformer
Notes
“setter” work for this property is performed by
_make_transformer()
. The reason for this unconventional behavior is noted in distribution.setter, which behaves similarly

property
size
¶ Size of the original (untransformed) space for the dimension

property
transformed_size
¶ Size of the transformed space for the dimension

abstract property
bounds
¶ Dimension bounds in the original space

abstract property
transformed_bounds
¶ Dimension bounds in the warped space

property
name
¶ A name associated with the dimension
 Returns
 String, tuple, or None
_name

abstract
distance
(self, a, b) → numbers.Number¶ Calculate distance between two points in the dimension’s bounds

abstract
get_params
(self) → dict¶ Get dict of parameters used to initialize the Dimension, or their defaults

class
hyperparameter_hunter.space.dimensions.
NumericalDimension
(low, high, **kwargs)¶ Bases:
hyperparameter_hunter.space.dimensions.Dimension
,abc.ABC
Abstract base class for strictly numerical
Dimension
subclasses Parameters
 low: Number
Lower bound (inclusive)
 high: Number
Upper bound (inclusive)
 **kwargs: Dict
Additional kwargs passed through from the concrete class to
Dimension
 Attributes
bounds
Dimension bounds in the original space
distribution
Class used for random sampling of points within the space
name
A name associated with the dimension
 prior
size
Size of the original (untransformed) space for the dimension
transformed_bounds
Dimension bounds in the warped space
transformed_size
Size of the transformed space for the dimension
transformer
Class used to transform and inversetransform samples in the space
Methods
distance
(self, a, b)Calculate distance between two points in the dimension’s bounds
get_params
(self)Get dict of parameters used to initialize the Dimension, or their defaults
inverse_transform
(self, data_t)Inverse transform samples from the warped space back to the original space
rvs
(self[, n_samples, random_state])Draw random samples.
transform
(self, data)Transform samples from the original space into a warped space

property
bounds
¶ Dimension bounds in the original space

distance
(self, a, b)¶ Calculate distance between two points in the dimension’s bounds
 Returns
 Number
Absolute value of the difference between a and b

class
hyperparameter_hunter.space.dimensions.
Real
(low, high, prior='uniform', transform='identity', name=None)¶ Bases:
hyperparameter_hunter.space.dimensions.NumericalDimension
Search space dimension that can assume any real value in a given range
 Parameters
 low: Float
Lower bound (inclusive)
 high: Float
Upper bound (inclusive)
 prior: {“uniform”, “loguniform”}, default=”uniform”
Distribution to use when sampling random points for this dimension. If “uniform”, points are sampled uniformly between the lower and upper bounds. If “loguniform”, points are sampled uniformly between log10(lower) and log10(upper)
 transform: {“identity”, “normalize”}, default=”identity”
Transformation to apply to the original space. If “identity”, the transformed space is the same as the original space. If “normalize”, the transformed space is scaled between 0 and 1
 name: String, tuple, or None, default=None
A name associated with the dimension
 Attributes
 distribution: rv_generic
See documentation of
_make_distribution()
ordistribution()
 transform_: String
Original value passed through the transform kwarg  Because
transform()
exists transformer: Transformer
See documentation of
_make_transformer()
ortransformer()
Methods
distance
(self, a, b)Calculate distance between two points in the dimension’s bounds
get_params
(self)Get dict of parameters used to initialize the Real, or their defaults
inverse_transform
(self, data_t)Inverse transform samples from the warped space back to the original space
rvs
(self[, n_samples, random_state])Draw random samples.
transform
(self, data)Transform samples from the original space into a warped space

inverse_transform
(self, data_t)¶ Inverse transform samples from the warped space back to the original space
 Parameters
 data_t: List
Samples to inverse transform. Should be of shape (<# samples>,
transformed_size
)
 Returns
 List
Samples transformed back to original space. Will be shape (<# samples>,
size
)

property
transformed_bounds
¶ Dimension bounds in the warped space
 Returns
 low: Float
0.0 if
transform_`="normalize". If :attr:`transform_`="identity" and :attr:`prior`="uniform", then :attr:`low
. Else log10(low) high: Float
1.0 if
transform_`="normalize". If :attr:`transform_`="identity" and :attr:`prior`="uniform", then :attr:`high
. Else log10(high)

get_params
(self) → dict¶ Get dict of parameters used to initialize the Real, or their defaults

class
hyperparameter_hunter.space.dimensions.
Integer
(low, high, transform='identity', name=None)¶ Bases:
hyperparameter_hunter.space.dimensions.NumericalDimension
Search space dimension that can assume any integer value in a given range
 Parameters
 low: Int
Lower bound (inclusive)
 high: Int
Upper bound (inclusive)
 transform: {“identity”, “normalize”}, default=”identity”
Transformation to apply to the original space. If “identity”, the transformed space is the same as the original space. If “normalize”, the transformed space is scaled between 0 and 1
 name: String, tuple, or None, default=None
A name associated with the dimension
 Attributes
 distribution: rv_generic
See documentation of
_make_distribution()
ordistribution()
 transform_: String
Original value passed through the transform kwarg  Because
transform()
exists transformer: Transformer
See documentation of
_make_transformer()
ortransformer()
Methods
distance
(self, a, b)Calculate distance between two points in the dimension’s bounds
get_params
(self)Get dict of parameters used to initialize the Integer, or their defaults
inverse_transform
(self, data_t)Inverse transform samples from the warped space back to the original space
rvs
(self[, n_samples, random_state])Draw random samples.
transform
(self, data)Transform samples from the original space into a warped space

inverse_transform
(self, data_t)¶ Inverse transform samples from the warped space back to the original space
 Parameters
 data_t: List
Samples to inverse transform. Should be of shape (<# samples>,
transformed_size
)
 Returns
 List
Samples transformed back to original space. Will be shape (<# samples>,
size
)

property
transformed_bounds
¶ Dimension bounds in the warped space
 Returns
 low: Int
0 if
transform_`="normalize", else :attr:`low
 high: Int
1 if
transform_`="normalize", else :attr:`high

get_params
(self) → dict¶ Get dict of parameters used to initialize the Integer, or their defaults

class
hyperparameter_hunter.space.dimensions.
Categorical
(categories: list, prior: list = None, transform='onehot', optional=False, name=None)¶ Bases:
hyperparameter_hunter.space.dimensions.Dimension
Search space dimension that can assume any categorical value in a given list
 Parameters
 categories: List
Sequence of possible categories of shape (n_categories,)
 prior: List, or None, default=None
If list, prior probabilities for each category of shape (categories,). By default all categories are equally likely
 transform: {“onehot”, “identity”}, default=”onehot”
Transformation to apply to the original space. If “identity”, the transformed space is the same as the original space. If “onehot”, the transformed space is a onehot encoded representation of the original space
 optional: Boolean, default=False
Intended for use by
FeatureEngineer
when optimizing anEngineerStep
. Specifically, this enables searching through a space in which an EngineerStep either may or may not be used. This is contrary to Categorical’s usual function of creating a space comprising multiple categories. When optional = True, the space created will represent any of the values in categories either being included in the entire FeatureEngineer process, or being skipped entirely. Internally, a value excluded by optional is represented by a sentinel value that signals it should be removed from the containing list, so optional will not work for choosing between a single value and None, for example name: String, tuple, or None, default=None
A name associated with the dimension
 Attributes
 categories: Tuple
Original value passed through the categories kwarg, cast to a tuple. If optional is True, then an instance of
RejectedOptional
will be appended to categories distribution: rv_generic
See documentation of
_make_distribution()
ordistribution()
 optional: Boolean
Original value passed through the optional kwarg
 prior: List, or None
Original value passed through the prior kwarg
 prior_actual: List
Calculated prior value, initially equivalent to
prior
, but then set to a default array if None transform_: String
Original value passed through the transform kwarg  Because
transform()
exists transformer: Transformer
See documentation of
_make_transformer()
ortransformer()
Methods
distance
(self, a, b)Calculate distance between two points in the dimension’s bounds
get_params
(self)Get dict of parameters used to initialize the Categorical, or their defaults
inverse_transform
(self, data_t)Inverse transform samples from the warped space back to the original space
rvs
(self[, n_samples, random_state])Draw random samples.
transform
(self, data)Transform samples from the original space into a warped space

rvs
(self, n_samples=None, random_state=None)¶ Draw random samples. Samples are in the original (untransformed) space. They must be transformed before being passed to a model or minimizer via
transform()
 Parameters
 n_samples: Int (optional)
Number of samples to be drawn. If not given, a single sample will be returned
 random_state: Int, RandomState, or None, default=None
Set random state to something other than None for reproducible results
 Returns
 List
Randomly drawn samples from the original space

property
transformed_size
¶ Size of the transformed space for the dimension
 Returns
 Int
1 if
transform_
== “identity”1 if
transform_
== “onehot” and length ofcategories
is 1 or 2Length of
categories
in all other cases

property
bounds
¶ Dimension bounds in the original space
 Returns
 Tuple
categories

property
transformed_bounds
¶ Dimension bounds in the warped space
 Returns
 Tuple, or list
If
transformed_size
== 1, then a tuple of (0.0, 1.0). Otherwise, returns a list containingtransformed_size
many tuples of (0.0, 1.0)
Notes
transformed_size
== 1 when the length ofcategories
== 2, so if there are two items in categories, (0.0, 1.0) is returned. If there are three items in categories, [(0.0, 1.0), (0.0, 1.0), (0.0, 1.0)] is returned, and so on.Because transformed_bounds uses
transformed_size
, it is affected bytransform_
. Specifically, the returns described above are fortransform_
== “onehot” (default).Examples
>>> Categorical(["a", "b"]).transformed_bounds (0.0, 1.0) >>> Categorical(["a", "b", "c"]).transformed_bounds [(0.0, 1.0), (0.0, 1.0), (0.0, 1.0)] >>> Categorical(["a", "b", "c", "d"]).transformed_bounds [(0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0)]

distance
(self, a, b) → int¶ Calculate distance between two points in the dimension’s bounds
 Parameters
 a
First category
 b
Second category
 Returns
 Int
0 if a == b. Else 1 (because categories have no order)

get_params
(self) → dict¶ Get dict of parameters used to initialize the Categorical, or their defaults
hyperparameter_hunter.space.space_core module¶
Defines utilities intended for internal use only, most notably
hyperparameter_hunter.space.space_core.Space
. These tools are used behind the scenes by
hyperparameter_hunter.optimization.protocol_core.BaseOptPro
to combine instances of
dimensions defined in hyperparameter_hunter.space.dimensions
into a usable hyperparameter
search Space
Related¶
hyperparameter_hunter.space.dimensions
Defines concrete descendants of
hyperparameter_hunter.space.dimensions.Dimension
, which are intended for direct use.hyperparameter_hunter.space.space_core.Space
is used to combine these Dimension instances
Notes¶
Many of the tools defined herein (although substantially modified) are based on those provided by
the excellent [ScikitOptimize](https://github.com/scikitoptimize/scikitoptimize) library. See
hyperparameter_hunter.optimization.backends.skopt
for a copy of SKOpt’s license

hyperparameter_hunter.space.space_core.
check_dimension
(dimension, transform=None)¶ Turn a provided dimension description into a dimension object. Checks that the provided dimension falls into one of the supported types, listed below in the description of dimension
 Parameters
 dimension: Tuple, list, or Dimension
Search space Dimension. May be any of the following: * (lower_bound, upper_bound) tuple (Real or Integer) * (lower_bound, upper_bound, prior) tuple (Real) * List of categories (Categorical) * Dimension instance (Real, Integer or Categorical)
 transform: {“identity”, “normalize”, “onehot”} (optional)
Categorical dimensions support “onehot” or “identity”. See Categorical documentation for more information
Real and Integer dimensions support “identity” or “normalize”. See Real or Integer documentation for more information
 Returns
 dimension: Dimension
Dimension instance created from the provided dimension description. If dimension is already an instance of Dimension, it is returned unchanged

class
hyperparameter_hunter.space.space_core.
Space
(dimensions)¶ Bases:
object
Initialize a search space from given specifications
 Parameters
 dimensions: List
List of search space Dimension instances or representatives. Each search dimension may be any of the following: * (lower_bound, upper_bound) tuple (Real or Integer) * (lower_bound, upper_bound, prior) tuple (Real) * List of categories (Categorical) * Dimension instance (Real, Integer or Categorical)
Notes
The upper and lower bounds are inclusive for Integer dimensions
 Attributes
bounds
The dimension bounds, in the original space
is_categorical
Whether
dimensions
contains exclusively Categorical dimensionsis_real
Whether
dimensions
contains exclusively Real dimensionsn_dims
Dimensionality of the original space
transformed_bounds
The dimension bounds, in the warped space
transformed_n_dims
Dimensionality of the warped space
Methods
distance
(self, point_a, point_b)Compute distance between two points in this space.
get_by_name
(self, name[, use_location, default])Retrieve a single dimension by its name
inverse_transform
(self, data_t)Inverse transform samples from the warped space back to the original space
names
(self[, use_location])Retrieve the names, or locations of all dimensions in the hyperparameter search space
rvs
(self[, n_samples, random_state])Draw random samples.
transform
(self, data)Transform samples from the original space into a warped space

rvs
(self, n_samples=1, random_state=None)¶ Draw random samples. Samples are in the original (untransformed) space. They must be transformed before being passed to a model or minimizer via
transform()
 Parameters
 n_samples: Int, default=1
Number of samples to be drawn from the space
 random_state: Int, RandomState, or None, default=None
Set random state to something other than None for reproducible results
 Returns
 List
Randomly drawn samples from the original space. Will be a list of lists, of shape (n_samples,
n_dims
)

transform
(self, data)¶ Transform samples from the original space into a warped space
 Parameters
 data: List
Samples to transform. Should be of shape (<# samples>,
n_dims
)
 Returns
 data_t: List
Samples transformed into a warped space. Will be of shape (<# samples>,
transformed_n_dims
)
Notes
Expected to be used to project samples into a suitable space for numerical optimization

inverse_transform
(self, data_t)¶ Inverse transform samples from the warped space back to the original space
 Parameters
 data_t: List
Samples to inverse transform. Should be of shape (<# samples>,
transformed_n_dims
)
 Returns
 List
Samples transformed back to the original space. Will be of shape (<# samples>,
n_dims
)

property
n_dims
¶ Dimensionality of the original space
 Returns
 Int
Length of
dimensions

property
transformed_n_dims
¶ Dimensionality of the warped space
 Returns
 Int
Sum of the transformed_size of all dimensions in
dimensions

property
bounds
¶ The dimension bounds, in the original space
 Returns
 List
Collection of the bounds of each dimension in
dimensions

property
transformed_bounds
¶ The dimension bounds, in the warped space
 Returns
 List
Collection of the transformed_bounds of each dimension in
dimensions

property
is_real
¶ Whether
dimensions
contains exclusively Real dimensions Returns
 Boolean
True if all dimensions in
dimensions
are Real. Else, False

property
is_categorical
¶ Whether
dimensions
contains exclusively Categorical dimensions Returns
 Boolean
True if all dimensions in
dimensions
are Categorical. Else, False

names
(self, use_location=True)¶ Retrieve the names, or locations of all dimensions in the hyperparameter search space
 Parameters
 use_location: Boolean, default=True
If True and a dimension has a nonnull attribute called ‘location’, its value will be used instead of ‘name’
 Returns
 names: List
A list of strings or tuples, in which each value is the name or location of the dimension at that index

get_by_name
(self, name, use_location=True, default=<object object at 0x7fe1747f95f0>)¶ Retrieve a single dimension by its name
 Parameters
 name: Tuple, or str
Name of the dimension in
dimensions
to return use_location: Boolean, default=True
If True and a dimension has a nonnull attribute called “location”, its value will be used instead of that dimension’s “name”
 default: Any (optional)
If given and name is not found, default will be returned. Otherwise, KeyError will be raised when name is not found
 Returns
 Dimension
Dimension subclass in
dimensions
, whose “name” attribute is equal to name

distance
(self, point_a, point_b)¶ Compute distance between two points in this space. Both point_a and point_b are expected to be of the same length as
dimensions
, with values corresponding to the Dimension bounds ofdimensions
 Parameters
 point_a: List
First point
 point_b: List
Second point
 Returns
 Number
Distance between point_a and point_b

hyperparameter_hunter.space.space_core.
normalize_dimensions
(dimensions)¶ Create a Space where all dimensions are instructed to be normalized to unit range. Note that this doesn’t really return normalized dimensions. It just returns the given dimensions, with each one’s transform set to the appropriate value, so that when each dimension’s
transform()
is called, the dimensions are actually normalized Parameters
 dimensions: List
List of search space dimensions. Each search dimension can be defined as any of the following: 1) a (lower_bound, upper_bound) tuple (for Real or Integer dimensions). 2) A (lower_bound, upper_bound, “prior”) tuple (for Real dimensions). 3) A list of categories (for Categorical dimensions). 4) An instance of a Dimension object (Real, Integer, or Categorical)
 Returns
hyperparameter_hunter.space.Space
Hyperparameter space class instance, in which dimensions have been instructed to be normalized to unit range upon invocation of the transform method
 Raises
 RuntimeError
If a processed element of dimensions is not one of: Real, Integer, Categorical
Notes
The upper and lower bounds are inclusive for Integer dimensions
Module contents¶
Defines tools for declaring individual Dimension ranges, as well as a collective Space for managing groups of Dimension instances
Notes¶
Many of the tools defined herein (although substantially modified) are based on those provided by
the excellent [ScikitOptimize](https://github.com/scikitoptimize/scikitoptimize) library. See
hyperparameter_hunter.optimization.backends.skopt
for a copy of SKOpt’s license