hyperparameter_hunter.optimization package

Submodules

hyperparameter_hunter.optimization.protocol_core module

This module defines the base Optimization Protocol classes. The classes defined herein are not intended for direct use, but are rather parent classes to those defined in hyperparameter_hunter.optimization.backends.skopt.protocols

Module contents

class hyperparameter_hunter.optimization.BayesianOptPro(target_metric=None, iterations=1, verbose=1, read_experiments=True, reporter_parameters=None, warn_on_re_ask=False, base_estimator='GP', n_initial_points=10, acquisition_function='gp_hedge', acquisition_optimizer='auto', random_state=32, acquisition_function_kwargs=None, acquisition_optimizer_kwargs=None, n_random_starts='DEPRECATED', callbacks=None, base_estimator_kwargs=None)

Bases: hyperparameter_hunter.optimization.protocol_core.SKOptPro

Base class for SKOpt-based Optimization Protocols

There are two important methods for all SKOptPro descendants that should be invoked after initialization:

  1. forge_experiment()

  2. go()

Parameters
target_metric: Tuple, default=(“oof”, <:attr:`environment.Environment.metrics`[0]>)

Rarely necessary to explicitly provide this, as the default is usually sufficient. Path denoting the metric to be used to compare Experiment performance. The first value should be one of [“oof”, “holdout”, “in_fold”]. The second value should be the name of a metric being recorded according to environment.Environment.metrics_params. See the documentation for metrics.get_formatted_target_metric() for more info. Any values returned by, or given as the target_metric input to, get_formatted_target_metric() are acceptable values for BaseOptPro.target_metric

iterations: Int, default=1

Number of Experiments to conduct during optimization upon invoking BaseOptPro.go()

verbose: {0, 1, 2}, default=1

Verbosity mode for console logging. 0: Silent. 1: Show only logs from the Optimization Protocol. 2: In addition to logs shown when verbose=1, also show the logs from individual Experiments

read_experiments: Boolean, default=True

If True, all Experiment records that fit in the current space and guidelines, and match algorithm_name, will be read in and used to fit any optimizers

reporter_parameters: Dict, or None, default=None

Additional parameters passed to reporting.OptimizationReporter.__init__(). Note: Unless provided explicitly, the key “do_maximize” will be added by default to reporter_params, with a value inferred from the direction of target_metric in G.Env.metrics. In nearly all cases, the “do_maximize” key should be ignored, as there are very few reasons to explicitly include it

warn_on_re_ask: Boolean, default=False

If True, and the internal optimizer recommends a point that has already been evaluated on invocation of ask, a warning is logged before recommending a random point. Either way, a random point is used instead of already-evaluated recommendations. However, logging the fact that this has taken place can be useful to indicate that the optimizer may be stalling, especially if it repeatedly recommends the same point. In these cases, if the suggested point is not optimal, it can be helpful to switch a different OptPro (especially DummyOptPro), which will suggest points using different criteria

Other Parameters
base_estimator: {SKLearn Regressor, “GP”, “RF”, “ET”, “GBRT”, “DUMMY”}, default=”GP”

If not string, should inherit from sklearn.base.RegressorMixin. In addition, the predict method should have an optional return_std argument, which returns std(Y | x), along with E[Y | x].

If base_estimator is a string in {“GP”, “RF”, “ET”, “GBRT”, “DUMMY”}, a surrogate model corresponding to the relevant X_minimize function is created

n_initial_points: Int, default=10

Number of complete evaluation points necessary before allowing Experiments to be approximated with base_estimator. Any valid Experiment records found will count as initialization points. If enough Experiment records are not found, additional points will be randomly sampled

acquisition_function:{“LCB”, “EI”, “PI”, “gp_hedge”}, default=”gp_hedge”

Function to minimize over the posterior distribution. Can be any of the following:

  • “LCB”: Lower confidence bound

  • “EI”: Negative expected improvement

  • “PI”: Negative probability of improvement

  • “gp_hedge”: Probabilistically choose one of the above three acquisition functions at every iteration

    • The gains g_i are initialized to zero

    • At every iteration,

      • Each acquisition function is optimised independently to propose a candidate point X_i

      • Out of all these candidate points, the next point X_best is chosen by softmax(eta g_i)

      • After fitting the surrogate model with (X_best, y_best), the gains are updated such that g_i -= mu(X_i)

acquisition_optimizer: {“sampling”, “lbfgs”, “auto”}, default=”auto”

Method to minimize the acquisition function. The fit model is updated with the optimal value obtained by optimizing acq_func with acq_optimizer

  • “sampling”: acq_func is optimized by computing acq_func at n_initial_points randomly sampled points.

  • “lbfgs”: acq_func is optimized by

    • Randomly sampling n_restarts_optimizer (from acq_optimizer_kwargs) points

    • “lbfgs” is run for 20 iterations with these initial points to find local minima

    • The optimal of these local minima is used to update the prior

  • “auto”: acq_optimizer is configured on the basis of the base_estimator and the search space. If the space is Categorical or if the provided estimator is based on tree-models, then this is set to “sampling”

random_state: Int, `RandomState` instance, or None, default=None

Set to something other than None for reproducible results

acquisition_function_kwargs: Dict, or None, default=dict(xi=0.01, kappa=1.96)

Additional arguments passed to the acquisition function

acquisition_optimizer_kwargs: Dict, or None, default=dict(n_points=10000, n_restarts_optimizer=5, n_jobs=1)

Additional arguments passed to the acquisition optimizer

n_random_starts: …

Deprecated since version 3.0.0: Use n_initial_points, instead. Will be removed in 3.2.0

callbacks: Callable, list of callables, or None, default=[]

If callable, then callbacks(self.optimizer_result) is called after each update to optimizer. If list, then each callable is called

base_estimator_kwargs: Dict, or None, default={}

Additional arguments passed to base_estimator when it is initialized

Notes

To provide initial input points for evaluation, individual Experiments can be executed prior to instantiating an Optimization Protocol. The results of these Experiments will automatically be detected and cherished by the optimizer.

SKOptPro and its children in optimization rely heavily on the utilities provided by the Scikit-Optimize library, so thank you to the creators and contributors for their excellent work.

Attributes
search_space_size

The number of different hyperparameter permutations possible given the current

source_script

Methods

forge_experiment(self, model_initializer[, …])

Define hyperparameter search scaffold for building Experiments during optimization

go(self[, force_ready])

Execute hyperparameter optimization, building an Experiment for each iteration

source_script = None
class hyperparameter_hunter.optimization.GradientBoostedRegressionTreeOptPro(target_metric=None, iterations=1, verbose=1, read_experiments=True, reporter_parameters=None, warn_on_re_ask=False, base_estimator='GBRT', n_initial_points=10, acquisition_function='EI', acquisition_optimizer='sampling', random_state=32, acquisition_function_kwargs=None, acquisition_optimizer_kwargs=None, n_random_starts='DEPRECATED', callbacks=None, base_estimator_kwargs=None)

Bases: hyperparameter_hunter.optimization.protocol_core.SKOptPro

Base class for SKOpt-based Optimization Protocols

There are two important methods for all SKOptPro descendants that should be invoked after initialization:

  1. forge_experiment()

  2. go()

Parameters
target_metric: Tuple, default=(“oof”, <:attr:`environment.Environment.metrics`[0]>)

Rarely necessary to explicitly provide this, as the default is usually sufficient. Path denoting the metric to be used to compare Experiment performance. The first value should be one of [“oof”, “holdout”, “in_fold”]. The second value should be the name of a metric being recorded according to environment.Environment.metrics_params. See the documentation for metrics.get_formatted_target_metric() for more info. Any values returned by, or given as the target_metric input to, get_formatted_target_metric() are acceptable values for BaseOptPro.target_metric

iterations: Int, default=1

Number of Experiments to conduct during optimization upon invoking BaseOptPro.go()

verbose: {0, 1, 2}, default=1

Verbosity mode for console logging. 0: Silent. 1: Show only logs from the Optimization Protocol. 2: In addition to logs shown when verbose=1, also show the logs from individual Experiments

read_experiments: Boolean, default=True

If True, all Experiment records that fit in the current space and guidelines, and match algorithm_name, will be read in and used to fit any optimizers

reporter_parameters: Dict, or None, default=None

Additional parameters passed to reporting.OptimizationReporter.__init__(). Note: Unless provided explicitly, the key “do_maximize” will be added by default to reporter_params, with a value inferred from the direction of target_metric in G.Env.metrics. In nearly all cases, the “do_maximize” key should be ignored, as there are very few reasons to explicitly include it

warn_on_re_ask: Boolean, default=False

If True, and the internal optimizer recommends a point that has already been evaluated on invocation of ask, a warning is logged before recommending a random point. Either way, a random point is used instead of already-evaluated recommendations. However, logging the fact that this has taken place can be useful to indicate that the optimizer may be stalling, especially if it repeatedly recommends the same point. In these cases, if the suggested point is not optimal, it can be helpful to switch a different OptPro (especially DummyOptPro), which will suggest points using different criteria

Other Parameters
base_estimator: {SKLearn Regressor, “GP”, “RF”, “ET”, “GBRT”, “DUMMY”}, default=”GP”

If not string, should inherit from sklearn.base.RegressorMixin. In addition, the predict method should have an optional return_std argument, which returns std(Y | x), along with E[Y | x].

If base_estimator is a string in {“GP”, “RF”, “ET”, “GBRT”, “DUMMY”}, a surrogate model corresponding to the relevant X_minimize function is created

n_initial_points: Int, default=10

Number of complete evaluation points necessary before allowing Experiments to be approximated with base_estimator. Any valid Experiment records found will count as initialization points. If enough Experiment records are not found, additional points will be randomly sampled

acquisition_function:{“LCB”, “EI”, “PI”, “gp_hedge”}, default=”gp_hedge”

Function to minimize over the posterior distribution. Can be any of the following:

  • “LCB”: Lower confidence bound

  • “EI”: Negative expected improvement

  • “PI”: Negative probability of improvement

  • “gp_hedge”: Probabilistically choose one of the above three acquisition functions at every iteration

    • The gains g_i are initialized to zero

    • At every iteration,

      • Each acquisition function is optimised independently to propose a candidate point X_i

      • Out of all these candidate points, the next point X_best is chosen by softmax(eta g_i)

      • After fitting the surrogate model with (X_best, y_best), the gains are updated such that g_i -= mu(X_i)

acquisition_optimizer: {“sampling”, “lbfgs”, “auto”}, default=”auto”

Method to minimize the acquisition function. The fit model is updated with the optimal value obtained by optimizing acq_func with acq_optimizer

  • “sampling”: acq_func is optimized by computing acq_func at n_initial_points randomly sampled points.

  • “lbfgs”: acq_func is optimized by

    • Randomly sampling n_restarts_optimizer (from acq_optimizer_kwargs) points

    • “lbfgs” is run for 20 iterations with these initial points to find local minima

    • The optimal of these local minima is used to update the prior

  • “auto”: acq_optimizer is configured on the basis of the base_estimator and the search space. If the space is Categorical or if the provided estimator is based on tree-models, then this is set to “sampling”

random_state: Int, `RandomState` instance, or None, default=None

Set to something other than None for reproducible results

acquisition_function_kwargs: Dict, or None, default=dict(xi=0.01, kappa=1.96)

Additional arguments passed to the acquisition function

acquisition_optimizer_kwargs: Dict, or None, default=dict(n_points=10000, n_restarts_optimizer=5, n_jobs=1)

Additional arguments passed to the acquisition optimizer

n_random_starts: …

Deprecated since version 3.0.0: Use n_initial_points, instead. Will be removed in 3.2.0

callbacks: Callable, list of callables, or None, default=[]

If callable, then callbacks(self.optimizer_result) is called after each update to optimizer. If list, then each callable is called

base_estimator_kwargs: Dict, or None, default={}

Additional arguments passed to base_estimator when it is initialized

Notes

To provide initial input points for evaluation, individual Experiments can be executed prior to instantiating an Optimization Protocol. The results of these Experiments will automatically be detected and cherished by the optimizer.

SKOptPro and its children in optimization rely heavily on the utilities provided by the Scikit-Optimize library, so thank you to the creators and contributors for their excellent work.

Attributes
search_space_size

The number of different hyperparameter permutations possible given the current

source_script

Methods

forge_experiment(self, model_initializer[, …])

Define hyperparameter search scaffold for building Experiments during optimization

go(self[, force_ready])

Execute hyperparameter optimization, building an Experiment for each iteration

source_script = None
hyperparameter_hunter.optimization.GBRT

alias of hyperparameter_hunter.optimization.backends.skopt.protocols.GradientBoostedRegressionTreeOptPro

class hyperparameter_hunter.optimization.RandomForestOptPro(target_metric=None, iterations=1, verbose=1, read_experiments=True, reporter_parameters=None, warn_on_re_ask=False, base_estimator='RF', n_initial_points=10, acquisition_function='EI', acquisition_optimizer='sampling', random_state=32, acquisition_function_kwargs=None, acquisition_optimizer_kwargs=None, n_random_starts='DEPRECATED', callbacks=None, base_estimator_kwargs=None)

Bases: hyperparameter_hunter.optimization.protocol_core.SKOptPro

Base class for SKOpt-based Optimization Protocols

There are two important methods for all SKOptPro descendants that should be invoked after initialization:

  1. forge_experiment()

  2. go()

Parameters
target_metric: Tuple, default=(“oof”, <:attr:`environment.Environment.metrics`[0]>)

Rarely necessary to explicitly provide this, as the default is usually sufficient. Path denoting the metric to be used to compare Experiment performance. The first value should be one of [“oof”, “holdout”, “in_fold”]. The second value should be the name of a metric being recorded according to environment.Environment.metrics_params. See the documentation for metrics.get_formatted_target_metric() for more info. Any values returned by, or given as the target_metric input to, get_formatted_target_metric() are acceptable values for BaseOptPro.target_metric

iterations: Int, default=1

Number of Experiments to conduct during optimization upon invoking BaseOptPro.go()

verbose: {0, 1, 2}, default=1

Verbosity mode for console logging. 0: Silent. 1: Show only logs from the Optimization Protocol. 2: In addition to logs shown when verbose=1, also show the logs from individual Experiments

read_experiments: Boolean, default=True

If True, all Experiment records that fit in the current space and guidelines, and match algorithm_name, will be read in and used to fit any optimizers

reporter_parameters: Dict, or None, default=None

Additional parameters passed to reporting.OptimizationReporter.__init__(). Note: Unless provided explicitly, the key “do_maximize” will be added by default to reporter_params, with a value inferred from the direction of target_metric in G.Env.metrics. In nearly all cases, the “do_maximize” key should be ignored, as there are very few reasons to explicitly include it

warn_on_re_ask: Boolean, default=False

If True, and the internal optimizer recommends a point that has already been evaluated on invocation of ask, a warning is logged before recommending a random point. Either way, a random point is used instead of already-evaluated recommendations. However, logging the fact that this has taken place can be useful to indicate that the optimizer may be stalling, especially if it repeatedly recommends the same point. In these cases, if the suggested point is not optimal, it can be helpful to switch a different OptPro (especially DummyOptPro), which will suggest points using different criteria

Other Parameters
base_estimator: {SKLearn Regressor, “GP”, “RF”, “ET”, “GBRT”, “DUMMY”}, default=”GP”

If not string, should inherit from sklearn.base.RegressorMixin. In addition, the predict method should have an optional return_std argument, which returns std(Y | x), along with E[Y | x].

If base_estimator is a string in {“GP”, “RF”, “ET”, “GBRT”, “DUMMY”}, a surrogate model corresponding to the relevant X_minimize function is created

n_initial_points: Int, default=10

Number of complete evaluation points necessary before allowing Experiments to be approximated with base_estimator. Any valid Experiment records found will count as initialization points. If enough Experiment records are not found, additional points will be randomly sampled

acquisition_function:{“LCB”, “EI”, “PI”, “gp_hedge”}, default=”gp_hedge”

Function to minimize over the posterior distribution. Can be any of the following:

  • “LCB”: Lower confidence bound

  • “EI”: Negative expected improvement

  • “PI”: Negative probability of improvement

  • “gp_hedge”: Probabilistically choose one of the above three acquisition functions at every iteration

    • The gains g_i are initialized to zero

    • At every iteration,

      • Each acquisition function is optimised independently to propose a candidate point X_i

      • Out of all these candidate points, the next point X_best is chosen by softmax(eta g_i)

      • After fitting the surrogate model with (X_best, y_best), the gains are updated such that g_i -= mu(X_i)

acquisition_optimizer: {“sampling”, “lbfgs”, “auto”}, default=”auto”

Method to minimize the acquisition function. The fit model is updated with the optimal value obtained by optimizing acq_func with acq_optimizer

  • “sampling”: acq_func is optimized by computing acq_func at n_initial_points randomly sampled points.

  • “lbfgs”: acq_func is optimized by

    • Randomly sampling n_restarts_optimizer (from acq_optimizer_kwargs) points

    • “lbfgs” is run for 20 iterations with these initial points to find local minima

    • The optimal of these local minima is used to update the prior

  • “auto”: acq_optimizer is configured on the basis of the base_estimator and the search space. If the space is Categorical or if the provided estimator is based on tree-models, then this is set to “sampling”

random_state: Int, `RandomState` instance, or None, default=None

Set to something other than None for reproducible results

acquisition_function_kwargs: Dict, or None, default=dict(xi=0.01, kappa=1.96)

Additional arguments passed to the acquisition function

acquisition_optimizer_kwargs: Dict, or None, default=dict(n_points=10000, n_restarts_optimizer=5, n_jobs=1)

Additional arguments passed to the acquisition optimizer

n_random_starts: …

Deprecated since version 3.0.0: Use n_initial_points, instead. Will be removed in 3.2.0

callbacks: Callable, list of callables, or None, default=[]

If callable, then callbacks(self.optimizer_result) is called after each update to optimizer. If list, then each callable is called

base_estimator_kwargs: Dict, or None, default={}

Additional arguments passed to base_estimator when it is initialized

Notes

To provide initial input points for evaluation, individual Experiments can be executed prior to instantiating an Optimization Protocol. The results of these Experiments will automatically be detected and cherished by the optimizer.

SKOptPro and its children in optimization rely heavily on the utilities provided by the Scikit-Optimize library, so thank you to the creators and contributors for their excellent work.

Attributes
search_space_size

The number of different hyperparameter permutations possible given the current

source_script

Methods

forge_experiment(self, model_initializer[, …])

Define hyperparameter search scaffold for building Experiments during optimization

go(self[, force_ready])

Execute hyperparameter optimization, building an Experiment for each iteration

source_script = None
hyperparameter_hunter.optimization.RF

alias of hyperparameter_hunter.optimization.backends.skopt.protocols.RandomForestOptPro

class hyperparameter_hunter.optimization.ExtraTreesOptPro(target_metric=None, iterations=1, verbose=1, read_experiments=True, reporter_parameters=None, warn_on_re_ask=False, base_estimator='ET', n_initial_points=10, acquisition_function='EI', acquisition_optimizer='sampling', random_state=32, acquisition_function_kwargs=None, acquisition_optimizer_kwargs=None, n_random_starts='DEPRECATED', callbacks=None, base_estimator_kwargs=None)

Bases: hyperparameter_hunter.optimization.protocol_core.SKOptPro

Base class for SKOpt-based Optimization Protocols

There are two important methods for all SKOptPro descendants that should be invoked after initialization:

  1. forge_experiment()

  2. go()

Parameters
target_metric: Tuple, default=(“oof”, <:attr:`environment.Environment.metrics`[0]>)

Rarely necessary to explicitly provide this, as the default is usually sufficient. Path denoting the metric to be used to compare Experiment performance. The first value should be one of [“oof”, “holdout”, “in_fold”]. The second value should be the name of a metric being recorded according to environment.Environment.metrics_params. See the documentation for metrics.get_formatted_target_metric() for more info. Any values returned by, or given as the target_metric input to, get_formatted_target_metric() are acceptable values for BaseOptPro.target_metric

iterations: Int, default=1

Number of Experiments to conduct during optimization upon invoking BaseOptPro.go()

verbose: {0, 1, 2}, default=1

Verbosity mode for console logging. 0: Silent. 1: Show only logs from the Optimization Protocol. 2: In addition to logs shown when verbose=1, also show the logs from individual Experiments

read_experiments: Boolean, default=True

If True, all Experiment records that fit in the current space and guidelines, and match algorithm_name, will be read in and used to fit any optimizers

reporter_parameters: Dict, or None, default=None

Additional parameters passed to reporting.OptimizationReporter.__init__(). Note: Unless provided explicitly, the key “do_maximize” will be added by default to reporter_params, with a value inferred from the direction of target_metric in G.Env.metrics. In nearly all cases, the “do_maximize” key should be ignored, as there are very few reasons to explicitly include it

warn_on_re_ask: Boolean, default=False

If True, and the internal optimizer recommends a point that has already been evaluated on invocation of ask, a warning is logged before recommending a random point. Either way, a random point is used instead of already-evaluated recommendations. However, logging the fact that this has taken place can be useful to indicate that the optimizer may be stalling, especially if it repeatedly recommends the same point. In these cases, if the suggested point is not optimal, it can be helpful to switch a different OptPro (especially DummyOptPro), which will suggest points using different criteria

Other Parameters
base_estimator: {SKLearn Regressor, “GP”, “RF”, “ET”, “GBRT”, “DUMMY”}, default=”GP”

If not string, should inherit from sklearn.base.RegressorMixin. In addition, the predict method should have an optional return_std argument, which returns std(Y | x), along with E[Y | x].

If base_estimator is a string in {“GP”, “RF”, “ET”, “GBRT”, “DUMMY”}, a surrogate model corresponding to the relevant X_minimize function is created

n_initial_points: Int, default=10

Number of complete evaluation points necessary before allowing Experiments to be approximated with base_estimator. Any valid Experiment records found will count as initialization points. If enough Experiment records are not found, additional points will be randomly sampled

acquisition_function:{“LCB”, “EI”, “PI”, “gp_hedge”}, default=”gp_hedge”

Function to minimize over the posterior distribution. Can be any of the following:

  • “LCB”: Lower confidence bound

  • “EI”: Negative expected improvement

  • “PI”: Negative probability of improvement

  • “gp_hedge”: Probabilistically choose one of the above three acquisition functions at every iteration

    • The gains g_i are initialized to zero

    • At every iteration,

      • Each acquisition function is optimised independently to propose a candidate point X_i

      • Out of all these candidate points, the next point X_best is chosen by softmax(eta g_i)

      • After fitting the surrogate model with (X_best, y_best), the gains are updated such that g_i -= mu(X_i)

acquisition_optimizer: {“sampling”, “lbfgs”, “auto”}, default=”auto”

Method to minimize the acquisition function. The fit model is updated with the optimal value obtained by optimizing acq_func with acq_optimizer

  • “sampling”: acq_func is optimized by computing acq_func at n_initial_points randomly sampled points.

  • “lbfgs”: acq_func is optimized by

    • Randomly sampling n_restarts_optimizer (from acq_optimizer_kwargs) points

    • “lbfgs” is run for 20 iterations with these initial points to find local minima

    • The optimal of these local minima is used to update the prior

  • “auto”: acq_optimizer is configured on the basis of the base_estimator and the search space. If the space is Categorical or if the provided estimator is based on tree-models, then this is set to “sampling”

random_state: Int, `RandomState` instance, or None, default=None

Set to something other than None for reproducible results

acquisition_function_kwargs: Dict, or None, default=dict(xi=0.01, kappa=1.96)

Additional arguments passed to the acquisition function

acquisition_optimizer_kwargs: Dict, or None, default=dict(n_points=10000, n_restarts_optimizer=5, n_jobs=1)

Additional arguments passed to the acquisition optimizer

n_random_starts: …

Deprecated since version 3.0.0: Use n_initial_points, instead. Will be removed in 3.2.0

callbacks: Callable, list of callables, or None, default=[]

If callable, then callbacks(self.optimizer_result) is called after each update to optimizer. If list, then each callable is called

base_estimator_kwargs: Dict, or None, default={}

Additional arguments passed to base_estimator when it is initialized

Notes

To provide initial input points for evaluation, individual Experiments can be executed prior to instantiating an Optimization Protocol. The results of these Experiments will automatically be detected and cherished by the optimizer.

SKOptPro and its children in optimization rely heavily on the utilities provided by the Scikit-Optimize library, so thank you to the creators and contributors for their excellent work.

Attributes
search_space_size

The number of different hyperparameter permutations possible given the current

source_script

Methods

forge_experiment(self, model_initializer[, …])

Define hyperparameter search scaffold for building Experiments during optimization

go(self[, force_ready])

Execute hyperparameter optimization, building an Experiment for each iteration

source_script = None
hyperparameter_hunter.optimization.ET

alias of hyperparameter_hunter.optimization.backends.skopt.protocols.ExtraTreesOptPro

class hyperparameter_hunter.optimization.DummyOptPro(target_metric=None, iterations=1, verbose=1, read_experiments=True, reporter_parameters=None, warn_on_re_ask=False, base_estimator='DUMMY', n_initial_points=10, acquisition_function='EI', acquisition_optimizer='sampling', random_state=32, acquisition_function_kwargs=None, acquisition_optimizer_kwargs=None, n_random_starts='DEPRECATED', callbacks=None, base_estimator_kwargs=None)

Bases: hyperparameter_hunter.optimization.protocol_core.SKOptPro

Base class for SKOpt-based Optimization Protocols

There are two important methods for all SKOptPro descendants that should be invoked after initialization:

  1. forge_experiment()

  2. go()

Parameters
target_metric: Tuple, default=(“oof”, <:attr:`environment.Environment.metrics`[0]>)

Rarely necessary to explicitly provide this, as the default is usually sufficient. Path denoting the metric to be used to compare Experiment performance. The first value should be one of [“oof”, “holdout”, “in_fold”]. The second value should be the name of a metric being recorded according to environment.Environment.metrics_params. See the documentation for metrics.get_formatted_target_metric() for more info. Any values returned by, or given as the target_metric input to, get_formatted_target_metric() are acceptable values for BaseOptPro.target_metric

iterations: Int, default=1

Number of Experiments to conduct during optimization upon invoking BaseOptPro.go()

verbose: {0, 1, 2}, default=1

Verbosity mode for console logging. 0: Silent. 1: Show only logs from the Optimization Protocol. 2: In addition to logs shown when verbose=1, also show the logs from individual Experiments

read_experiments: Boolean, default=True

If True, all Experiment records that fit in the current space and guidelines, and match algorithm_name, will be read in and used to fit any optimizers

reporter_parameters: Dict, or None, default=None

Additional parameters passed to reporting.OptimizationReporter.__init__(). Note: Unless provided explicitly, the key “do_maximize” will be added by default to reporter_params, with a value inferred from the direction of target_metric in G.Env.metrics. In nearly all cases, the “do_maximize” key should be ignored, as there are very few reasons to explicitly include it

warn_on_re_ask: Boolean, default=False

If True, and the internal optimizer recommends a point that has already been evaluated on invocation of ask, a warning is logged before recommending a random point. Either way, a random point is used instead of already-evaluated recommendations. However, logging the fact that this has taken place can be useful to indicate that the optimizer may be stalling, especially if it repeatedly recommends the same point. In these cases, if the suggested point is not optimal, it can be helpful to switch a different OptPro (especially DummyOptPro), which will suggest points using different criteria

Other Parameters
base_estimator: {SKLearn Regressor, “GP”, “RF”, “ET”, “GBRT”, “DUMMY”}, default=”GP”

If not string, should inherit from sklearn.base.RegressorMixin. In addition, the predict method should have an optional return_std argument, which returns std(Y | x), along with E[Y | x].

If base_estimator is a string in {“GP”, “RF”, “ET”, “GBRT”, “DUMMY”}, a surrogate model corresponding to the relevant X_minimize function is created

n_initial_points: Int, default=10

Number of complete evaluation points necessary before allowing Experiments to be approximated with base_estimator. Any valid Experiment records found will count as initialization points. If enough Experiment records are not found, additional points will be randomly sampled

acquisition_function:{“LCB”, “EI”, “PI”, “gp_hedge”}, default=”gp_hedge”

Function to minimize over the posterior distribution. Can be any of the following:

  • “LCB”: Lower confidence bound

  • “EI”: Negative expected improvement

  • “PI”: Negative probability of improvement

  • “gp_hedge”: Probabilistically choose one of the above three acquisition functions at every iteration

    • The gains g_i are initialized to zero

    • At every iteration,

      • Each acquisition function is optimised independently to propose a candidate point X_i

      • Out of all these candidate points, the next point X_best is chosen by softmax(eta g_i)

      • After fitting the surrogate model with (X_best, y_best), the gains are updated such that g_i -= mu(X_i)

acquisition_optimizer: {“sampling”, “lbfgs”, “auto”}, default=”auto”

Method to minimize the acquisition function. The fit model is updated with the optimal value obtained by optimizing acq_func with acq_optimizer

  • “sampling”: acq_func is optimized by computing acq_func at n_initial_points randomly sampled points.

  • “lbfgs”: acq_func is optimized by

    • Randomly sampling n_restarts_optimizer (from acq_optimizer_kwargs) points

    • “lbfgs” is run for 20 iterations with these initial points to find local minima

    • The optimal of these local minima is used to update the prior

  • “auto”: acq_optimizer is configured on the basis of the base_estimator and the search space. If the space is Categorical or if the provided estimator is based on tree-models, then this is set to “sampling”

random_state: Int, `RandomState` instance, or None, default=None

Set to something other than None for reproducible results

acquisition_function_kwargs: Dict, or None, default=dict(xi=0.01, kappa=1.96)

Additional arguments passed to the acquisition function

acquisition_optimizer_kwargs: Dict, or None, default=dict(n_points=10000, n_restarts_optimizer=5, n_jobs=1)

Additional arguments passed to the acquisition optimizer

n_random_starts: …

Deprecated since version 3.0.0: Use n_initial_points, instead. Will be removed in 3.2.0

callbacks: Callable, list of callables, or None, default=[]

If callable, then callbacks(self.optimizer_result) is called after each update to optimizer. If list, then each callable is called

base_estimator_kwargs: Dict, or None, default={}

Additional arguments passed to base_estimator when it is initialized

Notes

To provide initial input points for evaluation, individual Experiments can be executed prior to instantiating an Optimization Protocol. The results of these Experiments will automatically be detected and cherished by the optimizer.

SKOptPro and its children in optimization rely heavily on the utilities provided by the Scikit-Optimize library, so thank you to the creators and contributors for their excellent work.

Attributes
search_space_size

The number of different hyperparameter permutations possible given the current

source_script

Methods

forge_experiment(self, model_initializer[, …])

Define hyperparameter search scaffold for building Experiments during optimization

go(self[, force_ready])

Execute hyperparameter optimization, building an Experiment for each iteration

source_script = None
class hyperparameter_hunter.optimization.BayesianOptimization(**kwargs)

Bases: hyperparameter_hunter.optimization.backends.skopt.protocols.BayesianOptPro

Deprecated since version 3.0.0a2: Will be removed in 3.2.0. Renamed to BayesianOptPro

Attributes
search_space_size

The number of different hyperparameter permutations possible given the current

source_script

Methods

forge_experiment(self, model_initializer[, …])

Define hyperparameter search scaffold for building Experiments during optimization

get_ready(self)

Prepare for optimization by finalizing hyperparameter space and identifying similar Experiments.

go(self[, force_ready])

Execute hyperparameter optimization, building an Experiment for each iteration

set_dimensions(self)

Locate given hyperparameters that are space choice declarations and add them to dimensions

set_experiment_guidelines(self, \*args, …)

Deprecated since version 3.0.0a2.

source_script = None
class hyperparameter_hunter.optimization.GradientBoostedRegressionTreeOptimization(**kwargs)

Bases: hyperparameter_hunter.optimization.backends.skopt.protocols.GradientBoostedRegressionTreeOptPro

Deprecated since version 3.0.0a2: Will be removed in 3.2.0. Renamed to GradientBoostedRegressionTreeOptPro

Attributes
search_space_size

The number of different hyperparameter permutations possible given the current

source_script

Methods

forge_experiment(self, model_initializer[, …])

Define hyperparameter search scaffold for building Experiments during optimization

get_ready(self)

Prepare for optimization by finalizing hyperparameter space and identifying similar Experiments.

go(self[, force_ready])

Execute hyperparameter optimization, building an Experiment for each iteration

set_dimensions(self)

Locate given hyperparameters that are space choice declarations and add them to dimensions

set_experiment_guidelines(self, \*args, …)

Deprecated since version 3.0.0a2.

source_script = None
class hyperparameter_hunter.optimization.RandomForestOptimization(**kwargs)

Bases: hyperparameter_hunter.optimization.backends.skopt.protocols.RandomForestOptPro

Deprecated since version 3.0.0a2: Will be removed in 3.2.0. Renamed to RandomForestOptPro

Attributes
search_space_size

The number of different hyperparameter permutations possible given the current

source_script

Methods

forge_experiment(self, model_initializer[, …])

Define hyperparameter search scaffold for building Experiments during optimization

get_ready(self)

Prepare for optimization by finalizing hyperparameter space and identifying similar Experiments.

go(self[, force_ready])

Execute hyperparameter optimization, building an Experiment for each iteration

set_dimensions(self)

Locate given hyperparameters that are space choice declarations and add them to dimensions

set_experiment_guidelines(self, \*args, …)

Deprecated since version 3.0.0a2.

source_script = None
class hyperparameter_hunter.optimization.ExtraTreesOptimization(**kwargs)

Bases: hyperparameter_hunter.optimization.backends.skopt.protocols.ExtraTreesOptPro

Deprecated since version 3.0.0a2: Will be removed in 3.2.0. Renamed to ExtraTreesOptPro

Attributes
search_space_size

The number of different hyperparameter permutations possible given the current

source_script

Methods

forge_experiment(self, model_initializer[, …])

Define hyperparameter search scaffold for building Experiments during optimization

get_ready(self)

Prepare for optimization by finalizing hyperparameter space and identifying similar Experiments.

go(self[, force_ready])

Execute hyperparameter optimization, building an Experiment for each iteration

set_dimensions(self)

Locate given hyperparameters that are space choice declarations and add them to dimensions

set_experiment_guidelines(self, \*args, …)

Deprecated since version 3.0.0a2.

source_script = None
class hyperparameter_hunter.optimization.DummySearch(**kwargs)

Bases: hyperparameter_hunter.optimization.backends.skopt.protocols.DummyOptPro

Deprecated since version 3.0.0a2: Will be removed in 3.2.0. Renamed to DummyOptPro

Attributes
search_space_size

The number of different hyperparameter permutations possible given the current

source_script

Methods

forge_experiment(self, model_initializer[, …])

Define hyperparameter search scaffold for building Experiments during optimization

get_ready(self)

Prepare for optimization by finalizing hyperparameter space and identifying similar Experiments.

go(self[, force_ready])

Execute hyperparameter optimization, building an Experiment for each iteration

set_dimensions(self)

Locate given hyperparameters that are space choice declarations and add them to dimensions

set_experiment_guidelines(self, \*args, …)

Deprecated since version 3.0.0a2.

source_script = None