Index
Top-level package for evotorch.
algorithms
special
¶
This namespace contains the implementations of various evolutionary algorithms.
cmaes
¶
This namespace contains the CMAES class
CMAES (SearchAlgorithm, SinglePopulationAlgorithmMixin)
¶
This is a GPU-accelerated and vectorized implementation, based on pycma (version r3.2.2) and the below references.
References:
Nikolaus Hansen, Youhei Akimoto, and Petr Baudis.
CMA-ES/pycma on Github. Zenodo, DOI:10.5281/zenodo.2559634,
February 2019.
<https://github.com/CMA-ES/pycma>
Nikolaus Hansen, Andreas Ostermeier (2001).
Completely Derandomized Self-Adaptation in Evolution Strategies.
Nikolaus Hansen (2016).
The CMA Evolution Strategy: A Tutorial.
Source code in evotorch/algorithms/cmaes.py
class CMAES(SearchAlgorithm, SinglePopulationAlgorithmMixin):
"""
CMAES: Covariance Matrix Adaptation Evolution Strategy.
This is a GPU-accelerated and vectorized implementation, based on pycma (version r3.2.2)
and the below references.
References:
Nikolaus Hansen, Youhei Akimoto, and Petr Baudis.
CMA-ES/pycma on Github. Zenodo, DOI:10.5281/zenodo.2559634,
February 2019.
<https://github.com/CMA-ES/pycma>
Nikolaus Hansen, Andreas Ostermeier (2001).
Completely Derandomized Self-Adaptation in Evolution Strategies.
Nikolaus Hansen (2016).
The CMA Evolution Strategy: A Tutorial.
"""
def __init__(
self,
problem: Problem,
*,
stdev_init: Real,
popsize: Optional[int] = None,
center_init: Optional[Vector] = None,
c_m: Real = 1.0,
c_sigma: Optional[Real] = None,
c_sigma_ratio: Real = 1.0,
damp_sigma: Optional[Real] = None,
damp_sigma_ratio: Real = 1.0,
c_c: Optional[Real] = None,
c_c_ratio: Real = 1.0,
c_1: Optional[Real] = None,
c_1_ratio: Real = 1.0,
c_mu: Optional[Real] = None,
c_mu_ratio: Real = 1.0,
active: bool = True,
csa_squared: bool = False,
stdev_min: Optional[Real] = None,
stdev_max: Optional[Real] = None,
separable: bool = False,
limit_C_decomposition: bool = True,
obj_index: Optional[int] = None,
):
"""
`__init__(...)`: Initialize the CMAES solver.
Args:
problem (Problem): The problem object which is being worked on.
stdev_init (Real): Initial step-size
popsize: Population size. Can be specified as an int,
or can be left as None in which case the CMA-ES rule of thumb is applied:
popsize = 4 + floor(3 log d) where d is the dimension
center_init: Initial center point of the search distribution.
Can be given as a Solution or as a 1-D array.
If left as None, an initial center point is generated
with the help of the problem object's `generate_values(...)`
method.
c_m (Real): Learning rate for updating the mean
of the search distribution. By default the value is 1.
c_sigma (Optional[Real]): Learning rate for updating the step size. If None,
then the CMA-ES rules of thumb will be applied.
c_sigma_ratio (Real): Multiplier on the learning rate for the step size.
if c_sigma has been left as None, can be used to rescale the default c_sigma value.
damp_sigma (Optional[Real]): Damping factor for updating the step size. If None,
then the CMA-ES rules of thumb will be applied.
damp_sigma_ratio (Real): Multiplier on the damping factor for the step size.
if damp_sigma has been left as None, can be used to rescale the default damp_sigma value.
c_c (Optional[Real]): Learning rate for updating the rank-1 evolution path.
If None, then the CMA-ES rules of thumb will be applied.
c_c_ratio (Real): Multiplier on the learning rate for the rank-1 evolution path.
if c_c has been left as None, can be used to rescale the default c_c value.
c_1 (Optional[Real]): Learning rate for the rank-1 update to the covariance matrix.
If None, then the CMA-ES rules of thumb will be applied.
c_1_ratio (Real): Multiplier on the learning rate for the rank-1 update to the covariance matrix.
if c_1 has been left as None, can be used to rescale the default c_1 value.
c_mu (Optional[Real]): Learning rate for the rank-mu update to the covariance matrix.
If None, then the CMA-ES rules of thumb will be applied.
c_mu_ratio (Real): Multiplier on the learning rate for the rank-mu update to the covariance matrix.
if c_mu has been left as None, can be used to rescale the default c_mu value.
active (bool): Whether to use Active CMA-ES. Defaults to True, consistent with the tutorial paper and pycma.
csa_squared (bool): Whether to use the squared rule ("CSA_squared" in pycma) for the step-size adapation.
This effectively corresponds to taking the natural gradient for the evolution path on the step size,
rather than the default CMA-ES rule of thumb.
stdev_min (Optional[Real]): Minimum allowed standard deviation of the search
distribution. Leaving this as None means that no such
boundary is to be used.
Can be given as None or as a scalar.
stdev_max (Optional[Real]): Maximum allowed standard deviation of the search
distribution. Leaving this as None means that no such
boundary is to be used.
Can be given as None or as a scalar.
separable (bool): Provide this as True if you would like the problem
to be treated as a separable one. Treating a problem
as separable means to adapt only the diagonal parts
of the covariance matrix and to keep the non-diagonal
parts 0. High dimensional problems result in large
covariance matrices on which operating is computationally
expensive. Therefore, for such high dimensional problems,
setting `separable` as True might be useful.
limit_C_decomposition (bool): Whether to limit the frequency of decomposition of the shape matrix C
Setting this to True (default) means that C will not be decomposed every generation
This degrades the quality of the sampling and updates, but provides a guarantee of O(d^2) time complexity.
This option can be used with separable=True (e.g. for experimental reasons) but the performance will only degrade
without time-complexity benefits.
obj_index (Optional[int]): Objective index according to which evaluation
of the solution will be done.
"""
# Initialize the base class
SearchAlgorithm.__init__(self, problem, center=self._get_center, stepsize=self._get_sigma)
# Ensure that the problem is numeric
problem.ensure_numeric()
# CMAES can't handle problem bounds. Ensure that it is unbounded
problem.ensure_unbounded()
# Store the objective index
self._obj_index = problem.normalize_obj_index(obj_index)
# Track d = solution length for reference in initialization of hyperparameters
d = self._problem.solution_length
# === Initialize population ===
if not popsize:
# Default value used in CMA-ES literature 4 + floor(3 log n)
popsize = 4 + int(np.floor(3 * np.log(d)))
self.popsize = int(popsize)
# Half popsize, referred to as mu in CMA-ES literature
self.mu = int(np.floor(popsize / 2))
self._population = problem.generate_batch(popsize=popsize)
# === Initialize search distribution ===
self.separable = separable
# If `center_init` is not given, generate an initial solution
# with the help of the problem object.
# If it is given as a Solution, then clone the solution's values
# as a PyTorch tensor.
# Otherwise, use the given initial solution as the starting
# point in the search space.
if center_init is None:
center_init = self._problem.generate_values(1)
elif isinstance(center_init, Solution):
center_init = center_init.values.clone()
# Store the center
self.m = self._problem.make_tensor(center_init).squeeze()
valid_shaped_m = (self.m.ndim == 1) and (len(self.m) == self._problem.solution_length)
if not valid_shaped_m:
raise ValueError(
f"The initial center point was expected as a vector of length {self._problem.solution_length}."
" However, the provided `center_init` has (or implies) a different shape."
)
# Store the initial step size
self.sigma = self._problem.make_tensor(stdev_init)
if separable:
# Initialize C as the diagonal vector. Note that when separable, the eigendecomposition is not needed
self.C = self._problem.make_ones(d)
# In this case A is simply the square root of elements of C
self.A = self._problem.make_ones(d)
else:
# Initialize C = AA^T all diagonal.
self.C = self._problem.make_I(d)
self.A = self.C.clone()
# === Initialize raw weights ===
# Conditioned on popsize
# w_i = log((lambda + 1) / 2) - log(i) for i = 1 ... lambda
raw_weights = self.problem.make_tensor(np.log((popsize + 1) / 2) - torch.log(torch.arange(popsize) + 1))
# positive valued weights are the first mu
positive_weights = raw_weights[: self.mu]
negative_weights = raw_weights[self.mu :]
# Variance effective selection mass of positive weights
# Not affected by future updates to raw_weights
self.mu_eff = torch.sum(positive_weights).pow(2.0) / torch.sum(positive_weights.pow(2.0))
# === Initialize search parameters ===
# Conditioned on weights
# Store fixed information
self.c_m = c_m
self.active = active
self.csa_squared = csa_squared
self.stdev_min = stdev_min
self.stdev_max = stdev_max
# Learning rate for step-size adaption
if c_sigma is None:
c_sigma = (self.mu_eff + 2.0) / (d + self.mu_eff + 3)
self.c_sigma = c_sigma_ratio * c_sigma
# Damping factor for step-size adapation
if damp_sigma is None:
damp_sigma = 1 + 2 * max(0, torch.sqrt((self.mu_eff - 1) / (d + 1)) - 1) + self.c_sigma
self.damp_sigma = damp_sigma_ratio * damp_sigma
# Learning rate for evolution path for rank-1 update
if c_c is None:
# Branches on separability
if separable:
c_c = (1 + (1 / d) + (self.mu_eff / d)) / (d**0.5 + (1 / d) + 2 * (self.mu_eff / d))
else:
c_c = (4 + self.mu_eff / d) / (d + (4 + 2 * self.mu_eff / d))
self.c_c = c_c_ratio * c_c
# Learning rate for rank-1 update to covariance matrix
if c_1 is None:
# Branches on separability
if separable:
c_1 = 1.0 / (d + 2.0 * np.sqrt(d) + self.mu_eff / d)
else:
c_1 = min(1, popsize / 6) * 2 / ((d + 1.3) ** 2.0 + self.mu_eff)
self.c_1 = c_1_ratio * c_1
# Learning rate for rank-mu update to covariance matrix
if c_mu is None:
# Branches on separability
if separable:
c_mu = (0.25 + self.mu_eff + (1.0 / self.mu_eff) - 2) / (d + 4 * np.sqrt(d) + (self.mu_eff / 2.0))
else:
c_mu = min(
1 - self.c_1, 2 * ((0.25 + self.mu_eff - 2 + (1 / self.mu_eff)) / ((d + 2) ** 2.0 + self.mu_eff))
)
self.c_mu = c_mu_ratio * c_mu
# The 'variance aware' coefficient used for the additive component of the evolution path for sigma
self.variance_discount_sigma = torch.sqrt(self.c_sigma * (2 - self.c_sigma) * self.mu_eff)
# The 'variance aware' coefficient used for the additive component of the evolution path for rank-1 updates
self.variance_discount_c = torch.sqrt(self.c_c * (2 - self.c_c) * self.mu_eff)
# === Finalize weights ===
# Conditioned on search parameters and raw weights
# Positive weights always sum to 1
positive_weights = positive_weights / torch.sum(positive_weights)
if self.active:
# Active CMA-ES: negative weights sum to alpha
# Get the variance effective selection mass of negative weights
mu_eff_neg = torch.sum(negative_weights).pow(2.0) / torch.sum(negative_weights.pow(2.0))
# Alpha is the minimum of the following 3 terms
alpha_mu = 1 + self.c_1 / self.c_mu
alpha_mu_eff = 1 + 2 * mu_eff_neg / (self.mu_eff + 2)
alpha_pos_def = (1 - self.c_mu - self.c_1) / (d * self.c_mu)
alpha = min([alpha_mu, alpha_mu_eff, alpha_pos_def])
# Rescale negative weights
negative_weights = alpha * negative_weights / torch.sum(torch.abs(negative_weights))
else:
# Negative weights are simply zero
negative_weights = torch.zeros_like(negative_weights)
# Concatenate final weights
self.weights = torch.cat([positive_weights, negative_weights], dim=-1)
# === Some final setup ===
# Initialize the evolution paths
self.p_sigma = 0.0
self.p_c = 0.0
# Hansen's approximation to the expectation of ||x|| x ~ N(0, I_d).
# Note that we could use the exact formulation with Gamma functions, but we'll retain this form for consistency
self.unbiased_expectation = np.sqrt(d) * (1 - (1 / (4 * d)) + 1 / (21 * d**2))
self.last_ex = None
# How often to decompose C
if limit_C_decomposition:
self.decompose_C_freq = max(1, int(np.floor(_safe_divide(1, 10 * d * (self.c_1.cpu() + self.c_mu.cpu())))))
else:
self.decompose_C_freq = 1
# Use the SinglePopulationAlgorithmMixin to enable additional status reports regarding the population.
SinglePopulationAlgorithmMixin.__init__(self)
@property
def population(self) -> SolutionBatch:
"""Population generated by the CMA-ES algorithm"""
return self._population
def _get_center(self) -> torch.Tensor:
"""Get the center of search distribution, m"""
return self.m
def _get_sigma(self) -> float:
"""Get the step-size of the search distribution, sigma"""
return float(self.sigma.cpu())
@property
def obj_index(self) -> int:
"""Index of the objective being focused on"""
return self._obj_index
def sample_distribution(self, num_samples: Optional[int] = None) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
"""Sample the population. All 3 representations of solutions are returned for easy calculations of updates.
Note that the computation time of this operation of O(d^2 num_samples) unless separable, in which case O(d num_samples)
Args:
num_samples (Optional[int]): The number of samples to draw. If None, then the population size is used
Returns:
zs (torch.Tensor): A tensor of shape [num_samples, d] of samples from the local coordinate space e.g. z_i ~ N(0, I_d)
ys (torch.Tensor): A tensor of shape [num_samples, d] of samples from the shaped coordinate space e.g. y_i ~ N(0, C)
xs (torch.Tensor): A tensor of shape [num_samples, d] of samples from the search space e.g. x_i ~ N(m, sigma^2 C)
"""
if num_samples is None:
num_samples = self.popsize
# Generate z values
zs = self._problem.make_gaussian(num_solutions=num_samples)
# Construct ys = A zs
if self.separable:
# In the separable case A is diagonal so is represented as a single vector
ys = self.A.unsqueeze(0) * zs
else:
ys = (self.A @ zs.T).T
# Construct xs = m + sigma ys
xs = self.m.unsqueeze(0) + self.sigma * ys
return zs, ys, xs
def get_population_weights(self, xs: torch.Tensor) -> torch.Tensor:
"""Get the assigned weights of the population (e.g. evaluate, rank and return)
Args:
xs (torch.Tensor): The population samples drawn from N(mu, sigma^2 C)
Returns:
assigned_weights (torch.Tensor): A [popsize, ] dimensional tensor of ordered weights
"""
# Computation is O(popsize * F_time) where F_time is the evalutation time per sample
# Fill the population
self._population.set_values(xs)
# Evaluate the current population
self.problem.evaluate(self._population)
# Sort the population
indices = self._population.argsort(obj_index=self.obj_index)
# Invert the sorting of the population to obtain the ranks
# Note that these ranks start at zero, but this is fine as we are just using them for indexing
ranks = torch.zeros_like(indices)
ranks[indices] = torch.arange(self.popsize, dtype=indices.dtype, device=indices.device)
# Get weights corresponding to each rank
assigned_weights = self.weights[ranks]
return assigned_weights
def update_m(self, zs: torch.Tensor, ys: torch.Tensor, assigned_weights: torch.Tensor) -> torch.Tensor:
"""Update the center of the search distribution m
With zs and ys retained from sampling, this operation is O(popsize d), as it involves summing across popsize d-dimensional vectors.
Args:
zs (torch.Tensor): A tensor of shape [popsize, d] of samples from the local coordinate space e.g. z_i ~ N(0, I_d)
ys (torch.Tensor): A tensor of shape [popsize, d] of samples from the shaped coordinate space e.g. y_i ~ N(0, C)
assigned_weights (torch.Tensor): A [popsize, ] dimensional tensor of ordered weights
Returns:
local_m_displacement (torch.Tensor): A tensor of shape [d], corresponding to the local transformation of m,
(1/sigma) (C^-1/2) (m' - m) where m' is the updated m
shaped_m_displacement (torch.Tensor): A tensor of shape [d], corresponding to the shaped transformation of m,
(1/sigma) (m' - m) where m' is the updated m
"""
# Get the top-mu weights
top_mu = torch.topk(assigned_weights, k=self.mu)
top_mu_weights = top_mu.values
top_mu_indices = top_mu.indices
# Compute the weighted recombination in local coordinate space
local_m_displacement = torch.sum(top_mu_weights.unsqueeze(-1) * zs[top_mu_indices], dim=0)
# Compute the weighted recombination in shaped coordinate space
shaped_m_displacement = torch.sum(top_mu_weights.unsqueeze(-1) * ys[top_mu_indices], dim=0)
# Update m
self.m = self.m + self.c_m * self.sigma * shaped_m_displacement
# Return the weighted recombinations
return local_m_displacement, shaped_m_displacement
def update_p_sigma(self, local_m_displacement: torch.Tensor) -> None:
"""Update the evolution path for sigma, p_sigma
This operation is bounded O(d), as is simply the sum of vectors
Args:
local_m_displacement (torch.Tensor): The weighted recombination of local samples zs, corresponding to
(1/sigma) (C^-1/2) (m' - m) where m' is the updated m
"""
self.p_sigma = (1 - self.c_sigma) * self.p_sigma + self.variance_discount_sigma * local_m_displacement
def update_sigma(self) -> None:
"""Update the step size sigma according to its evolution path p_sigma
This operation is bounded O(d), with the most expensive component being the norm of the evolution path, a d-dimensional vector.
"""
d = self._problem.solution_length
# Compute the exponential update
if self.csa_squared:
# Exponential update based on natural gradient maximizing squared norm of p_sigma
exponential_update = (torch.norm(self.p_sigma).pow(2.0) / d - 1) / 2
else:
# Exponential update increasing likelihood p_sigma having expected norm
exponential_update = torch.norm(self.p_sigma) / self.unbiased_expectation - 1
# Rescale exponential update based on learning rate + damping factor
exponential_update = (self.c_sigma / self.damp_sigma) * exponential_update
# Multiplicative update to sigma
self.sigma = self.sigma * torch.exp(exponential_update)
def update_p_c(self, shaped_m_displacement: torch.Tensor, h_sig: torch.Tensor) -> None:
"""Update the evolution path for rank-1 update, p_c
This operation is bounded O(d), as is simply the sum of vectors
Args:
local_m_displacement (torch.Tensor): The weighted recombination of shaped samples ys, corresponding to
(1/sigma) (m' - m) where m' is the updated m
h_sig (torch.Tensor): Whether to stall the update based on the evolution path on sigma, p_sigma, expressed as a torch float
"""
self.p_c = (1 - self.c_c) * self.p_c + h_sig * self.variance_discount_c * shaped_m_displacement
def update_C(self, zs: torch.Tensor, ys: torch.Tensor, assigned_weights: torch.Tensor, h_sig: torch.Tensor) -> None:
"""Update the covariance shape matrix C based on rank-1 and rank-mu updates
This operation is bounded O(d^2 popsize), which is associated with computing the rank-mu update (summing across popsize d*d matrices)
Args:
zs (torch.Tensor): A tensor of shape [popsize, d] of samples from the local coordinate space e.g. z_i ~ N(0, I_d)
ys (torch.Tensor): A tensor of shape [popsize, d] of samples from the shaped coordinate space e.g. y_i ~ N(0, C)
assigned_weights (torch.Tensor): A [popsize, ] dimensional tensor of ordered weights
h_sig (torch.Tensor): Whether to stall the update based on the evolution path on sigma, p_sigma, expressed as a torch float
"""
d = self._problem.solution_length
# If using Active CMA-ES, reweight negative weights
if self.active:
assigned_weights = torch.where(
assigned_weights > 0, assigned_weights, d * assigned_weights / torch.norm(zs, dim=-1).pow(2.0)
)
c1a = self.c_1 * (1 - (1 - h_sig**2) * self.c_c * (2 - self.c_c)) # adjust for variance loss
weighted_pc = (self.c_1 / (c1a + 1e-23)) ** 0.5
if self.separable:
# Rank-1 update
r1_update = c1a * (self.p_c.pow(2.0) - self.C)
# Rank-mu update
rmu_update = self.c_mu * torch.sum(
assigned_weights.unsqueeze(-1) * (ys.pow(2.0) - self.C.unsqueeze(0)), dim=0
)
else:
# Rank-1 update
r1_update = c1a * (torch.outer(weighted_pc * self.p_c, weighted_pc * self.p_c) - self.C)
# Rank-mu update
rmu_update = self.c_mu * (
torch.sum(assigned_weights.unsqueeze(-1).unsqueeze(-1) * (ys.unsqueeze(1) * ys.unsqueeze(2)), dim=0)
- torch.sum(self.weights) * self.C
)
# Update C
self.C = self.C + r1_update + rmu_update
def decompose_C(self) -> None:
"""Perform the decomposition C = AA^T using a cholesky decomposition
Note that traditionally CMA-ES uses the eigendecomposition C = BDDB^-1. In our case,
we keep track of zs, ys and xs when sampling, so we never need C^-1/2.
Therefore, a cholesky decomposition is all that is necessary. This generally requires
O(d^3/3) operations, rather than the more costly O(d^3) operations associated with the eigendecomposition.
"""
if self.separable:
self.A = self.C.pow(0.5)
else:
self.A = torch.linalg.cholesky(self.C)
def _step(self):
"""Perform a step of the CMA-ES solver"""
# === Sampling, evaluation and ranking ===
# Sample the search distribution
zs, ys, xs = self.sample_distribution()
# Get the weights assigned to each solution
assigned_weights = self.get_population_weights(xs)
# === Center adaption ===
local_m_displacement, shaped_m_displacement = self.update_m(zs, ys, assigned_weights)
# === Step size adaption ===
# Update evolution path p_sigma
self.update_p_sigma(local_m_displacement)
# Update sigma
self.update_sigma()
# Compute h_sig, a boolean flag for stalling the update to p_c
h_sig = _h_sig(self.p_sigma, self.c_sigma, self._steps_count)
# === Unscaled covariance adapation ===
# Update evolution path p_c
self.update_p_c(shaped_m_displacement, h_sig)
# Update the covariance shape C
self.update_C(zs, ys, assigned_weights, h_sig)
# === Post-step corrections ===
# Limit element-wise standard deviation of sigma^2 C
if self.stdev_min is not None or self.stdev_max is not None:
self.C = _limit_stdev(self.sigma, self.C, self.stdev_min, self.stdev_max)
# Decompose C
if (self._steps_count + 1) % self.decompose_C_freq == 0:
self.decompose_C()
obj_index: int
property
readonly
¶
Index of the objective being focused on
population: SolutionBatch
property
readonly
¶
Population generated by the CMA-ES algorithm
__init__(self, problem, *, stdev_init, popsize=None, center_init=None, c_m=1.0, c_sigma=None, c_sigma_ratio=1.0, damp_sigma=None, damp_sigma_ratio=1.0, c_c=None, c_c_ratio=1.0, c_1=None, c_1_ratio=1.0, c_mu=None, c_mu_ratio=1.0, active=True, csa_squared=False, stdev_min=None, stdev_max=None, separable=False, limit_C_decomposition=True, obj_index=None)
special
¶
__init__(...): Initialize the CMAES solver.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object which is being worked on. |
required |
stdev_init |
Real |
Initial step-size |
required |
popsize |
Optional[int] |
Population size. Can be specified as an int, or can be left as None in which case the CMA-ES rule of thumb is applied: popsize = 4 + floor(3 log d) where d is the dimension |
None |
center_init |
Union[Iterable[float], torch.Tensor] |
Initial center point of the search distribution.
Can be given as a Solution or as a 1-D array.
If left as None, an initial center point is generated
with the help of the problem object's |
None |
c_m |
Real |
Learning rate for updating the mean of the search distribution. By default the value is 1. |
1.0 |
c_sigma |
Optional[Real] |
Learning rate for updating the step size. If None, then the CMA-ES rules of thumb will be applied. |
None |
c_sigma_ratio |
Real |
Multiplier on the learning rate for the step size. if c_sigma has been left as None, can be used to rescale the default c_sigma value. |
1.0 |
damp_sigma |
Optional[Real] |
Damping factor for updating the step size. If None, then the CMA-ES rules of thumb will be applied. |
None |
damp_sigma_ratio |
Real |
Multiplier on the damping factor for the step size. if damp_sigma has been left as None, can be used to rescale the default damp_sigma value. |
1.0 |
c_c |
Optional[Real] |
Learning rate for updating the rank-1 evolution path. If None, then the CMA-ES rules of thumb will be applied. |
None |
c_c_ratio |
Real |
Multiplier on the learning rate for the rank-1 evolution path. if c_c has been left as None, can be used to rescale the default c_c value. |
1.0 |
c_1 |
Optional[Real] |
Learning rate for the rank-1 update to the covariance matrix. If None, then the CMA-ES rules of thumb will be applied. |
None |
c_1_ratio |
Real |
Multiplier on the learning rate for the rank-1 update to the covariance matrix. if c_1 has been left as None, can be used to rescale the default c_1 value. |
1.0 |
c_mu |
Optional[Real] |
Learning rate for the rank-mu update to the covariance matrix. If None, then the CMA-ES rules of thumb will be applied. |
None |
c_mu_ratio |
Real |
Multiplier on the learning rate for the rank-mu update to the covariance matrix. if c_mu has been left as None, can be used to rescale the default c_mu value. |
1.0 |
active |
bool |
Whether to use Active CMA-ES. Defaults to True, consistent with the tutorial paper and pycma. |
True |
csa_squared |
bool |
Whether to use the squared rule ("CSA_squared" in pycma) for the step-size adapation. This effectively corresponds to taking the natural gradient for the evolution path on the step size, rather than the default CMA-ES rule of thumb. |
False |
stdev_min |
Optional[Real] |
Minimum allowed standard deviation of the search distribution. Leaving this as None means that no such boundary is to be used. Can be given as None or as a scalar. |
None |
stdev_max |
Optional[Real] |
Maximum allowed standard deviation of the search distribution. Leaving this as None means that no such boundary is to be used. Can be given as None or as a scalar. |
None |
separable |
bool |
Provide this as True if you would like the problem
to be treated as a separable one. Treating a problem
as separable means to adapt only the diagonal parts
of the covariance matrix and to keep the non-diagonal
parts 0. High dimensional problems result in large
covariance matrices on which operating is computationally
expensive. Therefore, for such high dimensional problems,
setting |
False |
limit_C_decomposition |
bool |
Whether to limit the frequency of decomposition of the shape matrix C Setting this to True (default) means that C will not be decomposed every generation This degrades the quality of the sampling and updates, but provides a guarantee of O(d^2) time complexity. This option can be used with separable=True (e.g. for experimental reasons) but the performance will only degrade without time-complexity benefits. |
True |
obj_index |
Optional[int] |
Objective index according to which evaluation of the solution will be done. |
None |
Source code in evotorch/algorithms/cmaes.py
def __init__(
self,
problem: Problem,
*,
stdev_init: Real,
popsize: Optional[int] = None,
center_init: Optional[Vector] = None,
c_m: Real = 1.0,
c_sigma: Optional[Real] = None,
c_sigma_ratio: Real = 1.0,
damp_sigma: Optional[Real] = None,
damp_sigma_ratio: Real = 1.0,
c_c: Optional[Real] = None,
c_c_ratio: Real = 1.0,
c_1: Optional[Real] = None,
c_1_ratio: Real = 1.0,
c_mu: Optional[Real] = None,
c_mu_ratio: Real = 1.0,
active: bool = True,
csa_squared: bool = False,
stdev_min: Optional[Real] = None,
stdev_max: Optional[Real] = None,
separable: bool = False,
limit_C_decomposition: bool = True,
obj_index: Optional[int] = None,
):
"""
`__init__(...)`: Initialize the CMAES solver.
Args:
problem (Problem): The problem object which is being worked on.
stdev_init (Real): Initial step-size
popsize: Population size. Can be specified as an int,
or can be left as None in which case the CMA-ES rule of thumb is applied:
popsize = 4 + floor(3 log d) where d is the dimension
center_init: Initial center point of the search distribution.
Can be given as a Solution or as a 1-D array.
If left as None, an initial center point is generated
with the help of the problem object's `generate_values(...)`
method.
c_m (Real): Learning rate for updating the mean
of the search distribution. By default the value is 1.
c_sigma (Optional[Real]): Learning rate for updating the step size. If None,
then the CMA-ES rules of thumb will be applied.
c_sigma_ratio (Real): Multiplier on the learning rate for the step size.
if c_sigma has been left as None, can be used to rescale the default c_sigma value.
damp_sigma (Optional[Real]): Damping factor for updating the step size. If None,
then the CMA-ES rules of thumb will be applied.
damp_sigma_ratio (Real): Multiplier on the damping factor for the step size.
if damp_sigma has been left as None, can be used to rescale the default damp_sigma value.
c_c (Optional[Real]): Learning rate for updating the rank-1 evolution path.
If None, then the CMA-ES rules of thumb will be applied.
c_c_ratio (Real): Multiplier on the learning rate for the rank-1 evolution path.
if c_c has been left as None, can be used to rescale the default c_c value.
c_1 (Optional[Real]): Learning rate for the rank-1 update to the covariance matrix.
If None, then the CMA-ES rules of thumb will be applied.
c_1_ratio (Real): Multiplier on the learning rate for the rank-1 update to the covariance matrix.
if c_1 has been left as None, can be used to rescale the default c_1 value.
c_mu (Optional[Real]): Learning rate for the rank-mu update to the covariance matrix.
If None, then the CMA-ES rules of thumb will be applied.
c_mu_ratio (Real): Multiplier on the learning rate for the rank-mu update to the covariance matrix.
if c_mu has been left as None, can be used to rescale the default c_mu value.
active (bool): Whether to use Active CMA-ES. Defaults to True, consistent with the tutorial paper and pycma.
csa_squared (bool): Whether to use the squared rule ("CSA_squared" in pycma) for the step-size adapation.
This effectively corresponds to taking the natural gradient for the evolution path on the step size,
rather than the default CMA-ES rule of thumb.
stdev_min (Optional[Real]): Minimum allowed standard deviation of the search
distribution. Leaving this as None means that no such
boundary is to be used.
Can be given as None or as a scalar.
stdev_max (Optional[Real]): Maximum allowed standard deviation of the search
distribution. Leaving this as None means that no such
boundary is to be used.
Can be given as None or as a scalar.
separable (bool): Provide this as True if you would like the problem
to be treated as a separable one. Treating a problem
as separable means to adapt only the diagonal parts
of the covariance matrix and to keep the non-diagonal
parts 0. High dimensional problems result in large
covariance matrices on which operating is computationally
expensive. Therefore, for such high dimensional problems,
setting `separable` as True might be useful.
limit_C_decomposition (bool): Whether to limit the frequency of decomposition of the shape matrix C
Setting this to True (default) means that C will not be decomposed every generation
This degrades the quality of the sampling and updates, but provides a guarantee of O(d^2) time complexity.
This option can be used with separable=True (e.g. for experimental reasons) but the performance will only degrade
without time-complexity benefits.
obj_index (Optional[int]): Objective index according to which evaluation
of the solution will be done.
"""
# Initialize the base class
SearchAlgorithm.__init__(self, problem, center=self._get_center, stepsize=self._get_sigma)
# Ensure that the problem is numeric
problem.ensure_numeric()
# CMAES can't handle problem bounds. Ensure that it is unbounded
problem.ensure_unbounded()
# Store the objective index
self._obj_index = problem.normalize_obj_index(obj_index)
# Track d = solution length for reference in initialization of hyperparameters
d = self._problem.solution_length
# === Initialize population ===
if not popsize:
# Default value used in CMA-ES literature 4 + floor(3 log n)
popsize = 4 + int(np.floor(3 * np.log(d)))
self.popsize = int(popsize)
# Half popsize, referred to as mu in CMA-ES literature
self.mu = int(np.floor(popsize / 2))
self._population = problem.generate_batch(popsize=popsize)
# === Initialize search distribution ===
self.separable = separable
# If `center_init` is not given, generate an initial solution
# with the help of the problem object.
# If it is given as a Solution, then clone the solution's values
# as a PyTorch tensor.
# Otherwise, use the given initial solution as the starting
# point in the search space.
if center_init is None:
center_init = self._problem.generate_values(1)
elif isinstance(center_init, Solution):
center_init = center_init.values.clone()
# Store the center
self.m = self._problem.make_tensor(center_init).squeeze()
valid_shaped_m = (self.m.ndim == 1) and (len(self.m) == self._problem.solution_length)
if not valid_shaped_m:
raise ValueError(
f"The initial center point was expected as a vector of length {self._problem.solution_length}."
" However, the provided `center_init` has (or implies) a different shape."
)
# Store the initial step size
self.sigma = self._problem.make_tensor(stdev_init)
if separable:
# Initialize C as the diagonal vector. Note that when separable, the eigendecomposition is not needed
self.C = self._problem.make_ones(d)
# In this case A is simply the square root of elements of C
self.A = self._problem.make_ones(d)
else:
# Initialize C = AA^T all diagonal.
self.C = self._problem.make_I(d)
self.A = self.C.clone()
# === Initialize raw weights ===
# Conditioned on popsize
# w_i = log((lambda + 1) / 2) - log(i) for i = 1 ... lambda
raw_weights = self.problem.make_tensor(np.log((popsize + 1) / 2) - torch.log(torch.arange(popsize) + 1))
# positive valued weights are the first mu
positive_weights = raw_weights[: self.mu]
negative_weights = raw_weights[self.mu :]
# Variance effective selection mass of positive weights
# Not affected by future updates to raw_weights
self.mu_eff = torch.sum(positive_weights).pow(2.0) / torch.sum(positive_weights.pow(2.0))
# === Initialize search parameters ===
# Conditioned on weights
# Store fixed information
self.c_m = c_m
self.active = active
self.csa_squared = csa_squared
self.stdev_min = stdev_min
self.stdev_max = stdev_max
# Learning rate for step-size adaption
if c_sigma is None:
c_sigma = (self.mu_eff + 2.0) / (d + self.mu_eff + 3)
self.c_sigma = c_sigma_ratio * c_sigma
# Damping factor for step-size adapation
if damp_sigma is None:
damp_sigma = 1 + 2 * max(0, torch.sqrt((self.mu_eff - 1) / (d + 1)) - 1) + self.c_sigma
self.damp_sigma = damp_sigma_ratio * damp_sigma
# Learning rate for evolution path for rank-1 update
if c_c is None:
# Branches on separability
if separable:
c_c = (1 + (1 / d) + (self.mu_eff / d)) / (d**0.5 + (1 / d) + 2 * (self.mu_eff / d))
else:
c_c = (4 + self.mu_eff / d) / (d + (4 + 2 * self.mu_eff / d))
self.c_c = c_c_ratio * c_c
# Learning rate for rank-1 update to covariance matrix
if c_1 is None:
# Branches on separability
if separable:
c_1 = 1.0 / (d + 2.0 * np.sqrt(d) + self.mu_eff / d)
else:
c_1 = min(1, popsize / 6) * 2 / ((d + 1.3) ** 2.0 + self.mu_eff)
self.c_1 = c_1_ratio * c_1
# Learning rate for rank-mu update to covariance matrix
if c_mu is None:
# Branches on separability
if separable:
c_mu = (0.25 + self.mu_eff + (1.0 / self.mu_eff) - 2) / (d + 4 * np.sqrt(d) + (self.mu_eff / 2.0))
else:
c_mu = min(
1 - self.c_1, 2 * ((0.25 + self.mu_eff - 2 + (1 / self.mu_eff)) / ((d + 2) ** 2.0 + self.mu_eff))
)
self.c_mu = c_mu_ratio * c_mu
# The 'variance aware' coefficient used for the additive component of the evolution path for sigma
self.variance_discount_sigma = torch.sqrt(self.c_sigma * (2 - self.c_sigma) * self.mu_eff)
# The 'variance aware' coefficient used for the additive component of the evolution path for rank-1 updates
self.variance_discount_c = torch.sqrt(self.c_c * (2 - self.c_c) * self.mu_eff)
# === Finalize weights ===
# Conditioned on search parameters and raw weights
# Positive weights always sum to 1
positive_weights = positive_weights / torch.sum(positive_weights)
if self.active:
# Active CMA-ES: negative weights sum to alpha
# Get the variance effective selection mass of negative weights
mu_eff_neg = torch.sum(negative_weights).pow(2.0) / torch.sum(negative_weights.pow(2.0))
# Alpha is the minimum of the following 3 terms
alpha_mu = 1 + self.c_1 / self.c_mu
alpha_mu_eff = 1 + 2 * mu_eff_neg / (self.mu_eff + 2)
alpha_pos_def = (1 - self.c_mu - self.c_1) / (d * self.c_mu)
alpha = min([alpha_mu, alpha_mu_eff, alpha_pos_def])
# Rescale negative weights
negative_weights = alpha * negative_weights / torch.sum(torch.abs(negative_weights))
else:
# Negative weights are simply zero
negative_weights = torch.zeros_like(negative_weights)
# Concatenate final weights
self.weights = torch.cat([positive_weights, negative_weights], dim=-1)
# === Some final setup ===
# Initialize the evolution paths
self.p_sigma = 0.0
self.p_c = 0.0
# Hansen's approximation to the expectation of ||x|| x ~ N(0, I_d).
# Note that we could use the exact formulation with Gamma functions, but we'll retain this form for consistency
self.unbiased_expectation = np.sqrt(d) * (1 - (1 / (4 * d)) + 1 / (21 * d**2))
self.last_ex = None
# How often to decompose C
if limit_C_decomposition:
self.decompose_C_freq = max(1, int(np.floor(_safe_divide(1, 10 * d * (self.c_1.cpu() + self.c_mu.cpu())))))
else:
self.decompose_C_freq = 1
# Use the SinglePopulationAlgorithmMixin to enable additional status reports regarding the population.
SinglePopulationAlgorithmMixin.__init__(self)
decompose_C(self)
¶
Perform the decomposition C = AA^T using a cholesky decomposition Note that traditionally CMA-ES uses the eigendecomposition C = BDDB^-1. In our case, we keep track of zs, ys and xs when sampling, so we never need C^-½. Therefore, a cholesky decomposition is all that is necessary. This generally requires O(d^3/3) operations, rather than the more costly O(d^3) operations associated with the eigendecomposition.
Source code in evotorch/algorithms/cmaes.py
def decompose_C(self) -> None:
"""Perform the decomposition C = AA^T using a cholesky decomposition
Note that traditionally CMA-ES uses the eigendecomposition C = BDDB^-1. In our case,
we keep track of zs, ys and xs when sampling, so we never need C^-1/2.
Therefore, a cholesky decomposition is all that is necessary. This generally requires
O(d^3/3) operations, rather than the more costly O(d^3) operations associated with the eigendecomposition.
"""
if self.separable:
self.A = self.C.pow(0.5)
else:
self.A = torch.linalg.cholesky(self.C)
get_population_weights(self, xs)
¶
Get the assigned weights of the population (e.g. evaluate, rank and return)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
xs |
torch.Tensor |
The population samples drawn from N(mu, sigma^2 C) |
required |
Returns:
| Type | Description |
|---|---|
assigned_weights (torch.Tensor) |
A [popsize, ] dimensional tensor of ordered weights |
Source code in evotorch/algorithms/cmaes.py
def get_population_weights(self, xs: torch.Tensor) -> torch.Tensor:
"""Get the assigned weights of the population (e.g. evaluate, rank and return)
Args:
xs (torch.Tensor): The population samples drawn from N(mu, sigma^2 C)
Returns:
assigned_weights (torch.Tensor): A [popsize, ] dimensional tensor of ordered weights
"""
# Computation is O(popsize * F_time) where F_time is the evalutation time per sample
# Fill the population
self._population.set_values(xs)
# Evaluate the current population
self.problem.evaluate(self._population)
# Sort the population
indices = self._population.argsort(obj_index=self.obj_index)
# Invert the sorting of the population to obtain the ranks
# Note that these ranks start at zero, but this is fine as we are just using them for indexing
ranks = torch.zeros_like(indices)
ranks[indices] = torch.arange(self.popsize, dtype=indices.dtype, device=indices.device)
# Get weights corresponding to each rank
assigned_weights = self.weights[ranks]
return assigned_weights
sample_distribution(self, num_samples=None)
¶
Sample the population. All 3 representations of solutions are returned for easy calculations of updates. Note that the computation time of this operation of O(d^2 num_samples) unless separable, in which case O(d num_samples)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
num_samples |
Optional[int] |
The number of samples to draw. If None, then the population size is used |
None |
Returns:
| Type | Description |
|---|---|
zs (torch.Tensor) |
A tensor of shape [num_samples, d] of samples from the local coordinate space e.g. z_i ~ N(0, I_d) ys (torch.Tensor): A tensor of shape [num_samples, d] of samples from the shaped coordinate space e.g. y_i ~ N(0, C) xs (torch.Tensor): A tensor of shape [num_samples, d] of samples from the search space e.g. x_i ~ N(m, sigma^2 C) |
Source code in evotorch/algorithms/cmaes.py
def sample_distribution(self, num_samples: Optional[int] = None) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
"""Sample the population. All 3 representations of solutions are returned for easy calculations of updates.
Note that the computation time of this operation of O(d^2 num_samples) unless separable, in which case O(d num_samples)
Args:
num_samples (Optional[int]): The number of samples to draw. If None, then the population size is used
Returns:
zs (torch.Tensor): A tensor of shape [num_samples, d] of samples from the local coordinate space e.g. z_i ~ N(0, I_d)
ys (torch.Tensor): A tensor of shape [num_samples, d] of samples from the shaped coordinate space e.g. y_i ~ N(0, C)
xs (torch.Tensor): A tensor of shape [num_samples, d] of samples from the search space e.g. x_i ~ N(m, sigma^2 C)
"""
if num_samples is None:
num_samples = self.popsize
# Generate z values
zs = self._problem.make_gaussian(num_solutions=num_samples)
# Construct ys = A zs
if self.separable:
# In the separable case A is diagonal so is represented as a single vector
ys = self.A.unsqueeze(0) * zs
else:
ys = (self.A @ zs.T).T
# Construct xs = m + sigma ys
xs = self.m.unsqueeze(0) + self.sigma * ys
return zs, ys, xs
update_C(self, zs, ys, assigned_weights, h_sig)
¶
Update the covariance shape matrix C based on rank-1 and rank-mu updates This operation is bounded O(d^2 popsize), which is associated with computing the rank-mu update (summing across popsize d*d matrices)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
zs |
torch.Tensor |
A tensor of shape [popsize, d] of samples from the local coordinate space e.g. z_i ~ N(0, I_d) |
required |
ys |
torch.Tensor |
A tensor of shape [popsize, d] of samples from the shaped coordinate space e.g. y_i ~ N(0, C) |
required |
assigned_weights |
torch.Tensor |
A [popsize, ] dimensional tensor of ordered weights |
required |
h_sig |
torch.Tensor |
Whether to stall the update based on the evolution path on sigma, p_sigma, expressed as a torch float |
required |
Source code in evotorch/algorithms/cmaes.py
def update_C(self, zs: torch.Tensor, ys: torch.Tensor, assigned_weights: torch.Tensor, h_sig: torch.Tensor) -> None:
"""Update the covariance shape matrix C based on rank-1 and rank-mu updates
This operation is bounded O(d^2 popsize), which is associated with computing the rank-mu update (summing across popsize d*d matrices)
Args:
zs (torch.Tensor): A tensor of shape [popsize, d] of samples from the local coordinate space e.g. z_i ~ N(0, I_d)
ys (torch.Tensor): A tensor of shape [popsize, d] of samples from the shaped coordinate space e.g. y_i ~ N(0, C)
assigned_weights (torch.Tensor): A [popsize, ] dimensional tensor of ordered weights
h_sig (torch.Tensor): Whether to stall the update based on the evolution path on sigma, p_sigma, expressed as a torch float
"""
d = self._problem.solution_length
# If using Active CMA-ES, reweight negative weights
if self.active:
assigned_weights = torch.where(
assigned_weights > 0, assigned_weights, d * assigned_weights / torch.norm(zs, dim=-1).pow(2.0)
)
c1a = self.c_1 * (1 - (1 - h_sig**2) * self.c_c * (2 - self.c_c)) # adjust for variance loss
weighted_pc = (self.c_1 / (c1a + 1e-23)) ** 0.5
if self.separable:
# Rank-1 update
r1_update = c1a * (self.p_c.pow(2.0) - self.C)
# Rank-mu update
rmu_update = self.c_mu * torch.sum(
assigned_weights.unsqueeze(-1) * (ys.pow(2.0) - self.C.unsqueeze(0)), dim=0
)
else:
# Rank-1 update
r1_update = c1a * (torch.outer(weighted_pc * self.p_c, weighted_pc * self.p_c) - self.C)
# Rank-mu update
rmu_update = self.c_mu * (
torch.sum(assigned_weights.unsqueeze(-1).unsqueeze(-1) * (ys.unsqueeze(1) * ys.unsqueeze(2)), dim=0)
- torch.sum(self.weights) * self.C
)
# Update C
self.C = self.C + r1_update + rmu_update
update_m(self, zs, ys, assigned_weights)
¶
Update the center of the search distribution m With zs and ys retained from sampling, this operation is O(popsize d), as it involves summing across popsize d-dimensional vectors.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
zs |
torch.Tensor |
A tensor of shape [popsize, d] of samples from the local coordinate space e.g. z_i ~ N(0, I_d) |
required |
ys |
torch.Tensor |
A tensor of shape [popsize, d] of samples from the shaped coordinate space e.g. y_i ~ N(0, C) |
required |
assigned_weights |
torch.Tensor |
A [popsize, ] dimensional tensor of ordered weights |
required |
Returns:
| Type | Description |
|---|---|
local_m_displacement (torch.Tensor) |
A tensor of shape [d], corresponding to the local transformation of m, (1/sigma) (C^-½) (m' - m) where m' is the updated m shaped_m_displacement (torch.Tensor): A tensor of shape [d], corresponding to the shaped transformation of m, (1/sigma) (m' - m) where m' is the updated m |
Source code in evotorch/algorithms/cmaes.py
def update_m(self, zs: torch.Tensor, ys: torch.Tensor, assigned_weights: torch.Tensor) -> torch.Tensor:
"""Update the center of the search distribution m
With zs and ys retained from sampling, this operation is O(popsize d), as it involves summing across popsize d-dimensional vectors.
Args:
zs (torch.Tensor): A tensor of shape [popsize, d] of samples from the local coordinate space e.g. z_i ~ N(0, I_d)
ys (torch.Tensor): A tensor of shape [popsize, d] of samples from the shaped coordinate space e.g. y_i ~ N(0, C)
assigned_weights (torch.Tensor): A [popsize, ] dimensional tensor of ordered weights
Returns:
local_m_displacement (torch.Tensor): A tensor of shape [d], corresponding to the local transformation of m,
(1/sigma) (C^-1/2) (m' - m) where m' is the updated m
shaped_m_displacement (torch.Tensor): A tensor of shape [d], corresponding to the shaped transformation of m,
(1/sigma) (m' - m) where m' is the updated m
"""
# Get the top-mu weights
top_mu = torch.topk(assigned_weights, k=self.mu)
top_mu_weights = top_mu.values
top_mu_indices = top_mu.indices
# Compute the weighted recombination in local coordinate space
local_m_displacement = torch.sum(top_mu_weights.unsqueeze(-1) * zs[top_mu_indices], dim=0)
# Compute the weighted recombination in shaped coordinate space
shaped_m_displacement = torch.sum(top_mu_weights.unsqueeze(-1) * ys[top_mu_indices], dim=0)
# Update m
self.m = self.m + self.c_m * self.sigma * shaped_m_displacement
# Return the weighted recombinations
return local_m_displacement, shaped_m_displacement
update_p_c(self, shaped_m_displacement, h_sig)
¶
Update the evolution path for rank-1 update, p_c This operation is bounded O(d), as is simply the sum of vectors
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
local_m_displacement |
torch.Tensor |
The weighted recombination of shaped samples ys, corresponding to (1/sigma) (m' - m) where m' is the updated m |
required |
h_sig |
torch.Tensor |
Whether to stall the update based on the evolution path on sigma, p_sigma, expressed as a torch float |
required |
Source code in evotorch/algorithms/cmaes.py
def update_p_c(self, shaped_m_displacement: torch.Tensor, h_sig: torch.Tensor) -> None:
"""Update the evolution path for rank-1 update, p_c
This operation is bounded O(d), as is simply the sum of vectors
Args:
local_m_displacement (torch.Tensor): The weighted recombination of shaped samples ys, corresponding to
(1/sigma) (m' - m) where m' is the updated m
h_sig (torch.Tensor): Whether to stall the update based on the evolution path on sigma, p_sigma, expressed as a torch float
"""
self.p_c = (1 - self.c_c) * self.p_c + h_sig * self.variance_discount_c * shaped_m_displacement
update_p_sigma(self, local_m_displacement)
¶
Update the evolution path for sigma, p_sigma This operation is bounded O(d), as is simply the sum of vectors
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
local_m_displacement |
torch.Tensor |
The weighted recombination of local samples zs, corresponding to (1/sigma) (C^-½) (m' - m) where m' is the updated m |
required |
Source code in evotorch/algorithms/cmaes.py
def update_p_sigma(self, local_m_displacement: torch.Tensor) -> None:
"""Update the evolution path for sigma, p_sigma
This operation is bounded O(d), as is simply the sum of vectors
Args:
local_m_displacement (torch.Tensor): The weighted recombination of local samples zs, corresponding to
(1/sigma) (C^-1/2) (m' - m) where m' is the updated m
"""
self.p_sigma = (1 - self.c_sigma) * self.p_sigma + self.variance_discount_sigma * local_m_displacement
update_sigma(self)
¶
Update the step size sigma according to its evolution path p_sigma This operation is bounded O(d), with the most expensive component being the norm of the evolution path, a d-dimensional vector.
Source code in evotorch/algorithms/cmaes.py
def update_sigma(self) -> None:
"""Update the step size sigma according to its evolution path p_sigma
This operation is bounded O(d), with the most expensive component being the norm of the evolution path, a d-dimensional vector.
"""
d = self._problem.solution_length
# Compute the exponential update
if self.csa_squared:
# Exponential update based on natural gradient maximizing squared norm of p_sigma
exponential_update = (torch.norm(self.p_sigma).pow(2.0) / d - 1) / 2
else:
# Exponential update increasing likelihood p_sigma having expected norm
exponential_update = torch.norm(self.p_sigma) / self.unbiased_expectation - 1
# Rescale exponential update based on learning rate + damping factor
exponential_update = (self.c_sigma / self.damp_sigma) * exponential_update
# Multiplicative update to sigma
self.sigma = self.sigma * torch.exp(exponential_update)
distributed
special
¶
gaussian
¶
CEM (GaussianSearchAlgorithm)
¶
The cross-entropy method (CEM) (Rubinstein, 1999).
This CEM implementation is focused on continuous optimization, and follows the variant explained in Duan et al. (2016).
The adaptive population size mechanism explained in Toklu et al. (2020)
(and previously used in the accompanying source code of the study
Salimans et al. (2017)) is supported, where the population size in an
iteration keeps increasing until a certain numberof interactions with
the simulator of the reinforcement learning environment is made.
See the initialization arguments num_interactions, popsize_max.
References:
Rubinstein, R. (1999). The cross-entropy method for combinatorial
and continuous optimization.
Methodology and computing in applied probability, 1(2), 127-190.
Duan, Y., Chen, X., Houthooft, R., Schulman, J., Abbeel, P. (2016).
Benchmarking deep reinforcement learning for continuous control.
International conference on machine learning. PMLR, 2016.
Salimans, T., Ho, J., Chen, X., Sidor, S. and Sutskever, I. (2017).
Evolution Strategies as a Scalable Alternative to
Reinforcement Learning.
Toklu, N.E., Liskowski, P., Srivastava, R.K. (2020).
ClipUp: A Simple and Powerful Optimizer
for Distribution-based Policy Evolution.
Parallel Problem Solving from Nature (PPSN 2020).
Source code in evotorch/algorithms/distributed/gaussian.py
class CEM(GaussianSearchAlgorithm):
"""
The cross-entropy method (CEM) (Rubinstein, 1999).
This CEM implementation is focused on continuous optimization,
and follows the variant explained in Duan et al. (2016).
The adaptive population size mechanism explained in Toklu et al. (2020)
(and previously used in the accompanying source code of the study
Salimans et al. (2017)) is supported, where the population size in an
iteration keeps increasing until a certain numberof interactions with
the simulator of the reinforcement learning environment is made.
See the initialization arguments `num_interactions`, `popsize_max`.
References:
Rubinstein, R. (1999). The cross-entropy method for combinatorial
and continuous optimization.
Methodology and computing in applied probability, 1(2), 127-190.
Duan, Y., Chen, X., Houthooft, R., Schulman, J., Abbeel, P. (2016).
Benchmarking deep reinforcement learning for continuous control.
International conference on machine learning. PMLR, 2016.
Salimans, T., Ho, J., Chen, X., Sidor, S. and Sutskever, I. (2017).
Evolution Strategies as a Scalable Alternative to
Reinforcement Learning.
Toklu, N.E., Liskowski, P., Srivastava, R.K. (2020).
ClipUp: A Simple and Powerful Optimizer
for Distribution-based Policy Evolution.
Parallel Problem Solving from Nature (PPSN 2020).
"""
DISTRIBUTION_TYPE = SeparableGaussian
DISTRIBUTION_PARAMS = NotImplemented # To be filled by the CEM instance
def __init__(
self,
problem: Problem,
*,
popsize: int,
parenthood_ratio: float,
stdev_init: Optional[RealOrVector] = None,
radius_init: Optional[RealOrVector] = None,
num_interactions: Optional[int] = None,
popsize_max: Optional[int] = None,
center_init: Optional[RealOrVector] = None,
stdev_min: Optional[RealOrVector] = None,
stdev_max: Optional[RealOrVector] = None,
stdev_max_change: Optional[Union[float, RealOrVector]] = None,
obj_index: Optional[int] = None,
distributed: bool = False,
popsize_weighted_grad_avg: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the search algorithm.
Args:
problem: The problem object to work on.
popsize: The population size.
parenthood_ratio: Expected as a float larger than 0 and smaller
than 1. For example, setting this value to 0.1 means that
the top 10% of the population will be declared as the parents,
and those parents will be used for updating the population.
The amount of parents is always computed according to the
specified `popsize`, not according to the adapted population
size, and not according to `popsize_max`.
stdev_init: The initial standard deviation of the search
distribution, expressed as a scalar or as an array.
Determines the initial coverage area of the search
distribution.
If one wishes to configure the coverage area via the
argument `radius_init` instead, then `stdev_init` is expected
as None.
radius_init: The initial radius of the search distribution,
expressed as a scalar.
Determines the initial coverage area of the search
distribution.
Here, "radius" is defined as the norm of the search
distribution.
If one wishes to configure the coverage area via the
argument `stdev_init` instead, then `radius_init` is expected
as None.
num_interactions: When given as an integer n,
it is ensured that a population has interacted with
the GymProblem's environment n times. If this target
has not been reached yet, then the population is declared
too small, and gets extended with more samples,
until n amount of interactions is reached.
When given as None, popsize is the only configuration
affecting the size of a population.
popsize_max: Having `num_interactions` set as an integer
might cause the effective population size jump to
unnecesarily large numbers. To prevent this,
one can set `popsize_max` to specify an upper
bound for the effective population size.
center_init: The initial center solution.
Can be left as None.
stdev_min: The minimum value for the standard deviation
values of the Gaussian search distribution.
Can be left as None (which is the default),
or can be given as a scalar or as a 1-dimensional array.
stdev_max: The maximum value for the standard deviation
values of the Gaussian search distribution.
Can be left as None (which is the default),
or can be given as a scalar or as a 1-dimensional array.
stdev_max_change: The maximum update ratio allowed on the
standard deviation. Expected as None if no such limiter
is needed, or as a real number within 0.0 and 1.0 otherwise.
In the PGPE implementation of Ha (2017, 2018), a value of
0.2 (20%) was used.
For this CEM implementation, the default is None.
obj_index: Index of the objective according to which the
gradient estimations will be done.
For single-objective problems, this can be left as None.
distributed: Whether or not the gradient computation will
be distributed. If `distributed` is given as False and
the problem is not parallelized, then everything will
be centralized (i.e. the entire computation will happen
in the main process).
If `distributed` is given as False, and the problem
is parallelized, then the population will be created
in the main process and then sent to remote workers
for parallelized evaluation, and then the remote fitnesses
will be collected by the main process again for computing
the search gradients.
If `distributed` is given as True, and the problem
is parallelized, then the search algorithm itself will
be distributed, in the sense that each remote actor will
generate its own population (such that the total population
size across all these actors becomes equal to `popsize`)
and will compute its own gradient, and then the main process
will collect these gradients, compute the averaged gradients
and update the main search distribution.
Non-distributed mode has the advantage of keeping the
population in the main process, which is good when one wishes
to do detailed monitoring during the evolutionary process,
but has the disadvantage of having to pass the solutions to
the remote actors and having to collect fitnesses, which
might result in increased interprocess communication traffic.
On the other hand, while it is not possible to monitor the
population in distributed mode, the distributed mode has the
advantage of significantly reducing the interprocess
communication traffic, since the only things communicated
with the remote actors are the search distributions (not the
solutions) and the gradients.
popsize_weighted_grad_avg: Only to be used in distributed mode.
(where being in distributed mode means `distributed` is given
as True). In distributed mode, each actor remotely samples
its own solution batches and computes its own gradients.
These gradients are then collected, and a final average
gradient is computed.
If `popsize_weighted_grad_avg` is True, then, while averaging
over the gradients, each gradient will have its own weight
that is computed according to how many solutions were sampled
by the actor that produced the gradient.
If `popsize_weighted_grad_avg` is False, then, there will not
be weighted averaging (or, each gradient will have equal
weight).
If `popsize_weighted_grad_avg` is None, then, the gradient
weights will be equal a value for `num_interactions` is given
(because `num_interactions` affects the number of solutions
according to the episode lengths, and popsize-weighting the
gradients could be misleading); and the gradient weights will
be weighted according to the sub-population (i.e. sub-batch)
sizes if `num_interactions` is left as None.
The default value for `popsize_weighted_grad_avg` is None.
When the distributed mode is disabled (i.e. when `distributed`
is False), then the argument `popsize_weighted_grad_avg` is
expected as None.
"""
self.DISTRIBUTION_PARAMS = {"parenthood_ratio": float(parenthood_ratio)}
super().__init__(
problem,
popsize=popsize,
center_learning_rate=1.0,
stdev_learning_rate=1.0,
stdev_init=stdev_init,
radius_init=radius_init,
popsize_max=popsize_max,
num_interactions=num_interactions,
optimizer=None,
optimizer_config=None,
ranking_method=None,
center_init=center_init,
stdev_min=stdev_min,
stdev_max=stdev_max,
stdev_max_change=stdev_max_change,
obj_index=obj_index,
distributed=distributed,
popsize_weighted_grad_avg=popsize_weighted_grad_avg,
)
DISTRIBUTION_TYPE (Distribution)
¶
Separable Multivariate Gaussian, as used by PGPE
Source code in evotorch/algorithms/distributed/gaussian.py
class SeparableGaussian(Distribution):
"""Separable Multivariate Gaussian, as used by PGPE"""
MANDATORY_PARAMETERS = {"mu", "sigma"}
OPTIONAL_PARAMETERS = {"divide_mu_grad_by", "divide_sigma_grad_by", "parenthood_ratio"}
PARAMETER_NDIMS = {"mu": 1, "sigma": 1}
@classmethod
def _unbatched_functional_sample(cls, num_solutions: int, mu: torch.Tensor, sigma: torch.Tensor) -> torch.Tensor:
[L] = mu.shape
[sigma_L] = sigma.shape
if L != sigma_L:
raise ValueError(f"The lengths of `mu` ({L}) and `sigma` ({sigma_L}) do not match.")
mu = mu.expand(int(num_solutions), L)
return torch.normal(mu, sigma)
@classmethod
def functional_sample(cls, num_solutions: int, parameters: dict) -> torch.Tensor:
"""
Sample and return separable Gaussian noise
This is a static utility method, which allows one to sample separable
Gaussian noise, without having to instantiate the distribution class
`SeparableGaussian`.
Args:
num_solutions: Number of solutions (or 1-dimensional tensors)
that will be sampled.
parameters: A parameter dictionary. Within this parameter
dictionary, the item `mu` is expected to store the mean, and
the item `sigma` is expected to store the standard deviation,
each in the form of a 1-dimensional tensor.
Returns:
Sampled separable Gaussian noise, as a PyTorch tensor.
If `mu` and/or `sigma` was given as tensors with 2 or more
dimensions (instead of only 1 dimension), the extra leftmost
dimensions will be interpreted as batch dimensions, and therefore,
this returned tensor will also have batch dimensions.
"""
from .decorators import expects_ndim
for k in parameters.keys():
if (k not in cls.MANDATORY_PARAMETERS) and (k not in cls.OPTIONAL_PARAMETERS):
raise ValueError(f"{cls.__name__} encountered an unrecognized parameter: {repr(k)}")
mu = parameters["mu"]
sigma = parameters["sigma"]
return expects_ndim(cls._unbatched_functional_sample, (None, 1, 1), randomness="different")(
num_solutions, mu, sigma
)
def __init__(
self,
parameters: dict,
*,
solution_length: Optional[int] = None,
device: Optional[Device] = None,
dtype: Optional[DType] = None,
):
[mu_length] = parameters["mu"].shape
[sigma_length] = parameters["sigma"].shape
if solution_length is None:
solution_length = mu_length
else:
if solution_length != mu_length:
raise ValueError(
f"The argument `solution_length` does not match the length of `mu` provided in `parameters`."
f" solution_length={solution_length},"
f' parameters["mu"]={mu_length}.'
)
if mu_length != sigma_length:
raise ValueError(
f"The tensors `mu` and `sigma` provided within `parameters` have mismatching lengths."
f' parameters["mu"]={mu_length},'
f' parameters["sigma"]={sigma_length}.'
)
super().__init__(
solution_length=solution_length,
parameters=parameters,
device=device,
dtype=dtype,
)
@property
def mu(self) -> torch.Tensor:
return self.parameters["mu"]
@mu.setter
def mu(self, new_mu: Iterable):
self.parameters["mu"] = torch.as_tensor(new_mu, dtype=self.dtype, device=self.device)
@property
def sigma(self) -> torch.Tensor:
return self.parameters["sigma"]
@sigma.setter
def sigma(self, new_sigma: Iterable):
self.parameters["sigma"] = torch.as_tensor(new_sigma, dtype=self.dtype, device=self.device)
def _fill(self, out: torch.Tensor, *, generator: Optional[torch.Generator] = None):
self.make_gaussian(out=out, center=self.mu, stdev=self.sigma, generator=generator)
def _divide_grad(self, param_name: str, grad: torch.Tensor, weights: torch.Tensor) -> torch.Tensor:
option = f"divide_{param_name}_grad_by"
if option in self.parameters:
div_by_what = self.parameters[option]
if div_by_what == "num_solutions":
[num_solutions] = weights.shape
grad = grad / num_solutions
elif div_by_what == "num_directions":
[num_solutions] = weights.shape
num_directions = num_solutions // 2
grad = grad / num_directions
elif div_by_what == "total_weight":
total_weight = torch.sum(torch.abs(weights))
grad = grad / total_weight
elif div_by_what == "weight_stdev":
weight_stdev = torch.std(weights)
grad = grad / weight_stdev
else:
raise ValueError(f"The parameter {option} has an unrecognized value: {div_by_what}")
return grad
def _compute_gradients_via_parenthood_ratio(self, samples: torch.Tensor, weights: torch.Tensor) -> dict:
[num_samples, _] = samples.shape
num_elites = math.floor(num_samples * self.parameters["parenthood_ratio"])
elite_indices = weights.argsort(descending=True)[:num_elites]
elites = samples[elite_indices, :]
return {
"mu": torch.mean(elites, dim=0) - self.parameters["mu"],
"sigma": torch.std(elites, dim=0) - self.parameters["sigma"],
}
def _compute_gradients(self, samples: torch.Tensor, weights: torch.Tensor, ranking_used: Optional[str]) -> dict:
if "parenthood_ratio" in self.parameters:
return self._compute_gradients_via_parenthood_ratio(samples, weights)
else:
mu = self.mu
sigma = self.sigma
# Compute the scaled noises, that is, the noise vectors which
# were used for generating the solutions
# (solution = scaled_noise + center)
scaled_noises = samples - mu
# Make sure that the weights (utilities) are 0-centered
# (Otherwise the formulations would have to consider a bias term)
if ranking_used not in ("centered", "normalized"):
weights = weights - torch.mean(weights)
mu_grad = self._divide_grad(
"mu",
total(dot(weights, scaled_noises)),
weights,
)
sigma_grad = self._divide_grad(
"sigma",
total(dot(weights, ((scaled_noises**2) - (sigma**2)) / sigma)),
weights,
)
return {
"mu": mu_grad,
"sigma": sigma_grad,
}
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "SeparableGaussian":
mu_grad = gradients["mu"]
sigma_grad = gradients["sigma"]
new_mu = self.mu + self._follow_gradient("mu", mu_grad, learning_rates=learning_rates, optimizers=optimizers)
new_sigma = self.sigma + self._follow_gradient(
"sigma", sigma_grad, learning_rates=learning_rates, optimizers=optimizers
)
return self.modified_copy(mu=new_mu, sigma=new_sigma)
def relative_entropy(dist_0: "SeparableGaussian", dist_1: "SeparableGaussian") -> float:
mu_0 = dist_0.parameters["mu"]
mu_1 = dist_1.parameters["mu"]
sigma_0 = dist_0.parameters["sigma"]
sigma_1 = dist_1.parameters["sigma"]
cov_0 = sigma_0.pow(2.0)
cov_1 = sigma_1.pow(2.0)
mu_delta = mu_1 - mu_0
trace_cov = torch.sum(cov_0 / cov_1)
k = dist_0.solution_length
scaled_mu = torch.sum(mu_delta.pow(2.0) / cov_1)
log_det = torch.sum(torch.log(cov_1)) - torch.sum(torch.log(cov_0))
return 0.5 * (trace_cov - k + scaled_mu + log_det)
functional_sample(num_solutions, parameters)
classmethod
¶Sample and return separable Gaussian noise
This is a static utility method, which allows one to sample separable
Gaussian noise, without having to instantiate the distribution class
SeparableGaussian.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
num_solutions |
int |
Number of solutions (or 1-dimensional tensors) that will be sampled. |
required |
parameters |
dict |
A parameter dictionary. Within this parameter
dictionary, the item |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
Sampled separable Gaussian noise, as a PyTorch tensor.
If |
Source code in evotorch/algorithms/distributed/gaussian.py
@classmethod
def functional_sample(cls, num_solutions: int, parameters: dict) -> torch.Tensor:
"""
Sample and return separable Gaussian noise
This is a static utility method, which allows one to sample separable
Gaussian noise, without having to instantiate the distribution class
`SeparableGaussian`.
Args:
num_solutions: Number of solutions (or 1-dimensional tensors)
that will be sampled.
parameters: A parameter dictionary. Within this parameter
dictionary, the item `mu` is expected to store the mean, and
the item `sigma` is expected to store the standard deviation,
each in the form of a 1-dimensional tensor.
Returns:
Sampled separable Gaussian noise, as a PyTorch tensor.
If `mu` and/or `sigma` was given as tensors with 2 or more
dimensions (instead of only 1 dimension), the extra leftmost
dimensions will be interpreted as batch dimensions, and therefore,
this returned tensor will also have batch dimensions.
"""
from .decorators import expects_ndim
for k in parameters.keys():
if (k not in cls.MANDATORY_PARAMETERS) and (k not in cls.OPTIONAL_PARAMETERS):
raise ValueError(f"{cls.__name__} encountered an unrecognized parameter: {repr(k)}")
mu = parameters["mu"]
sigma = parameters["sigma"]
return expects_ndim(cls._unbatched_functional_sample, (None, 1, 1), randomness="different")(
num_solutions, mu, sigma
)
update_parameters(self, gradients, *, learning_rates=None, optimizers=None)
¶Do an update on the distribution by following the given gradients.
It is expected that the inheriting class has its own implementation for this method.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
gradients |
dict |
Gradients, as a dictionary, which will be used for computing the necessary updates. |
required |
learning_rates |
Optional[dict] |
A dictionary which contains learning rates for parameters that will be updated using a learning rate coefficient. |
None |
optimizers |
Optional[dict] |
A dictionary which contains optimizer objects for parameters that will be updated using an adaptive optimizer. |
None |
Returns:
| Type | Description |
|---|---|
SeparableGaussian |
The updated copy of the distribution. |
Source code in evotorch/algorithms/distributed/gaussian.py
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "SeparableGaussian":
mu_grad = gradients["mu"]
sigma_grad = gradients["sigma"]
new_mu = self.mu + self._follow_gradient("mu", mu_grad, learning_rates=learning_rates, optimizers=optimizers)
new_sigma = self.sigma + self._follow_gradient(
"sigma", sigma_grad, learning_rates=learning_rates, optimizers=optimizers
)
return self.modified_copy(mu=new_mu, sigma=new_sigma)
__init__(self, problem, *, popsize, parenthood_ratio, stdev_init=None, radius_init=None, num_interactions=None, popsize_max=None, center_init=None, stdev_min=None, stdev_max=None, stdev_max_change=None, obj_index=None, distributed=False, popsize_weighted_grad_avg=None)
special
¶
__init__(...): Initialize the search algorithm.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object to work on. |
required |
popsize |
int |
The population size. |
required |
parenthood_ratio |
float |
Expected as a float larger than 0 and smaller
than 1. For example, setting this value to 0.1 means that
the top 10% of the population will be declared as the parents,
and those parents will be used for updating the population.
The amount of parents is always computed according to the
specified |
required |
stdev_init |
Union[float, Iterable[float], torch.Tensor] |
The initial standard deviation of the search
distribution, expressed as a scalar or as an array.
Determines the initial coverage area of the search
distribution.
If one wishes to configure the coverage area via the
argument |
None |
radius_init |
Union[float, Iterable[float], torch.Tensor] |
The initial radius of the search distribution,
expressed as a scalar.
Determines the initial coverage area of the search
distribution.
Here, "radius" is defined as the norm of the search
distribution.
If one wishes to configure the coverage area via the
argument |
None |
num_interactions |
Optional[int] |
When given as an integer n, it is ensured that a population has interacted with the GymProblem's environment n times. If this target has not been reached yet, then the population is declared too small, and gets extended with more samples, until n amount of interactions is reached. When given as None, popsize is the only configuration affecting the size of a population. |
None |
popsize_max |
Optional[int] |
Having |
None |
center_init |
Union[float, Iterable[float], torch.Tensor] |
The initial center solution. Can be left as None. |
None |
stdev_min |
Union[float, Iterable[float], torch.Tensor] |
The minimum value for the standard deviation values of the Gaussian search distribution. Can be left as None (which is the default), or can be given as a scalar or as a 1-dimensional array. |
None |
stdev_max |
Union[float, Iterable[float], torch.Tensor] |
The maximum value for the standard deviation values of the Gaussian search distribution. Can be left as None (which is the default), or can be given as a scalar or as a 1-dimensional array. |
None |
stdev_max_change |
Union[float, Iterable[float], torch.Tensor] |
The maximum update ratio allowed on the standard deviation. Expected as None if no such limiter is needed, or as a real number within 0.0 and 1.0 otherwise. In the PGPE implementation of Ha (2017, 2018), a value of 0.2 (20%) was used. For this CEM implementation, the default is None. |
None |
obj_index |
Optional[int] |
Index of the objective according to which the gradient estimations will be done. For single-objective problems, this can be left as None. |
None |
distributed |
bool |
Whether or not the gradient computation will
be distributed. If |
False |
popsize_weighted_grad_avg |
Optional[bool] |
Only to be used in distributed mode.
(where being in distributed mode means |
None |
Source code in evotorch/algorithms/distributed/gaussian.py
def __init__(
self,
problem: Problem,
*,
popsize: int,
parenthood_ratio: float,
stdev_init: Optional[RealOrVector] = None,
radius_init: Optional[RealOrVector] = None,
num_interactions: Optional[int] = None,
popsize_max: Optional[int] = None,
center_init: Optional[RealOrVector] = None,
stdev_min: Optional[RealOrVector] = None,
stdev_max: Optional[RealOrVector] = None,
stdev_max_change: Optional[Union[float, RealOrVector]] = None,
obj_index: Optional[int] = None,
distributed: bool = False,
popsize_weighted_grad_avg: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the search algorithm.
Args:
problem: The problem object to work on.
popsize: The population size.
parenthood_ratio: Expected as a float larger than 0 and smaller
than 1. For example, setting this value to 0.1 means that
the top 10% of the population will be declared as the parents,
and those parents will be used for updating the population.
The amount of parents is always computed according to the
specified `popsize`, not according to the adapted population
size, and not according to `popsize_max`.
stdev_init: The initial standard deviation of the search
distribution, expressed as a scalar or as an array.
Determines the initial coverage area of the search
distribution.
If one wishes to configure the coverage area via the
argument `radius_init` instead, then `stdev_init` is expected
as None.
radius_init: The initial radius of the search distribution,
expressed as a scalar.
Determines the initial coverage area of the search
distribution.
Here, "radius" is defined as the norm of the search
distribution.
If one wishes to configure the coverage area via the
argument `stdev_init` instead, then `radius_init` is expected
as None.
num_interactions: When given as an integer n,
it is ensured that a population has interacted with
the GymProblem's environment n times. If this target
has not been reached yet, then the population is declared
too small, and gets extended with more samples,
until n amount of interactions is reached.
When given as None, popsize is the only configuration
affecting the size of a population.
popsize_max: Having `num_interactions` set as an integer
might cause the effective population size jump to
unnecesarily large numbers. To prevent this,
one can set `popsize_max` to specify an upper
bound for the effective population size.
center_init: The initial center solution.
Can be left as None.
stdev_min: The minimum value for the standard deviation
values of the Gaussian search distribution.
Can be left as None (which is the default),
or can be given as a scalar or as a 1-dimensional array.
stdev_max: The maximum value for the standard deviation
values of the Gaussian search distribution.
Can be left as None (which is the default),
or can be given as a scalar or as a 1-dimensional array.
stdev_max_change: The maximum update ratio allowed on the
standard deviation. Expected as None if no such limiter
is needed, or as a real number within 0.0 and 1.0 otherwise.
In the PGPE implementation of Ha (2017, 2018), a value of
0.2 (20%) was used.
For this CEM implementation, the default is None.
obj_index: Index of the objective according to which the
gradient estimations will be done.
For single-objective problems, this can be left as None.
distributed: Whether or not the gradient computation will
be distributed. If `distributed` is given as False and
the problem is not parallelized, then everything will
be centralized (i.e. the entire computation will happen
in the main process).
If `distributed` is given as False, and the problem
is parallelized, then the population will be created
in the main process and then sent to remote workers
for parallelized evaluation, and then the remote fitnesses
will be collected by the main process again for computing
the search gradients.
If `distributed` is given as True, and the problem
is parallelized, then the search algorithm itself will
be distributed, in the sense that each remote actor will
generate its own population (such that the total population
size across all these actors becomes equal to `popsize`)
and will compute its own gradient, and then the main process
will collect these gradients, compute the averaged gradients
and update the main search distribution.
Non-distributed mode has the advantage of keeping the
population in the main process, which is good when one wishes
to do detailed monitoring during the evolutionary process,
but has the disadvantage of having to pass the solutions to
the remote actors and having to collect fitnesses, which
might result in increased interprocess communication traffic.
On the other hand, while it is not possible to monitor the
population in distributed mode, the distributed mode has the
advantage of significantly reducing the interprocess
communication traffic, since the only things communicated
with the remote actors are the search distributions (not the
solutions) and the gradients.
popsize_weighted_grad_avg: Only to be used in distributed mode.
(where being in distributed mode means `distributed` is given
as True). In distributed mode, each actor remotely samples
its own solution batches and computes its own gradients.
These gradients are then collected, and a final average
gradient is computed.
If `popsize_weighted_grad_avg` is True, then, while averaging
over the gradients, each gradient will have its own weight
that is computed according to how many solutions were sampled
by the actor that produced the gradient.
If `popsize_weighted_grad_avg` is False, then, there will not
be weighted averaging (or, each gradient will have equal
weight).
If `popsize_weighted_grad_avg` is None, then, the gradient
weights will be equal a value for `num_interactions` is given
(because `num_interactions` affects the number of solutions
according to the episode lengths, and popsize-weighting the
gradients could be misleading); and the gradient weights will
be weighted according to the sub-population (i.e. sub-batch)
sizes if `num_interactions` is left as None.
The default value for `popsize_weighted_grad_avg` is None.
When the distributed mode is disabled (i.e. when `distributed`
is False), then the argument `popsize_weighted_grad_avg` is
expected as None.
"""
self.DISTRIBUTION_PARAMS = {"parenthood_ratio": float(parenthood_ratio)}
super().__init__(
problem,
popsize=popsize,
center_learning_rate=1.0,
stdev_learning_rate=1.0,
stdev_init=stdev_init,
radius_init=radius_init,
popsize_max=popsize_max,
num_interactions=num_interactions,
optimizer=None,
optimizer_config=None,
ranking_method=None,
center_init=center_init,
stdev_min=stdev_min,
stdev_max=stdev_max,
stdev_max_change=stdev_max_change,
obj_index=obj_index,
distributed=distributed,
popsize_weighted_grad_avg=popsize_weighted_grad_avg,
)
GaussianSearchAlgorithm (SearchAlgorithm, SinglePopulationAlgorithmMixin)
¶
Base class for search algorithms based on Gaussian distribution.
Source code in evotorch/algorithms/distributed/gaussian.py
class GaussianSearchAlgorithm(SearchAlgorithm, SinglePopulationAlgorithmMixin):
"""
Base class for search algorithms based on Gaussian distribution.
"""
DISTRIBUTION_TYPE = NotImplemented
DISTRIBUTION_PARAMS = NotImplemented
def __init__(
self,
problem: Problem,
*,
popsize: int,
center_learning_rate: float,
stdev_learning_rate: float,
stdev_init: Optional[RealOrVector] = None,
radius_init: Optional[RealOrVector] = None,
num_interactions: Optional[int] = None,
popsize_max: Optional[int] = None,
optimizer=None,
optimizer_config: Optional[dict] = None,
ranking_method: Optional[str] = None,
center_init: Optional[RealOrVector] = None,
stdev_min: Optional[RealOrVector] = None,
stdev_max: Optional[RealOrVector] = None,
stdev_max_change: Optional[RealOrVector] = None,
obj_index: Optional[int] = None,
distributed: bool = False,
popsize_weighted_grad_avg: Optional[bool] = None,
ensure_even_popsize: bool = False,
):
# Ensure that the problem is numeric
problem.ensure_numeric()
# The distribution-based algorithms we consider here cannot handle strict lower and upper bound constraints.
# Therefore, we ensure that the given problem is unbounded.
problem.ensure_unbounded()
# Initialize the SearchAlgorithm, which is the parent class
SearchAlgorithm.__init__(
self,
problem,
center=self._get_mu,
stdev=self._get_sigma,
mean_eval=self._get_mean_eval,
)
self._ensure_even_popsize = bool(ensure_even_popsize)
if not distributed:
# self.add_status_getters({"median_eval": self._get_median_eval})
if num_interactions is not None:
self.add_status_getters({"popsize": self._get_popsize})
if self._ensure_even_popsize:
if (popsize % 2) != 0:
raise ValueError(
f"`popsize` was expected as an even number. However, the received `popsize` is {popsize}."
)
if center_init is None:
# If a starting point for the search distribution is not given,
# then we use the problem object to generate us one.
mu = problem.generate_values(1).reshape(-1)
else:
# If a starting point for the search distribution is given,
# then we make sure that its length, dtype, and device
# are correct.
mu = problem.ensure_tensor_length_and_dtype(center_init, allow_scalar=False, about="center_init")
# Get the standard deviation or the radius configuration from the arguments
stdev_init = to_stdev_init(
solution_length=problem.solution_length, stdev_init=stdev_init, radius_init=radius_init
)
# Make sure that the provided initial standard deviation is
# of correct length, dtype, and device.
sigma = problem.ensure_tensor_length_and_dtype(stdev_init, about="stdev_init", allow_scalar=False)
# Create the distribution
dist_cls = self.DISTRIBUTION_TYPE
dist_params = deepcopy(self.DISTRIBUTION_PARAMS) if self.DISTRIBUTION_PARAMS is not None else {}
dist_params.update({"mu": mu, "sigma": sigma})
self._distribution: Distribution = dist_cls(dist_params, dtype=problem.dtype, device=problem.device)
# Store the following keyword arguments to use later
self._popsize = int(popsize)
self._popsize_max = None if popsize_max is None else int(popsize_max)
self._num_interactions = None if num_interactions is None else int(num_interactions)
self._center_learning_rate = float(center_learning_rate)
self._stdev_learning_rate = float(stdev_learning_rate)
self._optimizer = self._initialize_optimizer(self._center_learning_rate, optimizer, optimizer_config)
self._ranking_method = None if ranking_method is None else str(ranking_method)
self._stdev_min = (
None
if stdev_min is None
else problem.ensure_tensor_length_and_dtype(stdev_min, about="stdev_min", allow_scalar=True)
)
self._stdev_max = (
None
if stdev_max is None
else problem.ensure_tensor_length_and_dtype(stdev_max, about="stdev_max", allow_scalar=True)
)
self._stdev_max_change = (
None
if stdev_max_change is None
else problem.ensure_tensor_length_and_dtype(stdev_max_change, about="stdev_max_change", allow_scalar=True)
)
self._obj_index = problem.normalize_obj_index(obj_index)
if distributed and (problem.num_actors > 0):
# If the algorithm is initialized in distributed mode, and also if the problem is configured
# for parallelization, then the _step method becomes an alias for _step_distributed
self._step = self._step_distributed
else:
# Otherwise, the _step method becomes an alias for _step_non_distributed
self._step = self._step_non_distributed
if popsize_weighted_grad_avg is None:
self._popsize_weighted_grad_avg = num_interactions is None
else:
if not distributed:
raise ValueError(
"The argument `popsize_weighted_grad_avg` can only be used in distributed mode."
" (i.e. when the argument `distributed` is given as True)."
" When `distributed` is False, please leave `popsize_weighted_grad_avg` as None."
)
self._popsize_weighted_grad_avg = bool(popsize_weighted_grad_avg)
self._mean_eval: Optional[float] = None
self._population: Optional[SolutionBatch] = None
self._first_iter: bool = True
# We would like to add the reporting capabilities of the mixin class `singlePopulationAlgorithmMixin`.
# However, we exclude "mean_eval" from the reporting services requested from `SinglePopulationAlgorithmMixin`
# because this class has its own reporting mechanism for `mean_eval`.
# Additionally, we enable the reporting services of `SinglePopulationAlgorithmMixin` only when we are
# in the non-distributed mode. This is because we do not have a centrally stored population at all in the
# distributed mode.
SinglePopulationAlgorithmMixin.__init__(self, exclude="mean_eval", enable=(not distributed))
def _initialize_optimizer(
self, learning_rate: float, optimizer=None, optimizer_config: Optional[dict] = None
) -> object:
if optimizer is None:
return None
elif isinstance(optimizer, str):
center_optim_cls = get_optimizer_class(optimizer, optimizer_config)
return center_optim_cls(
stepsize=float(learning_rate),
dtype=self._distribution.dtype,
solution_length=self._distribution.solution_length,
device=self._distribution.device,
)
else:
return optimizer
def _step(self):
raise NotImplementedError
def _step_distributed(self):
# Use the problem object's `sample_and_compute_gradients` method
# to do parallelized and distributed gradient computation
fetched = self.problem.sample_and_compute_gradients(
self._distribution,
self._popsize,
popsize_max=self._popsize_max,
obj_index=self._obj_index,
num_interactions=self._num_interactions,
ranking_method=self._ranking_method,
ensure_even_popsize=self._ensure_even_popsize,
)
# The method `sample_and_compute_gradients(...)` returns a list of dictionaries, each dictionary being
# the result of a different remote computation.
# For each remote computation, the list will contain a dictionary that looks like this:
# {"gradients": <gradients dictionary here>, "num_solutions": ..., "mean_eval": ...}
# We will now accumulate all the gradients, num_solutions, and mean_evals in their own lists.
# So, in the end, we will have a list of gradients, a list of num_solutions, and a list of
# mean_eval.
# These lists will be stored by the following temporary class:
class list_of:
gradients = []
num_solutions = []
mean_eval = []
# We are now filling the lists declared above
n = len(fetched)
for i in range(n):
list_of.gradients.append(fetched[i]["gradients"])
list_of.num_solutions.append(fetched[i]["num_solutions"])
list_of.mean_eval.append(fetched[i]["mean_eval"])
# Here, we get the keys of our gradient dictionaries.
# For most simple Gaussian distributions, grad_keys should be {"mu", "sigma"}.
grad_keys = set(list_of.gradients[0].keys())
# We now find the total number of solutions and the overall average mean_eval.
# The overall average mean will be reported to the user.
total_num_solutions = 0
total_weighted_eval = 0
for i in range(n):
total_num_solutions += list_of.num_solutions[i]
total_weighted_eval += float(list_of.num_solutions[i] * list_of.mean_eval[i])
avg_mean_eval = total_weighted_eval / total_num_solutions
# For each gradient (in most cases among 'mu' and 'sigma'), we allocate a new 0-filled tensor.
avg_gradients = {}
for key in grad_keys:
avg_gradients[key] = self._distribution.make_zeros(num_solutions=1).reshape(-1)
# Below, we iterate over all collected results and add their gradients, in a weighted manner, onto the
# `avg_gradients` we allocated above.
# At the end, `avg_gradients` will store the weighted-averaged gradients to be followed by the algorithm.
for i in range(n):
# For each collected result, we compute a weight for the gradient, which is the number of solutions
# sampled divided by the total number of sampled solutions.
num_solutions = list_of.num_solutions[i]
if self._popsize_weighted_grad_avg:
# If we are to weigh each gradient by its popsize (i.e. by its sample size)
# then the its weight is computed as its number of solutions divided by the
# total number of solutions
weight = num_solutions / total_num_solutions
else:
# If we are NOT to weigh each gradient by its popsize (i.e. by its sample size)
# then the weight of this gradient simply becomes 1 divided by the number of gradients.
weight = 1 / n
for key in grad_keys:
grad = list_of.gradients[i][key]
avg_gradients[key] += weight * grad
self._update_distribution(avg_gradients)
self._mean_eval = avg_mean_eval
def _step_non_distributed(self):
# First, we define an inner function which fills the current population by sampling from the distribution.
def fill_and_eval_pop():
# This inner function is responsible for filling the main population with samples
# and evaluate them.
if self._num_interactions is None:
# If num_interactions is configured as None, this means that we are not going to adapt
# the population size according to the number of simulation interactions reported
# by the problem object.
# We first make sure that the population (which is to be of constant size, since we are
# not in the adaptive population size mode) is allocated.
if self._population is None:
self._population = SolutionBatch(
self.problem, popsize=self._popsize, device=self._distribution.device, empty=True
)
# Now, we do in-place sampling on the population.
self._distribution.sample(out=self._population.access_values(), generator=self.problem)
# Finally, here, the solutions are evaluated.
self.problem.evaluate(self._population)
else:
# If num_interactions is not None, then this means that we have a threshold for the number
# of simulator interactions to reach before declaring the phase of sampling complete.
# In other words, we have to adapt our population size according to the number of simulator
# interactions reported by the problem object.
# The 'total_interaction_count' status reported by the problem object shows the global interaction count.
# Therefore, to properly count the simulator interactions we made during this generation, we need
# to get the interaction count before starting our sampling and evaluation operations.
first_num_interactions = self.problem.status.get("total_interaction_count", 0)
# We will keep allocating and evaluating new populations until the interaction count threshold is reached.
# These newly allocated populations will eventually concatenated into one.
# The not-yet-concatenated populations and the total allocated population size will be stored below:
populations = []
total_popsize = 0
# Below, we repeatedly allocate, sample, and evaluate, until our thresholds are reached.
while True:
# Allocate a new population
newpop = SolutionBatch(
self.problem,
popsize=self._popsize,
like=self._population,
empty=True,
)
# Update the total population size
total_popsize += len(newpop)
# Sample new solutions within the newly allocated population
self._distribution.sample(out=newpop.access_values(), generator=self.problem)
# Evaluate the new population
self.problem.evaluate(newpop)
# Add the newly allocated and evaluated population into the populations list
populations.append(newpop)
# In addition to the num_interactions threshold, we might also have a popsize_max threshold.
# We now check this threshold.
if (self._popsize_max is not None) and (total_popsize >= self._popsize_max):
# If the popsize_max threshold is reached, we leave the loop.
break
# We now compute the number of interactions we have made during this while loop.
interactions_made = self.problem.status["total_interaction_count"] - first_num_interactions
if interactions_made > self._num_interactions:
# If the number of interactions exceeds our threshold, we leave the loop.
break
# Finally, we concatenate all our populations into one.
self._population = SolutionBatch.cat(populations)
if self._first_iter:
# If we are computing the first generation, we just sample from our distribution and evaluate
# the solutions.
fill_and_eval_pop()
self._first_iter = False
else:
# If we are computing next generations, then we need to compute the gradients of the last
# generation, sample a new population, and evaluate the new population's solutions.
samples = self._population.access_values(keep_evals=True)
fitnesses = self._population.access_evals()[:, self._obj_index]
obj_sense = self.problem.senses[self._obj_index]
ranking_method = self._ranking_method
gradients = self._distribution.compute_gradients(
samples, fitnesses, objective_sense=obj_sense, ranking_method=ranking_method
)
self._update_distribution(gradients)
fill_and_eval_pop()
def _update_distribution(self, gradients: dict):
# This is where we follow the gradients with the help of the stored Distribution object.
# First, we check whether or not we will need to do a controlled update on the
# standard deviation (do we have imposed lower and upper bounds for the standard deviation,
# and do we have a maximum change limiter?)
controlled_stdev_update = (
(self._stdev_min is not None) or (self._stdev_max is not None) or (self._stdev_max_change is not None)
)
if controlled_stdev_update:
# If the standard deviation update needs to be controlled, we store the standard deviation just before
# the update. We will use this later.
old_sigma = self._distribution.sigma
# Here, we determine for which distribution parameter we have a learning rate and for which distribution
# parameter we have an optimizer.
learning_rates = {}
optimizers = {}
if self._optimizer is not None:
# If there is an optimizer, then we declare that "mu" has an optimizer
optimizers["mu"] = self._optimizer
else:
# If we do not have an optimizer, then we declare that "mu" has a raw learning rate coefficient
learning_rates["mu"] = self._center_learning_rate
# Here, we declare that "sigma" has a learning rate
learning_rates["sigma"] = self._stdev_learning_rate
# With the help of the Distribution object's `update_parameters(...)` method, we follow the gradients
updated_dist = self._distribution.update_parameters(
gradients, learning_rates=learning_rates, optimizers=optimizers
)
if controlled_stdev_update:
# If our standard deviation update needs to be controlled, then, considering the pre-update
# standard deviation, we ensure that the update constraints (lower and upper bounds and maximum change)
# are not violated.
updated_dist = updated_dist.modified_copy(
sigma=modify_tensor(
old_sigma,
updated_dist.sigma,
lb=self._stdev_min,
ub=self._stdev_max,
max_change=self._stdev_max_change,
)
)
# Now we can declare that our main distribution is the updated one
self._distribution = updated_dist
def _get_mu(self) -> torch.Tensor:
return self._distribution.parameters["mu"]
def _get_sigma(self) -> torch.Tensor:
return self._distribution.parameters["sigma"]
def _get_mean_eval(self) -> Optional[float]:
if self._population is None:
return self._mean_eval
else:
return float(torch.mean(self._population.evals[:, self._obj_index]))
# def _get_median_eval(self) -> Optional[float]:
# if self._population is None:
# return None
# else:
# return float(torch.median(self._population.evals[:, self._obj_index]))
def _get_popsize(self) -> int:
return 0 if self._population is None else len(self._population)
@property
def optimizer(self):
"""
Get the optimizer used by this search algorithm.
If an optimizer is not being used, the result will be `None`.
If a PyTorch optimizer is being used, the result will be an instance of
`torch.optim.Optimizer`.
If the returned optimizer is "clipup", then the returned object will be
an instance of `evotorch.optimizers.ClipUp`.
The returned optimizer object can be used for reading/writing the
hyperparameters. For example, to read the learning of the optimizer,
one can do:
```python
learning_rate = my_search_algorithm.optimizer.param_groups[0]["lr"]
```
One can also update the learning rate like this:
```python
my_search_algorithm.optimizer.param_groups[0]["lr"] = new_learning_rate
```
**Note for when updating the learning rate of ClipUp.**
At the moment of initialization, if one provides `center_learning_rate`
but the maximum speed is not specified (i.e. the search algorithm is not
given something like `optimizer_config={"max_speed": ...}`), then, the
maximum speed is initialized as `2*center_learning_rate`. However, when
this `center_learning_rate` is later modified (via
`my_search_algorithm.optimizer.param_groups[0]["lr"] = new_center_learning_rate`
the maximum speed will NOT be automatically adjusted.
Therefore, when updating the center learning rate of ClipUp, consider
also adjusting the maximum speed of ClipUp via:
`my_search_algorithm.optimizer.param_groups[0]["max_speed"] = ...`
"""
return None if self._optimizer is None else self._optimizer.contained_optimizer
@property
def population(self) -> Optional[SolutionBatch]:
"""
The population, represented by a SolutionBatch.
If the population is not initialized yet, the retrieved value will
be None.
Also note that, if this algorithm is in distributed mode, the
retrieved value will be None, since the distributed mode causes the
population to be generated in the remote actors, and not in the main
process.
"""
return self._population
@property
def obj_index(self) -> int:
"""
Index of the focused objective
"""
return self._obj_index
obj_index: int
property
readonly
¶
Index of the focused objective
optimizer
property
readonly
¶
Get the optimizer used by this search algorithm.
If an optimizer is not being used, the result will be None.
If a PyTorch optimizer is being used, the result will be an instance of
torch.optim.Optimizer.
If the returned optimizer is "clipup", then the returned object will be
an instance of evotorch.optimizers.ClipUp.
The returned optimizer object can be used for reading/writing the hyperparameters. For example, to read the learning of the optimizer, one can do:
One can also update the learning rate like this:
Note for when updating the learning rate of ClipUp.
At the moment of initialization, if one provides center_learning_rate
but the maximum speed is not specified (i.e. the search algorithm is not
given something like optimizer_config={"max_speed": ...}), then, the
maximum speed is initialized as 2*center_learning_rate. However, when
this center_learning_rate is later modified (via
my_search_algorithm.optimizer.param_groups[0]["lr"] = new_center_learning_rate
the maximum speed will NOT be automatically adjusted.
Therefore, when updating the center learning rate of ClipUp, consider
also adjusting the maximum speed of ClipUp via:
my_search_algorithm.optimizer.param_groups[0]["max_speed"] = ...
population: Optional[evotorch.core.SolutionBatch]
property
readonly
¶
The population, represented by a SolutionBatch.
If the population is not initialized yet, the retrieved value will be None. Also note that, if this algorithm is in distributed mode, the retrieved value will be None, since the distributed mode causes the population to be generated in the remote actors, and not in the main process.
PGPE (GaussianSearchAlgorithm)
¶
This implementation is the symmetric-sampling variant proposed in the paper Sehnke et al. (2010).
Inspired by the PGPE implementations used in the studies of Ha (2017, 2019), and by the evolution strategy variant of Salimans et al. (2017), this PGPE implementation uses 0-centered ranking by default. The default optimizer for this PGPE implementation is ClipUp (Toklu et al., 2020).
References:
Frank Sehnke, Christian Osendorfer, Thomas Ruckstiess,
Alex Graves, Jan Peters, Jurgen Schmidhuber (2010).
Parameter-exploring Policy Gradients.
Neural Networks 23(4), 551-559.
David Ha (2017). Evolving Stable Strategies.
<http://blog.otoro.net/2017/11/12/evolving-stable-strategies/>
Salimans, T., Ho, J., Chen, X., Sidor, S. and Sutskever, I. (2017).
Evolution Strategies as a Scalable Alternative to
Reinforcement Learning.
David Ha (2019). Reinforcement Learning for Improving Agent Design.
Artificial life 25 (4), 352-365.
Toklu, N.E., Liskowski, P., Srivastava, R.K. (2020).
ClipUp: A Simple and Powerful Optimizer
for Distribution-based Policy Evolution.
Parallel Problem Solving from Nature (PPSN 2020).
Source code in evotorch/algorithms/distributed/gaussian.py
class PGPE(GaussianSearchAlgorithm):
"""
PGPE: Policy gradient with parameter-based exploration.
This implementation is the symmetric-sampling variant proposed
in the paper Sehnke et al. (2010).
Inspired by the PGPE implementations used in the studies
of Ha (2017, 2019), and by the evolution strategy variant of
Salimans et al. (2017), this PGPE implementation uses 0-centered
ranking by default.
The default optimizer for this PGPE implementation is ClipUp
(Toklu et al., 2020).
References:
Frank Sehnke, Christian Osendorfer, Thomas Ruckstiess,
Alex Graves, Jan Peters, Jurgen Schmidhuber (2010).
Parameter-exploring Policy Gradients.
Neural Networks 23(4), 551-559.
David Ha (2017). Evolving Stable Strategies.
<http://blog.otoro.net/2017/11/12/evolving-stable-strategies/>
Salimans, T., Ho, J., Chen, X., Sidor, S. and Sutskever, I. (2017).
Evolution Strategies as a Scalable Alternative to
Reinforcement Learning.
David Ha (2019). Reinforcement Learning for Improving Agent Design.
Artificial life 25 (4), 352-365.
Toklu, N.E., Liskowski, P., Srivastava, R.K. (2020).
ClipUp: A Simple and Powerful Optimizer
for Distribution-based Policy Evolution.
Parallel Problem Solving from Nature (PPSN 2020).
"""
DISTRIBUTION_TYPE = NotImplemented # To be filled by the PGPE instance
DISTRIBUTION_PARAMS = NotImplemented # To be filled by the PGPE instance
def __init__(
self,
problem: Problem,
*,
popsize: int,
center_learning_rate: float,
stdev_learning_rate: float,
stdev_init: Optional[RealOrVector] = None,
radius_init: Optional[RealOrVector] = None,
num_interactions: Optional[int] = None,
popsize_max: Optional[int] = None,
optimizer="clipup",
optimizer_config: Optional[dict] = None,
ranking_method: Optional[str] = "centered",
center_init: Optional[RealOrVector] = None,
stdev_min: Optional[RealOrVector] = None,
stdev_max: Optional[RealOrVector] = None,
stdev_max_change: Optional[RealOrVector] = 0.2,
symmetric: bool = True,
obj_index: Optional[int] = None,
distributed: bool = False,
popsize_weighted_grad_avg: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the PGPE algorithm.
Args:
problem: The problem object which is being worked on.
The problem must have its dtype defined
(which means it works on Solution objects,
not with custom Solution objects).
Also, the problem must be single-objective.
popsize: The population size.
In the case of PGPE, `popsize` is expected as an even number
in non-distributed mode. In distributed mode, PGPE will
ensure that each sub-population size assigned to a remote
actor is an even number.
This behavior is because PGPE does symmetric sampling
(i.e. solutions are sampled in pairs).
center_learning_rate: The learning rate for the center
of the search distribution.
stdev_learning_rate: The learning rate for the standard
deviation values of the search distribution.
stdev_init: The initial standard deviation of the search
distribution, expressed as a scalar or as an array.
Determines the initial coverage area of the search
distribution.
If one wishes to configure the coverage area via the
argument `radius_init` instead, then `stdev_init` is expected
as None.
radius_init: The initial radius of the search distribution,
expressed as a scalar.
Determines the initial coverage area of the search
distribution.
Here, "radius" is defined as the norm of the search
distribution.
If one wishes to configure the coverage area via the
argument `stdev_init` instead, then `radius_init` is expected
as None.
num_interactions: When given as an integer n,
it is ensured that a population has interacted with
the GymProblem's environment n times. If this target
has not been reached yet, then the population is declared
too small, and gets extended with more samples,
until n amount of interactions is reached.
When given as None, popsize is the only configuration
affecting the size of a population.
popsize_max: Having `num_interactions` set as an integer
might cause the effective population size jump to
unnecesarily large numbers. To prevent this,
one can set `popsize_max` to specify an upper
bound for the effective population size.
optimizer: The optimizer to be used while following the
estimated the gradients.
Can be given as None if a momentum-based optimizer
is not required.
Otherwise, can be given as a str containing the name
of the optimizer (e.g. 'adam', 'clipup');
or as an instance of evotorch.optimizers.TorchOptimizer
or evotorch.optimizers.ClipUp.
The default is 'clipup'.
Note that, for ClipUp, the default maximum speed is set
as twice the given `center_learning_rate`.
This maximum speed can be configured by passing
`{"max_speed": ...}` to `optimizer_config`.
optimizer_config: Configuration which will be passed
to the optimizer as keyword arguments.
See `evotorch.optimizers` for details about
which optimizer accepts which keyword arguments.
ranking_method: Which ranking method will be used for
fitness shaping. See the documentation of
`evotorch.ranking.rank(...)` for details.
As in the study of Salimans et al. (2017),
the default is 'centered'.
Can be given as None if no such ranking is required.
center_init: The initial center solution.
Can be left as None.
stdev_min: Lower bound for the standard deviation value/array.
Can be given as a real number, or as an array of real numbers.
stdev_max: Upper bound for the standard deviation value/array.
Can be given as a real number, or as an array of real numbers.
stdev_max_change: The maximum update ratio allowed on the
standard deviation. Expected as None if no such limiter
is needed, or as a real number within 0.0 and 1.0 otherwise.
Like in the implementation of Ha (2017, 2018),
the default value for this setting is 0.2, meaning that
the update on the standard deviation values can not be
more than 20% of their original values.
symmetric: Whether or not the solutions will be sampled
in a symmetric/mirrored/antithetic manner.
The default is True.
obj_index: Index of the objective according to which the
gradient estimations will be done.
For single-objective problems, this can be left as None.
distributed: Whether or not the gradient computation will
be distributed. If `distributed` is given as False and
the problem is not parallelized, then everything will
be centralized (i.e. the entire computation will happen
in the main process).
If `distributed` is given as False, and the problem
is parallelized, then the population will be created
in the main process and then sent to remote workers
for parallelized evaluation, and then the remote fitnesses
will be collected by the main process again for computing
the search gradients.
If `distributed` is given as True, and the problem
is parallelized, then the search algorithm itself will
be distributed, in the sense that each remote actor will
generate its own population (such that the total population
size across all these actors becomes equal to `popsize`)
and will compute its own gradient, and then the main process
will collect these gradients, compute the averaged gradients
and update the main search distribution.
Non-distributed mode has the advantage of keeping the
population in the main process, which is good when one wishes
to do detailed monitoring during the evolutionary process,
but has the disadvantage of having to pass the solutions to
the remote actors and having to collect fitnesses, which
might result in increased interprocess communication traffic.
On the other hand, while it is not possible to monitor the
population in distributed mode, the distributed mode has the
advantage of significantly reducing the interprocess
communication traffic, since the only things communicated
with the remote actors are the search distributions (not the
solutions) and the gradients.
popsize_weighted_grad_avg: Only to be used in distributed mode.
(where being in distributed mode means `distributed` is given
as True). In distributed mode, each actor remotely samples
its own solution batches and computes its own gradients.
These gradients are then collected, and a final average
gradient is computed.
If `popsize_weighted_grad_avg` is True, then, while averaging
over the gradients, each gradient will have its own weight
that is computed according to how many solutions were sampled
by the actor that produced the gradient.
If `popsize_weighted_grad_avg` is False, then, there will not
be weighted averaging (or, each gradient will have equal
weight).
If `popsize_weighted_grad_avg` is None, then, the gradient
weights will be equal a value for `num_interactions` is given
(because `num_interactions` affects the number of solutions
according to the episode lengths, and popsize-weighting the
gradients could be misleading); and the gradient weights will
be weighted according to the sub-population (i.e. sub-batch)
sizes if `num_interactions` is left as None.
The default value for `popsize_weighted_grad_avg` is None.
When the distributed mode is disabled (i.e. when `distributed`
is False), then the argument `popsize_weighted_grad_avg` is
expected as None.
"""
if symmetric:
self.DISTRIBUTION_TYPE = SymmetricSeparableGaussian
divide_by = "num_directions"
else:
self.DISTRIBUTION_TYPE = SeparableGaussian
divide_by = "num_solutions"
self.DISTRIBUTION_PARAMS = {"divide_mu_grad_by": divide_by, "divide_sigma_grad_by": divide_by}
super().__init__(
problem,
popsize=popsize,
center_learning_rate=center_learning_rate,
stdev_learning_rate=stdev_learning_rate,
stdev_init=stdev_init,
radius_init=radius_init,
popsize_max=popsize_max,
num_interactions=num_interactions,
optimizer=optimizer,
optimizer_config=optimizer_config,
ranking_method=ranking_method,
center_init=center_init,
stdev_min=stdev_min,
stdev_max=stdev_max,
stdev_max_change=stdev_max_change,
obj_index=obj_index,
distributed=distributed,
popsize_weighted_grad_avg=popsize_weighted_grad_avg,
ensure_even_popsize=symmetric,
)
__init__(self, problem, *, popsize, center_learning_rate, stdev_learning_rate, stdev_init=None, radius_init=None, num_interactions=None, popsize_max=None, optimizer='clipup', optimizer_config=None, ranking_method='centered', center_init=None, stdev_min=None, stdev_max=None, stdev_max_change=0.2, symmetric=True, obj_index=None, distributed=False, popsize_weighted_grad_avg=None)
special
¶
__init__(...): Initialize the PGPE algorithm.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object which is being worked on. The problem must have its dtype defined (which means it works on Solution objects, not with custom Solution objects). Also, the problem must be single-objective. |
required |
popsize |
int |
The population size.
In the case of PGPE, |
required |
center_learning_rate |
float |
The learning rate for the center of the search distribution. |
required |
stdev_learning_rate |
float |
The learning rate for the standard deviation values of the search distribution. |
required |
stdev_init |
Union[float, Iterable[float], torch.Tensor] |
The initial standard deviation of the search
distribution, expressed as a scalar or as an array.
Determines the initial coverage area of the search
distribution.
If one wishes to configure the coverage area via the
argument |
None |
radius_init |
Union[float, Iterable[float], torch.Tensor] |
The initial radius of the search distribution,
expressed as a scalar.
Determines the initial coverage area of the search
distribution.
Here, "radius" is defined as the norm of the search
distribution.
If one wishes to configure the coverage area via the
argument |
None |
num_interactions |
Optional[int] |
When given as an integer n, it is ensured that a population has interacted with the GymProblem's environment n times. If this target has not been reached yet, then the population is declared too small, and gets extended with more samples, until n amount of interactions is reached. When given as None, popsize is the only configuration affecting the size of a population. |
None |
popsize_max |
Optional[int] |
Having |
None |
optimizer |
The optimizer to be used while following the
estimated the gradients.
Can be given as None if a momentum-based optimizer
is not required.
Otherwise, can be given as a str containing the name
of the optimizer (e.g. 'adam', 'clipup');
or as an instance of evotorch.optimizers.TorchOptimizer
or evotorch.optimizers.ClipUp.
The default is 'clipup'.
Note that, for ClipUp, the default maximum speed is set
as twice the given |
'clipup' |
|
optimizer_config |
Optional[dict] |
Configuration which will be passed
to the optimizer as keyword arguments.
See |
None |
ranking_method |
Optional[str] |
Which ranking method will be used for
fitness shaping. See the documentation of
|
'centered' |
center_init |
Union[float, Iterable[float], torch.Tensor] |
The initial center solution. Can be left as None. |
None |
stdev_min |
Union[float, Iterable[float], torch.Tensor] |
Lower bound for the standard deviation value/array. Can be given as a real number, or as an array of real numbers. |
None |
stdev_max |
Union[float, Iterable[float], torch.Tensor] |
Upper bound for the standard deviation value/array. Can be given as a real number, or as an array of real numbers. |
None |
stdev_max_change |
Union[float, Iterable[float], torch.Tensor] |
The maximum update ratio allowed on the standard deviation. Expected as None if no such limiter is needed, or as a real number within 0.0 and 1.0 otherwise. Like in the implementation of Ha (2017, 2018), the default value for this setting is 0.2, meaning that the update on the standard deviation values can not be more than 20% of their original values. |
0.2 |
symmetric |
bool |
Whether or not the solutions will be sampled in a symmetric/mirrored/antithetic manner. The default is True. |
True |
obj_index |
Optional[int] |
Index of the objective according to which the gradient estimations will be done. For single-objective problems, this can be left as None. |
None |
distributed |
bool |
Whether or not the gradient computation will
be distributed. If |
False |
popsize_weighted_grad_avg |
Optional[bool] |
Only to be used in distributed mode.
(where being in distributed mode means |
None |
Source code in evotorch/algorithms/distributed/gaussian.py
def __init__(
self,
problem: Problem,
*,
popsize: int,
center_learning_rate: float,
stdev_learning_rate: float,
stdev_init: Optional[RealOrVector] = None,
radius_init: Optional[RealOrVector] = None,
num_interactions: Optional[int] = None,
popsize_max: Optional[int] = None,
optimizer="clipup",
optimizer_config: Optional[dict] = None,
ranking_method: Optional[str] = "centered",
center_init: Optional[RealOrVector] = None,
stdev_min: Optional[RealOrVector] = None,
stdev_max: Optional[RealOrVector] = None,
stdev_max_change: Optional[RealOrVector] = 0.2,
symmetric: bool = True,
obj_index: Optional[int] = None,
distributed: bool = False,
popsize_weighted_grad_avg: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the PGPE algorithm.
Args:
problem: The problem object which is being worked on.
The problem must have its dtype defined
(which means it works on Solution objects,
not with custom Solution objects).
Also, the problem must be single-objective.
popsize: The population size.
In the case of PGPE, `popsize` is expected as an even number
in non-distributed mode. In distributed mode, PGPE will
ensure that each sub-population size assigned to a remote
actor is an even number.
This behavior is because PGPE does symmetric sampling
(i.e. solutions are sampled in pairs).
center_learning_rate: The learning rate for the center
of the search distribution.
stdev_learning_rate: The learning rate for the standard
deviation values of the search distribution.
stdev_init: The initial standard deviation of the search
distribution, expressed as a scalar or as an array.
Determines the initial coverage area of the search
distribution.
If one wishes to configure the coverage area via the
argument `radius_init` instead, then `stdev_init` is expected
as None.
radius_init: The initial radius of the search distribution,
expressed as a scalar.
Determines the initial coverage area of the search
distribution.
Here, "radius" is defined as the norm of the search
distribution.
If one wishes to configure the coverage area via the
argument `stdev_init` instead, then `radius_init` is expected
as None.
num_interactions: When given as an integer n,
it is ensured that a population has interacted with
the GymProblem's environment n times. If this target
has not been reached yet, then the population is declared
too small, and gets extended with more samples,
until n amount of interactions is reached.
When given as None, popsize is the only configuration
affecting the size of a population.
popsize_max: Having `num_interactions` set as an integer
might cause the effective population size jump to
unnecesarily large numbers. To prevent this,
one can set `popsize_max` to specify an upper
bound for the effective population size.
optimizer: The optimizer to be used while following the
estimated the gradients.
Can be given as None if a momentum-based optimizer
is not required.
Otherwise, can be given as a str containing the name
of the optimizer (e.g. 'adam', 'clipup');
or as an instance of evotorch.optimizers.TorchOptimizer
or evotorch.optimizers.ClipUp.
The default is 'clipup'.
Note that, for ClipUp, the default maximum speed is set
as twice the given `center_learning_rate`.
This maximum speed can be configured by passing
`{"max_speed": ...}` to `optimizer_config`.
optimizer_config: Configuration which will be passed
to the optimizer as keyword arguments.
See `evotorch.optimizers` for details about
which optimizer accepts which keyword arguments.
ranking_method: Which ranking method will be used for
fitness shaping. See the documentation of
`evotorch.ranking.rank(...)` for details.
As in the study of Salimans et al. (2017),
the default is 'centered'.
Can be given as None if no such ranking is required.
center_init: The initial center solution.
Can be left as None.
stdev_min: Lower bound for the standard deviation value/array.
Can be given as a real number, or as an array of real numbers.
stdev_max: Upper bound for the standard deviation value/array.
Can be given as a real number, or as an array of real numbers.
stdev_max_change: The maximum update ratio allowed on the
standard deviation. Expected as None if no such limiter
is needed, or as a real number within 0.0 and 1.0 otherwise.
Like in the implementation of Ha (2017, 2018),
the default value for this setting is 0.2, meaning that
the update on the standard deviation values can not be
more than 20% of their original values.
symmetric: Whether or not the solutions will be sampled
in a symmetric/mirrored/antithetic manner.
The default is True.
obj_index: Index of the objective according to which the
gradient estimations will be done.
For single-objective problems, this can be left as None.
distributed: Whether or not the gradient computation will
be distributed. If `distributed` is given as False and
the problem is not parallelized, then everything will
be centralized (i.e. the entire computation will happen
in the main process).
If `distributed` is given as False, and the problem
is parallelized, then the population will be created
in the main process and then sent to remote workers
for parallelized evaluation, and then the remote fitnesses
will be collected by the main process again for computing
the search gradients.
If `distributed` is given as True, and the problem
is parallelized, then the search algorithm itself will
be distributed, in the sense that each remote actor will
generate its own population (such that the total population
size across all these actors becomes equal to `popsize`)
and will compute its own gradient, and then the main process
will collect these gradients, compute the averaged gradients
and update the main search distribution.
Non-distributed mode has the advantage of keeping the
population in the main process, which is good when one wishes
to do detailed monitoring during the evolutionary process,
but has the disadvantage of having to pass the solutions to
the remote actors and having to collect fitnesses, which
might result in increased interprocess communication traffic.
On the other hand, while it is not possible to monitor the
population in distributed mode, the distributed mode has the
advantage of significantly reducing the interprocess
communication traffic, since the only things communicated
with the remote actors are the search distributions (not the
solutions) and the gradients.
popsize_weighted_grad_avg: Only to be used in distributed mode.
(where being in distributed mode means `distributed` is given
as True). In distributed mode, each actor remotely samples
its own solution batches and computes its own gradients.
These gradients are then collected, and a final average
gradient is computed.
If `popsize_weighted_grad_avg` is True, then, while averaging
over the gradients, each gradient will have its own weight
that is computed according to how many solutions were sampled
by the actor that produced the gradient.
If `popsize_weighted_grad_avg` is False, then, there will not
be weighted averaging (or, each gradient will have equal
weight).
If `popsize_weighted_grad_avg` is None, then, the gradient
weights will be equal a value for `num_interactions` is given
(because `num_interactions` affects the number of solutions
according to the episode lengths, and popsize-weighting the
gradients could be misleading); and the gradient weights will
be weighted according to the sub-population (i.e. sub-batch)
sizes if `num_interactions` is left as None.
The default value for `popsize_weighted_grad_avg` is None.
When the distributed mode is disabled (i.e. when `distributed`
is False), then the argument `popsize_weighted_grad_avg` is
expected as None.
"""
if symmetric:
self.DISTRIBUTION_TYPE = SymmetricSeparableGaussian
divide_by = "num_directions"
else:
self.DISTRIBUTION_TYPE = SeparableGaussian
divide_by = "num_solutions"
self.DISTRIBUTION_PARAMS = {"divide_mu_grad_by": divide_by, "divide_sigma_grad_by": divide_by}
super().__init__(
problem,
popsize=popsize,
center_learning_rate=center_learning_rate,
stdev_learning_rate=stdev_learning_rate,
stdev_init=stdev_init,
radius_init=radius_init,
popsize_max=popsize_max,
num_interactions=num_interactions,
optimizer=optimizer,
optimizer_config=optimizer_config,
ranking_method=ranking_method,
center_init=center_init,
stdev_min=stdev_min,
stdev_max=stdev_max,
stdev_max_change=stdev_max_change,
obj_index=obj_index,
distributed=distributed,
popsize_weighted_grad_avg=popsize_weighted_grad_avg,
ensure_even_popsize=symmetric,
)
SNES (GaussianSearchAlgorithm)
¶
Inspired by the implementation at: http://schaul.site44.com/code/snes.py
Reference:
Schaul, T., Glasmachers, T., Schmidhuber, J. (2011).
High Dimensions and Heavy Tails for Natural Evolution Strategies.
Proceedings of the 13th annual conference on Genetic and evolutionary
computation (GECCO 2011).
Source code in evotorch/algorithms/distributed/gaussian.py
class SNES(GaussianSearchAlgorithm):
"""
SNES: Separable Natural Evolution Strategies
Inspired by the implementation at: http://schaul.site44.com/code/snes.py
Reference:
Schaul, T., Glasmachers, T., Schmidhuber, J. (2011).
High Dimensions and Heavy Tails for Natural Evolution Strategies.
Proceedings of the 13th annual conference on Genetic and evolutionary
computation (GECCO 2011).
"""
DISTRIBUTION_TYPE = ExpSeparableGaussian
DISTRIBUTION_PARAMS = None
def __init__(
self,
problem: Problem,
*,
stdev_init: Optional[RealOrVector] = None,
radius_init: Optional[RealOrVector] = None,
popsize: Optional[int] = None,
center_learning_rate: Optional[float] = None,
stdev_learning_rate: Optional[float] = None,
scale_learning_rate: bool = True,
num_interactions: Optional[int] = None,
popsize_max: Optional[int] = None,
optimizer=None,
optimizer_config: Optional[dict] = None,
ranking_method: Optional[str] = "nes",
center_init: Optional[RealOrVector] = None,
stdev_min: Optional[RealOrVector] = None,
stdev_max: Optional[RealOrVector] = None,
stdev_max_change: Optional[RealOrVector] = None,
obj_index: Optional[int] = None,
distributed: bool = False,
popsize_weighted_grad_avg: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the SNES algorithm.
Args:
problem: The problem object which is being worked on.
stdev_init: The initial standard deviation of the search
distribution, expressed as a scalar or as an array.
Determines the initial coverage area of the search
distribution.
If one wishes to configure the coverage area via the
argument `radius_init` instead, then `stdev_init` is expected
as None.
radius_init: The initial radius of the search distribution,
expressed as a scalar.
Determines the initial coverage area of the search
distribution.
Here, "radius" is defined as the norm of the search
distribution.
If one wishes to configure the coverage area via the
argument `stdev_init` instead, then `radius_init` is expected
as None.
popsize: Population size. Can be specified as an int,
or can be left as None to let the solver decide.
In the case of SNES, `popsize` can be left as None,
in which case the default `popsize` will be computed
as `4 + floor(3 * log(n))` where `n` is the length
of a solution.
center_learning_rate: Learning rate for updating the mean
of the search distribution. Default value is 1.0
stdev_learning_rate: Learning rate for updating the covariance
matrix of the search distribution.
The default value is `0.2 * (3 + log(n)) / sqrt(n)`
where `n` is the length of a solution.
scale_learning_rate: For SNES, there is a default standard
deviation learning rate value which is computed as
`0.2 * (3 + log(n)) / sqrt(n)` (where `n` is the solution
length).
If scale_learning_rate is True (which is the default),
then the effective learning rate for the standard deviation
becomes the provided `stdev_learning_rate` multiplied by this
default value. If `scale_learning_rate` is False, then the
effective standard deviation learning rate becomes
equal to the provided `stdev_learning_rate` value.
num_interactions: When given as an integer n,
it is ensured that a population has interacted with
the GymProblem's environment n times. If this target
has not been reached yet, then the population is declared
too small, and gets extended with more samples,
until n amount of interactions is reached.
When given as None, popsize is the only configuration
affecting the size of a population.
popsize_max: Having `num_interactions` set as an integer
might cause the effective population size jump to
unnecesarily large numbers. To prevent this,
one can set `popsize_max` to specify an upper
bound for the effective population size.
num_interactions: When given as an integer n,
it is ensured that a population has interacted with
the GymProblem's environment n times. If this target
has not been reached yet, then the population is declared
too small, and gets extended with more samples,
until n amount of interactions is reached.
When given as None, popsize is the only configuration
affecting the size of a population.
popsize_max: Having `num_interactions` set as an integer
might cause the effective population size jump to
unnecesarily large numbers. To prevent this,
one can set `popsize_max` to specify an upper
bound for the effective population size.
optimizer: The optimizer to be used while following the
estimated the gradients.
Can be given as None if a momentum-based optimizer
is not required.
Otherwise, can be given as a str containing the name
of the optimizer (e.g. 'adam', 'clipup');
or as an instance of evotorch.optimizers.TorchOptimizer
or evotorch.optimizers.ClipUp.
The default is None.
Note that, for ClipUp, the default maximum speed is set
as twice the given `center_learning_rate`.
This maximum speed can be configured by passing
`{"max_speed": ...}` to `optimizer_config`.
optimizer_config: Configuration which will be passed
to the optimizer as keyword arguments.
See `evotorch.optimizers` for details about
which optimizer accepts which keyword arguments.
ranking_method: Which ranking method will be used for
fitness shaping. See the documentation of
`evotorch.ranking.rank(...)` for details.
The default is 'nes'.
Can be given as None if no such ranking is required.
center_init: The initial center solution.
Can be left as None.
stdev_min: Minimum values for the standard deviation.
Expected as a 1-dimensional array to serve as a limiter
to the diagonals of the covariance matrix's square root.
stdev_max: Maximum values for the standard deviation.
Expected as a 1-dimensional array to serve as a limiter
to the diagonals of the covariance matrix's square root.
stdev_max_change: Maximum change allowed for when updating
the square roort of the covariance matrix.
obj_index: Index of the objective according to which the
gradient estimations will be done.
For single-objective problems, this can be left as None.
distributed: Whether or not the gradient computation will
be distributed. If `distributed` is given as False and
the problem is not parallelized, then everything will
be centralized (i.e. the entire computation will happen
in the main process).
If `distributed` is given as False, and the problem
is parallelized, then the population will be created
in the main process and then sent to remote workers
for parallelized evaluation, and then the remote fitnesses
will be collected by the main process again for computing
the search gradients.
If `distributed` is given as True, and the problem
is parallelized, then the search algorithm itself will
be distributed, in the sense that each remote actor will
generate its own population (such that the total population
size across all these actors becomes equal to `popsize`)
and will compute its own gradient, and then the main process
will collect these gradients, compute the averaged gradients
and update the main search distribution.
Non-distributed mode has the advantage of keeping the
population in the main process, which is good when one wishes
to do detailed monitoring during the evolutionary process,
but has the disadvantage of having to pass the solutions to
the remote actors and having to collect fitnesses, which
might result in increased interprocess communication traffic.
On the other hand, while it is not possible to monitor the
population in distributed mode, the distributed mode has the
advantage of significantly reducing the interprocess
communication traffic, since the only things communicated
with the remote actors are the search distributions (not the
solutions) and the gradients.
popsize_weighted_grad_avg: Only to be used in distributed mode.
(where being in distributed mode means `distributed` is given
as True). In distributed mode, each actor remotely samples
its own solution batches and computes its own gradients.
These gradients are then collected, and a final average
gradient is computed.
If `popsize_weighted_grad_avg` is True, then, while averaging
over the gradients, each gradient will have its own weight
that is computed according to how many solutions were sampled
by the actor that produced the gradient.
If `popsize_weighted_grad_avg` is False, then, there will not
be weighted averaging (or, each gradient will have equal
weight).
If `popsize_weighted_grad_avg` is None, then, the gradient
weights will be equal a value for `num_interactions` is given
(because `num_interactions` affects the number of solutions
according to the episode lengths, and popsize-weighting the
gradients could be misleading); and the gradient weights will
be weighted according to the sub-population (i.e. sub-batch)
sizes if `num_interactions` is left as None.
The default value for `popsize_weighted_grad_avg` is None.
When the distributed mode is disabled (i.e. when `distributed`
is False), then the argument `popsize_weighted_grad_avg` is
expected as None.
"""
if popsize is None:
popsize = int(4 + math.floor(3 * math.log(problem.solution_length)))
if center_learning_rate is None:
center_learning_rate = 1.0
def default_stdev_lr():
n = problem.solution_length
return 0.2 * (3 + math.log(n)) / math.sqrt(n)
if stdev_learning_rate is None:
stdev_learning_rate = default_stdev_lr()
else:
stdev_learning_rate = float(stdev_learning_rate)
if scale_learning_rate:
stdev_learning_rate *= default_stdev_lr()
super().__init__(
problem,
popsize=popsize,
center_learning_rate=center_learning_rate,
stdev_learning_rate=stdev_learning_rate,
stdev_init=stdev_init,
radius_init=radius_init,
popsize_max=popsize_max,
num_interactions=num_interactions,
optimizer=optimizer,
optimizer_config=optimizer_config,
ranking_method=ranking_method,
center_init=center_init,
stdev_min=stdev_min,
stdev_max=stdev_max,
stdev_max_change=stdev_max_change,
obj_index=obj_index,
distributed=distributed,
popsize_weighted_grad_avg=popsize_weighted_grad_avg,
)
DISTRIBUTION_TYPE (SeparableGaussian)
¶
Exponential Separable Multivariate Gaussian, as used by SNES
Source code in evotorch/algorithms/distributed/gaussian.py
class ExpSeparableGaussian(SeparableGaussian):
"""Exponential Separable Multivariate Gaussian, as used by SNES"""
MANDATORY_PARAMETERS = {"mu", "sigma"}
OPTIONAL_PARAMETERS = set()
PARAMETER_NDIMS = {"mu": 1, "sigma": 1}
def _compute_gradients(self, samples: torch.Tensor, weights: torch.Tensor, ranking_used: Optional[str]) -> dict:
if ranking_used != "nes":
weights = weights / torch.sum(torch.abs(weights))
scaled_noises = samples - self.mu
raw_noises = scaled_noises / self.sigma
mu_grad = total(dot(weights, scaled_noises))
sigma_grad = total(dot(weights, (raw_noises**2) - 1))
return {"mu": mu_grad, "sigma": sigma_grad}
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "ExpSeparableGaussian":
mu_grad = gradients["mu"]
sigma_grad = gradients["sigma"]
new_mu = self.mu + self._follow_gradient("mu", mu_grad, learning_rates=learning_rates, optimizers=optimizers)
new_sigma = self.sigma * torch.exp(
0.5 * self._follow_gradient("sigma", sigma_grad, learning_rates=learning_rates, optimizers=optimizers)
)
return self.modified_copy(mu=new_mu, sigma=new_sigma)
update_parameters(self, gradients, *, learning_rates=None, optimizers=None)
¶Do an update on the distribution by following the given gradients.
It is expected that the inheriting class has its own implementation for this method.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
gradients |
dict |
Gradients, as a dictionary, which will be used for computing the necessary updates. |
required |
learning_rates |
Optional[dict] |
A dictionary which contains learning rates for parameters that will be updated using a learning rate coefficient. |
None |
optimizers |
Optional[dict] |
A dictionary which contains optimizer objects for parameters that will be updated using an adaptive optimizer. |
None |
Returns:
| Type | Description |
|---|---|
ExpSeparableGaussian |
The updated copy of the distribution. |
Source code in evotorch/algorithms/distributed/gaussian.py
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "ExpSeparableGaussian":
mu_grad = gradients["mu"]
sigma_grad = gradients["sigma"]
new_mu = self.mu + self._follow_gradient("mu", mu_grad, learning_rates=learning_rates, optimizers=optimizers)
new_sigma = self.sigma * torch.exp(
0.5 * self._follow_gradient("sigma", sigma_grad, learning_rates=learning_rates, optimizers=optimizers)
)
return self.modified_copy(mu=new_mu, sigma=new_sigma)
__init__(self, problem, *, stdev_init=None, radius_init=None, popsize=None, center_learning_rate=None, stdev_learning_rate=None, scale_learning_rate=True, num_interactions=None, popsize_max=None, optimizer=None, optimizer_config=None, ranking_method='nes', center_init=None, stdev_min=None, stdev_max=None, stdev_max_change=None, obj_index=None, distributed=False, popsize_weighted_grad_avg=None)
special
¶
__init__(...): Initialize the SNES algorithm.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object which is being worked on. |
required |
stdev_init |
Union[float, Iterable[float], torch.Tensor] |
The initial standard deviation of the search
distribution, expressed as a scalar or as an array.
Determines the initial coverage area of the search
distribution.
If one wishes to configure the coverage area via the
argument |
None |
radius_init |
Union[float, Iterable[float], torch.Tensor] |
The initial radius of the search distribution,
expressed as a scalar.
Determines the initial coverage area of the search
distribution.
Here, "radius" is defined as the norm of the search
distribution.
If one wishes to configure the coverage area via the
argument |
None |
popsize |
Optional[int] |
Population size. Can be specified as an int,
or can be left as None to let the solver decide.
In the case of SNES, |
None |
center_learning_rate |
Optional[float] |
Learning rate for updating the mean of the search distribution. Default value is 1.0 |
None |
stdev_learning_rate |
Optional[float] |
Learning rate for updating the covariance
matrix of the search distribution.
The default value is |
None |
scale_learning_rate |
bool |
For SNES, there is a default standard
deviation learning rate value which is computed as
|
True |
num_interactions |
Optional[int] |
When given as an integer n, it is ensured that a population has interacted with the GymProblem's environment n times. If this target has not been reached yet, then the population is declared too small, and gets extended with more samples, until n amount of interactions is reached. When given as None, popsize is the only configuration affecting the size of a population. |
None |
popsize_max |
Optional[int] |
Having |
None |
num_interactions |
Optional[int] |
When given as an integer n, it is ensured that a population has interacted with the GymProblem's environment n times. If this target has not been reached yet, then the population is declared too small, and gets extended with more samples, until n amount of interactions is reached. When given as None, popsize is the only configuration affecting the size of a population. |
None |
popsize_max |
Optional[int] |
Having |
None |
optimizer |
The optimizer to be used while following the
estimated the gradients.
Can be given as None if a momentum-based optimizer
is not required.
Otherwise, can be given as a str containing the name
of the optimizer (e.g. 'adam', 'clipup');
or as an instance of evotorch.optimizers.TorchOptimizer
or evotorch.optimizers.ClipUp.
The default is None.
Note that, for ClipUp, the default maximum speed is set
as twice the given |
None |
|
optimizer_config |
Optional[dict] |
Configuration which will be passed
to the optimizer as keyword arguments.
See |
None |
ranking_method |
Optional[str] |
Which ranking method will be used for
fitness shaping. See the documentation of
|
'nes' |
center_init |
Union[float, Iterable[float], torch.Tensor] |
The initial center solution. Can be left as None. |
None |
stdev_min |
Union[float, Iterable[float], torch.Tensor] |
Minimum values for the standard deviation. Expected as a 1-dimensional array to serve as a limiter to the diagonals of the covariance matrix's square root. |
None |
stdev_max |
Union[float, Iterable[float], torch.Tensor] |
Maximum values for the standard deviation. Expected as a 1-dimensional array to serve as a limiter to the diagonals of the covariance matrix's square root. |
None |
stdev_max_change |
Union[float, Iterable[float], torch.Tensor] |
Maximum change allowed for when updating the square roort of the covariance matrix. |
None |
obj_index |
Optional[int] |
Index of the objective according to which the gradient estimations will be done. For single-objective problems, this can be left as None. |
None |
distributed |
bool |
Whether or not the gradient computation will
be distributed. If |
False |
popsize_weighted_grad_avg |
Optional[bool] |
Only to be used in distributed mode.
(where being in distributed mode means |
None |
Source code in evotorch/algorithms/distributed/gaussian.py
def __init__(
self,
problem: Problem,
*,
stdev_init: Optional[RealOrVector] = None,
radius_init: Optional[RealOrVector] = None,
popsize: Optional[int] = None,
center_learning_rate: Optional[float] = None,
stdev_learning_rate: Optional[float] = None,
scale_learning_rate: bool = True,
num_interactions: Optional[int] = None,
popsize_max: Optional[int] = None,
optimizer=None,
optimizer_config: Optional[dict] = None,
ranking_method: Optional[str] = "nes",
center_init: Optional[RealOrVector] = None,
stdev_min: Optional[RealOrVector] = None,
stdev_max: Optional[RealOrVector] = None,
stdev_max_change: Optional[RealOrVector] = None,
obj_index: Optional[int] = None,
distributed: bool = False,
popsize_weighted_grad_avg: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the SNES algorithm.
Args:
problem: The problem object which is being worked on.
stdev_init: The initial standard deviation of the search
distribution, expressed as a scalar or as an array.
Determines the initial coverage area of the search
distribution.
If one wishes to configure the coverage area via the
argument `radius_init` instead, then `stdev_init` is expected
as None.
radius_init: The initial radius of the search distribution,
expressed as a scalar.
Determines the initial coverage area of the search
distribution.
Here, "radius" is defined as the norm of the search
distribution.
If one wishes to configure the coverage area via the
argument `stdev_init` instead, then `radius_init` is expected
as None.
popsize: Population size. Can be specified as an int,
or can be left as None to let the solver decide.
In the case of SNES, `popsize` can be left as None,
in which case the default `popsize` will be computed
as `4 + floor(3 * log(n))` where `n` is the length
of a solution.
center_learning_rate: Learning rate for updating the mean
of the search distribution. Default value is 1.0
stdev_learning_rate: Learning rate for updating the covariance
matrix of the search distribution.
The default value is `0.2 * (3 + log(n)) / sqrt(n)`
where `n` is the length of a solution.
scale_learning_rate: For SNES, there is a default standard
deviation learning rate value which is computed as
`0.2 * (3 + log(n)) / sqrt(n)` (where `n` is the solution
length).
If scale_learning_rate is True (which is the default),
then the effective learning rate for the standard deviation
becomes the provided `stdev_learning_rate` multiplied by this
default value. If `scale_learning_rate` is False, then the
effective standard deviation learning rate becomes
equal to the provided `stdev_learning_rate` value.
num_interactions: When given as an integer n,
it is ensured that a population has interacted with
the GymProblem's environment n times. If this target
has not been reached yet, then the population is declared
too small, and gets extended with more samples,
until n amount of interactions is reached.
When given as None, popsize is the only configuration
affecting the size of a population.
popsize_max: Having `num_interactions` set as an integer
might cause the effective population size jump to
unnecesarily large numbers. To prevent this,
one can set `popsize_max` to specify an upper
bound for the effective population size.
num_interactions: When given as an integer n,
it is ensured that a population has interacted with
the GymProblem's environment n times. If this target
has not been reached yet, then the population is declared
too small, and gets extended with more samples,
until n amount of interactions is reached.
When given as None, popsize is the only configuration
affecting the size of a population.
popsize_max: Having `num_interactions` set as an integer
might cause the effective population size jump to
unnecesarily large numbers. To prevent this,
one can set `popsize_max` to specify an upper
bound for the effective population size.
optimizer: The optimizer to be used while following the
estimated the gradients.
Can be given as None if a momentum-based optimizer
is not required.
Otherwise, can be given as a str containing the name
of the optimizer (e.g. 'adam', 'clipup');
or as an instance of evotorch.optimizers.TorchOptimizer
or evotorch.optimizers.ClipUp.
The default is None.
Note that, for ClipUp, the default maximum speed is set
as twice the given `center_learning_rate`.
This maximum speed can be configured by passing
`{"max_speed": ...}` to `optimizer_config`.
optimizer_config: Configuration which will be passed
to the optimizer as keyword arguments.
See `evotorch.optimizers` for details about
which optimizer accepts which keyword arguments.
ranking_method: Which ranking method will be used for
fitness shaping. See the documentation of
`evotorch.ranking.rank(...)` for details.
The default is 'nes'.
Can be given as None if no such ranking is required.
center_init: The initial center solution.
Can be left as None.
stdev_min: Minimum values for the standard deviation.
Expected as a 1-dimensional array to serve as a limiter
to the diagonals of the covariance matrix's square root.
stdev_max: Maximum values for the standard deviation.
Expected as a 1-dimensional array to serve as a limiter
to the diagonals of the covariance matrix's square root.
stdev_max_change: Maximum change allowed for when updating
the square roort of the covariance matrix.
obj_index: Index of the objective according to which the
gradient estimations will be done.
For single-objective problems, this can be left as None.
distributed: Whether or not the gradient computation will
be distributed. If `distributed` is given as False and
the problem is not parallelized, then everything will
be centralized (i.e. the entire computation will happen
in the main process).
If `distributed` is given as False, and the problem
is parallelized, then the population will be created
in the main process and then sent to remote workers
for parallelized evaluation, and then the remote fitnesses
will be collected by the main process again for computing
the search gradients.
If `distributed` is given as True, and the problem
is parallelized, then the search algorithm itself will
be distributed, in the sense that each remote actor will
generate its own population (such that the total population
size across all these actors becomes equal to `popsize`)
and will compute its own gradient, and then the main process
will collect these gradients, compute the averaged gradients
and update the main search distribution.
Non-distributed mode has the advantage of keeping the
population in the main process, which is good when one wishes
to do detailed monitoring during the evolutionary process,
but has the disadvantage of having to pass the solutions to
the remote actors and having to collect fitnesses, which
might result in increased interprocess communication traffic.
On the other hand, while it is not possible to monitor the
population in distributed mode, the distributed mode has the
advantage of significantly reducing the interprocess
communication traffic, since the only things communicated
with the remote actors are the search distributions (not the
solutions) and the gradients.
popsize_weighted_grad_avg: Only to be used in distributed mode.
(where being in distributed mode means `distributed` is given
as True). In distributed mode, each actor remotely samples
its own solution batches and computes its own gradients.
These gradients are then collected, and a final average
gradient is computed.
If `popsize_weighted_grad_avg` is True, then, while averaging
over the gradients, each gradient will have its own weight
that is computed according to how many solutions were sampled
by the actor that produced the gradient.
If `popsize_weighted_grad_avg` is False, then, there will not
be weighted averaging (or, each gradient will have equal
weight).
If `popsize_weighted_grad_avg` is None, then, the gradient
weights will be equal a value for `num_interactions` is given
(because `num_interactions` affects the number of solutions
according to the episode lengths, and popsize-weighting the
gradients could be misleading); and the gradient weights will
be weighted according to the sub-population (i.e. sub-batch)
sizes if `num_interactions` is left as None.
The default value for `popsize_weighted_grad_avg` is None.
When the distributed mode is disabled (i.e. when `distributed`
is False), then the argument `popsize_weighted_grad_avg` is
expected as None.
"""
if popsize is None:
popsize = int(4 + math.floor(3 * math.log(problem.solution_length)))
if center_learning_rate is None:
center_learning_rate = 1.0
def default_stdev_lr():
n = problem.solution_length
return 0.2 * (3 + math.log(n)) / math.sqrt(n)
if stdev_learning_rate is None:
stdev_learning_rate = default_stdev_lr()
else:
stdev_learning_rate = float(stdev_learning_rate)
if scale_learning_rate:
stdev_learning_rate *= default_stdev_lr()
super().__init__(
problem,
popsize=popsize,
center_learning_rate=center_learning_rate,
stdev_learning_rate=stdev_learning_rate,
stdev_init=stdev_init,
radius_init=radius_init,
popsize_max=popsize_max,
num_interactions=num_interactions,
optimizer=optimizer,
optimizer_config=optimizer_config,
ranking_method=ranking_method,
center_init=center_init,
stdev_min=stdev_min,
stdev_max=stdev_max,
stdev_max_change=stdev_max_change,
obj_index=obj_index,
distributed=distributed,
popsize_weighted_grad_avg=popsize_weighted_grad_avg,
)
XNES (GaussianSearchAlgorithm)
¶
Inspired by the implementation at: http://schaul.site44.com/code/xnes.py https://github.com/pybrain/pybrain/blob/master/pybrain/optimization/distributionbased/xnes.py
Reference
Glasmachers, Tobias, et al. Exponential natural evolution strategies. Proceedings of the 12th annual conference on Genetic and evolutionary computation (GECCO 2010).
Source code in evotorch/algorithms/distributed/gaussian.py
class XNES(GaussianSearchAlgorithm):
"""
XNES: Exponential Natural Evolution Strategies
Inspired by the implementation at:
http://schaul.site44.com/code/xnes.py
https://github.com/pybrain/pybrain/blob/master/pybrain/optimization/distributionbased/xnes.py
Reference:
Glasmachers, Tobias, et al.
Exponential natural evolution strategies.
Proceedings of the 12th annual conference on Genetic and evolutionary
computation (GECCO 2010).
"""
DISTRIBUTION_TYPE = ExpGaussian
DISTRIBUTION_PARAMS = None
def __init__(
self,
problem: Problem,
*,
stdev_init: Optional[RealOrVector] = None,
radius_init: Optional[RealOrVector] = None,
popsize: Optional[int] = None,
center_learning_rate: Optional[float] = None,
stdev_learning_rate: Optional[float] = None,
scale_learning_rate: bool = True,
num_interactions: Optional[int] = None,
popsize_max: Optional[int] = None,
optimizer=None,
optimizer_config: Optional[dict] = None,
ranking_method: Optional[str] = "nes",
center_init: Optional[RealOrVector] = None,
obj_index: Optional[int] = None,
distributed: bool = False,
popsize_weighted_grad_avg: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the XNES algorithm.
Args:
problem: The problem object which is being worked on.
stdev_init: The initial standard deviation of the search
distribution, expressed as a scalar or as an array.
Determines the initial coverage area of the search
distribution.
If one wishes to configure the coverage area via the
argument `radius_init` instead, then `stdev_init` is expected
as None.
radius_init: The initial radius of the search distribution,
expressed as a scalar.
Determines the initial coverage area of the search
distribution.
Here, "radius" is defined as the norm of the search
distribution.
If one wishes to configure the coverage area via the
argument `stdev_init` instead, then `radius_init` is expected
as None.
popsize: Population size. Can be specified as an int,
or can be left as None to let the solver decide.
In the case of SNES, `popsize` can be left as None,
in which case the default `popsize` will be computed
as `4 + floor(3 * log(n))` where `n` is the length
of a solution.
center_learning_rate: Learning rate for updating the mean
of the search distribution. Default value is 1.0
stdev_learning_rate: Learning rate for updating the covariance
matrix of the search distribution.
The default value is `0.6 * (3 + log(n)) / (n * sqrt(n))`
where `n` is the length of a solution.
scale_learning_rate: For SNES, there is a default standard
deviation learning rate value which is computed as
`0.6 * (3 + log(n)) / (n * sqrt(n))` (where `n` is the solution
length).
If scale_learning_rate is True (which is the default),
then the effective learning rate for the standard deviation
becomes the provided `stdev_learning_rate` multiplied by this
default value. If `scale_learning_rate` is False, then the
effective standard deviation learning rate becomes
equal to the provided `stdev_learning_rate` value.
num_interactions: When given as an integer n,
it is ensured that a population has interacted with
the GymProblem's environment n times. If this target
has not been reached yet, then the population is declared
too small, and gets extended with more samples,
until n amount of interactions is reached.
When given as None, popsize is the only configuration
affecting the size of a population.
popsize_max: Having `num_interactions` set as an integer
might cause the effective population size jump to
unnecesarily large numbers. To prevent this,
one can set `popsize_max` to specify an upper
bound for the effective population size.
num_interactions: When given as an integer n,
it is ensured that a population has interacted with
the GymProblem's environment n times. If this target
has not been reached yet, then the population is declared
too small, and gets extended with more samples,
until n amount of interactions is reached.
When given as None, popsize is the only configuration
affecting the size of a population.
optimizer: The optimizer to be used while following the
estimated the gradients.
Can be given as None if a momentum-based optimizer
is not required.
Otherwise, can be given as a str containing the name
of the optimizer (e.g. 'adam', 'clipup');
or as an instance of evotorch.optimizers.TorchOptimizer
or evotorch.optimizers.ClipUp.
The default is None.
Note that, for ClipUp, the default maximum speed is set
as twice the given `center_learning_rate`.
This maximum speed can be configured by passing
`{"max_speed": ...}` to `optimizer_config`.
optimizer_config: Configuration which will be passed
to the optimizer as keyword arguments.
See `evotorch.optimizers` for details about
which optimizer accepts which keyword arguments.
ranking_method: Which ranking method will be used for
fitness shaping. See the documentation of
`evotorch.ranking.rank(...)` for details.
The default is 'nes'.
Can be given as None if no such ranking is required.
center_init: The initial center solution.
Can be left as None.
obj_index: Index of the objective according to which the
gradient estimations will be done.
For single-objective problems, this can be left as None.
distributed: Whether or not the gradient computation will
be distributed. If `distributed` is given as False and
the problem is not parallelized, then everything will
be centralized (i.e. the entire computation will happen
in the main process).
If `distributed` is given as False, and the problem
is parallelized, then the population will be created
in the main process and then sent to remote workers
for parallelized evaluation, and then the remote fitnesses
will be collected by the main process again for computing
the search gradients.
If `distributed` is given as True, and the problem
is parallelized, then the search algorithm itself will
be distributed, in the sense that each remote actor will
generate its own population (such that the total population
size across all these actors becomes equal to `popsize`)
and will compute its own gradient, and then the main process
will collect these gradients, compute the averaged gradients
and update the main search distribution.
Non-distributed mode has the advantage of keeping the
population in the main process, which is good when one wishes
to do detailed monitoring during the evolutionary process,
but has the disadvantage of having to pass the solutions to
the remote actors and having to collect fitnesses, which
might result in increased interprocess communication traffic.
On the other hand, while it is not possible to monitor the
population in distributed mode, the distributed mode has the
advantage of significantly reducing the interprocess
communication traffic, since the only things communicated
with the remote actors are the search distributions (not the
solutions) and the gradients.
popsize_weighted_grad_avg: Only to be used in distributed mode.
(where being in distributed mode means `distributed` is given
as True). In distributed mode, each actor remotely samples
its own solution batches and computes its own gradients.
These gradients are then collected, and a final average
gradient is computed.
If `popsize_weighted_grad_avg` is True, then, while averaging
over the gradients, each gradient will have its own weight
that is computed according to how many solutions were sampled
by the actor that produced the gradient.
If `popsize_weighted_grad_avg` is False, then, there will not
be weighted averaging (or, each gradient will have equal
weight).
If `popsize_weighted_grad_avg` is None, then, the gradient
weights will be equal a value for `num_interactions` is given
(because `num_interactions` affects the number of solutions
according to the episode lengths, and popsize-weighting the
gradients could be misleading); and the gradient weights will
be weighted according to the sub-population (i.e. sub-batch)
sizes if `num_interactions` is left as None.
The default value for `popsize_weighted_grad_avg` is None.
When the distributed mode is disabled (i.e. when `distributed`
is False), then the argument `popsize_weighted_grad_avg` is
expected as None.
"""
if popsize is None:
popsize = int(4 + math.floor(3 * math.log(problem.solution_length)))
if center_learning_rate is None:
center_learning_rate = 1.0
def default_stdev_lr():
n = problem.solution_length
return 0.6 * (3 + math.log(n)) / (n * math.sqrt(n))
if stdev_learning_rate is None:
stdev_learning_rate = default_stdev_lr()
else:
stdev_learning_rate = float(stdev_learning_rate)
if scale_learning_rate:
stdev_learning_rate *= default_stdev_lr()
super().__init__(
problem,
popsize=popsize,
center_learning_rate=center_learning_rate,
stdev_learning_rate=stdev_learning_rate,
stdev_init=stdev_init,
radius_init=radius_init,
popsize_max=popsize_max,
num_interactions=num_interactions,
optimizer=optimizer,
optimizer_config=optimizer_config,
ranking_method=ranking_method,
center_init=center_init,
stdev_min=None,
stdev_max=None,
stdev_max_change=None,
obj_index=obj_index,
distributed=distributed,
popsize_weighted_grad_avg=popsize_weighted_grad_avg,
)
DISTRIBUTION_TYPE (Distribution)
¶
Exponential Multivariate Gaussian, as used by XNES
Source code in evotorch/algorithms/distributed/gaussian.py
class ExpGaussian(Distribution):
"""Exponential Multivariate Gaussian, as used by XNES"""
# Corresponding to mu and A in symbols used in xNES paper
MANDATORY_PARAMETERS = {"mu", "sigma"}
# Inverse of sigma, numerically more stable to track this independently to sigma
OPTIONAL_PARAMETERS = {"sigma_inv"}
PARAMETER_NDIMS = {"mu": 1, "sigma": 2, "sigma_inv": 2}
def __init__(
self,
parameters: dict,
*,
solution_length: Optional[int] = None,
device: Optional[Device] = None,
dtype: Optional[DType] = None,
):
[mu_length] = parameters["mu"].shape
# Make sigma 2D
if len(parameters["sigma"].shape) == 1:
parameters["sigma"] = torch.diag(parameters["sigma"])
# Automatically generate sigma_inv if not provided
if "sigma_inv" not in parameters:
parameters["sigma_inv"] = torch.inverse(parameters["sigma"])
[sigma_length, _] = parameters["sigma"].shape
if solution_length is None:
solution_length = mu_length
else:
if solution_length != mu_length:
raise ValueError(
f"The argument `solution_length` does not match the length of `mu` provided in `parameters`."
f" solution_length={solution_length},"
f' parameters["mu"]={mu_length}.'
)
if mu_length != sigma_length:
raise ValueError(
f"The tensors `mu` and `sigma` provided within `parameters` have mismatching lengths."
f' parameters["mu"]={mu_length},'
f' parameters["sigma"]={sigma_length}.'
)
super().__init__(
solution_length=solution_length,
parameters=parameters,
device=device,
dtype=dtype,
)
# Make identity matrix as this is used throughout in gradient computation
self.eye = self.make_I(solution_length)
@property
def mu(self) -> torch.Tensor:
"""Getter for mu
Returns:
mu (torch.Tensor): The center of the search distribution
"""
return self.parameters["mu"]
@mu.setter
def mu(self, new_mu: Iterable):
"""Setter for mu
Args:
new_mu (torch.Tensor): The new value of mu
"""
self.parameters["mu"] = torch.as_tensor(new_mu, dtype=self.dtype, device=self.device)
@property
def cov(self) -> torch.Tensor:
"""The covariance matrix A^T A"""
return self.sigma.transpose(0, 1) @ self.sigma
@property
def sigma(self) -> torch.Tensor:
"""Getter for sigma
Returns:
sigma (torch.Tensor): The square root of the covariance matrix
"""
return self.parameters["sigma"]
@property
def sigma_inv(self) -> torch.Tensor:
"""Getter for sigma_inv
Returns:
sigma_inv (torch.Tensor): The inverse square root of the covariance matrix
"""
if "sigma_inv" in self.parameters:
return self.parameters["sigma_inv"]
else:
return torch.inverse(self.parameters["sigma"])
@property
def A(self) -> torch.Tensor:
"""Alias for self.sigma, for notational consistency with paper"""
return self.sigma
@property
def A_inv(self) -> torch.Tensor:
"""Alias for self.sigma_inv, for notational consistency with paper"""
return self.sigma_inv
@sigma.setter
def sigma(self, new_sigma: Iterable):
"""Setter for sigma
Args:
new_sigma (torch.Tensor): The new value of sigma, the square root of the covariance matrix
"""
self.parameters["sigma"] = torch.as_tensor(new_sigma, dtype=self.dtype, device=self.device)
def to_global_coordinates(self, local_coordinates: torch.Tensor) -> torch.Tensor:
"""Map samples from local coordinate space N(0, I_d) to global coordinate space N(mu, A^T A)
This function is the inverse of to_local_coordinates
Args:
local_coordinates (torch.Tensor): The local coordinates sampled from N(0, I_d)
Returns:
global_coordinates (torch.Tensor): The global coordinates sampled from N(mu, A^T A)
"""
# Global samples are constructed as x = mu + A z where z is local coordinate
# We use transpose here to simplify the batched application of A
return self.mu.unsqueeze(0) + (self.A @ local_coordinates.T).T
def to_local_coordinates(self, global_coordinates: torch.Tensor) -> torch.Tensor:
"""Map samples from global coordinate space N(mu, A^T A) to local coordinate space N(0, I_d)
This function is the inverse of to_global_coordinates
Args:
global_coordinates (torch.Tensor): The global coordinates sampled from N(mu, A^T A)
Returns:
local_coordinates (torch.Tensor): The local coordinates sampled from N(0, I_d)
"""
# Global samples are constructed as x = mu + A z where z is local coordinate
# Therefore, we can recover z according to z = A_inv (x - mu)
return (self.A_inv @ (global_coordinates - self.mu.unsqueeze(0)).T).T
def _fill(self, out: torch.Tensor, *, generator: Optional[torch.Generator] = None):
"""Fill a tensor with samples from N(mu, A^T A)
Args:
out (torch.Tensor): The tensor to fill
generator (Optional[torch.Generator]): A generator to use to generate random values
"""
# Fill with local coordinates from N(0, I_d)
self.make_gaussian(out=out, generator=generator)
# Map local coordinates to global coordinate system
out[:] = self.to_global_coordinates(out)
def _compute_gradients(self, samples: torch.Tensor, weights: torch.Tensor, ranking_used: Optional[str]) -> dict:
"""Compute the gradients with respect to a given set of samples and weights
Args:
samples (torch.Tensor): Samples drawn from N(mu, A^T A), ideally using self._fill
weights (torch.Tensor): Weights e.g. fitnesses or utilities assigned to samples
ranking_used (optional[str]): The ranking method used to compute weights
Returns:
grads (dict): A dictionary containing the approximated natural gradient on d and M
"""
# Compute the local coordinates
local_coordinates = self.to_local_coordinates(samples)
# Make sure that the weights (utilities) are 0-centered
# (Otherwise the formulations would have to consider a bias term)
if ranking_used not in ("centered", "normalized"):
weights = weights - torch.mean(weights)
d_grad = total(dot(weights, local_coordinates))
local_coordinates_outer = local_coordinates.unsqueeze(1) * local_coordinates.unsqueeze(2)
M_grad = torch.sum(
weights.unsqueeze(-1).unsqueeze(-1) * (local_coordinates_outer - self.eye.unsqueeze(0)), dim=0
)
return {
"d": d_grad,
"M": M_grad,
}
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "ExpGaussian":
d_grad = gradients["d"]
M_grad = gradients["M"]
if "d" not in learning_rates:
learning_rates["d"] = learning_rates["mu"]
if "M" not in learning_rates:
learning_rates["M"] = learning_rates["sigma"]
# Follow gradients for d, and M
update_d = self._follow_gradient("d", d_grad, learning_rates=learning_rates, optimizers=optimizers)
update_M = self._follow_gradient("M", M_grad, learning_rates=learning_rates, optimizers=optimizers)
# Fold into parameters mu, A and A inv
new_mu = self.mu + torch.mv(self.A, update_d)
new_A = self.A @ torch.matrix_exp(0.5 * update_M)
new_A_inv = torch.matrix_exp(-0.5 * update_M) @ self.A_inv
# Return modified distribution
return self.modified_copy(mu=new_mu, sigma=new_A, sigma_inv=new_A_inv)
A: Tensor
property
readonly
¶Alias for self.sigma, for notational consistency with paper
A_inv: Tensor
property
readonly
¶Alias for self.sigma_inv, for notational consistency with paper
cov: Tensor
property
readonly
¶The covariance matrix A^T A
mu: Tensor
property
writable
¶Getter for mu
Returns:
| Type | Description |
|---|---|
mu (torch.Tensor) |
The center of the search distribution |
sigma: Tensor
property
writable
¶Getter for sigma
Returns:
| Type | Description |
|---|---|
sigma (torch.Tensor) |
The square root of the covariance matrix |
sigma_inv: Tensor
property
readonly
¶Getter for sigma_inv
Returns:
| Type | Description |
|---|---|
sigma_inv (torch.Tensor) |
The inverse square root of the covariance matrix |
to_global_coordinates(self, local_coordinates)
¶Map samples from local coordinate space N(0, I_d) to global coordinate space N(mu, A^T A) This function is the inverse of to_local_coordinates
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
local_coordinates |
torch.Tensor |
The local coordinates sampled from N(0, I_d) |
required |
Returns:
| Type | Description |
|---|---|
global_coordinates (torch.Tensor) |
The global coordinates sampled from N(mu, A^T A) |
Source code in evotorch/algorithms/distributed/gaussian.py
def to_global_coordinates(self, local_coordinates: torch.Tensor) -> torch.Tensor:
"""Map samples from local coordinate space N(0, I_d) to global coordinate space N(mu, A^T A)
This function is the inverse of to_local_coordinates
Args:
local_coordinates (torch.Tensor): The local coordinates sampled from N(0, I_d)
Returns:
global_coordinates (torch.Tensor): The global coordinates sampled from N(mu, A^T A)
"""
# Global samples are constructed as x = mu + A z where z is local coordinate
# We use transpose here to simplify the batched application of A
return self.mu.unsqueeze(0) + (self.A @ local_coordinates.T).T
to_local_coordinates(self, global_coordinates)
¶Map samples from global coordinate space N(mu, A^T A) to local coordinate space N(0, I_d) This function is the inverse of to_global_coordinates
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
global_coordinates |
torch.Tensor |
The global coordinates sampled from N(mu, A^T A) |
required |
Returns:
| Type | Description |
|---|---|
local_coordinates (torch.Tensor) |
The local coordinates sampled from N(0, I_d) |
Source code in evotorch/algorithms/distributed/gaussian.py
def to_local_coordinates(self, global_coordinates: torch.Tensor) -> torch.Tensor:
"""Map samples from global coordinate space N(mu, A^T A) to local coordinate space N(0, I_d)
This function is the inverse of to_global_coordinates
Args:
global_coordinates (torch.Tensor): The global coordinates sampled from N(mu, A^T A)
Returns:
local_coordinates (torch.Tensor): The local coordinates sampled from N(0, I_d)
"""
# Global samples are constructed as x = mu + A z where z is local coordinate
# Therefore, we can recover z according to z = A_inv (x - mu)
return (self.A_inv @ (global_coordinates - self.mu.unsqueeze(0)).T).T
update_parameters(self, gradients, *, learning_rates=None, optimizers=None)
¶Do an update on the distribution by following the given gradients.
It is expected that the inheriting class has its own implementation for this method.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
gradients |
dict |
Gradients, as a dictionary, which will be used for computing the necessary updates. |
required |
learning_rates |
Optional[dict] |
A dictionary which contains learning rates for parameters that will be updated using a learning rate coefficient. |
None |
optimizers |
Optional[dict] |
A dictionary which contains optimizer objects for parameters that will be updated using an adaptive optimizer. |
None |
Returns:
| Type | Description |
|---|---|
ExpGaussian |
The updated copy of the distribution. |
Source code in evotorch/algorithms/distributed/gaussian.py
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "ExpGaussian":
d_grad = gradients["d"]
M_grad = gradients["M"]
if "d" not in learning_rates:
learning_rates["d"] = learning_rates["mu"]
if "M" not in learning_rates:
learning_rates["M"] = learning_rates["sigma"]
# Follow gradients for d, and M
update_d = self._follow_gradient("d", d_grad, learning_rates=learning_rates, optimizers=optimizers)
update_M = self._follow_gradient("M", M_grad, learning_rates=learning_rates, optimizers=optimizers)
# Fold into parameters mu, A and A inv
new_mu = self.mu + torch.mv(self.A, update_d)
new_A = self.A @ torch.matrix_exp(0.5 * update_M)
new_A_inv = torch.matrix_exp(-0.5 * update_M) @ self.A_inv
# Return modified distribution
return self.modified_copy(mu=new_mu, sigma=new_A, sigma_inv=new_A_inv)
__init__(self, problem, *, stdev_init=None, radius_init=None, popsize=None, center_learning_rate=None, stdev_learning_rate=None, scale_learning_rate=True, num_interactions=None, popsize_max=None, optimizer=None, optimizer_config=None, ranking_method='nes', center_init=None, obj_index=None, distributed=False, popsize_weighted_grad_avg=None)
special
¶
__init__(...): Initialize the XNES algorithm.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object which is being worked on. |
required |
stdev_init |
Union[float, Iterable[float], torch.Tensor] |
The initial standard deviation of the search
distribution, expressed as a scalar or as an array.
Determines the initial coverage area of the search
distribution.
If one wishes to configure the coverage area via the
argument |
None |
radius_init |
Union[float, Iterable[float], torch.Tensor] |
The initial radius of the search distribution,
expressed as a scalar.
Determines the initial coverage area of the search
distribution.
Here, "radius" is defined as the norm of the search
distribution.
If one wishes to configure the coverage area via the
argument |
None |
popsize |
Optional[int] |
Population size. Can be specified as an int,
or can be left as None to let the solver decide.
In the case of SNES, |
None |
center_learning_rate |
Optional[float] |
Learning rate for updating the mean of the search distribution. Default value is 1.0 |
None |
stdev_learning_rate |
Optional[float] |
Learning rate for updating the covariance
matrix of the search distribution.
The default value is |
None |
scale_learning_rate |
bool |
For SNES, there is a default standard
deviation learning rate value which is computed as
|
True |
num_interactions |
Optional[int] |
When given as an integer n, it is ensured that a population has interacted with the GymProblem's environment n times. If this target has not been reached yet, then the population is declared too small, and gets extended with more samples, until n amount of interactions is reached. When given as None, popsize is the only configuration affecting the size of a population. |
None |
popsize_max |
Optional[int] |
Having |
None |
num_interactions |
Optional[int] |
When given as an integer n, it is ensured that a population has interacted with the GymProblem's environment n times. If this target has not been reached yet, then the population is declared too small, and gets extended with more samples, until n amount of interactions is reached. When given as None, popsize is the only configuration affecting the size of a population. |
None |
optimizer |
The optimizer to be used while following the
estimated the gradients.
Can be given as None if a momentum-based optimizer
is not required.
Otherwise, can be given as a str containing the name
of the optimizer (e.g. 'adam', 'clipup');
or as an instance of evotorch.optimizers.TorchOptimizer
or evotorch.optimizers.ClipUp.
The default is None.
Note that, for ClipUp, the default maximum speed is set
as twice the given |
None |
|
optimizer_config |
Optional[dict] |
Configuration which will be passed
to the optimizer as keyword arguments.
See |
None |
ranking_method |
Optional[str] |
Which ranking method will be used for
fitness shaping. See the documentation of
|
'nes' |
center_init |
Union[float, Iterable[float], torch.Tensor] |
The initial center solution. Can be left as None. |
None |
obj_index |
Optional[int] |
Index of the objective according to which the gradient estimations will be done. For single-objective problems, this can be left as None. |
None |
distributed |
bool |
Whether or not the gradient computation will
be distributed. If |
False |
popsize_weighted_grad_avg |
Optional[bool] |
Only to be used in distributed mode.
(where being in distributed mode means |
None |
Source code in evotorch/algorithms/distributed/gaussian.py
def __init__(
self,
problem: Problem,
*,
stdev_init: Optional[RealOrVector] = None,
radius_init: Optional[RealOrVector] = None,
popsize: Optional[int] = None,
center_learning_rate: Optional[float] = None,
stdev_learning_rate: Optional[float] = None,
scale_learning_rate: bool = True,
num_interactions: Optional[int] = None,
popsize_max: Optional[int] = None,
optimizer=None,
optimizer_config: Optional[dict] = None,
ranking_method: Optional[str] = "nes",
center_init: Optional[RealOrVector] = None,
obj_index: Optional[int] = None,
distributed: bool = False,
popsize_weighted_grad_avg: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the XNES algorithm.
Args:
problem: The problem object which is being worked on.
stdev_init: The initial standard deviation of the search
distribution, expressed as a scalar or as an array.
Determines the initial coverage area of the search
distribution.
If one wishes to configure the coverage area via the
argument `radius_init` instead, then `stdev_init` is expected
as None.
radius_init: The initial radius of the search distribution,
expressed as a scalar.
Determines the initial coverage area of the search
distribution.
Here, "radius" is defined as the norm of the search
distribution.
If one wishes to configure the coverage area via the
argument `stdev_init` instead, then `radius_init` is expected
as None.
popsize: Population size. Can be specified as an int,
or can be left as None to let the solver decide.
In the case of SNES, `popsize` can be left as None,
in which case the default `popsize` will be computed
as `4 + floor(3 * log(n))` where `n` is the length
of a solution.
center_learning_rate: Learning rate for updating the mean
of the search distribution. Default value is 1.0
stdev_learning_rate: Learning rate for updating the covariance
matrix of the search distribution.
The default value is `0.6 * (3 + log(n)) / (n * sqrt(n))`
where `n` is the length of a solution.
scale_learning_rate: For SNES, there is a default standard
deviation learning rate value which is computed as
`0.6 * (3 + log(n)) / (n * sqrt(n))` (where `n` is the solution
length).
If scale_learning_rate is True (which is the default),
then the effective learning rate for the standard deviation
becomes the provided `stdev_learning_rate` multiplied by this
default value. If `scale_learning_rate` is False, then the
effective standard deviation learning rate becomes
equal to the provided `stdev_learning_rate` value.
num_interactions: When given as an integer n,
it is ensured that a population has interacted with
the GymProblem's environment n times. If this target
has not been reached yet, then the population is declared
too small, and gets extended with more samples,
until n amount of interactions is reached.
When given as None, popsize is the only configuration
affecting the size of a population.
popsize_max: Having `num_interactions` set as an integer
might cause the effective population size jump to
unnecesarily large numbers. To prevent this,
one can set `popsize_max` to specify an upper
bound for the effective population size.
num_interactions: When given as an integer n,
it is ensured that a population has interacted with
the GymProblem's environment n times. If this target
has not been reached yet, then the population is declared
too small, and gets extended with more samples,
until n amount of interactions is reached.
When given as None, popsize is the only configuration
affecting the size of a population.
optimizer: The optimizer to be used while following the
estimated the gradients.
Can be given as None if a momentum-based optimizer
is not required.
Otherwise, can be given as a str containing the name
of the optimizer (e.g. 'adam', 'clipup');
or as an instance of evotorch.optimizers.TorchOptimizer
or evotorch.optimizers.ClipUp.
The default is None.
Note that, for ClipUp, the default maximum speed is set
as twice the given `center_learning_rate`.
This maximum speed can be configured by passing
`{"max_speed": ...}` to `optimizer_config`.
optimizer_config: Configuration which will be passed
to the optimizer as keyword arguments.
See `evotorch.optimizers` for details about
which optimizer accepts which keyword arguments.
ranking_method: Which ranking method will be used for
fitness shaping. See the documentation of
`evotorch.ranking.rank(...)` for details.
The default is 'nes'.
Can be given as None if no such ranking is required.
center_init: The initial center solution.
Can be left as None.
obj_index: Index of the objective according to which the
gradient estimations will be done.
For single-objective problems, this can be left as None.
distributed: Whether or not the gradient computation will
be distributed. If `distributed` is given as False and
the problem is not parallelized, then everything will
be centralized (i.e. the entire computation will happen
in the main process).
If `distributed` is given as False, and the problem
is parallelized, then the population will be created
in the main process and then sent to remote workers
for parallelized evaluation, and then the remote fitnesses
will be collected by the main process again for computing
the search gradients.
If `distributed` is given as True, and the problem
is parallelized, then the search algorithm itself will
be distributed, in the sense that each remote actor will
generate its own population (such that the total population
size across all these actors becomes equal to `popsize`)
and will compute its own gradient, and then the main process
will collect these gradients, compute the averaged gradients
and update the main search distribution.
Non-distributed mode has the advantage of keeping the
population in the main process, which is good when one wishes
to do detailed monitoring during the evolutionary process,
but has the disadvantage of having to pass the solutions to
the remote actors and having to collect fitnesses, which
might result in increased interprocess communication traffic.
On the other hand, while it is not possible to monitor the
population in distributed mode, the distributed mode has the
advantage of significantly reducing the interprocess
communication traffic, since the only things communicated
with the remote actors are the search distributions (not the
solutions) and the gradients.
popsize_weighted_grad_avg: Only to be used in distributed mode.
(where being in distributed mode means `distributed` is given
as True). In distributed mode, each actor remotely samples
its own solution batches and computes its own gradients.
These gradients are then collected, and a final average
gradient is computed.
If `popsize_weighted_grad_avg` is True, then, while averaging
over the gradients, each gradient will have its own weight
that is computed according to how many solutions were sampled
by the actor that produced the gradient.
If `popsize_weighted_grad_avg` is False, then, there will not
be weighted averaging (or, each gradient will have equal
weight).
If `popsize_weighted_grad_avg` is None, then, the gradient
weights will be equal a value for `num_interactions` is given
(because `num_interactions` affects the number of solutions
according to the episode lengths, and popsize-weighting the
gradients could be misleading); and the gradient weights will
be weighted according to the sub-population (i.e. sub-batch)
sizes if `num_interactions` is left as None.
The default value for `popsize_weighted_grad_avg` is None.
When the distributed mode is disabled (i.e. when `distributed`
is False), then the argument `popsize_weighted_grad_avg` is
expected as None.
"""
if popsize is None:
popsize = int(4 + math.floor(3 * math.log(problem.solution_length)))
if center_learning_rate is None:
center_learning_rate = 1.0
def default_stdev_lr():
n = problem.solution_length
return 0.6 * (3 + math.log(n)) / (n * math.sqrt(n))
if stdev_learning_rate is None:
stdev_learning_rate = default_stdev_lr()
else:
stdev_learning_rate = float(stdev_learning_rate)
if scale_learning_rate:
stdev_learning_rate *= default_stdev_lr()
super().__init__(
problem,
popsize=popsize,
center_learning_rate=center_learning_rate,
stdev_learning_rate=stdev_learning_rate,
stdev_init=stdev_init,
radius_init=radius_init,
popsize_max=popsize_max,
num_interactions=num_interactions,
optimizer=optimizer,
optimizer_config=optimizer_config,
ranking_method=ranking_method,
center_init=center_init,
stdev_min=None,
stdev_max=None,
stdev_max_change=None,
obj_index=obj_index,
distributed=distributed,
popsize_weighted_grad_avg=popsize_weighted_grad_avg,
)
functional
special
¶
Purely functional implementations of optimization algorithms.
Reasoning.
PyTorch has a functional API within its namespace torch.func.
In addition to allowing one to choose a pure functional programming style,
torch.func enables powerful batched operations via torch.func.vmap.
To be able to work with the functional programming style of torch.func,
EvoTorch introduces functional implementations of evolutionary search
algorithms and optimizers within the namespace
evotorch.algorithms.functional.
These algorithm implementations are compatible with torch.func.vmap,
and therefore they can perform batched evolutionary searches
(e.g. they can work on not just a single population, but on batches
of populations). Such batched searches can be helpful in the following
scenarios:
Scenario 1: Nested optimization. The main optimization problem at hand might have internal optimization problems. Therefore, when the main optimization problem's fitness function is reached, the internal optimization problem will have to be solved for each solution of the main problem. In such a scenario, one might want to use a functional evolutionary search for the inner optimization problem, so that a batch of populations is formed where each batch item represents a separate population associated with a separate solution of the main problem.
Scenario 2: Batched hyperparameter search. If the user is interested in using a search algorithm that has a functional implementation, the user might want to implement a hyperparameter search in such a way that there is a batch of hyperparameters (instead of just a single set of hyperparameters), and the search is performed on a batch of populations. In such a setting, each population within the population batch is associated with a different hyperparameter set within the hyperparameter batch.
Example: cross entropy method. Let us assume that we have the following fitness function, whose output we wish to minimize:
import torch
def f(x: torch.Tensor) -> torch.Tensor:
assert x.ndim == 2, "Please pass `x` as a 2-dimensional tensor"
return torch.sum(x**2, dim=-1)
Let us initialize our search from a random point:
Now we can initialize our cross entropy method like this:
from evotorch.algorithms.functional import cem, cem_ask, cem_tell
state = cem(
#
# Center point of the initial search distribution:
center_init=center_init,
#
#
# Standard deviation of the initial search distribution:
stdev_init=10.0,
#
#
# Top half of the population are to be chosen as parents:
parenthood_ratio=0.5,
#
#
# We wish to minimize the fitnesses:
objective_sense="min",
#
#
# A standard deviation item is not allowed to change more than
# 1% of its original value:
stdev_max_change=0.01,
)
At this point, we have an initial state of our cross entropy method search,
stored by the variable state. Now, we can implement a loop and perform
multiple generations of evolutionary search like this:
num_generations = 1000
for generation in range(1, 1 + num_generations):
# Ask for a new population (of size 1000) from cross entropy method
solutions = cem_ask(state, popsize=1000)
# At this point, `solutions` is a regular PyTorch tensor, ready to be
# passed to the function `f`.
# `solutions` is a 2-dimensional tensor of shape (N, L) where `N`
# is the number of solutions, and `L` is the length of a solution.
# Our example fitness function `f` is implemented in such a way that
# we can pass our 2-dimensional `solutions` tensor into it directly.
# We will receive `fitnesses` as a 1-dimensional tensor of length `N`.
fitnesses = f(solutions)
# Let us report the mean of fitnesses to see the progress
print("Generation:", generation, " Mean of fitnesses:", torch.mean(fitnesses))
# Now, we inform cross entropy method of the latest state of the search,
# the latest population, and the latest fitnesses, so that it can give us
# the next state of the search.
state = cem_tell(state, solutions, fitnesses)
At the end of the evolutionary search (or, actually, at any point), one can
analyze the state tuple to get information about the current status of the
search distribution. These state tuples are named tuples, and therefore, the
data they store are labeled.
In the case of cross entropy method, the latest center of the search
distribution can be obtained via:
latest_center = state.center
# Note, in the case of pgpe, this would be:
# latest_center = state.optimizer_state.center
Notes on manipulating the evolutionary search.
If, at any point of the search, you would like to change a hyperparameter,
you can do so by creating a modified copy of your latest state tuple,
and pass it to the ask method of your evolutionary search (which,
in the case of cross entropy method, is cem_ask).
Similarly, if you wish to change the center point of the search, you can
pass a modified state tuple containing the new center point to cem_ask.
Notes on batching.
In regular non-batched cases, functional search algorithms expect the
center_init argument as a 1-dimensional tensor. If center_init is given
as a tensor with 2 or more dimensions, the extra leftmost dimensions will
be considered as batch dimensions, and therefore the evolutionary search
itself will be batched (which means that the ask method of the search
algorithm will return a batch of populations). Furthermore, certain
hyperparameters can also be given in batches. See the specific
documentation of the functional algorithms to see which hyperparameters
support batching.
When working with batched populations, it is important to make sure that the fitness function can work with arbitrary amount of dimensions (not just 2 dimensions). One way to implement such fitness functions with the help of the rowwise decorator:
from evotorch.decorators import rowwise
@rowwise
def f(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x**2)
When decorated with @rowwise, we can implement our function as if the
tensor x is a 1-dimensional tensor. If the decorated f receives x
not as a vector, but as a matrix, then it will do the same operation
on each row of the matrix, in a vectorized manner. If x has 3 or
more dimensions, they will be considered as extra batch dimensions,
affecting the shape of the resulting tensor.
Example: gradient-based search.
This namespace also provides functional implementations of various gradient
based optimizers. The reasoning behind the existence of these implementations
is two-fold: (i) these optimizers are used by the functional pgpe
implementation (for handling the momentum); and (ii) having these optimizers
with a similar API allows user to switch back-and-forth between evolutionary
and gradient-based search for solving the same problem, hopefully without
having to change the code too much.
Let us consider the same fitness function again, in its @rowwise form so
that it can work with a single vector or a batch of such vectors:
from evotorch.decorators import rowwise
@rowwise
def f(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x**2)
To solve this optimization problem using the Adam optimizer, one can do the following:
from evotorch.algorithms.functional import adam, adam_ask, adam_tell
from torch.func import grad
# Prepare an initial search point
solution_length = 1000
center_init = torch.randn(solution_length, dtype=torch.float32) * 10
# Initialize the Adam optimizer
state = adam(
center_init=center_init,
center_learning_rate=0.001,
beta1=0.9,
beta2=0.999,
epsilon=1e-8,
)
num_iterations = 1000
for iteration in range(1, 1 + num_iterations):
# Get the current search point of the Adam search
center = adam_ask(state)
# Get the gradient.
# Negative, because we want to minimize f.
gradient = -(grad(f)(center))
# Inform the Adam optimizer of the gradient to follow, and get the next
# state of the search
state = adam_tell(state, follow_grad=gradient)
# Store the final solution
final_solution = adam_ask(state)
# or, alternatively:
# final_solution = state.center
Solving a stateful Problem object using functional algorithms. If you wish to solve a stateful Problem using a functional optimization algorithm, you can obtain a callable evaluator out of that Problem object, and then use it for computing the fitnesses. See the following example:
from evotorch import Problem, SolutionBatch
from evotorch.algorithms.functional import cem, cem_ask, cem_tell
class MyProblem(Problem):
def __init__(self): ...
def _evaluate_batch(self, batch: SolutionBatch):
# Stateful batch evaluation code goes here
...
# Instantiate the problem
problem = MyProblem()
# Make a callable fitness evaluator
fproblem = problem.make_callable_evaluator()
# Make an initial solution
center_init = torch.randn(problem.solution_length, dtype=torch.float32) * 10
# Prepare a cross entropy method search
state = cem(
center_init=center_init,
stdev_init=10.0,
parenthood_ratio=0.5,
objective_sense="min",
stdev_max_change=0.01,
)
num_generations = 1000
for generation in range(1, 1 + num_generations):
# Get a population
solutions = cem_ask(state, popsize=1000)
# Call the evaluator to get the fitnesses
fitnesses = fproblem(solutions)
# Let us report the mean of fitnesses to see the progress
print("Generation:", generation, " Mean of fitnesses:", torch.mean(fitnesses))
# Now, we inform cross entropy method of the latest state of the search,
# the latest population, and the latest fitnesses, so that it can give us
# the next state of the search.
state = cem_tell(state, solutions, fitnesses)
# Center of the latest search distribution
latest_center = state.center
funcadam
¶
AdamState (tuple)
¶
AdamState(center, center_learning_rate, beta1, beta2, epsilon, m, v, t)
Source code in evotorch/algorithms/functional/funcadam.py
adam(*, center_init, center_learning_rate=0.001, beta1=0.9, beta2=0.999, epsilon=1e-08)
¶
Initialize an Adam optimizer and return its initial state.
Reference:
Kingma, D. P. and J. Ba (2015).
Adam: A method for stochastic optimization.
In Proceedings of 3rd International Conference on Learning Representations.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
center_init |
Union[torch.Tensor, numpy.ndarray] |
Starting point for the Adam search. Expected as a PyTorch tensor with at least 1 dimension. If there are 2 or more dimensions, the extra leftmost dimensions are interpreted as batch dimensions. |
required |
center_learning_rate |
Union[numbers.Number, numpy.ndarray, torch.Tensor] |
Learning rate (i.e. the step size) for the Adam updates. Can be a scalar or a multidimensional tensor. If given as a tensor with multiple dimensions, those dimensions will be interpreted as batch dimensions. |
0.001 |
beta1 |
Union[numbers.Number, numpy.ndarray, torch.Tensor] |
beta1 hyperparameter for the Adam optimizer. Can be a scalar or a multidimensional tensor. If given as a tensor with multiple dimensions, those dimensions will be interpreted as batch dimensions. |
0.9 |
beta2 |
Union[numbers.Number, numpy.ndarray, torch.Tensor] |
beta2 hyperparameter for the Adam optimizer. Can be a scalar or a multidimensional tensor. If given as a tensor with multiple dimensions, those dimensions will be interpreted as batch dimensions. |
0.999 |
epsilon |
Union[numbers.Number, numpy.ndarray, torch.Tensor] |
epsilon hyperparameter for the Adam optimizer. Can be a scalar or a multidimensional tensor. If given as a tensor with multiple dimensions, those dimensions will be interpreted as batch dimensions. |
1e-08 |
Returns:
| Type | Description |
|---|---|
AdamState |
A named tuple of type |
Source code in evotorch/algorithms/functional/funcadam.py
def adam(
*,
center_init: BatchableVector,
center_learning_rate: BatchableScalar = 0.001,
beta1: BatchableScalar = 0.9,
beta2: BatchableScalar = 0.999,
epsilon: BatchableScalar = 1e-8,
) -> AdamState:
"""
Initialize an Adam optimizer and return its initial state.
Reference:
Kingma, D. P. and J. Ba (2015).
Adam: A method for stochastic optimization.
In Proceedings of 3rd International Conference on Learning Representations.
Args:
center_init: Starting point for the Adam search.
Expected as a PyTorch tensor with at least 1 dimension.
If there are 2 or more dimensions, the extra leftmost dimensions
are interpreted as batch dimensions.
center_learning_rate: Learning rate (i.e. the step size) for the Adam
updates. Can be a scalar or a multidimensional tensor.
If given as a tensor with multiple dimensions, those dimensions
will be interpreted as batch dimensions.
beta1: beta1 hyperparameter for the Adam optimizer.
Can be a scalar or a multidimensional tensor.
If given as a tensor with multiple dimensions, those dimensions
will be interpreted as batch dimensions.
beta2: beta2 hyperparameter for the Adam optimizer.
Can be a scalar or a multidimensional tensor.
If given as a tensor with multiple dimensions, those dimensions
will be interpreted as batch dimensions.
epsilon: epsilon hyperparameter for the Adam optimizer.
Can be a scalar or a multidimensional tensor.
If given as a tensor with multiple dimensions, those dimensions
will be interpreted as batch dimensions.
Returns:
A named tuple of type `AdamState`, representing the initial state
of the Adam optimizer.
"""
center_init = torch.as_tensor(center_init)
dtype = center_init.dtype
device = center_init.device
def as_tensor(x) -> torch.Tensor:
return torch.as_tensor(x, dtype=dtype, device=device)
center_learning_rate = as_tensor(center_learning_rate)
beta1 = as_tensor(beta1)
beta2 = as_tensor(beta2)
epsilon = as_tensor(epsilon)
m = torch.zeros_like(center_init)
v = torch.zeros_like(center_init)
t = torch.zeros(center_init.shape[:-1], dtype=dtype, device=device)
return AdamState(
center=center_init,
center_learning_rate=center_learning_rate,
beta1=beta1,
beta2=beta2,
epsilon=epsilon,
m=m,
v=v,
t=t,
)
adam_ask(state)
¶
Get the search point stored by the given AdamState.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state |
AdamState |
The current state of the Adam optimizer. |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The search point as a 1-dimensional tensor in the non-batched case, or as a multi-dimensional tensor if the Adam search is batched. |
Source code in evotorch/algorithms/functional/funcadam.py
def adam_ask(state: AdamState) -> torch.Tensor:
"""
Get the search point stored by the given `AdamState`.
Args:
state: The current state of the Adam optimizer.
Returns:
The search point as a 1-dimensional tensor in the non-batched case,
or as a multi-dimensional tensor if the Adam search is batched.
"""
return state.center
adam_tell(state, *, follow_grad)
¶
Tell the Adam optimizer the current gradient to get its next state.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state |
AdamState |
The current state of the Adam optimizer. |
required |
follow_grad |
Union[torch.Tensor, numpy.ndarray] |
Gradient at the current point of the Adam search. Can be a 1-dimensional tensor in the non-batched case, or a multi-dimensional tensor in the batched case. |
required |
Returns:
| Type | Description |
|---|---|
AdamState |
The updated state of Adam with the given gradient applied. |
Source code in evotorch/algorithms/functional/funcadam.py
def adam_tell(state: AdamState, *, follow_grad: BatchableVector) -> AdamState:
"""
Tell the Adam optimizer the current gradient to get its next state.
Args:
state: The current state of the Adam optimizer.
follow_grad: Gradient at the current point of the Adam search.
Can be a 1-dimensional tensor in the non-batched case,
or a multi-dimensional tensor in the batched case.
Returns:
The updated state of Adam with the given gradient applied.
"""
new_center, new_m, new_v, new_t = _adam_step(
follow_grad,
state.center,
state.center_learning_rate,
state.beta1,
state.beta2,
state.epsilon,
state.m,
state.v,
state.t,
)
return AdamState(
center=new_center,
center_learning_rate=state.center_learning_rate,
beta1=state.beta1,
beta2=state.beta2,
epsilon=state.epsilon,
m=new_m,
v=new_v,
t=new_t,
)
funccem
¶
CEMState (tuple)
¶
CEMState(center, stdev, stdev_min, stdev_max, stdev_max_change, parenthood_ratio, maximize)
Source code in evotorch/algorithms/functional/funccem.py
cem(*, center_init, parenthood_ratio, objective_sense, stdev_init=None, radius_init=None, stdev_min=None, stdev_max=None, stdev_max_change=None)
¶
Get an initial state for the cross entropy method (CEM).
The received initial state, a named tuple of type CEMState, is to be
passed to the function cem_ask(...) to receive the solutions belonging
to the first generation of the evolutionary search.
References:
Rubinstein, R. (1999). The cross-entropy method for combinatorial
and continuous optimization.
Methodology and computing in applied probability, 1(2), 127-190.
Duan, Y., Chen, X., Houthooft, R., Schulman, J., Abbeel, P. (2016).
Benchmarking deep reinforcement learning for continuous control.
International conference on machine learning. PMLR, 2016.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
center_init |
Union[torch.Tensor, numpy.ndarray] |
Center (i.e. mean) of the initial search distribution.
Expected as a PyTorch tensor with at least 1 dimension.
If the given |
required |
stdev_init |
Union[float, torch.Tensor, numpy.ndarray] |
Standard deviation of the initial search distribution.
If this is given as a scalar |
None |
radius_init |
Union[float, numbers.Number, numpy.ndarray, torch.Tensor] |
Radius for the initial search distribution, representing
the euclidean norm for the first standard deviation vector.
Setting this value as |
None |
parenthood_ratio |
float |
Proportion of the solutions that will be chosen as the parents for the next generation. For example, if this is given as 0.5, the top 50% of the solutions will be chosen as parents. |
required |
objective_sense |
str |
Expected as a string, either as 'min' or as 'max'. Determines if the goal is to minimize or is to maximize. |
required |
stdev_min |
Union[float, torch.Tensor, numpy.ndarray] |
Minimum allowed standard deviation for the search distribution. Can be given as a scalar or as a tensor with one or more dimensions. When given with at least 2 dimensions, the extra leftmost dimensions will be interpreted as batch dimensions. |
None |
stdev_max |
Union[float, torch.Tensor, numpy.ndarray] |
Maximum allowed standard deviation for the search distribution. Can be given as a scalar or as a tensor with one or more dimensions. When given with at least 2 dimensions, the extra leftmost dimensions will be interpreted as batch dimensions. |
None |
stdev_max_change |
Union[float, torch.Tensor, numpy.ndarray] |
Maximum allowed change for the standard deviation
vector. If this is given as a scalar, this scalar will serve as a
limiter for the change of the entire standard deviation vector.
For example, a scalar value of 0.2 means that the elements of the
standard deviation vector cannot change more than the 20% of their
original values. If this is given as a vector (i.e. as a
1-dimensional tensor), each element of |
None |
Returns:
| Type | Description |
|---|---|
CEMState |
A named tuple, of type |
Source code in evotorch/algorithms/functional/funccem.py
def cem(
*,
center_init: BatchableVector,
parenthood_ratio: float,
objective_sense: str,
stdev_init: Optional[Union[float, BatchableVector]] = None,
radius_init: Optional[Union[float, BatchableScalar]] = None,
stdev_min: Optional[Union[float, BatchableVector]] = None,
stdev_max: Optional[Union[float, BatchableVector]] = None,
stdev_max_change: Optional[Union[float, BatchableVector]] = None,
) -> CEMState:
"""
Get an initial state for the cross entropy method (CEM).
The received initial state, a named tuple of type `CEMState`, is to be
passed to the function `cem_ask(...)` to receive the solutions belonging
to the first generation of the evolutionary search.
References:
Rubinstein, R. (1999). The cross-entropy method for combinatorial
and continuous optimization.
Methodology and computing in applied probability, 1(2), 127-190.
Duan, Y., Chen, X., Houthooft, R., Schulman, J., Abbeel, P. (2016).
Benchmarking deep reinforcement learning for continuous control.
International conference on machine learning. PMLR, 2016.
Args:
center_init: Center (i.e. mean) of the initial search distribution.
Expected as a PyTorch tensor with at least 1 dimension.
If the given `center` tensor has more than 1 dimensions, the extra
leftmost dimensions will be interpreted as batch dimensions.
stdev_init: Standard deviation of the initial search distribution.
If this is given as a scalar `s`, the standard deviation for the
search distribution will be interpreted as `[s, s, ..., s]` whose
length is the same with the length of `center_init`.
If this is given as a 1-dimensional tensor, the given tensor will
be interpreted as the standard deviation vector.
If this is given as a tensor with at least 2 dimensions, the extra
leftmost dimension(s) will be interpreted as batch dimensions.
If you wish to express the coverage area of the initial search
distribution in terms of "radius" instead, you can leave
`stdev_init` as None, and provide a value for the argument
`radius_init`.
radius_init: Radius for the initial search distribution, representing
the euclidean norm for the first standard deviation vector.
Setting this value as `r` means that the standard deviation
vector will be initialized as a vector `[s, s, ..., s]`
whose norm will be equal to `r`. In the non-batched case,
`radius_init` is expected as a scalar value.
If `radius_init` is given as a tensor with 1 or more
dimensions, those dimensions will be considered as batch
dimensions. If you wish to express the coverage are of the initial
search distribution in terms of the standard deviation values
instead, you can leave `radius_init` as None, and provide a value
for the argument `stdev_init`.
parenthood_ratio: Proportion of the solutions that will be chosen as
the parents for the next generation. For example, if this is
given as 0.5, the top 50% of the solutions will be chosen as
parents.
objective_sense: Expected as a string, either as 'min' or as 'max'.
Determines if the goal is to minimize or is to maximize.
stdev_min: Minimum allowed standard deviation for the search
distribution. Can be given as a scalar or as a tensor with one or
more dimensions. When given with at least 2 dimensions, the extra
leftmost dimensions will be interpreted as batch dimensions.
stdev_max: Maximum allowed standard deviation for the search
distribution. Can be given as a scalar or as a tensor with one or
more dimensions. When given with at least 2 dimensions, the extra
leftmost dimensions will be interpreted as batch dimensions.
stdev_max_change: Maximum allowed change for the standard deviation
vector. If this is given as a scalar, this scalar will serve as a
limiter for the change of the entire standard deviation vector.
For example, a scalar value of 0.2 means that the elements of the
standard deviation vector cannot change more than the 20% of their
original values. If this is given as a vector (i.e. as a
1-dimensional tensor), each element of `stdev_max_change` will
serve as a limiter to its corresponding element within the standard
deviation vector. If `stdev_max_change` is given as a tensor with
at least 2 dimensions, the extra leftmost dimension(s) will be
interpreted as batch dimensions.
If you do not wish to have such a limiter, you can leave this as
None.
Returns:
A named tuple, of type `CEMState`, storing the hyperparameters and the
initial state of the cross entropy method.
"""
from .misc import _get_stdev_init
center_init = torch.as_tensor(center_init)
if center_init.ndim < 1:
raise ValueError(
"The center of the search distribution for the functional CEM was expected"
" as a tensor with at least 1 dimension."
f" However, the encountered `center_init` is {center_init}, of shape {center_init.shape}."
)
solution_length = center_init.shape[-1]
if solution_length == 0:
raise ValueError("Solution length cannot be 0")
stdev_init = _get_stdev_init(center_init=center_init, stdev_init=stdev_init, radius_init=radius_init)
device = center_init.device
dtype = center_init.dtype
def as_vector_like_center(x: Iterable, vector_name: str) -> torch.Tensor:
x = torch.as_tensor(x, dtype=dtype, device=device)
if x.ndim == 0:
x = x.repeat(solution_length)
else:
if x.shape[-1] != solution_length:
raise ValueError(
f"`{vector_name}` has an incompatible length."
f" The length of `{vector_name}`: {x.shape[-1]},"
f" but the solution length implied by the provided `center_init` is {solution_length}."
)
return x
if stdev_min is None:
stdev_min = 0.0
stdev_min = as_vector_like_center(stdev_min, "stdev_min")
if stdev_max is None:
stdev_max = float("inf")
stdev_max = as_vector_like_center(stdev_max, "stdev_max")
if stdev_max_change is None:
stdev_max_change = float("inf")
stdev_max_change = as_vector_like_center(stdev_max_change, "stdev_max_change")
parenthood_ratio = float(parenthood_ratio)
if objective_sense == "min":
maximize = False
elif objective_sense == "max":
maximize = True
else:
raise ValueError(
f"`objective_sense` was expected as 'min' or 'max', but it was received as {repr(objective_sense)}"
)
return CEMState(
center=center_init,
stdev=stdev_init,
stdev_min=stdev_min,
stdev_max=stdev_max,
stdev_max_change=stdev_max_change,
parenthood_ratio=parenthood_ratio,
maximize=maximize,
)
cem_ask(state, *, popsize)
¶
Obtain a population from cross entropy method, given the state.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state |
CEMState |
The current state of the cross entropy method search. |
required |
popsize |
int |
Number of solutions to be generated for the requested population. |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
Population, as a tensor of at least 2 dimensions. |
Source code in evotorch/algorithms/functional/funccem.py
def cem_ask(state: CEMState, *, popsize: int) -> torch.Tensor:
"""
Obtain a population from cross entropy method, given the state.
Args:
state: The current state of the cross entropy method search.
popsize: Number of solutions to be generated for the requested
population.
Returns:
Population, as a tensor of at least 2 dimensions.
"""
return _cem_ask(state.center, state.stdev, state.parenthood_ratio, popsize)
cem_tell(state, values, evals)
¶
Given the old state and the evals (fitnesses), obtain the next state.
From this state tuple, the center point of the search distribution can be
obtained via the field .center.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state |
CEMState |
The old state of the cross entropy method search. |
required |
values |
Tensor |
The most recent population, as a PyTorch tensor. |
required |
evals |
Tensor |
Evaluation results (i.e. fitnesses) for the solutions expressed
by |
required |
Returns:
| Type | Description |
|---|---|
CEMState |
The new state of the cross entropy method search. |
Source code in evotorch/algorithms/functional/funccem.py
def cem_tell(state: CEMState, values: torch.Tensor, evals: torch.Tensor) -> CEMState:
"""
Given the old state and the evals (fitnesses), obtain the next state.
From this state tuple, the center point of the search distribution can be
obtained via the field `.center`.
Args:
state: The old state of the cross entropy method search.
values: The most recent population, as a PyTorch tensor.
evals: Evaluation results (i.e. fitnesses) for the solutions expressed
by `values`. For example, if `values` is shaped `(N, L)`, this means
that there are `N` solutions (of length `L`). So, `evals` is
expected as a 1-dimensional tensor of length `N`, where `evals[i]`
expresses the fitness of the solution `values[i, :]`.
If `values` is shaped `(B, N, L)`, then there is also a batch
dimension, so, `evals` is expected as a 2-dimensional tensor of
shape `(B, N)`.
Returns:
The new state of the cross entropy method search.
"""
new_center, new_stdev = _cem_tell(
state.stdev_min,
state.stdev_max,
state.stdev_max_change,
state.parenthood_ratio,
state.maximize,
state.center,
state.stdev,
values,
evals,
)
return CEMState(
center=new_center,
stdev=new_stdev,
stdev_min=state.stdev_min,
stdev_max=state.stdev_max,
stdev_max_change=state.stdev_max_change,
parenthood_ratio=state.parenthood_ratio,
maximize=state.maximize,
)
funcclipup
¶
ClipUpState (tuple)
¶
ClipUpState(center, velocity, center_learning_rate, momentum, max_speed)
Source code in evotorch/algorithms/functional/funcclipup.py
clipup(*, center_init, momentum=0.9, center_learning_rate=None, max_speed=None)
¶
Initialize the ClipUp optimizer and return its initial state.
Reference:
Toklu, N. E., Liskowski, P., & Srivastava, R. K. (2020, September).
ClipUp: A Simple and Powerful Optimizer for Distribution-Based Policy Evolution.
In International Conference on Parallel Problem Solving from Nature (pp. 515-527).
Springer, Cham.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
center_init |
Union[torch.Tensor, numpy.ndarray] |
Starting point for the ClipUp search. Expected as a PyTorch tensor with at least 1 dimension. If there are 2 or more dimensions, the extra leftmost dimensions are interpreted as batch dimensions. |
required |
center_learning_rate |
Union[numbers.Number, numpy.ndarray, torch.Tensor] |
Learning rate (i.e. the step size) for the ClipUp updates. Can be a scalar or a multidimensional tensor. If given as a tensor with multiple dimensions, those dimensions will be interpreted as batch dimensions. |
None |
max_speed |
Union[numbers.Number, numpy.ndarray, torch.Tensor] |
Maximum speed, expected as a scalar. The euclidean norm
of the velocity (i.e. of the update vector) is not allowed to
exceed |
None |
Source code in evotorch/algorithms/functional/funcclipup.py
def clipup(
*,
center_init: BatchableVector,
momentum: BatchableScalar = 0.9,
center_learning_rate: Optional[BatchableScalar] = None,
max_speed: Optional[BatchableScalar] = None,
) -> ClipUpState:
"""
Initialize the ClipUp optimizer and return its initial state.
Reference:
Toklu, N. E., Liskowski, P., & Srivastava, R. K. (2020, September).
ClipUp: A Simple and Powerful Optimizer for Distribution-Based Policy Evolution.
In International Conference on Parallel Problem Solving from Nature (pp. 515-527).
Springer, Cham.
Args:
center_init: Starting point for the ClipUp search.
Expected as a PyTorch tensor with at least 1 dimension.
If there are 2 or more dimensions, the extra leftmost dimensions
are interpreted as batch dimensions.
center_learning_rate: Learning rate (i.e. the step size) for the ClipUp
updates. Can be a scalar or a multidimensional tensor.
If given as a tensor with multiple dimensions, those dimensions
will be interpreted as batch dimensions.
max_speed: Maximum speed, expected as a scalar. The euclidean norm
of the velocity (i.e. of the update vector) is not allowed to
exceed `max_speed`.
If given as a tensor with multiple dimensions, those dimensions
will be interpreted as batch dimensions.
"""
center_init = torch.as_tensor(center_init)
dtype = center_init.dtype
device = center_init.device
def as_tensor(x) -> torch.Tensor:
return torch.as_tensor(x, dtype=dtype, device=device)
if (center_learning_rate is None) and (max_speed is None):
raise ValueError("Both `center_learning_rate` and `max_speed` is missing. At least one of them is needed.")
elif (center_learning_rate is not None) and (max_speed is None):
center_learning_rate = as_tensor(center_learning_rate)
max_speed = center_learning_rate * 2.0
elif (center_learning_rate is None) and (max_speed is not None):
max_speed = as_tensor(max_speed)
center_learning_rate = max_speed / 2.0
else:
center_learning_rate = as_tensor(center_learning_rate)
max_speed = as_tensor(max_speed)
velocity = torch.zeros_like(center_init)
momentum = as_tensor(momentum)
return ClipUpState(
center=center_init,
velocity=velocity,
center_learning_rate=center_learning_rate,
momentum=momentum,
max_speed=max_speed,
)
clipup_ask(state)
¶
Get the search point stored by the given ClipUpState.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state |
ClipUpState |
The current state of the ClipUp optimizer. |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The search point as a 1-dimensional tensor in the non-batched case, or as a multi-dimensional tensor if the ClipUp search is batched. |
Source code in evotorch/algorithms/functional/funcclipup.py
def clipup_ask(state: ClipUpState) -> torch.Tensor:
"""
Get the search point stored by the given `ClipUpState`.
Args:
state: The current state of the ClipUp optimizer.
Returns:
The search point as a 1-dimensional tensor in the non-batched case,
or as a multi-dimensional tensor if the ClipUp search is batched.
"""
return state.center
clipup_tell(state, *, follow_grad)
¶
Tell the ClipUp optimizer the current gradient to get its next state.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state |
ClipUpState |
The current state of the ClipUp optimizer. |
required |
follow_grad |
Union[torch.Tensor, numpy.ndarray] |
Gradient at the current point of the Adam search. Can be a 1-dimensional tensor in the non-batched case, or a multi-dimensional tensor in the batched case. |
required |
Returns:
| Type | Description |
|---|---|
ClipUpState |
The updated state of ClipUp with the given gradient applied. |
Source code in evotorch/algorithms/functional/funcclipup.py
def clipup_tell(state: ClipUpState, *, follow_grad: BatchableVector) -> ClipUpState:
"""
Tell the ClipUp optimizer the current gradient to get its next state.
Args:
state: The current state of the ClipUp optimizer.
follow_grad: Gradient at the current point of the Adam search.
Can be a 1-dimensional tensor in the non-batched case,
or a multi-dimensional tensor in the batched case.
Returns:
The updated state of ClipUp with the given gradient applied.
"""
velocity, center = _clipup_step(
follow_grad,
state.center,
state.velocity,
state.center_learning_rate,
state.momentum,
state.max_speed,
)
return ClipUpState(
center=center,
velocity=velocity,
center_learning_rate=state.center_learning_rate,
momentum=state.momentum,
max_speed=state.max_speed,
)
funcpgpe
¶
PGPEState (tuple)
¶
PGPEState(optimizer, optimizer_state, stdev, stdev_learning_rate, stdev_min, stdev_max, stdev_max_change, ranking_method, maximize, symmetric)
Source code in evotorch/algorithms/functional/funcpgpe.py
class PGPEState(NamedTuple):
optimizer: Union[str, tuple] # "adam" or (adam, adam_ask, adam_tell)
optimizer_state: tuple
stdev: torch.Tensor
stdev_learning_rate: torch.Tensor
stdev_min: torch.Tensor
stdev_max: torch.Tensor
stdev_max_change: torch.Tensor
ranking_method: str
maximize: bool
symmetric: bool
__getnewargs__(self)
special
¶
__new__(_cls, optimizer, optimizer_state, stdev, stdev_learning_rate, stdev_min, stdev_max, stdev_max_change, ranking_method, maximize, symmetric)
special
staticmethod
¶
Create new instance of PGPEState(optimizer, optimizer_state, stdev, stdev_learning_rate, stdev_min, stdev_max, stdev_max_change, ranking_method, maximize, symmetric)
__repr__(self)
special
¶
pgpe(*, center_init, center_learning_rate, stdev_learning_rate, objective_sense, ranking_method='centered', optimizer='clipup', optimizer_config=None, stdev_init=None, radius_init=None, stdev_min=None, stdev_max=None, stdev_max_change=0.2, symmetric=True)
¶
Get an initial state for the PGPE algorithm.
The received initial state, a named tuple of type PGPEState, is to be
passed to the function pgpe_ask(...) to receive the solutions belonging
to the first generation of the evolutionary search.
Inspired by the PGPE implementations used in the studies of Ha (2017, 2019), and by the evolution strategy variant of Salimans et al. (2017), this PGPE implementation uses 0-centered ranking by default. The default optimizer for this PGPE implementation is ClipUp (Toklu et al., 2020).
References:
Frank Sehnke, Christian Osendorfer, Thomas Ruckstiess,
Alex Graves, Jan Peters, Jurgen Schmidhuber (2010).
Parameter-exploring Policy Gradients.
Neural Networks 23(4), 551-559.
David Ha (2017). Evolving Stable Strategies.
<http://blog.otoro.net/2017/11/12/evolving-stable-strategies/>
Salimans, T., Ho, J., Chen, X., Sidor, S. and Sutskever, I. (2017).
Evolution Strategies as a Scalable Alternative to
Reinforcement Learning.
David Ha (2019). Reinforcement Learning for Improving Agent Design.
Artificial life 25 (4), 352-365.
Toklu, N.E., Liskowski, P., Srivastava, R.K. (2020).
ClipUp: A Simple and Powerful Optimizer
for Distribution-based Policy Evolution.
Parallel Problem Solving from Nature (PPSN 2020).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
center_init |
Union[torch.Tensor, numpy.ndarray] |
Center (i.e. mean) of the initial search distribution.
Expected as a PyTorch tensor with at least 1 dimension.
If the given |
required |
center_learning_rate |
Union[numbers.Number, numpy.ndarray, torch.Tensor] |
Learning rate for when updating the center of the search distribution. For normal cases, this is expected as a scalar. If given as an n-dimensional tensor (where n>0), the extra dimensions will be considered as batch dimensions. |
required |
stdev_learning_rate |
Union[numbers.Number, numpy.ndarray, torch.Tensor] |
Learning rate for when updating the standard deviation of the search distribution. For normal cases, this is expected as a scalar. If given as an n-dimensional tensor (where n>0), the extra dimensions will be considered as batch dimensions. |
required |
objective_sense |
str |
Expected as a string, either as 'min' or as 'max'. Determines if the goal is to minimize or is to maximize. |
required |
ranking_method |
str |
Determines how the fitnesses will be ranked before computing the gradients. Among the choices are "centered" (a linear ranking where the worst solution gets the rank -0.5 and the best solution gets the rank +0.5), "linear" (a linear ranking where the worst solution gets the rank 0 and the best solution gets the rank 1), "nes" (the ranking method that is used by the natural evolution strategies), and "raw" (no ranking). |
'centered' |
optimizer |
Union[str, tuple] |
Functional optimizer to use when updating the center of the
search distribution. The functional optimizer can be expressed via
a string, or via a tuple.
If given as string, the valid choices are:
"clipup" (for the ClipUp optimizer),
"adam" (for the Adam optimizer),
"sgd" (for regular gradient ascent/descent).
If given as a tuple, the tuple should be in the form
|
'clipup' |
optimizer_config |
Optional[dict] |
Optionally a dictionary, containing the hyperparameters for the optimizer. |
None |
stdev_init |
Union[float, torch.Tensor, numpy.ndarray] |
Standard deviation of the initial search distribution.
If this is given as a scalar |
None |
radius_init |
Union[float, numbers.Number, numpy.ndarray, torch.Tensor] |
Radius for the initial search distribution, representing
the euclidean norm for the first standard deviation vector.
Setting this value as |
None |
stdev_min |
Union[float, torch.Tensor, numpy.ndarray] |
Minimum allowed standard deviation for the search distribution. Can be given as a scalar or as a tensor with one or more dimensions. When given with at least 2 dimensions, the extra leftmost dimensions will be interpreted as batch dimensions. |
None |
stdev_max |
Union[float, torch.Tensor, numpy.ndarray] |
Maximum allowed standard deviation for the search distribution. Can be given as a scalar or as a tensor with one or more dimensions. When given with at least 2 dimensions, the extra leftmost dimensions will be interpreted as batch dimensions. |
None |
stdev_max_change |
Union[float, torch.Tensor, numpy.ndarray] |
Maximum allowed change for the standard deviation
vector. If this is given as a scalar, this scalar will serve as a
limiter for the change of the entire standard deviation vector.
For example, a scalar value of 0.2 means that the elements of the
standard deviation vector cannot change more than the 20% of their
original values. If this is given as a vector (i.e. as a
1-dimensional tensor), each element of |
0.2 |
symmetric |
bool |
Whether or not symmetric (i.e. antithetic) sampling will be done while generating a new population. |
True |
Returns:
| Type | Description |
|---|---|
PGPEState |
A named tuple, of type |
Source code in evotorch/algorithms/functional/funcpgpe.py
def pgpe(
*,
center_init: BatchableVector,
center_learning_rate: BatchableScalar,
stdev_learning_rate: BatchableScalar,
objective_sense: str,
ranking_method: str = "centered",
optimizer: Union[str, tuple] = "clipup", # or "adam" or "sgd"
optimizer_config: Optional[dict] = None,
stdev_init: Optional[Union[float, BatchableVector]] = None,
radius_init: Optional[Union[float, BatchableScalar]] = None,
stdev_min: Optional[Union[float, BatchableVector]] = None,
stdev_max: Optional[Union[float, BatchableVector]] = None,
stdev_max_change: Optional[Union[float, BatchableVector]] = 0.2,
symmetric: bool = True,
) -> PGPEState:
"""
Get an initial state for the PGPE algorithm.
The received initial state, a named tuple of type `PGPEState`, is to be
passed to the function `pgpe_ask(...)` to receive the solutions belonging
to the first generation of the evolutionary search.
Inspired by the PGPE implementations used in the studies
of Ha (2017, 2019), and by the evolution strategy variant of
Salimans et al. (2017), this PGPE implementation uses 0-centered
ranking by default.
The default optimizer for this PGPE implementation is ClipUp
(Toklu et al., 2020).
References:
Frank Sehnke, Christian Osendorfer, Thomas Ruckstiess,
Alex Graves, Jan Peters, Jurgen Schmidhuber (2010).
Parameter-exploring Policy Gradients.
Neural Networks 23(4), 551-559.
David Ha (2017). Evolving Stable Strategies.
<http://blog.otoro.net/2017/11/12/evolving-stable-strategies/>
Salimans, T., Ho, J., Chen, X., Sidor, S. and Sutskever, I. (2017).
Evolution Strategies as a Scalable Alternative to
Reinforcement Learning.
David Ha (2019). Reinforcement Learning for Improving Agent Design.
Artificial life 25 (4), 352-365.
Toklu, N.E., Liskowski, P., Srivastava, R.K. (2020).
ClipUp: A Simple and Powerful Optimizer
for Distribution-based Policy Evolution.
Parallel Problem Solving from Nature (PPSN 2020).
Args:
center_init: Center (i.e. mean) of the initial search distribution.
Expected as a PyTorch tensor with at least 1 dimension.
If the given `center` tensor has more than 1 dimensions, the extra
leftmost dimensions will be interpreted as batch dimensions.
center_learning_rate: Learning rate for when updating the center of the
search distribution.
For normal cases, this is expected as a scalar. If given as an
n-dimensional tensor (where n>0), the extra dimensions will be
considered as batch dimensions.
stdev_learning_rate: Learning rate for when updating the standard
deviation of the search distribution.
For normal cases, this is expected as a scalar. If given as an
n-dimensional tensor (where n>0), the extra dimensions will be
considered as batch dimensions.
objective_sense: Expected as a string, either as 'min' or as 'max'.
Determines if the goal is to minimize or is to maximize.
ranking_method: Determines how the fitnesses will be ranked before
computing the gradients. Among the choices are
"centered" (a linear ranking where the worst solution gets the rank
-0.5 and the best solution gets the rank +0.5),
"linear" (a linear ranking where the worst solution gets the rank
0 and the best solution gets the rank 1),
"nes" (the ranking method that is used by the natural evolution
strategies), and
"raw" (no ranking).
optimizer: Functional optimizer to use when updating the center of the
search distribution. The functional optimizer can be expressed via
a string, or via a tuple.
If given as string, the valid choices are:
"clipup" (for the ClipUp optimizer),
"adam" (for the Adam optimizer),
"sgd" (for regular gradient ascent/descent).
If given as a tuple, the tuple should be in the form
`(optim, optim_ask, optim_tell)`, where the objects
`optim`, `optim_ask`, and `optim_tell` are the functions for
initializing the optimizer, asking (for the current search point),
and telling (the gradient to follow).
The function `optim` should expect keyword arguments for its
hyperparameters, and should return a state tuple of the optimizer.
The function `optim_ask` should expect the state tuple of the
optimizer, and should return the current search point as a tensor.
The function `optim_tell` should expect the state tuple of the
optimizer as a positional argument, and the gradient via the
keyword argument `follow_grad`.
optimizer_config: Optionally a dictionary, containing the
hyperparameters for the optimizer.
stdev_init: Standard deviation of the initial search distribution.
If this is given as a scalar `s`, the standard deviation for the
search distribution will be interpreted as `[s, s, ..., s]` whose
length is the same with the length of `center_init`.
If this is given as a 1-dimensional tensor, the given tensor will
be interpreted as the standard deviation vector.
If this is given as a tensor with at least 2 dimensions, the extra
leftmost dimension(s) will be interpreted as batch dimensions.
If you wish to express the coverage area of the initial search
distribution in terms of "radius" instead, you can leave
`stdev_init` as None, and provide a value for the argument
`radius_init`.
radius_init: Radius for the initial search distribution, representing
the euclidean norm for the first standard deviation vector.
Setting this value as `r` means that the standard deviation
vector will be initialized as a vector `[s, s, ..., s]`
whose norm will be equal to `r`. In the non-batched case,
`radius_init` is expected as a scalar value.
If `radius_init` is given as a tensor with 1 or more
dimensions, those dimensions will be considered as batch
dimensions. If you wish to express the coverage are of the initial
search distribution in terms of the standard deviation values
instead, you can leave `radius_init` as None, and provide a value
for the argument `stdev_init`.
stdev_min: Minimum allowed standard deviation for the search
distribution. Can be given as a scalar or as a tensor with one or
more dimensions. When given with at least 2 dimensions, the extra
leftmost dimensions will be interpreted as batch dimensions.
stdev_max: Maximum allowed standard deviation for the search
distribution. Can be given as a scalar or as a tensor with one or
more dimensions. When given with at least 2 dimensions, the extra
leftmost dimensions will be interpreted as batch dimensions.
stdev_max_change: Maximum allowed change for the standard deviation
vector. If this is given as a scalar, this scalar will serve as a
limiter for the change of the entire standard deviation vector.
For example, a scalar value of 0.2 means that the elements of the
standard deviation vector cannot change more than the 20% of their
original values. If this is given as a vector (i.e. as a
1-dimensional tensor), each element of `stdev_max_change` will
serve as a limiter to its corresponding element within the standard
deviation vector. If `stdev_max_change` is given as a tensor with
at least 2 dimensions, the extra leftmost dimension(s) will be
interpreted as batch dimensions.
If you do not wish to have such a limiter, you can leave this as
None.
symmetric: Whether or not symmetric (i.e. antithetic) sampling will be
done while generating a new population.
Returns:
A named tuple, of type `CEMState`, storing the hyperparameters and the
initial state of the cross entropy method.
"""
from .misc import _get_stdev_init, get_functional_optimizer
center_init = torch.as_tensor(center_init)
if center_init.ndim < 1:
raise ValueError(
"The center of the search distribution for the functional PGPE was expected"
" as a tensor with at least 1 dimension."
f" However, the encountered `center` is {center_init}, of shape {center_init.shape}."
)
solution_length = center_init.shape[-1]
if solution_length == 0:
raise ValueError("Solution length cannot be 0")
stdev_init = _get_stdev_init(center_init=center_init, stdev_init=stdev_init, radius_init=radius_init)
device = center_init.device
dtype = center_init.dtype
def as_tensor(x) -> torch.Tensor:
return torch.as_tensor(x, dtype=dtype, device=device)
def as_vector_like_center(x: Iterable, vector_name: str) -> torch.Tensor:
x = as_tensor(x)
if x.ndim == 0:
x = x.repeat(solution_length)
else:
if x.shape[-1] != solution_length:
raise ValueError(
f"`{vector_name}` has an incompatible length."
f" The length of `{vector_name}`: {x.shape[-1]},"
f" but the solution length implied by the provided `center_init` is {solution_length}."
)
return x
center_learning_rate = as_tensor(center_learning_rate)
stdev_learning_rate = as_tensor(stdev_learning_rate)
if objective_sense == "min":
maximize = False
elif objective_sense == "max":
maximize = True
else:
raise ValueError(
f"`objective_sense` was expected as 'min' or 'max', but it was received as {repr(objective_sense)}"
)
ranking_method = str(ranking_method)
if stdev_min is None:
stdev_min = 0.0
stdev_min = as_vector_like_center(stdev_min, "stdev_min")
if stdev_max is None:
stdev_max = float("inf")
stdev_max = as_vector_like_center(stdev_max, "stdev_max")
if stdev_max_change is None:
stdev_max_change = float("inf")
stdev_max_change = as_vector_like_center(stdev_max_change, "stdev_max_change")
if optimizer_config is None:
optimizer_config = {}
optimizer_init_func, _, _ = get_functional_optimizer(optimizer)
optimizer_state = optimizer_init_func(
center_init=center_init, center_learning_rate=center_learning_rate, **optimizer_config
)
symmetric = bool(symmetric)
return PGPEState(
optimizer=optimizer,
optimizer_state=optimizer_state,
stdev=stdev_init,
stdev_learning_rate=stdev_learning_rate,
stdev_min=stdev_min,
stdev_max=stdev_max,
stdev_max_change=stdev_max_change,
ranking_method=ranking_method,
maximize=maximize,
symmetric=symmetric,
)
pgpe_ask(state, *, popsize)
¶
Obtain a population from the PGPE algorithm.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state |
PGPEState |
The current state of PGPE. |
required |
popsize |
int |
Number of solutions to be generated for the requested population. |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
Population, as a tensor of at least 2 dimensions. |
Source code in evotorch/algorithms/functional/funcpgpe.py
def pgpe_ask(state: PGPEState, *, popsize: int) -> torch.Tensor:
"""
Obtain a population from the PGPE algorithm.
Args:
state: The current state of PGPE.
popsize: Number of solutions to be generated for the requested
population.
Returns:
Population, as a tensor of at least 2 dimensions.
"""
from .misc import get_functional_optimizer
_, optimizer_ask, _ = get_functional_optimizer(state.optimizer)
center = optimizer_ask(state.optimizer_state)
stdev = state.stdev
sample_func = _symmetic_sample if state.symmetric else _nonsymmetric_sample
return sample_func(popsize, mu=center, sigma=stdev)
pgpe_tell(state, values, evals)
¶
Given the old state and the evals (fitnesses), obtain the next state.
From this state tuple, the center point of the search distribution can be
obtained via the field .optimizer_state.center.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state |
PGPEState |
The old state of the cross entropy method search. |
required |
values |
Tensor |
The most recent population, as a PyTorch tensor. |
required |
evals |
Tensor |
Evaluation results (i.e. fitnesses) for the solutions expressed
by |
required |
Returns:
| Type | Description |
|---|---|
PGPEState |
The new state of PGPE. |
Source code in evotorch/algorithms/functional/funcpgpe.py
def pgpe_tell(state: PGPEState, values: torch.Tensor, evals: torch.Tensor) -> PGPEState:
"""
Given the old state and the evals (fitnesses), obtain the next state.
From this state tuple, the center point of the search distribution can be
obtained via the field `.optimizer_state.center`.
Args:
state: The old state of the cross entropy method search.
values: The most recent population, as a PyTorch tensor.
evals: Evaluation results (i.e. fitnesses) for the solutions expressed
by `values`. For example, if `values` is shaped `(N, L)`, this means
that there are `N` solutions (of length `L`). So, `evals` is
expected as a 1-dimensional tensor of length `N`, where `evals[i]`
expresses the fitness of the solution `values[i, :]`.
If `values` is shaped `(B, N, L)`, then there is also a batch
dimension, so, `evals` is expected as a 2-dimensional tensor of
shape `(B, N)`.
Returns:
The new state of PGPE.
"""
from .misc import get_functional_optimizer
_, optimizer_ask, optimizer_tell = get_functional_optimizer(state.optimizer)
grad_func = _symmetric_grad if state.symmetric else _nonsymmetric_grad
objective_sense = "max" if state.maximize else "min"
grads = grad_func(
values,
evals,
mu=optimizer_ask(state.optimizer_state),
sigma=state.stdev,
objective_sense=objective_sense,
ranking_method=state.ranking_method,
)
new_optimizer_state = optimizer_tell(state.optimizer_state, follow_grad=grads["mu"])
target_stdev = _follow_stdev_grad(state.stdev, state.stdev_learning_rate, grads["sigma"])
new_stdev = modify_vector(
state.stdev, target_stdev, lb=state.stdev_min, ub=state.stdev_max, max_change=state.stdev_max_change
)
return PGPEState(
optimizer=state.optimizer,
optimizer_state=new_optimizer_state,
stdev=new_stdev,
stdev_learning_rate=state.stdev_learning_rate,
stdev_min=state.stdev_min,
stdev_max=state.stdev_max,
stdev_max_change=state.stdev_max_change,
ranking_method=state.ranking_method,
maximize=state.maximize,
symmetric=state.symmetric,
)
funcsgd
¶
SGDState (tuple)
¶
SGDState(center, velocity, center_learning_rate, momentum)
Source code in evotorch/algorithms/functional/funcsgd.py
sgd(*, center_init, center_learning_rate, momentum=None)
¶
Initialize the gradient ascent/descent search and get its initial state.
Reference regarding the momentum behavior:
Polyak, B. T. (1964).
Some methods of speeding up the convergence of iteration methods.
USSR Computational Mathematics and Mathematical Physics, 4(5):1–17.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
center_init |
Union[torch.Tensor, numpy.ndarray] |
Starting point for the gradient ascent/descent. Expected as a PyTorch tensor with at least 1 dimension. If there are 2 or more dimensions, the extra leftmost dimensions are interpreted as batch dimensions. |
required |
center_learning_rate |
Union[numbers.Number, numpy.ndarray, torch.Tensor] |
Learning rate (i.e. the step size) for gradient ascent/descent. Can be a scalar or a multidimensional tensor. If given as a tensor with multiple dimensions, those dimensions will be interpreted as batch dimensions. |
required |
momentum |
Union[numbers.Number, numpy.ndarray, torch.Tensor] |
Momentum coefficient, expected as a scalar. If provided as a scalar, Polyak-style momentum will be enabled. If given as a tensor with multiple dimensions, those dimensions will be interpreted as batch dimensions. |
None |
Source code in evotorch/algorithms/functional/funcsgd.py
def sgd(
*,
center_init: BatchableVector,
center_learning_rate: BatchableScalar,
momentum: Optional[BatchableScalar] = None,
) -> SGDState:
"""
Initialize the gradient ascent/descent search and get its initial state.
Reference regarding the momentum behavior:
Polyak, B. T. (1964).
Some methods of speeding up the convergence of iteration methods.
USSR Computational Mathematics and Mathematical Physics, 4(5):1–17.
Args:
center_init: Starting point for the gradient ascent/descent.
Expected as a PyTorch tensor with at least 1 dimension.
If there are 2 or more dimensions, the extra leftmost dimensions
are interpreted as batch dimensions.
center_learning_rate: Learning rate (i.e. the step size) for gradient
ascent/descent. Can be a scalar or a multidimensional tensor.
If given as a tensor with multiple dimensions, those dimensions
will be interpreted as batch dimensions.
momentum: Momentum coefficient, expected as a scalar.
If provided as a scalar, Polyak-style momentum will be enabled.
If given as a tensor with multiple dimensions, those dimensions
will be interpreted as batch dimensions.
"""
center_init = torch.as_tensor(center_init)
dtype = center_init.dtype
device = center_init.device
def as_tensor(x) -> torch.Tensor:
return torch.as_tensor(x, dtype=dtype, device=device)
velocity = torch.zeros_like(center_init)
center_learning_rate = as_tensor(center_learning_rate)
momentum = as_tensor(0.0) if momentum is None else as_tensor(momentum)
return SGDState(
center=center_init,
velocity=velocity,
center_learning_rate=center_learning_rate,
momentum=momentum,
)
sgd_ask(state)
¶
Get the search point stored by the given SGDState.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state |
SGDState |
The current state of gradient ascent/descent. |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The search point as a 1-dimensional tensor in the non-batched case, or as a multi-dimensional tensor if the search is batched. |
Source code in evotorch/algorithms/functional/funcsgd.py
def sgd_ask(state: SGDState) -> torch.Tensor:
"""
Get the search point stored by the given `SGDState`.
Args:
state: The current state of gradient ascent/descent.
Returns:
The search point as a 1-dimensional tensor in the non-batched case,
or as a multi-dimensional tensor if the search is batched.
"""
return state.center
sgd_tell(state, *, follow_grad)
¶
Tell the gradient ascent/descent the current gradient to get its next state.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state |
SGDState |
The current state of gradient ascent/descent. |
required |
follow_grad |
Union[torch.Tensor, numpy.ndarray] |
Gradient at the current point of the search. Can be a 1-dimensional tensor in the non-batched case, or a multi-dimensional tensor in the batched case. |
required |
Returns:
| Type | Description |
|---|---|
SGDState |
The updated state of gradient ascent/descent, with the given gradient applied. |
Source code in evotorch/algorithms/functional/funcsgd.py
def sgd_tell(state: SGDState, *, follow_grad: BatchableVector) -> SGDState:
"""
Tell the gradient ascent/descent the current gradient to get its next state.
Args:
state: The current state of gradient ascent/descent.
follow_grad: Gradient at the current point of the search.
Can be a 1-dimensional tensor in the non-batched case,
or a multi-dimensional tensor in the batched case.
Returns:
The updated state of gradient ascent/descent, with the given gradient
applied.
"""
velocity, center = _sgd_step(
follow_grad,
state.center,
state.velocity,
state.center_learning_rate,
state.momentum,
)
return SGDState(
center=center,
velocity=velocity,
center_learning_rate=state.center_learning_rate,
momentum=state.momentum,
)
misc
¶
OptimizerFunctions (tuple)
¶
get_functional_optimizer(optimizer)
¶
Get a tuple of optimizer-related functions, from the given optimizer name.
For example, if the given string is "adam", the returned tuple will be
(adam, adam_ask, adam_tell), where
adam
is the function that will initialize the Adam optimizer,
adam_ask
is the function that will get the current search point as a tensor, and
adam_tell
is the function that will expect the gradient and will return the updated
state of the Adam search after applying the given gradient.
In addition to "adam", the strings "clipup" and "sgd" are also supported.
If the given optimizer is a 3-element tuple, then, the three elements within the tuple are assumed to be the initialization, ask, and tell functions of a custom optimizer, and those functions are returned in the same order.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
optimizer |
Union[str, tuple] |
The optimizer name as a string, or a 3-element tuple representing the functions related to the optimizer. |
required |
Returns:
| Type | Description |
|---|---|
tuple |
A 3-element tuple in the form
|
Source code in evotorch/algorithms/functional/misc.py
def get_functional_optimizer(optimizer: Union[str, tuple]) -> tuple:
"""
Get a tuple of optimizer-related functions, from the given optimizer name.
For example, if the given string is "adam", the returned tuple will be
`(adam, adam_ask, adam_tell)`, where
[adam][evotorch.algorithms.functional.funcadam.adam]
is the function that will initialize the Adam optimizer,
[adam_ask][evotorch.algorithms.functional.funcadam.adam_ask]
is the function that will get the current search point as a tensor, and
[adam_tell][evotorch.algorithms.functional.funcadam.adam_tell]
is the function that will expect the gradient and will return the updated
state of the Adam search after applying the given gradient.
In addition to "adam", the strings "clipup" and "sgd" are also supported.
If the given optimizer is a 3-element tuple, then, the three elements
within the tuple are assumed to be the initialization, ask, and tell
functions of a custom optimizer, and those functions are returned
in the same order.
Args:
optimizer: The optimizer name as a string, or a 3-element tuple
representing the functions related to the optimizer.
Returns:
A 3-element tuple in the form
`(optimizer, optimizer_ask, optimizer_tell)`, where each element
is a function, the first one being responsible for initializing
the optimizer and returning its first state.
"""
from .funcadam import adam, adam_ask, adam_tell
from .funcclipup import clipup, clipup_ask, clipup_tell
from .funcsgd import sgd, sgd_ask, sgd_tell
if optimizer == "adam":
return OptimizerFunctions(initialize=adam, ask=adam_ask, tell=adam_tell)
elif optimizer == "clipup":
return OptimizerFunctions(initialize=clipup, ask=clipup_ask, tell=clipup_tell)
elif optimizer in ("sgd", "sga", "momentum"):
return OptimizerFunctions(initialize=sgd, ask=sgd_ask, tell=sgd_tell)
elif isinstance(optimizer, str):
raise ValueError(f"Unrecognized functional optimizer name: {optimizer}")
elif isinstance(optimizer, Iterable):
a, b, c = optimizer
return OptimizerFunctions(initialize=a, ask=b, tell=c)
else:
raise TypeError(
f"`get_functional_optimizer(...)` received an unrecognized argument: {repr(optimizer)}"
f" (of type {type(optimizer)})"
)
ga
¶
Genetic algorithm variants: GeneticAlgorithm, Cosyne.
Cosyne (SearchAlgorithm, SinglePopulationAlgorithmMixin)
¶
Implementation of the CoSyNE algorithm.
References:
F.Gomez, J.Schmidhuber, R.Miikkulainen, M.Mitchell (2008).
Accelerated Neural Evolution through Cooperatively Coevolved Synapses.
Journal of Machine Learning Research 9 (5).
Source code in evotorch/algorithms/ga.py
class Cosyne(SearchAlgorithm, SinglePopulationAlgorithmMixin):
"""
Implementation of the CoSyNE algorithm.
References:
F.Gomez, J.Schmidhuber, R.Miikkulainen, M.Mitchell (2008).
Accelerated Neural Evolution through Cooperatively Coevolved Synapses.
Journal of Machine Learning Research 9 (5).
"""
def __init__(
self,
problem: Problem,
*,
popsize: int,
tournament_size: int,
mutation_stdev: Optional[float],
mutation_probability: Optional[float] = None,
permute_all: bool = False,
num_elites: Optional[int] = None,
elitism_ratio: Optional[float] = None,
eta: Optional[float] = None,
num_children: Optional[int] = None,
):
"""
`__init__(...)`: Initialize the Cosyne instance.
Args:
problem: The problem object to work on.
popsize: Population size, as an integer.
tournament_size: Tournament size, for tournament selection.
mutation_stdev: Standard deviation of the Gaussian mutation.
See [GaussianMutation][evotorch.operators.real.GaussianMutation] for more information.
mutation_probability: Elementwise Gaussian mutation probability.
Defaults to None.
See [GaussianMutation][evotorch.operators.real.GaussianMutation] for more information.
permute_all: If given as True, all solutions are subject to
permutation. If given as False (which is the default),
there will be a selection procedure for each decision
variable.
num_elites: Optionally expected as an integer, specifying the
number of elites to pass to the next generation.
Cannot be used together with the argument `elitism_ratio`.
elitism_ratio: Optionally expected as a real number between
0 and 1, specifying the amount of elites to pass to the
next generation. For example, 0.1 means that the best 10%
of the population are accepted as elites and passed onto
the next generation.
Cannot be used together with the argument `num_elites`.
eta: Optionally expected as an integer, specifying the eta
hyperparameter for the simulated binary cross-over (SBX).
If left as None, one-point cross-over will be used instead.
num_children: Number of children to generate at each iteration.
If left as None, then this number is half of the population
size.
"""
problem.ensure_numeric()
SearchAlgorithm.__init__(self, problem)
if mutation_stdev is None:
if mutation_probability is not None:
raise ValueError(
f"`mutation_probability` was set to {mutation_probability}, but `mutation_stdev` is None, "
"which means, mutation is disabled. If you want to enable the mutation, be sure to provide "
"`mutation_stdev` as well."
)
self.mutation_op = None
else:
self.mutation_op = GaussianMutation(
self._problem,
stdev=mutation_stdev,
mutation_probability=mutation_probability,
)
cross_over_kwargs = {"tournament_size": tournament_size}
if num_children is None:
cross_over_kwargs["cross_over_rate"] = 2.0
else:
cross_over_kwargs["num_children"] = num_children
if eta is None:
self._cross_over_op = OnePointCrossOver(self._problem, **cross_over_kwargs)
else:
self._cross_over_op = SimulatedBinaryCrossOver(self._problem, eta=eta, **cross_over_kwargs)
self._permutation_op = CosynePermutation(self._problem, permute_all=permute_all)
self._popsize = int(popsize)
if num_elites is not None and elitism_ratio is None:
self._num_elites = int(num_elites)
elif num_elites is None and elitism_ratio is not None:
self._num_elites = int(self._popsize * elitism_ratio)
elif num_elites is None and elitism_ratio is None:
self._num_elites = None
else:
raise ValueError(
"Received both `num_elites` and `elitism_ratio`. Please provide only one of them, or none of them."
)
self._population = SolutionBatch(problem, device=problem.device, popsize=self._popsize)
self._first_generation: bool = True
# GAStatusMixin.__init__(self)
SinglePopulationAlgorithmMixin.__init__(self)
@property
def population(self) -> SolutionBatch:
return self._population
def _step(self):
if self._first_generation:
self._first_generation = False
self._problem.evaluate(self._population)
to_merge = []
num_elites = self._num_elites
num_parents = int(self._popsize / 4)
num_relevant = max((0 if num_elites is None else num_elites), num_parents)
sorted_relevant = self._population.take_best(num_relevant)
if self._num_elites is not None and self._num_elites >= 1:
to_merge.append(sorted_relevant[:num_elites].clone())
parents = sorted_relevant[:num_parents]
children = self._cross_over_op(parents)
if self.mutation_op is not None:
children = self.mutation_op(children)
permuted = self._permutation_op(self._population)
to_merge.extend([children, permuted])
extended_population = SolutionBatch(merging_of=to_merge)
self._problem.evaluate(extended_population)
self._population = extended_population.take_best(self._popsize)
__init__(self, problem, *, popsize, tournament_size, mutation_stdev, mutation_probability=None, permute_all=False, num_elites=None, elitism_ratio=None, eta=None, num_children=None)
special
¶
__init__(...): Initialize the Cosyne instance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object to work on. |
required |
popsize |
int |
Population size, as an integer. |
required |
tournament_size |
int |
Tournament size, for tournament selection. |
required |
mutation_stdev |
Optional[float] |
Standard deviation of the Gaussian mutation. See GaussianMutation for more information. |
required |
mutation_probability |
Optional[float] |
Elementwise Gaussian mutation probability. Defaults to None. See GaussianMutation for more information. |
None |
permute_all |
bool |
If given as True, all solutions are subject to permutation. If given as False (which is the default), there will be a selection procedure for each decision variable. |
False |
num_elites |
Optional[int] |
Optionally expected as an integer, specifying the
number of elites to pass to the next generation.
Cannot be used together with the argument |
None |
elitism_ratio |
Optional[float] |
Optionally expected as a real number between
0 and 1, specifying the amount of elites to pass to the
next generation. For example, 0.1 means that the best 10%
of the population are accepted as elites and passed onto
the next generation.
Cannot be used together with the argument |
None |
eta |
Optional[float] |
Optionally expected as an integer, specifying the eta hyperparameter for the simulated binary cross-over (SBX). If left as None, one-point cross-over will be used instead. |
None |
num_children |
Optional[int] |
Number of children to generate at each iteration. If left as None, then this number is half of the population size. |
None |
Source code in evotorch/algorithms/ga.py
def __init__(
self,
problem: Problem,
*,
popsize: int,
tournament_size: int,
mutation_stdev: Optional[float],
mutation_probability: Optional[float] = None,
permute_all: bool = False,
num_elites: Optional[int] = None,
elitism_ratio: Optional[float] = None,
eta: Optional[float] = None,
num_children: Optional[int] = None,
):
"""
`__init__(...)`: Initialize the Cosyne instance.
Args:
problem: The problem object to work on.
popsize: Population size, as an integer.
tournament_size: Tournament size, for tournament selection.
mutation_stdev: Standard deviation of the Gaussian mutation.
See [GaussianMutation][evotorch.operators.real.GaussianMutation] for more information.
mutation_probability: Elementwise Gaussian mutation probability.
Defaults to None.
See [GaussianMutation][evotorch.operators.real.GaussianMutation] for more information.
permute_all: If given as True, all solutions are subject to
permutation. If given as False (which is the default),
there will be a selection procedure for each decision
variable.
num_elites: Optionally expected as an integer, specifying the
number of elites to pass to the next generation.
Cannot be used together with the argument `elitism_ratio`.
elitism_ratio: Optionally expected as a real number between
0 and 1, specifying the amount of elites to pass to the
next generation. For example, 0.1 means that the best 10%
of the population are accepted as elites and passed onto
the next generation.
Cannot be used together with the argument `num_elites`.
eta: Optionally expected as an integer, specifying the eta
hyperparameter for the simulated binary cross-over (SBX).
If left as None, one-point cross-over will be used instead.
num_children: Number of children to generate at each iteration.
If left as None, then this number is half of the population
size.
"""
problem.ensure_numeric()
SearchAlgorithm.__init__(self, problem)
if mutation_stdev is None:
if mutation_probability is not None:
raise ValueError(
f"`mutation_probability` was set to {mutation_probability}, but `mutation_stdev` is None, "
"which means, mutation is disabled. If you want to enable the mutation, be sure to provide "
"`mutation_stdev` as well."
)
self.mutation_op = None
else:
self.mutation_op = GaussianMutation(
self._problem,
stdev=mutation_stdev,
mutation_probability=mutation_probability,
)
cross_over_kwargs = {"tournament_size": tournament_size}
if num_children is None:
cross_over_kwargs["cross_over_rate"] = 2.0
else:
cross_over_kwargs["num_children"] = num_children
if eta is None:
self._cross_over_op = OnePointCrossOver(self._problem, **cross_over_kwargs)
else:
self._cross_over_op = SimulatedBinaryCrossOver(self._problem, eta=eta, **cross_over_kwargs)
self._permutation_op = CosynePermutation(self._problem, permute_all=permute_all)
self._popsize = int(popsize)
if num_elites is not None and elitism_ratio is None:
self._num_elites = int(num_elites)
elif num_elites is None and elitism_ratio is not None:
self._num_elites = int(self._popsize * elitism_ratio)
elif num_elites is None and elitism_ratio is None:
self._num_elites = None
else:
raise ValueError(
"Received both `num_elites` and `elitism_ratio`. Please provide only one of them, or none of them."
)
self._population = SolutionBatch(problem, device=problem.device, popsize=self._popsize)
self._first_generation: bool = True
# GAStatusMixin.__init__(self)
SinglePopulationAlgorithmMixin.__init__(self)
ExtendedPopulationMixin
¶
A mixin class that provides the method _make_extended_population(...).
This mixin class assumes that the inheriting class has the properties
problem (of type Problem), which provide
and population (of type SolutionBatch),
which provide the associated problem object and the current population,
respectively.
The class which inherits this mixin class gains the method
_make_extended_population(...). This new method applies the operators
specified during the initialization phase of this mixin class on the
current population, produces children, and then returns an extended
population.
Source code in evotorch/algorithms/ga.py
class ExtendedPopulationMixin:
"""
A mixin class that provides the method `_make_extended_population(...)`.
This mixin class assumes that the inheriting class has the properties
`problem` (of type [Problem][evotorch.core.Problem]), which provide
and `population` (of type [SolutionBatch][evotorch.core.SolutionBatch]),
which provide the associated problem object and the current population,
respectively.
The class which inherits this mixin class gains the method
`_make_extended_population(...)`. This new method applies the operators
specified during the initialization phase of this mixin class on the
current population, produces children, and then returns an extended
population.
"""
def __init__(
self,
*,
re_evaluate: bool,
re_evaluate_parents_first: Optional[bool] = None,
operators: Optional[Iterable] = None,
allow_empty_operators_list: bool = False,
):
"""
`__init__(...)`: Initialize the ExtendedPopulationMixin.
Args:
re_evaluate: Whether or not to re-evaluate the parent population
at every generation. When dealing with problems where the
fitness and/or feature evaluations are stochastic, one might
want to set this as True. On the other hand, for when the
fitness and/or feature evaluations are deterministic, one
might prefer to set this as False for efficiency.
re_evaluate_parents_first: This is to be specified only when
`re_evaluate` is True (otherwise to be left as None).
If this is given as True, then it will be assumed that the
provided operators require the parents to be evaluated.
If this is given as False, then it will be assumed that the
provided operators work without looking at the parents'
fitnesses (in which case both parents and children can be
evaluated in a single vectorized computation cycle).
If this is left as None, then whether or not the operators
need to know the parent evaluations will be determined
automatically as follows:
if the operators contain at least one cross-over operator
then `re_evaluate_parents_first` will be internally set as
True; otherwise `re_evaluate_parents_first` will be internally
set as False.
operators: List of operators to apply on the current population
for generating a new extended population.
allow_empty_operators_list: Whether or not to allow the operator
list to be empty. The default and the recommended value
is False. For cases where the inheriting class wants to
decide the operators later (via the attribute `_operators`)
this can be set as True.
"""
self._operators = [] if operators is None else list(operators)
if (not allow_empty_operators_list) and (len(self._operators) == 0):
raise ValueError("Received `operators` as an empty sequence. Please provide at least one operator.")
self._using_cross_over: bool = False
for operator in self._operators:
if isinstance(operator, CrossOver):
self._using_cross_over = True
break
self._re_evaluate: bool = bool(re_evaluate)
if re_evaluate_parents_first is None:
self._re_evaluate_parents_first = self._using_cross_over
else:
if not self._re_evaluate:
raise ValueError(
"Encountered the argument `re_evaluate_parents_first` as something other than None."
" However, `re_evaluate` is given as False."
" Please use `re_evaluate_parents_first` only when `re_evaluate` is True."
)
self._re_evaluate_parents_first = bool(re_evaluate_parents_first)
self._first_iter: bool = True
def _make_extended_population(self, split: bool = False) -> Union[SolutionBatch, tuple]:
"""
Make and return a new extended population that is evaluated.
Args:
split: If False, then the extended population will be returned
as a single SolutionBatch object which contains both the
parents and the children.
If True, then the extended population will be returned
as a pair of SolutionBatch objects, the first one being
the parents and the second one being the children.
Returns:
The extended population.
"""
# Get the problem object and the population
problem: Problem = self.problem
population: SolutionBatch = self.population
if self._re_evaluate:
# This is the case where our mixin is configured to re-evaluate the parents at every generation.
# Set the first iteration indicator to False
self._first_iter = False
if self._re_evaluate_parents_first:
# This is the case where our mixin is configured to evaluate the parents separately first.
# This is a sub-case of `_re_evaluate=True`.
# Evaluate the population, which stores the parents
problem.evaluate(population)
# Now that our parents are evaluated, we use the operators on them and get the children.
children = _use_operators(population, self._operators)
# Evaluate the children
problem.evaluate(children)
if split:
# If our mixin is configured to return the population and the children, then we return a tuple
# containing them as separate items.
return population, children
else:
# If our mixin is configured to return the population and the children in a single batch,
# then we concatenate the population and the children and return the resulting combined batch.
return SolutionBatch.cat([population, children])
else:
# This is the case where our mixin is configured to evaluate the parents and the children in one go.
# This is a sub-case of `_re_evaluate=True`.
# Use the operators on the parent solutions. It does not matter whether or not the parents are evaluated.
children = _use_operators(population, self._operators)
# Form an extended population by concatenating the population and the children.
extended_population = SolutionBatch.cat([population, children])
# Evaluate the extended population in one go.
problem.evaluate(extended_population)
if split:
# The method was configured to return the parents and the children separately.
# Because we combined them earlier for evaluating them in one go, we will split them now.
# Get the number of parents
num_parents = len(population)
# Get the newly evaluated copies of the parents from the extended population
parents = extended_population[:num_parents]
# Get the children from the extended population
children = extended_population[num_parents:]
# Return the newly evaluated copies of the parents and the children separately.
return parents, children
else:
# The method was configured to return the parents and the children in a single SolutionBatch.
# Here, we just return the extended population that we already have produced.
return extended_population
else:
# This is the case where our mixin is configured NOT to re-evaluate the parents at every generation.
if self._first_iter:
# The first iteration indicator (`_first_iter`) is True. So, this is the first iteration.
# We set `_first_iter` to False for future generations.
self._first_iter = False
# We not evaluate the parent population (because the parents are expected to be non-evaluated at the
# beginning).
problem.evaluate(population)
# Here, we assume that the parents are already evaluated. We apply our operators on the parents.
children = _use_operators(population, self._operators)
# Now, we evaluate the children.
problem.evaluate(children)
if split:
# Return the population and the children separately if `split=True`.
return population, children
else:
# Return the population and the children in a single SolutionBatch if `split=False`.
return SolutionBatch.cat([population, children])
@property
def re_evaluate(self) -> bool:
"""
Whether or not this search algorithm re-evaluates the parents
"""
return self._re_evaluate
@property
def re_evaluate_parents_first(self) -> Optional[bool]:
"""
Whether or not this search algorithm re-evaluates the parents separately.
This property is relevant only when `re_evaluate` is True.
If `re_evaluate` is False, then this property will return None.
"""
if self._re_evaluate:
return self._re_evaluate_parents_first
else:
return None
re_evaluate: bool
property
readonly
¶
Whether or not this search algorithm re-evaluates the parents
re_evaluate_parents_first: Optional[bool]
property
readonly
¶
Whether or not this search algorithm re-evaluates the parents separately.
This property is relevant only when re_evaluate is True.
If re_evaluate is False, then this property will return None.
__init__(self, *, re_evaluate, re_evaluate_parents_first=None, operators=None, allow_empty_operators_list=False)
special
¶
__init__(...): Initialize the ExtendedPopulationMixin.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
re_evaluate |
bool |
Whether or not to re-evaluate the parent population at every generation. When dealing with problems where the fitness and/or feature evaluations are stochastic, one might want to set this as True. On the other hand, for when the fitness and/or feature evaluations are deterministic, one might prefer to set this as False for efficiency. |
required |
re_evaluate_parents_first |
Optional[bool] |
This is to be specified only when
|
None |
operators |
Optional[Iterable] |
List of operators to apply on the current population for generating a new extended population. |
None |
allow_empty_operators_list |
bool |
Whether or not to allow the operator
list to be empty. The default and the recommended value
is False. For cases where the inheriting class wants to
decide the operators later (via the attribute |
False |
Source code in evotorch/algorithms/ga.py
def __init__(
self,
*,
re_evaluate: bool,
re_evaluate_parents_first: Optional[bool] = None,
operators: Optional[Iterable] = None,
allow_empty_operators_list: bool = False,
):
"""
`__init__(...)`: Initialize the ExtendedPopulationMixin.
Args:
re_evaluate: Whether or not to re-evaluate the parent population
at every generation. When dealing with problems where the
fitness and/or feature evaluations are stochastic, one might
want to set this as True. On the other hand, for when the
fitness and/or feature evaluations are deterministic, one
might prefer to set this as False for efficiency.
re_evaluate_parents_first: This is to be specified only when
`re_evaluate` is True (otherwise to be left as None).
If this is given as True, then it will be assumed that the
provided operators require the parents to be evaluated.
If this is given as False, then it will be assumed that the
provided operators work without looking at the parents'
fitnesses (in which case both parents and children can be
evaluated in a single vectorized computation cycle).
If this is left as None, then whether or not the operators
need to know the parent evaluations will be determined
automatically as follows:
if the operators contain at least one cross-over operator
then `re_evaluate_parents_first` will be internally set as
True; otherwise `re_evaluate_parents_first` will be internally
set as False.
operators: List of operators to apply on the current population
for generating a new extended population.
allow_empty_operators_list: Whether or not to allow the operator
list to be empty. The default and the recommended value
is False. For cases where the inheriting class wants to
decide the operators later (via the attribute `_operators`)
this can be set as True.
"""
self._operators = [] if operators is None else list(operators)
if (not allow_empty_operators_list) and (len(self._operators) == 0):
raise ValueError("Received `operators` as an empty sequence. Please provide at least one operator.")
self._using_cross_over: bool = False
for operator in self._operators:
if isinstance(operator, CrossOver):
self._using_cross_over = True
break
self._re_evaluate: bool = bool(re_evaluate)
if re_evaluate_parents_first is None:
self._re_evaluate_parents_first = self._using_cross_over
else:
if not self._re_evaluate:
raise ValueError(
"Encountered the argument `re_evaluate_parents_first` as something other than None."
" However, `re_evaluate` is given as False."
" Please use `re_evaluate_parents_first` only when `re_evaluate` is True."
)
self._re_evaluate_parents_first = bool(re_evaluate_parents_first)
self._first_iter: bool = True
GeneticAlgorithm (SearchAlgorithm, SinglePopulationAlgorithmMixin, ExtendedPopulationMixin)
¶
A genetic algorithm implementation.
Basic usage. Let us consider a single-objective optimization problem where the goal is to minimize the L2 norm of a continuous tensor of length 10:
from evotorch import Problem
from evotorch.algorithms import GeneticAlgorithm
from evotorch.operators import OnePointCrossOver, GaussianMutation
import torch
def f(x: torch.Tensor) -> torch.Tensor:
return torch.linalg.norm(x)
problem = Problem(
"min",
f,
initial_bounds=(-10.0, 10.0),
solution_length=10,
)
For solving this problem, a genetic algorithm could be instantiated as follows:
ga = GeneticAlgorithm(
problem,
popsize=100,
operators=[
OnePointCrossOver(problem, tournament_size=4),
GaussianMutation(problem, stdev=0.1),
],
)
The genetic algorithm instantiated above is configured to have a population size of 100, and is configured to perform the following operations on the population at each generation: (i) select solutions with a tournament of size 4, and produce children from the selected solutions by applying one-point cross-over; (ii) apply a gaussian mutation on the values of the solutions produced by the previous step, the amount of the mutation being sampled according to a standard deviation of 0.1. Once instantiated, this GeneticAlgorithm instance can be used with an API compatible with other search algorithms, as shown below:
from evotorch.logging import StdOutLogger
_ = StdOutLogger(ga) # Report the evolution's progress to standard output
ga.run(100) # Run the algorithm for 100 generations
print("Solution with best fitness ever:", ga.status["best"])
print("Current population's best:", ga.status["pop_best"])
Please also note:
- The operators are always executed according to the order specified within
the
operatorsargument. - There are more operators available in the namespace evotorch.operators.
- By default, GeneticAlgorithm is elitist. In the elitist mode, an extended
population is formed from parent solutions and child solutions, and the
best n solutions of this extended population are accepted as the next
generation. If you wish to switch to a non-elitist mode (where children
unconditionally replace the worst-performing parents), you can use the
initialization argument
elitist=False. - It is not mandatory to specify a cross-over operator. When a cross-over operator is missing, the GeneticAlgorithm will work like a simple evolution strategy implementation which produces children by mutating the parents, and then replaces the parents (where the criteria for replacing the parents depend on whether or not elitism is enabled).
- To be able to deal with stochastic fitness functions correctly,
GeneticAlgorithm re-evaluates previously evaluated parents as well.
When you are sure that the fitness function is deterministic,
you can pass the initialization argument
re_evaluate=Falseto prevent unnecessary computations.
Integer decision variables.
GeneticAlgorithm can be used on problems with dtype declared as integer
(e.g. torch.int32, torch.int64, etc.).
Within the field of discrete optimization, it is common to encounter
one or more of these scenarios:
- The search space of the problem has a special structure that one will wish to exploit (within the cross-over and/or mutation operators) to be able to reach the (near-)optimum within a reasonable amount of time.
- The problem is partially or fully combinatorial.
- The problem is constrained in such a way that arbitrarily sampling discrete values for its decision variables might cause infeasibility.
Considering all these scenarios, it is difficult to come up with general cross-over and mutation operators that will work across various discrete optimization problems, and it is common to design problem-specific operators. In EvoTorch, it is possible to define custom operators and use them with GeneticAlgorithm, which is required when using GeneticAlgorithm on a problem with a non-float dtype.
As an example, let us consider the following discrete optimization problem:
def f(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x)
problem = Problem(
"min",
f,
bounds=(-10, 10),
solution_length=10,
dtype=torch.int64,
)
Although EvoTorch does provide a very simple and generic (usable with float and int dtypes) cross-over named OnePointCrossOver (a cross-over which randomly decides a cutting point for each pair of parents, cuts them from those points and recombines them), it can be desirable and necessary to implement a custom cross-over operator. One can inherit from CrossOver to define a custom cross-over operator, as shown below:
from evotorch import SolutionBatch
from evotorch.operators import CrossOver
class CustomCrossOver(CrossOver):
def _do_cross_over(
self,
parents1: torch.Tensor,
parents2: torch.Tensor,
) -> SolutionBatch:
# parents1 is a tensor storing the decision values of the first
# half of the chosen parents.
# parents2 is a tensor storing the decision values of the second
# half of the chosen parents.
# We expect that the lengths of parents1 and parents2 are equal.
assert len(parents1) == len(parents2)
# Allocate an empty SolutionBatch that will store the children
childpop = SolutionBatch(self.problem, popsize=num_parents, empty=True)
# Gain access to the decision values tensor of the newly allocated
# childpop
childpop_values = childpop.access_values()
# Here we somehow fill `childpop_values` by recombining the parents.
# The most common thing to do is to produce two children by
# combining parents1[0] and parents2[0], to produce the next two
# children parents1[1] and parents2[1], and so on.
childpop_values[:] = ...
# Return the child population
return childpop
One can define a custom mutation operator by inheriting from Operator, as shown below:
class CustomMutation(Operator):
def _do(self, solutions: SolutionBatch):
# Get the decision values tensor of the solutions
sln_values = solutions.access_values()
# do in-place modifications to the decision values
sln_values[:] = ...
Alternatively, you could define the mutation operator as a function:
def my_mutation_function(original_values: torch.Tensor) -> torch.Tensor:
# Somehow produce mutated copies of the original values
mutated_values = ...
# Return the mutated values
return mutated_values
With these defined operators, we are now ready to instantiate our GeneticAlgorithm:
ga = GeneticAlgorithm(
problem,
popsize=100,
operators=[
CustomCrossOver(problem, tournament_size=4),
CustomMutation(problem),
# -- or, if you chose to define the mutation as a function: --
# my_mutation_function,
],
)
Non-numeric or variable-length solutions.
GeneticAlgorithm can also work on problems whose dtype is declared
as object, where dtype=object means that a solution's value(s) can be
expressed via a tensor, a numpy array, a scalar, a tuple, a list, a
dictionary.
Like in the previously discussed case (where dtype is an integer type),
one has to define custom operators when working on problems with
dtype=object. A custom cross-over definition specialized for
dtype=object looks like this:
from evotorch.tools import ObjectArray
class CrossOverForObjectDType(CrossOver):
def _do_cross_over(
self,
parents1: ObjectArray,
parents2: ObjectArray,
) -> SolutionBatch:
# Allocate an empty SolutionBatch that will store the children
childpop = SolutionBatch(self.problem, popsize=num_parents, empty=True)
# Gain access to the decision values ObjectArray of the newly allocated
# childpop
childpop_values = childpop.access_values()
# Here we somehow fill `childpop_values` by recombining the parents.
# The most common thing to do is to produce two children by
# combining parents1[0] and parents2[0], to produce the next two
# children parents1[1] and parents2[1], and so on.
childpop_values[:] = ...
# Return the child population
return childpop
A custom mutation operator specialized for dtype=object looks like this:
class MutationForObjectDType(Operator):
def _do(self, solutions: SolutionBatch):
# Get the decision values ObjectArray of the solutions
sln_values = solutions.access_values()
# do in-place modifications to the decision values
sln_values[:] = ...
A custom mutation function specialized for dtype=object looks like this:
def mutation_for_object_dtype(original_values: ObjectArray) -> ObjectArray:
# Somehow produce mutated copies of the original values
mutated_values = ...
# Return the mutated values
return mutated_values
With these operators defined, one can instantiate the GeneticAlgorithm:
ga = GeneticAlgorithm(
problem_with_object_dtype,
popsize=100,
operators=[
CrossOverForObjectDType(problem_with_object_dtype, tournament_size=4),
MutationForObjectDType(problem_with_object_dtype),
# -- or, if you chose to define the mutation as a function: --
# mutation_for_object_dtype,
],
)
Multiple objectives. GeneticAlgorithm can work on problems with multiple objectives. When there are multiple objectives, GeneticAlgorithm will compare the solutions according to their pareto-ranks and their crowding distances, like done by the NSGA-II algorithm (Deb, 2002).
References:
Sean Luke, 2013, Essentials of Metaheuristics, Lulu, second edition
available for free at http://cs.gmu.edu/~sean/book/metaheuristics/
Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, T. Meyarivan (2002).
A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II.
Source code in evotorch/algorithms/ga.py
class GeneticAlgorithm(SearchAlgorithm, SinglePopulationAlgorithmMixin, ExtendedPopulationMixin):
"""
A genetic algorithm implementation.
**Basic usage.**
Let us consider a single-objective optimization problem where the goal is to
minimize the L2 norm of a continuous tensor of length 10:
```python
from evotorch import Problem
from evotorch.algorithms import GeneticAlgorithm
from evotorch.operators import OnePointCrossOver, GaussianMutation
import torch
def f(x: torch.Tensor) -> torch.Tensor:
return torch.linalg.norm(x)
problem = Problem(
"min",
f,
initial_bounds=(-10.0, 10.0),
solution_length=10,
)
```
For solving this problem, a genetic algorithm could be instantiated as
follows:
```python
ga = GeneticAlgorithm(
problem,
popsize=100,
operators=[
OnePointCrossOver(problem, tournament_size=4),
GaussianMutation(problem, stdev=0.1),
],
)
```
The genetic algorithm instantiated above is configured to have a population
size of 100, and is configured to perform the following operations on the
population at each generation:
(i) select solutions with a tournament of size 4, and produce children from
the selected solutions by applying one-point cross-over;
(ii) apply a gaussian mutation on the values of the solutions produced by
the previous step, the amount of the mutation being sampled according to a
standard deviation of 0.1.
Once instantiated, this GeneticAlgorithm instance can be used with an API
compatible with other search algorithms, as shown below:
```python
from evotorch.logging import StdOutLogger
_ = StdOutLogger(ga) # Report the evolution's progress to standard output
ga.run(100) # Run the algorithm for 100 generations
print("Solution with best fitness ever:", ga.status["best"])
print("Current population's best:", ga.status["pop_best"])
```
Please also note:
- The operators are always executed according to the order specified within
the `operators` argument.
- There are more operators available in the namespace
[evotorch.operators][evotorch.operators].
- By default, GeneticAlgorithm is elitist. In the elitist mode, an extended
population is formed from parent solutions and child solutions, and the
best n solutions of this extended population are accepted as the next
generation. If you wish to switch to a non-elitist mode (where children
unconditionally replace the worst-performing parents), you can use the
initialization argument `elitist=False`.
- It is not mandatory to specify a cross-over operator. When a cross-over
operator is missing, the GeneticAlgorithm will work like a simple
evolution strategy implementation which produces children by mutating
the parents, and then replaces the parents (where the criteria for
replacing the parents depend on whether or not elitism is enabled).
- To be able to deal with stochastic fitness functions correctly,
GeneticAlgorithm re-evaluates previously evaluated parents as well.
When you are sure that the fitness function is deterministic,
you can pass the initialization argument `re_evaluate=False` to prevent
unnecessary computations.
**Integer decision variables.**
GeneticAlgorithm can be used on problems with `dtype` declared as integer
(e.g. `torch.int32`, `torch.int64`, etc.).
Within the field of discrete optimization, it is common to encounter
one or more of these scenarios:
- The search space of the problem has a special structure that one will
wish to exploit (within the cross-over and/or mutation operators) to
be able to reach the (near-)optimum within a reasonable amount of time.
- The problem is partially or fully combinatorial.
- The problem is constrained in such a way that arbitrarily sampling
discrete values for its decision variables might cause infeasibility.
Considering all these scenarios, it is difficult to come up with general
cross-over and mutation operators that will work across various discrete
optimization problems, and it is common to design problem-specific
operators. In EvoTorch, it is possible to define custom operators and
use them with GeneticAlgorithm, which is required when using
GeneticAlgorithm on a problem with a non-float dtype.
As an example, let us consider the following discrete optimization
problem:
```python
def f(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x)
problem = Problem(
"min",
f,
bounds=(-10, 10),
solution_length=10,
dtype=torch.int64,
)
```
Although EvoTorch does provide a very simple and generic (usable with float
and int dtypes) cross-over named
[OnePointCrossOver][evotorch.operators.real.OnePointCrossOver]
(a cross-over which randomly decides a cutting point for each pair of
parents, cuts them from those points and recombines them), it can be
desirable and necessary to implement a custom cross-over operator.
One can inherit from [CrossOver][evotorch.operators.base.CrossOver] to
define a custom cross-over operator, as shown below:
```python
from evotorch import SolutionBatch
from evotorch.operators import CrossOver
class CustomCrossOver(CrossOver):
def _do_cross_over(
self,
parents1: torch.Tensor,
parents2: torch.Tensor,
) -> SolutionBatch:
# parents1 is a tensor storing the decision values of the first
# half of the chosen parents.
# parents2 is a tensor storing the decision values of the second
# half of the chosen parents.
# We expect that the lengths of parents1 and parents2 are equal.
assert len(parents1) == len(parents2)
# Allocate an empty SolutionBatch that will store the children
childpop = SolutionBatch(self.problem, popsize=num_parents, empty=True)
# Gain access to the decision values tensor of the newly allocated
# childpop
childpop_values = childpop.access_values()
# Here we somehow fill `childpop_values` by recombining the parents.
# The most common thing to do is to produce two children by
# combining parents1[0] and parents2[0], to produce the next two
# children parents1[1] and parents2[1], and so on.
childpop_values[:] = ...
# Return the child population
return childpop
```
One can define a custom mutation operator by inheriting from
[Operator][evotorch.operators.base.Operator], as shown below:
```python
class CustomMutation(Operator):
def _do(self, solutions: SolutionBatch):
# Get the decision values tensor of the solutions
sln_values = solutions.access_values()
# do in-place modifications to the decision values
sln_values[:] = ...
```
Alternatively, you could define the mutation operator as a function:
```python
def my_mutation_function(original_values: torch.Tensor) -> torch.Tensor:
# Somehow produce mutated copies of the original values
mutated_values = ...
# Return the mutated values
return mutated_values
```
With these defined operators, we are now ready to instantiate our
GeneticAlgorithm:
```python
ga = GeneticAlgorithm(
problem,
popsize=100,
operators=[
CustomCrossOver(problem, tournament_size=4),
CustomMutation(problem),
# -- or, if you chose to define the mutation as a function: --
# my_mutation_function,
],
)
```
**Non-numeric or variable-length solutions.**
GeneticAlgorithm can also work on problems whose `dtype` is declared
as `object`, where `dtype=object` means that a solution's value(s) can be
expressed via a tensor, a numpy array, a scalar, a tuple, a list, a
dictionary.
Like in the previously discussed case (where dtype is an integer type),
one has to define custom operators when working on problems with
`dtype=object`. A custom cross-over definition specialized for
`dtype=object` looks like this:
```python
from evotorch.tools import ObjectArray
class CrossOverForObjectDType(CrossOver):
def _do_cross_over(
self,
parents1: ObjectArray,
parents2: ObjectArray,
) -> SolutionBatch:
# Allocate an empty SolutionBatch that will store the children
childpop = SolutionBatch(self.problem, popsize=num_parents, empty=True)
# Gain access to the decision values ObjectArray of the newly allocated
# childpop
childpop_values = childpop.access_values()
# Here we somehow fill `childpop_values` by recombining the parents.
# The most common thing to do is to produce two children by
# combining parents1[0] and parents2[0], to produce the next two
# children parents1[1] and parents2[1], and so on.
childpop_values[:] = ...
# Return the child population
return childpop
```
A custom mutation operator specialized for `dtype=object` looks like this:
```python
class MutationForObjectDType(Operator):
def _do(self, solutions: SolutionBatch):
# Get the decision values ObjectArray of the solutions
sln_values = solutions.access_values()
# do in-place modifications to the decision values
sln_values[:] = ...
```
A custom mutation function specialized for `dtype=object` looks like this:
```python
def mutation_for_object_dtype(original_values: ObjectArray) -> ObjectArray:
# Somehow produce mutated copies of the original values
mutated_values = ...
# Return the mutated values
return mutated_values
```
With these operators defined, one can instantiate the GeneticAlgorithm:
```python
ga = GeneticAlgorithm(
problem_with_object_dtype,
popsize=100,
operators=[
CrossOverForObjectDType(problem_with_object_dtype, tournament_size=4),
MutationForObjectDType(problem_with_object_dtype),
# -- or, if you chose to define the mutation as a function: --
# mutation_for_object_dtype,
],
)
```
**Multiple objectives.**
GeneticAlgorithm can work on problems with multiple objectives.
When there are multiple objectives, GeneticAlgorithm will compare the
solutions according to their pareto-ranks and their crowding distances,
like done by the NSGA-II algorithm (Deb, 2002).
References:
Sean Luke, 2013, Essentials of Metaheuristics, Lulu, second edition
available for free at http://cs.gmu.edu/~sean/book/metaheuristics/
Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, T. Meyarivan (2002).
A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II.
"""
def __init__(
self,
problem: Problem,
*,
operators: Iterable,
popsize: int,
elitist: bool = True,
re_evaluate: bool = True,
re_evaluate_parents_first: Optional[bool] = None,
_allow_empty_operator_list: bool = False,
):
"""
`__init__(...)`: Initialize the GeneticAlgorithm.
Args:
problem: The problem to optimize.
operators: Operators to be used by the genetic algorithm.
Expected as an iterable, such as a list or a tuple.
Each item within this iterable object is expected either
as an instance of [Operator][evotorch.operators.base.Operator],
or as a function which receives the decision values of
multiple solutions in a PyTorch tensor (or in an
[ObjectArray][evotorch.tools.objectarray.ObjectArray]
for when dtype is `object`) and returns a modified copy.
popsize: Population size.
elitist: Whether or not this genetic algorithm will behave in an
elitist manner. This argument controls how the genetic
algorithm will form the next generation from the parents
and the children. In elitist mode (i.e. with `elitist=True`),
the procedure to be followed by this genetic algorithm is:
(i) form an extended population which consists of
both the parents and the children,
(ii) sort the extended population from best to worst,
(iii) select the best `n` solutions as the new generation where
`n` is `popsize`.
In non-elitist mode (i.e. with `elitist=False`), the worst `m`
solutions within the parent population are replaced with
the children, `m` being the number of produced children.
re_evaluate: Whether or not to evaluate the solutions
that were already evaluated in the previous generations.
By default, this is set as True.
The reason behind this default setting is that,
in problems where the evaluation procedure is noisy,
by re-evaluating the already-evaluated solutions,
we prevent the bad solutions that were luckily evaluated
from hanging onto the population.
Instead, at every generation, each solution must go through
the evaluation procedure again and prove their worth.
For problems whose evaluation procedures are NOT noisy,
the user might consider turning re_evaluate to False
for saving computational cycles.
re_evaluate_parents_first: This is to be specified only when
`re_evaluate` is True (otherwise to be left as None).
If this is given as True, then it will be assumed that the
provided operators require the parents to be evaluated.
If this is given as False, then it will be assumed that the
provided operators work without looking at the parents'
fitnesses (in which case both parents and children can be
evaluated in a single vectorized computation cycle).
If this is left as None, then whether or not the operators
need to know the parent evaluations will be determined
automatically as follows:
if the operators contain at least one cross-over operator
then `re_evaluate_parents_first` will be internally set as
True; otherwise `re_evaluate_parents_first` will be internally
set as False.
"""
SearchAlgorithm.__init__(self, problem)
self._popsize = int(popsize)
self._elitist: bool = bool(elitist)
self._population = problem.generate_batch(self._popsize)
ExtendedPopulationMixin.__init__(
self,
re_evaluate=re_evaluate,
re_evaluate_parents_first=re_evaluate_parents_first,
operators=operators,
allow_empty_operators_list=_allow_empty_operator_list,
)
SinglePopulationAlgorithmMixin.__init__(self)
@property
def population(self) -> SolutionBatch:
"""Get the population"""
return self._population
def _step(self):
# Get the population size
popsize = self._popsize
if self._elitist:
# This is where we handle the elitist mode.
# Produce and get an extended population in a single SolutionBatch
extended_population = self._make_extended_population(split=False)
# From the extended population, take the best n solutions, n being the popsize.
self._population = extended_population.take_best(popsize)
else:
# This is where we handle the non-elitist mode.
# Take the parent solutions (ensured to be evaluated) and the children separately.
parents, children = self._make_extended_population(split=True)
# Get the number of children
num_children = len(children)
if num_children < popsize:
# If the number of children is less than the population size, then we keep the best m solutions from
# the parents, m being `popsize - num_children`.
chosen_parents = self._population.take_best(popsize - num_children)
# Combine the children with the chosen parents, and declare them as the new population.
self._population = SolutionBatch.cat([chosen_parents, children])
elif num_children == popsize:
# If the number of children is the same with the population size, then these children are declared as
# the new population.
self._population = children
else:
# If the number of children is more than the population size, then we take the best n solutions from
# these children, n being the population size. These chosen children are then declared as the new
# population.
self._population = children.take_best(self._popsize)
population: SolutionBatch
property
readonly
¶
Get the population
__init__(self, problem, *, operators, popsize, elitist=True, re_evaluate=True, re_evaluate_parents_first=None, _allow_empty_operator_list=False)
special
¶
__init__(...): Initialize the GeneticAlgorithm.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem to optimize. |
required |
operators |
Iterable |
Operators to be used by the genetic algorithm.
Expected as an iterable, such as a list or a tuple.
Each item within this iterable object is expected either
as an instance of Operator,
or as a function which receives the decision values of
multiple solutions in a PyTorch tensor (or in an
ObjectArray
for when dtype is |
required |
popsize |
int |
Population size. |
required |
elitist |
bool |
Whether or not this genetic algorithm will behave in an
elitist manner. This argument controls how the genetic
algorithm will form the next generation from the parents
and the children. In elitist mode (i.e. with |
True |
re_evaluate |
bool |
Whether or not to evaluate the solutions that were already evaluated in the previous generations. By default, this is set as True. The reason behind this default setting is that, in problems where the evaluation procedure is noisy, by re-evaluating the already-evaluated solutions, we prevent the bad solutions that were luckily evaluated from hanging onto the population. Instead, at every generation, each solution must go through the evaluation procedure again and prove their worth. For problems whose evaluation procedures are NOT noisy, the user might consider turning re_evaluate to False for saving computational cycles. |
True |
re_evaluate_parents_first |
Optional[bool] |
This is to be specified only when
|
None |
Source code in evotorch/algorithms/ga.py
def __init__(
self,
problem: Problem,
*,
operators: Iterable,
popsize: int,
elitist: bool = True,
re_evaluate: bool = True,
re_evaluate_parents_first: Optional[bool] = None,
_allow_empty_operator_list: bool = False,
):
"""
`__init__(...)`: Initialize the GeneticAlgorithm.
Args:
problem: The problem to optimize.
operators: Operators to be used by the genetic algorithm.
Expected as an iterable, such as a list or a tuple.
Each item within this iterable object is expected either
as an instance of [Operator][evotorch.operators.base.Operator],
or as a function which receives the decision values of
multiple solutions in a PyTorch tensor (or in an
[ObjectArray][evotorch.tools.objectarray.ObjectArray]
for when dtype is `object`) and returns a modified copy.
popsize: Population size.
elitist: Whether or not this genetic algorithm will behave in an
elitist manner. This argument controls how the genetic
algorithm will form the next generation from the parents
and the children. In elitist mode (i.e. with `elitist=True`),
the procedure to be followed by this genetic algorithm is:
(i) form an extended population which consists of
both the parents and the children,
(ii) sort the extended population from best to worst,
(iii) select the best `n` solutions as the new generation where
`n` is `popsize`.
In non-elitist mode (i.e. with `elitist=False`), the worst `m`
solutions within the parent population are replaced with
the children, `m` being the number of produced children.
re_evaluate: Whether or not to evaluate the solutions
that were already evaluated in the previous generations.
By default, this is set as True.
The reason behind this default setting is that,
in problems where the evaluation procedure is noisy,
by re-evaluating the already-evaluated solutions,
we prevent the bad solutions that were luckily evaluated
from hanging onto the population.
Instead, at every generation, each solution must go through
the evaluation procedure again and prove their worth.
For problems whose evaluation procedures are NOT noisy,
the user might consider turning re_evaluate to False
for saving computational cycles.
re_evaluate_parents_first: This is to be specified only when
`re_evaluate` is True (otherwise to be left as None).
If this is given as True, then it will be assumed that the
provided operators require the parents to be evaluated.
If this is given as False, then it will be assumed that the
provided operators work without looking at the parents'
fitnesses (in which case both parents and children can be
evaluated in a single vectorized computation cycle).
If this is left as None, then whether or not the operators
need to know the parent evaluations will be determined
automatically as follows:
if the operators contain at least one cross-over operator
then `re_evaluate_parents_first` will be internally set as
True; otherwise `re_evaluate_parents_first` will be internally
set as False.
"""
SearchAlgorithm.__init__(self, problem)
self._popsize = int(popsize)
self._elitist: bool = bool(elitist)
self._population = problem.generate_batch(self._popsize)
ExtendedPopulationMixin.__init__(
self,
re_evaluate=re_evaluate,
re_evaluate_parents_first=re_evaluate_parents_first,
operators=operators,
allow_empty_operators_list=_allow_empty_operator_list,
)
SinglePopulationAlgorithmMixin.__init__(self)
SteadyStateGA (GeneticAlgorithm)
¶
Thin wrapper around GeneticAlgorithm for compatibility with old code.
This SteadyStateGA class is equivalent to
GeneticAlgorithm except that
SteadyStateGA provides an additional method named use(...) for
specifying a cross-over and/or a mutation operator.
The method use(...) exists only for API compatibility with the previous
versions of EvoTorch. It is recommended to specify the operators via
the keyword argument operators instead.
Source code in evotorch/algorithms/ga.py
class SteadyStateGA(GeneticAlgorithm):
"""
Thin wrapper around GeneticAlgorithm for compatibility with old code.
This `SteadyStateGA` class is equivalent to
[GeneticAlgorithm][evotorch.algorithms.ga.GeneticAlgorithm] except that
`SteadyStateGA` provides an additional method named `use(...)` for
specifying a cross-over and/or a mutation operator.
The method `use(...)` exists only for API compatibility with the previous
versions of EvoTorch. It is recommended to specify the operators via
the keyword argument `operators` instead.
"""
def __init__(
self,
problem: Problem,
*,
popsize: int,
operators: Optional[Iterable] = None,
elitist: bool = True,
re_evaluate: bool = True,
re_evaluate_parents_first: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the SteadyStateGA.
Args:
problem: The problem to optimize.
operators: Optionally, an iterable of operators to be used by the
genetic algorithm. Each item within this iterable object is
expected either as an instance of
[Operator][evotorch.operators.base.Operator],
or as a function which receives the decision values of
multiple solutions in a PyTorch tensor (or in an
[ObjectArray][evotorch.tools.objectarray.ObjectArray]
for when dtype is `object`) and returns a modified copy.
If this is omitted, then it will be required to specify the
operators via the `use(...)` method.
popsize: Population size.
elitist: Whether or not this genetic algorithm will behave in an
elitist manner. This argument controls how the genetic
algorithm will form the next generation from the parents
and the children. In elitist mode (i.e. with `elitist=True`),
the procedure to be followed by this genetic algorithm is:
(i) form an extended population which consists of
both the parents and the children,
(ii) sort the extended population from best to worst,
(iii) select the best `n` solutions as the new generation where
`n` is `popsize`.
In non-elitist mode (i.e. with `elitist=False`), the worst `m`
solutions within the parent population are replaced with
the children, `m` being the number of produced children.
re_evaluate: Whether or not to evaluate the solutions
that were already evaluated in the previous generations.
By default, this is set as True.
The reason behind this default setting is that,
in problems where the evaluation procedure is noisy,
by re-evaluating the already-evaluated solutions,
we prevent the bad solutions that were luckily evaluated
from hanging onto the population.
Instead, at every generation, each solution must go through
the evaluation procedure again and prove their worth.
For problems whose evaluation procedures are NOT noisy,
the user might consider turning re_evaluate to False
for saving computational cycles.
re_evaluate_parents_first: This is to be specified only when
`re_evaluate` is True (otherwise to be left as None).
If this is given as True, then it will be assumed that the
provided operators require the parents to be evaluated.
If this is given as False, then it will be assumed that the
provided operators work without looking at the parents'
fitnesses (in which case both parents and children can be
evaluated in a single vectorized computation cycle).
If this is left as None, then whether or not the operators
need to know the parent evaluations will be determined
automatically as follows:
if the operators contain at least one cross-over operator
then `re_evaluate_parents_first` will be internally set as
True; otherwise `re_evaluate_parents_first` will be internally
set as False.
Additional note specific to `SteadyStateGA`: if the argument
`operators` is not given (or is given as an empty list), and
also `re_evaluate_parents_first` is left as None, then
`SteadyStateGA` will assume that the operators will be later
given via the `use(...)` method, and that these operators will
require the parents to be evaluated first (equivalent to
setting `re_evaluate_parents_first` as True).
"""
if operators is None:
operators = []
self._cross_over_op: Optional[Callable] = None
self._mutation_op: Optional[Callable] = None
self._forbid_use_method: bool = False
self._prepare_ops: bool = False
if (len(operators) == 0) and re_evaluate and (re_evaluate_parents_first is None):
re_evaluate_parents_first = True
super().__init__(
problem,
operators=operators,
popsize=popsize,
elitist=elitist,
re_evaluate=re_evaluate,
re_evaluate_parents_first=re_evaluate_parents_first,
_allow_empty_operator_list=True,
)
def use(self, operator: Callable):
"""
Specify the cross-over or the mutation operator to use.
This method exists for compatibility with previous EvoTorch code.
Instead of using this method, it is recommended to specify the
operators via the `operators` keyword argument while initializing
this class.
Using this method, one can specify one cross-over operator and one
mutation operator that will be used during the evolutionary search.
Specifying multiple cross-over operators or multiple mutation operators
is not allowed. When the cross-over and mutation operators are
specified via `use(...)`, the order of execution will always be
arranged such that the cross-over comes first and the mutation comes
comes second. If desired, one can specify only the cross-over operator
or only the mutation operator.
Please note that the `operators` keyword argument works differently,
and offers more flexibility for defining the procedure to follow at
each generation. In more details, the `operators` keyword argument
allows one to specify multiple cross-over and/or multiple mutation
operators, and those operators will be executed in the specified
order.
Args:
operator: The operator to be registered to SteadyStateGA.
If the specified operator is cross-over (i.e. an instance
of [CrossOver][evotorch.operators.base.CrossOver]),
then this operator will be registered for the cross-over
phase. If the specified operator is an operator that is
not of the cross-over type (i.e. any instance of
[Operator][evotorch.operators.base.Operator] that is not
[CrossOver][evotorch.operators.base.CrossOver]) or if it is
just a function which receives the decision values as a PyTorch
tensor (or, in the case where `dtype` of the problem is
`object` as an instance of
[ObjectArray][evotorch.tools.objectarray.ObjectArray]) and
returns a modified copy, then that operator will be registered
for the mutation phase of the genetic algorithm.
"""
if self._forbid_use_method:
raise RuntimeError(
"The method `use(...)` cannot be called anymore, because the evolutionary search has started."
)
if len(self._operators) > 0:
raise RuntimeError(
f"The method `use(...)` cannot be called"
f" because an operator list was provided while initializing this {type(self).__name__} instance."
)
if isinstance(operator, CrossOver):
if self._cross_over_op is not None:
raise ValueError(
f"The method `use(...)` received this cross-over operator as its argument:"
f" {operator} (of type {type(operator)})."
f" However, a cross-over operator was already set:"
f" {self._cross_over_op} (of type {type(self._cross_over_op)})."
)
self._cross_over_op = operator
self._prepare_ops = True
else:
if self._mutation_op is not None:
raise ValueError(
f"The method `use(...)` received this mutation operator as its argument:"
f" {operator} (of type {type(operator)})."
f" However, a mutation operator was already set:"
f" {self._mutation_op} (of type {type(self._mutation_op)})."
)
self._mutation_op = operator
self._prepare_ops = True
def _step(self):
self._forbid_use_method = True
if self._prepare_ops:
self._prepare_ops = False
if self._cross_over_op is not None:
self._operators.append(self._cross_over_op)
if self._mutation_op is not None:
self._operators.append(self._mutation_op)
else:
if len(self._operators) == 0:
raise RuntimeError(
f"This {type(self).__name__} instance does not know how to proceed, "
f" because neither the `operators` keyword argument was used during initialization"
f" nor was the `use(...)` method called later."
)
super()._step()
__init__(self, problem, *, popsize, operators=None, elitist=True, re_evaluate=True, re_evaluate_parents_first=None)
special
¶
__init__(...): Initialize the SteadyStateGA.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem to optimize. |
required |
operators |
Optional[Iterable] |
Optionally, an iterable of operators to be used by the
genetic algorithm. Each item within this iterable object is
expected either as an instance of
Operator,
or as a function which receives the decision values of
multiple solutions in a PyTorch tensor (or in an
ObjectArray
for when dtype is |
None |
popsize |
int |
Population size. |
required |
elitist |
bool |
Whether or not this genetic algorithm will behave in an
elitist manner. This argument controls how the genetic
algorithm will form the next generation from the parents
and the children. In elitist mode (i.e. with |
True |
re_evaluate |
bool |
Whether or not to evaluate the solutions that were already evaluated in the previous generations. By default, this is set as True. The reason behind this default setting is that, in problems where the evaluation procedure is noisy, by re-evaluating the already-evaluated solutions, we prevent the bad solutions that were luckily evaluated from hanging onto the population. Instead, at every generation, each solution must go through the evaluation procedure again and prove their worth. For problems whose evaluation procedures are NOT noisy, the user might consider turning re_evaluate to False for saving computational cycles. |
True |
re_evaluate_parents_first |
Optional[bool] |
This is to be specified only when
|
None |
Source code in evotorch/algorithms/ga.py
def __init__(
self,
problem: Problem,
*,
popsize: int,
operators: Optional[Iterable] = None,
elitist: bool = True,
re_evaluate: bool = True,
re_evaluate_parents_first: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the SteadyStateGA.
Args:
problem: The problem to optimize.
operators: Optionally, an iterable of operators to be used by the
genetic algorithm. Each item within this iterable object is
expected either as an instance of
[Operator][evotorch.operators.base.Operator],
or as a function which receives the decision values of
multiple solutions in a PyTorch tensor (or in an
[ObjectArray][evotorch.tools.objectarray.ObjectArray]
for when dtype is `object`) and returns a modified copy.
If this is omitted, then it will be required to specify the
operators via the `use(...)` method.
popsize: Population size.
elitist: Whether or not this genetic algorithm will behave in an
elitist manner. This argument controls how the genetic
algorithm will form the next generation from the parents
and the children. In elitist mode (i.e. with `elitist=True`),
the procedure to be followed by this genetic algorithm is:
(i) form an extended population which consists of
both the parents and the children,
(ii) sort the extended population from best to worst,
(iii) select the best `n` solutions as the new generation where
`n` is `popsize`.
In non-elitist mode (i.e. with `elitist=False`), the worst `m`
solutions within the parent population are replaced with
the children, `m` being the number of produced children.
re_evaluate: Whether or not to evaluate the solutions
that were already evaluated in the previous generations.
By default, this is set as True.
The reason behind this default setting is that,
in problems where the evaluation procedure is noisy,
by re-evaluating the already-evaluated solutions,
we prevent the bad solutions that were luckily evaluated
from hanging onto the population.
Instead, at every generation, each solution must go through
the evaluation procedure again and prove their worth.
For problems whose evaluation procedures are NOT noisy,
the user might consider turning re_evaluate to False
for saving computational cycles.
re_evaluate_parents_first: This is to be specified only when
`re_evaluate` is True (otherwise to be left as None).
If this is given as True, then it will be assumed that the
provided operators require the parents to be evaluated.
If this is given as False, then it will be assumed that the
provided operators work without looking at the parents'
fitnesses (in which case both parents and children can be
evaluated in a single vectorized computation cycle).
If this is left as None, then whether or not the operators
need to know the parent evaluations will be determined
automatically as follows:
if the operators contain at least one cross-over operator
then `re_evaluate_parents_first` will be internally set as
True; otherwise `re_evaluate_parents_first` will be internally
set as False.
Additional note specific to `SteadyStateGA`: if the argument
`operators` is not given (or is given as an empty list), and
also `re_evaluate_parents_first` is left as None, then
`SteadyStateGA` will assume that the operators will be later
given via the `use(...)` method, and that these operators will
require the parents to be evaluated first (equivalent to
setting `re_evaluate_parents_first` as True).
"""
if operators is None:
operators = []
self._cross_over_op: Optional[Callable] = None
self._mutation_op: Optional[Callable] = None
self._forbid_use_method: bool = False
self._prepare_ops: bool = False
if (len(operators) == 0) and re_evaluate and (re_evaluate_parents_first is None):
re_evaluate_parents_first = True
super().__init__(
problem,
operators=operators,
popsize=popsize,
elitist=elitist,
re_evaluate=re_evaluate,
re_evaluate_parents_first=re_evaluate_parents_first,
_allow_empty_operator_list=True,
)
use(self, operator)
¶
Specify the cross-over or the mutation operator to use.
This method exists for compatibility with previous EvoTorch code.
Instead of using this method, it is recommended to specify the
operators via the operators keyword argument while initializing
this class.
Using this method, one can specify one cross-over operator and one
mutation operator that will be used during the evolutionary search.
Specifying multiple cross-over operators or multiple mutation operators
is not allowed. When the cross-over and mutation operators are
specified via use(...), the order of execution will always be
arranged such that the cross-over comes first and the mutation comes
comes second. If desired, one can specify only the cross-over operator
or only the mutation operator.
Please note that the operators keyword argument works differently,
and offers more flexibility for defining the procedure to follow at
each generation. In more details, the operators keyword argument
allows one to specify multiple cross-over and/or multiple mutation
operators, and those operators will be executed in the specified
order.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
operator |
Callable |
The operator to be registered to SteadyStateGA.
If the specified operator is cross-over (i.e. an instance
of CrossOver),
then this operator will be registered for the cross-over
phase. If the specified operator is an operator that is
not of the cross-over type (i.e. any instance of
Operator that is not
CrossOver) or if it is
just a function which receives the decision values as a PyTorch
tensor (or, in the case where |
required |
Source code in evotorch/algorithms/ga.py
def use(self, operator: Callable):
"""
Specify the cross-over or the mutation operator to use.
This method exists for compatibility with previous EvoTorch code.
Instead of using this method, it is recommended to specify the
operators via the `operators` keyword argument while initializing
this class.
Using this method, one can specify one cross-over operator and one
mutation operator that will be used during the evolutionary search.
Specifying multiple cross-over operators or multiple mutation operators
is not allowed. When the cross-over and mutation operators are
specified via `use(...)`, the order of execution will always be
arranged such that the cross-over comes first and the mutation comes
comes second. If desired, one can specify only the cross-over operator
or only the mutation operator.
Please note that the `operators` keyword argument works differently,
and offers more flexibility for defining the procedure to follow at
each generation. In more details, the `operators` keyword argument
allows one to specify multiple cross-over and/or multiple mutation
operators, and those operators will be executed in the specified
order.
Args:
operator: The operator to be registered to SteadyStateGA.
If the specified operator is cross-over (i.e. an instance
of [CrossOver][evotorch.operators.base.CrossOver]),
then this operator will be registered for the cross-over
phase. If the specified operator is an operator that is
not of the cross-over type (i.e. any instance of
[Operator][evotorch.operators.base.Operator] that is not
[CrossOver][evotorch.operators.base.CrossOver]) or if it is
just a function which receives the decision values as a PyTorch
tensor (or, in the case where `dtype` of the problem is
`object` as an instance of
[ObjectArray][evotorch.tools.objectarray.ObjectArray]) and
returns a modified copy, then that operator will be registered
for the mutation phase of the genetic algorithm.
"""
if self._forbid_use_method:
raise RuntimeError(
"The method `use(...)` cannot be called anymore, because the evolutionary search has started."
)
if len(self._operators) > 0:
raise RuntimeError(
f"The method `use(...)` cannot be called"
f" because an operator list was provided while initializing this {type(self).__name__} instance."
)
if isinstance(operator, CrossOver):
if self._cross_over_op is not None:
raise ValueError(
f"The method `use(...)` received this cross-over operator as its argument:"
f" {operator} (of type {type(operator)})."
f" However, a cross-over operator was already set:"
f" {self._cross_over_op} (of type {type(self._cross_over_op)})."
)
self._cross_over_op = operator
self._prepare_ops = True
else:
if self._mutation_op is not None:
raise ValueError(
f"The method `use(...)` received this mutation operator as its argument:"
f" {operator} (of type {type(operator)})."
f" However, a mutation operator was already set:"
f" {self._mutation_op} (of type {type(self._mutation_op)})."
)
self._mutation_op = operator
self._prepare_ops = True
mapelites
¶
MAPElites (SearchAlgorithm, SinglePopulationAlgorithmMixin, ExtendedPopulationMixin)
¶
Implementation of the MAPElites algorithm.
In MAPElites, we deal with optimization problems where, in addition to the fitness, there are additional evaluation data ("features") that are computed during the phase of evaluation. To ensure a diversity of the solutions, the population is organized into cells of features.
Reference:
Jean-Baptiste Mouret and Jeff Clune (2015).
Illuminating search spaces by mapping elites.
arXiv preprint arXiv:1504.04909 (2015).
As an example, let us imagine that our problem has two features.
Let us call these features feat0 and feat1.
Let us also imagine that we wish to organize feat0 according to
the boundaries [(-inf, 0), (0, 10), (10, 20), (20, +inf)] and feat1
according to the boundaries [(-inf, 0), (0, 50), (50, +inf)].
Our population gets organized into:
+inf
^
|
f | | | |
e | pop[0] | pop[1] | pop[ 2] | pop[ 3]
a 50 -|- --------+--------+---------+---------
t | pop[4] | pop[5] | pop[ 6] | pop[ 7]
1 0 -|- --------+--------|---------+---------
| pop[8] | pop[9] | pop[10] | pop[11]
| | | |
<-----------------|--------|---------|----------->
-inf | 0 10 20 +inf
| feat0
|
v
-inf
where pop[i] is short for population[i], that is, the i-th solution
of the population.
Which problems can be solved by MAPElites?
The problems that can be addressed by MAPElites are the problems with
one objective, and with its eval_data_length (additional evaluation
data length) set as an integer greater than or equal to 1.
For example, let us imagine an optimization problem where we handle
2 features. The evaluation function for such a problem could look like:
def f(x: torch.Tensor) -> torch.Tensor:
# Somehow compute the fitness
fitness = ...
# Somehow compute the value for the first feature
feat0 = ...
# Somehow compute the value for the second feature
feat1 = ...
# Prepare an evaluation result tensor for the solution
eval_result = torch.tensor([fitness, feat0, feat1], device=x.device)
# Here, we return the eval_result.
# Notice that `eval_result` is a 1-dimensional tensor of length 3,
# where the item with index 0 is the fitness, and the items with
# indices 1 and 2 represent the two features of the solution.
# Please also note that, in vectorized mode, we would receive `n`
# solutions, and the evaluation result tensor would have to be of shape
# (n, 3).
return eval_result
The problem definition then would look like this:
from evotorch import Problem
problem = Problem(
"min",
f,
initial_bounds=(..., ...),
solution_length=...,
eval_data_length=2, # we have 2 features
)
Using MAPElites.
Let us continue using the example problem shown above, where we have
two features.
The first step towards configuring MAPElites is to come up with a
hypergrid tensor, from in the lower and upper bound for each
feature on each cell will be expressed. The hypergrid tensor is structured
like this:
hypergrid = torch.tensor(
[
[
[
feat0_lower_bound_for_cell0,
feat0_upper_bound_for_cell0,
],
[
feat1_lower_bound_for_cell0,
feat1_upper_bound_for_cell0,
],
],
[
[
feat0_lower_bound_for_cell1,
feat0_upper_bound_for_cell1,
],
[
feat1_lower_bound_for_cell1,
feat1_upper_bound_for_cell1,
],
],
[
[
feat0_lower_bound_for_cell2,
feat0_upper_bound_for_cell2,
],
[
feat1_lower_bound_for_cell2,
feat1_upper_bound_for_cell2,
],
],
...,
],
dtype=problem.eval_dtype,
device=problem.device,
)
that is, the item with index i,j,0 represents the lower bound for the
j-th feature in i-th cell, and the item with index i,j,1 represents the
upper bound for the j-th feature in i-th cell.
Specifying lower and upper bounds for each feature and for each cell can
be tedious. MAPElites provides a static helper function named
make_feature_grid
which asks for how many bins are desired for each feature, and then
produces a hypergrid tensor. For example, if we want 10 bins for feature
feat0 and 5 bins for feature feat1, then, we could do:
hypergrid = MAPElites.make_feature_grid(
lower_bounds=[
global_lower_bound_for_feat0,
global_lower_bound_for_feat1,
],
upper_bounds=[
global_upper_bound_for_feat0,
global_upper_bound_for_feat1,
],
num_bins=[10, 5],
dtype=problem.eval_dtype,
device=problem.device,
)
Now that hypergrid is prepared, one can instantiate MAPElites like
this:
searcher = MAPElites(
problem,
operators=[...], # list of operators like in GeneticAlgorithm
feature_grid=hypergrid,
)
where the keyword argument operators is a list that contains functions
or instances of Operator, like expected
by GeneticAlgorithm.
Once MAPElites is instantiated, it can be run like most of the search
algorithm implementations provided by EvoTorch, as shown below:
from evotorch.logging import StdOutLogger
_ = StdOutLogger(ga) # Report the evolution's progress to standard output
searcher.run(100) # Run MAPElites for 100 generations
print(dict(searcher.status)) # Show the final status dictionary
Vectorization capabilities. According to the basic definition of the MAPElites algorithm, a cell is first chosen, then mutated, and then the mutated solution is placed back into the most suitable cell (if the cell is not filled yet or if the fitness of the newly mutated solution is better than the existing solution in that cell). When vectorization, and especially GPU-based parallelization is available, picking and mutating solutions one by one can be wasteful in terms of performance. Therefore, this MAPElites implementation mutates the entire population, evaluates all of the mutated solutions, and places all of them back into the most suitable cells, all in such a way that the vectorization and/or GPU-based parallelization can be exploited.
Source code in evotorch/algorithms/mapelites.py
class MAPElites(SearchAlgorithm, SinglePopulationAlgorithmMixin, ExtendedPopulationMixin):
"""
Implementation of the MAPElites algorithm.
In MAPElites, we deal with optimization problems where, in addition
to the fitness, there are additional evaluation data ("features") that
are computed during the phase of evaluation. To ensure a diversity
of the solutions, the population is organized into cells of features.
Reference:
Jean-Baptiste Mouret and Jeff Clune (2015).
Illuminating search spaces by mapping elites.
arXiv preprint arXiv:1504.04909 (2015).
As an example, let us imagine that our problem has two features.
Let us call these features `feat0` and `feat1`.
Let us also imagine that we wish to organize `feat0` according to
the boundaries `[(-inf, 0), (0, 10), (10, 20), (20, +inf)]` and `feat1`
according to the boundaries `[(-inf, 0), (0, 50), (50, +inf)]`.
Our population gets organized into:
```text
+inf
^
|
f | | | |
e | pop[0] | pop[1] | pop[ 2] | pop[ 3]
a 50 -|- --------+--------+---------+---------
t | pop[4] | pop[5] | pop[ 6] | pop[ 7]
1 0 -|- --------+--------|---------+---------
| pop[8] | pop[9] | pop[10] | pop[11]
| | | |
<-----------------|--------|---------|----------->
-inf | 0 10 20 +inf
| feat0
|
v
-inf
```
where `pop[i]` is short for `population[i]`, that is, the i-th solution
of the population.
**Which problems can be solved by MAPElites?**
The problems that can be addressed by MAPElites are the problems with
one objective, and with its `eval_data_length` (additional evaluation
data length) set as an integer greater than or equal to 1.
For example, let us imagine an optimization problem where we handle
2 features. The evaluation function for such a problem could look like:
```python
def f(x: torch.Tensor) -> torch.Tensor:
# Somehow compute the fitness
fitness = ...
# Somehow compute the value for the first feature
feat0 = ...
# Somehow compute the value for the second feature
feat1 = ...
# Prepare an evaluation result tensor for the solution
eval_result = torch.tensor([fitness, feat0, feat1], device=x.device)
# Here, we return the eval_result.
# Notice that `eval_result` is a 1-dimensional tensor of length 3,
# where the item with index 0 is the fitness, and the items with
# indices 1 and 2 represent the two features of the solution.
# Please also note that, in vectorized mode, we would receive `n`
# solutions, and the evaluation result tensor would have to be of shape
# (n, 3).
return eval_result
```
The problem definition then would look like this:
```python
from evotorch import Problem
problem = Problem(
"min",
f,
initial_bounds=(..., ...),
solution_length=...,
eval_data_length=2, # we have 2 features
)
```
**Using MAPElites.**
Let us continue using the example `problem` shown above, where we have
two features.
The first step towards configuring MAPElites is to come up with a
hypergrid tensor, from in the lower and upper bound for each
feature on each cell will be expressed. The hypergrid tensor is structured
like this:
```python
hypergrid = torch.tensor(
[
[
[
feat0_lower_bound_for_cell0,
feat0_upper_bound_for_cell0,
],
[
feat1_lower_bound_for_cell0,
feat1_upper_bound_for_cell0,
],
],
[
[
feat0_lower_bound_for_cell1,
feat0_upper_bound_for_cell1,
],
[
feat1_lower_bound_for_cell1,
feat1_upper_bound_for_cell1,
],
],
[
[
feat0_lower_bound_for_cell2,
feat0_upper_bound_for_cell2,
],
[
feat1_lower_bound_for_cell2,
feat1_upper_bound_for_cell2,
],
],
...,
],
dtype=problem.eval_dtype,
device=problem.device,
)
```
that is, the item with index `i,j,0` represents the lower bound for the
j-th feature in i-th cell, and the item with index `i,j,1` represents the
upper bound for the j-th feature in i-th cell.
Specifying lower and upper bounds for each feature and for each cell can
be tedious. MAPElites provides a static helper function named
[make_feature_grid][evotorch.algorithms.mapelites.MAPElites.make_feature_grid]
which asks for how many bins are desired for each feature, and then
produces a hypergrid tensor. For example, if we want 10 bins for feature
`feat0` and 5 bins for feature `feat1`, then, we could do:
```python
hypergrid = MAPElites.make_feature_grid(
lower_bounds=[
global_lower_bound_for_feat0,
global_lower_bound_for_feat1,
],
upper_bounds=[
global_upper_bound_for_feat0,
global_upper_bound_for_feat1,
],
num_bins=[10, 5],
dtype=problem.eval_dtype,
device=problem.device,
)
```
Now that `hypergrid` is prepared, one can instantiate `MAPElites` like
this:
```python
searcher = MAPElites(
problem,
operators=[...], # list of operators like in GeneticAlgorithm
feature_grid=hypergrid,
)
```
where the keyword argument `operators` is a list that contains functions
or instances of [Operator][evotorch.operators.base.Operator], like expected
by [GeneticAlgorithm][evotorch.algorithms.ga.GeneticAlgorithm].
Once `MAPElites` is instantiated, it can be run like most of the search
algorithm implementations provided by EvoTorch, as shown below:
```python
from evotorch.logging import StdOutLogger
_ = StdOutLogger(ga) # Report the evolution's progress to standard output
searcher.run(100) # Run MAPElites for 100 generations
print(dict(searcher.status)) # Show the final status dictionary
```
**Vectorization capabilities.**
According to the basic definition of the MAPElites algorithm, a cell is
first chosen, then mutated, and then the mutated solution is placed back
into the most suitable cell (if the cell is not filled yet or if the
fitness of the newly mutated solution is better than the existing solution
in that cell). When vectorization, and especially GPU-based parallelization
is available, picking and mutating solutions one by one can be wasteful in
terms of performance. Therefore, this MAPElites implementation mutates the
entire population, evaluates all of the mutated solutions, and places all
of them back into the most suitable cells, all in such a way that the
vectorization and/or GPU-based parallelization can be exploited.
"""
def __init__(
self,
problem: Problem,
*,
operators: Iterable,
feature_grid: Iterable,
re_evaluate: bool = True,
re_evaluate_parents_first: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the MAPElites algorithm.
Args:
problem: The problem object to work on. This problem object
is expected to have one objective, and also have its
`eval_data_length` set as an integer that is greater than
or equal to 1.
operators: Operators to be used by the MAPElites algorithm.
Expected as an iterable, such as a list or a tuple.
Each item within this iterable object is expected either
as an instance of [Operator][evotorch.operators.base.Operator],
or as a function which receives the decision values of
multiple solutions in a PyTorch tensor and returns a modified
copy.
re_evaluate: Whether or not to evaluate the solutions
that were already evaluated in the previous generations.
By default, this is set as True.
The reason behind this default setting is that,
in problems where the evaluation procedure is noisy,
by re-evaluating the already-evaluated solutions,
we prevent the bad solutions that were luckily evaluated
from hanging onto the population.
Instead, at every generation, each solution must go through
the evaluation procedure again and prove their worth.
For problems whose evaluation procedures are NOT noisy,
the user might consider turning re_evaluate to False
for saving computational cycles.
re_evaluate_parents_first: This is to be specified only when
`re_evaluate` is True (otherwise to be left as None).
If this is given as True, then it will be assumed that the
provided operators require the parents to be evaluated.
If this is given as False, then it will be assumed that the
provided operators work without looking at the parents'
fitnesses (in which case both parents and children can be
evaluated in a single vectorized computation cycle).
If this is left as None, then whether or not the operators
need to know the parent evaluations will be determined
automatically as follows:
if the operators contain at least one cross-over operator
then `re_evaluate_parents_first` will be internally set as
True; otherwise `re_evaluate_parents_first` will be internally
set as False.
"""
problem.ensure_single_objective()
problem.ensure_numeric()
SearchAlgorithm.__init__(self, problem)
self._feature_grid = problem.as_tensor(feature_grid, use_eval_dtype=True)
self._sense = self._problem.senses[0]
self._popsize = self._feature_grid.shape[0]
self._population = problem.generate_batch(self._popsize)
self._filled = torch.zeros(self._popsize, dtype=torch.bool, device=self._population.device)
ExtendedPopulationMixin.__init__(
self,
re_evaluate=re_evaluate,
re_evaluate_parents_first=re_evaluate_parents_first,
operators=operators,
allow_empty_operators_list=False,
)
SinglePopulationAlgorithmMixin.__init__(self)
@property
def population(self) -> SolutionBatch:
"""
Get the population as a SolutionBatch object
In this MAP-Elites implementation, i-th solution corresponds to the
solution belonging to the i-th cell of the MAP-Elites hypergrid.
If `filled[i]` is True, then this means that the i-th cell is filled,
and therefore `population[i]` will get the solution belonging to the
i-th cell.
"""
return self._population
@property
def filled(self) -> torch.Tensor:
"""
Get a boolean tensor specifying whether or not the i-th cell is filled.
In this MAP-Elites implementation, i-th solution within the population
corresponds to the solution belonging to the i-th cell of the MAP-Elites
hypergrid. If `filled[i]` is True, then the solution stored in the i-th
cell satisfies the feature boundaries imposed by that cell.
If `filled[i]` is False, then the solution stored in the i-th cell
does not satisfy those boundaries, and therefore does not really belong
in that cell.
"""
from ..tools.readonlytensor import as_read_only_tensor
with torch.no_grad():
return as_read_only_tensor(self._filled)
def _step(self):
# Form an extended population from the parents and from the children
extended_population = self._make_extended_population(split=False)
# Get the most suitable solutions for each cell of the hypergrid.
# values[i, :] stores the decision values most suitable for the i-th cell.
# evals[i, :] stores the evaluation results most suitable for the i-th cell.
# if the suggested i-th solution completely satisfies the boundaries imposed by the i-th cell,
# then suitable_mask[i] will be True.
values, evals, suitable = _best_solution_considering_all_features(
self._sense,
extended_population.values.as_subclass(torch.Tensor),
extended_population.evals.as_subclass(torch.Tensor),
self._feature_grid,
)
# Place the most suitable decision values and evaluation results into the current population.
self._population.access_values(keep_evals=True)[:] = values
self._population.access_evals()[:] = evals
# If there was a suitable solution for the i-th cell, fill[i] is to be set as True.
self._filled[:] = suitable
@staticmethod
def make_feature_grid(
lower_bounds: Iterable,
upper_bounds: Iterable,
num_bins: Union[int, torch.Tensor],
*,
device: Optional[Device] = None,
dtype: Optional[DType] = None,
) -> torch.Tensor:
"""
Make a hypergrid for the MAPElites algorithm.
The [MAPElites][evotorch.algorithms.mapelites.MAPElites] organizes its
population not only according to the fitness, but also according to the
additional evaluation data which are interpreted as the additional features
of the solutions. To organize the current population according to these
[MAPElites][evotorch.algorithms.mapelites.MAPElites] requires "cells",
each cell having a lower and an upper bound for each feature.
`make_map_elites_grid(...)` is a helper function which generates the
required hypergrid of features in such a way that each cell, for each
feature, has the same interval.
The result of this function is a PyTorch tensor, which can be passed to
the `feature_grid` keyword argument of
[MAPElites][evotorch.algorithms.mapelites.MAPElites].
Args:
lower_bounds: The lower bounds, as a 1-dimensional sequence of numbers.
The length of this sequence must be equal to the number of
features, and the i-th element must express the lower bound
of the i-th feature.
upper_bounds: The upper bounds, as a 1-dimensional sequence of numbers.
The length of this sequence must be equal to the number of
features, and the i-th element must express the upper bound
of the i-th feature.
num_bins: Can be given as an integer or as a sequence of integers.
If given as an integer `n`, then there will be `n` bins for each
feature on the hypergrid. If given as a sequence of integers,
then the i-th element of the sequence will express the number of
bins for the i-th feature.
Returns:
The hypergrid, as a PyTorch tensor.
"""
cast_args = {}
if device is not None:
cast_args["device"] = torch.device(device)
if dtype is not None:
cast_args["dtype"] = to_torch_dtype(dtype)
has_casting = len(cast_args) > 0
if has_casting:
lower_bounds = torch.as_tensor(lower_bounds, **cast_args)
upper_bounds = torch.as_tensor(upper_bounds, **cast_args)
if (not isinstance(lower_bounds, torch.Tensor)) or (not isinstance(upper_bounds, torch.Tensor)):
raise TypeError(
f"While preparing the map elites hypergrid with device={device} and dtype={dtype},"
f"`lower_bounds` and `upper_bounds` were expected as tensors, but their types are different."
f" The type of `lower_bounds` is {type(lower_bounds)}."
f" The type of `upper_bounds` is {type(upper_bounds)}."
)
if lower_bounds.device != upper_bounds.device:
raise ValueError(
f"Cannot determine on which device to place the map elites grid,"
f" because `lower_bounds` and `upper_bounds` are on different devices."
f" The device of `lower_bounds` is {lower_bounds.device}."
f" The device of `upper_bounds` is {upper_bounds.device}."
)
if lower_bounds.dtype != upper_bounds.dtype:
raise ValueError(
f"Cannot determine the dtype of the map elites grid,"
f" because `lower_bounds` and `upper_bounds` have different dtypes."
f" The dtype of `lower_bounds` is {lower_bounds.dtype}."
f" The dtype of `upper_bounds` is {upper_bounds.dtype}."
)
if lower_bounds.size() != upper_bounds.size():
raise ValueError("`lower_bounds` and `upper_bounds` have incompatible shapes")
if lower_bounds.dim() != 1:
raise ValueError("Only 1D tensors are supported for `lower_bounds` and for `upper_bounds`")
dtype = lower_bounds.dtype
device = lower_bounds.device
num_bins = torch.as_tensor(num_bins, dtype=torch.int64, device=device)
if num_bins.dim() == 0:
num_bins = num_bins.expand(lower_bounds.size())
p_inf = torch.tensor([float("inf")], dtype=dtype, device=device)
n_inf = torch.tensor([float("-inf")], dtype=dtype, device=device)
def _make_feature_grid(lb, ub, num_bins):
sp = torch.linspace(lb, ub, num_bins - 1, device=device)
sp = torch.cat((n_inf, sp, p_inf))
return sp.unfold(dimension=0, size=2, step=1).unsqueeze(1)
f_grids = [_make_feature_grid(*bounds) for bounds in zip(lower_bounds, upper_bounds, num_bins)]
return torch.stack([torch.cat(c) for c in itertools.product(*f_grids)])
filled: Tensor
property
readonly
¶
Get a boolean tensor specifying whether or not the i-th cell is filled.
In this MAP-Elites implementation, i-th solution within the population
corresponds to the solution belonging to the i-th cell of the MAP-Elites
hypergrid. If filled[i] is True, then the solution stored in the i-th
cell satisfies the feature boundaries imposed by that cell.
If filled[i] is False, then the solution stored in the i-th cell
does not satisfy those boundaries, and therefore does not really belong
in that cell.
population: SolutionBatch
property
readonly
¶
Get the population as a SolutionBatch object
In this MAP-Elites implementation, i-th solution corresponds to the
solution belonging to the i-th cell of the MAP-Elites hypergrid.
If filled[i] is True, then this means that the i-th cell is filled,
and therefore population[i] will get the solution belonging to the
i-th cell.
__init__(self, problem, *, operators, feature_grid, re_evaluate=True, re_evaluate_parents_first=None)
special
¶
__init__(...): Initialize the MAPElites algorithm.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object to work on. This problem object
is expected to have one objective, and also have its
|
required |
operators |
Iterable |
Operators to be used by the MAPElites algorithm. Expected as an iterable, such as a list or a tuple. Each item within this iterable object is expected either as an instance of Operator, or as a function which receives the decision values of multiple solutions in a PyTorch tensor and returns a modified copy. |
required |
re_evaluate |
bool |
Whether or not to evaluate the solutions that were already evaluated in the previous generations. By default, this is set as True. The reason behind this default setting is that, in problems where the evaluation procedure is noisy, by re-evaluating the already-evaluated solutions, we prevent the bad solutions that were luckily evaluated from hanging onto the population. Instead, at every generation, each solution must go through the evaluation procedure again and prove their worth. For problems whose evaluation procedures are NOT noisy, the user might consider turning re_evaluate to False for saving computational cycles. |
True |
re_evaluate_parents_first |
Optional[bool] |
This is to be specified only when
|
None |
Source code in evotorch/algorithms/mapelites.py
def __init__(
self,
problem: Problem,
*,
operators: Iterable,
feature_grid: Iterable,
re_evaluate: bool = True,
re_evaluate_parents_first: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the MAPElites algorithm.
Args:
problem: The problem object to work on. This problem object
is expected to have one objective, and also have its
`eval_data_length` set as an integer that is greater than
or equal to 1.
operators: Operators to be used by the MAPElites algorithm.
Expected as an iterable, such as a list or a tuple.
Each item within this iterable object is expected either
as an instance of [Operator][evotorch.operators.base.Operator],
or as a function which receives the decision values of
multiple solutions in a PyTorch tensor and returns a modified
copy.
re_evaluate: Whether or not to evaluate the solutions
that were already evaluated in the previous generations.
By default, this is set as True.
The reason behind this default setting is that,
in problems where the evaluation procedure is noisy,
by re-evaluating the already-evaluated solutions,
we prevent the bad solutions that were luckily evaluated
from hanging onto the population.
Instead, at every generation, each solution must go through
the evaluation procedure again and prove their worth.
For problems whose evaluation procedures are NOT noisy,
the user might consider turning re_evaluate to False
for saving computational cycles.
re_evaluate_parents_first: This is to be specified only when
`re_evaluate` is True (otherwise to be left as None).
If this is given as True, then it will be assumed that the
provided operators require the parents to be evaluated.
If this is given as False, then it will be assumed that the
provided operators work without looking at the parents'
fitnesses (in which case both parents and children can be
evaluated in a single vectorized computation cycle).
If this is left as None, then whether or not the operators
need to know the parent evaluations will be determined
automatically as follows:
if the operators contain at least one cross-over operator
then `re_evaluate_parents_first` will be internally set as
True; otherwise `re_evaluate_parents_first` will be internally
set as False.
"""
problem.ensure_single_objective()
problem.ensure_numeric()
SearchAlgorithm.__init__(self, problem)
self._feature_grid = problem.as_tensor(feature_grid, use_eval_dtype=True)
self._sense = self._problem.senses[0]
self._popsize = self._feature_grid.shape[0]
self._population = problem.generate_batch(self._popsize)
self._filled = torch.zeros(self._popsize, dtype=torch.bool, device=self._population.device)
ExtendedPopulationMixin.__init__(
self,
re_evaluate=re_evaluate,
re_evaluate_parents_first=re_evaluate_parents_first,
operators=operators,
allow_empty_operators_list=False,
)
SinglePopulationAlgorithmMixin.__init__(self)
make_feature_grid(lower_bounds, upper_bounds, num_bins, *, device=None, dtype=None)
staticmethod
¶
Make a hypergrid for the MAPElites algorithm.
The MAPElites organizes its
population not only according to the fitness, but also according to the
additional evaluation data which are interpreted as the additional features
of the solutions. To organize the current population according to these
MAPElites requires "cells",
each cell having a lower and an upper bound for each feature.
make_map_elites_grid(...) is a helper function which generates the
required hypergrid of features in such a way that each cell, for each
feature, has the same interval.
The result of this function is a PyTorch tensor, which can be passed to
the feature_grid keyword argument of
MAPElites.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
lower_bounds |
Iterable |
The lower bounds, as a 1-dimensional sequence of numbers. The length of this sequence must be equal to the number of features, and the i-th element must express the lower bound of the i-th feature. |
required |
upper_bounds |
Iterable |
The upper bounds, as a 1-dimensional sequence of numbers. The length of this sequence must be equal to the number of features, and the i-th element must express the upper bound of the i-th feature. |
required |
num_bins |
Union[int, torch.Tensor] |
Can be given as an integer or as a sequence of integers.
If given as an integer |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The hypergrid, as a PyTorch tensor. |
Source code in evotorch/algorithms/mapelites.py
@staticmethod
def make_feature_grid(
lower_bounds: Iterable,
upper_bounds: Iterable,
num_bins: Union[int, torch.Tensor],
*,
device: Optional[Device] = None,
dtype: Optional[DType] = None,
) -> torch.Tensor:
"""
Make a hypergrid for the MAPElites algorithm.
The [MAPElites][evotorch.algorithms.mapelites.MAPElites] organizes its
population not only according to the fitness, but also according to the
additional evaluation data which are interpreted as the additional features
of the solutions. To organize the current population according to these
[MAPElites][evotorch.algorithms.mapelites.MAPElites] requires "cells",
each cell having a lower and an upper bound for each feature.
`make_map_elites_grid(...)` is a helper function which generates the
required hypergrid of features in such a way that each cell, for each
feature, has the same interval.
The result of this function is a PyTorch tensor, which can be passed to
the `feature_grid` keyword argument of
[MAPElites][evotorch.algorithms.mapelites.MAPElites].
Args:
lower_bounds: The lower bounds, as a 1-dimensional sequence of numbers.
The length of this sequence must be equal to the number of
features, and the i-th element must express the lower bound
of the i-th feature.
upper_bounds: The upper bounds, as a 1-dimensional sequence of numbers.
The length of this sequence must be equal to the number of
features, and the i-th element must express the upper bound
of the i-th feature.
num_bins: Can be given as an integer or as a sequence of integers.
If given as an integer `n`, then there will be `n` bins for each
feature on the hypergrid. If given as a sequence of integers,
then the i-th element of the sequence will express the number of
bins for the i-th feature.
Returns:
The hypergrid, as a PyTorch tensor.
"""
cast_args = {}
if device is not None:
cast_args["device"] = torch.device(device)
if dtype is not None:
cast_args["dtype"] = to_torch_dtype(dtype)
has_casting = len(cast_args) > 0
if has_casting:
lower_bounds = torch.as_tensor(lower_bounds, **cast_args)
upper_bounds = torch.as_tensor(upper_bounds, **cast_args)
if (not isinstance(lower_bounds, torch.Tensor)) or (not isinstance(upper_bounds, torch.Tensor)):
raise TypeError(
f"While preparing the map elites hypergrid with device={device} and dtype={dtype},"
f"`lower_bounds` and `upper_bounds` were expected as tensors, but their types are different."
f" The type of `lower_bounds` is {type(lower_bounds)}."
f" The type of `upper_bounds` is {type(upper_bounds)}."
)
if lower_bounds.device != upper_bounds.device:
raise ValueError(
f"Cannot determine on which device to place the map elites grid,"
f" because `lower_bounds` and `upper_bounds` are on different devices."
f" The device of `lower_bounds` is {lower_bounds.device}."
f" The device of `upper_bounds` is {upper_bounds.device}."
)
if lower_bounds.dtype != upper_bounds.dtype:
raise ValueError(
f"Cannot determine the dtype of the map elites grid,"
f" because `lower_bounds` and `upper_bounds` have different dtypes."
f" The dtype of `lower_bounds` is {lower_bounds.dtype}."
f" The dtype of `upper_bounds` is {upper_bounds.dtype}."
)
if lower_bounds.size() != upper_bounds.size():
raise ValueError("`lower_bounds` and `upper_bounds` have incompatible shapes")
if lower_bounds.dim() != 1:
raise ValueError("Only 1D tensors are supported for `lower_bounds` and for `upper_bounds`")
dtype = lower_bounds.dtype
device = lower_bounds.device
num_bins = torch.as_tensor(num_bins, dtype=torch.int64, device=device)
if num_bins.dim() == 0:
num_bins = num_bins.expand(lower_bounds.size())
p_inf = torch.tensor([float("inf")], dtype=dtype, device=device)
n_inf = torch.tensor([float("-inf")], dtype=dtype, device=device)
def _make_feature_grid(lb, ub, num_bins):
sp = torch.linspace(lb, ub, num_bins - 1, device=device)
sp = torch.cat((n_inf, sp, p_inf))
return sp.unfold(dimension=0, size=2, step=1).unsqueeze(1)
f_grids = [_make_feature_grid(*bounds) for bounds in zip(lower_bounds, upper_bounds, num_bins)]
return torch.stack([torch.cat(c) for c in itertools.product(*f_grids)])
pycmaes
¶
This namespace contains the PyCMAES class, which is a wrapper
for the CMA-ES implementation of the cma package.
PyCMAES (SearchAlgorithm, SinglePopulationAlgorithmMixin)
¶
This is an interface class between the CMAES implementation
within the cma package developed within the GitHub repository
CMA-ES/pycma.
References:
Nikolaus Hansen, Youhei Akimoto, and Petr Baudis.
CMA-ES/pycma on Github. Zenodo, DOI:10.5281/zenodo.2559634,
February 2019.
<https://github.com/CMA-ES/pycma>
Nikolaus Hansen, Andreas Ostermeier (2001).
Completely Derandomized Self-Adaptation in Evolution Strategies.
Source code in evotorch/algorithms/pycmaes.py
class PyCMAES(SearchAlgorithm, SinglePopulationAlgorithmMixin):
"""
PyCMAES: Covariance Matrix Adaptation Evolution Strategy.
This is an interface class between the CMAES implementation
within the `cma` package developed within the GitHub repository
CMA-ES/pycma.
References:
Nikolaus Hansen, Youhei Akimoto, and Petr Baudis.
CMA-ES/pycma on Github. Zenodo, DOI:10.5281/zenodo.2559634,
February 2019.
<https://github.com/CMA-ES/pycma>
Nikolaus Hansen, Andreas Ostermeier (2001).
Completely Derandomized Self-Adaptation in Evolution Strategies.
"""
def __init__(
self,
problem: Problem,
*,
stdev_init: RealOrVector, # sigma0
popsize: Optional[int] = None, # popsize
center_init: Optional[Vector] = None, # x0
center_learning_rate: Optional[float] = None, # CMA_cmean
cov_learning_rate: Optional[float] = None, # CMA_on
rankmu_learning_rate: Optional[float] = None, # CMA_rankmu
rankone_learning_rate: Optional[float] = None, # CMA_rankone
stdev_min: Optional[Union[float, np.ndarray]] = None, # minstd
stdev_max: Optional[Union[float, np.ndarray]] = None, # maxstd
separable: bool = False, # CMA_diagonal
obj_index: Optional[int] = None,
cma_options: dict = {},
):
"""
`__init__(...)`: Initialize the PyCMAES solver.
Args:
problem: The problem object which is being worked on.
stdev_init: Initial standard deviation as a scalar or
as a 1-dimensional array.
popsize: Population size. Can be specified as an int,
or can be left as None to let the CMAES solver
decide the population size according to the length
of a solution.
center_init: Initial center point of the search distribution.
Can be given as a Solution or as a 1-D array.
If left as None, an initial center point is generated
with the help of the problem object's `generate_values(...)`
method.
center_learning_rate: Learning rate for updating the mean
of the search distribution. Leaving this as None
means that the CMAES solver is to use its own default,
which is documented as 1.0.
cov_learning_rate: Learning rate for updating the covariance
matrix of the search distribution. This hyperparameter
acts as a common multiplier for rank_one update and rank_mu
update of the covariance matrix. Leaving this as None
means that the CMAES solver is to use its own default,
which is documented as 1.0.
rankmu_learning_rate: Learning rate for the rank_mu update
of the covariance matrix of the search distribution.
Leaving this as None means that the CMAES solver is to use
its own default, which is documented as 1.0.
rankone_learning_rate: Learning rate for the rank_one update
of the covariance matrix of the search distribution.
Leaving this as None means that the CMAES solver is to use
its own default, which is documented as 1.0.
stdev_min: Minimum allowed standard deviation of the search
distribution. Leaving this as None means that no such
boundary is to be used.
Can be given as None, as a scalar, or as a 1-dimensional
array.
stdev_max: Maximum allowed standard deviation of the search
distribution. Leaving this as None means that no such
boundary is to be used.
Can be given as None, as a scalar, or as a 1-dimensional
array.
separable: Provide this as True if you would like the problem
to be treated as a separable one. Treating a problem
as separable means to adapt only the diagonal parts
of the covariance matrix and to keep the non-diagonal
parts 0. High dimensional problems result in large
covariance matrices on which operating is computationally
expensive. Therefore, for such high dimensional problems,
setting `separable` as True might be useful.
If, instead, you would like to configure on which
iterations the diagonal parts of the covariance matrix
are to be adapted, then it is recommended to leave
`separable` as False and set a new value for the key
"CMA_diagonal" via `cma_options` (see the official
documentation of pycma for details regarding the
"CMA_diagonal" setting).
obj_index: Objective index according to which evaluation
of the solution will be done.
cma_options: Any other configuration for the CMAES solver
can be passed via the cma_options dictionary.
"""
# Make sure that the cma module is installed
if cma is None:
raise ImportError(f"The class {type(self).__name__} is only available if the package `cma` is installed.")
# Initialize the base class
SearchAlgorithm.__init__(self, problem, center=self._get_center)
# Ensure that the problem is numeric
problem.ensure_numeric()
# Store the objective index
self._obj_index = problem.normalize_obj_index(obj_index)
# If `center_init` is not given, generate an initial solution
# with the help of the problem object.
# Otherwise, use the given initial solution as the starting
# point in the search space.
if center_init is None:
x0 = self._problem.generate_values(1).to("cpu").view(-1).numpy().astype(dtype=float)
else:
x0 = numpy_copy(center_init, dtype=float)
# Store the initial standard deviations
sigma0 = numpy_copy(stdev_init, dtype=float)
# Generate an options dictionary to pass to the cma solver.
inopts = {}
for k, v in cma_options.items():
if isinstance(v, torch.Tensor):
v = numpy_copy(v, dtype=float)
inopts[k] = v
# Remove the number of iterations boundary
if "maxiter" not in inopts:
inopts["maxiter"] = np.inf
# Below is a temporary helper function for safely storing the configuration items.
# This inner function updates the `inopts` variable.
def store_opt(key: str, long_name: str, value: Any, converter: Callable):
# Here, `key` represents the configuration key used by pycma
# `long_name` represents the configuration's long name used by this class
# `value` is the configuration value associated with `key`.
# Declare that this inner function accesses the `inopts` variable.
nonlocal inopts
if value is None:
# If the provided `value` is None, then there is no configuration to store.
# So, we just leave this inner function.
return
if key in inopts:
# If the given `key` already exists within `inopts`, this means that the configuration was specified
# twice: via the keyword argument `cma_options` AND via a keyword argument.
# We raise an error and inform the user about this redundancy.
raise ValueError(
f"The configuration {repr(key)} was redundantly provided"
f" both via the initialization argument {long_name}"
f" and via the cma_options dictionary."
f" {long_name}={repr(value)};"
f" cma_options[{repr(key)}]={repr(inopts[key])}."
)
inopts[key] = converter(value)
# Temporary helper function which makes sure that `x` is a numpy array or a float.
def array_or_float(x):
if is_sequence(x):
return numpy_copy(x)
else:
return float(x)
# Store the cma configuration received through the initialization arguments (and raise error if there is
# redundancy with the cma_options dictionary).
store_opt("popsize", "popsize", popsize, int)
store_opt("CMA_cmean", "center_learning_rate", center_learning_rate, float)
store_opt("CMA_on", "cov_learning_rate", cov_learning_rate, float)
store_opt("CMA_rankmu", "rankmu_learning_rate", rankmu_learning_rate, float)
store_opt("CMA_rankone", "rankone_learning_rate", rankone_learning_rate, float)
store_opt("minstd", "stdev_min", stdev_min, array_or_float)
store_opt("maxstd", "stdev_max", stdev_max, array_or_float)
if separable:
store_opt("CMA_diagonal", "separable", separable, bool)
# If the problem defines lower and upper bounds, pass these into the options dict.
def process_bounds(bounds: RealOrVector) -> np.ndarray:
if bounds is None:
return None
else:
if is_sequence(bounds):
bounds = numpy_copy(bounds)
else:
bounds = np.array(float(bounds)).repeat(self._problem.solution_length)
return bounds
lb = process_bounds(self._problem.lower_bounds)
ub = process_bounds(self._problem.upper_bounds)
register_bounds = False
if lb is not None and ub is None:
ub = np.array(np.inf).repeat(self._problem.solution_length)
register_bounds = True
elif lb is None and ub is not None:
lb = np.array(-(np.inf)).repeat(self._problem.solution_length)
register_bounds = True
elif lb is not None and ub is not None:
register_bounds = True
if register_bounds:
inopts["bounds"] = [lb, ub]
# Generate a random seed using the problem object for the sake of reproducibility.
if "seed" not in inopts:
inopts["seed"] = int(self._problem.make_randint(tuple(), n=(2**32) - 100) + 100)
# Instantiate the CMAEvolutionStrategy with the prepared configuration items.
self._es = cma.CMAEvolutionStrategy(x0, sigma0, inopts)
# Initialize the population.
self._population: SolutionBatch = self._problem.generate_batch(self._es.popsize, empty=True)
# Use the SinglePopulationAlgorithmMixin to enable additional status reports regarding the population.
SinglePopulationAlgorithmMixin.__init__(self)
@property
def population(self) -> SolutionBatch:
"""Population generated by the CMA-ES algorithm"""
return self._population
def _step(self):
"""Perform a step of the CMA-ES solver"""
asked = self._es.ask()
self._population.access_values()[:] = torch.as_tensor(
np.asarray(asked), dtype=self._problem.dtype, device=self._population.device
)
self._problem.evaluate(self._population)
scores = numpy_copy(self._population.utility(self._obj_index), dtype=float)
self._es.tell(asked, -1.0 * scores)
def _get_center(self) -> torch.Tensor:
return torch.as_tensor(self._es.result[5], dtype=self._population.dtype, device=self._population.device)
@property
def obj_index(self) -> int:
"""Index of the objective being focused on"""
return self._obj_index
obj_index: int
property
readonly
¶
Index of the objective being focused on
population: SolutionBatch
property
readonly
¶
Population generated by the CMA-ES algorithm
__init__(self, problem, *, stdev_init, popsize=None, center_init=None, center_learning_rate=None, cov_learning_rate=None, rankmu_learning_rate=None, rankone_learning_rate=None, stdev_min=None, stdev_max=None, separable=False, obj_index=None, cma_options={})
special
¶
__init__(...): Initialize the PyCMAES solver.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object which is being worked on. |
required |
stdev_init |
Union[float, Iterable[float], torch.Tensor] |
Initial standard deviation as a scalar or as a 1-dimensional array. |
required |
popsize |
Optional[int] |
Population size. Can be specified as an int, or can be left as None to let the CMAES solver decide the population size according to the length of a solution. |
None |
center_init |
Union[Iterable[float], torch.Tensor] |
Initial center point of the search distribution.
Can be given as a Solution or as a 1-D array.
If left as None, an initial center point is generated
with the help of the problem object's |
None |
center_learning_rate |
Optional[float] |
Learning rate for updating the mean of the search distribution. Leaving this as None means that the CMAES solver is to use its own default, which is documented as 1.0. |
None |
cov_learning_rate |
Optional[float] |
Learning rate for updating the covariance matrix of the search distribution. This hyperparameter acts as a common multiplier for rank_one update and rank_mu update of the covariance matrix. Leaving this as None means that the CMAES solver is to use its own default, which is documented as 1.0. |
None |
rankmu_learning_rate |
Optional[float] |
Learning rate for the rank_mu update of the covariance matrix of the search distribution. Leaving this as None means that the CMAES solver is to use its own default, which is documented as 1.0. |
None |
rankone_learning_rate |
Optional[float] |
Learning rate for the rank_one update of the covariance matrix of the search distribution. Leaving this as None means that the CMAES solver is to use its own default, which is documented as 1.0. |
None |
stdev_min |
Union[float, numpy.ndarray] |
Minimum allowed standard deviation of the search distribution. Leaving this as None means that no such boundary is to be used. Can be given as None, as a scalar, or as a 1-dimensional array. |
None |
stdev_max |
Union[float, numpy.ndarray] |
Maximum allowed standard deviation of the search distribution. Leaving this as None means that no such boundary is to be used. Can be given as None, as a scalar, or as a 1-dimensional array. |
None |
separable |
bool |
Provide this as True if you would like the problem
to be treated as a separable one. Treating a problem
as separable means to adapt only the diagonal parts
of the covariance matrix and to keep the non-diagonal
parts 0. High dimensional problems result in large
covariance matrices on which operating is computationally
expensive. Therefore, for such high dimensional problems,
setting |
False |
obj_index |
Optional[int] |
Objective index according to which evaluation of the solution will be done. |
None |
cma_options |
dict |
Any other configuration for the CMAES solver can be passed via the cma_options dictionary. |
{} |
Source code in evotorch/algorithms/pycmaes.py
def __init__(
self,
problem: Problem,
*,
stdev_init: RealOrVector, # sigma0
popsize: Optional[int] = None, # popsize
center_init: Optional[Vector] = None, # x0
center_learning_rate: Optional[float] = None, # CMA_cmean
cov_learning_rate: Optional[float] = None, # CMA_on
rankmu_learning_rate: Optional[float] = None, # CMA_rankmu
rankone_learning_rate: Optional[float] = None, # CMA_rankone
stdev_min: Optional[Union[float, np.ndarray]] = None, # minstd
stdev_max: Optional[Union[float, np.ndarray]] = None, # maxstd
separable: bool = False, # CMA_diagonal
obj_index: Optional[int] = None,
cma_options: dict = {},
):
"""
`__init__(...)`: Initialize the PyCMAES solver.
Args:
problem: The problem object which is being worked on.
stdev_init: Initial standard deviation as a scalar or
as a 1-dimensional array.
popsize: Population size. Can be specified as an int,
or can be left as None to let the CMAES solver
decide the population size according to the length
of a solution.
center_init: Initial center point of the search distribution.
Can be given as a Solution or as a 1-D array.
If left as None, an initial center point is generated
with the help of the problem object's `generate_values(...)`
method.
center_learning_rate: Learning rate for updating the mean
of the search distribution. Leaving this as None
means that the CMAES solver is to use its own default,
which is documented as 1.0.
cov_learning_rate: Learning rate for updating the covariance
matrix of the search distribution. This hyperparameter
acts as a common multiplier for rank_one update and rank_mu
update of the covariance matrix. Leaving this as None
means that the CMAES solver is to use its own default,
which is documented as 1.0.
rankmu_learning_rate: Learning rate for the rank_mu update
of the covariance matrix of the search distribution.
Leaving this as None means that the CMAES solver is to use
its own default, which is documented as 1.0.
rankone_learning_rate: Learning rate for the rank_one update
of the covariance matrix of the search distribution.
Leaving this as None means that the CMAES solver is to use
its own default, which is documented as 1.0.
stdev_min: Minimum allowed standard deviation of the search
distribution. Leaving this as None means that no such
boundary is to be used.
Can be given as None, as a scalar, or as a 1-dimensional
array.
stdev_max: Maximum allowed standard deviation of the search
distribution. Leaving this as None means that no such
boundary is to be used.
Can be given as None, as a scalar, or as a 1-dimensional
array.
separable: Provide this as True if you would like the problem
to be treated as a separable one. Treating a problem
as separable means to adapt only the diagonal parts
of the covariance matrix and to keep the non-diagonal
parts 0. High dimensional problems result in large
covariance matrices on which operating is computationally
expensive. Therefore, for such high dimensional problems,
setting `separable` as True might be useful.
If, instead, you would like to configure on which
iterations the diagonal parts of the covariance matrix
are to be adapted, then it is recommended to leave
`separable` as False and set a new value for the key
"CMA_diagonal" via `cma_options` (see the official
documentation of pycma for details regarding the
"CMA_diagonal" setting).
obj_index: Objective index according to which evaluation
of the solution will be done.
cma_options: Any other configuration for the CMAES solver
can be passed via the cma_options dictionary.
"""
# Make sure that the cma module is installed
if cma is None:
raise ImportError(f"The class {type(self).__name__} is only available if the package `cma` is installed.")
# Initialize the base class
SearchAlgorithm.__init__(self, problem, center=self._get_center)
# Ensure that the problem is numeric
problem.ensure_numeric()
# Store the objective index
self._obj_index = problem.normalize_obj_index(obj_index)
# If `center_init` is not given, generate an initial solution
# with the help of the problem object.
# Otherwise, use the given initial solution as the starting
# point in the search space.
if center_init is None:
x0 = self._problem.generate_values(1).to("cpu").view(-1).numpy().astype(dtype=float)
else:
x0 = numpy_copy(center_init, dtype=float)
# Store the initial standard deviations
sigma0 = numpy_copy(stdev_init, dtype=float)
# Generate an options dictionary to pass to the cma solver.
inopts = {}
for k, v in cma_options.items():
if isinstance(v, torch.Tensor):
v = numpy_copy(v, dtype=float)
inopts[k] = v
# Remove the number of iterations boundary
if "maxiter" not in inopts:
inopts["maxiter"] = np.inf
# Below is a temporary helper function for safely storing the configuration items.
# This inner function updates the `inopts` variable.
def store_opt(key: str, long_name: str, value: Any, converter: Callable):
# Here, `key` represents the configuration key used by pycma
# `long_name` represents the configuration's long name used by this class
# `value` is the configuration value associated with `key`.
# Declare that this inner function accesses the `inopts` variable.
nonlocal inopts
if value is None:
# If the provided `value` is None, then there is no configuration to store.
# So, we just leave this inner function.
return
if key in inopts:
# If the given `key` already exists within `inopts`, this means that the configuration was specified
# twice: via the keyword argument `cma_options` AND via a keyword argument.
# We raise an error and inform the user about this redundancy.
raise ValueError(
f"The configuration {repr(key)} was redundantly provided"
f" both via the initialization argument {long_name}"
f" and via the cma_options dictionary."
f" {long_name}={repr(value)};"
f" cma_options[{repr(key)}]={repr(inopts[key])}."
)
inopts[key] = converter(value)
# Temporary helper function which makes sure that `x` is a numpy array or a float.
def array_or_float(x):
if is_sequence(x):
return numpy_copy(x)
else:
return float(x)
# Store the cma configuration received through the initialization arguments (and raise error if there is
# redundancy with the cma_options dictionary).
store_opt("popsize", "popsize", popsize, int)
store_opt("CMA_cmean", "center_learning_rate", center_learning_rate, float)
store_opt("CMA_on", "cov_learning_rate", cov_learning_rate, float)
store_opt("CMA_rankmu", "rankmu_learning_rate", rankmu_learning_rate, float)
store_opt("CMA_rankone", "rankone_learning_rate", rankone_learning_rate, float)
store_opt("minstd", "stdev_min", stdev_min, array_or_float)
store_opt("maxstd", "stdev_max", stdev_max, array_or_float)
if separable:
store_opt("CMA_diagonal", "separable", separable, bool)
# If the problem defines lower and upper bounds, pass these into the options dict.
def process_bounds(bounds: RealOrVector) -> np.ndarray:
if bounds is None:
return None
else:
if is_sequence(bounds):
bounds = numpy_copy(bounds)
else:
bounds = np.array(float(bounds)).repeat(self._problem.solution_length)
return bounds
lb = process_bounds(self._problem.lower_bounds)
ub = process_bounds(self._problem.upper_bounds)
register_bounds = False
if lb is not None and ub is None:
ub = np.array(np.inf).repeat(self._problem.solution_length)
register_bounds = True
elif lb is None and ub is not None:
lb = np.array(-(np.inf)).repeat(self._problem.solution_length)
register_bounds = True
elif lb is not None and ub is not None:
register_bounds = True
if register_bounds:
inopts["bounds"] = [lb, ub]
# Generate a random seed using the problem object for the sake of reproducibility.
if "seed" not in inopts:
inopts["seed"] = int(self._problem.make_randint(tuple(), n=(2**32) - 100) + 100)
# Instantiate the CMAEvolutionStrategy with the prepared configuration items.
self._es = cma.CMAEvolutionStrategy(x0, sigma0, inopts)
# Initialize the population.
self._population: SolutionBatch = self._problem.generate_batch(self._es.popsize, empty=True)
# Use the SinglePopulationAlgorithmMixin to enable additional status reports regarding the population.
SinglePopulationAlgorithmMixin.__init__(self)
restarter
special
¶
This namespace contains the implementations of various restart mechanisms
modify_restart
¶
IPOP (ModifyingRestart)
¶
Source code in evotorch/algorithms/restarter/modify_restart.py
class IPOP(ModifyingRestart):
def __init__(
self,
problem: Problem,
algorithm_class: Type[SearchAlgorithm],
algorithm_args: dict = {},
min_fitness_stdev: float = 1e-9,
popsize_multiplier: float = 2,
):
"""IPOP restart, terminates algorithm when minimum standard deviation in fitness values is hit, multiplies the population size in that case
References:
Glasmachers, Tobias, and Oswin Krause.
"The hessian estimation evolution strategy."
PPSN 2020
Args:
problem (Problem): A Problem to solve
algorithm_class (Type[SearchAlgorithm]): The class of the search algorithm to restart
algorithm_args (dict): Arguments to pass to the search algorithm on restart
min_fitness_stdev (float): The minimum standard deviation in fitnesses; going below this will trigger a restart
popsize_multiplier (float): A multiplier on the population size within algorithm_args
"""
super().__init__(problem, algorithm_class, algorithm_args)
self.min_fitness_stdev = min_fitness_stdev
self.popsize_multiplier = popsize_multiplier
def _search_algorithm_terminated(self) -> bool:
# Additional check on standard deviation of fitnesses of population
if self.search_algorithm.population.evals.std() < self.min_fitness_stdev:
return True
return super()._search_algorithm_terminated()
def _modify_algorithm_args(self) -> None:
# Only modify arguments if this isn't the first restart
if self.num_restarts >= 1:
new_args = deepcopy(self._algorithm_args)
# Multiply population size
new_args["popsize"] = int(self.popsize_multiplier * len(self.search_algorithm.population))
self._algorithm_args = new_args
__init__(self, problem, algorithm_class, algorithm_args={}, min_fitness_stdev=1e-09, popsize_multiplier=2)
special
¶
IPOP restart, terminates algorithm when minimum standard deviation in fitness values is hit, multiplies the population size in that case
References
Glasmachers, Tobias, and Oswin Krause. "The hessian estimation evolution strategy." PPSN 2020
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
A Problem to solve |
required |
algorithm_class |
Type[SearchAlgorithm] |
The class of the search algorithm to restart |
required |
algorithm_args |
dict |
Arguments to pass to the search algorithm on restart |
{} |
min_fitness_stdev |
float |
The minimum standard deviation in fitnesses; going below this will trigger a restart |
1e-09 |
popsize_multiplier |
float |
A multiplier on the population size within algorithm_args |
2 |
Source code in evotorch/algorithms/restarter/modify_restart.py
def __init__(
self,
problem: Problem,
algorithm_class: Type[SearchAlgorithm],
algorithm_args: dict = {},
min_fitness_stdev: float = 1e-9,
popsize_multiplier: float = 2,
):
"""IPOP restart, terminates algorithm when minimum standard deviation in fitness values is hit, multiplies the population size in that case
References:
Glasmachers, Tobias, and Oswin Krause.
"The hessian estimation evolution strategy."
PPSN 2020
Args:
problem (Problem): A Problem to solve
algorithm_class (Type[SearchAlgorithm]): The class of the search algorithm to restart
algorithm_args (dict): Arguments to pass to the search algorithm on restart
min_fitness_stdev (float): The minimum standard deviation in fitnesses; going below this will trigger a restart
popsize_multiplier (float): A multiplier on the population size within algorithm_args
"""
super().__init__(problem, algorithm_class, algorithm_args)
self.min_fitness_stdev = min_fitness_stdev
self.popsize_multiplier = popsize_multiplier
restart
¶
Restart (SearchAlgorithm)
¶
Source code in evotorch/algorithms/restarter/restart.py
class Restart(SearchAlgorithm):
def __init__(
self,
problem: Problem,
algorithm_class: Type[SearchAlgorithm],
algorithm_args: dict = {},
**kwargs: Any,
):
"""Base class for independent restarts methods
Args:
problem (Problem): A Problem to solve
algorithm_class (Type[SearchAlgorithm]): The class of the search algorithm to restart
algorithm_args (dict): Arguments to pass to the search algorithm on restart
"""
SearchAlgorithm.__init__(
self,
problem,
search_algorithm=self._get_sa_status,
num_restarts=self._get_num_restarts,
algorithm_terminated=self._search_algorithm_terminated,
**kwargs,
)
self._algorithm_class = algorithm_class
self._algorithm_args = algorithm_args
self.num_restarts = 0
self._restart()
def _get_sa_status(self) -> dict:
"""Status dictionary of search algorithm"""
return self.search_algorithm.status
def _get_num_restarts(self) -> int:
"""Number of restarts (including the first start) so far"""
return self.num_restarts
def _restart(self) -> None:
"""Restart the search algorithm"""
self.search_algorithm = self._algorithm_class(self._problem, **self._algorithm_args)
self.num_restarts += 1
def _search_algorithm_terminated(self) -> bool:
"""Boolean flag for search algorithm terminated"""
return self.search_algorithm.is_terminated
def _step(self):
# Step the search algorithm
self.search_algorithm.step()
# If stepping the search algorithm has reached a terminal state, restart the search algorithm
if self._search_algorithm_terminated():
self._restart()
__init__(self, problem, algorithm_class, algorithm_args={}, **kwargs)
special
¶
Base class for independent restarts methods
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
A Problem to solve |
required |
algorithm_class |
Type[SearchAlgorithm] |
The class of the search algorithm to restart |
required |
algorithm_args |
dict |
Arguments to pass to the search algorithm on restart |
{} |
Source code in evotorch/algorithms/restarter/restart.py
def __init__(
self,
problem: Problem,
algorithm_class: Type[SearchAlgorithm],
algorithm_args: dict = {},
**kwargs: Any,
):
"""Base class for independent restarts methods
Args:
problem (Problem): A Problem to solve
algorithm_class (Type[SearchAlgorithm]): The class of the search algorithm to restart
algorithm_args (dict): Arguments to pass to the search algorithm on restart
"""
SearchAlgorithm.__init__(
self,
problem,
search_algorithm=self._get_sa_status,
num_restarts=self._get_num_restarts,
algorithm_terminated=self._search_algorithm_terminated,
**kwargs,
)
self._algorithm_class = algorithm_class
self._algorithm_args = algorithm_args
self.num_restarts = 0
self._restart()
searchalgorithm
¶
This namespace contains SearchAlgorithm, the base class for all
evolutionary algorithms.
LazyReporter
¶
This class provides an interface of storing and reporting status. This class is designed to be inherited by other classes.
Let us assume that we have the following class inheriting from LazyReporter:
class Example(LazyReporter):
def __init__(self):
LazyReporter.__init__(self, a=self._get_a, b=self._get_b)
def _get_a(self):
return ... # return the status 'a'
def _get_b(self):
return ... # return the status 'b'
At its initialization phase, this Example class registers its methods
_get_a and _get_b as its status providers.
Having the LazyReporter interface, the Example class gains a status
property:
ex = Example()
print(ex.status["a"]) # Get the status 'a'
print(ex.status["b"]) # Get the status 'b'
Once a status is queried, its computation result is stored to be re-used later. After running the code above, if we query the status 'a' again:
then the status 'a' is not computed again (i.e. _get_a is not
called again). Instead, the stored status value of 'a' is re-used.
To force re-computation of the status values, one can execute:
Or the Example instance can clear its status from within one of its methods:
Source code in evotorch/algorithms/searchalgorithm.py
class LazyReporter:
"""
This class provides an interface of storing and reporting status.
This class is designed to be inherited by other classes.
Let us assume that we have the following class inheriting from
LazyReporter:
```python
class Example(LazyReporter):
def __init__(self):
LazyReporter.__init__(self, a=self._get_a, b=self._get_b)
def _get_a(self):
return ... # return the status 'a'
def _get_b(self):
return ... # return the status 'b'
```
At its initialization phase, this Example class registers its methods
``_get_a`` and ``_get_b`` as its status providers.
Having the LazyReporter interface, the Example class gains a ``status``
property:
```python
ex = Example()
print(ex.status["a"]) # Get the status 'a'
print(ex.status["b"]) # Get the status 'b'
```
Once a status is queried, its computation result is stored to be re-used
later. After running the code above, if we query the status 'a' again:
```python
print(ex.status["a"]) # Getting the status 'a' again
```
then the status 'a' is not computed again (i.e. ``_get_a`` is not
called again). Instead, the stored status value of 'a' is re-used.
To force re-computation of the status values, one can execute:
```python
ex.clear_status()
```
Or the Example instance can clear its status from within one of its
methods:
```python
class Example(LazyReporter):
...
def some_method(self):
...
self.clear_status()
```
"""
@staticmethod
def _missing_status_producer():
return None
def __init__(self, **kwargs):
"""
`__init__(...)`: Initialize the LazyReporter instance.
Args:
kwargs: Keyword arguments, mapping the status keys to the
methods or functions providing the status values.
"""
self.__getters = kwargs
self.__computed = {}
def get_status_value(self, key: Any) -> Any:
"""
Get the specified status value.
Args:
key: The key (i.e. the name) of the status variable.
"""
if key not in self.__computed:
self.__computed[key] = self.__getters[key]()
return self.__computed[key]
def has_status_key(self, key: Any) -> bool:
"""
Return True if there is a status variable with the specified key.
Otherwise, return False.
Args:
key: The key (i.e. the name) of the status variable whose
existence is to be checked.
Returns:
True if there is such a key; False otherwise.
"""
return key in self.__getters
def iter_status_keys(self):
"""Iterate over the status keys."""
return self.__getters.keys()
def clear_status(self):
"""Clear all the stored values of the status variables."""
self.__computed.clear()
def is_status_computed(self, key) -> bool:
"""
Return True if the specified status is computed yet.
Return False otherwise.
Args:
key: The key (i.e. the name) of the status variable.
Returns:
True if the status of the given key is computed; False otherwise.
"""
return key in self.__computed
def update_status(self, additional_status: Mapping):
"""
Update the stored status with an external dict-like object.
The given dict-like object can override existing status keys
with new values, and also bring new keys to the status.
Args:
additional_status: A dict-like object storing the status update.
"""
for k, v in additional_status.items():
if k not in self.__getters:
self.__getters[k] = LazyReporter._missing_status_producer
self.__computed[k] = v
def add_status_getters(self, getters: Mapping):
"""
Register additional status-getting functions.
Args:
getters: A dictionary-like object where the keys are the
additional status variable names, and values are functions
which are expected to compute/retrieve the values for those
status variables.
"""
self.__getters.update(getters)
@property
def status(self) -> "LazyStatusDict":
"""Get a LazyStatusDict which is bound to this LazyReporter."""
return LazyStatusDict(self)
status: LazyStatusDict
property
readonly
¶
Get a LazyStatusDict which is bound to this LazyReporter.
__init__(self, **kwargs)
special
¶
__init__(...): Initialize the LazyReporter instance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
kwargs |
Keyword arguments, mapping the status keys to the methods or functions providing the status values. |
{} |
Source code in evotorch/algorithms/searchalgorithm.py
add_status_getters(self, getters)
¶
Register additional status-getting functions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
getters |
Mapping |
A dictionary-like object where the keys are the additional status variable names, and values are functions which are expected to compute/retrieve the values for those status variables. |
required |
Source code in evotorch/algorithms/searchalgorithm.py
def add_status_getters(self, getters: Mapping):
"""
Register additional status-getting functions.
Args:
getters: A dictionary-like object where the keys are the
additional status variable names, and values are functions
which are expected to compute/retrieve the values for those
status variables.
"""
self.__getters.update(getters)
clear_status(self)
¶
get_status_value(self, key)
¶
Get the specified status value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Any |
The key (i.e. the name) of the status variable. |
required |
Source code in evotorch/algorithms/searchalgorithm.py
has_status_key(self, key)
¶
Return True if there is a status variable with the specified key. Otherwise, return False.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Any |
The key (i.e. the name) of the status variable whose existence is to be checked. |
required |
Returns:
| Type | Description |
|---|---|
bool |
True if there is such a key; False otherwise. |
Source code in evotorch/algorithms/searchalgorithm.py
def has_status_key(self, key: Any) -> bool:
"""
Return True if there is a status variable with the specified key.
Otherwise, return False.
Args:
key: The key (i.e. the name) of the status variable whose
existence is to be checked.
Returns:
True if there is such a key; False otherwise.
"""
return key in self.__getters
is_status_computed(self, key)
¶
Return True if the specified status is computed yet. Return False otherwise.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
The key (i.e. the name) of the status variable. |
required |
Returns:
| Type | Description |
|---|---|
bool |
True if the status of the given key is computed; False otherwise. |
Source code in evotorch/algorithms/searchalgorithm.py
iter_status_keys(self)
¶
update_status(self, additional_status)
¶
Update the stored status with an external dict-like object. The given dict-like object can override existing status keys with new values, and also bring new keys to the status.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
additional_status |
Mapping |
A dict-like object storing the status update. |
required |
Source code in evotorch/algorithms/searchalgorithm.py
def update_status(self, additional_status: Mapping):
"""
Update the stored status with an external dict-like object.
The given dict-like object can override existing status keys
with new values, and also bring new keys to the status.
Args:
additional_status: A dict-like object storing the status update.
"""
for k, v in additional_status.items():
if k not in self.__getters:
self.__getters[k] = LazyReporter._missing_status_producer
self.__computed[k] = v
LazyStatusDict (Mapping)
¶
A Mapping subclass used by the status property of a LazyReporter.
The interface of this object is similar to a read-only dictionary.
Source code in evotorch/algorithms/searchalgorithm.py
class LazyStatusDict(Mapping):
"""
A Mapping subclass used by the `status` property of a `LazyReporter`.
The interface of this object is similar to a read-only dictionary.
"""
def __init__(self, lazy_reporter: LazyReporter):
"""
`__init__(...)`: Initialize the LazyStatusDict object.
Args:
lazy_reporter: The LazyReporter object whose status is to be
accessed.
"""
super().__init__()
self.__lazy_reporter = lazy_reporter
def __getitem__(self, key: Any) -> Any:
result = self.__lazy_reporter.get_status_value(key)
if isinstance(result, (torch.Tensor, ObjectArray)):
result = as_read_only_tensor(result)
return result
def __len__(self) -> int:
return len(list(self.__lazy_reporter.iter_status_keys()))
def __iter__(self):
for k in self.__lazy_reporter.iter_status_keys():
yield k
def __contains__(self, key: Any) -> bool:
return self.__lazy_reporter.has_status_key(key)
def _to_string(self) -> str:
with io.StringIO() as f:
print("<" + type(self).__name__, file=f)
for k in self.__lazy_reporter.iter_status_keys():
if self.__lazy_reporter.is_status_computed(k):
r = repr(self.__lazy_reporter.get_status_value(k))
else:
r = "<not yet computed>"
print(" ", k, "=", r, file=f)
print(">", end="", file=f)
f.seek(0)
entire_str = f.read()
return entire_str
def __str__(self) -> str:
return self._to_string()
def __repr__(self) -> str:
return self._to_string()
__init__(self, lazy_reporter)
special
¶
__init__(...): Initialize the LazyStatusDict object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
lazy_reporter |
LazyReporter |
The LazyReporter object whose status is to be accessed. |
required |
SearchAlgorithm (LazyReporter)
¶
Base class for all evolutionary search algorithms.
An algorithm developer is expected to inherit from this base class,
and override the method named _step() to define how a single
step of this new algorithm is performed.
For each core status dictionary element, a new method is expected
to exist within the inheriting class. These status reporting
methods are then registered via the keyword arguments of the
__init__(...) method of SearchAlgorithm.
To sum up, a newly developed algorithm inheriting from this base class is expected in this structure:
from evotorch import Problem
class MyNewAlgorithm(SearchAlgorithm):
def __init__(self, problem: Problem):
SearchAlgorithm.__init__(
self, problem, status1=self._get_status1, status2=self._get_status2, ...
)
def _step(self):
# Code that defines how a step of this algorithm
# should work goes here.
...
def _get_status1(self):
# The value returned by this function will be shown
# in the status dictionary, associated with the key
# 'status1'.
return ...
def _get_status2(self):
# The value returned by this function will be shown
# in the status dictionary, associated with the key
# 'status2'.
return ...
Source code in evotorch/algorithms/searchalgorithm.py
class SearchAlgorithm(LazyReporter):
"""
Base class for all evolutionary search algorithms.
An algorithm developer is expected to inherit from this base class,
and override the method named `_step()` to define how a single
step of this new algorithm is performed.
For each core status dictionary element, a new method is expected
to exist within the inheriting class. These status reporting
methods are then registered via the keyword arguments of the
`__init__(...)` method of `SearchAlgorithm`.
To sum up, a newly developed algorithm inheriting from this base
class is expected in this structure:
```python
from evotorch import Problem
class MyNewAlgorithm(SearchAlgorithm):
def __init__(self, problem: Problem):
SearchAlgorithm.__init__(
self, problem, status1=self._get_status1, status2=self._get_status2, ...
)
def _step(self):
# Code that defines how a step of this algorithm
# should work goes here.
...
def _get_status1(self):
# The value returned by this function will be shown
# in the status dictionary, associated with the key
# 'status1'.
return ...
def _get_status2(self):
# The value returned by this function will be shown
# in the status dictionary, associated with the key
# 'status2'.
return ...
```
"""
def __init__(self, problem: Problem, **kwargs):
"""
Initialize the SearchAlgorithm instance.
Args:
problem: Problem to work with.
kwargs: Any additional keyword argument, in the form of `k=f`,
is accepted in this manner: for each pair of `k` and `f`,
`k` is accepted as the status key (i.e. a status variable
name), and `f` is accepted as a function (probably a method
of the inheriting class) that will generate the value of that
status variable.
"""
super().__init__(**kwargs)
self._problem = problem
self._before_step_hook = Hook()
self._after_step_hook = Hook()
self._log_hook = Hook()
self._end_of_run_hook = Hook()
self._steps_count: int = 0
self._first_step_datetime: Optional[datetime] = None
@property
def problem(self) -> Problem:
"""
The problem object which is being worked on.
"""
return self._problem
@property
def before_step_hook(self) -> Hook:
"""
Use this Hook to add more behavior to the search algorithm
to be performed just before executing a step.
"""
return self._before_step_hook
@property
def after_step_hook(self) -> Hook:
"""
Use this Hook to add more behavior to the search algorithm
to be performed just after executing a step.
The dictionaries returned by the functions registered into
this Hook will be accumulated and added into the status
dictionary of the search algorithm.
"""
return self._after_step_hook
@property
def log_hook(self) -> Hook:
"""
Use this Hook to add more behavior to the search algorithm
at the moment of logging the constructed status dictionary.
This Hook is executed after the execution of `after_step_hook`
is complete.
The functions in this Hook are assumed to expect a single
argument, that is the status dictionary of the search algorithm.
"""
return self._log_hook
@property
def end_of_run_hook(self) -> Hook:
"""
Use this Hook to add more behavior to the search algorithm
at the end of a run.
This Hook is executed after all the generations of a run
are done.
The functions in this Hook are assumed to expect a single
argument, that is the status dictionary of the search algorithm.
"""
return self._end_of_run_hook
@property
def step_count(self) -> int:
"""
Number of search steps performed.
This is equivalent to the number of generations, or to the
number of iterations.
"""
return self._steps_count
@property
def steps_count(self) -> int:
"""
Deprecated alias for the `step_count` property.
It is recommended to use the `step_count` property instead.
"""
return self._steps_count
def step(self):
"""
Perform a step of the search algorithm.
"""
self._before_step_hook()
self.clear_status()
if self._first_step_datetime is None:
self._first_step_datetime = datetime.now()
self._step()
self._steps_count += 1
self.update_status({"iter": self._steps_count})
self.update_status(self._problem.status)
extra_status = self._after_step_hook.accumulate_dict()
self.update_status(extra_status)
if len(self._log_hook) >= 1:
self._log_hook(dict(self.status))
def _step(self):
"""
Algorithm developers are expected to override this method
in an inheriting subclass.
The code which defines how a step of the evolutionary algorithm
is performed goes here.
"""
raise NotImplementedError
def run(self, num_generations: int, *, reset_first_step_datetime: bool = True):
"""
Run the algorithm for the given number of generations
(i.e. iterations).
Args:
num_generations: Number of generations.
reset_first_step_datetime: If this argument is given as True,
then, the datetime of the first search step will be forgotten.
Forgetting the first step's datetime means that the first step
taken by this new run will be the new first step datetime.
"""
if reset_first_step_datetime:
self.reset_first_step_datetime()
for _ in range(int(num_generations)):
self.step()
if len(self._end_of_run_hook) >= 1:
self._end_of_run_hook(dict(self.status))
@property
def first_step_datetime(self) -> Optional[datetime]:
"""
Get the datetime when the algorithm took the first search step.
If a step is not taken at all, then the result will be None.
"""
return self._first_step_datetime
def reset_first_step_datetime(self):
"""
Reset (or forget) the first step's datetime.
"""
self._first_step_datetime = None
@property
def is_terminated(self) -> bool:
"""Whether the algorithm is in a terminal state"""
return False
after_step_hook: Hook
property
readonly
¶
Use this Hook to add more behavior to the search algorithm to be performed just after executing a step.
The dictionaries returned by the functions registered into this Hook will be accumulated and added into the status dictionary of the search algorithm.
before_step_hook: Hook
property
readonly
¶
Use this Hook to add more behavior to the search algorithm to be performed just before executing a step.
end_of_run_hook: Hook
property
readonly
¶
Use this Hook to add more behavior to the search algorithm at the end of a run.
This Hook is executed after all the generations of a run are done.
The functions in this Hook are assumed to expect a single argument, that is the status dictionary of the search algorithm.
first_step_datetime: Optional[datetime.datetime]
property
readonly
¶
Get the datetime when the algorithm took the first search step. If a step is not taken at all, then the result will be None.
is_terminated: bool
property
readonly
¶
Whether the algorithm is in a terminal state
log_hook: Hook
property
readonly
¶
Use this Hook to add more behavior to the search algorithm at the moment of logging the constructed status dictionary.
This Hook is executed after the execution of after_step_hook
is complete.
The functions in this Hook are assumed to expect a single argument, that is the status dictionary of the search algorithm.
problem: Problem
property
readonly
¶
The problem object which is being worked on.
step_count: int
property
readonly
¶
Number of search steps performed.
This is equivalent to the number of generations, or to the number of iterations.
steps_count: int
property
readonly
¶
Deprecated alias for the step_count property.
It is recommended to use the step_count property instead.
__init__(self, problem, **kwargs)
special
¶
Initialize the SearchAlgorithm instance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
Problem to work with. |
required |
kwargs |
Any additional keyword argument, in the form of |
{} |
Source code in evotorch/algorithms/searchalgorithm.py
def __init__(self, problem: Problem, **kwargs):
"""
Initialize the SearchAlgorithm instance.
Args:
problem: Problem to work with.
kwargs: Any additional keyword argument, in the form of `k=f`,
is accepted in this manner: for each pair of `k` and `f`,
`k` is accepted as the status key (i.e. a status variable
name), and `f` is accepted as a function (probably a method
of the inheriting class) that will generate the value of that
status variable.
"""
super().__init__(**kwargs)
self._problem = problem
self._before_step_hook = Hook()
self._after_step_hook = Hook()
self._log_hook = Hook()
self._end_of_run_hook = Hook()
self._steps_count: int = 0
self._first_step_datetime: Optional[datetime] = None
reset_first_step_datetime(self)
¶
run(self, num_generations, *, reset_first_step_datetime=True)
¶
Run the algorithm for the given number of generations (i.e. iterations).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
num_generations |
int |
Number of generations. |
required |
reset_first_step_datetime |
bool |
If this argument is given as True, then, the datetime of the first search step will be forgotten. Forgetting the first step's datetime means that the first step taken by this new run will be the new first step datetime. |
True |
Source code in evotorch/algorithms/searchalgorithm.py
def run(self, num_generations: int, *, reset_first_step_datetime: bool = True):
"""
Run the algorithm for the given number of generations
(i.e. iterations).
Args:
num_generations: Number of generations.
reset_first_step_datetime: If this argument is given as True,
then, the datetime of the first search step will be forgotten.
Forgetting the first step's datetime means that the first step
taken by this new run will be the new first step datetime.
"""
if reset_first_step_datetime:
self.reset_first_step_datetime()
for _ in range(int(num_generations)):
self.step()
if len(self._end_of_run_hook) >= 1:
self._end_of_run_hook(dict(self.status))
step(self)
¶
Perform a step of the search algorithm.
Source code in evotorch/algorithms/searchalgorithm.py
def step(self):
"""
Perform a step of the search algorithm.
"""
self._before_step_hook()
self.clear_status()
if self._first_step_datetime is None:
self._first_step_datetime = datetime.now()
self._step()
self._steps_count += 1
self.update_status({"iter": self._steps_count})
self.update_status(self._problem.status)
extra_status = self._after_step_hook.accumulate_dict()
self.update_status(extra_status)
if len(self._log_hook) >= 1:
self._log_hook(dict(self.status))
SinglePopulationAlgorithmMixin
¶
A mixin class that can be inherited by a SearchAlgorithm subclass.
This mixin class assumes that the inheriting class has the following members:
problem: The problem object that is associated with the search algorithm. This attribute is already provided by the SearchAlgorithm base class.population: An attribute or a (possibly read-only) property which stores the population of the search algorithm as aSolutionBatchinstance.
This mixin class also assumes that the inheriting class might
contain an attribute (or a property) named obj_index.
If there is such an attribute and its value is not None, then this
mixin class assumes that obj_index represents the index of the
objective that is being focused on.
Upon initialization, this mixin class first determines whether or not
the algorithm is a single-objective one.
In more details, if there is an attribute named obj_index (and its
value is not None), or if the associated problem has only one objective,
then this mixin class assumes that the inheriting SearchAlgorithm is a
single objective algorithm.
Otherwise, it is assumed that the underlying algorithm works (or might
work) on multiple objectives.
In the single-objective case, this mixin class brings the inheriting
SearchAlgorithm the ability to report the following:
pop_best (best solution of the population),
pop_best_eval (evaluation result of the population's best solution),
mean_eval (mean evaluation result of the population),
median_eval (median evaluation result of the population).
In the multi-objective case, for each objective i, this mixin class
brings the inheriting SearchAlgorithm the ability to report the following:
obj<i>_pop_best (best solution of the population according),
obj<i>_pop_best_eval (evaluation result of the population's best
solution),
obj<i>_mean_eval (mean evaluation result of the population)
obj<iP_median_eval (median evaluation result of the population).
Source code in evotorch/algorithms/searchalgorithm.py
class SinglePopulationAlgorithmMixin:
"""
A mixin class that can be inherited by a SearchAlgorithm subclass.
This mixin class assumes that the inheriting class has the following
members:
- `problem`: The problem object that is associated with the search
algorithm. This attribute is already provided by the SearchAlgorithm
base class.
- `population`: An attribute or a (possibly read-only) property which
stores the population of the search algorithm as a `SolutionBatch`
instance.
This mixin class also assumes that the inheriting class _might_
contain an attribute (or a property) named `obj_index`.
If there is such an attribute and its value is not None, then this
mixin class assumes that `obj_index` represents the index of the
objective that is being focused on.
Upon initialization, this mixin class first determines whether or not
the algorithm is a single-objective one.
In more details, if there is an attribute named `obj_index` (and its
value is not None), or if the associated problem has only one objective,
then this mixin class assumes that the inheriting SearchAlgorithm is a
single objective algorithm.
Otherwise, it is assumed that the underlying algorithm works (or might
work) on multiple objectives.
In the single-objective case, this mixin class brings the inheriting
SearchAlgorithm the ability to report the following:
`pop_best` (best solution of the population),
`pop_best_eval` (evaluation result of the population's best solution),
`mean_eval` (mean evaluation result of the population),
`median_eval` (median evaluation result of the population).
In the multi-objective case, for each objective `i`, this mixin class
brings the inheriting SearchAlgorithm the ability to report the following:
`obj<i>_pop_best` (best solution of the population according),
`obj<i>_pop_best_eval` (evaluation result of the population's best
solution),
`obj<i>_mean_eval` (mean evaluation result of the population)
`obj<iP_median_eval` (median evaluation result of the population).
"""
class ObjectiveStatusReporter:
REPORTABLES = {"pop_best", "pop_best_eval", "mean_eval", "median_eval"}
def __init__(
self,
algorithm: SearchAlgorithm,
*,
obj_index: int,
to_report: str,
):
self.__algorithm = algorithm
self.__obj_index = int(obj_index)
if to_report not in self.REPORTABLES:
raise ValueError(f"Unrecognized report request: {to_report}")
self.__to_report = to_report
@property
def population(self) -> SolutionBatch:
return self.__algorithm.population
@property
def obj_index(self) -> int:
return self.__obj_index
def get_status_value(self, status_key: str) -> Any:
return self.__algorithm.get_status_value(status_key)
def has_status_key(self, status_key: str) -> bool:
return self.__algorithm.has_status_key(status_key)
def _get_pop_best(self):
i = self.population.argbest(self.obj_index)
return clone(self.population[i])
def _get_pop_best_eval(self):
pop_best = None
pop_best_keys = ("pop_best", f"obj{self.obj_index}_pop_best")
for pop_best_key in pop_best_keys:
if self.has_status_key(pop_best_key):
pop_best = self.get_status_value(pop_best_key)
break
if (pop_best is not None) and pop_best.is_evaluated:
return float(pop_best.evals[self.obj_index])
else:
return None
@torch.no_grad()
def _get_mean_eval(self):
return float(torch.mean(self.population.access_evals(self.obj_index)))
@torch.no_grad()
def _get_median_eval(self):
return float(torch.median(self.population.access_evals(self.obj_index)))
def __call__(self):
return getattr(self, "_get_" + self.__to_report)()
def __init__(self, *, exclude: Optional[Iterable] = None, enable: bool = True):
if not enable:
return
ObjectiveStatusReporter = self.ObjectiveStatusReporter
reportables = ObjectiveStatusReporter.REPORTABLES
single_obj: Optional[int] = None
self.__exclude = set() if exclude is None else set(exclude)
if hasattr(self, "obj_index") and (self.obj_index is not None):
single_obj = self.obj_index
elif len(self.problem.senses) == 1:
single_obj = 0
if single_obj is not None:
for reportable in reportables:
if reportable not in self.__exclude:
self.add_status_getters(
{reportable: ObjectiveStatusReporter(self, obj_index=single_obj, to_report=reportable)}
)
else:
for i_obj in range(len(self.problem.senses)):
for reportable in reportables:
if reportable not in self.__exclude:
self.add_status_getters(
{
f"obj{i_obj}_{reportable}": ObjectiveStatusReporter(
self, obj_index=i_obj, to_report=reportable
),
}
)
core
¶
Definitions of the core classes: Problem, Solution, and SolutionBatch.
ActorSeeds (tuple)
¶
AllRemoteEnvs
¶
Representation of all remote reinforcement learning instances stored by the ray actors.
An instance of this class is to be obtained from a main (i.e. non-remote) Problem object, as follows:
remote_envs = my_problem.all_remote_envs()
A remote method f() on all remote environments can then be executed as follows:
results = remote_envs.f()
Given that there are n actors, results contains n objects,
the i-th object being the method's result from the i-th actor.
An alternative to the example above is like this:
results = my_problem.all_remote_envs().f()
Source code in evotorch/core.py
class AllRemoteEnvs:
"""
Representation of all remote reinforcement learning instances
stored by the ray actors.
An instance of this class is to be obtained from a main
(i.e. non-remote) Problem object, as follows:
remote_envs = my_problem.all_remote_envs()
A remote method f() on all remote environments can then
be executed as follows:
results = remote_envs.f()
Given that there are `n` actors, `results` contains `n` objects,
the i-th object being the method's result from the i-th actor.
An alternative to the example above is like this:
results = my_problem.all_remote_envs().f()
"""
def __init__(self, actors: list):
self._actors = actors
def __getattr__(self, attr_name: str) -> Any:
return RemoteMethod(attr_name, self._actors, on_env=True)
AllRemoteProblems
¶
Representation of all remote problem instances stored by the ray actors.
An instance of this class is to be obtained from a main (i.e. non-remote) Problem object, as follows:
remote_probs = my_problem.all_remote_problems()
A remote method f() on all remote Problem instances can then be executed as follows:
results = remote_probs.f()
Given that there are n actors, results contains n objects,
the i-th object being the method's result from the i-th actor.
An alternative to the example above is like this:
results = my_problem.all_remote_problems().f()
Source code in evotorch/core.py
class AllRemoteProblems:
"""
Representation of all remote problem instances stored by the ray actors.
An instance of this class is to be obtained from a main
(i.e. non-remote) Problem object, as follows:
remote_probs = my_problem.all_remote_problems()
A remote method f() on all remote Problem instances can then
be executed as follows:
results = remote_probs.f()
Given that there are `n` actors, `results` contains `n` objects,
the i-th object being the method's result from the i-th actor.
An alternative to the example above is like this:
results = my_problem.all_remote_problems().f()
"""
def __init__(self, actors: list):
self._actors = actors
def __getattr__(self, attr_name: str) -> Any:
return RemoteMethod(attr_name, self._actors)
BoundsPair (tuple)
¶
ParetoInfo (tuple)
¶
Problem (TensorMakerMixin, Serializable)
¶
Representation of a problem to be optimized.
The simplest way to use this class is to instantiate it with an external fitness function.
Let us imagine that we have the following fitness function:
A problem definition can be made around this fitness function as follows:
from evotorch import Problem
problem = Problem(
"min", f, # Goal is to minimize f (would be "max" for maximization)
solution_length=10, # Length of a solution is 10
initial_bounds=(-5.0, 5.0), # Bounds for sampling a new solution
dtype=torch.float32, # dtype of a solution
)
Vectorized problem definitions. To boost the runtime performance, one might want to define a vectorized fitness function where the fitnesses of multiple solutions are computed in a batched manner using the vectorization capabilities of PyTorch. A vectorized problem definition can be made as follows:
from evotorch.decorators import vectorized
@vectorized
def vf(solutions: torch.Tensor) -> torch.Tensor:
return torch.linalg.norm(solutions ** 2, dim=-1)
problem = Problem(
"min", vf, # Goal is to minimize vf (would be "max" for maximization)
solution_length=10, # Length of a solution is 10
initial_bounds=(-5.0, 5.0), # Bounds for sampling a new solution
dtype=torch.float32, # dtype of a solution
)
Parallelization across multiple CPUs. An optimization problem can be configured to parallelize its evaluation operations across multiple CPUs as follows:
Exploiting hardware accelerators. As an alternative to CPU-based parallelization, one might prefer to use the parallelized computation capabilities of a hardware accelerator such as CUDA. To load the problem onto a cuda device (for example, onto "cuda:0"), one can do:
from evotorch.decorators import vectorized
@vectorized
def vf(solutions: torch.Tensor) -> torch.Tensor:
return ...
problem = Problem("min", vf, ..., device="cuda:0")
Exploiting multiple GPUs in parallel.
One can also keep the entire population on the CPU, and split and distribute
it to multiple GPUs for GPU-accelerated and parallelized fitness evaluation.
For this, the main device of the problem is set as CPU, but the fitness
function is decorated with evotorch.decorators.on_cuda.
from evotorch.decorators import on_cuda, vectorized
@on_cuda
@vectorized
def vf(solutions: torch.Tensor) -> torch.Tensor:
return ...
problem = Problem(
"min",
vf,
...,
num_actors=N, # where N>1 and equal to the number of GPUs
# Note: if you are on a computer or on a ray cluster with multiple
# GPUs, you might prefer to use the string "num_gpus" instead of an
# integer N, which will cause the number of available GPUs to be
# counted, and the number of actors to be configured as that count.
#
num_gpus_per_actor=1, # each GPU is assigned to an actor
device="cpu",
)
Defining problems via inheritance. A problem can also be defined via inheritance. Using inheritance, one can define problems which carry their own additional data, and/or update their states as more solutions are evaluated, and/or have custom procedures for sampling new solutions, etc.
As a first example, let us define a parameterized problem. In this example
problem, the goal is to minimize x^(2q), q being a parameter of the
problem. The definition of such a problem can be as follows:
from evotorch import Problem, Solution
class MyProblem(Problem):
def __init__(self, q: float):
self.q = float(q)
super().__init__(
objective_sense="min", # the goal is to minimize
solution_length=10, # a solution has the length 10
initial_bounds=(-5.0, 5.0), # sample new solutions from within [-5, 5]
dtype=torch.float32, # the dtype of a solution is float32
# num_actors=..., # if parallelization via multiple actors is desired
)
def _evaluate(self, solution: Solution):
# This is where we declare the procedure of evaluating a solution
# Get the decision values of the solution as a PyTorch tensor
x = solution.values
# Compute the fitness
fitness = torch.sum(x ** (2 * self.q))
# Register the fitness into the Solution object
solution.set_evaluation(fitness)
This parameterized problem can be instantiated as follows (let's say with q=3):
Defining vectorized problems via inheritance.
Vectorization can be used with inheritance-based problem definitions as well.
Please see the following example where the method _evaluate_batch
is used instead of _evaluate for vectorization:
from evotorch import Problem, SolutionBatch
class MyVectorizedProblem(Problem):
def __init__(self, q: float):
self.q = float(q)
super().__init__(
objective_sense="min", # the goal is to minimize
solution_length=10, # a solution has the length 10
initial_bounds=(-5.0, 5.0), # sample new solutions from within [-5, 5]
dtype=torch.float32, # the dtype of a solution is float32
# num_actors=..., # if parallelization via multiple actors is desired
# device="cuda:0", # if hardware acceleration is desired
)
def _evaluate_batch(self, solutions: SolutionBatch):
# Get the decision values of all the solutions in a 2D PyTorch tensor:
xs = solutions.values
# Compute the fitnesses
fitnesses = torch.sum(x ** (2 * self.q), dim=-1)
# Register the fitness into the Solution object
solutions.set_evals(fitnesses)
Using multiple GPUs from a problem defined via inheritance.
The previous example demonstrating the use of multiple GPUs showed how
an independent fitness function can be decorated via
evotorch.decorators.on_cuda. Instead of using an independent fitness
function, if one wishes to define a problem by subclassing Problem,
the overriden method _evaluate_batch(...) has to be decorated by
evotorch.decorators.on_cuda. Like in the previous multi-GPU example,
let us assume that we want to parallelize the fitness evaluation
across N GPUs (where N>1). The inheritance-based code to achieve this
can look like this:
from evotorch import Problem, SolutionBatch
from evotorch.decorators import on_cuda
class MyMultiGPUProblem(Problem):
def __init__(self):
...
super().__init__(
objective_sense="min", # the goal is to minimize
solution_length=10, # a solution has the length 10
initial_bounds=(-5.0, 5.0), # sample new solutions from within [-5, 5]
dtype=torch.float32, # the dtype of a solution is float32
num_actors=N, # allocate N actors
# Note: if you are on a computer or on a ray cluster with multiple
# GPUs, you might prefer to use the string "num_gpus" instead of an
# integer N, which will cause the number of available GPUs to be
# counted, and the number of actors to be configured as that count.
#
num_gpus_per_actor=1, # for each actor, assign a cuda device
device="cpu", # keep the main population on the CPU
)
@on_cuda
def _evaluate_batch(self, solutions: SolutionBatch):
# Get the decision values of all the solutions in a 2D PyTorch tensor:
xs = solutions.values
# Compute the fitnesses
fitnesses = ...
# Register the fitness into the Solution object
solutions.set_evals(fitnesses)
Customizing how initial solutions are sampled.
Instead of sampling solutions from within an interval, one might wish to
define a special procedure for generating new solutions. This can be
achieved by overriding the _fill(...) method of the Problem class.
Please see the example below.
class MyProblemWithCustomizedFilling(Problem):
def __init__(self):
super().__init__(
objective_sense="min",
solution_length=10,
dtype=torch.float32,
# we do not set initial_bounds because we have a manual procedure
# for initializing solutions
)
def _evaluate_batch(
self, solutions: SolutionBatch
): ... # code to compute and fill the fitnesses goes here
def _fill(self, values: torch.Tensor):
# `values` is an empty tensor of shape (n, m) where n is the number
# of solutions and m is the solution length.
# The responsibility of this method is to fill this tensor.
# In the case of this example, let us say that we wish the new
# solutions to have values sampled from a standard normal distribution.
values.normal_()
Defining manually-structured optimization problems.
The dtype of an optimization problem can be set as object.
When the dtype is set as an object, it means that a solution's
value can be a PyTorch tensor, or a numpy array, or a Python list,
or a Python dictionary, or a string, or a scalar, or None.
This gives the user enough flexibility to express non-numeric
optimization problems and/or problems where each solution has its
own length, or even its own structure.
In the example below, we define an optimization problem where a solution is represented by a Python list and where each solution can have its own length. For simplicity, we define the fitness function as the sum of the values of a solution.
from evotorch import Problem, SolutionBatch
from evotorch.tools import ObjectArray
import random
import torch
class MyCustomStructuredProblem(Problem):
def __init__(self):
super().__init__(
objective_sense="min",
dtype=object,
)
def _evaluate_batch(self, solutions: SolutionBatch):
# Get the number of solutions
n = len(solutions)
# Allocate a PyTorch tensor that will store the fitnesses
fitnesses = torch.empty(n, dtype=torch.float32)
# Fitness is computed as the sum of numeric values stored
# by a solution.
for i in range(n):
# Get the values stored by a solution (which, in the case of
# this example, is a Python list, because we initialize them
# so in the _fill method).
sln_values = solutions[i].values
fitnesses[i] = sum(sln_values)
# Set the fitnesses
solutions.set_evals(fitnesses)
def _fill(self, values: ObjectArray):
# At this point, we have an ObjectArray of length `n`.
# This means, we need to fill the values of `n` solutions.
# `values[i]` represents the values of the i-th solution.
# Initially, `values[i]` is None.
# It is up to us how `values[i]` will be filled.
# Let us make each solution be initialized as a list of
# random length, containing random real numbers.
for i in range(len(values)):
ith_solution_length = random.randint(1, 10)
ith_solution_values = [random.random() for _ in range(ith_solution_length)]
values[i] = ith_solution_values
Multi-objective optimization. A multi-objective optimization problem can be expressed by using multiple objective senses. As an example, let us consider an optimization problem where the first objective sense is minimization and the second objective sense is maximization. When working with an external fitness function, the code to express such an optimization problem would look like this:
from evotorch import Problem
from evotorch.decorators import vectorized
import torch
@vectorized
def f(x: torch.Tensor) -> torch.Tensor:
# (Note that if dtype is object, x will be of type ObjectArray,
# and not a PyTorch tensor)
# Code to compute the fitnesses goes here.
# Our resulting tensor `fitnesses` is expected to have a shape (n, m)
# where n is the number of solutions and m is the number of objectives
# (which is 2 in the case of this example).
# `fitnesses[i, k]` is expected to store the fitness value belonging
# to the i-th solution according to the k-th objective.
fitnesses: torch.Tensor = ...
return fitnesses
problem = Problem(["min", "max"], f, ...)
A multi-objective problem defined via class inheritance would look like this:
from evotorch import Problem, SolutionBatch
class MyMultiObjectiveProblem(Problem):
def __init__(self):
super().__init__(objective_sense=["min", "max"], ...)
def _evaluate_batch(self, solutions: SolutionBatch):
# Code to compute the fitnesses goes here.
# `fitnesses[i, k]` is expected to store the fitness value belonging
# to the i-th solution according to the k-th objective.
fitnesses: torch.Tensor = ...
# Set the fitnesses
solutions.set_evals(fitnesses)
How to solve a problem.
If the optimization problem is single-objective and its dtype is a float
(e.g. torch.float32, torch.float64, etc.), then it can be solved using
any search algorithm implemented in EvoTorch. Let us assume that we have
such an optimization problem stored by the variable prob. We could use
the cross entropy method)
to solve it:
from evotorch import Problem
from evotorch.algorithms import CEM
from evotorch.logging import StdOutLogger
def f(x: torch.Tensor) -> torch.Tensor: ...
prob = Problem("min", f, solution_length=..., dtype=torch.float32)
searcher = CEM(
problem,
# The keyword arguments below refer to hyperparameters specific to the
# cross entropy method algorithm. It is recommended to tune these
# hyperparameters according to the problem at hand.
popsize=100, # population size
parenthood_ratio=0.5, # 0.5 means better half of solutions become parents
stdev_init=10.0, # initial standard deviation of the search distribution
)
_ = StdOutLogger(searcher) # to report the progress onto the screen
searcher.run(50) # run for 50 generations
print("Center of the search distribution:", searcher.status["center"])
print("Solution with best fitness ever:", searcher.status["best"])
See the namespace evotorch.algorithms to see the algorithms implemented within EvoTorch.
If the optimization problem at hand has an integer dtype (e.g. torch.int64),
or has the object dtype, or has multiple objectives, then distribution-based
search algorithms such as CEM cannot be used (since those algorithms were
implemented with continuous decision variables and with single-objective
problems in mind). In such cases, one can use the algorithm named
SteadyState.
Please also note that, while using
SteadyStateGA on a problem with an
integer dtype or with the object dtype, one will have to define manual
cross-over and mutation operators specialized to the solution structure
of the problem at hand. Please see the documentation of
SteadyStateGA for details.
Source code in evotorch/core.py
class Problem(TensorMakerMixin, Serializable):
"""
Representation of a problem to be optimized.
The simplest way to use this class is to instantiate it with an
external fitness function.
Let us imagine that we have the following fitness function:
```
import torch
def f(solution: torch.Tensor) -> torch.Tensor:
return torch.linalg.norm(solution)
```
A problem definition can be made around this fitness function as follows:
```
from evotorch import Problem
problem = Problem(
"min", f, # Goal is to minimize f (would be "max" for maximization)
solution_length=10, # Length of a solution is 10
initial_bounds=(-5.0, 5.0), # Bounds for sampling a new solution
dtype=torch.float32, # dtype of a solution
)
```
**Vectorized problem definitions.**
To boost the runtime performance, one might want to define a vectorized
fitness function where the fitnesses of multiple solutions are computed
in a batched manner using the vectorization capabilities of PyTorch.
A vectorized problem definition can be made as follows:
```
from evotorch.decorators import vectorized
@vectorized
def vf(solutions: torch.Tensor) -> torch.Tensor:
return torch.linalg.norm(solutions ** 2, dim=-1)
problem = Problem(
"min", vf, # Goal is to minimize vf (would be "max" for maximization)
solution_length=10, # Length of a solution is 10
initial_bounds=(-5.0, 5.0), # Bounds for sampling a new solution
dtype=torch.float32, # dtype of a solution
)
```
**Parallelization across multiple CPUs.**
An optimization problem can be configured to parallelize its evaluation
operations across multiple CPUs as follows:
```python
problem = Problem("min", f, ..., num_actors=4) # will use 4 actors
```
**Exploiting hardware accelerators.**
As an alternative to CPU-based parallelization, one might prefer to use
the parallelized computation capabilities of a hardware accelerator such
as CUDA. To load the problem onto a cuda device (for example, onto
"cuda:0"), one can do:
```python
from evotorch.decorators import vectorized
@vectorized
def vf(solutions: torch.Tensor) -> torch.Tensor:
return ...
problem = Problem("min", vf, ..., device="cuda:0")
```
**Exploiting multiple GPUs in parallel.**
One can also keep the entire population on the CPU, and split and distribute
it to multiple GPUs for GPU-accelerated and parallelized fitness evaluation.
For this, the main device of the problem is set as CPU, but the fitness
function is decorated with `evotorch.decorators.on_cuda`.
```python
from evotorch.decorators import on_cuda, vectorized
@on_cuda
@vectorized
def vf(solutions: torch.Tensor) -> torch.Tensor:
return ...
problem = Problem(
"min",
vf,
...,
num_actors=N, # where N>1 and equal to the number of GPUs
# Note: if you are on a computer or on a ray cluster with multiple
# GPUs, you might prefer to use the string "num_gpus" instead of an
# integer N, which will cause the number of available GPUs to be
# counted, and the number of actors to be configured as that count.
#
num_gpus_per_actor=1, # each GPU is assigned to an actor
device="cpu",
)
```
**Defining problems via inheritance.**
A problem can also be defined via inheritance.
Using inheritance, one can define problems which carry their own additional
data, and/or update their states as more solutions are evaluated,
and/or have custom procedures for sampling new solutions, etc.
As a first example, let us define a parameterized problem. In this example
problem, the goal is to minimize `x^(2q)`, `q` being a parameter of the
problem. The definition of such a problem can be as follows:
```python
from evotorch import Problem, Solution
class MyProblem(Problem):
def __init__(self, q: float):
self.q = float(q)
super().__init__(
objective_sense="min", # the goal is to minimize
solution_length=10, # a solution has the length 10
initial_bounds=(-5.0, 5.0), # sample new solutions from within [-5, 5]
dtype=torch.float32, # the dtype of a solution is float32
# num_actors=..., # if parallelization via multiple actors is desired
)
def _evaluate(self, solution: Solution):
# This is where we declare the procedure of evaluating a solution
# Get the decision values of the solution as a PyTorch tensor
x = solution.values
# Compute the fitness
fitness = torch.sum(x ** (2 * self.q))
# Register the fitness into the Solution object
solution.set_evaluation(fitness)
```
This parameterized problem can be instantiated as follows (let's say with q=3):
```python
problem = MyProblem(q=3)
```
**Defining vectorized problems via inheritance.**
Vectorization can be used with inheritance-based problem definitions as well.
Please see the following example where the method `_evaluate_batch`
is used instead of `_evaluate` for vectorization:
```python
from evotorch import Problem, SolutionBatch
class MyVectorizedProblem(Problem):
def __init__(self, q: float):
self.q = float(q)
super().__init__(
objective_sense="min", # the goal is to minimize
solution_length=10, # a solution has the length 10
initial_bounds=(-5.0, 5.0), # sample new solutions from within [-5, 5]
dtype=torch.float32, # the dtype of a solution is float32
# num_actors=..., # if parallelization via multiple actors is desired
# device="cuda:0", # if hardware acceleration is desired
)
def _evaluate_batch(self, solutions: SolutionBatch):
# Get the decision values of all the solutions in a 2D PyTorch tensor:
xs = solutions.values
# Compute the fitnesses
fitnesses = torch.sum(x ** (2 * self.q), dim=-1)
# Register the fitness into the Solution object
solutions.set_evals(fitnesses)
```
**Using multiple GPUs from a problem defined via inheritance.**
The previous example demonstrating the use of multiple GPUs showed how
an independent fitness function can be decorated via
`evotorch.decorators.on_cuda`. Instead of using an independent fitness
function, if one wishes to define a problem by subclassing `Problem`,
the overriden method `_evaluate_batch(...)` has to be decorated by
`evotorch.decorators.on_cuda`. Like in the previous multi-GPU example,
let us assume that we want to parallelize the fitness evaluation
across N GPUs (where N>1). The inheritance-based code to achieve this
can look like this:
```python
from evotorch import Problem, SolutionBatch
from evotorch.decorators import on_cuda
class MyMultiGPUProblem(Problem):
def __init__(self):
...
super().__init__(
objective_sense="min", # the goal is to minimize
solution_length=10, # a solution has the length 10
initial_bounds=(-5.0, 5.0), # sample new solutions from within [-5, 5]
dtype=torch.float32, # the dtype of a solution is float32
num_actors=N, # allocate N actors
# Note: if you are on a computer or on a ray cluster with multiple
# GPUs, you might prefer to use the string "num_gpus" instead of an
# integer N, which will cause the number of available GPUs to be
# counted, and the number of actors to be configured as that count.
#
num_gpus_per_actor=1, # for each actor, assign a cuda device
device="cpu", # keep the main population on the CPU
)
@on_cuda
def _evaluate_batch(self, solutions: SolutionBatch):
# Get the decision values of all the solutions in a 2D PyTorch tensor:
xs = solutions.values
# Compute the fitnesses
fitnesses = ...
# Register the fitness into the Solution object
solutions.set_evals(fitnesses)
```
**Customizing how initial solutions are sampled.**
Instead of sampling solutions from within an interval, one might wish to
define a special procedure for generating new solutions. This can be
achieved by overriding the `_fill(...)` method of the Problem class.
Please see the example below.
```python
class MyProblemWithCustomizedFilling(Problem):
def __init__(self):
super().__init__(
objective_sense="min",
solution_length=10,
dtype=torch.float32,
# we do not set initial_bounds because we have a manual procedure
# for initializing solutions
)
def _evaluate_batch(
self, solutions: SolutionBatch
): ... # code to compute and fill the fitnesses goes here
def _fill(self, values: torch.Tensor):
# `values` is an empty tensor of shape (n, m) where n is the number
# of solutions and m is the solution length.
# The responsibility of this method is to fill this tensor.
# In the case of this example, let us say that we wish the new
# solutions to have values sampled from a standard normal distribution.
values.normal_()
```
**Defining manually-structured optimization problems.**
The `dtype` of an optimization problem can be set as `object`.
When the `dtype` is set as an `object`, it means that a solution's
value can be a PyTorch tensor, or a numpy array, or a Python list,
or a Python dictionary, or a string, or a scalar, or `None`.
This gives the user enough flexibility to express non-numeric
optimization problems and/or problems where each solution has its
own length, or even its own structure.
In the example below, we define an optimization problem where a
solution is represented by a Python list and where each solution can
have its own length. For simplicity, we define the fitness function
as the sum of the values of a solution.
```python
from evotorch import Problem, SolutionBatch
from evotorch.tools import ObjectArray
import random
import torch
class MyCustomStructuredProblem(Problem):
def __init__(self):
super().__init__(
objective_sense="min",
dtype=object,
)
def _evaluate_batch(self, solutions: SolutionBatch):
# Get the number of solutions
n = len(solutions)
# Allocate a PyTorch tensor that will store the fitnesses
fitnesses = torch.empty(n, dtype=torch.float32)
# Fitness is computed as the sum of numeric values stored
# by a solution.
for i in range(n):
# Get the values stored by a solution (which, in the case of
# this example, is a Python list, because we initialize them
# so in the _fill method).
sln_values = solutions[i].values
fitnesses[i] = sum(sln_values)
# Set the fitnesses
solutions.set_evals(fitnesses)
def _fill(self, values: ObjectArray):
# At this point, we have an ObjectArray of length `n`.
# This means, we need to fill the values of `n` solutions.
# `values[i]` represents the values of the i-th solution.
# Initially, `values[i]` is None.
# It is up to us how `values[i]` will be filled.
# Let us make each solution be initialized as a list of
# random length, containing random real numbers.
for i in range(len(values)):
ith_solution_length = random.randint(1, 10)
ith_solution_values = [random.random() for _ in range(ith_solution_length)]
values[i] = ith_solution_values
```
**Multi-objective optimization.**
A multi-objective optimization problem can be expressed by using multiple
objective senses. As an example, let us consider an optimization problem
where the first objective sense is minimization and the second objective
sense is maximization. When working with an external fitness function,
the code to express such an optimization problem would look like this:
```python
from evotorch import Problem
from evotorch.decorators import vectorized
import torch
@vectorized
def f(x: torch.Tensor) -> torch.Tensor:
# (Note that if dtype is object, x will be of type ObjectArray,
# and not a PyTorch tensor)
# Code to compute the fitnesses goes here.
# Our resulting tensor `fitnesses` is expected to have a shape (n, m)
# where n is the number of solutions and m is the number of objectives
# (which is 2 in the case of this example).
# `fitnesses[i, k]` is expected to store the fitness value belonging
# to the i-th solution according to the k-th objective.
fitnesses: torch.Tensor = ...
return fitnesses
problem = Problem(["min", "max"], f, ...)
```
A multi-objective problem defined via class inheritance would look like this:
```python
from evotorch import Problem, SolutionBatch
class MyMultiObjectiveProblem(Problem):
def __init__(self):
super().__init__(objective_sense=["min", "max"], ...)
def _evaluate_batch(self, solutions: SolutionBatch):
# Code to compute the fitnesses goes here.
# `fitnesses[i, k]` is expected to store the fitness value belonging
# to the i-th solution according to the k-th objective.
fitnesses: torch.Tensor = ...
# Set the fitnesses
solutions.set_evals(fitnesses)
```
**How to solve a problem.**
If the optimization problem is single-objective and its dtype is a float
(e.g. torch.float32, torch.float64, etc.), then it can be solved using
any search algorithm implemented in EvoTorch. Let us assume that we have
such an optimization problem stored by the variable `prob`. We could use
the [cross entropy method][evotorch.algorithms.distributed.gaussian.CEM])
to solve it:
```python
from evotorch import Problem
from evotorch.algorithms import CEM
from evotorch.logging import StdOutLogger
def f(x: torch.Tensor) -> torch.Tensor: ...
prob = Problem("min", f, solution_length=..., dtype=torch.float32)
searcher = CEM(
problem,
# The keyword arguments below refer to hyperparameters specific to the
# cross entropy method algorithm. It is recommended to tune these
# hyperparameters according to the problem at hand.
popsize=100, # population size
parenthood_ratio=0.5, # 0.5 means better half of solutions become parents
stdev_init=10.0, # initial standard deviation of the search distribution
)
_ = StdOutLogger(searcher) # to report the progress onto the screen
searcher.run(50) # run for 50 generations
print("Center of the search distribution:", searcher.status["center"])
print("Solution with best fitness ever:", searcher.status["best"])
```
See the namespace [evotorch.algorithms][evotorch.algorithms] to see the
algorithms implemented within EvoTorch.
If the optimization problem at hand has an integer dtype (e.g. torch.int64),
or has the `object` dtype, or has multiple objectives, then distribution-based
search algorithms such as CEM cannot be used (since those algorithms were
implemented with continuous decision variables and with single-objective
problems in mind). In such cases, one can use the algorithm named
[SteadyState][evotorch.algorithms.ga.SteadyStateGA].
Please also note that, while using
[SteadyStateGA][evotorch.algorithms.ga.SteadyStateGA] on a problem with an
integer dtype or with the `object` dtype, one will have to define manual
cross-over and mutation operators specialized to the solution structure
of the problem at hand. Please see the documentation of
[SteadyStateGA][evotorch.algorithms.ga.SteadyStateGA] for details.
"""
def __init__(
self,
objective_sense: ObjectiveSense,
objective_func: Optional[Callable] = None,
*,
initial_bounds: Optional[BoundsPairLike] = None,
bounds: Optional[BoundsPairLike] = None,
solution_length: Optional[int] = None,
dtype: Optional[DType] = None,
eval_dtype: Optional[DType] = None,
device: Optional[Device] = None,
eval_data_length: Optional[int] = None,
seed: Optional[int] = None,
num_actors: Optional[Union[int, str]] = None,
actor_config: Optional[dict] = None,
num_gpus_per_actor: Optional[Union[int, float, str]] = None,
num_subbatches: Optional[int] = None,
subbatch_size: Optional[int] = None,
store_solution_stats: Optional[bool] = None,
vectorized: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the Problem object.
Args:
objective_sense: A string, or a sequence of strings.
For a single-objective problem, a single string
("min" or "max", for minimization or maximization)
is enough.
For a problem with `n` objectives, a sequence
of strings (e.g. a list of strings) of length `n` is
required, each string in the sequence being "min" or
"max". This argument specifies the goal of the
optimization.
initial_bounds: In which interval will the values of a
new solution will be initialized.
Expected as a tuple, each element being either a
scalar, or a vector of length `n`, `n` being the
length of a solution.
If a manual solution initialization is preferred
(instead of an interval-based initialization),
one can leave `initial_bounds` as None, and override
the `_fill(...)` method in the inheriting subclass.
bounds: Interval in which all the solutions must always
reside.
Expected as a tuple, each element being either a
scalar, or a vector of length `n`, `n` being the
length of a solution.
This argument is optional, and can be left as None
if one does not wish to declare hard bounds on the
decision values of the problem.
If `bounds` is specified, `initial_bounds` is missing,
and `_fill(...)` is not overriden, then `bounds` will
also serve as the `initial_bounds`.
solution_length: Length of a solution.
Required for all fixed-length numeric optimization
problems.
For variable-length problems (which might or might not
be numeric), one is expected to leave `solution_length`
as None, and declare `dtype` as `object`.
dtype: dtype (data type) of the data stored by a solution.
Can be given as a string (e.g. "float32"),
or as a numpy dtype (e.g. `numpy.dtype("float32")`),
or as a PyTorch dtype (e.g. `torch.float32`).
Alternatively, if the problem is variable-length
and/or non-numeric, one is expected to declare `dtype`
as `object`.
eval_dtype: dtype to be used for storing the evaluations
(or fitnesses, or scores, or costs, or losses)
of the solutions.
Can be given as a string (e.g. "float32"),
or as a numpy dtype (e.g. `numpy.dtype("float32")`),
or as a PyTorch dtype (e.g. `torch.float32`).
`eval_dtype` must always refer to a "float" data type,
therefore, `object` is not accepted as a valid `eval_dtype`.
If `eval_dtype` is not specified (i.e. left as None),
then the following actions are taken to determine the
`eval_dtype`:
if `dtype` is "float16", `eval_dtype` becomes "float16";
if `dtype` is "bfloat16", `eval_dtype` becomes "bfloat16";
if `dtype` is "float32", `eval_dtype` becomes "float32";
if `dtype` is "float64", `eval_dtype` becomes "float64";
and for any other `dtype`, `eval_dtype` becomes "float32".
device: Default device in which a new population will be
generated. For non-numeric problems, this must be "cpu".
For numeric problems, this can be any device supported
by PyTorch (e.g. "cuda").
Note that, if the number of actors of the problem is configured
to be more than 1, `device` has to be "cpu" (or, equivalently,
left as None).
eval_data_length: In addition to evaluation results
(which are (un)fitnesses, or scores, or costs, or losses),
each solution can store extra evaluation data.
If storage of such extra evaluation data is required,
one can set this argument to an integer bigger than 0.
seed: Random seed to be used by the random number generator
attached to the problem object.
If left as None, no random number generator will be
attached, and the global random number generator of
PyTorch will be used instead.
num_actors: Number of actors to create for parallelized
evaluation of the solutions.
Certain string values are also accepted.
When given as "max" or as "num_cpus", the number of actors
will be equal to the number of all available CPUs in the ray
cluster.
When given as "num_gpus", the number of actors will be
equal to the number of all available GPUs in the ray
cluster, and each actor will be assigned a GPU.
There is also an option, "num_devices", which means that
both the numbers of CPUs and GPUs will be analyzed, and
new actors and GPUs for them will be allocated,
in a one-to-one mapping manner, if possible.
In more details, with `num_actors="num_devices"`, if
`device` is given as a GPU device, then it will be inferred
that the user wishes to put everything (including the
population) on a single GPU, and therefore there won't be
any allocation of actors nor GPUs.
With `num_actors="num_devices"` and with `device` set as
"cpu" (or as left as None), if there are multiple CPUs
and multiple GPUs, then `n` actors will be allocated
where `n` is the minimum among the number of CPUs
and the number of GPUs, so that there can be one-to-one
mapping between CPUs and GPUs (i.e. such that each actor
can be assigned an entire GPU).
If `num_actors` is given as "num_gpus" or "num_devices",
the argument `num_gpus_per_actor` must not be used,
and the `actor_config` dictionary must not contain the
key "num_gpus".
If `num_actors` is given as something other than "num_gpus"
or "num_devices", and if you wish to assign GPUs to each
actor, then please see the argument `num_gpus_per_actor`.
actor_config: A dictionary, representing the keyword arguments
to be passed to the options(...) used when creating the
ray actor objects. To be used for explicitly allocating
resources per each actor.
For example, for declaring that each actor is to use a GPU,
one can pass `actor_config=dict(num_gpus=1)`.
Can also be given as None (which is the default),
if no such options are to be passed.
num_gpus_per_actor: Number of GPUs to be allocated by each
remote actor.
The default behavior is to NOT allocate any GPU at all
(which is the default behavior of the ray library as well).
When given as a number `n`, each actor will be given
`n` GPUs (where `n` can be an integer, or can be a `float`
for fractional allocation).
When given as a string "max", then the available GPUs
across the entire ray cluster (or within the local computer
in the simplest cases) will be equally distributed among
the actors.
When given as a string "all", then each actor will have
access to all the GPUs (this will be achieved by suppressing
the environment variable `CUDA_VISIBLE_DEVICES` for each
actor).
When the problem is not distributed (i.e. when there are
no actors), this argument is expected to be left as None.
num_subbatches: If `num_subbatches` is None (assuming that
`subbatch_size` is also None), then, when evaluating a
population, the population will be split into n pieces, `n`
being the number of actors, and each actor will evaluate
its assigned piece. If `num_subbatches` is an integer `m`,
then the population will be split into `m` pieces,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
While using a distributed algorithm, this argument determines
how many sub-batches will be generated, and therefore,
how many gradients will be computed by the remote actors.
subbatch_size: If `subbatch_size` is None (assuming that
`num_subbatches` is also None), then, when evaluating a
population, the population will be split into `n` pieces, `n`
being the number of actors, and each actor will evaluate its
assigned piece. If `subbatch_size` is an integer `m`,
then the population will be split into pieces of size `m`,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
When there can be significant difference across the solutions
in terms of computational requirements, specifying a
`subbatch_size` can be beneficial, because, while one
actor is busy with a subbatch containing computationally
challenging solutions, other actors can accept more
tasks and save time.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
While using a distributed algorithm, this argument determines
the size of a sub-batch (or sub-population) sampled by a
remote actor for computing a gradient.
In distributed mode, it is expected that the population size
is divisible by `subbatch_size`.
store_solution_stats: Whether or not the problem object should
keep track of the best and worst solutions.
Can also be left as None (which is the default behavior),
in which case, it will store the best and worst solutions
only when the first solution batch it encounters is on the
cpu. This default behavior is to ensure that there is no
transfer between the cpu and a foreign computation device
(like the gpu) just for the sake of keeping the best and
the worst solutions.
vectorized: Set this to True if the provided fitness function
is vectorized but is not decorated via `@vectorized`.
"""
# Set the dtype for the decision variables of the Problem
if dtype is None:
self._dtype = torch.float32
elif is_dtype_object(dtype):
self._dtype = object
else:
self._dtype = to_torch_dtype(dtype)
_evolog.info(message_from(self, f"The `dtype` for the problem's decision variables is set as {self._dtype}"))
# Set the dtype for the solution evaluations (i.e. fitnesses and evaluation data)
if eval_dtype is not None:
# If an `eval_dtype` is explicitly stated, then accept it as the `_eval_dtype` of the Problem
self._eval_dtype = to_torch_dtype(eval_dtype)
else:
# This is the case where an `eval_dtype` is not explicitly stated by the user.
# We need to choose a default.
if self._dtype in (torch.float16, torch.bfloat16, torch.float64):
# If the `dtype` of the problem is a non-32-bit float type (i.e. float16, bfloat16, float64)
# then we use that as our `_eval_dtype` as well.
self._eval_dtype = self._dtype
else:
# For any other `dtype`, we use float32 as our `_eval_dtype`.
self._eval_dtype = torch.float32
_evolog.info(
message_from(
self, f"`eval_dtype` (the dtype of the fitnesses and evaluation data) is set as {self._eval_dtype}"
)
)
# Set the main device of the Problem object
self._device = torch.device("cpu") if device is None else torch.device(device)
_evolog.info(message_from(self, f"The `device` of the problem is set as {self._device}"))
# Declare the internal variable that might store the random number generator
self._generator: Optional[torch.Generator] = None
# Set the seed of the Problem object, if a seed is provided
self.manual_seed(seed)
# Declare the internal variables that will store the bounds and the solution length
self._initial_lower_bounds: Optional[torch.Tensor] = None
self._initial_upper_bounds: Optional[torch.Tensor] = None
self._lower_bounds: Optional[torch.Tensor] = None
self._upper_bounds: Optional[torch.Tensor] = None
self._solution_length: Optional[int] = None
if self._dtype is object:
# If dtype is given as `object`, then there are some runtime sanity checks to perform
if bounds is not None or initial_bounds is not None:
# With dtype as object, if bounds are given then we raise an error.
# This is because the `object` dtype implies that the decision values are not necessarily numeric,
# and therefore, we cannot have the guarantee of satisfying numeric bounds.
raise ValueError(
f"With dtype as {repr(dtype)}, expected to receive `initial_bounds` and/or `bounds` as None."
f" However, one or both of them is/are set as value(s) other than None."
)
if solution_length is not None:
# With dtype as object, if `solution_length` is provided, then we raise an error.
# This is because the `object` dtype implies that the solutions can be expressed via various
# containers, each with its own length, and therefore, a fixed solution length cannot be guaranteed.
raise ValueError(
f"With dtype as {repr(dtype)}, expected to receive `solution_length` as None."
f" However, received `solution_length` as {repr(solution_length)}."
)
if str(self._device) != "cpu":
# With dtype as object, if `device` is something other than "cpu", then we raise an error.
# This is because the `object` dtype implies that the decision values are stored by an ObjectArray,
# whose device is always "cpu".
raise ValueError(
f"With dtype as {repr(dtype)}, expected to receive `device` as 'cpu'."
f" However, received `device` as {repr(device)}."
)
else:
# If dtype is something other than `object`, then we need to make sure that we have a valid length for
# solutions, and also properly store the given numeric bounds.
if solution_length is None:
# With a numeric dtype, if solution length is missing, then we raise an error.
raise ValueError(
f"Together with a numeric dtype ({repr(dtype)}),"
f" expected to receive `solution_length` as an integer."
f" However, `solution_length` is None."
)
else:
# With a numeric dtype, if a solution length is provided, we make sure that it is integer.
solution_length = int(solution_length)
# Store the solution length
self._solution_length = solution_length
if (bounds is not None) or (initial_bounds is not None):
# This is the case where we have a dtype other than `object`, and either `bounds` or `initial_bounds`
# was provided.
initbnd_tuple_name = "initial_bounds"
bnd_tuple_name = "bounds"
if (bounds is not None) and (initial_bounds is None):
# With a numeric dtype, if strict bounds are given but initial bounds are not given, then we assume
# that the strict bounds also serve as the initial bounds.
# Therefore, we take clones of the strict bounds and use this clones as the initial bounds.
initial_bounds = clone(bounds)
initbnd_tuple_name = "bounds"
# Below is an internal helper function for some common operations for the (strict) bounds
# and for the initial bounds.
def process_bounds(bounds_tuple: BoundsPairLike, tuple_name: str) -> BoundsPair:
# This function receives the bounds_tuple (a tuple containing lower and upper bounds),
# and the string name of the bounds argument ("bounds" or "initial_bounds").
# What is returned is the bounds expressed as PyTorch tensors in the correct dtype and device.
nonlocal solution_length
# Extract the lower and upper bounds from the received bounds tuple.
lb, ub = bounds_tuple
# Make sure that the lower and upper bounds are expressed as tensors of correct dtype and device.
lb = self.make_tensor(lb)
ub = self.make_tensor(ub)
for bound_array in (lb, ub): # For each boundary tensor (lb and ub)
if bound_array.ndim not in (0, 1):
# If the boundary tensor is not as scalar and is not a 1-dimensional vector, then raise an
# error.
raise ValueError(
f"Lower and upper bounds are expected as scalars or as 1-dimensional vectors."
f" However, these given boundaries have incompatible shape:"
f" {bound_array} (of shape {bound_array.shape})."
)
if bound_array.ndim == 1:
if len(bound_array) != solution_length:
# In the case where the boundary tensor is a 1-dimensional vector, if this vector's length
# is not equal to the solution length, then we raise an error.
raise ValueError(
f"When boundaries are expressed as 1-dimensional vectors, their length are"
f" expected as the solution length of the Problem object."
f" However, while the problem's solution length is {solution_length},"
f" these given boundaries have incompatible length:"
f" {bound_array} (of length {len(bound_array)})."
)
# Return the processed forms of the lower and upper boundary tensors.
return lb, ub
# Process the initial bounds with the help of the internal function `process_bounds(...)`
init_lb, init_ub = process_bounds(initial_bounds, initbnd_tuple_name)
# Store the processed initial bounds
self._initial_lower_bounds = init_lb
self._initial_upper_bounds = init_ub
if bounds is not None:
# If there are strict bounds, then process those bounds with the help of `process_bounds(...)`.
lb, ub = process_bounds(bounds, bnd_tuple_name)
# Store the processed bounds
self._lower_bounds = lb
self._upper_bounds = ub
# Annotate the variable that will store the objective sense(s) of the problem
self._objective_sense: ObjectiveSense
# Below is an internal function which makes sure that a provided objective sense has a valid value
# (where valid values are "min" or "max")
def validate_sense(s: str):
if s not in ("min", "max"):
raise ValueError(
f"Invalid objective sense: {repr(s)}."
f"Instead, please provide the objective sense as 'min' or 'max'."
)
if not is_sequence(objective_sense):
# If the provided objective sense is not a sequence, then convert it to a single-element list
senses = [objective_sense]
num_senses = 1
else:
# If the provided objective sense is a sequence, then take a list copy of it
senses = list(objective_sense)
num_senses = len(objective_sense)
# Ensure that each provided objective sense is valid
for sense in senses:
validate_sense(sense)
if num_senses == 0:
# If the given sequence of objective senses is empty, then we raise an error.
raise ValueError(
"Encountered an empty sequence via `objective_sense`."
" For a single-objective problem, please set `objective_sense` as 'min' or 'max'."
" For a multi-objective problem, please set `objective_sense` as a sequence,"
" each element being 'min' or 'max'."
)
# Store the objective senses
self._senses: Iterable[str] = senses
# Store the provided objective function (which can be None)
self._objective_func: Optional[Callable] = objective_func
# Declare the instance variable that will store whether or not the external fitness function is
# vectorized, if such an external fitness function is given.
self._vectorized: Optional[bool]
# Store the information which indicates whether or not the given objective function is vectorized
if self._objective_func is None:
# This is the case where an external fitness function is not given.
# In this case, we expect the keyword argument `vectorized` to be left as None.
if vectorized is not None:
# If the keyword argument `vectorized` is something other than None, then we raise an error
# to let the user know.
raise ValueError(
f"This problem object received no external fitness function."
f" When not using an external fitness function, the keyword argument `vectorized`"
f" is expected to be left as None."
f" However, the value of the keyword argument `vectorized` is {vectorized}."
)
# At this point, we know that we do not have an external fitness function.
# The variable which is supposed to tell us whether or not the external fitness function is vectorized
# is therefore irrelevant. We just set it as None.
self._vectorized = None
else:
# This is the case where an external fitness function is given.
if (
hasattr(self._objective_func, "__evotorch_vectorized__")
and self._objective_func.__evotorch_vectorized__
):
# If the external fitness function has an attribute `__evotorch_vectorized__`, and the value of this
# attribute evaluates to True, then this is an indication that the fitness function was decorated
# with `@vectorized`.
if vectorized is not None:
# At this point, we know (or at least have the assumption) that the fitness function was decorated
# with `@vectorized`. Any boolean value given via the keyword argument `vectorized` would therefore
# be either redundant or conflicting.
# Therefore, in this case, if the keyword argument `vectorized` is anything other than None, we
# raise an error to inform the user.
raise ValueError(
f"Received a fitness function that was decorated via @vectorized."
f" When using such a fitness function, the keyword argument `vectorized`"
f" is expected to be left as None."
f" However, the value of the keyword argument `vectorized` is {vectorized}."
)
# Since we know that our fitness function declares itself as vectorized, we set the instance variable
# _vectorized as True.
self._vectorized = True
else:
# This is the case in which the fitness function does not appear to be decorated via `@vectorized`.
# In this case, if the keyword argument `vectorized` has a value that is equivalent to True,
# then the value of `_vectorized` becomes True. On the other hand, if the keyword argument `vectorized`
# was left as None or if it has a value that is equivalent to False, `_vectorized` becomes False.
self._vectorized = bool(vectorized)
# If the evaluation data length is explicitly stated, then convert it to an integer and store it.
# Otherwise, store the evaluation data length as 0.
self._eval_data_length = 0 if eval_data_length is None else int(eval_data_length)
# Initialize the actor index.
# If the problem is configured to be parallelized and the parallelization is triggered, then each remote
# copy will have a different integer value for `_actor_index`.
self._actor_index: Optional[int] = None
# Initialize the variable that might store the list of actors as None.
# If the problem is configured to be parallelized and the parallelization is triggered, then this variable
# will store references to the remote actors (each remote actor storing its own copy of this Problem
# instance).
self._actors: Optional[list] = None
# Initialize the variable that might store the ray ActorPool.
# If the problem is configured to be parallelized and the parallelization is triggered, then this variable
# will store the ray ActorPool that is generated out of the remote actors.
self._actor_pool: Optional[ActorPool] = None
# Store the ray actor configuration dictionary provided by the user (if any).
# When (or if) the parallelization is triggered, each actor will be created with this given configuration.
self._actor_config: Optional[dict] = None if actor_config is None else deepcopy(dict(actor_config))
# If given, store the sub-batch size or number of sub-batches.
# When the problem is parallelized, a sub-batch size determines the maximum size for a SolutionBatch
# that will be sent to a remote actor for parallel solution evaluation.
# Alternatively, num_subbatches determines into how many pieces will a SolutionBatch be split
# for parallelization.
# If both are None, then the main SolutionBatch will be split among the actors.
if (num_subbatches is not None) and (subbatch_size is not None):
raise ValueError(
f"Encountered both `num_subbatches` and `subbatch_size` as values other than None."
f" num_subbatches={num_subbatches}, subbatch_size={subbatch_size}."
f" Having both of them as values other than None cannot be accepted."
)
self._num_subbatches: Optional[int] = None if num_subbatches is None else int(num_subbatches)
self._subbatch_size: Optional[int] = None if subbatch_size is None else int(subbatch_size)
# Initialize the additional states to be loaded by the remote actor as None.
# If there are such additional states for remote actors, the inheriting class can fill this as a list
# of dictionaries.
self._remote_states: Optional[Iterable[dict]] = None
# Initialize a temporary internal variable which stores the resources available in the ray cluster.
# Most probably, we are interested in the resources "CPU" and "GPU".
ray_resources: Optional[dict] = None
# The following is an internal helper function which returns the amount of availability for a given
# resource in the ray cluster.
# If the requested resource is not available at all, None will be returned.
def get_ray_resource(resource_name: str) -> Any:
# Ensure that the ray cluster is initialized
ensure_ray()
nonlocal ray_resources
if ray_resources is None:
# If the ray resource information was not fetched, then fetch them and store them.
ray_resources = ray.available_resources()
# Return the information regarding the requested resource from the fetched resource information.
# If it turns out that the requested resource is not available at all, the result will be None.
return ray_resources.get(resource_name, None)
# Annotate the variable that will store the number of actors (to be created when the parallelization
# is triggered).
self._num_actors: int
if num_actors is None:
# If the argument `num_actors` is left as None, then we set `_num_actors` as 0, which means that
# there will be no parallelization.
self._num_actors = 0
elif isinstance(num_actors, str):
# This is the case where `num_actors` has a string value
if num_actors in ("max", "num_cpus"):
# If the `num_actors` argument was given as "max" or as "num_cpus", then we first read how many CPUs
# are available in the ray cluster, then convert it to integer (via computing its ceil value), and
# finally set `_num_actors` as this integer.
self._num_actors = math.ceil(get_ray_resource("CPU"))
elif num_actors == "num_gpus":
# If the `num_actors` argument was given as "num_gpus", then we first read how many GPUs are
# available in the ray cluster.
num_gpus = get_ray_resource("GPU")
if num_gpus is None:
# If there are no GPUs at all, then we raise an error
raise ValueError(
"The argument `num_actors` was encountered as 'num_gpus'."
" However, there does not seem to be any GPU available."
)
if num_gpus < 1e-4:
# If the number of available GPUs are 0 or close to 0, then we raise an error
raise ValueError(
f"The argument `num_actors` was encountered as 'num_gpus'."
f" However, the number of available GPUs are either 0 or close to 0 (= {num_gpus})."
)
if (actor_config is not None) and ("num_gpus" in actor_config):
# With `num_actors` argument given as "num_gpus", we will also allocate each GPU to an actor.
# If `actor_config` contains an item with key "num_gpus", then that configuration item would
# conflict with the GPU allocation we are about to do here.
# So, we raise an error.
raise ValueError(
"The argument `num_actors` was encountered as 'num_gpus'."
" With this configuration, the number of GPUs assigned to an actor is automatically determined."
" However, at the same time, the `actor_config` argument was received with the key 'num_gpus',"
" which causes a conflict."
)
if num_gpus_per_actor is not None:
# With `num_actors` argument given as "num_gpus", we will also allocate each GPU to an actor.
# If the argument `num_gpus_per_actor` is also stated, then such a configuration item would
# conflict with the GPU allocation we are about to do here.
# So, we raise an error.
raise ValueError(
f"The argument `num_actors` was encountered as 'num_gpus'."
f" With this configuration, the number of GPUs assigned to an actor is automatically determined."
f" However, at the same time, the `num_gpus_per_actor` argument was received with a value other"
f" than None ({repr(num_gpus_per_actor)}), which causes a conflict."
)
# Set the number of actors as the ceiled integer counterpart of the number of available GPUs
self._num_actors = math.ceil(num_gpus)
# We assign a GPU for each actor (by overriding the value for the argument `num_gpus_per_actor`).
num_gpus_per_actor = num_gpus / self._num_actors
elif num_actors == "num_devices":
# This is the case where `num_actors` has the string value "num_devices".
# With `num_actors` set as "num_devices", if there are any GPUs, the behavior is to assign a GPU
# to each actor. If there are conflicting configurations regarding how many GPUs are to be assigned
# to each actor, then we raise an error.
if (actor_config is not None) and ("num_gpus" in actor_config):
raise ValueError(
"The argument `num_actors` was encountered as 'num_devices'."
" With this configuration, the number of GPUs assigned to an actor is automatically determined."
" However, at the same time, the `actor_config` argument was received with the key 'num_gpus',"
" which causes a conflict."
)
if num_gpus_per_actor is not None:
raise ValueError(
f"The argument `num_actors` was encountered as 'num_devices'."
f" With this configuration, the number of GPUs assigned to an actor is automatically determined."
f" However, at the same time, the `num_gpus_per_actor` argument was received with a value other"
f" than None ({repr(num_gpus_per_actor)}), which causes a conflict."
)
if self._device != torch.device("cpu"):
# If the main device is not CPU, then the user most probably wishes to put all the
# computations (both evaluations and the population) on the GPU, without allocating
# any actor.
# So, we set `_num_actors` as None, and overwrite `num_gpus_per_actor` with None.
self._num_actors = None
num_gpus_per_actor = None
else:
# If the device argument is "cpu" or left as None, then we assume that actor allocations
# might be desired.
# Read how many CPUs and GPUs are available in the ray cluster.
num_cpus = get_ray_resource("CPU")
num_gpus = get_ray_resource("GPU")
# If we have multiple CPUs, then we continue with the actor allocation procedures.
if (num_gpus is None) or (num_gpus < 1e-4):
# If there are no GPUs, then we set the number of actors as the number of CPUs, and we
# set the number of GPUs per actor as None (which means that there will be no GPU
# assignment)
self._num_actors = math.ceil(num_cpus)
num_gpus_per_actor = None
else:
# If there are GPUs available, then we compute the minimum among the number of CPUs and
# GPUs, and this minimum value becomes the number of actors (so that there can be
# one-to-one mapping between actors and GPUs).
self._num_actors = math.ceil(min(num_cpus, num_gpus))
# We assign a GPU for each actor (by overriding the value for the argument
# `num_gpus_per_actor`).
if self._num_actors <= num_gpus:
num_gpus_per_actor = 1
else:
num_gpus_per_actor = num_gpus / self._num_actors
else:
# This is the case where `num_actors` is given as an unexpected string. We raise an error here.
raise ValueError(
f"Invalid string value for `num_actors`: {repr(num_actors)}."
f" The acceptable string values for `num_actors` are 'max', 'num_cpus', 'num_gpus', 'num_devices'."
)
else:
# This is the case where `num_actors` has a value which is not a string.
# In this case, we make sure that the given value is an integer, and then use this integer as our
# number of actors.
self._num_actors = int(num_actors)
if self._num_actors == 1:
_evolog.info(
message_from(
self,
(
"The number of actors that will be allocated for parallelized evaluation was encountered as 1."
" This number is automatically dropped to 0,"
" because having only 1 actor does not bring any benefit in terms of parallelization."
),
)
)
# Creating a single actor does not bring any benefit of parallelization.
# Therefore, at the end of all the computations above regarding the number of actors, if it turns out
# that the target number of actors is 1, we reduce it to 0 (meaning that no actor will be initialized).
self._num_actors = 0
# Since we are to allocate no actor, the value of the argument `num_gpus_per_actor` is meaningless.
# We therefore overwrite the value of that argument with None.
num_gpus_per_actor = None
_evolog.info(
message_from(
self, f"The number of actors that will be allocated for parallelized evaluation is {self._num_actors}"
)
)
if (self._num_actors >= 2) and (self._device != torch.device("cpu")):
detailed_error_msg = (
f"The number of actors that will be allocated for parallelized evaluation is {self._num_actors}."
" When the number of actors is at least 2,"
' the only supported value for the `device` argument is "cpu".'
f" However, `device` was received as {self._device}."
"\n\n---- Possible ways to fix the error: ----"
"\n\n"
"(1)"
" If both the population and the fitness evaluation operations can fit into the same device,"
f" try setting `device={self._device}` and `num_actors=0`."
"\n\n"
"(2)"
" If you would like to use N number of GPUs in parallel for fitness evaluation (where N>1),"
' set `device="cpu"` (so that the main process will keep the population on the cpu), set'
" `num_actors=N` and `num_gpus_per_actor=1` (to allocate an actor for each of the `N` GPUs),"
" and then, decorate your fitness function using `evotorch.decorators.on_cuda`"
" so that the fitness evaluation will be performed on the cuda device assigned to the actor."
" The code for achieving this can look like this:"
"\n\n"
" from evotorch import Problem\n"
" from evotorch.decorators import on_cuda, vectorized\n"
" import torch\n"
"\n"
" @on_cuda\n"
" @vectorized\n"
" def f(x: torch.Tensor) -> torch.Tensor:\n"
" ...\n"
"\n"
' problem = Problem("min", f, device="cpu", num_actors=N, num_gpus_per_actor=1)\n'
"\n"
"Or, it can look like this:\n"
"\n"
" from evotorch import Problem, SolutionBatch\n"
" from evotorch.decorators import on_cuda\n"
" import torch\n"
"\n"
" class MyProblem(Problem):\n"
" def __init__(self, ...):\n"
" super().__init__(\n"
' objective_sense="min", device="cpu", num_actors=N, num_gpus_per_actor=1, ...\n'
" )\n"
"\n"
" @on_cuda\n"
" def _evaluate_batch(self, batch: SolutionBatch):\n"
" ...\n"
"\n"
" problem = MyProblem(...)\n"
"\n"
"\n"
"(3)"
" Similarly to option (2), for when you wish to use N number of GPUs for fitness evaluation,"
' set `device="cpu"`, set `num_actors=N` and `num_gpus_per_actor=1`, then, within the evaluation'
' function, manually use the device `"cuda"` to accelerate the computation.'
"\n\n"
"--------------\n"
"Note for cases (2) and (3): if you are on a computer or on a ray cluster with multiple GPUs, you"
' might prefer to set `num_actors` as the string "num_gpus" instead of an integer N,'
" which will cause the number of available GPUs to be counted, and the number of actors to be"
" configured as that count."
)
raise ValueError(detailed_error_msg)
# Annotate the variable which will determine how many GPUs are to be assigned to each actor.
self._num_gpus_per_actor: Optional[Union[str, int, float]]
if (actor_config is not None) and ("num_gpus" in actor_config) and (num_gpus_per_actor is not None):
# If `actor_config` dictionary has the item "num_gpus" and also `num_gpus_per_actor` is not None,
# then there is a conflicting (or redundant) configuration. We raise an error here.
raise ValueError(
'The `actor_config` dictionary contains the key "num_gpus".'
" At the same time, `num_gpus_per_actor` has a value other than None."
" These two configurations are conflicting."
" Please specify the number of GPUs per actor either via the `actor_config` dictionary,"
" or via the `num_gpus_per_actor` argument, but not via both."
)
if num_gpus_per_actor is None:
# If the argument `num_gpus_per_actor` is not specified, then we set the attribute
# `_num_gpus_per_actor` as None, which means that no GPUs will be assigned to the actors.
self._num_gpus_per_actor = None
elif isinstance(num_gpus_per_actor, str):
# This is the case where `num_gpus_per_actor` is given as a string.
if num_gpus_per_actor == "max":
# This is the case where `num_gpus_per_actor` is given as "max".
num_gpus = get_ray_resource("GPU")
if num_gpus is None:
# With `num_gpus_per_actor` as "max", if there is no GPU available, then we set the attribute
# `_num_gpus_per_actor` as None, which means there will be no GPU assignment to the actors.
self._num_gpus_per_actor = None
else:
# With `num_gpus_per_actor` as "max", if there are GPUs available, then the available GPUs will
# be shared among the actors.
self._num_gpus_per_actor = num_gpus / self._num_actors
elif num_gpus_per_actor == "all":
# When `num_gpus_per_actor` is "all", we also set the attribute `_num_gpus_per_actor` as "all".
# When a remote actor is initialized, the remote actor will see that the Problem instance has its
# `_num_gpus_per_actor` set as "all", and it will remove the environment variable named
# "CUDA_VISIBLE_DEVICES" in its own environment.
# With "CUDA_VISIBLE_DEVICES" removed, an actor will see all the GPUs available in its own
# environment.
self._num_gpus_per_actor = "all"
else:
# This is the case where `num_gpus_per_actor` argument has an unexpected string value.
# We raise an error.
raise ValueError(
f"Invalid string value for `num_gpus_per_actor`: {repr(num_gpus_per_actor)}."
f' Acceptable string values for `num_gpus_per_actor` are: "max", "all".'
)
elif isinstance(num_gpus_per_actor, int):
# When the argument `num_gpus_per_actor` is set as an integer we just set the attribute
# `_num_gpus_per_actor` as this integer.
self._num_gpus_per_actor = num_gpus_per_actor
else:
# For anything else, we assume that `num_gpus_per_actor` is an object that is convertible to float.
# Therefore, we convert it to float and store it in the attribute `_num_gpus_per_actor`.
# Also, remember that, when `num_actors` is given as "num_gpus" or as "num_devices",
# the code above overrides the value for the argument `num_gpus_per_actor`, which means,
# this is the case that is activated when `num_actors` is "num_gpus" or "num_devices".
self._num_gpus_per_actor = float(num_gpus_per_actor)
if self._num_actors > 0:
_evolog.info(
message_from(self, f"Number of GPUs that will be allocated per actor is {self._num_gpus_per_actor}")
)
# Initialize the Hook instances (and the related status dictionary for the `_after_eval_hook`)
self._before_eval_hook: Hook = Hook()
self._after_eval_hook: Hook = Hook()
self._after_eval_status: dict = {}
self._remote_hook: Hook = Hook()
self._before_grad_hook: Hook = Hook()
self._after_grad_hook: Hook = Hook()
# Initialize various stats regarding the solutions encountered by this Problem instance.
self._store_solution_stats = None if store_solution_stats is None else bool(store_solution_stats)
self._best: Optional[list] = None
self._worst: Optional[list] = None
self._best_evals: Optional[torch.Tensor] = None
self._worst_evals: Optional[torch.Tensor] = None
# Initialize the boolean attribute which indicates whether or not this Problem instance (which can be
# the main instance or a remote instance on an actor) is "prepared" via the `_prepare` method.
self._prepared: bool = False
def manual_seed(self, seed: Optional[int] = None):
"""
Provide a manual seed for the Problem object.
If the given seed is None, then the Problem object will remove
its own stored generator, and start using the global generator
of PyTorch instead.
If the given seed is an integer, then the Problem object will
instantiate its own generator with the given seed.
Args:
seed: None for using the global PyTorch generator; an integer
for instantiating a new PyTorch generator with this given
integer seed, specific to this Problem object.
"""
if seed is None:
self._generator = None
else:
if self._generator is None:
self._generator = torch.Generator(device=self.device)
self._generator.manual_seed(seed)
@property
def dtype(self) -> DType:
"""
dtype of the Problem object.
The decision variables of the optimization problem are of this dtype.
"""
return self._dtype
@property
def device(self) -> Device:
"""
device of the Problem object.
New solutions and populations will be generated in this device.
"""
return self._device
@property
def aux_device(self) -> Device:
"""
Auxiliary device to help with the computations, most commonly for
speeding up the solution evaluations.
An auxiliary device is different than the main device of the Problem
object (the main device being expressed by the `device` property).
While the main device of the Problem object determines where the
solutions and the populations are stored (and also using which device
should a SearchAlgorithm instance communicate with the problem),
an auxiliary device is a device that might be used by the Problem
instance itself for its own computations (e.g. computations defined
within the methods `_evaluate(...)` or `_evaluate_batch(...)`).
If the problem's main device is something other than "cpu", that main
device is also seen as the auxiliary device, and therefore returned
by this property.
If the problem's main device is "cpu", then the auxiliary device
is decided as follows. If `num_gpus_per_actor` of the Problem object
was set as "all" and if this instance is a remote instance, then the
auxiliary device is guessed as "cuda:N" where N is the actor index.
In all other cases, the auxiliary device is "cuda" if cuda is
available, and "cpu" otherwise.
"""
cpu_device = torch.device("cpu")
if torch.device(self.device) == cpu_device:
if torch.cuda.is_available():
if isinstance(self._num_gpus_per_actor, str) and (self._num_gpus_per_actor == "all") and self.is_remote:
return torch.device("cuda", self.actor_index)
else:
return torch.device("cuda")
else:
return cpu_device
else:
return self.device
@property
def eval_dtype(self) -> DType:
"""
evaluation dtype of the Problem object.
The evaluation results of the solutions are stored according to this
dtype.
"""
return self._eval_dtype
@property
def generator(self) -> Optional[torch.Generator]:
"""
Random generator used by this Problem object.
Can also be None, which means that the Problem object will use the
global random generator of PyTorch.
"""
return self._generator
@property
def has_own_generator(self) -> bool:
"""
Whether or not the Problem object has its own random generator.
If this is True, then the Problem object will use its own
random generator when creating random values or tensors.
If this is False, then the Problem object will use the global
random generator when creating random values or tensors.
"""
return self.generator is not None
@property
def objective_sense(self) -> ObjectiveSense:
"""
Get the objective sense.
If the problem is single-objective, then a single string is returned.
If the problem is multi-objective, then the objective senses will be
returned in a list.
The returned string in the single-objective case, or each returned
string in the multi-objective case, is "min" or "max".
"""
if len(self.senses) == 1:
return self.senses[0]
else:
return self.senses
@property
def senses(self) -> Iterable[str]:
"""
Get the objective senses.
The return value is a list of strings, each string being
"min" or "max".
"""
return self._senses
@property
def is_single_objective(self) -> bool:
"""Whether or not the problem is single-objective"""
return len(self.senses) == 1
@property
def is_multi_objective(self) -> bool:
"""Whether or not the problem is multi-objective"""
return len(self.senses) > 1
def get_obj_order_descending(self) -> Iterable[bool]:
"""When sorting the solutions from best to worst according to each objective i, is the ordering descending?"""
result = []
for s in self.senses:
if s == "min":
result.append(False)
elif s == "max":
result.append(True)
else:
raise ValueError(f"Invalid sense: {repr(s)}")
return result
@property
def solution_length(self) -> Optional[int]:
"""
Get the solution length.
Problems with `dtype=None` do not have solution lengths.
For such problems, this property returns None.
"""
return self._solution_length
@property
def eval_data_length(self) -> int:
"""
Length of the extra evaluation data vector for each solution.
"""
return self._eval_data_length
@property
def initial_lower_bounds(self) -> Optional[torch.Tensor]:
"""
Initial lower bounds, for when initializing a new solution.
If such a bound was declared during the initialization phase,
the returned value is a torch tensor (in the form of a vector
or in the form of a scalar).
If no such bound was declared, the returned value is None.
"""
return self._initial_lower_bounds
@property
def initial_upper_bounds(self) -> Optional[torch.Tensor]:
"""
Initial upper bounds, for when initializing a new solution.
If such a bound was declared during the initialization phase,
the returned value is a torch tensor (in the form of a vector
or in the form of a scalar).
If no such bound was declared, the returned value is None.
"""
return self._initial_upper_bounds
@property
def lower_bounds(self) -> Optional[torch.Tensor]:
"""
Lower bounds for the allowed values of a solution.
If such a bound was declared during the initialization phase,
the returned value is a torch tensor (in the form of a vector
or in the form of a scalar).
If no such bound was declared, the returned value is None.
"""
return self._lower_bounds
@property
def upper_bounds(self) -> Optional[torch.Tensor]:
"""
Upper bounds for the allowed values of a solution.
If such a bound was declared during the initialization phase,
the returned value is a torch tensor (in the form of a vector
or in the form of a scalar).
If no such bound was declared, the returned value is None.
"""
return self._upper_bounds
def generate_values(self, num_solutions: int) -> Union[torch.Tensor, ObjectArray]:
"""
Generate decision values.
This function returns a tensor containing the decision values
for `n` new solutions, `n` being the integer passed as the `num_rows`
argument.
For numeric problems, this function generates the decision values
which respect `initial_bounds` (or `bounds`, if `initial_bounds`
was not provided).
If this type of initialization is not desired, one can override
this function and define a manual initialization scheme in the
inheriting subclass.
For non-numeric problems, it is expected that the inheriting subclass
will override the method `_fill(...)`.
Args:
num_solutions: For how many solutions will new decision values be
generated.
Returns:
A PyTorch tensor for numeric problems, an ObjectArray for
non-numeric problems.
"""
if self.dtype is object:
result = self.make_empty(num_solutions=num_solutions)
else:
result = torch.empty(tuple(), dtype=self.dtype, device=self.device)
result = result.expand(num_solutions, self.solution_length)
result = result + make_batched_false_for_vmap(result.device)
self._fill(result)
return result
def _fill(self, values: Iterable):
"""
Fill the provided `values` tensor with new decision values.
Inheriting subclasses can override this method to specialize how
new solutions are generated.
For numeric problems, this method already has an implementation
which samples the initial decision values uniformly from the
interval expressed by `initial_bounds` attribute.
For non-numeric problems, overriding this method is mandatory.
Args:
values: The tensor which is to be filled with the new decision
values.
"""
if self.dtype is object:
raise NotImplementedError(
"The dtype of this problem is object, therefore a manual implementation of the"
" method `_fill(...)` needs to be provided by the inheriting class."
)
else:
if (self.initial_lower_bounds is None) or (self.initial_upper_bounds is None):
raise RuntimeError(
"The default implementation of the method `_fill(...)` does not know how to initialize solutions"
" because it appears that this Problem object was not given neither `initial_bounds` nor `bounds`"
" during the moment of initialization."
" Please either instantiate this Problem object with `initial_bounds` and/or `bounds`, or override"
" the method `_fill(...)` to specify how solutions should be initialized."
)
else:
return self.make_uniform(
out=values,
lb=self.initial_lower_bounds,
ub=self.initial_upper_bounds,
)
def generate_batch(
self,
popsize: Optional[int] = None,
*,
empty: bool = False,
center: Optional[RealOrVector] = None,
stdev: Optional[RealOrVector] = None,
symmetric: bool = False,
) -> "SolutionBatch":
"""
Generate a new SolutionBatch.
Args:
popsize: Number of solutions that will be contained in the new
batch.
empty: Set this as True if you would like to receive the solutions
un-initialized.
center: Center point of the Gaussian distribution from which
the decision values will be sampled, as a scalar or as a
1-dimensional vector.
Can also be left as None.
If `center` is None and `stdev` is None, all the decision
values will be sampled from the interval specified by
`initial_bounds` (or by `bounds` if `initial_bounds` was not
specified).
If `center` is None and `stdev` is not None, a center point
will be sampled from within the interval specified by
`initial_bounds` or `bounds`, and the decision values will be
sampled from a Gaussian distribution around this center point.
stdev: Can be None (default) if the SolutionBatch is to contain
decision values sampled from the interval specified by
`initial_bounds` (or by `bounds` if `initial_bounds` was not
provided during the initialization phase).
Alternatively, a scalar or a 1-dimensional vector specifying
the standard deviation of the Gaussian distribution from which
the decision values will be sampled.
symmetric: To be used only when `stdev` is not None.
If `symmetric` is True, decision values will be sampled from
the Gaussian distribution in a symmetric (i.e. antithetic)
manner.
Otherwise, the decision values will be sampled in the
non-antithetic manner.
"""
if (center is None) and (stdev is None):
if symmetric:
raise ValueError(
f"The argument `symmetric` can be set as True only when `center` and `stdev` are provided."
f" Although `center` and `stdev` are None, `symmetric` was received as {symmetric}."
)
return SolutionBatch(self, popsize, empty=empty, device=self.device)
elif (center is not None) and (stdev is not None):
if empty:
raise ValueError(
f"When `center` and `stdev` are provided, the argument `empty` must be False."
f" However, the received value for `empty` is {empty}."
)
result = SolutionBatch(self, popsize, device=self.device, empty=True)
self.make_gaussian(out=result.access_values(), center=center, stdev=stdev, symmetric=symmetric)
return result
else:
raise ValueError(
f"The arguments `center` and `stdev` were expected to be None or non-None at the same time."
f" Received `center`: {center}."
f" Received `stdev`: {stdev}."
)
def _parallelize(self):
"""Create ray actors for parallelizing the solution evaluations."""
# If the problem was explicitly configured for
# NOT having parallelization, leave this function.
if (not isinstance(self._num_actors, str)) and (self._num_actors <= 0):
return
# If this problem object is a remote one,
# leave this function
# (because we do not want the remote worker
# to parallelize itself further)
if self._actor_index is not None:
return
# If the actors list is not None, then this means
# that the initialization of the parallelization mechanism
# was already completed. So, leave this function.
if self._actors is not None:
return
# Make sure that ray is initialized
ensure_ray()
number_of_actors = self._num_actors
# numpy's RandomState uses 32-bit unsigned integers
# for random seeds.
# So, the following value is the exclusive upper bound
# for a random seed.
supremum_seed = 2**32
# Generate an integer from the main problem object's
# random_state. From this integer, further seed integers
# will be computed, and these generated seeds will be
# used by the remote actors.
base_seed = int(self.make_randint(tuple(), n=supremum_seed))
# The following function returns a seed number for the actor
# number i.
def generate_actor_seed(i):
nonlocal base_seed, supremum_seed
return (base_seed + (i + 1)) % supremum_seed
all_seeds = []
j = 0
for i in range(number_of_actors):
actor_seeds = []
for _ in range(4):
actor_seeds.append(generate_actor_seed(j))
j += 1
all_seeds.append(tuple(actor_seeds))
if self._remote_states is None:
remote_states = [{} for _ in range(number_of_actors)]
else:
remote_states = self._remote_states
# Prepare the necessary actor config
config_per_actor = {}
if self._actor_config is not None:
config_per_actor.update(self._actor_config)
if isinstance(self._num_gpus_per_actor, (int, float)):
config_per_actor["num_gpus"] = self._num_gpus_per_actor
# Generate the actors, each with a unique seed.
if config_per_actor is None:
actors = [EvaluationActor.remote(self, i, all_seeds[i], remote_states[i]) for i in range(number_of_actors)]
else:
actors = [
EvaluationActor.options(**config_per_actor).remote(self, i, all_seeds[i], remote_states[i])
for i in range(number_of_actors)
]
self._actors = actors
self._actor_pool = ActorPool(self._actors)
self._remote_states = None
def all_remote_problems(self) -> AllRemoteProblems:
"""
Get an accessor which is used for running a method
on all remote clones of this Problem object.
For example, given a Problem object named `my_problem`,
also assuming that this Problem object is parallelized,
and therefore has `n` remote actors, a method `f()`
can be executed on all the remote instances as follows:
results = my_problem.all_remote_problems().f()
The variable `results` is a list of length `n`, the i-th
item of the list belonging to the method f's result
from the i-th actor.
Returns:
A method accessor for all the remote Problem objects.
"""
self._parallelize()
if self.is_remote:
raise RuntimeError(
"The method `all_remote_problems()` can only be used on the main (i.e. non-remote)"
" Problem instance."
" However, this Problem instance is on a remote actor."
)
return AllRemoteProblems(self._actors)
def all_remote_envs(self) -> AllRemoteEnvs:
"""
Get an accessor which is used for running a method
on all remote reinforcement learning environments.
This method can only be used on parallelized Problem
objects which have their `get_env()` methods defined.
For example, one can use this feature on a parallelized
GymProblem.
As an example, let us consider a parallelized GymProblem
object named `my_problem`. Given that `my_problem` has
`n` remote actors, a method `f()` can be executed
on all remote reinforcement learning environments as
follows:
results = my_problem.all_remote_envs().f()
The variable `results` is a list of length `n`, the i-th
item of the list belonging to the method f's result
from the i-th actor.
Returns:
A method accessor for all the remote reinforcement
learning environments.
"""
self._parallelize()
if self.is_remote:
raise RuntimeError(
"The method `all_remote_envs()` can only be used on the main (i.e. non-remote)"
" Problem instance."
" However, this Problem instance is on a remote actor."
)
return AllRemoteEnvs(self._actors)
def kill_actors(self):
"""
Kill all the remote actors used by the Problem instance.
One might use this method to release the resources used by the
remote actors.
"""
if not self.is_main:
raise RuntimeError(
"The method `kill_actors()` can only be used on the main (i.e. non-remote)"
" Problem instance."
" However, this Problem instance is on a remote actor."
)
for actor in self._actors:
ray.kill(actor)
self._actors = None
self._actor_pool = None
@property
def num_actors(self) -> int:
"""
Number of actors (to be) used for parallelization.
If the problem is configured for no parallelization,
the result will be 0.
"""
return self._num_actors
@property
def actors(self) -> Optional[list]:
"""
Get the ray actors, if the Problem object is distributed.
If the Problem object is not distributed and therefore
has no actors, then, the result will be None.
"""
return self._actors
@property
def actor_index(self) -> Optional[int]:
"""Return the actor index if this is a remote worker.
If this is not a remote worker, return None.
"""
return self._actor_index
@property
def is_remote(self) -> bool:
"""Returns True if this problem object lives in a remote ray actor.
Otherwise, returns False.
"""
return self._actor_index is not None
@property
def is_main(self) -> bool:
"""Returns True if this problem object lives in the main process
and not in a remote actor.
Otherwise, returns False.
"""
return self._actor_index is None
@property
def before_eval_hook(self) -> Hook:
"""
Get the Hook which stores the functions to call just before
evaluating a SolutionBatch.
The functions to be stored in this hook are expected to
accept one positional argument, that one argument being the
SolutionBatch which is about to be evaluated.
"""
return self._before_eval_hook
@property
def after_eval_hook(self) -> Hook:
"""
Get the Hook which stores the functions to call just after
evaluating a SolutionBatch.
The functions to be stored in this hook are expected to
accept one argument, that one argument being the SolutionBatch
whose evaluation has just been completed.
The dictionaries returned by the functions in this hook
are accumulated, and reported in the status dictionary of this
problem object.
"""
return self._after_eval_hook
@property
def before_grad_hook(self) -> Hook:
"""
Get the Hook which stores the functions to call just before
its `sample_and_compute_gradients(...)` operation.
"""
return self._before_grad_hook
@property
def after_grad_hook(self) -> Hook:
"""
Get the Hook which stores the functions to call just after
its `sample_and_compute_gradients(...)` operation.
The functions to be stored in this hook are expected to
accept one argument, that one argument being the gradients
dictionary (which was produced by the Problem object,
but not yet followed by the search algorithm).
The dictionaries returned by the functions in this hook
are accumulated, and reported in the status dictionary of this
problem object.
"""
return self._after_grad_hook
@property
def remote_hook(self) -> Hook:
"""
Get the Hook which stores the functions to call when this
Problem object is (re)created on a remote actor.
The functions in this hook should expect one positional
argument, that is the Problem object itself.
"""
return self._remote_hook
def _make_sync_data_for_actors(self) -> Any:
"""
Override this function for providing synchronization between
the main process and the remote actors.
The responsibility of this function is to prepare and return the
data to be sent to the remote actors for synchronization.
If this function returns NotImplemented, then there will be no
syncing.
If this function returns None, there will be no data sent to the
actors for syncing, however, syncing will still be enabled, and
the main actor will ask for sync data from the remote actors
after their jobs are finished.
"""
return NotImplemented
def _use_sync_data_from_main(self, received: Any):
"""
Override this function for providing synchronization between
the main process and the remote actors.
The responsibility of this function is to update the state
of the remote Problem object according to the synchronization
data received by the main process.
"""
pass
def _make_sync_data_for_main(self) -> Any:
"""
Override this function for providing synchronization between
the main process and the remote actors.
The responsibility of this function is to prepare and return the
data to be sent to the main Problem object by a remote actor.
"""
return NotImplemented
def _use_sync_data_from_actors(self, received: list):
"""
Override this function for providing synchronization between
the main process and the remote actors.
The responsibility of this function is to update the state
of the main Problem object according to the synchronization
data received by the remote actors.
"""
pass
def _make_pickle_data_for_main(self) -> dict:
"""
Override this function for preserving the state of a remote
actor in the main state dictionary when pickling a parallelized
problem.
The responsibility of this function is to return the state
of a problem object which lives in a remote actor.
If the remote clones of this problem do not need to be stateful
then you probably do not need to override this method.
"""
return {}
def _use_pickle_data_from_main(self, state: dict):
"""
Override this function for re-creating the internal state of
a problem instance living in a remote actor, by using the
given state dictionary.
If the remote clones of this problem do not need to be stateful
then you probably do not need to override this method.
"""
pass
def _sync_before(self) -> bool:
if self._actors is None:
return False
to_send = self._make_sync_data_for_actors()
if to_send is NotImplemented:
return False
if to_send is not None:
ray.get([actor.call.remote("_use_sync_data_from_main", [to_send], {}) for actor in self._actors])
return True
def _sync_after(self):
if self._actors is None:
return
received = ray.get([actor.call.remote("_make_sync_data_for_main", [], {}) for actor in self._actors])
self._use_sync_data_from_actors(received)
@torch.no_grad()
def _get_best_and_worst(self, batch: "SolutionBatch") -> Optional[dict]:
if self._store_solution_stats is None:
self._store_solution_stats = str(batch.device) == "cpu"
if not self._store_solution_stats:
return {}
senses = self.senses
nobjs = len(senses)
if self._best is None:
self._best_evals = self.make_empty(nobjs, device=batch.device, use_eval_dtype=True)
self._worst_evals = self.make_empty(nobjs, device=batch.device, use_eval_dtype=True)
for i_obj in range(nobjs):
if senses[i_obj] == "min":
self._best_evals[i_obj] = float("inf")
self._worst_evals[i_obj] = float("-inf")
elif senses[i_obj] == "max":
self._best_evals[i_obj] = float("-inf")
self._worst_evals[i_obj] = float("inf")
else:
raise ValueError(f"Invalid sense: {senses[i_obj]}")
self._best = [None] * nobjs
self._worst = [None] * nobjs
def first_is_better(a, b, i_obj):
if senses[i_obj] == "min":
return a < b
elif senses[i_obj] == "max":
return a > b
else:
raise ValueError(f"Invalid sense: {senses[i_obj]}")
def first_is_worse(a, b, i_obj):
if senses[i_obj] == "min":
return a > b
elif senses[i_obj] == "max":
return a < b
else:
raise ValueError(f"Invalid sense: {senses[i_obj]}")
best_sln_indices = [batch.argbest(i) for i in range(nobjs)]
worst_sln_indices = [batch.argworst(i) for i in range(nobjs)]
for i_obj in range(nobjs):
best_sln_index = best_sln_indices[i_obj]
worst_sln_index = worst_sln_indices[i_obj]
scores = batch.access_evals(i_obj)
best_score = scores[best_sln_index]
worst_score = scores[worst_sln_index]
if first_is_better(best_score, self._best_evals[i_obj], i_obj):
self._best_evals[i_obj] = best_score
self._best[i_obj] = batch[best_sln_index].clone()
if first_is_worse(worst_score, self._worst_evals[i_obj], i_obj):
self._worst_evals[i_obj] = worst_score
self._worst[i_obj] = batch[worst_sln_index].clone()
if len(senses) == 1:
return dict(
best=self._best[0],
worst=self._worst[0],
best_eval=float(self._best[0].evals[0]),
worst_eval=float(self._worst[0].evals[0]),
)
else:
return {"best": self._best, "worst": self._worst}
def compare_solutions(self, a: "Solution", b: "Solution", obj_index: Optional[int] = None) -> float:
"""
Compare two solutions.
It is assumed that both solutions are already evaluated.
Args:
a: The first solution.
b: The second solution.
obj_index: The objective index according to which the comparison
will be made.
Can be left as None if the problem is single-objective.
Returns:
A positive number if `a` is better;
a negative number if `b` is better;
0 if there is a tie.
"""
senses = self.senses
obj_index = self.normalize_obj_index(obj_index)
sense = senses[obj_index]
def score(s: Solution):
return s.evals[obj_index]
if sense == "max":
return score(a) - score(b)
elif sense == "min":
return score(b) - score(a)
else:
raise ValueError("Unrecognized sense: " + repr(sense))
def is_better(self, a: "Solution", b: "Solution", obj_index: Optional[int] = None) -> bool:
"""
Check whether or not the first solution is better.
It is assumed that both solutions are already evaluated.
Args:
a: The first solution.
b: The second solution.
obj_index: The objective index according to which the comparison
will be made.
Can be left as None if the problem is single-objective.
Returns:
True if `a` is better; False otherwise.
"""
return self.compare_solutions(a, b, obj_index) > 0
def is_worse(self, a: "Solution", b: "Solution", obj_index: Optional[int] = None) -> bool:
"""
Check whether or not the first solution is worse.
It is assumed that both solutions are already evaluated.
Args:
a: The first solution.
b: The second solution.
obj_index: The objective index according to which the comparison
will be made.
Can be left as None if the problem is single-objective.
Returns:
True if `a` is worse; False otherwise.
"""
return self.compare_solutions(a, b, obj_index) < 0
def _prepare(self) -> None:
"""Prepare a worker instance of the problem for evaluation. To be overridden by the user"""
pass
def _prepare_main(self) -> None:
"""Prepare the main instance of the problem for evaluation."""
self._share_attributes()
def _start_preparations(self) -> None:
"""Prepare the problem for evaluation. Calls self._prepare() if the self._prepared flag is not True."""
if not self._prepared:
if self.actors is None or self._num_actors == 0:
# Call prepare method for any problem class that is expected to do work
self._prepare()
if self.is_main:
# Call share method to distribute shared attributes to actors
self._prepare_main()
self._prepared = True
@property
def _nonserialized_attribs(self) -> List[str]:
return []
def _share_attributes(self) -> None:
if (self._actors is not None) and (len(self._actors) > 0):
for attrib_name in self._shared_attribs:
obj_ref = ray.put(getattr(self, attrib_name))
for actor in self.actors:
actor.call.remote("put_ray_object", [], {"obj_ref": obj_ref, "attrib_name": attrib_name})
def put_ray_object(self, obj_ref: ray.ObjectRef, attrib_name: str) -> None:
setattr(self, attrib_name, ray.get(obj_ref))
@property
def _shared_attribs(self) -> List[str]:
return []
def _device_of_fitness_function(self) -> Optional[Device]:
def device_of_fn(fn: Optional[Callable]) -> Optional[Device]:
if fn is None:
return None
else:
if hasattr(fn, "__evotorch_on_aux_device__") and fn.__evotorch_on_aux_device__:
return self.aux_device
elif hasattr(fn, "device"):
return fn.device
else:
return None
for candidate_fn in (self._objective_func, self._evaluate_all, self._evaluate_batch, self._evaluate):
device = device_of_fn(candidate_fn)
if device is not None:
if candidate_fn is self._evaluate_all:
raise RuntimeError(
"It seems that the `_evaluate_all(...)` method of this Problem object is either decorated"
" by @on_aux_device or by @on_device, or it is specifying a target device via a `device`"
" attribute. However, these decorators (or the `device` attribute) are not supported in the"
" case of `_evaluate_all(...)`."
" The reason is that the checking of the target device and the operations of moving the batch"
" onto the target device are handled by the default implementation of `_evaluate_all` itself."
" To specify a target device, consider decorating `_evaluate_batch(...)` or `_evaluate(...)`"
" instead."
)
return device
return None
def evaluate(self, x: Union["SolutionBatch", "Solution"]):
"""
Evaluate the given Solution or SolutionBatch.
Args:
x: The SolutionBatch to be evaluated.
"""
if isinstance(x, Solution):
batch = x.to_batch()
elif isinstance(x, SolutionBatch):
batch = x
else:
raise TypeError(
f"The method `evaluate(...)` expected a Solution or a SolutionBatch as its argument."
f" However, the received object is {repr(x)}, which is of type {repr(type(x))}."
)
self._parallelize()
if self.is_main:
self.before_eval_hook(batch)
must_sync_after = self._sync_before()
self._start_preparations()
self._evaluate_all(batch)
if must_sync_after:
self._sync_after()
if self.is_main:
self._after_eval_status = {}
best_and_worst = self._get_best_and_worst(batch)
if best_and_worst is not None:
self._after_eval_status.update(best_and_worst)
self._after_eval_status.update(self.after_eval_hook.accumulate_dict(batch))
def _evaluate_all(self, batch: "SolutionBatch"):
if self._actors is None:
fitness_device = self._device_of_fitness_function()
if fitness_device is None:
self._evaluate_batch(batch)
else:
original_device = batch.device
moved_batch = batch.to(fitness_device)
self._evaluate_batch(moved_batch)
batch.set_evals(moved_batch.evals.to(original_device))
else:
if self._num_subbatches is not None:
pieces = batch.split(self._num_subbatches)
elif self._subbatch_size is not None:
pieces = batch.split(max_size=self._subbatch_size)
else:
pieces = batch.split(len(self._actors))
# mapresult = self._actor_pool.map(lambda a, v: a.evaluate_batch.remote(v), list(pieces))
# for i, evals in enumerate(mapresult):
# row_begin, row_end = pieces.indices_of(i)
# batch._evdata[row_begin:row_end, :] = evals
mapresult = self._actor_pool.map_unordered(
lambda a, v: a.evaluate_batch_piece.remote(v[0], v[1]), list(enumerate(pieces))
)
for i, evals in mapresult:
row_begin, row_end = pieces.indices_of(i)
batch._evdata[row_begin:row_end, :] = evals
def _evaluate_batch(self, batch: "SolutionBatch"):
if self._vectorized and (self._objective_func is not None):
result = self._objective_func(batch.values)
if isinstance(result, tuple):
batch.set_evals(*result)
else:
batch.set_evals(result)
else:
for sln in batch:
self._evaluate(sln)
def _evaluate(self, solution: "Solution"):
if self._objective_func is not None:
result = self._objective_func(solution.values)
if isinstance(result, tuple):
solution.set_evals(*result)
else:
solution.set_evals(result)
else:
raise NotImplementedError
@property
def stores_solution_stats(self) -> Optional[bool]:
"""
Whether or not the best and worst solutions are kept.
"""
return self._store_solution_stats
@property
def status(self) -> dict:
"""
Status dictionary of the problem object, updated after the last
evaluation operation.
The dictionaries returned by the functions in `after_eval_hook`
are accumulated, and reported in this status dictionary.
"""
return self._after_eval_status
def ensure_numeric(self):
"""
Ensure that the problem has a numeric dtype.
Raises:
ValueError: if the problem has a non-numeric dtype.
"""
if is_dtype_object(self.dtype):
raise ValueError("Expected a problem with numeric dtype, but the dtype is object.")
def ensure_unbounded(self):
"""
Ensure that the problem has no strict lower and upper bounds.
Raises:
ValueError: if the problem has strict lower and upper bounds.
"""
if not (self.lower_bounds is None and self.upper_bounds is None):
raise ValueError("Expected an unbounded problem, but this problem has lower and/or upper bounds.")
def ensure_single_objective(self):
"""
Ensure that the problem has only one objective.
Raises:
ValueError: if the problem is multi-objective.
"""
n = len(self.senses)
if n > 1:
raise ValueError(f"Expected a single-objective problem, but this problem has {n} objectives.")
def normalize_obj_index(self, obj_index: Optional[int] = None) -> int:
"""
Normalize the objective index.
If the provided index is non-negative, it is ensured that the index
is valid.
If the provided index is negative, the objectives are counted in the
reverse order, and the corresponding non-negative index is returned.
For example, -1 is converted to a non-negative integer corresponding to
the last objective.
If the provided index is None and if the problem is single-objective,
the returned value is 0, which represents the only objective.
If the provided index is None and if the problem is multi-objective,
an error is raised.
Args:
obj_index: The non-normalized objective index.
Returns:
The normalized objective index, as a non-negative integer.
"""
if obj_index is None:
if len(self.senses) == 1:
return 0
else:
raise ValueError(
"This problem is multi-objective, therefore, an explicit objective index was expected."
" However, `obj_index` was found to be None."
)
else:
obj_index = int(obj_index)
if obj_index < 0:
obj_index = len(self.senses) + obj_index
if obj_index < 0 or obj_index >= len(self.senses):
raise IndexError("Objective index out of range.")
return obj_index
def _get_cloned_state(self, *, memo: dict) -> dict:
# Collect the inner states of the remote Problem clones
if self._actors is not None:
self._remote_states = ray.get(
[actor.call.remote("_make_pickle_data_for_main", [], {}) for actor in self._actors]
)
# Prepare the main state dictionary
result = {}
for k, v in self.__dict__.items():
if k in ("_actors", "_actor_pool") or k in self._nonserialized_attribs:
result[k] = None
else:
v_id = id(v)
if v_id in memo:
result[k] = memo[v_id]
else:
with _no_grad_if_basic_dtype(self.dtype):
result[k] = deep_clone(
v,
otherwise_deepcopy=True,
memo=memo,
)
return result
def _get_local_interaction_count(self) -> int:
"""
Get the number of simulator interactions this Problem encountered.
For problems focused on reinforcement learning, it is expected
that the subclass overrides this method to describe its own way
of getting the local interaction count.
When working on parallelized problems, what is returned here is
not necessarily synchronized with the other parallelized instance.
"""
raise NotImplementedError
def _get_local_episode_count(self) -> int:
"""
Get the number of episodes this Problem encountered.
For problems focused on reinforcement learning, it is expected
that the subclass overrides this method to describe its own way
of getting the local episode count.
When working on parallelized problems, what is returned here is
not necessarily synchronized with the other parallelized instance.
"""
raise NotImplementedError
def sample_and_compute_gradients(
self,
distribution,
popsize: int,
*,
num_interactions: Optional[int] = None,
popsize_max: Optional[int] = None,
obj_index: Optional[int] = None,
ranking_method: Optional[str] = None,
with_stats: bool = True,
ensure_even_popsize: bool = False,
) -> Union[list, dict]:
"""
Sample new solutions from the distribution and compute gradients.
The distribution can then be updated according to the computed
gradients.
If the problem is not parallelized, and `with_stats` is False,
then the result will be a single dictionary of gradients.
For example, in the case of a Gaussian distribution, the returned
gradients dictionary would look like this:
{
"mu": ..., # the gradient for the mean
"sigma": ..., # the gradient for the standard deviation
}
If the problem is not parallelized, and `with_stats` is True,
then the result will be a dictionary which contains in itself
the gradients dictionary, and additional elements for providing
further information. In the case of a Gaussian distribution,
the returned dictionary with additional stats would look like
this:
{
"gradients": {
"mu": ..., # the gradient for the mean
"sigma": ..., # the gradient for the standard deviation
},
"num_solutions": ..., # how many solutions were sampled
"mean_eval": ..., # Mean of all evaluations
}
If the problem is parallelized, then the gradient computation will
be distributed among the remote actors. In more details, each actor
will sample its own solutions (such that the total population size
across all remote actors will be near the provided `popsize`)
and will compute its own gradients, and will produce its own
additional stats (if `with_stats` is given as True).
These remote results will then be collected by the main process,
and the final result of this method will be a list of dictionaries,
each dictionary being the result of a remote gradient computation.
The sampled solutions are temporary, and will not be kept
(and will not be returned).
To customize how solutions are sampled and how gradients are
computed, one is encouraged to override
`_sample_and_compute_gradients(...)` (instead of overriding this
method directly.
Args:
distribution: The search distribution from which the solutions
will be sampled, and according to which the gradients will
be computed.
popsize: The number of solutions which will be sampled.
num_interactions: Number of simulator interactions that must
be completed (more solutions will be sampled until this
threshold is reached). This argument is to be used when
the problem has characteristics similar to reinforcement
learning, and an adaptive population size, depending on
the interactions made, is desired.
Otherwise, one can leave this argument as None, in which
case, there will not be any threshold based on number
of interactions.
popsize_max: To be used when `num_interactions` is provided,
as an additional criterion for ending the solution sampling
phase. This argument can be used to prevent the population
size from growing too much while trying to satisfy the
`num_interactions`. If not needed, `popsize_max` can be left
as None.
obj_index: Index of the objective according to which the gradients
will be computed. Can be left as None if the problem has only
one objective.
ranking_method: The solution ranking method to be used when
computing the gradients.
If not specified, the raw fitnesses will be used.
with_stats: If given as False, then the results dictionary will
only contain the gradients information. If given as True,
then the results dictionary will contain within itself
the gradients dictionary, and also additional elements for
providing further information.
The default is True.
ensure_even_popsize: If `ensure_even_popsize` is True and the
problem is not parallelized, then a `popsize` given as an odd
number will cause an error. If `ensure_even_popsize` is True
and the problem is parallelized, then the remote actors will
sample their own sub-populations in such a way that their
sizes are even.
If `ensure_even_popsize` is False, whether or not the
`popsize` is even will not be checked.
When the provided `distribution` is a symmetric (or
"mirrored", or "antithetic"), then this argument must be
given as True.
Returns:
A results dictionary when the problem is not parallelized,
or list of results dictionaries when the problem is parallelized.
"""
# For problems which are configured for parallelization, make sure that the actors are created.
self._parallelize()
# Below we check if there is an inconsistency in arguments.
if (num_interactions is None) and (popsize_max is not None):
# If `num_interactions` is None, then we assume that the user does not wish an adaptive population size.
# However, at the same time, if `popsize_max` is not None, then there is an inconsistency,
# because, `popsize_max` without `num_interactions` (therefore without adaptive population size)
# does not make sense.
# This is probably a configuration error, so, we inform the user by raising an error.
raise ValueError(
f"`popsize_max` was expected as None, because `num_interactions` is None."
f" However, `popsize_max` was found as {popsize_max}."
)
# The problem instance in the main process should trigger the `before_grad_hook`.
if self.is_main:
self._before_grad_hook()
if self.is_main and (self._actors is not None) and (len(self._actors) > 0):
# If this is the main process and the problem is parallelized, then we need to split the request
# into multiple tasks, and then execute those tasks in parallel using the problem's actor pool.
if self._subbatch_size is not None:
# If `subbatch_size` is provided, then we first make sure that `popsize` is divisible by
# `subbatch_size`
if (popsize % self._subbatch_size) != 0:
raise ValueError(
f"This Problem was created with `subbatch_size` as {self._subbatch_size}."
f" When doing remote gradient computation, the requested population size must be divisible by"
f" the `subbatch_size`."
f" However, the requested population size is {popsize}, and the remainder after dividing it"
f" by `subbatch_size` is not 0 (it is {popsize % self._subbatch_size})."
)
# After making sure that `popsize` and `subbatch_size` configurations are compatible, we declare that
# we are going to have n tasks, each task imposing a sample size of `subbatch_size`.
n = int(popsize // self._subbatch_size)
popsize_per_task = [self._subbatch_size for _ in range(n)]
elif self._num_subbatches is not None:
# If `num_subbatches` is provided, then we are going to have n tasks where n is equal to the given
# `num_subbatches`.
popsize_per_task = split_workload(popsize, self._num_subbatches)
else:
# If neither `subbatch_size` nor `num_subbatches` is given, then we will split the workload in such
# a way that each actor will have its share.
popsize_per_task = split_workload(popsize, len(self._actors))
if ensure_even_popsize:
# If `ensure_even_popsize` argument is True, then we need to make sure that each tasks's popsize is
# an even number.
for i in range(len(popsize_per_task)):
if (popsize_per_task[i] % 2) != 0:
# If the i-th actor's assigned popsize is not even, increase its assigned popsize by 1.
popsize_per_task[i] += 1
# The number of tasks is finally determined by the length of `popsize_per_task` list we created above.
num_tasks = len(popsize_per_task)
if num_interactions is None:
# If the argument `num_interactions` is not given, then, for each task, we declare that
# `num_interactions` is None.
num_inter_per_task = [None for _ in range(num_tasks)]
else:
# If the argument `num_interactions` is given, then we compute each task's target number of
# interactions from its sample size.
num_inter_per_task = [
math.ceil((popsize_per_task[i] / popsize) * num_interactions) for i in range(num_tasks)
]
if popsize_max is None:
# If the argument `popsize_max` is not given, then, for each task, we declare that
# `popsize_max` is None.
popsize_max_per_task = [None for _ in range(num_tasks)]
else:
# If the argument `popsize_max` is given, then we compute each task's target maximum population size
# from its sample size.
popsize_max_per_task = [
math.ceil((popsize_per_task[i] / popsize) * popsize_max) for i in range(num_tasks)
]
# We trigger the synchronization between the main process and the remote actors.
# If this problem instance has nothing to synchronize, then `must_sync_after` will be False.
must_sync_after = self._sync_before()
# Because we want to send the distribution to remote actors, we first copy the distribution to cpu
# (unless it is already on cpu)
dist_on_cpu = distribution.to("cpu")
# Here, we use our actor pool to execute our tasks in parallel.
result = list(
self._actor_pool.map_unordered(
(
lambda a, v: a.call.remote(
"_sample_and_compute_gradients",
[dist_on_cpu, v[0]],
{
"obj_index": obj_index,
"num_interactions": v[1],
"popsize_max": v[2],
"ranking_method": ranking_method,
},
)
),
list(zip(popsize_per_task, num_inter_per_task, popsize_max_per_task)),
)
)
# At this point, all the tensors within our collected results are on the CPU.
if torch.device(self.device) != torch.device("cpu"):
# If the main device of this problem instance is not CPU, then we move the tensors to the main device.
result = cast_tensors_in_container(result, device=self.device)
if must_sync_after:
# If a post-gradient synchronization is required, we trigger the synchronization operations.
self._sync_after()
# ####################################################
# # If this is the main process and the problem is parallelized, then we need to split the workload among
# # the remote actors, and then request each of them to compute their gradients.
#
# # We begin by getting the number of actors, and computing the `popsize` for each actor.
# num_actors = len(self._actors)
# popsize_per_actor = split_workload(popsize, num_actors)
#
# if ensure_even_popsize:
# # If `ensure_even_popsize` argument is True, then we need to make sure that each actor's popsize is
# # an even number.
# for i in range(len(popsize_per_actor)):
# if (popsize_per_actor[i] % 2) != 0:
# # If the i-th actor's assigned popsize is not even, increase its assigned popsize by 1.
# popsize_per_actor[i] += 1
#
# if num_interactions is None:
# # If `num_interactions` is None, then the `num_interactions` argument for each actor must also be
# # passed as None.
# num_int_per_actor = [None] * num_actors
# else:
# # If `num_interactions` is not None, then we split the `num_interactions` workload among the actors.
# num_int_per_actor = split_workload(num_interactions, num_actors)
#
# if popsize_max is None:
# # If `popsize_max` is None, then the `popsize_max` argument for each actor must also be None.
# popsize_max_per_actor = [None] * num_actors
# else:
# # If `popsize_max` is not None, then we split the `popsize_max` workload among the actors.
# popsize_max_per_actor = split_workload(popsize_max, num_actors)
#
# # We trigger the synchronization between the main process and the remote actors.
# # If this problem instance has nothing to synchronize, then `must_sync_after` will be False.
# must_sync_after = self._sync_before()
#
# # Because we want to send the distribution to remote actors, we first copy the distribution to cpu
# # (unless it is already on cpu)
# dist_on_cpu = distribution.to("cpu")
#
# # To each actor, we send the request of computing the gradients, and then collect the results
# result = ray.get(
# [
# self._actors[i].call.remote(
# "_gradient_computation_helper",
# [dist_on_cpu, popsize_per_actor[i]],
# dict(
# num_interactions=num_int_per_actor[i],
# popsize_max=popsize_max_per_actor[i],
# obj_index=obj_index,
# ranking_method=ranking_method,
# with_stats=with_stats,
# move_results_to_device="cpu",
# ),
# )
# for i in range(num_actors)
# ]
# )
#
# # At this point, all the tensors within our collected results are on the CPU.
#
# if torch.device(self.device) != torch.device("cpu"):
# # If the main device of this problem instance is not CPU, then we move the tensors to the main device.
# result = cast_tensors_in_container(result, device=device)
#
# if must_sync_after:
# # If a post-gradient synchronization is required, we trigger the synchronization operations.
# self._sync_after()
else:
# If the problem is not parallelized, then we request this instance itself to compute the gradients.
result = self._gradient_computation_helper(
distribution,
popsize,
popsize_max=popsize_max,
obj_index=obj_index,
ranking_method=ranking_method,
num_interactions=num_interactions,
with_stats=with_stats,
)
# The problem instance in the main process should trigger the `after_grad_hook`.
if self.is_main:
self._after_eval_status = self._after_grad_hook.accumulate_dict(result)
# We finally return the results
return result
def _gradient_computation_helper(
self,
distribution,
popsize: int,
*,
num_interactions: Optional[int] = None,
popsize_max: Optional[int] = None,
obj_index: Optional[int] = None,
ranking_method: Optional[str] = None,
with_stats: bool = True,
move_results_to_device: Optional[Device] = None,
) -> dict:
# This is a helper method which makes sure that the provided distribution is in the correct dtype and device.
# This method also makes sure that the results are moved to the desired device.
# At first, we make sure that the objective index is normalized
# (for example, the objective -1 is converted to the index of the last objective).
obj_index = self.normalize_obj_index(obj_index)
if (distribution.dtype != self.dtype) or (distribution.device != self.device):
# Make sure that the distribution is in the correct dtype and device
distribution = distribution.modified_copy(dtype=self.dtype, device=self.device)
# Call the protected method responsible for sampling solutions and computing the gradients
result = self._sample_and_compute_gradients(
distribution,
popsize,
popsize_max=popsize_max,
obj_index=obj_index,
num_interactions=num_interactions,
ranking_method=ranking_method,
)
if move_results_to_device is not None:
# If `move_results_to_device` is provided, move the results to the desired device
result = cast_tensors_in_container(result, device=move_results_to_device)
# Finally, return the result
if with_stats:
return result
else:
return result["gradients"]
@property
def _grad_device(self) -> Device:
"""
Get the device in which new solutions will be made in distributed mode.
In more details, in distributed mode, each actor creates its own
sub-populations, evaluates them, and computes its own gradient
(all such actor gradients eventually being collected by the
distribution-based search algorithm in the main process).
For some problem types, it can make sense for the remote actors to
create their temporary sub-populations on another device
(e.g. on the GPU that is allocated specifically for them).
For such situations, one is encouraged to override this property
and make it return whatever device is to be used.
Note that this property is used by the default implementation of the
method named `_sample_and_compute_grad(...)`. If the method named
`_sample_and_compute_grad(...)` is overriden, this property might not
be called at all.
This is the default (i.e. not-yet-overriden) implementation in the
Problem class, and performs the following operations to decide the
device:
(i) if the Problem object was given an external fitness function that
is decorated by @[on_device][evotorch.decorators.on_device],
or by @[on_aux_device][evotorch.decorators.on_aux_device],
or has a `device` attribute, then return the device requested by that
function; otherwise
(ii) if either one of the methods `_evaluate_batch`, and `_evaluate`
was decorated by @[on_device][evotorch.decorators.on_device]
or by @[on_aux_device][evotorch.decorators.on_aux_device],
or has a `device` attribute, then return the device requested by that
method; otherwise
(iii) return the main device of the Problem object.
"""
fitness_device = self._device_of_fitness_function()
return self.device if fitness_device is None else fitness_device
def _sample_and_compute_gradients(
self,
distribution,
popsize: int,
*,
obj_index: int,
num_interactions: Optional[int] = None,
popsize_max: Optional[int] = None,
ranking_method: Optional[str] = None,
) -> dict:
"""
This method contains the description of how the solutions are sampled
and the gradients are computed according to the given distribution.
One might override this method for customizing the procedure of
sampling solutions and the gradient computation, but this method does
have a default implementation.
This returns a dictionary which contains the gradients for the given
distribution, and also further information. For example, considering
a Gaussian distribution with parameters 'mu' and 'sigma', the result
is expected to look like this:
{
"gradients": {
"mu": ..., # the gradient for the mean (tensor)
"sigma": ..., # the gradient for the std.dev. (tensor)
},
"num_solutions": ..., # how many solutions were sampled (int)
"mean_eval": ..., # Mean of all evaluations (float)
}
A customized version of this method can add more items to the outer
dictionary.
Args:
distribution: The search distribution from which the solutions
will be sampled and according to which the gradients will
be computed. This method assumes that `distribution` is
given with this problem instance's dtype, and in this problem
instance's device.
popsize: Number of solutions to sample.
obj_index: Objective index, expected as an integer.
num_interactions: Number of simulator interactions that must be
reached before computing the gradients.
Having this argument as an integer implies that adaptive
population is requested: more solutions are to be sampled
until this number of simulator interactions are made.
Can also be None if this threshold is not needed.
popsize_max: Maximum population size for when the population
size is adaptive (where the adaptiveness is enabled when
`num_interactions` is not None).
Can be left as None if a maximum population size limit
is not needed.
ranking_method: Ranking method to be used when computing the
gradients. Can be left as None, in which case the raw
fitnesses will be used.
Returns:
A dictionary which contains the gradients, number of solutions,
mean of all the evaluation results, and optionally further
items (if customized to do so).
"""
# Annotate the variable which will store the temporary SolutionBatch for computing the local gradient.
resulting_batch: SolutionBatch
# Get the device in which the new solutions will be made.
grad_device = torch.device(self._grad_device)
distribution = distribution.to(grad_device)
# Below we define an inner utility function which samples and evaluates a new SolutionBatch.
# This newly evaluated SolutionBatch is returned.
def sample_evaluated_batch() -> SolutionBatch:
batch = SolutionBatch(self, popsize, device=grad_device)
distribution.sample(out=batch.access_values(), generator=self.generator)
self.evaluate(batch)
return batch
if num_interactions is None:
# If a `num_interactions` threshold is not given (i.e. is left as None), then we assume that an adaptive
# population is not desired.
# We therefore simply sample and evaluate a single SolutionBatch, and declare it as our main batch.
resulting_batch = sample_evaluated_batch()
else:
# If we have a `num_interactions` threshold, then we might have to sample more than one SolutionBatch
# (until `num_interactions` is reached).
# We start by defining a list (`batches`) which is to store all the batches we will sample.
batches = []
# We will have to count the number of all simulator interactions that we have encountered during the
# execution of this method. So, to count it correctly, we first get the interaction count that we already
# have before sampling and evaluating our new solutions.
interaction_count_at_first = self._get_local_interaction_count()
# Below is an inner function which returns how many simulator interactions we have done so far.
# It makes use of the variable `interaction_count_at_first` defined above.
def current_num_interactions() -> int:
return self._get_local_interaction_count() - interaction_count_at_first
# We also keep track of the total number of solutions.
# We might need this if there is a `popsize_max` threshold.
current_popsize = 0
# The main loop of the adaptive sampling.
while True:
# Sample and evaluate a new SolutionBatch, and add it to our batches list.
batches.append(sample_evaluated_batch())
# Increase our total population size by the size of the most recent batch.
current_popsize += popsize
if current_num_interactions() > num_interactions:
# If the number of interactions has reached or exceeded the `num_interactions` threshold,
# we exit the loop.
break
if (popsize_max is not None) and (current_popsize >= popsize_max):
# If we have `popsize_max` threshold and our total population size have reached or exceeded
# the `popsize_max` threshold, we exit the loop.
break
if len(batches) == 1:
# If we have only one batch in our batches list, that batch can be declared as our main batch.
resulting_batch = batches[0]
else:
# If we have multiple batches in our batches list, we concatenate all those batches and
# declare the result of the concatenation as our main batch.
resulting_batch = SolutionBatch.cat(batches)
# We take the solutions (`samples`) and the fitnesses from our main batch.
samples = resulting_batch.access_values(keep_evals=True)
fitnesses = resulting_batch.access_evals(obj_index)
# With the help of `samples` and `fitnesses`, we now compute our gradients.
grads = distribution.compute_gradients(
samples, fitnesses, objective_sense=self.senses[obj_index], ranking_method=ranking_method
)
if grad_device != self.device:
grads = cast_tensors_in_container(grads, device=self.device)
# Finally, we return the result, which is a dictionary containing the gradients and further information.
return {
"gradients": grads,
"num_solutions": len(resulting_batch),
"mean_eval": float(torch.mean(resulting_batch.access_evals(obj_index))),
}
def is_on_cpu(self) -> bool:
"""
Whether or not the Problem object has its device set as "cpu".
"""
return str(self.device) == "cpu"
def make_callable_evaluator(self, *, obj_index: Optional[int] = None) -> "ProblemBoundEvaluator":
"""
Get a callable evaluator for evaluating the given solutions.
Let us assume that we have a [Problem][evotorch.core.Problem]
declared like this:
```python
from evotorch import Problem
my_problem = Problem(
"min",
fitness_function_goes_here,
...,
)
```
Using the regular API of EvoTorch, one has to generate solutions for this
problem as follows:
```python
population_size = ...
my_solutions = my_problem.generate_batch(population_size)
```
For editing the decision values within the
[SolutionBatch][evotorch.core.SolutionBatch] `my_solutions`, one has to
do the following:
```python
new_decision_values = ...
my_solutions.set_values(new_decision_values)
```
Finally, to evaluate `my_solutions`, one would have to do these:
```python
my_problem.evaluate(my_solutions)
fitnesses = my_problem.evals
```
One could desire a different interface which is more compatible with
functional programming paradigm, especially when planning to use the
functional algorithms (such as, for example, the functional
[cem][evotorch.algorithms.functional.funccem.cem]).
To achieve this, one can do the following:
```python
f = my_problem.make_callable_evaluator()
```
Now, we have a new object `f`, which behaves like a function.
This function-like object expects a tensor of decision values, and
returns fitnesses, as shown below:
```python
random_decision_values = torch.randn(
population_size,
my_problem.solution_length,
dtype=my_problem.dtype,
)
fitnesses = f(random_decision_values)
```
**Parallelized fitness evaluation.**
If a `Problem` object is condifured to use parallelized evaluation with
the help of multiple actors, a callable evaluator made out of that
`Problem` object will also make use of those multiple actors.
**Additional batch dimensions.**
If a callable evaluator receives a tensor with 3 or more dimensions,
those extra leftmost dimensions will be considered as batch
dimensions. The returned fitness tensor will also preserve those batch
dimensions.
**Notes on vmap.**
`ProblemBoundEvaluator` is a shallow wrapper around a `Problem` object.
It does NOT transform the underlying problem object to its stateless
counterpart, and therefore it does NOT conform to pure functional
programming paradigm. Being stateful, it will NOT work correctly with
`vmap`. For batched evaluations, it is recommended to use extra batch
dimensions, instead of using `vmap`.
Args:
obj_index: The index of the objective according to which the
evaluations will be done. If the problem is single-objective,
this is not required. If the problem is multi-objective, this
needs to be given as an integer.
Returns:
A callable fitness evaluator, bound to this problem object.
"""
return ProblemBoundEvaluator(self, obj_index=obj_index)
actor_index: Optional[int]
property
readonly
¶
Return the actor index if this is a remote worker. If this is not a remote worker, return None.
actors: Optional[list]
property
readonly
¶
Get the ray actors, if the Problem object is distributed. If the Problem object is not distributed and therefore has no actors, then, the result will be None.
after_eval_hook: Hook
property
readonly
¶
Get the Hook which stores the functions to call just after evaluating a SolutionBatch.
The functions to be stored in this hook are expected to accept one argument, that one argument being the SolutionBatch whose evaluation has just been completed.
The dictionaries returned by the functions in this hook are accumulated, and reported in the status dictionary of this problem object.
after_grad_hook: Hook
property
readonly
¶
Get the Hook which stores the functions to call just after
its sample_and_compute_gradients(...) operation.
The functions to be stored in this hook are expected to accept one argument, that one argument being the gradients dictionary (which was produced by the Problem object, but not yet followed by the search algorithm).
The dictionaries returned by the functions in this hook are accumulated, and reported in the status dictionary of this problem object.
aux_device: Union[str, torch.device]
property
readonly
¶
Auxiliary device to help with the computations, most commonly for speeding up the solution evaluations.
An auxiliary device is different than the main device of the Problem
object (the main device being expressed by the device property).
While the main device of the Problem object determines where the
solutions and the populations are stored (and also using which device
should a SearchAlgorithm instance communicate with the problem),
an auxiliary device is a device that might be used by the Problem
instance itself for its own computations (e.g. computations defined
within the methods _evaluate(...) or _evaluate_batch(...)).
If the problem's main device is something other than "cpu", that main device is also seen as the auxiliary device, and therefore returned by this property.
If the problem's main device is "cpu", then the auxiliary device
is decided as follows. If num_gpus_per_actor of the Problem object
was set as "all" and if this instance is a remote instance, then the
auxiliary device is guessed as "cuda:N" where N is the actor index.
In all other cases, the auxiliary device is "cuda" if cuda is
available, and "cpu" otherwise.
before_eval_hook: Hook
property
readonly
¶
Get the Hook which stores the functions to call just before evaluating a SolutionBatch.
The functions to be stored in this hook are expected to accept one positional argument, that one argument being the SolutionBatch which is about to be evaluated.
before_grad_hook: Hook
property
readonly
¶
Get the Hook which stores the functions to call just before
its sample_and_compute_gradients(...) operation.
device: Union[str, torch.device]
property
readonly
¶
device of the Problem object.
New solutions and populations will be generated in this device.
dtype: Union[str, torch.dtype, numpy.dtype, Type]
property
readonly
¶
dtype of the Problem object.
The decision variables of the optimization problem are of this dtype.
eval_data_length: int
property
readonly
¶
Length of the extra evaluation data vector for each solution.
eval_dtype: Union[str, torch.dtype, numpy.dtype, Type]
property
readonly
¶
evaluation dtype of the Problem object.
The evaluation results of the solutions are stored according to this dtype.
generator: Optional[torch._C.Generator]
property
readonly
¶
Random generator used by this Problem object.
Can also be None, which means that the Problem object will use the global random generator of PyTorch.
has_own_generator: bool
property
readonly
¶
Whether or not the Problem object has its own random generator.
If this is True, then the Problem object will use its own random generator when creating random values or tensors. If this is False, then the Problem object will use the global random generator when creating random values or tensors.
initial_lower_bounds: Optional[torch.Tensor]
property
readonly
¶
Initial lower bounds, for when initializing a new solution.
If such a bound was declared during the initialization phase, the returned value is a torch tensor (in the form of a vector or in the form of a scalar). If no such bound was declared, the returned value is None.
initial_upper_bounds: Optional[torch.Tensor]
property
readonly
¶
Initial upper bounds, for when initializing a new solution.
If such a bound was declared during the initialization phase, the returned value is a torch tensor (in the form of a vector or in the form of a scalar). If no such bound was declared, the returned value is None.
is_main: bool
property
readonly
¶
Returns True if this problem object lives in the main process and not in a remote actor. Otherwise, returns False.
is_multi_objective: bool
property
readonly
¶
Whether or not the problem is multi-objective
is_remote: bool
property
readonly
¶
Returns True if this problem object lives in a remote ray actor. Otherwise, returns False.
is_single_objective: bool
property
readonly
¶
Whether or not the problem is single-objective
lower_bounds: Optional[torch.Tensor]
property
readonly
¶
Lower bounds for the allowed values of a solution.
If such a bound was declared during the initialization phase, the returned value is a torch tensor (in the form of a vector or in the form of a scalar). If no such bound was declared, the returned value is None.
num_actors: int
property
readonly
¶
Number of actors (to be) used for parallelization. If the problem is configured for no parallelization, the result will be 0.
objective_sense: Union[str, Iterable[str]]
property
readonly
¶
Get the objective sense.
If the problem is single-objective, then a single string is returned. If the problem is multi-objective, then the objective senses will be returned in a list.
The returned string in the single-objective case, or each returned string in the multi-objective case, is "min" or "max".
remote_hook: Hook
property
readonly
¶
Get the Hook which stores the functions to call when this Problem object is (re)created on a remote actor.
The functions in this hook should expect one positional argument, that is the Problem object itself.
senses: Iterable[str]
property
readonly
¶
Get the objective senses.
The return value is a list of strings, each string being "min" or "max".
solution_length: Optional[int]
property
readonly
¶
Get the solution length.
Problems with dtype=None do not have solution lengths.
For such problems, this property returns None.
status: dict
property
readonly
¶
Status dictionary of the problem object, updated after the last evaluation operation.
The dictionaries returned by the functions in after_eval_hook
are accumulated, and reported in this status dictionary.
stores_solution_stats: Optional[bool]
property
readonly
¶
Whether or not the best and worst solutions are kept.
upper_bounds: Optional[torch.Tensor]
property
readonly
¶
Upper bounds for the allowed values of a solution.
If such a bound was declared during the initialization phase, the returned value is a torch tensor (in the form of a vector or in the form of a scalar). If no such bound was declared, the returned value is None.
__init__(self, objective_sense, objective_func=None, *, initial_bounds=None, bounds=None, solution_length=None, dtype=None, eval_dtype=None, device=None, eval_data_length=None, seed=None, num_actors=None, actor_config=None, num_gpus_per_actor=None, num_subbatches=None, subbatch_size=None, store_solution_stats=None, vectorized=None)
special
¶
__init__(...): Initialize the Problem object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
objective_sense |
Union[str, Iterable[str]] |
A string, or a sequence of strings.
For a single-objective problem, a single string
("min" or "max", for minimization or maximization)
is enough.
For a problem with |
required |
initial_bounds |
Union[Iterable[Union[float, Iterable[float], torch.Tensor]], evotorch.core.BoundsPair] |
In which interval will the values of a
new solution will be initialized.
Expected as a tuple, each element being either a
scalar, or a vector of length |
None |
bounds |
Union[Iterable[Union[float, Iterable[float], torch.Tensor]], evotorch.core.BoundsPair] |
Interval in which all the solutions must always
reside.
Expected as a tuple, each element being either a
scalar, or a vector of length |
None |
solution_length |
Optional[int] |
Length of a solution.
Required for all fixed-length numeric optimization
problems.
For variable-length problems (which might or might not
be numeric), one is expected to leave |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
dtype (data type) of the data stored by a solution.
Can be given as a string (e.g. "float32"),
or as a numpy dtype (e.g. |
None |
eval_dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
dtype to be used for storing the evaluations
(or fitnesses, or scores, or costs, or losses)
of the solutions.
Can be given as a string (e.g. "float32"),
or as a numpy dtype (e.g. |
None |
device |
Union[str, torch.device] |
Default device in which a new population will be
generated. For non-numeric problems, this must be "cpu".
For numeric problems, this can be any device supported
by PyTorch (e.g. "cuda").
Note that, if the number of actors of the problem is configured
to be more than 1, |
None |
eval_data_length |
Optional[int] |
In addition to evaluation results (which are (un)fitnesses, or scores, or costs, or losses), each solution can store extra evaluation data. If storage of such extra evaluation data is required, one can set this argument to an integer bigger than 0. |
None |
seed |
Optional[int] |
Random seed to be used by the random number generator attached to the problem object. If left as None, no random number generator will be attached, and the global random number generator of PyTorch will be used instead. |
None |
num_actors |
Union[int, str] |
Number of actors to create for parallelized
evaluation of the solutions.
Certain string values are also accepted.
When given as "max" or as "num_cpus", the number of actors
will be equal to the number of all available CPUs in the ray
cluster.
When given as "num_gpus", the number of actors will be
equal to the number of all available GPUs in the ray
cluster, and each actor will be assigned a GPU.
There is also an option, "num_devices", which means that
both the numbers of CPUs and GPUs will be analyzed, and
new actors and GPUs for them will be allocated,
in a one-to-one mapping manner, if possible.
In more details, with |
None |
actor_config |
Optional[dict] |
A dictionary, representing the keyword arguments
to be passed to the options(...) used when creating the
ray actor objects. To be used for explicitly allocating
resources per each actor.
For example, for declaring that each actor is to use a GPU,
one can pass |
None |
num_gpus_per_actor |
Union[int, float, str] |
Number of GPUs to be allocated by each
remote actor.
The default behavior is to NOT allocate any GPU at all
(which is the default behavior of the ray library as well).
When given as a number |
None |
num_subbatches |
Optional[int] |
If |
None |
subbatch_size |
Optional[int] |
If |
None |
store_solution_stats |
Optional[bool] |
Whether or not the problem object should keep track of the best and worst solutions. Can also be left as None (which is the default behavior), in which case, it will store the best and worst solutions only when the first solution batch it encounters is on the cpu. This default behavior is to ensure that there is no transfer between the cpu and a foreign computation device (like the gpu) just for the sake of keeping the best and the worst solutions. |
None |
vectorized |
Optional[bool] |
Set this to True if the provided fitness function
is vectorized but is not decorated via |
None |
Source code in evotorch/core.py
def __init__(
self,
objective_sense: ObjectiveSense,
objective_func: Optional[Callable] = None,
*,
initial_bounds: Optional[BoundsPairLike] = None,
bounds: Optional[BoundsPairLike] = None,
solution_length: Optional[int] = None,
dtype: Optional[DType] = None,
eval_dtype: Optional[DType] = None,
device: Optional[Device] = None,
eval_data_length: Optional[int] = None,
seed: Optional[int] = None,
num_actors: Optional[Union[int, str]] = None,
actor_config: Optional[dict] = None,
num_gpus_per_actor: Optional[Union[int, float, str]] = None,
num_subbatches: Optional[int] = None,
subbatch_size: Optional[int] = None,
store_solution_stats: Optional[bool] = None,
vectorized: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the Problem object.
Args:
objective_sense: A string, or a sequence of strings.
For a single-objective problem, a single string
("min" or "max", for minimization or maximization)
is enough.
For a problem with `n` objectives, a sequence
of strings (e.g. a list of strings) of length `n` is
required, each string in the sequence being "min" or
"max". This argument specifies the goal of the
optimization.
initial_bounds: In which interval will the values of a
new solution will be initialized.
Expected as a tuple, each element being either a
scalar, or a vector of length `n`, `n` being the
length of a solution.
If a manual solution initialization is preferred
(instead of an interval-based initialization),
one can leave `initial_bounds` as None, and override
the `_fill(...)` method in the inheriting subclass.
bounds: Interval in which all the solutions must always
reside.
Expected as a tuple, each element being either a
scalar, or a vector of length `n`, `n` being the
length of a solution.
This argument is optional, and can be left as None
if one does not wish to declare hard bounds on the
decision values of the problem.
If `bounds` is specified, `initial_bounds` is missing,
and `_fill(...)` is not overriden, then `bounds` will
also serve as the `initial_bounds`.
solution_length: Length of a solution.
Required for all fixed-length numeric optimization
problems.
For variable-length problems (which might or might not
be numeric), one is expected to leave `solution_length`
as None, and declare `dtype` as `object`.
dtype: dtype (data type) of the data stored by a solution.
Can be given as a string (e.g. "float32"),
or as a numpy dtype (e.g. `numpy.dtype("float32")`),
or as a PyTorch dtype (e.g. `torch.float32`).
Alternatively, if the problem is variable-length
and/or non-numeric, one is expected to declare `dtype`
as `object`.
eval_dtype: dtype to be used for storing the evaluations
(or fitnesses, or scores, or costs, or losses)
of the solutions.
Can be given as a string (e.g. "float32"),
or as a numpy dtype (e.g. `numpy.dtype("float32")`),
or as a PyTorch dtype (e.g. `torch.float32`).
`eval_dtype` must always refer to a "float" data type,
therefore, `object` is not accepted as a valid `eval_dtype`.
If `eval_dtype` is not specified (i.e. left as None),
then the following actions are taken to determine the
`eval_dtype`:
if `dtype` is "float16", `eval_dtype` becomes "float16";
if `dtype` is "bfloat16", `eval_dtype` becomes "bfloat16";
if `dtype` is "float32", `eval_dtype` becomes "float32";
if `dtype` is "float64", `eval_dtype` becomes "float64";
and for any other `dtype`, `eval_dtype` becomes "float32".
device: Default device in which a new population will be
generated. For non-numeric problems, this must be "cpu".
For numeric problems, this can be any device supported
by PyTorch (e.g. "cuda").
Note that, if the number of actors of the problem is configured
to be more than 1, `device` has to be "cpu" (or, equivalently,
left as None).
eval_data_length: In addition to evaluation results
(which are (un)fitnesses, or scores, or costs, or losses),
each solution can store extra evaluation data.
If storage of such extra evaluation data is required,
one can set this argument to an integer bigger than 0.
seed: Random seed to be used by the random number generator
attached to the problem object.
If left as None, no random number generator will be
attached, and the global random number generator of
PyTorch will be used instead.
num_actors: Number of actors to create for parallelized
evaluation of the solutions.
Certain string values are also accepted.
When given as "max" or as "num_cpus", the number of actors
will be equal to the number of all available CPUs in the ray
cluster.
When given as "num_gpus", the number of actors will be
equal to the number of all available GPUs in the ray
cluster, and each actor will be assigned a GPU.
There is also an option, "num_devices", which means that
both the numbers of CPUs and GPUs will be analyzed, and
new actors and GPUs for them will be allocated,
in a one-to-one mapping manner, if possible.
In more details, with `num_actors="num_devices"`, if
`device` is given as a GPU device, then it will be inferred
that the user wishes to put everything (including the
population) on a single GPU, and therefore there won't be
any allocation of actors nor GPUs.
With `num_actors="num_devices"` and with `device` set as
"cpu" (or as left as None), if there are multiple CPUs
and multiple GPUs, then `n` actors will be allocated
where `n` is the minimum among the number of CPUs
and the number of GPUs, so that there can be one-to-one
mapping between CPUs and GPUs (i.e. such that each actor
can be assigned an entire GPU).
If `num_actors` is given as "num_gpus" or "num_devices",
the argument `num_gpus_per_actor` must not be used,
and the `actor_config` dictionary must not contain the
key "num_gpus".
If `num_actors` is given as something other than "num_gpus"
or "num_devices", and if you wish to assign GPUs to each
actor, then please see the argument `num_gpus_per_actor`.
actor_config: A dictionary, representing the keyword arguments
to be passed to the options(...) used when creating the
ray actor objects. To be used for explicitly allocating
resources per each actor.
For example, for declaring that each actor is to use a GPU,
one can pass `actor_config=dict(num_gpus=1)`.
Can also be given as None (which is the default),
if no such options are to be passed.
num_gpus_per_actor: Number of GPUs to be allocated by each
remote actor.
The default behavior is to NOT allocate any GPU at all
(which is the default behavior of the ray library as well).
When given as a number `n`, each actor will be given
`n` GPUs (where `n` can be an integer, or can be a `float`
for fractional allocation).
When given as a string "max", then the available GPUs
across the entire ray cluster (or within the local computer
in the simplest cases) will be equally distributed among
the actors.
When given as a string "all", then each actor will have
access to all the GPUs (this will be achieved by suppressing
the environment variable `CUDA_VISIBLE_DEVICES` for each
actor).
When the problem is not distributed (i.e. when there are
no actors), this argument is expected to be left as None.
num_subbatches: If `num_subbatches` is None (assuming that
`subbatch_size` is also None), then, when evaluating a
population, the population will be split into n pieces, `n`
being the number of actors, and each actor will evaluate
its assigned piece. If `num_subbatches` is an integer `m`,
then the population will be split into `m` pieces,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
While using a distributed algorithm, this argument determines
how many sub-batches will be generated, and therefore,
how many gradients will be computed by the remote actors.
subbatch_size: If `subbatch_size` is None (assuming that
`num_subbatches` is also None), then, when evaluating a
population, the population will be split into `n` pieces, `n`
being the number of actors, and each actor will evaluate its
assigned piece. If `subbatch_size` is an integer `m`,
then the population will be split into pieces of size `m`,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
When there can be significant difference across the solutions
in terms of computational requirements, specifying a
`subbatch_size` can be beneficial, because, while one
actor is busy with a subbatch containing computationally
challenging solutions, other actors can accept more
tasks and save time.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
While using a distributed algorithm, this argument determines
the size of a sub-batch (or sub-population) sampled by a
remote actor for computing a gradient.
In distributed mode, it is expected that the population size
is divisible by `subbatch_size`.
store_solution_stats: Whether or not the problem object should
keep track of the best and worst solutions.
Can also be left as None (which is the default behavior),
in which case, it will store the best and worst solutions
only when the first solution batch it encounters is on the
cpu. This default behavior is to ensure that there is no
transfer between the cpu and a foreign computation device
(like the gpu) just for the sake of keeping the best and
the worst solutions.
vectorized: Set this to True if the provided fitness function
is vectorized but is not decorated via `@vectorized`.
"""
# Set the dtype for the decision variables of the Problem
if dtype is None:
self._dtype = torch.float32
elif is_dtype_object(dtype):
self._dtype = object
else:
self._dtype = to_torch_dtype(dtype)
_evolog.info(message_from(self, f"The `dtype` for the problem's decision variables is set as {self._dtype}"))
# Set the dtype for the solution evaluations (i.e. fitnesses and evaluation data)
if eval_dtype is not None:
# If an `eval_dtype` is explicitly stated, then accept it as the `_eval_dtype` of the Problem
self._eval_dtype = to_torch_dtype(eval_dtype)
else:
# This is the case where an `eval_dtype` is not explicitly stated by the user.
# We need to choose a default.
if self._dtype in (torch.float16, torch.bfloat16, torch.float64):
# If the `dtype` of the problem is a non-32-bit float type (i.e. float16, bfloat16, float64)
# then we use that as our `_eval_dtype` as well.
self._eval_dtype = self._dtype
else:
# For any other `dtype`, we use float32 as our `_eval_dtype`.
self._eval_dtype = torch.float32
_evolog.info(
message_from(
self, f"`eval_dtype` (the dtype of the fitnesses and evaluation data) is set as {self._eval_dtype}"
)
)
# Set the main device of the Problem object
self._device = torch.device("cpu") if device is None else torch.device(device)
_evolog.info(message_from(self, f"The `device` of the problem is set as {self._device}"))
# Declare the internal variable that might store the random number generator
self._generator: Optional[torch.Generator] = None
# Set the seed of the Problem object, if a seed is provided
self.manual_seed(seed)
# Declare the internal variables that will store the bounds and the solution length
self._initial_lower_bounds: Optional[torch.Tensor] = None
self._initial_upper_bounds: Optional[torch.Tensor] = None
self._lower_bounds: Optional[torch.Tensor] = None
self._upper_bounds: Optional[torch.Tensor] = None
self._solution_length: Optional[int] = None
if self._dtype is object:
# If dtype is given as `object`, then there are some runtime sanity checks to perform
if bounds is not None or initial_bounds is not None:
# With dtype as object, if bounds are given then we raise an error.
# This is because the `object` dtype implies that the decision values are not necessarily numeric,
# and therefore, we cannot have the guarantee of satisfying numeric bounds.
raise ValueError(
f"With dtype as {repr(dtype)}, expected to receive `initial_bounds` and/or `bounds` as None."
f" However, one or both of them is/are set as value(s) other than None."
)
if solution_length is not None:
# With dtype as object, if `solution_length` is provided, then we raise an error.
# This is because the `object` dtype implies that the solutions can be expressed via various
# containers, each with its own length, and therefore, a fixed solution length cannot be guaranteed.
raise ValueError(
f"With dtype as {repr(dtype)}, expected to receive `solution_length` as None."
f" However, received `solution_length` as {repr(solution_length)}."
)
if str(self._device) != "cpu":
# With dtype as object, if `device` is something other than "cpu", then we raise an error.
# This is because the `object` dtype implies that the decision values are stored by an ObjectArray,
# whose device is always "cpu".
raise ValueError(
f"With dtype as {repr(dtype)}, expected to receive `device` as 'cpu'."
f" However, received `device` as {repr(device)}."
)
else:
# If dtype is something other than `object`, then we need to make sure that we have a valid length for
# solutions, and also properly store the given numeric bounds.
if solution_length is None:
# With a numeric dtype, if solution length is missing, then we raise an error.
raise ValueError(
f"Together with a numeric dtype ({repr(dtype)}),"
f" expected to receive `solution_length` as an integer."
f" However, `solution_length` is None."
)
else:
# With a numeric dtype, if a solution length is provided, we make sure that it is integer.
solution_length = int(solution_length)
# Store the solution length
self._solution_length = solution_length
if (bounds is not None) or (initial_bounds is not None):
# This is the case where we have a dtype other than `object`, and either `bounds` or `initial_bounds`
# was provided.
initbnd_tuple_name = "initial_bounds"
bnd_tuple_name = "bounds"
if (bounds is not None) and (initial_bounds is None):
# With a numeric dtype, if strict bounds are given but initial bounds are not given, then we assume
# that the strict bounds also serve as the initial bounds.
# Therefore, we take clones of the strict bounds and use this clones as the initial bounds.
initial_bounds = clone(bounds)
initbnd_tuple_name = "bounds"
# Below is an internal helper function for some common operations for the (strict) bounds
# and for the initial bounds.
def process_bounds(bounds_tuple: BoundsPairLike, tuple_name: str) -> BoundsPair:
# This function receives the bounds_tuple (a tuple containing lower and upper bounds),
# and the string name of the bounds argument ("bounds" or "initial_bounds").
# What is returned is the bounds expressed as PyTorch tensors in the correct dtype and device.
nonlocal solution_length
# Extract the lower and upper bounds from the received bounds tuple.
lb, ub = bounds_tuple
# Make sure that the lower and upper bounds are expressed as tensors of correct dtype and device.
lb = self.make_tensor(lb)
ub = self.make_tensor(ub)
for bound_array in (lb, ub): # For each boundary tensor (lb and ub)
if bound_array.ndim not in (0, 1):
# If the boundary tensor is not as scalar and is not a 1-dimensional vector, then raise an
# error.
raise ValueError(
f"Lower and upper bounds are expected as scalars or as 1-dimensional vectors."
f" However, these given boundaries have incompatible shape:"
f" {bound_array} (of shape {bound_array.shape})."
)
if bound_array.ndim == 1:
if len(bound_array) != solution_length:
# In the case where the boundary tensor is a 1-dimensional vector, if this vector's length
# is not equal to the solution length, then we raise an error.
raise ValueError(
f"When boundaries are expressed as 1-dimensional vectors, their length are"
f" expected as the solution length of the Problem object."
f" However, while the problem's solution length is {solution_length},"
f" these given boundaries have incompatible length:"
f" {bound_array} (of length {len(bound_array)})."
)
# Return the processed forms of the lower and upper boundary tensors.
return lb, ub
# Process the initial bounds with the help of the internal function `process_bounds(...)`
init_lb, init_ub = process_bounds(initial_bounds, initbnd_tuple_name)
# Store the processed initial bounds
self._initial_lower_bounds = init_lb
self._initial_upper_bounds = init_ub
if bounds is not None:
# If there are strict bounds, then process those bounds with the help of `process_bounds(...)`.
lb, ub = process_bounds(bounds, bnd_tuple_name)
# Store the processed bounds
self._lower_bounds = lb
self._upper_bounds = ub
# Annotate the variable that will store the objective sense(s) of the problem
self._objective_sense: ObjectiveSense
# Below is an internal function which makes sure that a provided objective sense has a valid value
# (where valid values are "min" or "max")
def validate_sense(s: str):
if s not in ("min", "max"):
raise ValueError(
f"Invalid objective sense: {repr(s)}."
f"Instead, please provide the objective sense as 'min' or 'max'."
)
if not is_sequence(objective_sense):
# If the provided objective sense is not a sequence, then convert it to a single-element list
senses = [objective_sense]
num_senses = 1
else:
# If the provided objective sense is a sequence, then take a list copy of it
senses = list(objective_sense)
num_senses = len(objective_sense)
# Ensure that each provided objective sense is valid
for sense in senses:
validate_sense(sense)
if num_senses == 0:
# If the given sequence of objective senses is empty, then we raise an error.
raise ValueError(
"Encountered an empty sequence via `objective_sense`."
" For a single-objective problem, please set `objective_sense` as 'min' or 'max'."
" For a multi-objective problem, please set `objective_sense` as a sequence,"
" each element being 'min' or 'max'."
)
# Store the objective senses
self._senses: Iterable[str] = senses
# Store the provided objective function (which can be None)
self._objective_func: Optional[Callable] = objective_func
# Declare the instance variable that will store whether or not the external fitness function is
# vectorized, if such an external fitness function is given.
self._vectorized: Optional[bool]
# Store the information which indicates whether or not the given objective function is vectorized
if self._objective_func is None:
# This is the case where an external fitness function is not given.
# In this case, we expect the keyword argument `vectorized` to be left as None.
if vectorized is not None:
# If the keyword argument `vectorized` is something other than None, then we raise an error
# to let the user know.
raise ValueError(
f"This problem object received no external fitness function."
f" When not using an external fitness function, the keyword argument `vectorized`"
f" is expected to be left as None."
f" However, the value of the keyword argument `vectorized` is {vectorized}."
)
# At this point, we know that we do not have an external fitness function.
# The variable which is supposed to tell us whether or not the external fitness function is vectorized
# is therefore irrelevant. We just set it as None.
self._vectorized = None
else:
# This is the case where an external fitness function is given.
if (
hasattr(self._objective_func, "__evotorch_vectorized__")
and self._objective_func.__evotorch_vectorized__
):
# If the external fitness function has an attribute `__evotorch_vectorized__`, and the value of this
# attribute evaluates to True, then this is an indication that the fitness function was decorated
# with `@vectorized`.
if vectorized is not None:
# At this point, we know (or at least have the assumption) that the fitness function was decorated
# with `@vectorized`. Any boolean value given via the keyword argument `vectorized` would therefore
# be either redundant or conflicting.
# Therefore, in this case, if the keyword argument `vectorized` is anything other than None, we
# raise an error to inform the user.
raise ValueError(
f"Received a fitness function that was decorated via @vectorized."
f" When using such a fitness function, the keyword argument `vectorized`"
f" is expected to be left as None."
f" However, the value of the keyword argument `vectorized` is {vectorized}."
)
# Since we know that our fitness function declares itself as vectorized, we set the instance variable
# _vectorized as True.
self._vectorized = True
else:
# This is the case in which the fitness function does not appear to be decorated via `@vectorized`.
# In this case, if the keyword argument `vectorized` has a value that is equivalent to True,
# then the value of `_vectorized` becomes True. On the other hand, if the keyword argument `vectorized`
# was left as None or if it has a value that is equivalent to False, `_vectorized` becomes False.
self._vectorized = bool(vectorized)
# If the evaluation data length is explicitly stated, then convert it to an integer and store it.
# Otherwise, store the evaluation data length as 0.
self._eval_data_length = 0 if eval_data_length is None else int(eval_data_length)
# Initialize the actor index.
# If the problem is configured to be parallelized and the parallelization is triggered, then each remote
# copy will have a different integer value for `_actor_index`.
self._actor_index: Optional[int] = None
# Initialize the variable that might store the list of actors as None.
# If the problem is configured to be parallelized and the parallelization is triggered, then this variable
# will store references to the remote actors (each remote actor storing its own copy of this Problem
# instance).
self._actors: Optional[list] = None
# Initialize the variable that might store the ray ActorPool.
# If the problem is configured to be parallelized and the parallelization is triggered, then this variable
# will store the ray ActorPool that is generated out of the remote actors.
self._actor_pool: Optional[ActorPool] = None
# Store the ray actor configuration dictionary provided by the user (if any).
# When (or if) the parallelization is triggered, each actor will be created with this given configuration.
self._actor_config: Optional[dict] = None if actor_config is None else deepcopy(dict(actor_config))
# If given, store the sub-batch size or number of sub-batches.
# When the problem is parallelized, a sub-batch size determines the maximum size for a SolutionBatch
# that will be sent to a remote actor for parallel solution evaluation.
# Alternatively, num_subbatches determines into how many pieces will a SolutionBatch be split
# for parallelization.
# If both are None, then the main SolutionBatch will be split among the actors.
if (num_subbatches is not None) and (subbatch_size is not None):
raise ValueError(
f"Encountered both `num_subbatches` and `subbatch_size` as values other than None."
f" num_subbatches={num_subbatches}, subbatch_size={subbatch_size}."
f" Having both of them as values other than None cannot be accepted."
)
self._num_subbatches: Optional[int] = None if num_subbatches is None else int(num_subbatches)
self._subbatch_size: Optional[int] = None if subbatch_size is None else int(subbatch_size)
# Initialize the additional states to be loaded by the remote actor as None.
# If there are such additional states for remote actors, the inheriting class can fill this as a list
# of dictionaries.
self._remote_states: Optional[Iterable[dict]] = None
# Initialize a temporary internal variable which stores the resources available in the ray cluster.
# Most probably, we are interested in the resources "CPU" and "GPU".
ray_resources: Optional[dict] = None
# The following is an internal helper function which returns the amount of availability for a given
# resource in the ray cluster.
# If the requested resource is not available at all, None will be returned.
def get_ray_resource(resource_name: str) -> Any:
# Ensure that the ray cluster is initialized
ensure_ray()
nonlocal ray_resources
if ray_resources is None:
# If the ray resource information was not fetched, then fetch them and store them.
ray_resources = ray.available_resources()
# Return the information regarding the requested resource from the fetched resource information.
# If it turns out that the requested resource is not available at all, the result will be None.
return ray_resources.get(resource_name, None)
# Annotate the variable that will store the number of actors (to be created when the parallelization
# is triggered).
self._num_actors: int
if num_actors is None:
# If the argument `num_actors` is left as None, then we set `_num_actors` as 0, which means that
# there will be no parallelization.
self._num_actors = 0
elif isinstance(num_actors, str):
# This is the case where `num_actors` has a string value
if num_actors in ("max", "num_cpus"):
# If the `num_actors` argument was given as "max" or as "num_cpus", then we first read how many CPUs
# are available in the ray cluster, then convert it to integer (via computing its ceil value), and
# finally set `_num_actors` as this integer.
self._num_actors = math.ceil(get_ray_resource("CPU"))
elif num_actors == "num_gpus":
# If the `num_actors` argument was given as "num_gpus", then we first read how many GPUs are
# available in the ray cluster.
num_gpus = get_ray_resource("GPU")
if num_gpus is None:
# If there are no GPUs at all, then we raise an error
raise ValueError(
"The argument `num_actors` was encountered as 'num_gpus'."
" However, there does not seem to be any GPU available."
)
if num_gpus < 1e-4:
# If the number of available GPUs are 0 or close to 0, then we raise an error
raise ValueError(
f"The argument `num_actors` was encountered as 'num_gpus'."
f" However, the number of available GPUs are either 0 or close to 0 (= {num_gpus})."
)
if (actor_config is not None) and ("num_gpus" in actor_config):
# With `num_actors` argument given as "num_gpus", we will also allocate each GPU to an actor.
# If `actor_config` contains an item with key "num_gpus", then that configuration item would
# conflict with the GPU allocation we are about to do here.
# So, we raise an error.
raise ValueError(
"The argument `num_actors` was encountered as 'num_gpus'."
" With this configuration, the number of GPUs assigned to an actor is automatically determined."
" However, at the same time, the `actor_config` argument was received with the key 'num_gpus',"
" which causes a conflict."
)
if num_gpus_per_actor is not None:
# With `num_actors` argument given as "num_gpus", we will also allocate each GPU to an actor.
# If the argument `num_gpus_per_actor` is also stated, then such a configuration item would
# conflict with the GPU allocation we are about to do here.
# So, we raise an error.
raise ValueError(
f"The argument `num_actors` was encountered as 'num_gpus'."
f" With this configuration, the number of GPUs assigned to an actor is automatically determined."
f" However, at the same time, the `num_gpus_per_actor` argument was received with a value other"
f" than None ({repr(num_gpus_per_actor)}), which causes a conflict."
)
# Set the number of actors as the ceiled integer counterpart of the number of available GPUs
self._num_actors = math.ceil(num_gpus)
# We assign a GPU for each actor (by overriding the value for the argument `num_gpus_per_actor`).
num_gpus_per_actor = num_gpus / self._num_actors
elif num_actors == "num_devices":
# This is the case where `num_actors` has the string value "num_devices".
# With `num_actors` set as "num_devices", if there are any GPUs, the behavior is to assign a GPU
# to each actor. If there are conflicting configurations regarding how many GPUs are to be assigned
# to each actor, then we raise an error.
if (actor_config is not None) and ("num_gpus" in actor_config):
raise ValueError(
"The argument `num_actors` was encountered as 'num_devices'."
" With this configuration, the number of GPUs assigned to an actor is automatically determined."
" However, at the same time, the `actor_config` argument was received with the key 'num_gpus',"
" which causes a conflict."
)
if num_gpus_per_actor is not None:
raise ValueError(
f"The argument `num_actors` was encountered as 'num_devices'."
f" With this configuration, the number of GPUs assigned to an actor is automatically determined."
f" However, at the same time, the `num_gpus_per_actor` argument was received with a value other"
f" than None ({repr(num_gpus_per_actor)}), which causes a conflict."
)
if self._device != torch.device("cpu"):
# If the main device is not CPU, then the user most probably wishes to put all the
# computations (both evaluations and the population) on the GPU, without allocating
# any actor.
# So, we set `_num_actors` as None, and overwrite `num_gpus_per_actor` with None.
self._num_actors = None
num_gpus_per_actor = None
else:
# If the device argument is "cpu" or left as None, then we assume that actor allocations
# might be desired.
# Read how many CPUs and GPUs are available in the ray cluster.
num_cpus = get_ray_resource("CPU")
num_gpus = get_ray_resource("GPU")
# If we have multiple CPUs, then we continue with the actor allocation procedures.
if (num_gpus is None) or (num_gpus < 1e-4):
# If there are no GPUs, then we set the number of actors as the number of CPUs, and we
# set the number of GPUs per actor as None (which means that there will be no GPU
# assignment)
self._num_actors = math.ceil(num_cpus)
num_gpus_per_actor = None
else:
# If there are GPUs available, then we compute the minimum among the number of CPUs and
# GPUs, and this minimum value becomes the number of actors (so that there can be
# one-to-one mapping between actors and GPUs).
self._num_actors = math.ceil(min(num_cpus, num_gpus))
# We assign a GPU for each actor (by overriding the value for the argument
# `num_gpus_per_actor`).
if self._num_actors <= num_gpus:
num_gpus_per_actor = 1
else:
num_gpus_per_actor = num_gpus / self._num_actors
else:
# This is the case where `num_actors` is given as an unexpected string. We raise an error here.
raise ValueError(
f"Invalid string value for `num_actors`: {repr(num_actors)}."
f" The acceptable string values for `num_actors` are 'max', 'num_cpus', 'num_gpus', 'num_devices'."
)
else:
# This is the case where `num_actors` has a value which is not a string.
# In this case, we make sure that the given value is an integer, and then use this integer as our
# number of actors.
self._num_actors = int(num_actors)
if self._num_actors == 1:
_evolog.info(
message_from(
self,
(
"The number of actors that will be allocated for parallelized evaluation was encountered as 1."
" This number is automatically dropped to 0,"
" because having only 1 actor does not bring any benefit in terms of parallelization."
),
)
)
# Creating a single actor does not bring any benefit of parallelization.
# Therefore, at the end of all the computations above regarding the number of actors, if it turns out
# that the target number of actors is 1, we reduce it to 0 (meaning that no actor will be initialized).
self._num_actors = 0
# Since we are to allocate no actor, the value of the argument `num_gpus_per_actor` is meaningless.
# We therefore overwrite the value of that argument with None.
num_gpus_per_actor = None
_evolog.info(
message_from(
self, f"The number of actors that will be allocated for parallelized evaluation is {self._num_actors}"
)
)
if (self._num_actors >= 2) and (self._device != torch.device("cpu")):
detailed_error_msg = (
f"The number of actors that will be allocated for parallelized evaluation is {self._num_actors}."
" When the number of actors is at least 2,"
' the only supported value for the `device` argument is "cpu".'
f" However, `device` was received as {self._device}."
"\n\n---- Possible ways to fix the error: ----"
"\n\n"
"(1)"
" If both the population and the fitness evaluation operations can fit into the same device,"
f" try setting `device={self._device}` and `num_actors=0`."
"\n\n"
"(2)"
" If you would like to use N number of GPUs in parallel for fitness evaluation (where N>1),"
' set `device="cpu"` (so that the main process will keep the population on the cpu), set'
" `num_actors=N` and `num_gpus_per_actor=1` (to allocate an actor for each of the `N` GPUs),"
" and then, decorate your fitness function using `evotorch.decorators.on_cuda`"
" so that the fitness evaluation will be performed on the cuda device assigned to the actor."
" The code for achieving this can look like this:"
"\n\n"
" from evotorch import Problem\n"
" from evotorch.decorators import on_cuda, vectorized\n"
" import torch\n"
"\n"
" @on_cuda\n"
" @vectorized\n"
" def f(x: torch.Tensor) -> torch.Tensor:\n"
" ...\n"
"\n"
' problem = Problem("min", f, device="cpu", num_actors=N, num_gpus_per_actor=1)\n'
"\n"
"Or, it can look like this:\n"
"\n"
" from evotorch import Problem, SolutionBatch\n"
" from evotorch.decorators import on_cuda\n"
" import torch\n"
"\n"
" class MyProblem(Problem):\n"
" def __init__(self, ...):\n"
" super().__init__(\n"
' objective_sense="min", device="cpu", num_actors=N, num_gpus_per_actor=1, ...\n'
" )\n"
"\n"
" @on_cuda\n"
" def _evaluate_batch(self, batch: SolutionBatch):\n"
" ...\n"
"\n"
" problem = MyProblem(...)\n"
"\n"
"\n"
"(3)"
" Similarly to option (2), for when you wish to use N number of GPUs for fitness evaluation,"
' set `device="cpu"`, set `num_actors=N` and `num_gpus_per_actor=1`, then, within the evaluation'
' function, manually use the device `"cuda"` to accelerate the computation.'
"\n\n"
"--------------\n"
"Note for cases (2) and (3): if you are on a computer or on a ray cluster with multiple GPUs, you"
' might prefer to set `num_actors` as the string "num_gpus" instead of an integer N,'
" which will cause the number of available GPUs to be counted, and the number of actors to be"
" configured as that count."
)
raise ValueError(detailed_error_msg)
# Annotate the variable which will determine how many GPUs are to be assigned to each actor.
self._num_gpus_per_actor: Optional[Union[str, int, float]]
if (actor_config is not None) and ("num_gpus" in actor_config) and (num_gpus_per_actor is not None):
# If `actor_config` dictionary has the item "num_gpus" and also `num_gpus_per_actor` is not None,
# then there is a conflicting (or redundant) configuration. We raise an error here.
raise ValueError(
'The `actor_config` dictionary contains the key "num_gpus".'
" At the same time, `num_gpus_per_actor` has a value other than None."
" These two configurations are conflicting."
" Please specify the number of GPUs per actor either via the `actor_config` dictionary,"
" or via the `num_gpus_per_actor` argument, but not via both."
)
if num_gpus_per_actor is None:
# If the argument `num_gpus_per_actor` is not specified, then we set the attribute
# `_num_gpus_per_actor` as None, which means that no GPUs will be assigned to the actors.
self._num_gpus_per_actor = None
elif isinstance(num_gpus_per_actor, str):
# This is the case where `num_gpus_per_actor` is given as a string.
if num_gpus_per_actor == "max":
# This is the case where `num_gpus_per_actor` is given as "max".
num_gpus = get_ray_resource("GPU")
if num_gpus is None:
# With `num_gpus_per_actor` as "max", if there is no GPU available, then we set the attribute
# `_num_gpus_per_actor` as None, which means there will be no GPU assignment to the actors.
self._num_gpus_per_actor = None
else:
# With `num_gpus_per_actor` as "max", if there are GPUs available, then the available GPUs will
# be shared among the actors.
self._num_gpus_per_actor = num_gpus / self._num_actors
elif num_gpus_per_actor == "all":
# When `num_gpus_per_actor` is "all", we also set the attribute `_num_gpus_per_actor` as "all".
# When a remote actor is initialized, the remote actor will see that the Problem instance has its
# `_num_gpus_per_actor` set as "all", and it will remove the environment variable named
# "CUDA_VISIBLE_DEVICES" in its own environment.
# With "CUDA_VISIBLE_DEVICES" removed, an actor will see all the GPUs available in its own
# environment.
self._num_gpus_per_actor = "all"
else:
# This is the case where `num_gpus_per_actor` argument has an unexpected string value.
# We raise an error.
raise ValueError(
f"Invalid string value for `num_gpus_per_actor`: {repr(num_gpus_per_actor)}."
f' Acceptable string values for `num_gpus_per_actor` are: "max", "all".'
)
elif isinstance(num_gpus_per_actor, int):
# When the argument `num_gpus_per_actor` is set as an integer we just set the attribute
# `_num_gpus_per_actor` as this integer.
self._num_gpus_per_actor = num_gpus_per_actor
else:
# For anything else, we assume that `num_gpus_per_actor` is an object that is convertible to float.
# Therefore, we convert it to float and store it in the attribute `_num_gpus_per_actor`.
# Also, remember that, when `num_actors` is given as "num_gpus" or as "num_devices",
# the code above overrides the value for the argument `num_gpus_per_actor`, which means,
# this is the case that is activated when `num_actors` is "num_gpus" or "num_devices".
self._num_gpus_per_actor = float(num_gpus_per_actor)
if self._num_actors > 0:
_evolog.info(
message_from(self, f"Number of GPUs that will be allocated per actor is {self._num_gpus_per_actor}")
)
# Initialize the Hook instances (and the related status dictionary for the `_after_eval_hook`)
self._before_eval_hook: Hook = Hook()
self._after_eval_hook: Hook = Hook()
self._after_eval_status: dict = {}
self._remote_hook: Hook = Hook()
self._before_grad_hook: Hook = Hook()
self._after_grad_hook: Hook = Hook()
# Initialize various stats regarding the solutions encountered by this Problem instance.
self._store_solution_stats = None if store_solution_stats is None else bool(store_solution_stats)
self._best: Optional[list] = None
self._worst: Optional[list] = None
self._best_evals: Optional[torch.Tensor] = None
self._worst_evals: Optional[torch.Tensor] = None
# Initialize the boolean attribute which indicates whether or not this Problem instance (which can be
# the main instance or a remote instance on an actor) is "prepared" via the `_prepare` method.
self._prepared: bool = False
all_remote_envs(self)
¶
Get an accessor which is used for running a method on all remote reinforcement learning environments.
This method can only be used on parallelized Problem
objects which have their get_env() methods defined.
For example, one can use this feature on a parallelized
GymProblem.
As an example, let us consider a parallelized GymProblem
object named my_problem. Given that my_problem has
n remote actors, a method f() can be executed
on all remote reinforcement learning environments as
follows:
results = my_problem.all_remote_envs().f()
The variable results is a list of length n, the i-th
item of the list belonging to the method f's result
from the i-th actor.
Returns:
| Type | Description |
|---|---|
AllRemoteEnvs |
A method accessor for all the remote reinforcement learning environments. |
Source code in evotorch/core.py
def all_remote_envs(self) -> AllRemoteEnvs:
"""
Get an accessor which is used for running a method
on all remote reinforcement learning environments.
This method can only be used on parallelized Problem
objects which have their `get_env()` methods defined.
For example, one can use this feature on a parallelized
GymProblem.
As an example, let us consider a parallelized GymProblem
object named `my_problem`. Given that `my_problem` has
`n` remote actors, a method `f()` can be executed
on all remote reinforcement learning environments as
follows:
results = my_problem.all_remote_envs().f()
The variable `results` is a list of length `n`, the i-th
item of the list belonging to the method f's result
from the i-th actor.
Returns:
A method accessor for all the remote reinforcement
learning environments.
"""
self._parallelize()
if self.is_remote:
raise RuntimeError(
"The method `all_remote_envs()` can only be used on the main (i.e. non-remote)"
" Problem instance."
" However, this Problem instance is on a remote actor."
)
return AllRemoteEnvs(self._actors)
all_remote_problems(self)
¶
Get an accessor which is used for running a method on all remote clones of this Problem object.
For example, given a Problem object named my_problem,
also assuming that this Problem object is parallelized,
and therefore has n remote actors, a method f()
can be executed on all the remote instances as follows:
results = my_problem.all_remote_problems().f()
The variable results is a list of length n, the i-th
item of the list belonging to the method f's result
from the i-th actor.
Returns:
| Type | Description |
|---|---|
AllRemoteProblems |
A method accessor for all the remote Problem objects. |
Source code in evotorch/core.py
def all_remote_problems(self) -> AllRemoteProblems:
"""
Get an accessor which is used for running a method
on all remote clones of this Problem object.
For example, given a Problem object named `my_problem`,
also assuming that this Problem object is parallelized,
and therefore has `n` remote actors, a method `f()`
can be executed on all the remote instances as follows:
results = my_problem.all_remote_problems().f()
The variable `results` is a list of length `n`, the i-th
item of the list belonging to the method f's result
from the i-th actor.
Returns:
A method accessor for all the remote Problem objects.
"""
self._parallelize()
if self.is_remote:
raise RuntimeError(
"The method `all_remote_problems()` can only be used on the main (i.e. non-remote)"
" Problem instance."
" However, this Problem instance is on a remote actor."
)
return AllRemoteProblems(self._actors)
compare_solutions(self, a, b, obj_index=None)
¶
Compare two solutions. It is assumed that both solutions are already evaluated.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
a |
Solution |
The first solution. |
required |
b |
Solution |
The second solution. |
required |
obj_index |
Optional[int] |
The objective index according to which the comparison will be made. Can be left as None if the problem is single-objective. |
None |
Returns:
| Type | Description |
|---|---|
float |
A positive number if |
Source code in evotorch/core.py
def compare_solutions(self, a: "Solution", b: "Solution", obj_index: Optional[int] = None) -> float:
"""
Compare two solutions.
It is assumed that both solutions are already evaluated.
Args:
a: The first solution.
b: The second solution.
obj_index: The objective index according to which the comparison
will be made.
Can be left as None if the problem is single-objective.
Returns:
A positive number if `a` is better;
a negative number if `b` is better;
0 if there is a tie.
"""
senses = self.senses
obj_index = self.normalize_obj_index(obj_index)
sense = senses[obj_index]
def score(s: Solution):
return s.evals[obj_index]
if sense == "max":
return score(a) - score(b)
elif sense == "min":
return score(b) - score(a)
else:
raise ValueError("Unrecognized sense: " + repr(sense))
ensure_numeric(self)
¶
Ensure that the problem has a numeric dtype.
Exceptions:
| Type | Description |
|---|---|
ValueError |
if the problem has a non-numeric dtype. |
ensure_single_objective(self)
¶
Ensure that the problem has only one objective.
Exceptions:
| Type | Description |
|---|---|
ValueError |
if the problem is multi-objective. |
Source code in evotorch/core.py
ensure_unbounded(self)
¶
Ensure that the problem has no strict lower and upper bounds.
Exceptions:
| Type | Description |
|---|---|
ValueError |
if the problem has strict lower and upper bounds. |
Source code in evotorch/core.py
def ensure_unbounded(self):
"""
Ensure that the problem has no strict lower and upper bounds.
Raises:
ValueError: if the problem has strict lower and upper bounds.
"""
if not (self.lower_bounds is None and self.upper_bounds is None):
raise ValueError("Expected an unbounded problem, but this problem has lower and/or upper bounds.")
evaluate(self, x)
¶
Evaluate the given Solution or SolutionBatch.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Union[SolutionBatch, Solution] |
The SolutionBatch to be evaluated. |
required |
Source code in evotorch/core.py
def evaluate(self, x: Union["SolutionBatch", "Solution"]):
"""
Evaluate the given Solution or SolutionBatch.
Args:
x: The SolutionBatch to be evaluated.
"""
if isinstance(x, Solution):
batch = x.to_batch()
elif isinstance(x, SolutionBatch):
batch = x
else:
raise TypeError(
f"The method `evaluate(...)` expected a Solution or a SolutionBatch as its argument."
f" However, the received object is {repr(x)}, which is of type {repr(type(x))}."
)
self._parallelize()
if self.is_main:
self.before_eval_hook(batch)
must_sync_after = self._sync_before()
self._start_preparations()
self._evaluate_all(batch)
if must_sync_after:
self._sync_after()
if self.is_main:
self._after_eval_status = {}
best_and_worst = self._get_best_and_worst(batch)
if best_and_worst is not None:
self._after_eval_status.update(best_and_worst)
self._after_eval_status.update(self.after_eval_hook.accumulate_dict(batch))
generate_batch(self, popsize=None, *, empty=False, center=None, stdev=None, symmetric=False)
¶
Generate a new SolutionBatch.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
popsize |
Optional[int] |
Number of solutions that will be contained in the new batch. |
None |
empty |
bool |
Set this as True if you would like to receive the solutions un-initialized. |
False |
center |
Union[float, Iterable[float], torch.Tensor] |
Center point of the Gaussian distribution from which
the decision values will be sampled, as a scalar or as a
1-dimensional vector.
Can also be left as None.
If |
None |
stdev |
Union[float, Iterable[float], torch.Tensor] |
Can be None (default) if the SolutionBatch is to contain
decision values sampled from the interval specified by
|
None |
symmetric |
bool |
To be used only when |
False |
Source code in evotorch/core.py
def generate_batch(
self,
popsize: Optional[int] = None,
*,
empty: bool = False,
center: Optional[RealOrVector] = None,
stdev: Optional[RealOrVector] = None,
symmetric: bool = False,
) -> "SolutionBatch":
"""
Generate a new SolutionBatch.
Args:
popsize: Number of solutions that will be contained in the new
batch.
empty: Set this as True if you would like to receive the solutions
un-initialized.
center: Center point of the Gaussian distribution from which
the decision values will be sampled, as a scalar or as a
1-dimensional vector.
Can also be left as None.
If `center` is None and `stdev` is None, all the decision
values will be sampled from the interval specified by
`initial_bounds` (or by `bounds` if `initial_bounds` was not
specified).
If `center` is None and `stdev` is not None, a center point
will be sampled from within the interval specified by
`initial_bounds` or `bounds`, and the decision values will be
sampled from a Gaussian distribution around this center point.
stdev: Can be None (default) if the SolutionBatch is to contain
decision values sampled from the interval specified by
`initial_bounds` (or by `bounds` if `initial_bounds` was not
provided during the initialization phase).
Alternatively, a scalar or a 1-dimensional vector specifying
the standard deviation of the Gaussian distribution from which
the decision values will be sampled.
symmetric: To be used only when `stdev` is not None.
If `symmetric` is True, decision values will be sampled from
the Gaussian distribution in a symmetric (i.e. antithetic)
manner.
Otherwise, the decision values will be sampled in the
non-antithetic manner.
"""
if (center is None) and (stdev is None):
if symmetric:
raise ValueError(
f"The argument `symmetric` can be set as True only when `center` and `stdev` are provided."
f" Although `center` and `stdev` are None, `symmetric` was received as {symmetric}."
)
return SolutionBatch(self, popsize, empty=empty, device=self.device)
elif (center is not None) and (stdev is not None):
if empty:
raise ValueError(
f"When `center` and `stdev` are provided, the argument `empty` must be False."
f" However, the received value for `empty` is {empty}."
)
result = SolutionBatch(self, popsize, device=self.device, empty=True)
self.make_gaussian(out=result.access_values(), center=center, stdev=stdev, symmetric=symmetric)
return result
else:
raise ValueError(
f"The arguments `center` and `stdev` were expected to be None or non-None at the same time."
f" Received `center`: {center}."
f" Received `stdev`: {stdev}."
)
generate_values(self, num_solutions)
¶
Generate decision values.
This function returns a tensor containing the decision values
for n new solutions, n being the integer passed as the num_rows
argument.
For numeric problems, this function generates the decision values
which respect initial_bounds (or bounds, if initial_bounds
was not provided).
If this type of initialization is not desired, one can override
this function and define a manual initialization scheme in the
inheriting subclass.
For non-numeric problems, it is expected that the inheriting subclass
will override the method _fill(...).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
num_solutions |
int |
For how many solutions will new decision values be generated. |
required |
Returns:
| Type | Description |
|---|---|
Union[torch.Tensor, evotorch.tools.objectarray.ObjectArray] |
A PyTorch tensor for numeric problems, an ObjectArray for non-numeric problems. |
Source code in evotorch/core.py
def generate_values(self, num_solutions: int) -> Union[torch.Tensor, ObjectArray]:
"""
Generate decision values.
This function returns a tensor containing the decision values
for `n` new solutions, `n` being the integer passed as the `num_rows`
argument.
For numeric problems, this function generates the decision values
which respect `initial_bounds` (or `bounds`, if `initial_bounds`
was not provided).
If this type of initialization is not desired, one can override
this function and define a manual initialization scheme in the
inheriting subclass.
For non-numeric problems, it is expected that the inheriting subclass
will override the method `_fill(...)`.
Args:
num_solutions: For how many solutions will new decision values be
generated.
Returns:
A PyTorch tensor for numeric problems, an ObjectArray for
non-numeric problems.
"""
if self.dtype is object:
result = self.make_empty(num_solutions=num_solutions)
else:
result = torch.empty(tuple(), dtype=self.dtype, device=self.device)
result = result.expand(num_solutions, self.solution_length)
result = result + make_batched_false_for_vmap(result.device)
self._fill(result)
return result
get_obj_order_descending(self)
¶
When sorting the solutions from best to worst according to each objective i, is the ordering descending?
Source code in evotorch/core.py
def get_obj_order_descending(self) -> Iterable[bool]:
"""When sorting the solutions from best to worst according to each objective i, is the ordering descending?"""
result = []
for s in self.senses:
if s == "min":
result.append(False)
elif s == "max":
result.append(True)
else:
raise ValueError(f"Invalid sense: {repr(s)}")
return result
is_better(self, a, b, obj_index=None)
¶
Check whether or not the first solution is better. It is assumed that both solutions are already evaluated.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
a |
Solution |
The first solution. |
required |
b |
Solution |
The second solution. |
required |
obj_index |
Optional[int] |
The objective index according to which the comparison will be made. Can be left as None if the problem is single-objective. |
None |
Returns:
| Type | Description |
|---|---|
bool |
True if |
Source code in evotorch/core.py
def is_better(self, a: "Solution", b: "Solution", obj_index: Optional[int] = None) -> bool:
"""
Check whether or not the first solution is better.
It is assumed that both solutions are already evaluated.
Args:
a: The first solution.
b: The second solution.
obj_index: The objective index according to which the comparison
will be made.
Can be left as None if the problem is single-objective.
Returns:
True if `a` is better; False otherwise.
"""
return self.compare_solutions(a, b, obj_index) > 0
is_on_cpu(self)
¶
is_worse(self, a, b, obj_index=None)
¶
Check whether or not the first solution is worse. It is assumed that both solutions are already evaluated.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
a |
Solution |
The first solution. |
required |
b |
Solution |
The second solution. |
required |
obj_index |
Optional[int] |
The objective index according to which the comparison will be made. Can be left as None if the problem is single-objective. |
None |
Returns:
| Type | Description |
|---|---|
bool |
True if |
Source code in evotorch/core.py
def is_worse(self, a: "Solution", b: "Solution", obj_index: Optional[int] = None) -> bool:
"""
Check whether or not the first solution is worse.
It is assumed that both solutions are already evaluated.
Args:
a: The first solution.
b: The second solution.
obj_index: The objective index according to which the comparison
will be made.
Can be left as None if the problem is single-objective.
Returns:
True if `a` is worse; False otherwise.
"""
return self.compare_solutions(a, b, obj_index) < 0
kill_actors(self)
¶
Kill all the remote actors used by the Problem instance.
One might use this method to release the resources used by the remote actors.
Source code in evotorch/core.py
def kill_actors(self):
"""
Kill all the remote actors used by the Problem instance.
One might use this method to release the resources used by the
remote actors.
"""
if not self.is_main:
raise RuntimeError(
"The method `kill_actors()` can only be used on the main (i.e. non-remote)"
" Problem instance."
" However, this Problem instance is on a remote actor."
)
for actor in self._actors:
ray.kill(actor)
self._actors = None
self._actor_pool = None
make_callable_evaluator(self, *, obj_index=None)
¶
Get a callable evaluator for evaluating the given solutions.
Let us assume that we have a Problem declared like this:
Using the regular API of EvoTorch, one has to generate solutions for this problem as follows:
For editing the decision values within the
SolutionBatch my_solutions, one has to
do the following:
Finally, to evaluate my_solutions, one would have to do these:
One could desire a different interface which is more compatible with functional programming paradigm, especially when planning to use the functional algorithms (such as, for example, the functional cem). To achieve this, one can do the following:
Now, we have a new object f, which behaves like a function.
This function-like object expects a tensor of decision values, and
returns fitnesses, as shown below:
random_decision_values = torch.randn(
population_size,
my_problem.solution_length,
dtype=my_problem.dtype,
)
fitnesses = f(random_decision_values)
Parallelized fitness evaluation.
If a Problem object is condifured to use parallelized evaluation with
the help of multiple actors, a callable evaluator made out of that
Problem object will also make use of those multiple actors.
Additional batch dimensions. If a callable evaluator receives a tensor with 3 or more dimensions, those extra leftmost dimensions will be considered as batch dimensions. The returned fitness tensor will also preserve those batch dimensions.
Notes on vmap.
ProblemBoundEvaluator is a shallow wrapper around a Problem object.
It does NOT transform the underlying problem object to its stateless
counterpart, and therefore it does NOT conform to pure functional
programming paradigm. Being stateful, it will NOT work correctly with
vmap. For batched evaluations, it is recommended to use extra batch
dimensions, instead of using vmap.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
obj_index |
Optional[int] |
The index of the objective according to which the evaluations will be done. If the problem is single-objective, this is not required. If the problem is multi-objective, this needs to be given as an integer. |
None |
Returns:
| Type | Description |
|---|---|
ProblemBoundEvaluator |
A callable fitness evaluator, bound to this problem object. |
Source code in evotorch/core.py
def make_callable_evaluator(self, *, obj_index: Optional[int] = None) -> "ProblemBoundEvaluator":
"""
Get a callable evaluator for evaluating the given solutions.
Let us assume that we have a [Problem][evotorch.core.Problem]
declared like this:
```python
from evotorch import Problem
my_problem = Problem(
"min",
fitness_function_goes_here,
...,
)
```
Using the regular API of EvoTorch, one has to generate solutions for this
problem as follows:
```python
population_size = ...
my_solutions = my_problem.generate_batch(population_size)
```
For editing the decision values within the
[SolutionBatch][evotorch.core.SolutionBatch] `my_solutions`, one has to
do the following:
```python
new_decision_values = ...
my_solutions.set_values(new_decision_values)
```
Finally, to evaluate `my_solutions`, one would have to do these:
```python
my_problem.evaluate(my_solutions)
fitnesses = my_problem.evals
```
One could desire a different interface which is more compatible with
functional programming paradigm, especially when planning to use the
functional algorithms (such as, for example, the functional
[cem][evotorch.algorithms.functional.funccem.cem]).
To achieve this, one can do the following:
```python
f = my_problem.make_callable_evaluator()
```
Now, we have a new object `f`, which behaves like a function.
This function-like object expects a tensor of decision values, and
returns fitnesses, as shown below:
```python
random_decision_values = torch.randn(
population_size,
my_problem.solution_length,
dtype=my_problem.dtype,
)
fitnesses = f(random_decision_values)
```
**Parallelized fitness evaluation.**
If a `Problem` object is condifured to use parallelized evaluation with
the help of multiple actors, a callable evaluator made out of that
`Problem` object will also make use of those multiple actors.
**Additional batch dimensions.**
If a callable evaluator receives a tensor with 3 or more dimensions,
those extra leftmost dimensions will be considered as batch
dimensions. The returned fitness tensor will also preserve those batch
dimensions.
**Notes on vmap.**
`ProblemBoundEvaluator` is a shallow wrapper around a `Problem` object.
It does NOT transform the underlying problem object to its stateless
counterpart, and therefore it does NOT conform to pure functional
programming paradigm. Being stateful, it will NOT work correctly with
`vmap`. For batched evaluations, it is recommended to use extra batch
dimensions, instead of using `vmap`.
Args:
obj_index: The index of the objective according to which the
evaluations will be done. If the problem is single-objective,
this is not required. If the problem is multi-objective, this
needs to be given as an integer.
Returns:
A callable fitness evaluator, bound to this problem object.
"""
return ProblemBoundEvaluator(self, obj_index=obj_index)
manual_seed(self, seed=None)
¶
Provide a manual seed for the Problem object.
If the given seed is None, then the Problem object will remove its own stored generator, and start using the global generator of PyTorch instead. If the given seed is an integer, then the Problem object will instantiate its own generator with the given seed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
seed |
Optional[int] |
None for using the global PyTorch generator; an integer for instantiating a new PyTorch generator with this given integer seed, specific to this Problem object. |
None |
Source code in evotorch/core.py
def manual_seed(self, seed: Optional[int] = None):
"""
Provide a manual seed for the Problem object.
If the given seed is None, then the Problem object will remove
its own stored generator, and start using the global generator
of PyTorch instead.
If the given seed is an integer, then the Problem object will
instantiate its own generator with the given seed.
Args:
seed: None for using the global PyTorch generator; an integer
for instantiating a new PyTorch generator with this given
integer seed, specific to this Problem object.
"""
if seed is None:
self._generator = None
else:
if self._generator is None:
self._generator = torch.Generator(device=self.device)
self._generator.manual_seed(seed)
normalize_obj_index(self, obj_index=None)
¶
Normalize the objective index.
If the provided index is non-negative, it is ensured that the index is valid.
If the provided index is negative, the objectives are counted in the reverse order, and the corresponding non-negative index is returned. For example, -1 is converted to a non-negative integer corresponding to the last objective.
If the provided index is None and if the problem is single-objective, the returned value is 0, which represents the only objective.
If the provided index is None and if the problem is multi-objective, an error is raised.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
obj_index |
Optional[int] |
The non-normalized objective index. |
None |
Returns:
| Type | Description |
|---|---|
int |
The normalized objective index, as a non-negative integer. |
Source code in evotorch/core.py
def normalize_obj_index(self, obj_index: Optional[int] = None) -> int:
"""
Normalize the objective index.
If the provided index is non-negative, it is ensured that the index
is valid.
If the provided index is negative, the objectives are counted in the
reverse order, and the corresponding non-negative index is returned.
For example, -1 is converted to a non-negative integer corresponding to
the last objective.
If the provided index is None and if the problem is single-objective,
the returned value is 0, which represents the only objective.
If the provided index is None and if the problem is multi-objective,
an error is raised.
Args:
obj_index: The non-normalized objective index.
Returns:
The normalized objective index, as a non-negative integer.
"""
if obj_index is None:
if len(self.senses) == 1:
return 0
else:
raise ValueError(
"This problem is multi-objective, therefore, an explicit objective index was expected."
" However, `obj_index` was found to be None."
)
else:
obj_index = int(obj_index)
if obj_index < 0:
obj_index = len(self.senses) + obj_index
if obj_index < 0 or obj_index >= len(self.senses):
raise IndexError("Objective index out of range.")
return obj_index
sample_and_compute_gradients(self, distribution, popsize, *, num_interactions=None, popsize_max=None, obj_index=None, ranking_method=None, with_stats=True, ensure_even_popsize=False)
¶
Sample new solutions from the distribution and compute gradients.
The distribution can then be updated according to the computed gradients.
If the problem is not parallelized, and with_stats is False,
then the result will be a single dictionary of gradients.
For example, in the case of a Gaussian distribution, the returned
gradients dictionary would look like this:
{
"mu": ..., # the gradient for the mean
"sigma": ..., # the gradient for the standard deviation
}
If the problem is not parallelized, and with_stats is True,
then the result will be a dictionary which contains in itself
the gradients dictionary, and additional elements for providing
further information. In the case of a Gaussian distribution,
the returned dictionary with additional stats would look like
this:
{
"gradients": {
"mu": ..., # the gradient for the mean
"sigma": ..., # the gradient for the standard deviation
},
"num_solutions": ..., # how many solutions were sampled
"mean_eval": ..., # Mean of all evaluations
}
If the problem is parallelized, then the gradient computation will
be distributed among the remote actors. In more details, each actor
will sample its own solutions (such that the total population size
across all remote actors will be near the provided popsize)
and will compute its own gradients, and will produce its own
additional stats (if with_stats is given as True).
These remote results will then be collected by the main process,
and the final result of this method will be a list of dictionaries,
each dictionary being the result of a remote gradient computation.
The sampled solutions are temporary, and will not be kept (and will not be returned).
To customize how solutions are sampled and how gradients are
computed, one is encouraged to override
_sample_and_compute_gradients(...) (instead of overriding this
method directly.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
distribution |
The search distribution from which the solutions will be sampled, and according to which the gradients will be computed. |
required | |
popsize |
int |
The number of solutions which will be sampled. |
required |
num_interactions |
Optional[int] |
Number of simulator interactions that must be completed (more solutions will be sampled until this threshold is reached). This argument is to be used when the problem has characteristics similar to reinforcement learning, and an adaptive population size, depending on the interactions made, is desired. Otherwise, one can leave this argument as None, in which case, there will not be any threshold based on number of interactions. |
None |
popsize_max |
Optional[int] |
To be used when |
None |
obj_index |
Optional[int] |
Index of the objective according to which the gradients will be computed. Can be left as None if the problem has only one objective. |
None |
ranking_method |
Optional[str] |
The solution ranking method to be used when computing the gradients. If not specified, the raw fitnesses will be used. |
None |
with_stats |
bool |
If given as False, then the results dictionary will only contain the gradients information. If given as True, then the results dictionary will contain within itself the gradients dictionary, and also additional elements for providing further information. The default is True. |
True |
ensure_even_popsize |
bool |
If |
False |
Returns:
| Type | Description |
|---|---|
Union[list, dict] |
A results dictionary when the problem is not parallelized, or list of results dictionaries when the problem is parallelized. |
Source code in evotorch/core.py
def sample_and_compute_gradients(
self,
distribution,
popsize: int,
*,
num_interactions: Optional[int] = None,
popsize_max: Optional[int] = None,
obj_index: Optional[int] = None,
ranking_method: Optional[str] = None,
with_stats: bool = True,
ensure_even_popsize: bool = False,
) -> Union[list, dict]:
"""
Sample new solutions from the distribution and compute gradients.
The distribution can then be updated according to the computed
gradients.
If the problem is not parallelized, and `with_stats` is False,
then the result will be a single dictionary of gradients.
For example, in the case of a Gaussian distribution, the returned
gradients dictionary would look like this:
{
"mu": ..., # the gradient for the mean
"sigma": ..., # the gradient for the standard deviation
}
If the problem is not parallelized, and `with_stats` is True,
then the result will be a dictionary which contains in itself
the gradients dictionary, and additional elements for providing
further information. In the case of a Gaussian distribution,
the returned dictionary with additional stats would look like
this:
{
"gradients": {
"mu": ..., # the gradient for the mean
"sigma": ..., # the gradient for the standard deviation
},
"num_solutions": ..., # how many solutions were sampled
"mean_eval": ..., # Mean of all evaluations
}
If the problem is parallelized, then the gradient computation will
be distributed among the remote actors. In more details, each actor
will sample its own solutions (such that the total population size
across all remote actors will be near the provided `popsize`)
and will compute its own gradients, and will produce its own
additional stats (if `with_stats` is given as True).
These remote results will then be collected by the main process,
and the final result of this method will be a list of dictionaries,
each dictionary being the result of a remote gradient computation.
The sampled solutions are temporary, and will not be kept
(and will not be returned).
To customize how solutions are sampled and how gradients are
computed, one is encouraged to override
`_sample_and_compute_gradients(...)` (instead of overriding this
method directly.
Args:
distribution: The search distribution from which the solutions
will be sampled, and according to which the gradients will
be computed.
popsize: The number of solutions which will be sampled.
num_interactions: Number of simulator interactions that must
be completed (more solutions will be sampled until this
threshold is reached). This argument is to be used when
the problem has characteristics similar to reinforcement
learning, and an adaptive population size, depending on
the interactions made, is desired.
Otherwise, one can leave this argument as None, in which
case, there will not be any threshold based on number
of interactions.
popsize_max: To be used when `num_interactions` is provided,
as an additional criterion for ending the solution sampling
phase. This argument can be used to prevent the population
size from growing too much while trying to satisfy the
`num_interactions`. If not needed, `popsize_max` can be left
as None.
obj_index: Index of the objective according to which the gradients
will be computed. Can be left as None if the problem has only
one objective.
ranking_method: The solution ranking method to be used when
computing the gradients.
If not specified, the raw fitnesses will be used.
with_stats: If given as False, then the results dictionary will
only contain the gradients information. If given as True,
then the results dictionary will contain within itself
the gradients dictionary, and also additional elements for
providing further information.
The default is True.
ensure_even_popsize: If `ensure_even_popsize` is True and the
problem is not parallelized, then a `popsize` given as an odd
number will cause an error. If `ensure_even_popsize` is True
and the problem is parallelized, then the remote actors will
sample their own sub-populations in such a way that their
sizes are even.
If `ensure_even_popsize` is False, whether or not the
`popsize` is even will not be checked.
When the provided `distribution` is a symmetric (or
"mirrored", or "antithetic"), then this argument must be
given as True.
Returns:
A results dictionary when the problem is not parallelized,
or list of results dictionaries when the problem is parallelized.
"""
# For problems which are configured for parallelization, make sure that the actors are created.
self._parallelize()
# Below we check if there is an inconsistency in arguments.
if (num_interactions is None) and (popsize_max is not None):
# If `num_interactions` is None, then we assume that the user does not wish an adaptive population size.
# However, at the same time, if `popsize_max` is not None, then there is an inconsistency,
# because, `popsize_max` without `num_interactions` (therefore without adaptive population size)
# does not make sense.
# This is probably a configuration error, so, we inform the user by raising an error.
raise ValueError(
f"`popsize_max` was expected as None, because `num_interactions` is None."
f" However, `popsize_max` was found as {popsize_max}."
)
# The problem instance in the main process should trigger the `before_grad_hook`.
if self.is_main:
self._before_grad_hook()
if self.is_main and (self._actors is not None) and (len(self._actors) > 0):
# If this is the main process and the problem is parallelized, then we need to split the request
# into multiple tasks, and then execute those tasks in parallel using the problem's actor pool.
if self._subbatch_size is not None:
# If `subbatch_size` is provided, then we first make sure that `popsize` is divisible by
# `subbatch_size`
if (popsize % self._subbatch_size) != 0:
raise ValueError(
f"This Problem was created with `subbatch_size` as {self._subbatch_size}."
f" When doing remote gradient computation, the requested population size must be divisible by"
f" the `subbatch_size`."
f" However, the requested population size is {popsize}, and the remainder after dividing it"
f" by `subbatch_size` is not 0 (it is {popsize % self._subbatch_size})."
)
# After making sure that `popsize` and `subbatch_size` configurations are compatible, we declare that
# we are going to have n tasks, each task imposing a sample size of `subbatch_size`.
n = int(popsize // self._subbatch_size)
popsize_per_task = [self._subbatch_size for _ in range(n)]
elif self._num_subbatches is not None:
# If `num_subbatches` is provided, then we are going to have n tasks where n is equal to the given
# `num_subbatches`.
popsize_per_task = split_workload(popsize, self._num_subbatches)
else:
# If neither `subbatch_size` nor `num_subbatches` is given, then we will split the workload in such
# a way that each actor will have its share.
popsize_per_task = split_workload(popsize, len(self._actors))
if ensure_even_popsize:
# If `ensure_even_popsize` argument is True, then we need to make sure that each tasks's popsize is
# an even number.
for i in range(len(popsize_per_task)):
if (popsize_per_task[i] % 2) != 0:
# If the i-th actor's assigned popsize is not even, increase its assigned popsize by 1.
popsize_per_task[i] += 1
# The number of tasks is finally determined by the length of `popsize_per_task` list we created above.
num_tasks = len(popsize_per_task)
if num_interactions is None:
# If the argument `num_interactions` is not given, then, for each task, we declare that
# `num_interactions` is None.
num_inter_per_task = [None for _ in range(num_tasks)]
else:
# If the argument `num_interactions` is given, then we compute each task's target number of
# interactions from its sample size.
num_inter_per_task = [
math.ceil((popsize_per_task[i] / popsize) * num_interactions) for i in range(num_tasks)
]
if popsize_max is None:
# If the argument `popsize_max` is not given, then, for each task, we declare that
# `popsize_max` is None.
popsize_max_per_task = [None for _ in range(num_tasks)]
else:
# If the argument `popsize_max` is given, then we compute each task's target maximum population size
# from its sample size.
popsize_max_per_task = [
math.ceil((popsize_per_task[i] / popsize) * popsize_max) for i in range(num_tasks)
]
# We trigger the synchronization between the main process and the remote actors.
# If this problem instance has nothing to synchronize, then `must_sync_after` will be False.
must_sync_after = self._sync_before()
# Because we want to send the distribution to remote actors, we first copy the distribution to cpu
# (unless it is already on cpu)
dist_on_cpu = distribution.to("cpu")
# Here, we use our actor pool to execute our tasks in parallel.
result = list(
self._actor_pool.map_unordered(
(
lambda a, v: a.call.remote(
"_sample_and_compute_gradients",
[dist_on_cpu, v[0]],
{
"obj_index": obj_index,
"num_interactions": v[1],
"popsize_max": v[2],
"ranking_method": ranking_method,
},
)
),
list(zip(popsize_per_task, num_inter_per_task, popsize_max_per_task)),
)
)
# At this point, all the tensors within our collected results are on the CPU.
if torch.device(self.device) != torch.device("cpu"):
# If the main device of this problem instance is not CPU, then we move the tensors to the main device.
result = cast_tensors_in_container(result, device=self.device)
if must_sync_after:
# If a post-gradient synchronization is required, we trigger the synchronization operations.
self._sync_after()
# ####################################################
# # If this is the main process and the problem is parallelized, then we need to split the workload among
# # the remote actors, and then request each of them to compute their gradients.
#
# # We begin by getting the number of actors, and computing the `popsize` for each actor.
# num_actors = len(self._actors)
# popsize_per_actor = split_workload(popsize, num_actors)
#
# if ensure_even_popsize:
# # If `ensure_even_popsize` argument is True, then we need to make sure that each actor's popsize is
# # an even number.
# for i in range(len(popsize_per_actor)):
# if (popsize_per_actor[i] % 2) != 0:
# # If the i-th actor's assigned popsize is not even, increase its assigned popsize by 1.
# popsize_per_actor[i] += 1
#
# if num_interactions is None:
# # If `num_interactions` is None, then the `num_interactions` argument for each actor must also be
# # passed as None.
# num_int_per_actor = [None] * num_actors
# else:
# # If `num_interactions` is not None, then we split the `num_interactions` workload among the actors.
# num_int_per_actor = split_workload(num_interactions, num_actors)
#
# if popsize_max is None:
# # If `popsize_max` is None, then the `popsize_max` argument for each actor must also be None.
# popsize_max_per_actor = [None] * num_actors
# else:
# # If `popsize_max` is not None, then we split the `popsize_max` workload among the actors.
# popsize_max_per_actor = split_workload(popsize_max, num_actors)
#
# # We trigger the synchronization between the main process and the remote actors.
# # If this problem instance has nothing to synchronize, then `must_sync_after` will be False.
# must_sync_after = self._sync_before()
#
# # Because we want to send the distribution to remote actors, we first copy the distribution to cpu
# # (unless it is already on cpu)
# dist_on_cpu = distribution.to("cpu")
#
# # To each actor, we send the request of computing the gradients, and then collect the results
# result = ray.get(
# [
# self._actors[i].call.remote(
# "_gradient_computation_helper",
# [dist_on_cpu, popsize_per_actor[i]],
# dict(
# num_interactions=num_int_per_actor[i],
# popsize_max=popsize_max_per_actor[i],
# obj_index=obj_index,
# ranking_method=ranking_method,
# with_stats=with_stats,
# move_results_to_device="cpu",
# ),
# )
# for i in range(num_actors)
# ]
# )
#
# # At this point, all the tensors within our collected results are on the CPU.
#
# if torch.device(self.device) != torch.device("cpu"):
# # If the main device of this problem instance is not CPU, then we move the tensors to the main device.
# result = cast_tensors_in_container(result, device=device)
#
# if must_sync_after:
# # If a post-gradient synchronization is required, we trigger the synchronization operations.
# self._sync_after()
else:
# If the problem is not parallelized, then we request this instance itself to compute the gradients.
result = self._gradient_computation_helper(
distribution,
popsize,
popsize_max=popsize_max,
obj_index=obj_index,
ranking_method=ranking_method,
num_interactions=num_interactions,
with_stats=with_stats,
)
# The problem instance in the main process should trigger the `after_grad_hook`.
if self.is_main:
self._after_eval_status = self._after_grad_hook.accumulate_dict(result)
# We finally return the results
return result
ProblemBoundEvaluator
¶
A callable fitness evaluator, bound to the given Problem.
A callable evaluator returned by the method
Problem.make_callable_evaluator is an instance of this class.
For details, please see the documentation of
Problem, and of its method
make_callable_evaluator.
Source code in evotorch/core.py
class ProblemBoundEvaluator:
"""
A callable fitness evaluator, bound to the given `Problem`.
A callable evaluator returned by the method
`Problem.make_callable_evaluator` is an instance of this class.
For details, please see the documentation of
[Problem][evotorch.core.Problem], and of its method
`make_callable_evaluator`.
"""
def __init__(self, problem: Problem, *, obj_index: Optional[int] = None):
"""
`__init__(...)`: Initialize the `ProblemBoundEvaluator`.
Args:
problem: The problem object to be wrapped.
obj_index: The objective index. Optional if the problem being
wrapped is single-objective. If the problem being wrapped
is multi-objective, this is expected as an integer.
"""
self._problem = problem
if not isinstance(self._problem, Problem):
clsname = type(self).__name__
raise TypeError(
f"In its initialization phase, {clsname} expected a `Problem` object,"
f" but found: {repr(self._problem)} (of type {repr(type(self._problem))})"
)
self._obj_index = self._problem.normalize_obj_index(obj_index)
self._problem.ensure_numeric()
if problem.dtype != problem.eval_dtype:
raise TypeError(
"The dtype of the decision values is not the same with the dtype of the evaluations."
" Currently, it is not supported to make callable evaluators out of problems whose"
" decision value dtypes are different than their evaluation dtypes."
)
def _make_empty_solution_batch(self, popsize: int) -> SolutionBatch:
return SolutionBatch(self._problem, popsize=popsize, empty=True, device="meta")
def _prepare_evaluated_solution_batch(self, values_2d: torch.Tensor) -> SolutionBatch:
num_solutions, solution_length = values_2d.shape
batch = self._make_empty_solution_batch(num_solutions)
batch._data = values_2d
batch._evdata = torch.empty_like(batch._evdata, device=values_2d.device)
self._problem.evaluate(batch)
return batch
def __call__(self, values: torch.Tensor) -> torch.Tensor:
"""
Evaluate the solutions expressed by the given `values` tensor.
Args:
values: Decision values. Expected as a tensor with at least
2 dimensions. If the number of dimensions is more than 2,
the extra leftmost dimensions will be considered as batch
dimensions.
Returns:
The fitnesses, as a tensor.
"""
ndim = values.ndim
if ndim == 0:
clsname = type(self).__name__
raise ValueError(
f"{clsname} was expecting a tensor with at least 1 dimension for solution evaluation."
f" However, it received a scalar: {values}"
)
solution_length = values.shape[-1]
original_batch_shape = values.shape[:-1]
values = values.reshape(-1, solution_length)
evaluated_batch = self._prepare_evaluated_solution_batch(values)
evals = evaluated_batch.evals[:, self._obj_index]
return evals.reshape(original_batch_shape).as_subclass(torch.Tensor)
__call__(self, values)
special
¶
Evaluate the solutions expressed by the given values tensor.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
values |
Tensor |
Decision values. Expected as a tensor with at least 2 dimensions. If the number of dimensions is more than 2, the extra leftmost dimensions will be considered as batch dimensions. |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The fitnesses, as a tensor. |
Source code in evotorch/core.py
def __call__(self, values: torch.Tensor) -> torch.Tensor:
"""
Evaluate the solutions expressed by the given `values` tensor.
Args:
values: Decision values. Expected as a tensor with at least
2 dimensions. If the number of dimensions is more than 2,
the extra leftmost dimensions will be considered as batch
dimensions.
Returns:
The fitnesses, as a tensor.
"""
ndim = values.ndim
if ndim == 0:
clsname = type(self).__name__
raise ValueError(
f"{clsname} was expecting a tensor with at least 1 dimension for solution evaluation."
f" However, it received a scalar: {values}"
)
solution_length = values.shape[-1]
original_batch_shape = values.shape[:-1]
values = values.reshape(-1, solution_length)
evaluated_batch = self._prepare_evaluated_solution_batch(values)
evals = evaluated_batch.evals[:, self._obj_index]
return evals.reshape(original_batch_shape).as_subclass(torch.Tensor)
__init__(self, problem, *, obj_index=None)
special
¶
__init__(...): Initialize the ProblemBoundEvaluator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object to be wrapped. |
required |
obj_index |
Optional[int] |
The objective index. Optional if the problem being wrapped is single-objective. If the problem being wrapped is multi-objective, this is expected as an integer. |
None |
Source code in evotorch/core.py
def __init__(self, problem: Problem, *, obj_index: Optional[int] = None):
"""
`__init__(...)`: Initialize the `ProblemBoundEvaluator`.
Args:
problem: The problem object to be wrapped.
obj_index: The objective index. Optional if the problem being
wrapped is single-objective. If the problem being wrapped
is multi-objective, this is expected as an integer.
"""
self._problem = problem
if not isinstance(self._problem, Problem):
clsname = type(self).__name__
raise TypeError(
f"In its initialization phase, {clsname} expected a `Problem` object,"
f" but found: {repr(self._problem)} (of type {repr(type(self._problem))})"
)
self._obj_index = self._problem.normalize_obj_index(obj_index)
self._problem.ensure_numeric()
if problem.dtype != problem.eval_dtype:
raise TypeError(
"The dtype of the decision values is not the same with the dtype of the evaluations."
" Currently, it is not supported to make callable evaluators out of problems whose"
" decision value dtypes are different than their evaluation dtypes."
)
RemoteMethod
¶
Representation of a method on a remote actor's contained Problem or reinforcement learning environment
Source code in evotorch/core.py
class RemoteMethod:
"""
Representation of a method on a remote actor's contained Problem
or reinforcement learning environment
"""
def __init__(self, method_name: str, actors: list, on_env: bool = False):
self._method_name = str(method_name)
self._actors = actors
self._on_env = bool(on_env)
def __call__(self, *args, **kwargs) -> Any:
def invoke(actor):
if self._on_env:
return actor.call_on_env.remote(self._method_name, args, kwargs)
else:
return actor.call.remote(self._method_name, args, kwargs)
return ray.get([invoke(actor) for actor in self._actors])
def __repr__(self) -> str:
if self._on_env:
further = ", on_env=True"
else:
further = ""
return f"<{type(self).__name__} {repr(self._method_name)}{further}>"
Solution (Serializable)
¶
Representation of a single Solution.
A Solution can be a reference to a row of a SolutionBatch (in which case it shares its storage with the SolutionBatch), or can be an independent solution. When the Solution shares its storage with a SolutionBatch, any modifications to its decision values and/or evaluation results will affect its parent SolutionBatch as well.
When a Solution object is cloned (via its clone() method,
or via the functions copy.copy(...) and copy.deepcopy(...),
a new independent Solution object will be created.
This new independent copy will NOT share its storage with
its original SolutionBatch anymore.
Source code in evotorch/core.py
class Solution(Serializable):
"""
Representation of a single Solution.
A Solution can be a reference to a row of a SolutionBatch
(in which case it shares its storage with the SolutionBatch),
or can be an independent solution.
When the Solution shares its storage with a SolutionBatch,
any modifications to its decision values and/or evaluation
results will affect its parent SolutionBatch as well.
When a Solution object is cloned (via its `clone()` method,
or via the functions `copy.copy(...)` and `copy.deepcopy(...)`,
a new independent Solution object will be created.
This new independent copy will NOT share its storage with
its original SolutionBatch anymore.
"""
def __init__(self, parent: SolutionBatch, index: int):
"""
`__init__(...)`: Initialize the Solution object.
Args:
parent: The parent SolutionBatch which stores the Solution.
index: Index of the solution in SolutionBatch.
"""
if not isinstance(parent, SolutionBatch):
raise TypeError(
f"Expected a SolutionBatch as a parent, but encountered {repr(parent)},"
f" which is of type {repr(type(parent))}."
)
index = int(index)
if index < 0:
index = len(parent) + index
if not ((index >= 0) and (index <= len(parent))):
raise IndexError(f"Invalid index: {index}")
self._batch: SolutionBatch = parent[index : index + 1]
def access_values(self, *, keep_evals: bool = False) -> torch.Tensor:
"""
Access the decision values tensor of the solution.
The received tensor will be mutable.
By default, it will be assumed that the user wishes to
obtain this tensor to change the decision values, and therefore,
the evaluation results associated with this solution will be
cleared (i.e. will be NaN).
Args:
keep_evals: When this is set to True, the evaluation results
associated with this solution will be kept (i.e. will NOT
be cleared).
Returns:
The decision values tensor of the solution.
"""
return self._batch.access_values(keep_evals=keep_evals)[0]
def access_evals(self) -> torch.Tensor:
"""
Access the evaluation results of the solution.
The received tensor will be mutable.
Returns:
The evaluation results tensor of the solution.
"""
return self._batch.access_evals()[0]
@property
def values(self) -> Any:
"""
Decision values of the solution
"""
return self._batch.values[0]
@property
def evals(self) -> torch.Tensor:
"""
Evaluation results of the solution in a 1-dimensional tensor.
"""
return self._batch.evals[0]
@property
def evaluation(self) -> torch.Tensor:
"""
Get the evaluation result.
If the problem is single-objective and the problem does not
allocate any space for extra evaluation data, then a scalar
is returned.
Otherwise, this property becomes equivalent to the `evals`
property, and a 1-dimensional tensor is returned.
"""
result = self.evals
if len(result) == 1:
result = result[0]
return result
def set_values(self, values: Any):
"""
Set the decision values of the Solution.
Note that modifying the decision values will result in the
evaluation results being getting cleared (in more details,
the evaluation results tensor will be filled with NaN values).
Args:
values: New decision values for this Solution.
"""
if is_dtype_object(self.dtype):
value_tensor = ObjectArray(1)
value_tensor[0] = values
else:
value_tensor = torch.as_tensor(values, dtype=self.dtype).reshape(1, -1)
self._batch.set_values(value_tensor)
def set_evals(self, evals: torch.Tensor, eval_data: Optional[Iterable] = None):
"""
Set the evaluation results of the Solution.
Args:
evals: New evaluation result(s) for the Solution.
For single-objective problems, this argument can be given
as a scalar.
When this argument is given as a scalar (for single-objective
cases) or as a tensor which is long enough to cover for
all the objectives but not for the extra evaluation data,
then the extra evaluation data will be cleared
(in more details, extra evaluation data will be filled with
NaN values).
eval_data: Optionally, the argument `eval_data` can be used to
specify extra evaluation data separately.
`eval_data` is expected as a 1-dimensional sequence.
"""
evals = torch.as_tensor(evals, dtype=self.eval_dtype, device=self.device)
if evals.ndim in (0, 1):
evals = evals.reshape(1, -1)
else:
raise ValueError(
f"`set_evals(...)` method of a Solution expects a 1-dimensional or a 2-dimensional"
f" evaluation tensor. However, the received evaluation tensor has {evals.ndim} dimensions"
f" (having a shape of {evals.shape})."
)
if eval_data is not None:
eval_data = torch.as_tensor(eval_data, dtype=self.eval_dtype, device=self.device)
if eval_data.ndim != 1:
raise ValueError(
f"The argument `eval_data` was expected as a 1-dimensional sequence."
f" However, the shape of `eval_data` is {eval_data.shape}."
)
eval_data = eval_data.reshape(1, -1)
self._batch.set_evals(evals, eval_data)
def set_evaluation(self, evaluation: RealOrVector, eval_data: Optional[Iterable] = None):
"""
Set the evaluation results of the Solution.
This method is an alias for `set_evals(...)`, added for having
a setter counterpart for the `evaluation` property of the Solution
class.
Args:
evaluation: New evaluation result(s) for the Solution.
For single-objective problems, this argument can be given
as a scalar.
When this argument is given as a scalar (for single-objective
cases) or as a tensor which is long enough to cover for
all the objectives but not for the extra evaluation data,
then the extra evaluation data will be cleared
(in more details, extra evaluation data will be filled with
NaN values).
eval_data: Optionally, the argument `eval_data` can be used to
specify extra evaluation data separately.
`eval_data` is expected as a 1-dimensional sequence.
"""
self.set_evals(evaluation, eval_data)
def objective_sense(self) -> ObjectiveSense:
"""
Get the objective sense(s) of this Solution's associated Problem.
If the problem is single-objective, then a single string is returned.
If the problem is multi-objective, then the objective senses will be
returned in a list.
The returned string in the single-objective case, or each returned
string in the multi-objective case, is "min" or "max".
"""
return self._batch.objective_sense
@property
def senses(self) -> Iterable[str]:
"""
Objective sense(s) of this Solution's associated Problem.
This is a list of strings, each string being "min" or "max".
"""
return self._batch.senses
@property
def is_evaluated(self) -> bool:
"""
Whether or not the Solution is fully evaluated.
This property returns True only when all of the evaluation results
for all objectives have numeric values other than NaN.
This property assumes that the extra evaluation data is optional,
and therefore does not take into consideration whether or not the
extra evaluation data contains NaN values.
In other words, while determining whether or not a solution is fully
evaluated, only the evaluation results corresponding to the
objectives are taken into account.
"""
num_objs = len(self.senses)
with torch.no_grad():
return not bool(torch.any(torch.isnan(self._batch.evals[0, :num_objs])))
@property
def dtype(self) -> DType:
"""
dtype of the decision values
"""
return self._batch.dtype
@property
def device(self) -> Device:
"""
The device storing the Solution
"""
return self._batch.device
@property
def eval_dtype(self) -> DType:
"""
dtype of the evaluation results
"""
return self._batch.eval_dtype
@staticmethod
def _rightmost_shape(shape: Iterable) -> torch.Size:
if len(shape) >= 2:
return torch.Size([int(shape[-1])])
else:
return torch.Size([])
@property
def shape(self) -> torch.Size:
"""
Shape of the decision values of the Solution
"""
return self._rightmost_shape(self._batch.values_shape)
def size(self) -> torch.Size:
"""
Shape of the decision values of the Solution
"""
return self.shape
@property
def eval_shape(self) -> torch.Size:
"""
Shape of the evaluation results
"""
return self._rightmost_shape(self._batch.eval_shape)
@property
def ndim(self) -> int:
"""
Number of dimensions of the decision values.
For numeric solutions (e.g. of dtype `torch.float32`), this returns
1, since such numeric solutions are kepts as 1-dimensional vectors.
When dtype is `object`, `ndim` is reported as whatever the contained
object reports as its `ndim`, or 0 if the contained object does not
have an `ndim` attribute.
"""
values = self.values
if hasattr(values, "ndim"):
return values.ndim
else:
return 0
def dim(self) -> int:
"""
This method returns the `ndim` attribute of this Solution.
"""
return self.ndim
def __len__(self) -> int:
return len(self.values)
def __iter__(self):
return self.values.__iter__()
def __reversed__(self):
return self.values.__reversed__()
def __getitem__(self, i):
return self.values.__getitem__(i)
def _to_string(self) -> str:
clsname = type(self).__name__
result = []
values = self._batch.access_values(keep_evals=True)[0]
evals = self._batch.access_evals()[0]
def write(*args):
for arg in args:
result.append(str(arg))
write("<", clsname, " values=", values)
if not torch.all(torch.isnan(evals)):
write(", evals=", evals)
write(">")
return "".join(result)
def __repr__(self) -> str:
return self._to_string()
def __str__(self) -> str:
return self._to_string()
def _get_cloned_state(self, *, memo: dict) -> dict:
with _no_grad_if_basic_dtype(self.dtype):
return deep_clone(
self.__dict__,
otherwise_deepcopy=True,
memo=memo,
)
def to(self, device: Device) -> "Solution":
"""
Get the counterpart of this Solution on the new device.
If the specified device is the device of this Solution,
then this Solution itself is returned.
If the specified device is a different device, then a clone
of this Solution on this different device is first
created, and then this new clone is returned.
Please note that the `to(...)` method is not supported when
the dtype is `object`.
Args:
device: The device on which the resulting Solution
will be stored.
Returns:
The Solution on the specified device.
"""
return Solution(self._batch.to(device), 0)
def to_batch(self) -> SolutionBatch:
"""
Get the single-row SolutionBatch counterpart of the Solution.
The returned SolutionBatch and the Solution have shared
storage, meaning that modifying one of them affects the other.
Returns:
The SolutionBatch counterpart of the Solution.
"""
return self._batch
device: Union[str, torch.device]
property
readonly
¶
The device storing the Solution
dtype: Union[str, torch.dtype, numpy.dtype, Type]
property
readonly
¶
dtype of the decision values
eval_dtype: Union[str, torch.dtype, numpy.dtype, Type]
property
readonly
¶
dtype of the evaluation results
eval_shape: Size
property
readonly
¶
Shape of the evaluation results
evals: Tensor
property
readonly
¶
Evaluation results of the solution in a 1-dimensional tensor.
evaluation: Tensor
property
readonly
¶
Get the evaluation result.
If the problem is single-objective and the problem does not
allocate any space for extra evaluation data, then a scalar
is returned.
Otherwise, this property becomes equivalent to the evals
property, and a 1-dimensional tensor is returned.
is_evaluated: bool
property
readonly
¶
Whether or not the Solution is fully evaluated.
This property returns True only when all of the evaluation results for all objectives have numeric values other than NaN.
This property assumes that the extra evaluation data is optional, and therefore does not take into consideration whether or not the extra evaluation data contains NaN values. In other words, while determining whether or not a solution is fully evaluated, only the evaluation results corresponding to the objectives are taken into account.
ndim: int
property
readonly
¶
Number of dimensions of the decision values.
For numeric solutions (e.g. of dtype torch.float32), this returns
1, since such numeric solutions are kepts as 1-dimensional vectors.
When dtype is object, ndim is reported as whatever the contained
object reports as its ndim, or 0 if the contained object does not
have an ndim attribute.
senses: Iterable[str]
property
readonly
¶
Objective sense(s) of this Solution's associated Problem.
This is a list of strings, each string being "min" or "max".
shape: Size
property
readonly
¶
Shape of the decision values of the Solution
values: Any
property
readonly
¶
Decision values of the solution
__init__(self, parent, index)
special
¶
__init__(...): Initialize the Solution object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
parent |
SolutionBatch |
The parent SolutionBatch which stores the Solution. |
required |
index |
int |
Index of the solution in SolutionBatch. |
required |
Source code in evotorch/core.py
def __init__(self, parent: SolutionBatch, index: int):
"""
`__init__(...)`: Initialize the Solution object.
Args:
parent: The parent SolutionBatch which stores the Solution.
index: Index of the solution in SolutionBatch.
"""
if not isinstance(parent, SolutionBatch):
raise TypeError(
f"Expected a SolutionBatch as a parent, but encountered {repr(parent)},"
f" which is of type {repr(type(parent))}."
)
index = int(index)
if index < 0:
index = len(parent) + index
if not ((index >= 0) and (index <= len(parent))):
raise IndexError(f"Invalid index: {index}")
self._batch: SolutionBatch = parent[index : index + 1]
access_evals(self)
¶
Access the evaluation results of the solution. The received tensor will be mutable.
Returns:
| Type | Description |
|---|---|
Tensor |
The evaluation results tensor of the solution. |
access_values(self, *, keep_evals=False)
¶
Access the decision values tensor of the solution. The received tensor will be mutable.
By default, it will be assumed that the user wishes to obtain this tensor to change the decision values, and therefore, the evaluation results associated with this solution will be cleared (i.e. will be NaN).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
keep_evals |
bool |
When this is set to True, the evaluation results associated with this solution will be kept (i.e. will NOT be cleared). |
False |
Returns:
| Type | Description |
|---|---|
Tensor |
The decision values tensor of the solution. |
Source code in evotorch/core.py
def access_values(self, *, keep_evals: bool = False) -> torch.Tensor:
"""
Access the decision values tensor of the solution.
The received tensor will be mutable.
By default, it will be assumed that the user wishes to
obtain this tensor to change the decision values, and therefore,
the evaluation results associated with this solution will be
cleared (i.e. will be NaN).
Args:
keep_evals: When this is set to True, the evaluation results
associated with this solution will be kept (i.e. will NOT
be cleared).
Returns:
The decision values tensor of the solution.
"""
return self._batch.access_values(keep_evals=keep_evals)[0]
dim(self)
¶
objective_sense(self)
¶
Get the objective sense(s) of this Solution's associated Problem.
If the problem is single-objective, then a single string is returned. If the problem is multi-objective, then the objective senses will be returned in a list.
The returned string in the single-objective case, or each returned string in the multi-objective case, is "min" or "max".
Source code in evotorch/core.py
def objective_sense(self) -> ObjectiveSense:
"""
Get the objective sense(s) of this Solution's associated Problem.
If the problem is single-objective, then a single string is returned.
If the problem is multi-objective, then the objective senses will be
returned in a list.
The returned string in the single-objective case, or each returned
string in the multi-objective case, is "min" or "max".
"""
return self._batch.objective_sense
set_evals(self, evals, eval_data=None)
¶
Set the evaluation results of the Solution.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
evals |
Tensor |
New evaluation result(s) for the Solution. For single-objective problems, this argument can be given as a scalar. When this argument is given as a scalar (for single-objective cases) or as a tensor which is long enough to cover for all the objectives but not for the extra evaluation data, then the extra evaluation data will be cleared (in more details, extra evaluation data will be filled with NaN values). |
required |
eval_data |
Optional[Iterable] |
Optionally, the argument |
None |
Source code in evotorch/core.py
def set_evals(self, evals: torch.Tensor, eval_data: Optional[Iterable] = None):
"""
Set the evaluation results of the Solution.
Args:
evals: New evaluation result(s) for the Solution.
For single-objective problems, this argument can be given
as a scalar.
When this argument is given as a scalar (for single-objective
cases) or as a tensor which is long enough to cover for
all the objectives but not for the extra evaluation data,
then the extra evaluation data will be cleared
(in more details, extra evaluation data will be filled with
NaN values).
eval_data: Optionally, the argument `eval_data` can be used to
specify extra evaluation data separately.
`eval_data` is expected as a 1-dimensional sequence.
"""
evals = torch.as_tensor(evals, dtype=self.eval_dtype, device=self.device)
if evals.ndim in (0, 1):
evals = evals.reshape(1, -1)
else:
raise ValueError(
f"`set_evals(...)` method of a Solution expects a 1-dimensional or a 2-dimensional"
f" evaluation tensor. However, the received evaluation tensor has {evals.ndim} dimensions"
f" (having a shape of {evals.shape})."
)
if eval_data is not None:
eval_data = torch.as_tensor(eval_data, dtype=self.eval_dtype, device=self.device)
if eval_data.ndim != 1:
raise ValueError(
f"The argument `eval_data` was expected as a 1-dimensional sequence."
f" However, the shape of `eval_data` is {eval_data.shape}."
)
eval_data = eval_data.reshape(1, -1)
self._batch.set_evals(evals, eval_data)
set_evaluation(self, evaluation, eval_data=None)
¶
Set the evaluation results of the Solution.
This method is an alias for set_evals(...), added for having
a setter counterpart for the evaluation property of the Solution
class.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
evaluation |
Union[float, Iterable[float], torch.Tensor] |
New evaluation result(s) for the Solution. For single-objective problems, this argument can be given as a scalar. When this argument is given as a scalar (for single-objective cases) or as a tensor which is long enough to cover for all the objectives but not for the extra evaluation data, then the extra evaluation data will be cleared (in more details, extra evaluation data will be filled with NaN values). |
required |
eval_data |
Optional[Iterable] |
Optionally, the argument |
None |
Source code in evotorch/core.py
def set_evaluation(self, evaluation: RealOrVector, eval_data: Optional[Iterable] = None):
"""
Set the evaluation results of the Solution.
This method is an alias for `set_evals(...)`, added for having
a setter counterpart for the `evaluation` property of the Solution
class.
Args:
evaluation: New evaluation result(s) for the Solution.
For single-objective problems, this argument can be given
as a scalar.
When this argument is given as a scalar (for single-objective
cases) or as a tensor which is long enough to cover for
all the objectives but not for the extra evaluation data,
then the extra evaluation data will be cleared
(in more details, extra evaluation data will be filled with
NaN values).
eval_data: Optionally, the argument `eval_data` can be used to
specify extra evaluation data separately.
`eval_data` is expected as a 1-dimensional sequence.
"""
self.set_evals(evaluation, eval_data)
set_values(self, values)
¶
Set the decision values of the Solution.
Note that modifying the decision values will result in the evaluation results being getting cleared (in more details, the evaluation results tensor will be filled with NaN values).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
values |
Any |
New decision values for this Solution. |
required |
Source code in evotorch/core.py
def set_values(self, values: Any):
"""
Set the decision values of the Solution.
Note that modifying the decision values will result in the
evaluation results being getting cleared (in more details,
the evaluation results tensor will be filled with NaN values).
Args:
values: New decision values for this Solution.
"""
if is_dtype_object(self.dtype):
value_tensor = ObjectArray(1)
value_tensor[0] = values
else:
value_tensor = torch.as_tensor(values, dtype=self.dtype).reshape(1, -1)
self._batch.set_values(value_tensor)
size(self)
¶
to(self, device)
¶
Get the counterpart of this Solution on the new device.
If the specified device is the device of this Solution, then this Solution itself is returned. If the specified device is a different device, then a clone of this Solution on this different device is first created, and then this new clone is returned.
Please note that the to(...) method is not supported when
the dtype is object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
device |
Union[str, torch.device] |
The device on which the resulting Solution will be stored. |
required |
Returns:
| Type | Description |
|---|---|
Solution |
The Solution on the specified device. |
Source code in evotorch/core.py
def to(self, device: Device) -> "Solution":
"""
Get the counterpart of this Solution on the new device.
If the specified device is the device of this Solution,
then this Solution itself is returned.
If the specified device is a different device, then a clone
of this Solution on this different device is first
created, and then this new clone is returned.
Please note that the `to(...)` method is not supported when
the dtype is `object`.
Args:
device: The device on which the resulting Solution
will be stored.
Returns:
The Solution on the specified device.
"""
return Solution(self._batch.to(device), 0)
to_batch(self)
¶
Get the single-row SolutionBatch counterpart of the Solution. The returned SolutionBatch and the Solution have shared storage, meaning that modifying one of them affects the other.
Returns:
| Type | Description |
|---|---|
SolutionBatch |
The SolutionBatch counterpart of the Solution. |
Source code in evotorch/core.py
def to_batch(self) -> SolutionBatch:
"""
Get the single-row SolutionBatch counterpart of the Solution.
The returned SolutionBatch and the Solution have shared
storage, meaning that modifying one of them affects the other.
Returns:
The SolutionBatch counterpart of the Solution.
"""
return self._batch
SolutionBatch (Serializable)
¶
Representation of a batch of solutions.
A SolutionBatch stores the decision values of multiple solutions in a single contiguous tensor. For numeric and fixed-length problems, this contiguous tensor is a PyTorch tensor. For not-necessarily-numeric and not-necessarily-fixed-length problems, this contiguous tensor is an ObjectArray.
The evalution results and extra evaluation data of the solutions are also stored in an additional contiguous tensor.
Interface-wise, a SolutionBatch behaves like a sequence of
Solution objects. One can get single Solution from a SolutionBatch
via the indexing operator ([]).
Additionally, one can iterate over each solution using
the for ... in ... statement.
One can also get a slice of a SolutionBatch.
The slicing of a SolutionBatch results in a new SolutionBatch.
With simple slicing, the obtained SolutionBatch shares its
memory with the original SolutionBatch.
With advanced slicing (i.e. the kind of slicing where the
solution indices are specified one by one, like:
mybatch[[0, 4, 2, 5]]), the obtained SolutionBatch is a copy,
and does not share any memory with its original.
The decision values of all the stored solutions in the batch can be obtained in a read-only tensor via:
values = batch.values
If one has modified decision values and wishes to put them
into the batch, the set_values(...) method can be used
as follows:
batch.set_values(modified_values)
The evaluation results of the solutions can be obtained in a read-only tensor via:
evals = batch.evals
If one has newly computed evaluation results, and wishes
to put them into the batch, the set_evals(...) method
can be used as follows:
batch.set_evals(newly_computed_evals)
Source code in evotorch/core.py
class SolutionBatch(Serializable):
"""
Representation of a batch of solutions.
A SolutionBatch stores the decision values of multiple solutions
in a single contiguous tensor. For numeric and fixed-length
problems, this contiguous tensor is a PyTorch tensor.
For not-necessarily-numeric and not-necessarily-fixed-length
problems, this contiguous tensor is an ObjectArray.
The evalution results and extra evaluation data of the solutions
are also stored in an additional contiguous tensor.
Interface-wise, a SolutionBatch behaves like a sequence of
Solution objects. One can get single Solution from a SolutionBatch
via the indexing operator (`[]`).
Additionally, one can iterate over each solution using
the `for ... in ...` statement.
One can also get a slice of a SolutionBatch.
The slicing of a SolutionBatch results in a new SolutionBatch.
With simple slicing, the obtained SolutionBatch shares its
memory with the original SolutionBatch.
With advanced slicing (i.e. the kind of slicing where the
solution indices are specified one by one, like:
`mybatch[[0, 4, 2, 5]]`), the obtained SolutionBatch is a copy,
and does not share any memory with its original.
The decision values of all the stored solutions in the batch
can be obtained in a read-only tensor via:
values = batch.values
If one has modified decision values and wishes to put them
into the batch, the `set_values(...)` method can be used
as follows:
batch.set_values(modified_values)
The evaluation results of the solutions can be obtained
in a read-only tensor via:
evals = batch.evals
If one has newly computed evaluation results, and wishes
to put them into the batch, the `set_evals(...)` method
can be used as follows:
batch.set_evals(newly_computed_evals)
"""
def __init__(
self,
problem: Optional[Problem] = None,
popsize: Optional[int] = None,
*,
device: Optional[Device] = None,
slice_of: Optional[Union[tuple, SolutionBatchSliceInfo]] = None,
like: Optional["SolutionBatch"] = None,
merging_of: Iterable["SolutionBatch"] = None,
empty: Optional[bool] = None,
):
self._num_objs: int
self._data: Union[torch.Tensor, ObjectArray]
self._descending: Iterable[bool]
self._slice: Optional[IndicesOrSlice] = None
if slice_of is not None:
expect_none(
"While making a new SolutionBatch via slicing",
problem=problem,
popsize=popsize,
device=device,
merging_of=merging_of,
like=like,
empty=empty,
)
source: "SolutionBatch"
slice_info: IndicesOrSlice
source, slice_info = slice_of
def safe_slice(t: torch.Tensor, slice_info):
d0 = t.ndim
t = t[slice_info]
d1 = t.ndim
if d0 != d1:
raise ValueError(
"Encountered an illegal slicing operation which would"
" change the shape of the stored tensor(s) of the"
" SolutionBatch."
)
return t
with torch.no_grad():
# self._data = source._data[slice_info]
# self._evdata = source._evdata[slice_info]
self._data = safe_slice(source._data, slice_info)
self._evdata = safe_slice(source._evdata, slice_info)
self._slice = slice_info
self._descending = source._descending
shares_storage = storage_ptr(self._data) == storage_ptr(source._data)
if not shares_storage:
self._descending = deepcopy(self._descending)
self._num_objs = source._num_objs
elif like is not None:
expect_none(
"While making a new SolutionBatch via the like=... argument",
merging_of=merging_of,
slice_of=slice_of,
)
self._data = empty_tensor_like(like._data, length=popsize, device=device)
self._evdata = empty_tensor_like(like._evdata, length=popsize, device=device)
self._evdata[:] = float("nan")
self._descending = like._descending
self._num_objs = like._num_objs
if not _opt_bool(empty, default=False):
self._fill_via_problem(problem)
elif merging_of is not None:
expect_none(
"While making a new SolutionBatch via merging",
problem=problem,
popsize=popsize,
device=device,
slice_of=slice_of,
like=like,
empty=empty,
)
# Convert `merging_of` into a list.
# While doing that, also count the total number of rows
batches = []
total_rows = 0
for batch in merging_of:
total_rows += len(batch)
batches.append(batch)
# Get essential attributes from the first batch
self._descending = deepcopy(batches[0]._descending)
self._num_objs = batches[0]._num_objs
if isinstance(batches[0]._data, ObjectArray):
def process_data(x):
return deepcopy(x)
self._data = ObjectArray(total_rows)
else:
def process_data(x):
return x
self._data = empty_tensor_like(batches[0]._data, length=total_rows)
self._evdata = empty_tensor_like(batches[0]._evdata, length=total_rows)
row_begin = 0
for batch in batches:
row_end = row_begin + len(batch)
self._data[row_begin:row_end] = process_data(batch._data)
self._evdata[row_begin:row_end] = batch._evdata
row_begin = row_end
elif problem is not None:
expect_none(
"While making a new SolutionBatch with a given problem",
slice_of=slice_of,
like=like,
merging_of=merging_of,
)
if device is None:
device = problem.device
self._num_objs = len(problem.senses)
if problem.dtype is object:
if str(device) != "cpu":
raise ValueError("Cannot create a batch containing arbitrary objects on a device other than cpu")
self._data = ObjectArray(popsize)
else:
self._data = torch.empty((popsize, problem.solution_length), device=device, dtype=problem.dtype)
if not _opt_bool(empty, default=False):
self._data[:] = problem.generate_values(len(self._data))
self._evdata = problem.make_nan(
popsize, self._num_objs + problem.eval_data_length, device=device, use_eval_dtype=True
)
self._descending = problem.get_obj_order_descending()
else:
raise ValueError("Invalid call to the __init__(...) of SolutionBatch")
def _normalize_row_index(self, i: int) -> int:
i = int(i)
org_i = i
if i < 0:
i = int(self._data.shape[0]) + i
if (i < 0) or (i > (self._data.shape[0] - 1)):
raise IndexError(f"Invalid row: {org_i}")
return i
def _normalize_obj_index(self, i: int) -> int:
i = int(i)
org_i = i
if i < 0:
i = self._num_objs + i
if (i < 0) or (i > (self._num_objs)):
raise IndexError(f"Invalid objective index: {org_i}")
return i
def _optionally_get_obj_index(self, i: Optional[int]) -> int:
if i is None:
if self._num_objs != 1:
raise ValueError(
f"The objective index was given as None."
f" However, the number of objectives is not 1,"
f" it is {self._num_objs}."
f" Therefore, the objective index is not optional,"
f" and must be provided as an integer, not as None."
)
return 0
else:
return self._normalize_obj_index(i)
@torch.no_grad()
def argsort(self, obj_index: Optional[int] = None) -> torch.Tensor:
"""Return the indices of solutions, sorted from best to worst.
Args:
obj_index: The objective index. Can be passed as None
if the problem is single-objective. Otherwise,
expected as an int.
Returns:
A PyTorch tensor, containing the solution indices,
sorted from the best solution to the worst.
"""
obj_index = self._optionally_get_obj_index(obj_index)
descending = self._descending[obj_index]
ev_col = self._evdata[:, obj_index]
return torch.argsort(ev_col, descending=descending)
@torch.no_grad()
def compute_pareto_ranks(self, crowdsort: bool = True) -> Tuple[torch.Tensor, torch.Tensor]:
"""
Compute the pareto-ranks of the solutions in the batch.
Args:
crowdsort: If given as True, each front in itself
will be sorted from the least crowding solution
to the most crowding solution.
If given as False, there will be no crowd-sorting.
Returns:
ranks (torch.Tensor): The computed pareto ranks, of shape [num_samples,]
Ranks are encoded in the form 'lowest is best'. So solutions in the pareto front will be assigned rank 0,
solutions in the next non-dominated set (excluding the pareto front) will be assigned rank 1, and so on.
crowdsort_ranks (Optional[torch.Tensor]): The computed crowd sort ranks, only returned if crowdsort=True. Otherwise, None is returned.
Solutions within a front are assigned a crowding score, as described in the above paper.
Then, the solution with the best crowding score within a front of size K is assigned rank 0, and the solution with the
worst crowding score within a front is assigned rank K-1.
"""
utils = self.utils()
ranks, crowdsort_ranks = _compute_pareto_ranks(utils, crowdsort)
return ranks, crowdsort_ranks
@torch.no_grad()
def arg_pareto_sort(self, crowdsort: bool = True) -> ParetoInfo:
"""
Pareto-sort the solutions in the batch.
The result is a namedtuple consisting of two elements:
`fronts` and `ranks`.
Let us assume that we have 5 solutions, and after a
pareto-sorting they ended up in this order:
front 0 (best front) : solution 1, solution 2
front 1 : solution 0, solution 4
front 2 (worst front): solution 3
Considering the example ordering above, the returned
ParetoInfo instance looks like this:
ParetoInfo(
fronts=[[1, 2], [0, 4], [3]],
ranks=tensor([1, 0, 0, 2, 1])
)
where `fronts` stores the solution indices grouped by
pareto fronts; and `ranks` stores, as a tensor of int64,
the pareto rank for each solution (where 0 means best
rank).
Args:
crowdsort: If given as True, each front in itself
will be sorted from the least crowding solution
to the most crowding solution.
If given as False, there will be no crowd-sorting.
Returns:
A ParetoInfo instance
"""
utils = self.utils()
fronts, ranks = _pareto_sort(utils, crowdsort)
return ParetoInfo(fronts=fronts, ranks=ranks)
@torch.no_grad()
def argbest(self, obj_index: Optional[int] = None) -> torch.Tensor:
"""Return the best solution's index
Args:
obj_index: The objective index. Can be passed as None
if the problem is single-objective. Otherwise,
expected as an int.
Returns:
The index of the best solution.
"""
obj_index = self._optionally_get_obj_index(obj_index)
descending = self._descending[obj_index]
argf = torch.argmax if descending else torch.argmin
return argf(self._evdata[:, obj_index])
@torch.no_grad()
def argworst(self, obj_index: Optional[int] = None) -> torch.Tensor:
"""Return the worst solution's index
Args:
obj_index: The objective index. Can be passed as None
if the problem is single-objective. Otherwise,
expected as an int.
Returns:
The index of the worst solution.
"""
obj_index = self._optionally_get_obj_index(obj_index)
descending = self._descending[obj_index]
argf = torch.argmin if descending else torch.argmax
return argf(self._evdata[:, obj_index])
def _get_objective_sign(self, i_obj: int) -> float:
if self._descending[i_obj]:
return 1.0
else:
return -1.0
@torch.no_grad()
def set_values(self, values: Any, *, solutions: MaybeIndicesOrSlice = None):
"""
Set the decision values of the solutions.
Args:
values: New decision values.
solutions: Optionally a list of integer indices or an instance
of `slice(...)`, to be used if one wishes to set the
decision values of only some of the solutions.
"""
if solutions is None:
solutions = slice(None, None, None)
self._data[solutions] = values
self._evdata[solutions] = float("nan")
@torch.no_grad()
def set_evals(
self,
evals: torch.Tensor,
eval_data: Optional[torch.Tensor] = None,
*,
solutions: MaybeIndicesOrSlice = None,
):
"""
Set the evaluations of the solutions.
Args:
evals: A numeric tensor which contains the evaluation results.
Acceptable shapes are as follows:
`(n,)` only to be used for single-objective problems, sets
the evaluation results of the target `n` solutions, and clears
(where clearing means to fill with NaN values)
extra evaluation data (if the problem has allocations for such
extra evaluation data);
`(n,m)` where `m` is the number of objectives, sets the
evaluation results of the target `n` solutions, and clears
their extra evaluation data;
`(n,m+q)` where `m` is the number of objectives and `q` is the
length of extra evaluation data, sets the evaluation result
and extra data of the target `n` solutions.
eval_data: To be used only when the problem has extra evaluation
data. Optionally, one can pass the extra evaluation data
separately via this argument (instead of jointly through
a single tensor via `evals`).
The expected shape of this tensor is `(n,q)` where `n`
is the number of solutions and `q` is the length of the
extra evaluation data.
solutions: Optionally a list of integer indices or an instance
of `slice(...)`, to be used if one wishes to set the
evaluations of only some of the solutions.
Raises:
ValueError: if the given tensor has an incompatible shape.
"""
if solutions is None:
solutions = slice(None, None, None)
num_solutions = self._evdata.shape[0]
elif isinstance(solutions, slice):
num_solutions = self._evdata[solutions].shape[0]
elif is_sequence(solutions):
num_solutions = len(solutions)
total_eval_width = self._evdata.shape[1]
num_objs = self._num_objs
num_data = total_eval_width - num_objs
if evals.ndim == 1:
if num_objs != 1:
raise ValueError(
f"The method `set_evals(...)` was given a 1-dimensional tensor."
f" However, the number of objectives of the problem at hand is {num_objs}, not 1."
f" 1-dimensional evaluation tensors can only be accepted if the problem"
f" has one objective."
)
evals = evals.reshape(-1, 1)
elif evals.ndim == 2:
pass # nothing to do here
else:
if num_objs == 1:
raise ValueError(
f"The method `set_evals(...)` received a tensor with {evals.ndim} dimensions."
f" Since the problem at hand has only one objective,"
f" 1-dimensional or 2-dimensional tensors are acceptable, but not {evals.ndim}-dimensional ones."
)
else:
raise ValueError(
f"The method `set_evals(...)` received a tensor with {evals.ndim} dimensions."
f" Since the problem at hand has more than one objective (there are {num_objs} objectives),"
f" only 2-dimensional tensors are acceptable, not {evals.ndim}-dimensional ones."
)
[nrows, ncols] = evals.shape
if nrows != num_solutions:
raise ValueError(
f"Trying to set the evaluations of {num_solutions} solutions, but the given tensor has {nrows} rows."
)
if eval_data is not None:
if eval_data.ndim != 2:
raise ValueError(
f"The `eval_data` argument was expected as a 2-dimensional tensor."
f" However, the shape of the given `eval_data` is {eval_data.shape}."
)
if eval_data.shape[1] != num_data:
raise ValueError(
f"The `eval_data` argument was expected to have {num_data} columns."
f" However, the received `eval_data` has the shape: {eval_data.shape}."
)
if ncols != num_objs:
raise ValueError(
f"The method `set_evals(...)` was used with `evals` and `eval_data` arguments."
f" When both of these arguments are provided, `evals` is expected either as a 1-dimensional tensor"
f" (for single-objective cases only), or as a tensor of shape (n, m) where n is the number of"
f" solutions, and m is the number of objectives."
f" However, while the problem at hand has {num_objs} objectives,"
f" the `evals` tensor has {ncols} columns."
)
if evals.shape[0] != eval_data.shape[0]:
raise ValueError(
f"The provided `evals` and `eval_data` tensors have incompatible shapes."
f" Shape of `evals`: {evals.shape},"
f" shape of `eval_data`: {eval_data.shape}."
)
self._evdata[solutions, :] = torch.hstack([evals, eval_data])
else:
if ncols == num_objs:
self._evdata[solutions, :num_objs] = evals
self._evdata[solutions, num_objs:] = float("nan")
elif ncols == total_eval_width:
self._evdata[solutions, :] = evals
else:
raise ValueError(
f"The method `set_evals(...)` received a tensor with {ncols} columns, which is incompatible."
f" Acceptable number of columns are: {num_objs}"
f" (for setting only the objective-associated evaluations and leave extra evaluation data as NaN), or"
f" {total_eval_width} (for setting both objective-associated evaluations and extra evaluation data)."
)
@property
def evals(self) -> torch.Tensor:
"""
Evaluation results of the solutions, in a ReadOnlyTensor
"""
from .tools.readonlytensor import as_read_only_tensor
with torch.no_grad():
return as_read_only_tensor(self._evdata)
@property
def values(self) -> Union[torch.Tensor, Iterable]:
"""
Decision values of the solutions, in a read-only tensor-like object
"""
from .tools.readonlytensor import as_read_only_tensor
with torch.no_grad():
return as_read_only_tensor(self._data)
# @property
# def unsafe_evals(self) -> torch.Tensor:
# """
# It is not recommended to use this property.
#
# Grants mutable access to the evaluations of the solutions.
# """
# return self._evdata
#
# @property
# def unsafe_values(self) -> Union[torch.Tensor, Iterable]:
# """
# It is not recommended to use this property.
#
# Grants mutable access to the decision values of the solutions.
# """
# return self._data
@torch.no_grad()
def access_evals(self, obj_index: Optional[int] = None) -> torch.Tensor:
"""
Get the internal mutable tensor storing the evaluations.
IMPORTANT: This method exposes the evaluation tensor of the batch
as it is, in mutable mode. It is therefore considered unsafe to rely
on this method. Before using this method, please consider using the
`evals` property for reading the evaluation results, and using the
`set_evals(...)` method which allows one to update the evaluations
without exposing any internal tensor.
When this method is used without any argument, the returned tensor
will be of shape `(n, m)`, where `n` is the number of solutions,
and `m` is the number of objectives plus the length of extra
evaluation data.
When this method is used with an integer argument specifying an
objective index, the returned tensor will be 1-dimensional
having a length of `n`, where `n` is the number of solutions.
In this case, the returned 1-dimensional tensor will be a view
upon the evaluation results of the specified objective.
The value `nan` (not-a-number) means not evaluated yet.
Args:
obj_index: None for getting the entire 2-dimensional evaluation
tensor; an objective index (as integer) for getting a
1-dimensional mutable slice of the evaluation tensor,
the slice being a view upon the evaluation results
regarding the specified objective.
Returns:
The mutable tensor storing the evaluation information.
"""
if obj_index is None:
return self._evdata
else:
return self._evdata[:, self._normalize_obj_index(obj_index)]
@torch.no_grad()
def access_values(self, *, keep_evals: bool = False) -> Union[torch.Tensor, ObjectArray]:
"""
Get the internal mutable tensor storing the decision values.
IMPORTANT: This method exposes the internal decision values tensor of
the batch as it is, in mutable mode. It is therefore considered unsafe
to rely on this method. Before using this method, please consider
using the `values` property for reading the decision values, and using
the `set_values(...)` method which allows one to update the decision
values without exposing any internal tensor.
IMPORTANT: The default assumption of this method is that the tensor
is requested for modification purposes. Therefore, by default, as soon
as this method is called, the evaluation results of the solutions will
be cleared (where clearing means that the evaluation results will be
filled with `NaN`s).
The reasoning behind this default behavior is to prevent the modified
solutions from having outdated evaluation results.
Args:
keep_evals: If set as False, the evaluation data of the solutions
will be cleared (i.e. will be filled with `NaN`s).
If set as True, the existing evaluation data will be kept.
Returns:
The mutable tensor storing the decision values.
"""
if not keep_evals:
self.forget_evals()
return self._data
@torch.no_grad()
def forget_evals(self, *, solutions: MaybeIndicesOrSlice = None):
"""
Forget the evaluations of the solutions.
The evaluation results will be cleared, which means that they will
be filled with `NaN`s.
"""
if solutions is None:
solutions = slice(None, None, None)
self._evdata[solutions, :] = float("nan")
@torch.no_grad()
def utility(
self,
obj_index: Optional[int] = None,
*,
ranking_method: Optional[str] = None,
check_nans: bool = True,
using_values_dtype: bool = False,
) -> torch.Tensor:
"""
Return numeric scores for each solution.
Utility scores are different from evaluation results,
in the sense that utilities monotonically increase from
bad solutions to good solutions, regardless of the
objective sense.
**If ranking method is passed as None:**
if the objective sense is 'max', the evaluation results are returned
as the utility scores; otherwise, if the objective sense is 'min',
the evaluation results multiplied by -1 are returned as the
utility scores.
**If the name of a ranking method is given** (e.g. 'centered'):
then the solutions are ranked (best solutions having the
highest rank), and those ranks are returned as the utility
scores.
**If an objective index is not provided:** (i.e. passed as None)
if the problem is multi-objective, the utility scores
for each objective is given, in a tensor shaped (n, m),
n being the number of solutions and m being the number
of objectives; otherwise, if the problem is single-objective,
the utility scores for each objective is given in a 1-dimensional
tensor of length n, n being the number of solutions.
**If an objective index is provided as an int:**
the utility scores are returned in a 1-dimensional tensor
of length n, n being the number of solutions.
Args:
obj_index: Expected as None, or as an integer.
In the single-objective case, None is equivalent to 0.
In the multi-objective case, None means "for each
objective".
ranking_method: If the utility scores are to be generated
according to a certain ranking method, pass here the name
of that ranking method as a str (e.g. 'centered').
check_nans: Check for nan (not-a-number) values in the
evaluation results, which is an indication of
unevaluated solutions.
using_values_dtype: If True, the utility values will be returned
using the dtype of the decision values.
If False, the utility values will be returned using the dtype
of the evaluation data.
The default is False.
Returns:
Utility scores, in a PyTorch tensor.
"""
if obj_index is not None:
obj_index = self._normalize_obj_index(obj_index)
evdata = self._evdata[:, obj_index]
if check_nans:
if torch.any(torch.isnan(evdata)):
raise ValueError(
"Cannot compute the utility values, because there are solutions which are not evaluated yet."
)
if ranking_method is None:
result = evdata * self._get_objective_sign(obj_index)
else:
result = rank(evdata, ranking_method=ranking_method, higher_is_better=self._descending[obj_index])
if using_values_dtype:
result = torch.as_tensor(result, dtype=self._data.dtype, device=self._data.device)
return result
else:
if self._num_objs == 1:
return self.utility(
0, ranking_method=ranking_method, check_nans=check_nans, using_values_dtype=using_values_dtype
)
else:
return torch.stack(
[
self.utility(
j,
ranking_method=ranking_method,
check_nans=check_nans,
using_values_dtype=using_values_dtype,
)
for j in range(self._num_objs)
],
).T
@torch.no_grad()
def utils(
self,
*,
ranking_method: Optional[str] = None,
check_nans: bool = True,
using_values_dtype: bool = False,
) -> torch.Tensor:
"""
Return numeric scores for each solution, and for each objective.
Utility scores are different from evaluation results,
in the sense that utilities monotonically increase from
bad solutions to good solutions, regardless of the
objective sense.
Unlike the method called `utility(...)`, this function returns
a 2-dimensional tensor even when the problem is single-objective.
The result of this method is always a 2-dimensional tensor of
shape `(n, m)`, `n` being the number of solutions, `m` being the
number of objectives.
Args:
ranking_method: If the utility scores are to be generated
according to a certain ranking method, pass here the name
of that ranking method as a str (e.g. 'centered').
check_nans: Check for nan (not-a-number) values in the
evaluation results, which is an indication of
unevaluated solutions.
using_values_dtype: If True, the utility values will be returned
using the dtype of the decision values.
If False, the utility values will be returned using the dtype
of the evaluation data.
The default is False.
Returns:
Utility scores, in a 2-dimensional PyTorch tensor.
"""
result = self.utility(
ranking_method=ranking_method, check_nans=check_nans, using_values_dtype=using_values_dtype
)
if result.ndim == 1:
result = result.view(len(result), 1)
return result
def split(self, num_pieces: Optional[int] = None, *, max_size: Optional[int] = None) -> "SolutionBatchPieces":
"""Split this SolutionBatch into a specified number of pieces,
or into an unspecified number of pieces where the maximum
size of each piece is specified.
Args:
num_pieces: Can be provided as an integer n, which means
that the this SolutionBatch will be split to n pieces.
Alternatively, can be left as None if the user intends
to set max_size as an integer instead.
max_size: Can be provided as an integer n, which means
that this SolutionBatch will be split to multiple
pieces, each piece containing n solutions at most.
Alternatively, can be left as None if the user intends
to set num_pieces as an integer instead.
Returns:
A SolutionBatchPieces object, which behaves like a list of
SolutionBatch objects, each object in the list being a
slice view of this SolutionBatch object.
"""
return SolutionBatchPieces(self, num_pieces=num_pieces, max_size=max_size)
@torch.no_grad()
def concat(self, other: Union["SolutionBatch", Iterable]) -> "SolutionBatch":
"""Concatenate this SolutionBatch with the other(s).
In this context, concatenation means that the solutions of
this SolutionBatch and of the others are collected in one big
SolutionBatch object.
Args:
other: A SolutionBatch, or a sequence of SolutionBatch objects.
Returns:
A new SolutionBatch object which is the result of the
concatenation.
"""
if isinstance(other, SolutionBatch):
lst = [self, other]
else:
lst = [self]
lst.extend(list(other))
return SolutionBatch(merging_of=lst)
def take(self, indices: Iterable) -> "SolutionBatch":
"""Make a new SolutionBatch containing the specified solutions.
Args:
indices: A sequence of solution indices. These specified
solutions will make it to the newly made SolutionBatch.
Returns:
The new SolutionBatch.
"""
if is_sequence(indices):
return type(self)(slice_of=(self, indices))
else:
raise TypeError("Expected a sequence of solution indices, but got a `{type(indices)}`")
def take_best(self, n: int, *, obj_index: Optional[int] = None) -> "SolutionBatch":
"""Make a new SolutionBatch containing the best `n` solutions.
Args:
n: Number of solutions which will be taken.
obj_index: Objective index according to which the best ones
will be taken.
If `obj_index` is left as None and the problem is multi-
objective, then the solutions will be ranked according to
their fronts, and according how crowding they are, and then
the topmost `n` solutions will be taken.
If `obj_index` is left as None and the problem is single-
objective, then that single objective will be taken as the
ranking criterion.
Returns:
The new SolutionBatch.
"""
if obj_index is None and self._num_objs >= 2:
ranks, crowdsort_ranks = self.compute_pareto_ranks(crowdsort=True)
# Combine the ranks, such that solutions with a better crowdsort rank are weighted above solutions with the same pareto rank **only**
combined_ranks = ranks.to(torch.float) + 0.1 * crowdsort_ranks.to(torch.float) / len(self)
indices = torch.argsort(combined_ranks)[:n]
else:
indices = self.argsort(obj_index)[:n]
return type(self)(slice_of=(self, indices))
def __getitem__(self, i):
if isinstance(i, slice) or is_sequence(i) or isinstance(i, type(...)):
return type(self)(slice_of=(self, i))
else:
return Solution(parent=self, index=i)
def __len__(self):
return int(self._data.shape[0])
def __iter__(self):
for i in range(len(self)):
yield self[i]
def _get_cloned_state(self, *, memo: dict) -> dict:
with _no_grad_if_basic_dtype(self.dtype):
return deep_clone(
self.__dict__,
otherwise_deepcopy=True,
memo=memo,
)
def to(self, device: Device) -> "SolutionBatch":
"""
Get the counterpart of this SolutionBatch on the new device.
If the specified device is the device of this SolutionBatch,
then this SolutionBatch itself is returned.
If the specified device is a different device, then a clone
of this SolutionBatch on this different device is first
created, and then this new clone is returned.
Please note that the `to(...)` method is not supported when
the dtype is `object`.
Args:
device: The device on which the resulting SolutionBatch
will be stored.
Returns:
The SolutionBatch on the specified device.
"""
if isinstance(self._data, ObjectArray):
raise ValueError("The `to(...)` method is not supported when the dtype is `object`.")
device = torch.device(device)
if device == self.device:
return self
else:
new_batch = SolutionBatch(like=self, device=device, empty=True)
with torch.no_grad():
new_batch._data[:] = self._data.to(device)
new_batch._evdata[:] = self._evdata.to(device)
return new_batch
@property
def device(self) -> Device:
"""
The device in which the solutions are stored.
"""
return self._data.device
@property
def dtype(self) -> DType:
"""
The dtype of the decision values of the solutions.
This property exists as an alias for the property
`.values_dtype`.
"""
return self._data.dtype
@property
def values_dtype(self) -> DType:
"""
The dtype of the decision values of the solutions.
"""
return self._data.dtype
@property
def eval_dtype(self) -> DType:
"""
The dtype of the evaluation results and extra evaluation data
of the solutions.
"""
return self._evdata.dtype
@property
def values_shape(self) -> torch.Size:
"""
The shape of the batch's decision values tensor, as a tuple (n, l),
where `n` is the number of solutions, and `l` is the length
of a single solution.
If `dtype=None`, then there is no fixed length.
Therefore, the shape is returned as (n,).
"""
return self._data.shape
@property
def eval_shape(self) -> torch.Size:
"""
The shape of the batch's evaluation tensor, as a tuple (n, l),
where `n` is the number of solutions, and `l` is an integer
which is equal to number of objectives plus the length of the
extra evaluation data, if any.
"""
return self._evdata.shape
@property
def solution_length(self) -> Optional[int]:
"""
Get the length of a solution, if this batch is numeric.
For non-numeric batches (i.e. batches with dtype=object),
`solution_length` is given as None.
"""
if self._data.ndim == 2:
return self._data.shape[1]
else:
return None
@property
def objective_sense(self) -> ObjectiveSense:
"""
Get the objective sense(s) of this batch's associated Problem.
If the problem is single-objective, then a single string is returned.
If the problem is multi-objective, then the objective senses will be
returned in a list.
The returned string in the single-objective case, or each returned
string in the multi-objective case, is "min" or "max".
"""
if len(self.senses) == 1:
return self.senses[0]
else:
return self.senses
@property
def senses(self) -> Iterable[str]:
"""
Objective sense(s) of this batch's associated Problem.
This is a list of strings, each string being "min" or "max".
"""
def desc_to_sense(desc: bool) -> str:
return "max" if desc else "min"
return [desc_to_sense(desc) for desc in self._descending]
@staticmethod
def cat(solution_batches: Iterable) -> "SolutionBatch":
"""
Concatenate multiple SolutionBatch instances into one.
Args:
solution_batches: An Iterable of SolutionBatch objects to
concatenate.
Returns:
The result of the concatenation, as a new SolutionBatch.
"""
first = None
rest = []
for i, batch in enumerate(solution_batches):
if not isinstance(batch, SolutionBatch):
raise TypeError(f"Expected a SolutionBatch but got {repr(batch)}")
if i == 0:
first = batch
else:
rest.append(batch)
return first.concat(rest)
device: Union[str, torch.device]
property
readonly
¶
The device in which the solutions are stored.
dtype: Union[str, torch.dtype, numpy.dtype, Type]
property
readonly
¶
The dtype of the decision values of the solutions.
This property exists as an alias for the property
.values_dtype.
eval_dtype: Union[str, torch.dtype, numpy.dtype, Type]
property
readonly
¶
The dtype of the evaluation results and extra evaluation data of the solutions.
eval_shape: Size
property
readonly
¶
The shape of the batch's evaluation tensor, as a tuple (n, l),
where n is the number of solutions, and l is an integer
which is equal to number of objectives plus the length of the
extra evaluation data, if any.
evals: Tensor
property
readonly
¶
Evaluation results of the solutions, in a ReadOnlyTensor
objective_sense: Union[str, Iterable[str]]
property
readonly
¶
Get the objective sense(s) of this batch's associated Problem.
If the problem is single-objective, then a single string is returned. If the problem is multi-objective, then the objective senses will be returned in a list.
The returned string in the single-objective case, or each returned string in the multi-objective case, is "min" or "max".
senses: Iterable[str]
property
readonly
¶
Objective sense(s) of this batch's associated Problem.
This is a list of strings, each string being "min" or "max".
solution_length: Optional[int]
property
readonly
¶
Get the length of a solution, if this batch is numeric.
For non-numeric batches (i.e. batches with dtype=object),
solution_length is given as None.
values: Union[torch.Tensor, Iterable]
property
readonly
¶
Decision values of the solutions, in a read-only tensor-like object
values_dtype: Union[str, torch.dtype, numpy.dtype, Type]
property
readonly
¶
The dtype of the decision values of the solutions.
values_shape: Size
property
readonly
¶
The shape of the batch's decision values tensor, as a tuple (n, l),
where n is the number of solutions, and l is the length
of a single solution.
If dtype=None, then there is no fixed length.
Therefore, the shape is returned as (n,).
access_evals(self, obj_index=None)
¶
Get the internal mutable tensor storing the evaluations.
IMPORTANT: This method exposes the evaluation tensor of the batch
as it is, in mutable mode. It is therefore considered unsafe to rely
on this method. Before using this method, please consider using the
evals property for reading the evaluation results, and using the
set_evals(...) method which allows one to update the evaluations
without exposing any internal tensor.
When this method is used without any argument, the returned tensor
will be of shape (n, m), where n is the number of solutions,
and m is the number of objectives plus the length of extra
evaluation data.
When this method is used with an integer argument specifying an
objective index, the returned tensor will be 1-dimensional
having a length of n, where n is the number of solutions.
In this case, the returned 1-dimensional tensor will be a view
upon the evaluation results of the specified objective.
The value nan (not-a-number) means not evaluated yet.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
obj_index |
Optional[int] |
None for getting the entire 2-dimensional evaluation tensor; an objective index (as integer) for getting a 1-dimensional mutable slice of the evaluation tensor, the slice being a view upon the evaluation results regarding the specified objective. |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The mutable tensor storing the evaluation information. |
Source code in evotorch/core.py
@torch.no_grad()
def access_evals(self, obj_index: Optional[int] = None) -> torch.Tensor:
"""
Get the internal mutable tensor storing the evaluations.
IMPORTANT: This method exposes the evaluation tensor of the batch
as it is, in mutable mode. It is therefore considered unsafe to rely
on this method. Before using this method, please consider using the
`evals` property for reading the evaluation results, and using the
`set_evals(...)` method which allows one to update the evaluations
without exposing any internal tensor.
When this method is used without any argument, the returned tensor
will be of shape `(n, m)`, where `n` is the number of solutions,
and `m` is the number of objectives plus the length of extra
evaluation data.
When this method is used with an integer argument specifying an
objective index, the returned tensor will be 1-dimensional
having a length of `n`, where `n` is the number of solutions.
In this case, the returned 1-dimensional tensor will be a view
upon the evaluation results of the specified objective.
The value `nan` (not-a-number) means not evaluated yet.
Args:
obj_index: None for getting the entire 2-dimensional evaluation
tensor; an objective index (as integer) for getting a
1-dimensional mutable slice of the evaluation tensor,
the slice being a view upon the evaluation results
regarding the specified objective.
Returns:
The mutable tensor storing the evaluation information.
"""
if obj_index is None:
return self._evdata
else:
return self._evdata[:, self._normalize_obj_index(obj_index)]
access_values(self, *, keep_evals=False)
¶
Get the internal mutable tensor storing the decision values.
IMPORTANT: This method exposes the internal decision values tensor of
the batch as it is, in mutable mode. It is therefore considered unsafe
to rely on this method. Before using this method, please consider
using the values property for reading the decision values, and using
the set_values(...) method which allows one to update the decision
values without exposing any internal tensor.
IMPORTANT: The default assumption of this method is that the tensor
is requested for modification purposes. Therefore, by default, as soon
as this method is called, the evaluation results of the solutions will
be cleared (where clearing means that the evaluation results will be
filled with NaNs).
The reasoning behind this default behavior is to prevent the modified
solutions from having outdated evaluation results.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
keep_evals |
bool |
If set as False, the evaluation data of the solutions
will be cleared (i.e. will be filled with |
False |
Returns:
| Type | Description |
|---|---|
Union[torch.Tensor, evotorch.tools.objectarray.ObjectArray] |
The mutable tensor storing the decision values. |
Source code in evotorch/core.py
@torch.no_grad()
def access_values(self, *, keep_evals: bool = False) -> Union[torch.Tensor, ObjectArray]:
"""
Get the internal mutable tensor storing the decision values.
IMPORTANT: This method exposes the internal decision values tensor of
the batch as it is, in mutable mode. It is therefore considered unsafe
to rely on this method. Before using this method, please consider
using the `values` property for reading the decision values, and using
the `set_values(...)` method which allows one to update the decision
values without exposing any internal tensor.
IMPORTANT: The default assumption of this method is that the tensor
is requested for modification purposes. Therefore, by default, as soon
as this method is called, the evaluation results of the solutions will
be cleared (where clearing means that the evaluation results will be
filled with `NaN`s).
The reasoning behind this default behavior is to prevent the modified
solutions from having outdated evaluation results.
Args:
keep_evals: If set as False, the evaluation data of the solutions
will be cleared (i.e. will be filled with `NaN`s).
If set as True, the existing evaluation data will be kept.
Returns:
The mutable tensor storing the decision values.
"""
if not keep_evals:
self.forget_evals()
return self._data
arg_pareto_sort(self, crowdsort=True)
¶
Pareto-sort the solutions in the batch.
The result is a namedtuple consisting of two elements:
fronts and ranks.
Let us assume that we have 5 solutions, and after a
pareto-sorting they ended up in this order:
front 0 (best front) : solution 1, solution 2
front 1 : solution 0, solution 4
front 2 (worst front): solution 3
Considering the example ordering above, the returned ParetoInfo instance looks like this:
ParetoInfo(
fronts=[[1, 2], [0, 4], [3]],
ranks=tensor([1, 0, 0, 2, 1])
)
where fronts stores the solution indices grouped by
pareto fronts; and ranks stores, as a tensor of int64,
the pareto rank for each solution (where 0 means best
rank).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
crowdsort |
bool |
If given as True, each front in itself will be sorted from the least crowding solution to the most crowding solution. If given as False, there will be no crowd-sorting. |
True |
Returns:
| Type | Description |
|---|---|
ParetoInfo |
A ParetoInfo instance |
Source code in evotorch/core.py
@torch.no_grad()
def arg_pareto_sort(self, crowdsort: bool = True) -> ParetoInfo:
"""
Pareto-sort the solutions in the batch.
The result is a namedtuple consisting of two elements:
`fronts` and `ranks`.
Let us assume that we have 5 solutions, and after a
pareto-sorting they ended up in this order:
front 0 (best front) : solution 1, solution 2
front 1 : solution 0, solution 4
front 2 (worst front): solution 3
Considering the example ordering above, the returned
ParetoInfo instance looks like this:
ParetoInfo(
fronts=[[1, 2], [0, 4], [3]],
ranks=tensor([1, 0, 0, 2, 1])
)
where `fronts` stores the solution indices grouped by
pareto fronts; and `ranks` stores, as a tensor of int64,
the pareto rank for each solution (where 0 means best
rank).
Args:
crowdsort: If given as True, each front in itself
will be sorted from the least crowding solution
to the most crowding solution.
If given as False, there will be no crowd-sorting.
Returns:
A ParetoInfo instance
"""
utils = self.utils()
fronts, ranks = _pareto_sort(utils, crowdsort)
return ParetoInfo(fronts=fronts, ranks=ranks)
argbest(self, obj_index=None)
¶
Return the best solution's index
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
obj_index |
Optional[int] |
The objective index. Can be passed as None if the problem is single-objective. Otherwise, expected as an int. |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The index of the best solution. |
Source code in evotorch/core.py
@torch.no_grad()
def argbest(self, obj_index: Optional[int] = None) -> torch.Tensor:
"""Return the best solution's index
Args:
obj_index: The objective index. Can be passed as None
if the problem is single-objective. Otherwise,
expected as an int.
Returns:
The index of the best solution.
"""
obj_index = self._optionally_get_obj_index(obj_index)
descending = self._descending[obj_index]
argf = torch.argmax if descending else torch.argmin
return argf(self._evdata[:, obj_index])
argsort(self, obj_index=None)
¶
Return the indices of solutions, sorted from best to worst.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
obj_index |
Optional[int] |
The objective index. Can be passed as None if the problem is single-objective. Otherwise, expected as an int. |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
A PyTorch tensor, containing the solution indices, sorted from the best solution to the worst. |
Source code in evotorch/core.py
@torch.no_grad()
def argsort(self, obj_index: Optional[int] = None) -> torch.Tensor:
"""Return the indices of solutions, sorted from best to worst.
Args:
obj_index: The objective index. Can be passed as None
if the problem is single-objective. Otherwise,
expected as an int.
Returns:
A PyTorch tensor, containing the solution indices,
sorted from the best solution to the worst.
"""
obj_index = self._optionally_get_obj_index(obj_index)
descending = self._descending[obj_index]
ev_col = self._evdata[:, obj_index]
return torch.argsort(ev_col, descending=descending)
argworst(self, obj_index=None)
¶
Return the worst solution's index
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
obj_index |
Optional[int] |
The objective index. Can be passed as None if the problem is single-objective. Otherwise, expected as an int. |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The index of the worst solution. |
Source code in evotorch/core.py
@torch.no_grad()
def argworst(self, obj_index: Optional[int] = None) -> torch.Tensor:
"""Return the worst solution's index
Args:
obj_index: The objective index. Can be passed as None
if the problem is single-objective. Otherwise,
expected as an int.
Returns:
The index of the worst solution.
"""
obj_index = self._optionally_get_obj_index(obj_index)
descending = self._descending[obj_index]
argf = torch.argmin if descending else torch.argmax
return argf(self._evdata[:, obj_index])
cat(solution_batches)
staticmethod
¶
Concatenate multiple SolutionBatch instances into one.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
solution_batches |
Iterable |
An Iterable of SolutionBatch objects to concatenate. |
required |
Returns:
| Type | Description |
|---|---|
SolutionBatch |
The result of the concatenation, as a new SolutionBatch. |
Source code in evotorch/core.py
@staticmethod
def cat(solution_batches: Iterable) -> "SolutionBatch":
"""
Concatenate multiple SolutionBatch instances into one.
Args:
solution_batches: An Iterable of SolutionBatch objects to
concatenate.
Returns:
The result of the concatenation, as a new SolutionBatch.
"""
first = None
rest = []
for i, batch in enumerate(solution_batches):
if not isinstance(batch, SolutionBatch):
raise TypeError(f"Expected a SolutionBatch but got {repr(batch)}")
if i == 0:
first = batch
else:
rest.append(batch)
return first.concat(rest)
compute_pareto_ranks(self, crowdsort=True)
¶
Compute the pareto-ranks of the solutions in the batch.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
crowdsort |
bool |
If given as True, each front in itself will be sorted from the least crowding solution to the most crowding solution. If given as False, there will be no crowd-sorting. |
True |
Returns:
| Type | Description |
|---|---|
ranks (torch.Tensor) |
The computed pareto ranks, of shape [num_samples,] Ranks are encoded in the form 'lowest is best'. So solutions in the pareto front will be assigned rank 0, solutions in the next non-dominated set (excluding the pareto front) will be assigned rank 1, and so on. crowdsort_ranks (Optional[torch.Tensor]): The computed crowd sort ranks, only returned if crowdsort=True. Otherwise, None is returned. Solutions within a front are assigned a crowding score, as described in the above paper. Then, the solution with the best crowding score within a front of size K is assigned rank 0, and the solution with the worst crowding score within a front is assigned rank K-1. |
Source code in evotorch/core.py
@torch.no_grad()
def compute_pareto_ranks(self, crowdsort: bool = True) -> Tuple[torch.Tensor, torch.Tensor]:
"""
Compute the pareto-ranks of the solutions in the batch.
Args:
crowdsort: If given as True, each front in itself
will be sorted from the least crowding solution
to the most crowding solution.
If given as False, there will be no crowd-sorting.
Returns:
ranks (torch.Tensor): The computed pareto ranks, of shape [num_samples,]
Ranks are encoded in the form 'lowest is best'. So solutions in the pareto front will be assigned rank 0,
solutions in the next non-dominated set (excluding the pareto front) will be assigned rank 1, and so on.
crowdsort_ranks (Optional[torch.Tensor]): The computed crowd sort ranks, only returned if crowdsort=True. Otherwise, None is returned.
Solutions within a front are assigned a crowding score, as described in the above paper.
Then, the solution with the best crowding score within a front of size K is assigned rank 0, and the solution with the
worst crowding score within a front is assigned rank K-1.
"""
utils = self.utils()
ranks, crowdsort_ranks = _compute_pareto_ranks(utils, crowdsort)
return ranks, crowdsort_ranks
concat(self, other)
¶
Concatenate this SolutionBatch with the other(s).
In this context, concatenation means that the solutions of this SolutionBatch and of the others are collected in one big SolutionBatch object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
other |
Union[SolutionBatch, Iterable] |
A SolutionBatch, or a sequence of SolutionBatch objects. |
required |
Returns:
| Type | Description |
|---|---|
SolutionBatch |
A new SolutionBatch object which is the result of the concatenation. |
Source code in evotorch/core.py
@torch.no_grad()
def concat(self, other: Union["SolutionBatch", Iterable]) -> "SolutionBatch":
"""Concatenate this SolutionBatch with the other(s).
In this context, concatenation means that the solutions of
this SolutionBatch and of the others are collected in one big
SolutionBatch object.
Args:
other: A SolutionBatch, or a sequence of SolutionBatch objects.
Returns:
A new SolutionBatch object which is the result of the
concatenation.
"""
if isinstance(other, SolutionBatch):
lst = [self, other]
else:
lst = [self]
lst.extend(list(other))
return SolutionBatch(merging_of=lst)
forget_evals(self, *, solutions=None)
¶
Forget the evaluations of the solutions.
The evaluation results will be cleared, which means that they will
be filled with NaNs.
Source code in evotorch/core.py
@torch.no_grad()
def forget_evals(self, *, solutions: MaybeIndicesOrSlice = None):
"""
Forget the evaluations of the solutions.
The evaluation results will be cleared, which means that they will
be filled with `NaN`s.
"""
if solutions is None:
solutions = slice(None, None, None)
self._evdata[solutions, :] = float("nan")
set_evals(self, evals, eval_data=None, *, solutions=None)
¶
Set the evaluations of the solutions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
evals |
Tensor |
A numeric tensor which contains the evaluation results.
Acceptable shapes are as follows:
|
required |
eval_data |
Optional[torch.Tensor] |
To be used only when the problem has extra evaluation
data. Optionally, one can pass the extra evaluation data
separately via this argument (instead of jointly through
a single tensor via |
None |
solutions |
Union[int, Iterable[int], slice] |
Optionally a list of integer indices or an instance
of |
None |
Exceptions:
| Type | Description |
|---|---|
ValueError |
if the given tensor has an incompatible shape. |
Source code in evotorch/core.py
@torch.no_grad()
def set_evals(
self,
evals: torch.Tensor,
eval_data: Optional[torch.Tensor] = None,
*,
solutions: MaybeIndicesOrSlice = None,
):
"""
Set the evaluations of the solutions.
Args:
evals: A numeric tensor which contains the evaluation results.
Acceptable shapes are as follows:
`(n,)` only to be used for single-objective problems, sets
the evaluation results of the target `n` solutions, and clears
(where clearing means to fill with NaN values)
extra evaluation data (if the problem has allocations for such
extra evaluation data);
`(n,m)` where `m` is the number of objectives, sets the
evaluation results of the target `n` solutions, and clears
their extra evaluation data;
`(n,m+q)` where `m` is the number of objectives and `q` is the
length of extra evaluation data, sets the evaluation result
and extra data of the target `n` solutions.
eval_data: To be used only when the problem has extra evaluation
data. Optionally, one can pass the extra evaluation data
separately via this argument (instead of jointly through
a single tensor via `evals`).
The expected shape of this tensor is `(n,q)` where `n`
is the number of solutions and `q` is the length of the
extra evaluation data.
solutions: Optionally a list of integer indices or an instance
of `slice(...)`, to be used if one wishes to set the
evaluations of only some of the solutions.
Raises:
ValueError: if the given tensor has an incompatible shape.
"""
if solutions is None:
solutions = slice(None, None, None)
num_solutions = self._evdata.shape[0]
elif isinstance(solutions, slice):
num_solutions = self._evdata[solutions].shape[0]
elif is_sequence(solutions):
num_solutions = len(solutions)
total_eval_width = self._evdata.shape[1]
num_objs = self._num_objs
num_data = total_eval_width - num_objs
if evals.ndim == 1:
if num_objs != 1:
raise ValueError(
f"The method `set_evals(...)` was given a 1-dimensional tensor."
f" However, the number of objectives of the problem at hand is {num_objs}, not 1."
f" 1-dimensional evaluation tensors can only be accepted if the problem"
f" has one objective."
)
evals = evals.reshape(-1, 1)
elif evals.ndim == 2:
pass # nothing to do here
else:
if num_objs == 1:
raise ValueError(
f"The method `set_evals(...)` received a tensor with {evals.ndim} dimensions."
f" Since the problem at hand has only one objective,"
f" 1-dimensional or 2-dimensional tensors are acceptable, but not {evals.ndim}-dimensional ones."
)
else:
raise ValueError(
f"The method `set_evals(...)` received a tensor with {evals.ndim} dimensions."
f" Since the problem at hand has more than one objective (there are {num_objs} objectives),"
f" only 2-dimensional tensors are acceptable, not {evals.ndim}-dimensional ones."
)
[nrows, ncols] = evals.shape
if nrows != num_solutions:
raise ValueError(
f"Trying to set the evaluations of {num_solutions} solutions, but the given tensor has {nrows} rows."
)
if eval_data is not None:
if eval_data.ndim != 2:
raise ValueError(
f"The `eval_data` argument was expected as a 2-dimensional tensor."
f" However, the shape of the given `eval_data` is {eval_data.shape}."
)
if eval_data.shape[1] != num_data:
raise ValueError(
f"The `eval_data` argument was expected to have {num_data} columns."
f" However, the received `eval_data` has the shape: {eval_data.shape}."
)
if ncols != num_objs:
raise ValueError(
f"The method `set_evals(...)` was used with `evals` and `eval_data` arguments."
f" When both of these arguments are provided, `evals` is expected either as a 1-dimensional tensor"
f" (for single-objective cases only), or as a tensor of shape (n, m) where n is the number of"
f" solutions, and m is the number of objectives."
f" However, while the problem at hand has {num_objs} objectives,"
f" the `evals` tensor has {ncols} columns."
)
if evals.shape[0] != eval_data.shape[0]:
raise ValueError(
f"The provided `evals` and `eval_data` tensors have incompatible shapes."
f" Shape of `evals`: {evals.shape},"
f" shape of `eval_data`: {eval_data.shape}."
)
self._evdata[solutions, :] = torch.hstack([evals, eval_data])
else:
if ncols == num_objs:
self._evdata[solutions, :num_objs] = evals
self._evdata[solutions, num_objs:] = float("nan")
elif ncols == total_eval_width:
self._evdata[solutions, :] = evals
else:
raise ValueError(
f"The method `set_evals(...)` received a tensor with {ncols} columns, which is incompatible."
f" Acceptable number of columns are: {num_objs}"
f" (for setting only the objective-associated evaluations and leave extra evaluation data as NaN), or"
f" {total_eval_width} (for setting both objective-associated evaluations and extra evaluation data)."
)
set_values(self, values, *, solutions=None)
¶
Set the decision values of the solutions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
values |
Any |
New decision values. |
required |
solutions |
Union[int, Iterable[int], slice] |
Optionally a list of integer indices or an instance
of |
None |
Source code in evotorch/core.py
@torch.no_grad()
def set_values(self, values: Any, *, solutions: MaybeIndicesOrSlice = None):
"""
Set the decision values of the solutions.
Args:
values: New decision values.
solutions: Optionally a list of integer indices or an instance
of `slice(...)`, to be used if one wishes to set the
decision values of only some of the solutions.
"""
if solutions is None:
solutions = slice(None, None, None)
self._data[solutions] = values
self._evdata[solutions] = float("nan")
split(self, num_pieces=None, *, max_size=None)
¶
Split this SolutionBatch into a specified number of pieces, or into an unspecified number of pieces where the maximum size of each piece is specified.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
num_pieces |
Optional[int] |
Can be provided as an integer n, which means that the this SolutionBatch will be split to n pieces. Alternatively, can be left as None if the user intends to set max_size as an integer instead. |
None |
max_size |
Optional[int] |
Can be provided as an integer n, which means that this SolutionBatch will be split to multiple pieces, each piece containing n solutions at most. Alternatively, can be left as None if the user intends to set num_pieces as an integer instead. |
None |
Returns:
| Type | Description |
|---|---|
SolutionBatchPieces |
A SolutionBatchPieces object, which behaves like a list of SolutionBatch objects, each object in the list being a slice view of this SolutionBatch object. |
Source code in evotorch/core.py
def split(self, num_pieces: Optional[int] = None, *, max_size: Optional[int] = None) -> "SolutionBatchPieces":
"""Split this SolutionBatch into a specified number of pieces,
or into an unspecified number of pieces where the maximum
size of each piece is specified.
Args:
num_pieces: Can be provided as an integer n, which means
that the this SolutionBatch will be split to n pieces.
Alternatively, can be left as None if the user intends
to set max_size as an integer instead.
max_size: Can be provided as an integer n, which means
that this SolutionBatch will be split to multiple
pieces, each piece containing n solutions at most.
Alternatively, can be left as None if the user intends
to set num_pieces as an integer instead.
Returns:
A SolutionBatchPieces object, which behaves like a list of
SolutionBatch objects, each object in the list being a
slice view of this SolutionBatch object.
"""
return SolutionBatchPieces(self, num_pieces=num_pieces, max_size=max_size)
take(self, indices)
¶
Make a new SolutionBatch containing the specified solutions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
indices |
Iterable |
A sequence of solution indices. These specified solutions will make it to the newly made SolutionBatch. |
required |
Returns:
| Type | Description |
|---|---|
SolutionBatch |
The new SolutionBatch. |
Source code in evotorch/core.py
def take(self, indices: Iterable) -> "SolutionBatch":
"""Make a new SolutionBatch containing the specified solutions.
Args:
indices: A sequence of solution indices. These specified
solutions will make it to the newly made SolutionBatch.
Returns:
The new SolutionBatch.
"""
if is_sequence(indices):
return type(self)(slice_of=(self, indices))
else:
raise TypeError("Expected a sequence of solution indices, but got a `{type(indices)}`")
take_best(self, n, *, obj_index=None)
¶
Make a new SolutionBatch containing the best n solutions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n |
int |
Number of solutions which will be taken. |
required |
obj_index |
Optional[int] |
Objective index according to which the best ones
will be taken.
If |
None |
Returns:
| Type | Description |
|---|---|
SolutionBatch |
The new SolutionBatch. |
Source code in evotorch/core.py
def take_best(self, n: int, *, obj_index: Optional[int] = None) -> "SolutionBatch":
"""Make a new SolutionBatch containing the best `n` solutions.
Args:
n: Number of solutions which will be taken.
obj_index: Objective index according to which the best ones
will be taken.
If `obj_index` is left as None and the problem is multi-
objective, then the solutions will be ranked according to
their fronts, and according how crowding they are, and then
the topmost `n` solutions will be taken.
If `obj_index` is left as None and the problem is single-
objective, then that single objective will be taken as the
ranking criterion.
Returns:
The new SolutionBatch.
"""
if obj_index is None and self._num_objs >= 2:
ranks, crowdsort_ranks = self.compute_pareto_ranks(crowdsort=True)
# Combine the ranks, such that solutions with a better crowdsort rank are weighted above solutions with the same pareto rank **only**
combined_ranks = ranks.to(torch.float) + 0.1 * crowdsort_ranks.to(torch.float) / len(self)
indices = torch.argsort(combined_ranks)[:n]
else:
indices = self.argsort(obj_index)[:n]
return type(self)(slice_of=(self, indices))
to(self, device)
¶
Get the counterpart of this SolutionBatch on the new device.
If the specified device is the device of this SolutionBatch, then this SolutionBatch itself is returned. If the specified device is a different device, then a clone of this SolutionBatch on this different device is first created, and then this new clone is returned.
Please note that the to(...) method is not supported when
the dtype is object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
device |
Union[str, torch.device] |
The device on which the resulting SolutionBatch will be stored. |
required |
Returns:
| Type | Description |
|---|---|
SolutionBatch |
The SolutionBatch on the specified device. |
Source code in evotorch/core.py
def to(self, device: Device) -> "SolutionBatch":
"""
Get the counterpart of this SolutionBatch on the new device.
If the specified device is the device of this SolutionBatch,
then this SolutionBatch itself is returned.
If the specified device is a different device, then a clone
of this SolutionBatch on this different device is first
created, and then this new clone is returned.
Please note that the `to(...)` method is not supported when
the dtype is `object`.
Args:
device: The device on which the resulting SolutionBatch
will be stored.
Returns:
The SolutionBatch on the specified device.
"""
if isinstance(self._data, ObjectArray):
raise ValueError("The `to(...)` method is not supported when the dtype is `object`.")
device = torch.device(device)
if device == self.device:
return self
else:
new_batch = SolutionBatch(like=self, device=device, empty=True)
with torch.no_grad():
new_batch._data[:] = self._data.to(device)
new_batch._evdata[:] = self._evdata.to(device)
return new_batch
utility(self, obj_index=None, *, ranking_method=None, check_nans=True, using_values_dtype=False)
¶
Return numeric scores for each solution.
Utility scores are different from evaluation results, in the sense that utilities monotonically increase from bad solutions to good solutions, regardless of the objective sense.
If ranking method is passed as None: if the objective sense is 'max', the evaluation results are returned as the utility scores; otherwise, if the objective sense is 'min', the evaluation results multiplied by -1 are returned as the utility scores.
If the name of a ranking method is given (e.g. 'centered'): then the solutions are ranked (best solutions having the highest rank), and those ranks are returned as the utility scores.
If an objective index is not provided: (i.e. passed as None) if the problem is multi-objective, the utility scores for each objective is given, in a tensor shaped (n, m), n being the number of solutions and m being the number of objectives; otherwise, if the problem is single-objective, the utility scores for each objective is given in a 1-dimensional tensor of length n, n being the number of solutions.
If an objective index is provided as an int: the utility scores are returned in a 1-dimensional tensor of length n, n being the number of solutions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
obj_index |
Optional[int] |
Expected as None, or as an integer. In the single-objective case, None is equivalent to 0. In the multi-objective case, None means "for each objective". |
None |
ranking_method |
Optional[str] |
If the utility scores are to be generated according to a certain ranking method, pass here the name of that ranking method as a str (e.g. 'centered'). |
None |
check_nans |
bool |
Check for nan (not-a-number) values in the evaluation results, which is an indication of unevaluated solutions. |
True |
using_values_dtype |
bool |
If True, the utility values will be returned using the dtype of the decision values. If False, the utility values will be returned using the dtype of the evaluation data. The default is False. |
False |
Returns:
| Type | Description |
|---|---|
Tensor |
Utility scores, in a PyTorch tensor. |
Source code in evotorch/core.py
@torch.no_grad()
def utility(
self,
obj_index: Optional[int] = None,
*,
ranking_method: Optional[str] = None,
check_nans: bool = True,
using_values_dtype: bool = False,
) -> torch.Tensor:
"""
Return numeric scores for each solution.
Utility scores are different from evaluation results,
in the sense that utilities monotonically increase from
bad solutions to good solutions, regardless of the
objective sense.
**If ranking method is passed as None:**
if the objective sense is 'max', the evaluation results are returned
as the utility scores; otherwise, if the objective sense is 'min',
the evaluation results multiplied by -1 are returned as the
utility scores.
**If the name of a ranking method is given** (e.g. 'centered'):
then the solutions are ranked (best solutions having the
highest rank), and those ranks are returned as the utility
scores.
**If an objective index is not provided:** (i.e. passed as None)
if the problem is multi-objective, the utility scores
for each objective is given, in a tensor shaped (n, m),
n being the number of solutions and m being the number
of objectives; otherwise, if the problem is single-objective,
the utility scores for each objective is given in a 1-dimensional
tensor of length n, n being the number of solutions.
**If an objective index is provided as an int:**
the utility scores are returned in a 1-dimensional tensor
of length n, n being the number of solutions.
Args:
obj_index: Expected as None, or as an integer.
In the single-objective case, None is equivalent to 0.
In the multi-objective case, None means "for each
objective".
ranking_method: If the utility scores are to be generated
according to a certain ranking method, pass here the name
of that ranking method as a str (e.g. 'centered').
check_nans: Check for nan (not-a-number) values in the
evaluation results, which is an indication of
unevaluated solutions.
using_values_dtype: If True, the utility values will be returned
using the dtype of the decision values.
If False, the utility values will be returned using the dtype
of the evaluation data.
The default is False.
Returns:
Utility scores, in a PyTorch tensor.
"""
if obj_index is not None:
obj_index = self._normalize_obj_index(obj_index)
evdata = self._evdata[:, obj_index]
if check_nans:
if torch.any(torch.isnan(evdata)):
raise ValueError(
"Cannot compute the utility values, because there are solutions which are not evaluated yet."
)
if ranking_method is None:
result = evdata * self._get_objective_sign(obj_index)
else:
result = rank(evdata, ranking_method=ranking_method, higher_is_better=self._descending[obj_index])
if using_values_dtype:
result = torch.as_tensor(result, dtype=self._data.dtype, device=self._data.device)
return result
else:
if self._num_objs == 1:
return self.utility(
0, ranking_method=ranking_method, check_nans=check_nans, using_values_dtype=using_values_dtype
)
else:
return torch.stack(
[
self.utility(
j,
ranking_method=ranking_method,
check_nans=check_nans,
using_values_dtype=using_values_dtype,
)
for j in range(self._num_objs)
],
).T
utils(self, *, ranking_method=None, check_nans=True, using_values_dtype=False)
¶
Return numeric scores for each solution, and for each objective. Utility scores are different from evaluation results, in the sense that utilities monotonically increase from bad solutions to good solutions, regardless of the objective sense.
Unlike the method called utility(...), this function returns
a 2-dimensional tensor even when the problem is single-objective.
The result of this method is always a 2-dimensional tensor of
shape (n, m), n being the number of solutions, m being the
number of objectives.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ranking_method |
Optional[str] |
If the utility scores are to be generated according to a certain ranking method, pass here the name of that ranking method as a str (e.g. 'centered'). |
None |
check_nans |
bool |
Check for nan (not-a-number) values in the evaluation results, which is an indication of unevaluated solutions. |
True |
using_values_dtype |
bool |
If True, the utility values will be returned using the dtype of the decision values. If False, the utility values will be returned using the dtype of the evaluation data. The default is False. |
False |
Returns:
| Type | Description |
|---|---|
Tensor |
Utility scores, in a 2-dimensional PyTorch tensor. |
Source code in evotorch/core.py
@torch.no_grad()
def utils(
self,
*,
ranking_method: Optional[str] = None,
check_nans: bool = True,
using_values_dtype: bool = False,
) -> torch.Tensor:
"""
Return numeric scores for each solution, and for each objective.
Utility scores are different from evaluation results,
in the sense that utilities monotonically increase from
bad solutions to good solutions, regardless of the
objective sense.
Unlike the method called `utility(...)`, this function returns
a 2-dimensional tensor even when the problem is single-objective.
The result of this method is always a 2-dimensional tensor of
shape `(n, m)`, `n` being the number of solutions, `m` being the
number of objectives.
Args:
ranking_method: If the utility scores are to be generated
according to a certain ranking method, pass here the name
of that ranking method as a str (e.g. 'centered').
check_nans: Check for nan (not-a-number) values in the
evaluation results, which is an indication of
unevaluated solutions.
using_values_dtype: If True, the utility values will be returned
using the dtype of the decision values.
If False, the utility values will be returned using the dtype
of the evaluation data.
The default is False.
Returns:
Utility scores, in a 2-dimensional PyTorch tensor.
"""
result = self.utility(
ranking_method=ranking_method, check_nans=check_nans, using_values_dtype=using_values_dtype
)
if result.ndim == 1:
result = result.view(len(result), 1)
return result
SolutionBatchPieces (Sequence)
¶
A collection of SolutionBatch slice views.
An instance of this class behaves like a read-only collection of SolutionBatch objects (each being a sliced view of a bigger SolutionBatch).
Source code in evotorch/core.py
class SolutionBatchPieces(Sequence):
"""A collection of SolutionBatch slice views.
An instance of this class behaves like a read-only collection of
SolutionBatch objects (each being a sliced view of a bigger
SolutionBatch).
"""
@torch.no_grad()
def __init__(self, batch: SolutionBatch, *, num_pieces: Optional[int] = None, max_size: Optional[int] = None):
"""
`__init__(...)`: Initialize the SolutionBatchPieces.
Args:
batch: The SolutionBatch which will be split into
multiple SolutionBatch views.
Each view itself is a SolutionBatch object,
but not independent, meaning that any modification
done to a SolutionBatch view will reflect on this
main batch.
num_pieces: Can be provided as an integer n, which means
that the main SolutionBatch will be split to n pieces.
Alternatively, can be left as None if the user intends
to set max_size as an integer instead.
max_size: Can be provided as an integer n, which means
that the main SolutionBatch will be split to multiple
pieces, each piece containing n solutions at most.
Alternatively, can be left as None if the user intends
to set num_pieces as an integer instead.
"""
self._batch = batch
self._pieces: List[SolutionBatch] = []
self._piece_sizes: List[int] = []
self._piece_slices: List[Tuple[int, int]] = []
total_size = len(self._batch)
if max_size is None and num_pieces is not None:
num_pieces = int(num_pieces)
# divide to pieces
base_size = total_size // num_pieces
rest = total_size - (base_size * num_pieces)
self._piece_sizes = [base_size] * num_pieces
for i in range(rest):
self._piece_sizes[i] += 1
elif max_size is not None and num_pieces is None:
max_size = int(max_size)
# divide to pieces
num_pieces = math.ceil(total_size / max_size)
current_total = 0
for i in range(num_pieces):
if current_total + max_size > total_size:
self._piece_sizes.append(total_size - current_total)
else:
self._piece_sizes.append(max_size)
current_total += max_size
elif max_size is not None and num_pieces is not None:
raise ValueError("Expected either max_size or num_pieces, received both.")
elif max_size is None and num_pieces is None:
raise ValueError("Expected either max_size or num_pieces, received none.")
current_begin = 0
for size in self._piece_sizes:
current_end = current_begin + size
self._piece_slices.append((current_begin, current_end))
current_begin = current_end
for slice_begin, slice_end in self._piece_slices:
self._pieces.append(self._batch[slice_begin:slice_end])
def __len__(self) -> int:
return len(self._pieces)
def __getitem__(self, i: Union[int, slice]) -> SolutionBatch:
return self._pieces[i]
def iter_with_indices(self):
"""Iterate over each `(piece, (i_begin, i_end))`
where `piece` is a SolutionBatch view, `i_begin` is the beginning
index of the SolutionBatch view in the main batch, `j_begin` is the
ending index (exclusive) of the SolutionBatch view in the main batch.
"""
for i in range(len(self._pieces)):
yield self._pieces[i], self._piece_slices[i]
def indices_of(self, n) -> tuple:
"""Get `(i_begin, i_end)` for the n-th piece
(i.e. the n-th sliced view of the main SolutionBatch)
where `i_begin` is the beginning index of the n-th piece,
`i_end` is the (exclusive) ending index of the n-th piece.
Args:
n: Specifies the index of the queried SolutionBatch view.
Returns:
Beginning and ending indices of the SolutionBatch view,
in a tuple.
"""
return self._piece_slices[n]
@property
def batch(self) -> SolutionBatch:
"""Get the main SolutionBatch object, in its non-split form"""
return self._batch
def _to_string(self) -> str:
f = io.StringIO()
print(f"<{type(self).__name__}", file=f)
n = len(self._pieces)
for i, piece in enumerate(self._pieces):
print(f" {piece}", end="", file=f)
if (i + 1) == n:
print(file=f)
else:
print(",", file=f)
print(">", file=f)
f.seek(0)
return f.read()
def __str__(self) -> str:
return self._to_string()
def __repr__(self) -> str:
return self._to_string()
batch: SolutionBatch
property
readonly
¶
Get the main SolutionBatch object, in its non-split form
__init__(self, batch, *, num_pieces=None, max_size=None)
special
¶
__init__(...): Initialize the SolutionBatchPieces.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
batch |
SolutionBatch |
The SolutionBatch which will be split into multiple SolutionBatch views. Each view itself is a SolutionBatch object, but not independent, meaning that any modification done to a SolutionBatch view will reflect on this main batch. |
required |
num_pieces |
Optional[int] |
Can be provided as an integer n, which means that the main SolutionBatch will be split to n pieces. Alternatively, can be left as None if the user intends to set max_size as an integer instead. |
None |
max_size |
Optional[int] |
Can be provided as an integer n, which means that the main SolutionBatch will be split to multiple pieces, each piece containing n solutions at most. Alternatively, can be left as None if the user intends to set num_pieces as an integer instead. |
None |
Source code in evotorch/core.py
@torch.no_grad()
def __init__(self, batch: SolutionBatch, *, num_pieces: Optional[int] = None, max_size: Optional[int] = None):
"""
`__init__(...)`: Initialize the SolutionBatchPieces.
Args:
batch: The SolutionBatch which will be split into
multiple SolutionBatch views.
Each view itself is a SolutionBatch object,
but not independent, meaning that any modification
done to a SolutionBatch view will reflect on this
main batch.
num_pieces: Can be provided as an integer n, which means
that the main SolutionBatch will be split to n pieces.
Alternatively, can be left as None if the user intends
to set max_size as an integer instead.
max_size: Can be provided as an integer n, which means
that the main SolutionBatch will be split to multiple
pieces, each piece containing n solutions at most.
Alternatively, can be left as None if the user intends
to set num_pieces as an integer instead.
"""
self._batch = batch
self._pieces: List[SolutionBatch] = []
self._piece_sizes: List[int] = []
self._piece_slices: List[Tuple[int, int]] = []
total_size = len(self._batch)
if max_size is None and num_pieces is not None:
num_pieces = int(num_pieces)
# divide to pieces
base_size = total_size // num_pieces
rest = total_size - (base_size * num_pieces)
self._piece_sizes = [base_size] * num_pieces
for i in range(rest):
self._piece_sizes[i] += 1
elif max_size is not None and num_pieces is None:
max_size = int(max_size)
# divide to pieces
num_pieces = math.ceil(total_size / max_size)
current_total = 0
for i in range(num_pieces):
if current_total + max_size > total_size:
self._piece_sizes.append(total_size - current_total)
else:
self._piece_sizes.append(max_size)
current_total += max_size
elif max_size is not None and num_pieces is not None:
raise ValueError("Expected either max_size or num_pieces, received both.")
elif max_size is None and num_pieces is None:
raise ValueError("Expected either max_size or num_pieces, received none.")
current_begin = 0
for size in self._piece_sizes:
current_end = current_begin + size
self._piece_slices.append((current_begin, current_end))
current_begin = current_end
for slice_begin, slice_end in self._piece_slices:
self._pieces.append(self._batch[slice_begin:slice_end])
indices_of(self, n)
¶
Get (i_begin, i_end) for the n-th piece
(i.e. the n-th sliced view of the main SolutionBatch)
where i_begin is the beginning index of the n-th piece,
i_end is the (exclusive) ending index of the n-th piece.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n |
Specifies the index of the queried SolutionBatch view. |
required |
Returns:
| Type | Description |
|---|---|
tuple |
Beginning and ending indices of the SolutionBatch view, in a tuple. |
Source code in evotorch/core.py
def indices_of(self, n) -> tuple:
"""Get `(i_begin, i_end)` for the n-th piece
(i.e. the n-th sliced view of the main SolutionBatch)
where `i_begin` is the beginning index of the n-th piece,
`i_end` is the (exclusive) ending index of the n-th piece.
Args:
n: Specifies the index of the queried SolutionBatch view.
Returns:
Beginning and ending indices of the SolutionBatch view,
in a tuple.
"""
return self._piece_slices[n]
iter_with_indices(self)
¶
Iterate over each (piece, (i_begin, i_end))
where piece is a SolutionBatch view, i_begin is the beginning
index of the SolutionBatch view in the main batch, j_begin is the
ending index (exclusive) of the SolutionBatch view in the main batch.
Source code in evotorch/core.py
def iter_with_indices(self):
"""Iterate over each `(piece, (i_begin, i_end))`
where `piece` is a SolutionBatch view, `i_begin` is the beginning
index of the SolutionBatch view in the main batch, `j_begin` is the
ending index (exclusive) of the SolutionBatch view in the main batch.
"""
for i in range(len(self._pieces)):
yield self._pieces[i], self._piece_slices[i]
decorators
¶
Module defining decorators for evotorch.
expects_ndim(*expected_ndims, *, allow_smaller_ndim=False, randomness='error')
¶
Decorator to declare the number of dimensions for each positional argument.
Let us imagine that we have a function f(a, b), where a and b are
PyTorch tensors. Let us also imagine that the function f is implemented
in such a way that a is assumed to be a 2-dimensional tensor, and b
is assumed to be a 1-dimensional tensor. In this case, the function f
can be decorated as follows:
from evotorch.decorators import expects_ndim
@expects_ndim(2, 1)
def f(a: torch.Tensor, b: torch.Tensor) -> torch.Tensor: ...
Once decorated like this, the function f will gain the following
additional behaviors:
- If less-than-expected number of dimensions are provided either for
aor forb, an error will be raised (unless the decorator is provided with the keyword argumentallow_smaller_ndim=True) - If either
aorbare given as tensors that have extra leftmost dimensions, those dimensions will be assumed as batch dimensions, and therefore, the functionfwill run in a vectorized manner (with the help ofvmapbehind the scene), and the result will be a tensor with extra leftmost dimension(s), representing a batch of resulting tensors. - For convenience, numpy arrays and scalar data that are subclasses
of
numbers.Numberwill be converted to PyTorch tensors first, and then will be processed.
To be able to take advantage of this decorator, please ensure that the
decorated function is a vmap-friendly function. Please also ensure
that the decorated function expects positional arguments only.
Randomness.
Like in torch.func.vmap, the behavior of the decorated function in
terms of randomness can be configured via a keyword argument named
randomness:
@expects_ndim(2, 1, randomness="error")
def f(a: torch.Tensor, b: torch.Tensor) -> torch.Tensor: ...
If randomness is set as "error", then, when there is batching, any
attempt to generate random data using PyTorch will raise an error.
If randomness is set as "different", then, a random generation
operation such as torch.randn(...) will produce a BatchedTensor,
where each batch item has its own re-sampled data.
If randomness is set as "same", then, a random generation operation
such as torch.randn(...) will produce a non-batched tensor containing
random data that is sampled only once.
Alternative usage.
expects_ndim has an alternative interface that allows one to use it
as a tool for temporarily wrapping/transforming other functions. Let us
consider again our example function f. Instead of using the decorator
syntax, one can do:
which will temporarily wrap the function f with the additional behaviors
mentioned above, and immediately call it with the arguments a and b.
Source code in evotorch/decorators.py
def expects_ndim( # noqa: C901
*expected_ndims,
allow_smaller_ndim: bool = False,
randomness: str = "error",
) -> Callable:
"""
Decorator to declare the number of dimensions for each positional argument.
Let us imagine that we have a function `f(a, b)`, where `a` and `b` are
PyTorch tensors. Let us also imagine that the function `f` is implemented
in such a way that `a` is assumed to be a 2-dimensional tensor, and `b`
is assumed to be a 1-dimensional tensor. In this case, the function `f`
can be decorated as follows:
```python
from evotorch.decorators import expects_ndim
@expects_ndim(2, 1)
def f(a: torch.Tensor, b: torch.Tensor) -> torch.Tensor: ...
```
Once decorated like this, the function `f` will gain the following
additional behaviors:
- If less-than-expected number of dimensions are provided either for
`a` or for `b`, an error will be raised (unless the decorator
is provided with the keyword argument `allow_smaller_ndim=True`)
- If either `a` or `b` are given as tensors that have extra leftmost
dimensions, those dimensions will be assumed as batch dimensions,
and therefore, the function `f` will run in a vectorized manner
(with the help of `vmap` behind the scene), and the result will be
a tensor with extra leftmost dimension(s), representing a batch
of resulting tensors.
- For convenience, numpy arrays and scalar data that are subclasses
of `numbers.Number` will be converted to PyTorch tensors first, and
then will be processed.
To be able to take advantage of this decorator, please ensure that the
decorated function is a `vmap`-friendly function. Please also ensure
that the decorated function expects positional arguments only.
**Randomness.**
Like in `torch.func.vmap`, the behavior of the decorated function in
terms of randomness can be configured via a keyword argument named
`randomness`:
```python
@expects_ndim(2, 1, randomness="error")
def f(a: torch.Tensor, b: torch.Tensor) -> torch.Tensor: ...
```
If `randomness` is set as "error", then, when there is batching, any
attempt to generate random data using PyTorch will raise an error.
If `randomness` is set as "different", then, a random generation
operation such as `torch.randn(...)` will produce a `BatchedTensor`,
where each batch item has its own re-sampled data.
If `randomness` is set as "same", then, a random generation operation
such as `torch.randn(...)` will produce a non-batched tensor containing
random data that is sampled only once.
**Alternative usage.**
`expects_ndim` has an alternative interface that allows one to use it
as a tool for temporarily wrapping/transforming other functions. Let us
consider again our example function `f`. Instead of using the decorator
syntax, one can do:
```python
result = expects_ndim(f, (2, 1))(a, b)
```
which will temporarily wrap the function `f` with the additional behaviors
mentioned above, and immediately call it with the arguments `a` and `b`.
"""
if (len(expected_ndims) == 2) and isinstance(expected_ndims[0], Callable) and isinstance(expected_ndims[1], tuple):
func_to_wrap, expected_ndims = expected_ndims
return expects_ndim(*expected_ndims, allow_smaller_ndim=allow_smaller_ndim, randomness=randomness)(func_to_wrap)
expected_ndims = tuple(
(None if expected_arg_ndim is None else int(expected_arg_ndim)) for expected_arg_ndim in expected_ndims
)
def expects_ndim_decorator(fn: Callable):
def expects_ndim_decorated(*args):
# The inner class below is responsible for accumulating the dtype and device info of the tensors
# encountered across the arguments received by the decorated function.
# Such dtype and device information will be used if one of the considered arguments is given as a native
# scalar object (i.e. float), when converting that native scalar object to a PyTorch tensor.
class tensor_info:
# At first, we initialize the set of encountered dtype and device info as None.
# They will be lazily filled if we ever need such information.
encountered_dtypes: Optional[set] = None
encountered_devices: Optional[set] = None
@classmethod
def update(cls):
# Collect and fill the dtype and device information if it is not filled yet.
if (cls.encountered_dtypes is None) or (cls.encountered_devices is None):
cls.encountered_dtypes = set()
cls.encountered_devices = set()
for expected_arg_ndim, arg in zip(expected_ndims, args):
if (expected_arg_ndims is not None) and isinstance(arg, torch.Tensor):
# If the argument has a declared expected ndim, and also if it is a PyTorch tensor,
# then we add its dtype and device information to the sets `encountered_dtypes` and
# `encountered_devices`.
cls.encountered_dtypes.add(arg.dtype)
cls.encountered_devices.add(arg.device)
@classmethod
def _get_unique_dtype(cls, error_msg: str) -> torch.dtype:
# Ensure that there is only one `dtype` and return it.
# If there is not exactly one dtype, then raise an error.
if len(cls.encountered_dtypes) == 1:
[dtype] = cls.encountered_dtypes
return dtype
else:
raise TypeError(error_msg)
@classmethod
def _get_unique_device(cls, error_msg: str) -> torch.device:
# Ensure that there is only one `device` and return it.
# If there is not exactly one device, then raise an error.
if len(cls.encountered_devices) == 1:
[device] = cls.encountered_devices
return device
else:
raise TypeError(error_msg)
@classmethod
def convert_scalar_to_tensor(cls, scalar: Number) -> torch.Tensor:
# This class method aims to convert a scalar to a PyTorch tensor.
# The dtype and device of the tensor counterpart of the scalar will be taken from the dtype and
# device information of the other tensors encountered so far.
# First, we update the dtype and device information that can be collected from the arguments.
cls.update()
# Get the device used by the tensor arguments.
device = cls._get_unique_device(
f"The function decorated with `expects_ndim` received the scalar argument {scalar}."
f" However, this scalar argument cannot be converted to a PyTorch tensor, because it is not"
" clear to which device should this scalar be moved."
" This might happen when none of the other considered arguments is a tensor,"
" or when there are multiple tensor arguments with conflicting devices."
f" Devices encountered across all the considered arguments are: {cls.encountered_devices}."
" To make this error go away, please consider making sure that other tensor arguments have a"
" consistent device, or passing this scalar as a PyTorch tensor so that no conversion is"
" needed."
)
if isinstance(scalar, (bool, np.bool_)):
# If the given scalar argument is a boolean, we declare the dtype of its tensor counterpart as
# torch.bool.
dtype = torch.bool
else:
# If the given scalar argument is not a boolean, we declare the dtype of its tensor counterpart
# as the dtype that is observed across the other arguments.
dtype = cls._get_unique_dtype(
f" The function decorated with `expects_ndim` received the scalar argument {scalar}."
" However, this scalar argument cannot be converted to a PyTorch tensor, because it is not"
" clear by which dtype should this scalar be represented in its tensor form."
" This might happen when none of the other considered arguments is a tensor,"
" or when there are multiple tensor arguments with different dtypes."
f" dtypes encountered across all the considered arguments are {cls.encountered_dtypes}."
" To make this error go away, please consider making sure that other tensor arguments have"
" a consistent dtype, or passing this scalar as a PyTorch tensor so that no conversion is"
" needed."
)
# Finally, using our new dtype and new device, we convert the scalar to a tensor.
return torch.as_tensor(scalar, dtype=dtype, device=device)
# First, we want to make sure that each positional argument is a PyTorch tensor.
# So, we initialize `new_args` as an empty list, which will be filled with the tensor counterparts
# of the original positional arguments.
new_args = []
for i_arg, (expected_arg_ndims, arg) in enumerate(zip(expected_ndims, args)):
if (expected_arg_ndims is None) or isinstance(arg, torch.Tensor):
# In this case, either the expected number of dimensions is given as None (indicating that the user
# does not wish any batching nor any conversion for this argument), or the argument is already
# a PyTorch tensor (so, no conversion to tensor needs to be done).
# We do not have to do anything in this case.
pass
elif isinstance(arg, (Number, np.bool_)):
# If the argument is a scalar `Number`, we convert it to a PyTorch tensor, the dtype and the device
# of it being determined with the help of the inner class `tensor_info`.
arg = tensor_info.convert_scalar_to_tensor(arg)
elif isinstance(arg, np.ndarray):
# If the argument is a numpy array, we convert it to a PyTorch tensor.
arg = torch.as_tensor(arg)
else:
# This is the case where an object of an unrecognized type is received. We do not know how to
# process this argument, and, naively trying to convert it to a PyTorch tensor could fail, or
# could generate an unexpected result. So, we raise an error.
raise TypeError(f"Received an argument of unexpected type: {arg} (of type {type(arg)})")
if (expected_arg_ndims is not None) and (arg.ndim < expected_arg_ndims) and (not allow_smaller_ndim):
# This is the case where the currently analyzed positional argument has less-than-expected number
# of dimensions, and we are not in the allow-smaller-ndim mode. So, we raise an error.
raise ValueError(
f"The argument with index {i_arg} has the shape {arg.shape}, having {arg.ndim} dimensions."
f" However, it was expected as a tensor with {expected_arg_ndims} dimensions."
)
# At this point, we know that `arg` is a proper PyTorch tensor. So, we add it into `new_args`.
new_args.append(arg)
wrapped_fn = fn
num_args = len(new_args)
wrapped_ndims = [
(None if expected_arg_ndim is None else arg.ndim)
for expected_arg_ndim, arg in zip(expected_ndims, new_args)
]
# The following loop will run until we know that no `vmap` is necessary.
while True:
# Within each iteration, at first, we assume that `vmap` is not necessary, and therefore, for each
# positional argument, the batching dimension is `None` (which means no argument will be batched).
needs_vmap = False
in_dims = [None for _ in new_args]
for i_arg in range(num_args):
# For each positional argument with index `i_arg`, we check whether or not there are extra leftmost
# dimensions.
if (wrapped_ndims[i_arg] is not None) and (wrapped_ndims[i_arg] > expected_ndims[i_arg]):
# This is the case where the number of dimensions associated with this positional argument is
# greater than its expected number of dimensions.
# We take note that there is at least one positional argument which requires `vmap`.
needs_vmap = True
# We declare that this argument's batching dimension is 0 (i.e. its leftmost dimension).
in_dims[i_arg] = 0
# Now that we marked the leftmost dimension of this argument as the batching dimension, we
# should not consider this dimension in the next iteration of this `while` loop. So, we
# decrease its number of not-yet-handled dimensions by 1.
wrapped_ndims[i_arg] -= 1
if needs_vmap:
# This is the case where there was at least one positional argument that needs `vmap`.
# Therefore, we wrap the function via `vmap`.
# Note that, after this `vmap` wrapping, if some of the positional arguments still have extra
# leftmost dimensions, another level of `vmap`-wrapping will be done by the next iteration of this
# `while` loop.
wrapped_fn = vmap(wrapped_fn, in_dims=tuple(in_dims), randomness=randomness)
else:
# This is the case where no positional argument with extra leftmost dimension was found.
# Either the positional arguments were non-batched to begin with, or the `vmap`-wrapping of the
# previous iterations of this `while` loop were sufficient. Therefore, we are now ready to quit
# this loop.
break
# Run the `vmap`-wrapped counterpart of the function and return its result
return wrapped_fn(*new_args)
return expects_ndim_decorated
return expects_ndim_decorator
on_aux_device(*args)
¶
Decorator that informs a problem object that this function wants to receive its solutions on the auxiliary device of the problem.
According to its default (non-overriden) implementation, a problem
object returns torch.device("cuda") as its auxiliary device if
PyTorch's cuda backend is available and if there is a visible cuda
device. Otherwise, the auxiliary device is returned as
torch.device("cpu").
The auxiliary device is meant as a secondary device (in addition
to the main device reported by the problem object's device
attribute) used mainly for boosting the performance of fitness
evaluations.
This decorator, therefore, tells a problem object that the fitness
function requests to receive its solutions on this secondary device.
What this decorator does is that it injects a new attribute named
__evotorch_on_aux_device__ onto the decorated callable object,
then sets that new attribute to True, and then return the decorated
callable object itself. Upon seeing this new attribute with the
value True, a Problem object will attempt
to move the solutions to its auxiliary device before calling the
decorated fitness function.
Let us imagine a fitness function f whose definition looks like:
In its not-yet-decorated form, the function f would be given x on the
main device of the associated problem object. However, if one decorates
f as follows:
from evotorch.decorators import on_aux_device
@on_aux_device
def f(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x, dim=-1)
then the Problem object will first move x onto its auxiliary device,
then will call f.
This decorator is useful on multi-GPU settings. For details, please see the following example:
from evotorch import Problem
from evotorch.decorators import on_aux_device
@on_aux_device
def f(x: torch.Tensor) -> torch.Tensor: ...
problem = Problem(
"min",
f,
num_actors=4,
num_gpus_per_actor=1,
device="cpu",
)
In the example code above, we assume that there are 4 GPUs available.
The main device of the problem is "cpu", which means the populations
will be generated on the cpu. When evaluating a population, the population
will be split into 4 subbatches (because we have 4 actors), and each
subbatch will be sent to an actor. Thanks to the decorator @on_aux_device,
the Problem instance on each actor will first move
its SolutionBatch to its auxiliary device
visible to the actor, and then the fitness function will perform its
fitness evaluations on that device. In summary, the actors will use their
associated auxiliary devices (most commonly "cuda") to evaluate the
fitnesses of the solutions in parallel.
This decorator can also be used to decorate the method _evaluate or
_evaluate_batch belonging to a custom subclass of
Problem. Please see the example below:
from evotorch import Problem
class MyCustomProblem(Problem):
def __init__(self):
super().__init__(
...,
device="cpu", # populations will be created on the cpu
...,
)
@on_aux_device("cuda") # evaluations will be on the auxiliary device
def _evaluate_batch(self, solutions: SolutionBatch):
fitnesses = ...
solutions.set_evals(fitnesses)
Source code in evotorch/decorators.py
def on_aux_device(*args) -> Callable:
"""
Decorator that informs a problem object that this function wants to
receive its solutions on the auxiliary device of the problem.
According to its default (non-overriden) implementation, a problem
object returns `torch.device("cuda")` as its auxiliary device if
PyTorch's cuda backend is available and if there is a visible cuda
device. Otherwise, the auxiliary device is returned as
`torch.device("cpu")`.
The auxiliary device is meant as a secondary device (in addition
to the main device reported by the problem object's `device`
attribute) used mainly for boosting the performance of fitness
evaluations.
This decorator, therefore, tells a problem object that the fitness
function requests to receive its solutions on this secondary device.
What this decorator does is that it injects a new attribute named
`__evotorch_on_aux_device__` onto the decorated callable object,
then sets that new attribute to `True`, and then return the decorated
callable object itself. Upon seeing this new attribute with the
value `True`, a [Problem][evotorch.core.Problem] object will attempt
to move the solutions to its auxiliary device before calling the
decorated fitness function.
Let us imagine a fitness function `f` whose definition looks like:
```python
import torch
def f(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x, dim=-1)
```
In its not-yet-decorated form, the function `f` would be given `x` on the
main device of the associated problem object. However, if one decorates
`f` as follows:
```python
from evotorch.decorators import on_aux_device
@on_aux_device
def f(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x, dim=-1)
```
then the Problem object will first move `x` onto its auxiliary device,
then will call `f`.
This decorator is useful on multi-GPU settings. For details, please see
the following example:
```python
from evotorch import Problem
from evotorch.decorators import on_aux_device
@on_aux_device
def f(x: torch.Tensor) -> torch.Tensor: ...
problem = Problem(
"min",
f,
num_actors=4,
num_gpus_per_actor=1,
device="cpu",
)
```
In the example code above, we assume that there are 4 GPUs available.
The main device of the problem is "cpu", which means the populations
will be generated on the cpu. When evaluating a population, the population
will be split into 4 subbatches (because we have 4 actors), and each
subbatch will be sent to an actor. Thanks to the decorator `@on_aux_device`,
the [Problem][evotorch.core.Problem] instance on each actor will first move
its [SolutionBatch][evotorch.core.SolutionBatch] to its auxiliary device
visible to the actor, and then the fitness function will perform its
fitness evaluations on that device. In summary, the actors will use their
associated auxiliary devices (most commonly "cuda") to evaluate the
fitnesses of the solutions in parallel.
This decorator can also be used to decorate the method `_evaluate` or
`_evaluate_batch` belonging to a custom subclass of
[Problem][evotorch.core.Problem]. Please see the example below:
```python
from evotorch import Problem
class MyCustomProblem(Problem):
def __init__(self):
super().__init__(
...,
device="cpu", # populations will be created on the cpu
...,
)
@on_aux_device("cuda") # evaluations will be on the auxiliary device
def _evaluate_batch(self, solutions: SolutionBatch):
fitnesses = ...
solutions.set_evals(fitnesses)
```
"""
return _simple_decorator("__evotorch_on_aux_device__", args, decorator_name="on_aux_device")
on_cuda(*args)
¶
Decorator that informs a problem object that this function wants to receive its solutions on a cuda device (optionally of the specified cuda index).
Decorating a fitness function like this:
is equivalent to:
Decorating a fitness function like this:
is equivalent to:
Please see the documentation of on_device for further details.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
args |
An optional positional arguments using which one can specify the index of the cuda device to use. |
() |
Source code in evotorch/decorators.py
def on_cuda(*args) -> Callable:
"""
Decorator that informs a problem object that this function wants to
receive its solutions on a cuda device (optionally of the specified
cuda index).
Decorating a fitness function like this:
```
@on_cuda
def f(...):
...
```
is equivalent to:
```
@on_device("cuda")
def f(...):
...
```
Decorating a fitness function like this:
```
@on_cuda(0)
def f(...):
...
```
is equivalent to:
```
@on_device("cuda:0")
def f(...):
...
```
Please see the documentation of [on_device][evotorch.decorators.on_device]
for further details.
Args:
args: An optional positional arguments using which one can specify
the index of the cuda device to use.
"""
# Get the number of arguments
nargs = len(args)
if nargs == 0:
# If the number of arguments is 0, then we assume that we are in this situation:
#
# @on_cuda()
# def f(...):
# ...
#
# There is no specified index, and we are not yet given which object to decorate.
# Therefore, we set both of them as None.
index = None
fn = None
elif nargs == 1:
# The number of arguments is 1. We begin by storing that single argument using a variable named `arg`.
arg = args[0]
if isinstance(arg, Callable):
# If the argument is a callable object, we assume that we are in this situation:
#
# @on_cuda
# def f(...):
# ...
# We are not given a cuda index
index = None
# We are given our function to decorate. We store that function using a variable named `fn`.
fn = arg
else:
# If the argument is not a callable object, we assume that it is a cuda index, and that we are in the
# following situation:
#
# @on_cuda(index)
# def f(...):
# ...
# We are given a cuda index. After making sure that it is an integer, we store it by a variable named
# `index`.
index = int(arg)
# At this moment, we do not know the function that is being decorated. So, we set `fn` as None.
fn = None
else:
# If the number of arguments is neither 0 nor 1, then this is an unexpected case.
# We raise an error to inform the user.
raise TypeError("`on_cuda(...)` received invalid number of arguments")
# Prepare the device as "cuda"
device_str = "cuda"
if index is not None:
# If a cuda index is given, then we add ":N" (where N is the index) to the end of `device_str`.
device_str += ":" + str(index)
# Prepare the decorator function which, upon being called with a function argument, wraps that function.
decorator = on_device(device_str)
# If the function that is being decorated is not known yet (i.e. if `fn` is None), then we return the
# decorator function. If the function is known, then we decorate and return it.
return decorator if fn is None else decorator(fn)
on_device(device)
¶
Decorator that informs a problem object that this function wants to receive its solutions on the specified device.
What this decorator does is that it injects a device attribute onto
the decorated callable object. Then, this callable object itself is
returned. Upon seeing the device attribute, the evaluate(...) method
of the Problem object will attempt to move the
solutions to that device.
Let us imagine a fitness function f whose definition looks like:
In its not-yet-decorated form, the function f would be given x on the
default device of the associated problem object. However, if one decorates
f as follows:
from evotorch.decorators import on_device
@on_device("cuda:0")
def f(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x, dim=-1)
then the Problem object will first move x onto the device cuda:0, and
then will call f.
This decorator is useful on multi-GPU settings. For details, please see the following example:
from evotorch import Problem
from evotorch.decorators import on_device
@on_device("cuda")
def f(x: torch.Tensor) -> torch.Tensor: ...
problem = Problem(
"min",
f,
num_actors=4,
num_gpus_per_actor=1,
device="cpu",
)
In the example code above, we assume that there are 4 GPUs available.
The main device of the problem is "cpu", which means the populations
will be generated on the cpu. When evaluating a population, the population
will be split into 4 subbatches (because we have 4 actors), and each
subbatch will be sent to an actor. Thanks to the decorator @on_device,
the Problem instance on each actor will first move
its SolutionBatch to the cuda device visible
to its actor, and then the fitness function f will perform its evaluation
operations on that SolutionBatch on the
the visible cuda. In summary, the actors will use their associated cuda
devices to evaluate the fitnesses of the solutions in parallel.
This decorator can also be used to decorate the method _evaluate or
_evaluate_batch belonging to a custom subclass of
Problem. Please see the example below:
from evotorch import Problem
class MyCustomProblem(Problem):
def __init__(self):
super().__init__(
...,
device="cpu", # populations will be created on the cpu
...,
)
@on_device("cuda") # fitness evaluations will happen on cuda
def _evaluate_batch(self, solutions: SolutionBatch):
fitnesses = ...
solutions.set_evals(fitnesses)
The attribute device that is added by this decorator can be used to
query the fitness device, and also to modify/update it:
@on_device("cpu")
def f(x: torch.Tensor) -> torch.Tensor: ...
print(f.device) # Prints: torch.device("cpu")
f.device = "cuda:0" # Evaluations will be done on cuda:0 from now on
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
device |
Union[str, torch.device] |
The device on which the decorated fitness function will work. |
required |
Source code in evotorch/decorators.py
def on_device(device: Device) -> Callable:
"""
Decorator that informs a problem object that this function wants to
receive its solutions on the specified device.
What this decorator does is that it injects a `device` attribute onto
the decorated callable object. Then, this callable object itself is
returned. Upon seeing the `device` attribute, the `evaluate(...)` method
of the [Problem][evotorch.core.Problem] object will attempt to move the
solutions to that device.
Let us imagine a fitness function `f` whose definition looks like:
```python
import torch
def f(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x, dim=-1)
```
In its not-yet-decorated form, the function `f` would be given `x` on the
default device of the associated problem object. However, if one decorates
`f` as follows:
```python
from evotorch.decorators import on_device
@on_device("cuda:0")
def f(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x, dim=-1)
```
then the Problem object will first move `x` onto the device cuda:0, and
then will call `f`.
This decorator is useful on multi-GPU settings. For details, please see
the following example:
```python
from evotorch import Problem
from evotorch.decorators import on_device
@on_device("cuda")
def f(x: torch.Tensor) -> torch.Tensor: ...
problem = Problem(
"min",
f,
num_actors=4,
num_gpus_per_actor=1,
device="cpu",
)
```
In the example code above, we assume that there are 4 GPUs available.
The main device of the problem is "cpu", which means the populations
will be generated on the cpu. When evaluating a population, the population
will be split into 4 subbatches (because we have 4 actors), and each
subbatch will be sent to an actor. Thanks to the decorator `@on_device`,
the [Problem][evotorch.core.Problem] instance on each actor will first move
its [SolutionBatch][evotorch.core.SolutionBatch] to the cuda device visible
to its actor, and then the fitness function `f` will perform its evaluation
operations on that [SolutionBatch][evotorch.core.SolutionBatch] on the
the visible cuda. In summary, the actors will use their associated cuda
devices to evaluate the fitnesses of the solutions in parallel.
This decorator can also be used to decorate the method `_evaluate` or
`_evaluate_batch` belonging to a custom subclass of
[Problem][evotorch.core.Problem]. Please see the example below:
```python
from evotorch import Problem
class MyCustomProblem(Problem):
def __init__(self):
super().__init__(
...,
device="cpu", # populations will be created on the cpu
...,
)
@on_device("cuda") # fitness evaluations will happen on cuda
def _evaluate_batch(self, solutions: SolutionBatch):
fitnesses = ...
solutions.set_evals(fitnesses)
```
The attribute `device` that is added by this decorator can be used to
query the fitness device, and also to modify/update it:
```python
@on_device("cpu")
def f(x: torch.Tensor) -> torch.Tensor: ...
print(f.device) # Prints: torch.device("cpu")
f.device = "cuda:0" # Evaluations will be done on cuda:0 from now on
```
Args:
device: The device on which the decorated fitness function will work.
"""
# Take the `torch.device` counterpart of `device`
device = torch.device(device)
def decorator(fn: Callable) -> Callable:
setattr(fn, "__evotorch_on_device__", True)
setattr(fn, "device", device)
return fn
return decorator
pass_info(*args)
¶
Decorates a callable so that the neuroevolution problem class (e.g. GymNE) will pass information regarding the task at hand, in the form of keyword arguments.
This decorator adds a new attribute named __evotorch_pass_info__ to the
decorated callable object, sets this new attribute to True, and then returns
the callable object itself. Upon seeing this attribute with the value True,
a neuroevolution problem class sends extra information as keyword arguments.
For example, in the case of GymNE or VecGymNE, the passed information would include dimensions of the observation and action spaces.
Examples:
@pass_info
class MyModule(nn.Module):
def __init__(self, obs_length: int, act_length: int, **kwargs):
# Because MyModule is decorated with @pass_info, it receives
# keyword arguments related to the environment "CartPole-v0",
# including obs_length and act_length.
...
problem = GymNE(
"CartPole-v0",
network=MyModule,
...,
)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fn_or_class |
Callable |
Function or class to decorate |
required |
Returns:
| Type | Description |
|---|---|
Callable |
Decorated function or class |
Source code in evotorch/decorators.py
def pass_info(*args) -> Callable:
"""
Decorates a callable so that the neuroevolution problem class (e.g. GymNE) will
pass information regarding the task at hand, in the form of keyword arguments.
This decorator adds a new attribute named `__evotorch_pass_info__` to the
decorated callable object, sets this new attribute to True, and then returns
the callable object itself. Upon seeing this attribute with the value `True`,
a neuroevolution problem class sends extra information as keyword arguments.
For example, in the case of [GymNE][evotorch.neuroevolution.GymNE] or
[VecGymNE][evotorch.neuroevolution.VecGymNE], the passed information would
include dimensions of the observation and action spaces.
Example:
```python
@pass_info
class MyModule(nn.Module):
def __init__(self, obs_length: int, act_length: int, **kwargs):
# Because MyModule is decorated with @pass_info, it receives
# keyword arguments related to the environment "CartPole-v0",
# including obs_length and act_length.
...
problem = GymNE(
"CartPole-v0",
network=MyModule,
...,
)
```
Args:
fn_or_class (Callable): Function or class to decorate
Returns:
Callable: Decorated function or class
"""
return _simple_decorator("__evotorch_pass_info__", args, decorator_name="pass_info")
rowwise(*args, *, randomness='error')
¶
Decorate a vector-expecting function to make it support batch dimensions.
To be able to decorate a function via @rowwise, the following conditions
are required to be satisfied:
(i) the function expects a single positional argument, which is a PyTorch
tensor;
(ii) the function is implemented with the assumption that the tensor it
receives is a vector (i.e. is 1-dimensional).
Let us consider the example below:
Notice how the implementation of the function f assumes that its argument
x is 1-dimensional, and based on that assumption, omits the dim
keyword argument when calling torch.sum(...).
Upon receiving a 1-dimensional tensor, this decorated function f will
perform its operations on the vector x, like how it would work without
the decorator @rowwise.
Upon receiving a 2-dimensional tensor, this decorated function f will
perform its operations on each row of x.
Upon receiving a tensor with 3 or more dimensions, this decorated function
f will interpret its input as a batch of matrices, and perform its
operations on each matrix within the batch.
Defining fitness functions for Problem objects.
The decorator @rowwise can be used for defining a fitness function for a
Problem object. The advantage of doing so is to be
able to implement the fitness function with the simple assumption that the
input is a vector (that stores decision values for a single solution),
and the output is a scalar (that represents the fitness of the solution).
The decorator @rowwise also flags the decorated function (like
@vectorized does), so, the fitness function is used correctly by the
Problem instance, in a vectorized manner. See the example below:
@rowwise
def fitness(decision_values: torch.Tensor) -> torch.Tensor:
return torch.sqrt(torch.sum(decision_values**2))
my_problem = Problem("min", fitness, ...)
In the example above, thanks to the decorator @rowwise, my_problem will
use fitness in a vectorized manner when evaluating a SolutionBatch,
even though fitness is defined in terms of a single solution.
Randomness.
Like in torch.func.vmap, the behavior of the decorated function in
terms of randomness can be configured via a keyword argument named
randomness:
If randomness is set as "error", then, when there is batching, any
attempt to generate random data using PyTorch will raise an error.
If randomness is set as "different", then, a random generation
operation such as torch.randn(...) will produce a BatchedTensor,
where each batch item has its own re-sampled data.
If randomness is set as "same", then, a random generation operation
such as torch.randn(...) will produce a non-batched tensor containing
random data that is sampled only once.
Source code in evotorch/decorators.py
def rowwise(*args, randomness: str = "error") -> Callable:
"""
Decorate a vector-expecting function to make it support batch dimensions.
To be able to decorate a function via `@rowwise`, the following conditions
are required to be satisfied:
(i) the function expects a single positional argument, which is a PyTorch
tensor;
(ii) the function is implemented with the assumption that the tensor it
receives is a vector (i.e. is 1-dimensional).
Let us consider the example below:
```python
@rowwise
def f(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x**2)
```
Notice how the implementation of the function `f` assumes that its argument
`x` is 1-dimensional, and based on that assumption, omits the `dim`
keyword argument when calling `torch.sum(...)`.
Upon receiving a 1-dimensional tensor, this decorated function `f` will
perform its operations on the vector `x`, like how it would work without
the decorator `@rowwise`.
Upon receiving a 2-dimensional tensor, this decorated function `f` will
perform its operations on each row of `x`.
Upon receiving a tensor with 3 or more dimensions, this decorated function
`f` will interpret its input as a batch of matrices, and perform its
operations on each matrix within the batch.
**Defining fitness functions for Problem objects.**
The decorator `@rowwise` can be used for defining a fitness function for a
[Problem][evotorch.core.Problem] object. The advantage of doing so is to be
able to implement the fitness function with the simple assumption that the
input is a vector (that stores decision values for a single solution),
and the output is a scalar (that represents the fitness of the solution).
The decorator `@rowwise` also flags the decorated function (like
`@vectorized` does), so, the fitness function is used correctly by the
`Problem` instance, in a vectorized manner. See the example below:
```python
@rowwise
def fitness(decision_values: torch.Tensor) -> torch.Tensor:
return torch.sqrt(torch.sum(decision_values**2))
my_problem = Problem("min", fitness, ...)
```
In the example above, thanks to the decorator `@rowwise`, `my_problem` will
use `fitness` in a vectorized manner when evaluating a `SolutionBatch`,
even though `fitness` is defined in terms of a single solution.
**Randomness.**
Like in `torch.func.vmap`, the behavior of the decorated function in
terms of randomness can be configured via a keyword argument named
`randomness`:
```python
@rowwise(randomness="error")
def f(x: torch.Tensor) -> torch.Tensor: ...
```
If `randomness` is set as "error", then, when there is batching, any
attempt to generate random data using PyTorch will raise an error.
If `randomness` is set as "different", then, a random generation
operation such as `torch.randn(...)` will produce a `BatchedTensor`,
where each batch item has its own re-sampled data.
If `randomness` is set as "same", then, a random generation operation
such as `torch.randn(...)` will produce a non-batched tensor containing
random data that is sampled only once.
"""
num_args = len(args)
if num_args == 0:
immediately_decorate = False
elif num_args == 1:
immediately_decorate = True
else:
raise TypeError("`rowwise` received invalid number of positional arguments")
def decorator(fn: Callable) -> Callable: # <- inner decorator
decorated = expects_ndim(fn, (1,), randomness=randomness)
decorated.__evotorch_vectorized__ = True
return decorated
return decorator(args[0]) if immediately_decorate else decorator
vectorized(*args)
¶
Decorates a fitness function so that the problem object (which can be an instance of evotorch.Problem) will send the fitness function a 2D tensor containing all the solutions, instead of a 1D tensor containing a single solution.
What this decorator does is that it adds the decorated fitness function a new
attribute named __evotorch_vectorized__, the value of this new attribute being
True. Upon seeing this new attribute, the problem object will send this function
multiple solutions so that vectorized operations on multiple solutions can be
performed by this fitness function.
Let us imagine that we have the following fitness function which works on a
single solution x, and returns a single fitness value:
...and let us now define the optimization problem associated with this fitness function:
While the fitness function f and the definition p1 form a valid problem
description, it does not use PyTorch to its full potential in terms of performance.
If we were to request the evaluation results on a population of solutions via
p1.evaluate(population), p1 would use a classic for loop to evaluate every
single solution within population one by one.
We could greatly increase our performance by:
(i) re-defining our fitness function in a vectorized manner, i.e. in such a way
that it will operate on many solutions and compute all of their fitnesses at once;
(ii) label our fitness function via @vectorized, so that the problem object
will be aware that this new fitness function expects n solutions and returns
n fitnesses. The re-designed and labeled fitness function looks like this:
from evotorch.decorators import vectorized
@vectorized
def f2(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x**2, dim=-1)
The problem description for f2 is:
In this last example, p2 will realize that f2 is decorated via @vectorized,
and will send it n solutions, and will receive and process n fitnesses.
Source code in evotorch/decorators.py
def vectorized(*args) -> Callable:
"""
Decorates a fitness function so that the problem object (which can be an instance
of [evotorch.Problem][evotorch.core.Problem]) will send the fitness function a 2D
tensor containing all the solutions, instead of a 1D tensor containing a single
solution.
What this decorator does is that it adds the decorated fitness function a new
attribute named `__evotorch_vectorized__`, the value of this new attribute being
True. Upon seeing this new attribute, the problem object will send this function
multiple solutions so that vectorized operations on multiple solutions can be
performed by this fitness function.
Let us imagine that we have the following fitness function which works on a
single solution `x`, and returns a single fitness value:
```python
import torch
def f(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x**2)
```
...and let us now define the optimization problem associated with this fitness
function:
```python
p1 = Problem("min", f, initial_bounds=(-10.0, 10.0), solution_length=5)
```
While the fitness function `f` and the definition `p1` form a valid problem
description, it does not use PyTorch to its full potential in terms of performance.
If we were to request the evaluation results on a population of solutions via
`p1.evaluate(population)`, `p1` would use a classic `for` loop to evaluate every
single solution within `population` one by one.
We could greatly increase our performance by:
(i) re-defining our fitness function in a vectorized manner, i.e. in such a way
that it will operate on many solutions and compute all of their fitnesses at once;
(ii) label our fitness function via `@vectorized`, so that the problem object
will be aware that this new fitness function expects `n` solutions and returns
`n` fitnesses. The re-designed and labeled fitness function looks like this:
```python
from evotorch.decorators import vectorized
@vectorized
def f2(x: torch.Tensor) -> torch.Tensor:
return torch.sum(x**2, dim=-1)
```
The problem description for `f2` is:
```python
p2 = Problem("min", f2, initial_bounds=(-10.0, 10.0), solution_length=5)
```
In this last example, `p2` will realize that `f2` is decorated via `@vectorized`,
and will send it `n` solutions, and will receive and process `n` fitnesses.
"""
return _simple_decorator("__evotorch_vectorized__", args, decorator_name="vectorized")
distributions
¶
Distribution (TensorMakerMixin, Serializable)
¶
Base class for any search distribution.
Source code in evotorch/distributions.py
class Distribution(TensorMakerMixin, Serializable):
"""
Base class for any search distribution.
"""
MANDATORY_PARAMETERS = set()
OPTIONAL_PARAMETERS = set()
PARAMETER_NDIMS = {}
functional_sample = NotImplemented
def __init__(
self, *, solution_length: int, parameters: dict, dtype: Optional[DType] = None, device: Optional[Device] = None
):
"""
`__init__(...)`: Initialize the Distribution.
It is expected that one of these two conditions is met:
(i) the inheriting search distribution class does not implement its
own `__init__(...)` method; or
(ii) the inheriting search distribution class has its own
`__init__(...)` method, and calls `Distribution.__init__(...)`
from there, during its initialization phase.
Args:
solution_length: Expected as an integer, this argument represents
the solution length.
parameters: Expected as a dictionary, this argument stores
the parameters of the search distribution.
For example, for a Gaussian distribution where `mu`
represents the mean, and `sigma` represents the coverage
area, this dictionary would have the keys "mu" and "sigma",
and each of these keys would map to a PyTorch tensor.
dtype: The dtype of the search distribution (e.g. torch.float32).
device: The device of the search distribution (e.g. "cpu").
"""
self.__solution_length: int = int(solution_length)
self.__parameters: dict
self.__dtype: torch.dtype
self.__device: torch.device
self.__check_correctness(parameters)
cast_kwargs = {}
if dtype is not None:
cast_kwargs["dtype"] = to_torch_dtype(dtype)
if device is not None:
cast_kwargs["device"] = torch.device(device)
if len(cast_kwargs) == 0:
self.__parameters = copy(parameters)
else:
self.__parameters = cast_tensors_in_container(parameters, **cast_kwargs)
self.__dtype = cast_kwargs.get("dtype", dtype_of_container(parameters))
self.__device = cast_kwargs.get("device", device_of_container(parameters))
def __check_correctness(self, parameters: dict):
found_mandatory = 0
for param_name in parameters.keys():
if param_name in self.MANDATORY_PARAMETERS:
found_mandatory += 1
elif param_name in self.OPTIONAL_PARAMETERS:
pass # nothing to do
else:
raise ValueError(f"Unrecognized parameter: {repr(param_name)}")
if found_mandatory < len(self.MANDATORY_PARAMETERS):
raise ValueError(
f"Not all mandatory parameters of this Distribution were specified."
f" Mandatory parameters of this distribution: {self.MANDATORY_PARAMETERS};"
f" optional parameters of this distribution: {self.OPTIONAL_PARAMETERS};"
f" encountered parameters: {set(parameters.keys())}."
)
def to(self, device: Device) -> "Distribution":
"""
Bring the Distribution onto a computational device.
If the given device is already the device of this Distribution,
then the Distribution itself will be returned.
If the given device is different than the device of this
Distribution, a copy of this Distribution on the given device
will be created and returned.
Args:
device: The computation device onto which the Distribution
will be brought.
Returns:
The Distribution on the target device.
"""
if torch.device(self.device) == torch.device(device):
return self
else:
cls = self.__class__
return cls(solution_length=self.solution_length, parameters=self.parameters, device=device)
def _fill(self, out: torch.Tensor, *, generator: Optional[torch.Generator] = None):
"""
Fill the given tensor with samples from this search distribution.
It is expected that the inheriting search distribution class
has its own implementation for this method.
Args:
out: The PyTorch tensor that will be filled with the samples.
This tensor is expected as 2-dimensional with its number
of columns equal to the solution length declared by this
distribution.
generator: Optionally a PyTorch generator, to be used for
sampling. None means that the global generator of PyTorch
is to be used.
"""
raise NotImplementedError
def sample(
self,
num_solutions: Optional[int] = None,
*,
out: Optional[torch.Tensor] = None,
generator: Any = None,
) -> torch.Tensor:
"""
Sample solutions from this search distribution.
Args:
num_solutions: How many solutions will be sampled.
If this argument is given as an integer and the argument
`out` is left as None, then a new PyTorch tensor, filled
with the samples from this distribution, will be generated
and returned. The number of rows of this new tensor will
be equal to the given `num_solutions`.
If the argument `num_solutions` is provided as an integer,
then the argument `out` is expected as None.
out: The PyTorch tensor that will be filled with the samples
of this distribution. This tensor is expected as a
2-dimensional tensor with its number of columns equal to
the solution length declared by this distribution.
If the argument `out` is provided as a tensor, then the
argument `num_solutions` is expected as None.
generator: Optionally a PyTorch generator or any object which
has a `generator` attribute (e.g. a Problem instance).
If left as None, the global generator of PyTorch will be
used.
Returns:
A 2-dimensional PyTorch tensor which stores the sampled solutions.
"""
if (num_solutions is not None) and (out is not None):
raise ValueError(
"Received both `num_solutions` and `out` with values other than None."
"Please provide only one of these arguments with a value other than None, not both of them."
)
elif (num_solutions is not None) and (out is None):
num_solutions = int(num_solutions)
out = self.make_empty(num_solutions=num_solutions)
out = out + make_batched_false_for_vmap(out.device)
elif (num_solutions is None) and (out is not None):
if out.ndim != 2:
raise ValueError(
f"The `sample(...)` method can fill only 2-dimensional tensors."
f" However, the provided `out` tensor has {out.ndim} dimensions, its shape being {out.shape}."
)
_, num_cols = out.shape
if num_cols != self.solution_length:
raise ValueError(
f"The solution length declared by this distribution is {self.solution_length}."
f" However, the provided `out` tensor has {num_cols} columns."
f" The `sample(...)` method can only work with tensors whose number of columns are equal"
f" to the declared solution length."
)
else:
raise ValueError(
"Received both `num_solutions` and `out` as None."
"Please provide one of these arguments with a value other than None."
)
self._fill(out, generator=generator)
return out
def _compute_gradients(self, samples: torch.Tensor, weights: torch.Tensor, ranking_used: Optional[str]) -> dict:
"""
Compute the gradients out of the samples (sampled solutions)
and weights (i.e. weights or ranks of the solutions, better
solutions having numerically higher weights).
It is expected that the inheriting class implements this method.
Args:
samples: The sampled solutions, as a 2-dimensional tensor.
weights: Solution weights, as a 1-dimensional tensor of length
`n`, `n` being the number of sampled solutions.
ranking_used: Ranking that was used to obtain the weights.
Returns:
The gradient(s) in a dictionary.
"""
raise NotImplementedError
def compute_gradients(
self,
samples: torch.Tensor,
fitnesses: torch.Tensor,
*,
objective_sense: str,
ranking_method: Optional[str] = None,
) -> dict:
"""
Compute and return gradients.
Args:
samples: The solutions that were sampled from this Distribution.
The tensor passed via this argument is expected to have
the same dtype and device with this Distribution.
fitnesses: The evaluation results of the sampled solutions.
If fitnesses are given with a different dtype (maybe because
the eval_dtype of the Problem object is different than its
decision variable dtype), then this method will first
create an internal copy of the fitnesses with the correct
dtype, and then will use those copied fitnesses for
computing the gradients.
objective_sense: The objective sense, expected as "min" or "max".
In the case of "min", lower fitness values will be regarded
as better (therefore, in this case, one can alternatively
refer to fitnesses as 'unfitnesses' or 'solution costs').
In the case of "max", higher fitness values will be regarded
as better.
ranking_method: The ranking method to be used.
Can be: "linear" (where ranks linearly go from 0 to 1);
"centered" (where ranks linearly go from -0.5 to +0.5);
"normalized" (where the standard-normalized fitnesses
serve as ranks); or "raw" (where the fitnesses themselves
serve as ranks).
The default is "raw".
Returns:
A dictionary which contains the gradient for each parameter of the
distribution.
"""
if objective_sense == "max":
higher_is_better = True
elif objective_sense == "min":
higher_is_better = False
else:
raise ValueError(
f'`objective_sense` was expected as "min" or as "max".'
f" However, it was encountered as {repr(objective_sense)}."
)
if ranking_method is None:
ranking_method = "raw"
# Make sure that the fitnesses are in the correct dtype
fitnesses = torch.as_tensor(fitnesses, dtype=self.dtype)
[num_samples, _] = samples.shape
[num_fitnesses] = fitnesses.shape
if num_samples != num_fitnesses:
raise ValueError(
f"The number of samples and the number of fitnesses do not match:" f" {num_samples} != {num_fitnesses}."
)
weights = rank(fitnesses, ranking_method=ranking_method, higher_is_better=higher_is_better)
return self._compute_gradients(samples, weights, ranking_method)
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "Distribution":
"""
Do an update on the distribution by following the given gradients.
It is expected that the inheriting class has its own implementation
for this method.
Args:
gradients: Gradients, as a dictionary, which will be used for
computing the necessary updates.
learning_rates: A dictionary which contains learning rates
for parameters that will be updated using a learning rate
coefficient.
optimizers: A dictionary which contains optimizer objects
for parameters that will be updated using an adaptive
optimizer.
Returns:
The updated copy of the distribution.
"""
raise NotImplementedError
def modified_copy(
self, *, dtype: Optional[DType] = None, device: Optional[Device] = None, **parameters
) -> "Distribution":
"""
Return a modified copy of this distribution.
Args:
dtype: The new dtype of the distribution.
device: The new device of the distribution.
parameters: Expected in the form of extra keyword arguments.
Each of these keyword arguments will cause the new distribution
to have a modified value for the specified parameter.
Returns:
The modified copy of the distribution.
"""
cls = self.__class__
if device is None:
device = self.device
if dtype is None:
dtype = self.dtype
new_parameters = copy(self.parameters)
new_parameters.update(parameters)
return cls(parameters=new_parameters, dtype=dtype, device=device)
def relative_entropy(dist_0: "Distribution", dist_1: "Distribution") -> float:
raise NotImplementedError
@property
def solution_length(self) -> int:
return self.__solution_length
@property
def device(self) -> torch.device:
return self.__device
@property
def dtype(self) -> torch.dtype:
return self.__dtype
@property
def parameters(self) -> dict:
return self.__parameters
def _follow_gradient(
self,
param_name: str,
x: torch.Tensor,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> torch.Tensor:
x = torch.as_tensor(x, dtype=self.dtype, device=self.device)
learning_rate, optimizer = self._get_learning_rate_and_optimizer(param_name, learning_rates, optimizers)
if (learning_rate is None) and (optimizer is None):
return x
elif (learning_rate is not None) and (optimizer is None):
return learning_rate * x
elif (learning_rate is None) and (optimizer is not None):
return optimizer.ascent(x)
else:
raise ValueError(
"Encountered both `learning_rate` and `optimizer` as values other than None."
" This method can only work if both of them are None or only one of them is not None."
)
@staticmethod
def _get_learning_rate_and_optimizer(
param_name: str, learning_rates: Optional[dict], optimizers: Optional[dict]
) -> tuple:
if learning_rates is None:
learning_rates = {}
if optimizers is None:
optimizers = {}
return learning_rates.get(param_name, None), optimizers.get(param_name, None)
@torch.no_grad()
def _get_cloned_state(self, *, memo: dict) -> dict:
return deep_clone(
self.__dict__,
otherwise_deepcopy=True,
memo=memo,
)
__init__(self, *, solution_length, parameters, dtype=None, device=None)
special
¶
__init__(...): Initialize the Distribution.
It is expected that one of these two conditions is met:
(i) the inheriting search distribution class does not implement its
own __init__(...) method; or
(ii) the inheriting search distribution class has its own
__init__(...) method, and calls Distribution.__init__(...)
from there, during its initialization phase.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
solution_length |
int |
Expected as an integer, this argument represents the solution length. |
required |
parameters |
dict |
Expected as a dictionary, this argument stores
the parameters of the search distribution.
For example, for a Gaussian distribution where |
required |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
The dtype of the search distribution (e.g. torch.float32). |
None |
device |
Union[str, torch.device] |
The device of the search distribution (e.g. "cpu"). |
None |
Source code in evotorch/distributions.py
def __init__(
self, *, solution_length: int, parameters: dict, dtype: Optional[DType] = None, device: Optional[Device] = None
):
"""
`__init__(...)`: Initialize the Distribution.
It is expected that one of these two conditions is met:
(i) the inheriting search distribution class does not implement its
own `__init__(...)` method; or
(ii) the inheriting search distribution class has its own
`__init__(...)` method, and calls `Distribution.__init__(...)`
from there, during its initialization phase.
Args:
solution_length: Expected as an integer, this argument represents
the solution length.
parameters: Expected as a dictionary, this argument stores
the parameters of the search distribution.
For example, for a Gaussian distribution where `mu`
represents the mean, and `sigma` represents the coverage
area, this dictionary would have the keys "mu" and "sigma",
and each of these keys would map to a PyTorch tensor.
dtype: The dtype of the search distribution (e.g. torch.float32).
device: The device of the search distribution (e.g. "cpu").
"""
self.__solution_length: int = int(solution_length)
self.__parameters: dict
self.__dtype: torch.dtype
self.__device: torch.device
self.__check_correctness(parameters)
cast_kwargs = {}
if dtype is not None:
cast_kwargs["dtype"] = to_torch_dtype(dtype)
if device is not None:
cast_kwargs["device"] = torch.device(device)
if len(cast_kwargs) == 0:
self.__parameters = copy(parameters)
else:
self.__parameters = cast_tensors_in_container(parameters, **cast_kwargs)
self.__dtype = cast_kwargs.get("dtype", dtype_of_container(parameters))
self.__device = cast_kwargs.get("device", device_of_container(parameters))
compute_gradients(self, samples, fitnesses, *, objective_sense, ranking_method=None)
¶
Compute and return gradients.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
samples |
Tensor |
The solutions that were sampled from this Distribution. The tensor passed via this argument is expected to have the same dtype and device with this Distribution. |
required |
fitnesses |
Tensor |
The evaluation results of the sampled solutions. If fitnesses are given with a different dtype (maybe because the eval_dtype of the Problem object is different than its decision variable dtype), then this method will first create an internal copy of the fitnesses with the correct dtype, and then will use those copied fitnesses for computing the gradients. |
required |
objective_sense |
str |
The objective sense, expected as "min" or "max". In the case of "min", lower fitness values will be regarded as better (therefore, in this case, one can alternatively refer to fitnesses as 'unfitnesses' or 'solution costs'). In the case of "max", higher fitness values will be regarded as better. |
required |
ranking_method |
Optional[str] |
The ranking method to be used. Can be: "linear" (where ranks linearly go from 0 to 1); "centered" (where ranks linearly go from -0.5 to +0.5); "normalized" (where the standard-normalized fitnesses serve as ranks); or "raw" (where the fitnesses themselves serve as ranks). The default is "raw". |
None |
Returns:
| Type | Description |
|---|---|
dict |
A dictionary which contains the gradient for each parameter of the distribution. |
Source code in evotorch/distributions.py
def compute_gradients(
self,
samples: torch.Tensor,
fitnesses: torch.Tensor,
*,
objective_sense: str,
ranking_method: Optional[str] = None,
) -> dict:
"""
Compute and return gradients.
Args:
samples: The solutions that were sampled from this Distribution.
The tensor passed via this argument is expected to have
the same dtype and device with this Distribution.
fitnesses: The evaluation results of the sampled solutions.
If fitnesses are given with a different dtype (maybe because
the eval_dtype of the Problem object is different than its
decision variable dtype), then this method will first
create an internal copy of the fitnesses with the correct
dtype, and then will use those copied fitnesses for
computing the gradients.
objective_sense: The objective sense, expected as "min" or "max".
In the case of "min", lower fitness values will be regarded
as better (therefore, in this case, one can alternatively
refer to fitnesses as 'unfitnesses' or 'solution costs').
In the case of "max", higher fitness values will be regarded
as better.
ranking_method: The ranking method to be used.
Can be: "linear" (where ranks linearly go from 0 to 1);
"centered" (where ranks linearly go from -0.5 to +0.5);
"normalized" (where the standard-normalized fitnesses
serve as ranks); or "raw" (where the fitnesses themselves
serve as ranks).
The default is "raw".
Returns:
A dictionary which contains the gradient for each parameter of the
distribution.
"""
if objective_sense == "max":
higher_is_better = True
elif objective_sense == "min":
higher_is_better = False
else:
raise ValueError(
f'`objective_sense` was expected as "min" or as "max".'
f" However, it was encountered as {repr(objective_sense)}."
)
if ranking_method is None:
ranking_method = "raw"
# Make sure that the fitnesses are in the correct dtype
fitnesses = torch.as_tensor(fitnesses, dtype=self.dtype)
[num_samples, _] = samples.shape
[num_fitnesses] = fitnesses.shape
if num_samples != num_fitnesses:
raise ValueError(
f"The number of samples and the number of fitnesses do not match:" f" {num_samples} != {num_fitnesses}."
)
weights = rank(fitnesses, ranking_method=ranking_method, higher_is_better=higher_is_better)
return self._compute_gradients(samples, weights, ranking_method)
modified_copy(self, *, dtype=None, device=None, **parameters)
¶
Return a modified copy of this distribution.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
The new dtype of the distribution. |
None |
device |
Union[str, torch.device] |
The new device of the distribution. |
None |
parameters |
Expected in the form of extra keyword arguments. Each of these keyword arguments will cause the new distribution to have a modified value for the specified parameter. |
{} |
Returns:
| Type | Description |
|---|---|
Distribution |
The modified copy of the distribution. |
Source code in evotorch/distributions.py
def modified_copy(
self, *, dtype: Optional[DType] = None, device: Optional[Device] = None, **parameters
) -> "Distribution":
"""
Return a modified copy of this distribution.
Args:
dtype: The new dtype of the distribution.
device: The new device of the distribution.
parameters: Expected in the form of extra keyword arguments.
Each of these keyword arguments will cause the new distribution
to have a modified value for the specified parameter.
Returns:
The modified copy of the distribution.
"""
cls = self.__class__
if device is None:
device = self.device
if dtype is None:
dtype = self.dtype
new_parameters = copy(self.parameters)
new_parameters.update(parameters)
return cls(parameters=new_parameters, dtype=dtype, device=device)
sample(self, num_solutions=None, *, out=None, generator=None)
¶
Sample solutions from this search distribution.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
num_solutions |
Optional[int] |
How many solutions will be sampled.
If this argument is given as an integer and the argument
|
None |
out |
Optional[torch.Tensor] |
The PyTorch tensor that will be filled with the samples
of this distribution. This tensor is expected as a
2-dimensional tensor with its number of columns equal to
the solution length declared by this distribution.
If the argument |
None |
generator |
Any |
Optionally a PyTorch generator or any object which
has a |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
A 2-dimensional PyTorch tensor which stores the sampled solutions. |
Source code in evotorch/distributions.py
def sample(
self,
num_solutions: Optional[int] = None,
*,
out: Optional[torch.Tensor] = None,
generator: Any = None,
) -> torch.Tensor:
"""
Sample solutions from this search distribution.
Args:
num_solutions: How many solutions will be sampled.
If this argument is given as an integer and the argument
`out` is left as None, then a new PyTorch tensor, filled
with the samples from this distribution, will be generated
and returned. The number of rows of this new tensor will
be equal to the given `num_solutions`.
If the argument `num_solutions` is provided as an integer,
then the argument `out` is expected as None.
out: The PyTorch tensor that will be filled with the samples
of this distribution. This tensor is expected as a
2-dimensional tensor with its number of columns equal to
the solution length declared by this distribution.
If the argument `out` is provided as a tensor, then the
argument `num_solutions` is expected as None.
generator: Optionally a PyTorch generator or any object which
has a `generator` attribute (e.g. a Problem instance).
If left as None, the global generator of PyTorch will be
used.
Returns:
A 2-dimensional PyTorch tensor which stores the sampled solutions.
"""
if (num_solutions is not None) and (out is not None):
raise ValueError(
"Received both `num_solutions` and `out` with values other than None."
"Please provide only one of these arguments with a value other than None, not both of them."
)
elif (num_solutions is not None) and (out is None):
num_solutions = int(num_solutions)
out = self.make_empty(num_solutions=num_solutions)
out = out + make_batched_false_for_vmap(out.device)
elif (num_solutions is None) and (out is not None):
if out.ndim != 2:
raise ValueError(
f"The `sample(...)` method can fill only 2-dimensional tensors."
f" However, the provided `out` tensor has {out.ndim} dimensions, its shape being {out.shape}."
)
_, num_cols = out.shape
if num_cols != self.solution_length:
raise ValueError(
f"The solution length declared by this distribution is {self.solution_length}."
f" However, the provided `out` tensor has {num_cols} columns."
f" The `sample(...)` method can only work with tensors whose number of columns are equal"
f" to the declared solution length."
)
else:
raise ValueError(
"Received both `num_solutions` and `out` as None."
"Please provide one of these arguments with a value other than None."
)
self._fill(out, generator=generator)
return out
to(self, device)
¶
Bring the Distribution onto a computational device.
If the given device is already the device of this Distribution, then the Distribution itself will be returned. If the given device is different than the device of this Distribution, a copy of this Distribution on the given device will be created and returned.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
device |
Union[str, torch.device] |
The computation device onto which the Distribution will be brought. |
required |
Returns:
| Type | Description |
|---|---|
Distribution |
The Distribution on the target device. |
Source code in evotorch/distributions.py
def to(self, device: Device) -> "Distribution":
"""
Bring the Distribution onto a computational device.
If the given device is already the device of this Distribution,
then the Distribution itself will be returned.
If the given device is different than the device of this
Distribution, a copy of this Distribution on the given device
will be created and returned.
Args:
device: The computation device onto which the Distribution
will be brought.
Returns:
The Distribution on the target device.
"""
if torch.device(self.device) == torch.device(device):
return self
else:
cls = self.__class__
return cls(solution_length=self.solution_length, parameters=self.parameters, device=device)
update_parameters(self, gradients, *, learning_rates=None, optimizers=None)
¶
Do an update on the distribution by following the given gradients.
It is expected that the inheriting class has its own implementation for this method.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
gradients |
dict |
Gradients, as a dictionary, which will be used for computing the necessary updates. |
required |
learning_rates |
Optional[dict] |
A dictionary which contains learning rates for parameters that will be updated using a learning rate coefficient. |
None |
optimizers |
Optional[dict] |
A dictionary which contains optimizer objects for parameters that will be updated using an adaptive optimizer. |
None |
Returns:
| Type | Description |
|---|---|
Distribution |
The updated copy of the distribution. |
Source code in evotorch/distributions.py
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "Distribution":
"""
Do an update on the distribution by following the given gradients.
It is expected that the inheriting class has its own implementation
for this method.
Args:
gradients: Gradients, as a dictionary, which will be used for
computing the necessary updates.
learning_rates: A dictionary which contains learning rates
for parameters that will be updated using a learning rate
coefficient.
optimizers: A dictionary which contains optimizer objects
for parameters that will be updated using an adaptive
optimizer.
Returns:
The updated copy of the distribution.
"""
raise NotImplementedError
ExpGaussian (Distribution)
¶
Exponential Multivariate Gaussian, as used by XNES
Source code in evotorch/distributions.py
class ExpGaussian(Distribution):
"""Exponential Multivariate Gaussian, as used by XNES"""
# Corresponding to mu and A in symbols used in xNES paper
MANDATORY_PARAMETERS = {"mu", "sigma"}
# Inverse of sigma, numerically more stable to track this independently to sigma
OPTIONAL_PARAMETERS = {"sigma_inv"}
PARAMETER_NDIMS = {"mu": 1, "sigma": 2, "sigma_inv": 2}
def __init__(
self,
parameters: dict,
*,
solution_length: Optional[int] = None,
device: Optional[Device] = None,
dtype: Optional[DType] = None,
):
[mu_length] = parameters["mu"].shape
# Make sigma 2D
if len(parameters["sigma"].shape) == 1:
parameters["sigma"] = torch.diag(parameters["sigma"])
# Automatically generate sigma_inv if not provided
if "sigma_inv" not in parameters:
parameters["sigma_inv"] = torch.inverse(parameters["sigma"])
[sigma_length, _] = parameters["sigma"].shape
if solution_length is None:
solution_length = mu_length
else:
if solution_length != mu_length:
raise ValueError(
f"The argument `solution_length` does not match the length of `mu` provided in `parameters`."
f" solution_length={solution_length},"
f' parameters["mu"]={mu_length}.'
)
if mu_length != sigma_length:
raise ValueError(
f"The tensors `mu` and `sigma` provided within `parameters` have mismatching lengths."
f' parameters["mu"]={mu_length},'
f' parameters["sigma"]={sigma_length}.'
)
super().__init__(
solution_length=solution_length,
parameters=parameters,
device=device,
dtype=dtype,
)
# Make identity matrix as this is used throughout in gradient computation
self.eye = self.make_I(solution_length)
@property
def mu(self) -> torch.Tensor:
"""Getter for mu
Returns:
mu (torch.Tensor): The center of the search distribution
"""
return self.parameters["mu"]
@mu.setter
def mu(self, new_mu: Iterable):
"""Setter for mu
Args:
new_mu (torch.Tensor): The new value of mu
"""
self.parameters["mu"] = torch.as_tensor(new_mu, dtype=self.dtype, device=self.device)
@property
def cov(self) -> torch.Tensor:
"""The covariance matrix A^T A"""
return self.sigma.transpose(0, 1) @ self.sigma
@property
def sigma(self) -> torch.Tensor:
"""Getter for sigma
Returns:
sigma (torch.Tensor): The square root of the covariance matrix
"""
return self.parameters["sigma"]
@property
def sigma_inv(self) -> torch.Tensor:
"""Getter for sigma_inv
Returns:
sigma_inv (torch.Tensor): The inverse square root of the covariance matrix
"""
if "sigma_inv" in self.parameters:
return self.parameters["sigma_inv"]
else:
return torch.inverse(self.parameters["sigma"])
@property
def A(self) -> torch.Tensor:
"""Alias for self.sigma, for notational consistency with paper"""
return self.sigma
@property
def A_inv(self) -> torch.Tensor:
"""Alias for self.sigma_inv, for notational consistency with paper"""
return self.sigma_inv
@sigma.setter
def sigma(self, new_sigma: Iterable):
"""Setter for sigma
Args:
new_sigma (torch.Tensor): The new value of sigma, the square root of the covariance matrix
"""
self.parameters["sigma"] = torch.as_tensor(new_sigma, dtype=self.dtype, device=self.device)
def to_global_coordinates(self, local_coordinates: torch.Tensor) -> torch.Tensor:
"""Map samples from local coordinate space N(0, I_d) to global coordinate space N(mu, A^T A)
This function is the inverse of to_local_coordinates
Args:
local_coordinates (torch.Tensor): The local coordinates sampled from N(0, I_d)
Returns:
global_coordinates (torch.Tensor): The global coordinates sampled from N(mu, A^T A)
"""
# Global samples are constructed as x = mu + A z where z is local coordinate
# We use transpose here to simplify the batched application of A
return self.mu.unsqueeze(0) + (self.A @ local_coordinates.T).T
def to_local_coordinates(self, global_coordinates: torch.Tensor) -> torch.Tensor:
"""Map samples from global coordinate space N(mu, A^T A) to local coordinate space N(0, I_d)
This function is the inverse of to_global_coordinates
Args:
global_coordinates (torch.Tensor): The global coordinates sampled from N(mu, A^T A)
Returns:
local_coordinates (torch.Tensor): The local coordinates sampled from N(0, I_d)
"""
# Global samples are constructed as x = mu + A z where z is local coordinate
# Therefore, we can recover z according to z = A_inv (x - mu)
return (self.A_inv @ (global_coordinates - self.mu.unsqueeze(0)).T).T
def _fill(self, out: torch.Tensor, *, generator: Optional[torch.Generator] = None):
"""Fill a tensor with samples from N(mu, A^T A)
Args:
out (torch.Tensor): The tensor to fill
generator (Optional[torch.Generator]): A generator to use to generate random values
"""
# Fill with local coordinates from N(0, I_d)
self.make_gaussian(out=out, generator=generator)
# Map local coordinates to global coordinate system
out[:] = self.to_global_coordinates(out)
def _compute_gradients(self, samples: torch.Tensor, weights: torch.Tensor, ranking_used: Optional[str]) -> dict:
"""Compute the gradients with respect to a given set of samples and weights
Args:
samples (torch.Tensor): Samples drawn from N(mu, A^T A), ideally using self._fill
weights (torch.Tensor): Weights e.g. fitnesses or utilities assigned to samples
ranking_used (optional[str]): The ranking method used to compute weights
Returns:
grads (dict): A dictionary containing the approximated natural gradient on d and M
"""
# Compute the local coordinates
local_coordinates = self.to_local_coordinates(samples)
# Make sure that the weights (utilities) are 0-centered
# (Otherwise the formulations would have to consider a bias term)
if ranking_used not in ("centered", "normalized"):
weights = weights - torch.mean(weights)
d_grad = total(dot(weights, local_coordinates))
local_coordinates_outer = local_coordinates.unsqueeze(1) * local_coordinates.unsqueeze(2)
M_grad = torch.sum(
weights.unsqueeze(-1).unsqueeze(-1) * (local_coordinates_outer - self.eye.unsqueeze(0)), dim=0
)
return {
"d": d_grad,
"M": M_grad,
}
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "ExpGaussian":
d_grad = gradients["d"]
M_grad = gradients["M"]
if "d" not in learning_rates:
learning_rates["d"] = learning_rates["mu"]
if "M" not in learning_rates:
learning_rates["M"] = learning_rates["sigma"]
# Follow gradients for d, and M
update_d = self._follow_gradient("d", d_grad, learning_rates=learning_rates, optimizers=optimizers)
update_M = self._follow_gradient("M", M_grad, learning_rates=learning_rates, optimizers=optimizers)
# Fold into parameters mu, A and A inv
new_mu = self.mu + torch.mv(self.A, update_d)
new_A = self.A @ torch.matrix_exp(0.5 * update_M)
new_A_inv = torch.matrix_exp(-0.5 * update_M) @ self.A_inv
# Return modified distribution
return self.modified_copy(mu=new_mu, sigma=new_A, sigma_inv=new_A_inv)
A: Tensor
property
readonly
¶
Alias for self.sigma, for notational consistency with paper
A_inv: Tensor
property
readonly
¶
Alias for self.sigma_inv, for notational consistency with paper
cov: Tensor
property
readonly
¶
The covariance matrix A^T A
mu: Tensor
property
writable
¶
Getter for mu
Returns:
| Type | Description |
|---|---|
mu (torch.Tensor) |
The center of the search distribution |
sigma: Tensor
property
writable
¶
Getter for sigma
Returns:
| Type | Description |
|---|---|
sigma (torch.Tensor) |
The square root of the covariance matrix |
sigma_inv: Tensor
property
readonly
¶
Getter for sigma_inv
Returns:
| Type | Description |
|---|---|
sigma_inv (torch.Tensor) |
The inverse square root of the covariance matrix |
to_global_coordinates(self, local_coordinates)
¶
Map samples from local coordinate space N(0, I_d) to global coordinate space N(mu, A^T A) This function is the inverse of to_local_coordinates
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
local_coordinates |
torch.Tensor |
The local coordinates sampled from N(0, I_d) |
required |
Returns:
| Type | Description |
|---|---|
global_coordinates (torch.Tensor) |
The global coordinates sampled from N(mu, A^T A) |
Source code in evotorch/distributions.py
def to_global_coordinates(self, local_coordinates: torch.Tensor) -> torch.Tensor:
"""Map samples from local coordinate space N(0, I_d) to global coordinate space N(mu, A^T A)
This function is the inverse of to_local_coordinates
Args:
local_coordinates (torch.Tensor): The local coordinates sampled from N(0, I_d)
Returns:
global_coordinates (torch.Tensor): The global coordinates sampled from N(mu, A^T A)
"""
# Global samples are constructed as x = mu + A z where z is local coordinate
# We use transpose here to simplify the batched application of A
return self.mu.unsqueeze(0) + (self.A @ local_coordinates.T).T
to_local_coordinates(self, global_coordinates)
¶
Map samples from global coordinate space N(mu, A^T A) to local coordinate space N(0, I_d) This function is the inverse of to_global_coordinates
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
global_coordinates |
torch.Tensor |
The global coordinates sampled from N(mu, A^T A) |
required |
Returns:
| Type | Description |
|---|---|
local_coordinates (torch.Tensor) |
The local coordinates sampled from N(0, I_d) |
Source code in evotorch/distributions.py
def to_local_coordinates(self, global_coordinates: torch.Tensor) -> torch.Tensor:
"""Map samples from global coordinate space N(mu, A^T A) to local coordinate space N(0, I_d)
This function is the inverse of to_global_coordinates
Args:
global_coordinates (torch.Tensor): The global coordinates sampled from N(mu, A^T A)
Returns:
local_coordinates (torch.Tensor): The local coordinates sampled from N(0, I_d)
"""
# Global samples are constructed as x = mu + A z where z is local coordinate
# Therefore, we can recover z according to z = A_inv (x - mu)
return (self.A_inv @ (global_coordinates - self.mu.unsqueeze(0)).T).T
update_parameters(self, gradients, *, learning_rates=None, optimizers=None)
¶
Do an update on the distribution by following the given gradients.
It is expected that the inheriting class has its own implementation for this method.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
gradients |
dict |
Gradients, as a dictionary, which will be used for computing the necessary updates. |
required |
learning_rates |
Optional[dict] |
A dictionary which contains learning rates for parameters that will be updated using a learning rate coefficient. |
None |
optimizers |
Optional[dict] |
A dictionary which contains optimizer objects for parameters that will be updated using an adaptive optimizer. |
None |
Returns:
| Type | Description |
|---|---|
ExpGaussian |
The updated copy of the distribution. |
Source code in evotorch/distributions.py
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "ExpGaussian":
d_grad = gradients["d"]
M_grad = gradients["M"]
if "d" not in learning_rates:
learning_rates["d"] = learning_rates["mu"]
if "M" not in learning_rates:
learning_rates["M"] = learning_rates["sigma"]
# Follow gradients for d, and M
update_d = self._follow_gradient("d", d_grad, learning_rates=learning_rates, optimizers=optimizers)
update_M = self._follow_gradient("M", M_grad, learning_rates=learning_rates, optimizers=optimizers)
# Fold into parameters mu, A and A inv
new_mu = self.mu + torch.mv(self.A, update_d)
new_A = self.A @ torch.matrix_exp(0.5 * update_M)
new_A_inv = torch.matrix_exp(-0.5 * update_M) @ self.A_inv
# Return modified distribution
return self.modified_copy(mu=new_mu, sigma=new_A, sigma_inv=new_A_inv)
ExpSeparableGaussian (SeparableGaussian)
¶
Exponential Separable Multivariate Gaussian, as used by SNES
Source code in evotorch/distributions.py
class ExpSeparableGaussian(SeparableGaussian):
"""Exponential Separable Multivariate Gaussian, as used by SNES"""
MANDATORY_PARAMETERS = {"mu", "sigma"}
OPTIONAL_PARAMETERS = set()
PARAMETER_NDIMS = {"mu": 1, "sigma": 1}
def _compute_gradients(self, samples: torch.Tensor, weights: torch.Tensor, ranking_used: Optional[str]) -> dict:
if ranking_used != "nes":
weights = weights / torch.sum(torch.abs(weights))
scaled_noises = samples - self.mu
raw_noises = scaled_noises / self.sigma
mu_grad = total(dot(weights, scaled_noises))
sigma_grad = total(dot(weights, (raw_noises**2) - 1))
return {"mu": mu_grad, "sigma": sigma_grad}
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "ExpSeparableGaussian":
mu_grad = gradients["mu"]
sigma_grad = gradients["sigma"]
new_mu = self.mu + self._follow_gradient("mu", mu_grad, learning_rates=learning_rates, optimizers=optimizers)
new_sigma = self.sigma * torch.exp(
0.5 * self._follow_gradient("sigma", sigma_grad, learning_rates=learning_rates, optimizers=optimizers)
)
return self.modified_copy(mu=new_mu, sigma=new_sigma)
update_parameters(self, gradients, *, learning_rates=None, optimizers=None)
¶
Do an update on the distribution by following the given gradients.
It is expected that the inheriting class has its own implementation for this method.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
gradients |
dict |
Gradients, as a dictionary, which will be used for computing the necessary updates. |
required |
learning_rates |
Optional[dict] |
A dictionary which contains learning rates for parameters that will be updated using a learning rate coefficient. |
None |
optimizers |
Optional[dict] |
A dictionary which contains optimizer objects for parameters that will be updated using an adaptive optimizer. |
None |
Returns:
| Type | Description |
|---|---|
ExpSeparableGaussian |
The updated copy of the distribution. |
Source code in evotorch/distributions.py
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "ExpSeparableGaussian":
mu_grad = gradients["mu"]
sigma_grad = gradients["sigma"]
new_mu = self.mu + self._follow_gradient("mu", mu_grad, learning_rates=learning_rates, optimizers=optimizers)
new_sigma = self.sigma * torch.exp(
0.5 * self._follow_gradient("sigma", sigma_grad, learning_rates=learning_rates, optimizers=optimizers)
)
return self.modified_copy(mu=new_mu, sigma=new_sigma)
FunctionalGradEstimator
¶
Represents the callable object returned by make_functional_grad_estimator.
Please see the documentation of make_functional_grad_estimator
Source code in evotorch/distributions.py
class FunctionalGradEstimator:
"""
Represents the callable object returned by `make_functional_grad_estimator`.
Please see the documentation of
[make_functional_grad_estimator][evotorch.distributions.make_functional_grad_estimator]
"""
def __init__(
self,
distribution_class: Type,
*,
function: Optional[Callable] = None,
objective_sense: str,
required_parameters: Iterable,
fixed_parameters: Optional[dict] = None,
ranking_method: Optional[str] = None,
return_samples: bool = False,
return_fitnesses: bool = False,
):
from .decorators import expects_ndim
self.__function = function
self.__objective_sense = None if objective_sense is None else str(objective_sense)
self.__distribution_class = distribution_class
self.__required_parameters = [str(element) for element in required_parameters]
if len(self.__required_parameters) == 0:
raise TypeError("`required_parameters` cannot be empty")
self.__required_param_pos = {
required_parameter: i_parameter for i_parameter, required_parameter in enumerate(self.__required_parameters)
}
self.__num_required_parameters = len(self.__required_parameters)
self.__fixed_parameters = {} if fixed_parameters is None else fixed_parameters
self.__return_samples = bool(return_samples)
self.__return_fitnesses = bool(return_fitnesses)
self.__ranking_method = None if ranking_method is None else str(ranking_method)
if self.__function is None:
leftmost_ndims = (None, None, 2, 1)
else:
leftmost_ndims = (
None,
None,
None,
)
self.__grad_batch = expects_ndim(
self.__grad,
leftmost_ndims + tuple(_get_param_ndim(distribution_class, p) for p in self.__required_parameters),
randomness="different",
)
def __grad(self, *args) -> torch.Tensor:
objective_sense = args[0]
ranking_method = args[1]
if self.__function is None:
samples = args[2]
fitnesses = args[3]
vectors = args[4:]
[num_solutions, _] = samples.shape
[num_fitnesses] = fitnesses.shape
if num_solutions != num_fitnesses:
raise ValueError("The length of the fitness vector does not match the number of samples")
else:
num_solutions = args[2]
vectors = args[3:]
samples = None
fitnesses = None
parameters = {**dict(zip(self.__required_parameters, vectors)), **(self.__fixed_parameters)}
distribution = self.__distribution_class(parameters)
if samples is None:
samples = distribution.sample(num_solutions)
fitnesses = self.__function(samples)
grads = distribution.compute_gradients(
samples, fitnesses, objective_sense=objective_sense, ranking_method=ranking_method
)
if self.__return_samples and self.__return_fitnesses:
return GradsWithSamplesAndFitnesses(grads=grads, samples=samples, fitnesses=fitnesses)
elif self.__return_samples:
return GradsWithSamples(grads=grads, samples=samples)
elif self.__return_fitnesses:
return GradsWithFitnesses(grads=grads, fitnesses=fitnesses)
else:
return grads
def __call__(self, *args, **parameter_kwargs) -> torch.Tensor:
parameters_need_filtering = False
if "objective_sense" in parameter_kwargs:
objective_sense = parameter_kwargs["objective_sense"]
parameters_need_filtering = True
else:
objective_sense = self.__objective_sense
if self.__objective_sense is None:
raise ValueError(
"The gradient estimator was not given an `objective_sense`, neither at its phase of initialization,"
" nor when it got called."
)
if "ranking_method" in parameter_kwargs:
ranking_method = parameter_kwargs["ranking_method"]
parameters_need_filtering = True
else:
ranking_method = self.__ranking_method
if parameters_need_filtering:
parameter_kwargs = {
k: v for k, v in parameter_kwargs.items() if k not in ("objective_sense", "ranking_method")
}
if self.__function is None:
samples = args[0]
fitnesses = args[1]
num_solutions = None
parameter_args = args[2:]
else:
samples = None
fitnesses = None
num_solutions = args[0]
parameter_args = args[1:]
num_parameter_args = len(parameter_args)
num_parameter_kwargs = len(parameter_kwargs)
if (num_parameter_args == 0) and (num_parameter_kwargs == self.__num_required_parameters):
parameters = [None] * self.__num_required_parameters
for parameter_name, parameter_value in parameter_kwargs.items():
parameter_pos = self.__required_param_pos[parameter_name]
parameters[parameter_pos] = parameter_value
elif (num_parameter_args == self.__num_required_parameters) and (num_parameter_kwargs == 0):
parameters = parameter_args
elif (num_parameter_args == 0) and (num_parameter_kwargs == 0):
raise TypeError("Missing parameter arguments")
elif (num_parameter_args > 0) and (num_parameter_kwargs > 0):
raise TypeError(
"Specifying some of the distribution parameters via positional arguments and some others"
" via keyword arguments is not supported."
" Please provide the distribution parameters only via positional arguments"
" or only via keyword arguments."
)
else:
raise TypeError("Invalid number of arguments")
if self.__function is None:
return self.__grad_batch(objective_sense, ranking_method, samples, fitnesses, *parameters)
else:
return self.__grad_batch(objective_sense, ranking_method, num_solutions, *parameters)
FunctionalSampler
¶
Represents a sampler returned by make_functional_sampler.
Please see the documentation of make_functional_sampler.
Source code in evotorch/distributions.py
class FunctionalSampler:
"""
Represents a sampler returned by `make_functional_sampler`.
Please see the documentation of
[make_functional_sampler][evotorch.distributions.make_functional_sampler].
"""
def __init__(
self, distribution_class: Type, *, required_parameters: Iterable, fixed_parameters: Optional[dict] = None
):
from .decorators import expects_ndim
self.__distribution_class = distribution_class
self.__required_parameters = [str(element) for element in required_parameters]
if len(self.__required_parameters) == 0:
raise TypeError("`required_parameters` cannot be empty")
self.__required_param_pos = {
required_parameter: i_parameter for i_parameter, required_parameter in enumerate(self.__required_parameters)
}
self.__num_required_parameters = len(self.__required_parameters)
self.__fixed_parameters = {} if fixed_parameters is None else fixed_parameters
self.__sample_batch = expects_ndim(
self.__sample,
(None,) + tuple(_get_param_ndim(distribution_class, p) for p in self.__required_parameters),
randomness="different",
)
def __sample(self, num_solutions: int, *parameters) -> torch.Tensor:
parameters = {**dict(zip(self.__required_parameters, parameters)), **(self.__fixed_parameters)}
if self.__distribution_class.functional_sample is NotImplemented:
distribution = self.__distribution_class(parameters)
return distribution.sample(num_solutions)
else:
return self.__distribution_class.functional_sample(num_solutions, parameters)
def __call__(self, num_samples: int, *parameter_args, **parameter_kwargs) -> torch.Tensor:
num_parameter_args = len(parameter_args)
num_parameter_kwargs = len(parameter_kwargs)
if (num_parameter_args == 0) and (num_parameter_kwargs == self.__num_required_parameters):
parameters = [None] * self.__num_required_parameters
for parameter_name, parameter_value in parameter_kwargs.items():
parameter_pos = self.__required_param_pos[parameter_name]
parameters[parameter_pos] = parameter_value
elif (num_parameter_args == self.__num_required_parameters) and (num_parameter_kwargs == 0):
parameters = parameter_args
elif (num_parameter_args == 0) and (num_parameter_kwargs == 0):
raise TypeError("Missing parameter arguments")
elif (num_parameter_args > 0) and (num_parameter_kwargs > 0):
raise TypeError(
"Specifying some of the distribution parameters via positional arguments and some others"
" via keyword arguments is not supported."
" Please provide the distribution parameters only via positional arguments"
" or only via keyword arguments."
)
else:
raise TypeError("Invalid number of arguments")
return self.__sample_batch(num_samples, *parameters)
GradsWithFitnesses (tuple)
¶
GradsWithSamples (tuple)
¶
GradsWithSamplesAndFitnesses (tuple)
¶
SeparableGaussian (Distribution)
¶
Separable Multivariate Gaussian, as used by PGPE
Source code in evotorch/distributions.py
class SeparableGaussian(Distribution):
"""Separable Multivariate Gaussian, as used by PGPE"""
MANDATORY_PARAMETERS = {"mu", "sigma"}
OPTIONAL_PARAMETERS = {"divide_mu_grad_by", "divide_sigma_grad_by", "parenthood_ratio"}
PARAMETER_NDIMS = {"mu": 1, "sigma": 1}
@classmethod
def _unbatched_functional_sample(cls, num_solutions: int, mu: torch.Tensor, sigma: torch.Tensor) -> torch.Tensor:
[L] = mu.shape
[sigma_L] = sigma.shape
if L != sigma_L:
raise ValueError(f"The lengths of `mu` ({L}) and `sigma` ({sigma_L}) do not match.")
mu = mu.expand(int(num_solutions), L)
return torch.normal(mu, sigma)
@classmethod
def functional_sample(cls, num_solutions: int, parameters: dict) -> torch.Tensor:
"""
Sample and return separable Gaussian noise
This is a static utility method, which allows one to sample separable
Gaussian noise, without having to instantiate the distribution class
`SeparableGaussian`.
Args:
num_solutions: Number of solutions (or 1-dimensional tensors)
that will be sampled.
parameters: A parameter dictionary. Within this parameter
dictionary, the item `mu` is expected to store the mean, and
the item `sigma` is expected to store the standard deviation,
each in the form of a 1-dimensional tensor.
Returns:
Sampled separable Gaussian noise, as a PyTorch tensor.
If `mu` and/or `sigma` was given as tensors with 2 or more
dimensions (instead of only 1 dimension), the extra leftmost
dimensions will be interpreted as batch dimensions, and therefore,
this returned tensor will also have batch dimensions.
"""
from .decorators import expects_ndim
for k in parameters.keys():
if (k not in cls.MANDATORY_PARAMETERS) and (k not in cls.OPTIONAL_PARAMETERS):
raise ValueError(f"{cls.__name__} encountered an unrecognized parameter: {repr(k)}")
mu = parameters["mu"]
sigma = parameters["sigma"]
return expects_ndim(cls._unbatched_functional_sample, (None, 1, 1), randomness="different")(
num_solutions, mu, sigma
)
def __init__(
self,
parameters: dict,
*,
solution_length: Optional[int] = None,
device: Optional[Device] = None,
dtype: Optional[DType] = None,
):
[mu_length] = parameters["mu"].shape
[sigma_length] = parameters["sigma"].shape
if solution_length is None:
solution_length = mu_length
else:
if solution_length != mu_length:
raise ValueError(
f"The argument `solution_length` does not match the length of `mu` provided in `parameters`."
f" solution_length={solution_length},"
f' parameters["mu"]={mu_length}.'
)
if mu_length != sigma_length:
raise ValueError(
f"The tensors `mu` and `sigma` provided within `parameters` have mismatching lengths."
f' parameters["mu"]={mu_length},'
f' parameters["sigma"]={sigma_length}.'
)
super().__init__(
solution_length=solution_length,
parameters=parameters,
device=device,
dtype=dtype,
)
@property
def mu(self) -> torch.Tensor:
return self.parameters["mu"]
@mu.setter
def mu(self, new_mu: Iterable):
self.parameters["mu"] = torch.as_tensor(new_mu, dtype=self.dtype, device=self.device)
@property
def sigma(self) -> torch.Tensor:
return self.parameters["sigma"]
@sigma.setter
def sigma(self, new_sigma: Iterable):
self.parameters["sigma"] = torch.as_tensor(new_sigma, dtype=self.dtype, device=self.device)
def _fill(self, out: torch.Tensor, *, generator: Optional[torch.Generator] = None):
self.make_gaussian(out=out, center=self.mu, stdev=self.sigma, generator=generator)
def _divide_grad(self, param_name: str, grad: torch.Tensor, weights: torch.Tensor) -> torch.Tensor:
option = f"divide_{param_name}_grad_by"
if option in self.parameters:
div_by_what = self.parameters[option]
if div_by_what == "num_solutions":
[num_solutions] = weights.shape
grad = grad / num_solutions
elif div_by_what == "num_directions":
[num_solutions] = weights.shape
num_directions = num_solutions // 2
grad = grad / num_directions
elif div_by_what == "total_weight":
total_weight = torch.sum(torch.abs(weights))
grad = grad / total_weight
elif div_by_what == "weight_stdev":
weight_stdev = torch.std(weights)
grad = grad / weight_stdev
else:
raise ValueError(f"The parameter {option} has an unrecognized value: {div_by_what}")
return grad
def _compute_gradients_via_parenthood_ratio(self, samples: torch.Tensor, weights: torch.Tensor) -> dict:
[num_samples, _] = samples.shape
num_elites = math.floor(num_samples * self.parameters["parenthood_ratio"])
elite_indices = weights.argsort(descending=True)[:num_elites]
elites = samples[elite_indices, :]
return {
"mu": torch.mean(elites, dim=0) - self.parameters["mu"],
"sigma": torch.std(elites, dim=0) - self.parameters["sigma"],
}
def _compute_gradients(self, samples: torch.Tensor, weights: torch.Tensor, ranking_used: Optional[str]) -> dict:
if "parenthood_ratio" in self.parameters:
return self._compute_gradients_via_parenthood_ratio(samples, weights)
else:
mu = self.mu
sigma = self.sigma
# Compute the scaled noises, that is, the noise vectors which
# were used for generating the solutions
# (solution = scaled_noise + center)
scaled_noises = samples - mu
# Make sure that the weights (utilities) are 0-centered
# (Otherwise the formulations would have to consider a bias term)
if ranking_used not in ("centered", "normalized"):
weights = weights - torch.mean(weights)
mu_grad = self._divide_grad(
"mu",
total(dot(weights, scaled_noises)),
weights,
)
sigma_grad = self._divide_grad(
"sigma",
total(dot(weights, ((scaled_noises**2) - (sigma**2)) / sigma)),
weights,
)
return {
"mu": mu_grad,
"sigma": sigma_grad,
}
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "SeparableGaussian":
mu_grad = gradients["mu"]
sigma_grad = gradients["sigma"]
new_mu = self.mu + self._follow_gradient("mu", mu_grad, learning_rates=learning_rates, optimizers=optimizers)
new_sigma = self.sigma + self._follow_gradient(
"sigma", sigma_grad, learning_rates=learning_rates, optimizers=optimizers
)
return self.modified_copy(mu=new_mu, sigma=new_sigma)
def relative_entropy(dist_0: "SeparableGaussian", dist_1: "SeparableGaussian") -> float:
mu_0 = dist_0.parameters["mu"]
mu_1 = dist_1.parameters["mu"]
sigma_0 = dist_0.parameters["sigma"]
sigma_1 = dist_1.parameters["sigma"]
cov_0 = sigma_0.pow(2.0)
cov_1 = sigma_1.pow(2.0)
mu_delta = mu_1 - mu_0
trace_cov = torch.sum(cov_0 / cov_1)
k = dist_0.solution_length
scaled_mu = torch.sum(mu_delta.pow(2.0) / cov_1)
log_det = torch.sum(torch.log(cov_1)) - torch.sum(torch.log(cov_0))
return 0.5 * (trace_cov - k + scaled_mu + log_det)
functional_sample(num_solutions, parameters)
classmethod
¶
Sample and return separable Gaussian noise
This is a static utility method, which allows one to sample separable
Gaussian noise, without having to instantiate the distribution class
SeparableGaussian.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
num_solutions |
int |
Number of solutions (or 1-dimensional tensors) that will be sampled. |
required |
parameters |
dict |
A parameter dictionary. Within this parameter
dictionary, the item |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
Sampled separable Gaussian noise, as a PyTorch tensor.
If |
Source code in evotorch/distributions.py
@classmethod
def functional_sample(cls, num_solutions: int, parameters: dict) -> torch.Tensor:
"""
Sample and return separable Gaussian noise
This is a static utility method, which allows one to sample separable
Gaussian noise, without having to instantiate the distribution class
`SeparableGaussian`.
Args:
num_solutions: Number of solutions (or 1-dimensional tensors)
that will be sampled.
parameters: A parameter dictionary. Within this parameter
dictionary, the item `mu` is expected to store the mean, and
the item `sigma` is expected to store the standard deviation,
each in the form of a 1-dimensional tensor.
Returns:
Sampled separable Gaussian noise, as a PyTorch tensor.
If `mu` and/or `sigma` was given as tensors with 2 or more
dimensions (instead of only 1 dimension), the extra leftmost
dimensions will be interpreted as batch dimensions, and therefore,
this returned tensor will also have batch dimensions.
"""
from .decorators import expects_ndim
for k in parameters.keys():
if (k not in cls.MANDATORY_PARAMETERS) and (k not in cls.OPTIONAL_PARAMETERS):
raise ValueError(f"{cls.__name__} encountered an unrecognized parameter: {repr(k)}")
mu = parameters["mu"]
sigma = parameters["sigma"]
return expects_ndim(cls._unbatched_functional_sample, (None, 1, 1), randomness="different")(
num_solutions, mu, sigma
)
update_parameters(self, gradients, *, learning_rates=None, optimizers=None)
¶
Do an update on the distribution by following the given gradients.
It is expected that the inheriting class has its own implementation for this method.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
gradients |
dict |
Gradients, as a dictionary, which will be used for computing the necessary updates. |
required |
learning_rates |
Optional[dict] |
A dictionary which contains learning rates for parameters that will be updated using a learning rate coefficient. |
None |
optimizers |
Optional[dict] |
A dictionary which contains optimizer objects for parameters that will be updated using an adaptive optimizer. |
None |
Returns:
| Type | Description |
|---|---|
SeparableGaussian |
The updated copy of the distribution. |
Source code in evotorch/distributions.py
def update_parameters(
self,
gradients: dict,
*,
learning_rates: Optional[dict] = None,
optimizers: Optional[dict] = None,
) -> "SeparableGaussian":
mu_grad = gradients["mu"]
sigma_grad = gradients["sigma"]
new_mu = self.mu + self._follow_gradient("mu", mu_grad, learning_rates=learning_rates, optimizers=optimizers)
new_sigma = self.sigma + self._follow_gradient(
"sigma", sigma_grad, learning_rates=learning_rates, optimizers=optimizers
)
return self.modified_copy(mu=new_mu, sigma=new_sigma)
SymmetricSeparableGaussian (SeparableGaussian)
¶
Symmetric (antithetic) separable Gaussian distribution as used by PGPE.
For example, if the desired number of samples (or number of solutions,
provided via the argument num_solutions) is 6, 3 "directions" will
be sampled. Each direction is a pair of solutions, where one of the
solutions is the center vector plus perturbation, and the other
solution is the center vector minus the same perturbation. Therefore,
such a symmetric population of size 6 looks like this:
___
solution[0]: center + sampled_perturbation[0] \
> direction0
solution[1]: center - sampled_perturbation[1] ___/
___
solution[2]: center + sampled_perturbation[2] \
> direction1
solution[3]: center - sampled_perturbation[3] ___/
___
solution[4]: center + sampled_perturbation[4] \
> direction2
solution[5]: center - sampled_perturbation[5] ___/
Source code in evotorch/distributions.py
class SymmetricSeparableGaussian(SeparableGaussian):
r"""
Symmetric (antithetic) separable Gaussian distribution as used by PGPE.
For example, if the desired number of samples (or number of solutions,
provided via the argument `num_solutions`) is 6, 3 "directions" will
be sampled. Each direction is a pair of solutions, where one of the
solutions is the center vector plus perturbation, and the other
solution is the center vector minus the same perturbation. Therefore,
such a symmetric population of size 6 looks like this:
```
___
solution[0]: center + sampled_perturbation[0] \
> direction0
solution[1]: center - sampled_perturbation[1] ___/
___
solution[2]: center + sampled_perturbation[2] \
> direction1
solution[3]: center - sampled_perturbation[3] ___/
___
solution[4]: center + sampled_perturbation[4] \
> direction2
solution[5]: center - sampled_perturbation[5] ___/
```
"""
MANDATORY_PARAMETERS = {"mu", "sigma"}
OPTIONAL_PARAMETERS = {"divide_mu_grad_by", "divide_sigma_grad_by", "parenthood_ratio"}
PARAMETER_NDIMS = {"mu": 1, "sigma": 1}
@classmethod
def _unbatched_functional_sample(cls, num_solutions: int, mu: torch.Tensor, sigma: torch.Tensor) -> torch.Tensor:
zeros = torch.zeros_like(mu)
num_solutions = int(num_solutions)
if (num_solutions % 2) != 0:
raise ValueError(
f"Number of solutions to be sampled from {cls.__name__} must be an even number."
f" However, the encountered `num_solutions` is {num_solutions}."
)
num_directions = num_solutions // 2
positive_ends = SeparableGaussian._unbatched_functional_sample(num_directions, zeros, sigma)
negative_ends = -positive_ends
positive_ends += mu
negative_ends += mu
combined_samples = vmap(torch.stack)([positive_ends, negative_ends]).reshape(-1, positive_ends.shape[-1])
return combined_samples
@classmethod
def functional_sample(cls, num_solutions: int, parameters: dict) -> torch.Tensor:
"""
Sample and return symmetric separable Gaussian noise
This is a static utility method, which allows one to sample symmetric
separable Gaussian noise, without having to instantiate the
distribution class `SymmetricSeparableGaussian`.
Args:
num_solutions: Number of solutions (or 1-dimensional tensors)
that will be sampled. Note that, since this distribution is
symmetric, `num_solutions` must be even.
parameters: A parameter dictionary. Within this parameter
dictionary, the item `mu` is expected to store the mean, and
the item `sigma` is expected to store the standard deviation,
each in the form of a 1-dimensional tensor.
Returns:
Sampled symmetric separable Gaussian noise, as a PyTorch tensor.
If `mu` and/or `sigma` was given as tensors with 2 or more
dimensions (instead of only 1 dimension), the extra leftmost
dimensions will be interpreted as batch dimensions, and therefore,
this returned tensor will also have batch dimensions.
"""
from .decorators import expects_ndim
for k in parameters.keys():
if (k not in cls.MANDATORY_PARAMETERS) and (k not in cls.OPTIONAL_PARAMETERS):
raise ValueError(f"{cls.__name__} encountered an unrecognized parameter: {repr(k)}")
mu = parameters["mu"]
sigma = parameters["sigma"]
return expects_ndim(cls._unbatched_functional_sample, (None, 1, 1), randomness="different")(
num_solutions, mu, sigma
)
def _fill(self, out: torch.Tensor, *, generator: Optional[torch.Generator] = None):
self.make_gaussian(out=out, center=self.mu, stdev=self.sigma, symmetric=True, generator=generator)
def _compute_gradients(
self,
samples: torch.Tensor,
weights: torch.Tensor,
ranking_used: Optional[str],
) -> dict:
if "parenthood_ratio" in self.parameters:
return self._compute_gradients_via_parenthood_ratio(samples, weights)
else:
mu = self.mu
sigma = self.sigma
# Make sure that the weights (utilities) are 0-centered
# (Otherwise the formulations would have to consider a bias term)
if ranking_used not in ("centered", "normalized"):
weights = weights - torch.mean(weights)
[nslns] = weights.shape
# ndirs = nslns // 2
# Compute the scaled noises, that is, the noise vectors which
# were used for generating the solutions
# (solution = scaled_noise + center)
scaled_noises = samples[0::2] - mu
# Separate the plus and the minus ends of the directions
fdplus = weights[0::2]
fdminus = weights[1::2]
# Considering that the population is stored like this:
# _
# solution0: center + scaled_noise0 \
# > direction0
# solution1: center - scaled_noise0 _/
# _
# solution2: center + scaled_noise1 \
# > direction1
# solution3: center - scaled_noise1 _/
#
# ...
# fdplus[0] becomes the utility of the plus end of direction0
# (i.e. utility of solution0)
# fdminus[0] becomes the utility of the minus end of direction0
# (i.e. utility of solution1)
# fdplus[1] becomes the utility of the plus end of direction1
# (i.e. utility of solution2)
# fdminus[1] becomes the utility of the minus end of direction1
# (i.e. utility of solution3)
# ... and so on...
grad_mu = self._divide_grad("mu", total(dot((fdplus - fdminus) / 2, scaled_noises)), weights)
grad_sigma = self._divide_grad(
"sigma",
total(dot(((fdplus + fdminus) / 2), ((scaled_noises**2) - (sigma**2)) / sigma)),
weights,
)
return {
"mu": grad_mu,
"sigma": grad_sigma,
}
functional_sample(num_solutions, parameters)
classmethod
¶
Sample and return symmetric separable Gaussian noise
This is a static utility method, which allows one to sample symmetric
separable Gaussian noise, without having to instantiate the
distribution class SymmetricSeparableGaussian.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
num_solutions |
int |
Number of solutions (or 1-dimensional tensors)
that will be sampled. Note that, since this distribution is
symmetric, |
required |
parameters |
dict |
A parameter dictionary. Within this parameter
dictionary, the item |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
Sampled symmetric separable Gaussian noise, as a PyTorch tensor.
If |
Source code in evotorch/distributions.py
@classmethod
def functional_sample(cls, num_solutions: int, parameters: dict) -> torch.Tensor:
"""
Sample and return symmetric separable Gaussian noise
This is a static utility method, which allows one to sample symmetric
separable Gaussian noise, without having to instantiate the
distribution class `SymmetricSeparableGaussian`.
Args:
num_solutions: Number of solutions (or 1-dimensional tensors)
that will be sampled. Note that, since this distribution is
symmetric, `num_solutions` must be even.
parameters: A parameter dictionary. Within this parameter
dictionary, the item `mu` is expected to store the mean, and
the item `sigma` is expected to store the standard deviation,
each in the form of a 1-dimensional tensor.
Returns:
Sampled symmetric separable Gaussian noise, as a PyTorch tensor.
If `mu` and/or `sigma` was given as tensors with 2 or more
dimensions (instead of only 1 dimension), the extra leftmost
dimensions will be interpreted as batch dimensions, and therefore,
this returned tensor will also have batch dimensions.
"""
from .decorators import expects_ndim
for k in parameters.keys():
if (k not in cls.MANDATORY_PARAMETERS) and (k not in cls.OPTIONAL_PARAMETERS):
raise ValueError(f"{cls.__name__} encountered an unrecognized parameter: {repr(k)}")
mu = parameters["mu"]
sigma = parameters["sigma"]
return expects_ndim(cls._unbatched_functional_sample, (None, 1, 1), randomness="different")(
num_solutions, mu, sigma
)
make_functional_grad_estimator(distribution_class, *, required_parameters, function=None, objective_sense=None, fixed_parameters=None, ranking_method=None, return_samples=False, return_fitnesses=False)
¶
Make a stateless gradient estimator function.
The returned function estimates gradients for the parameters of the specified distribution, either with the help of a fitness function, or with the help of a pair of tensors representing the samples (or solutions) and their associated fitnesses.
Usage 1: with the help of a fitness function.
Let us assume that we have a fitness function f, which receives a
matrix (i.e. 2-dimensional tensor) and returns a vector (i.e. a
1-dimensional tensor), where each the i-th row of the matrix represents
the i-th solution, and i-th element of the returned vector represents
the fitness of the i-th solution.
A functional gradient estimator for this function can be created like
this:
from evotorch.distributions import (
SymmetricSeparableGaussian,
make_functional_grad_estimator,
)
def f(x: torch.Tensor) -> torch.Tensor: ...
fgrad = make_functional_grad_estimator(
# The gradient estimator will use this distribution:
SymmetricSeparableGaussian,
# The gradient estimator will be bound to this fitness function:
function=f,
# We want to maximize the fitnesses returned by `f`
# (use "min" for minimizing them)
objective_sense="max",
# The distribution parameters "mu" and "sigma" are to be passed
# as arguments every time we call it as a function.
required_parameters=["mu", "sigma"],
# The fitnesses will be ranked according to this method:
ranking_method="centered", # the default is "raw"
)
Now that we have our gradient estimator fgrad, we can use it as a
function:
current_mu = ... # mu parameter vector
current_sigma = ... # sigma parameter vector
num_samples = ... # number of samples (temporary population size)
gradients = fgrad(num_samples, current_mu, current_sigma)
# or, alternatively:
# gradients = fgrad(num_samples, mu=current_mu, sigma=current_sigma)
# At this point, we have our `gradients`, which is in the form of a
# dictionary. Gradients for the parameters mu and sigma can be obtained
# from this dictionary like this:
grad_for_mu = gradients["mu"]
grad_for_sigma = gradients["sigma"]
Usage 2: without an explicit fitness function. Let us imagine a scenario where the procedure of computing the fitnesses is not so straightforward and therefore it is not possible to wrap it within a single fitness function. For such cases, we can create and use a gradient estimator that is not bound to any such fitness function:
from evotorch.distributions import (
SymmetricSeparableGaussian,
make_functional_sampler,
make_functional_grad_estimator,
)
estimate_grads = make_functional_grad_estimator(
# The gradient estimator will use this distribution:
SymmetricSeparableGaussian,
# We want to maximize the fitnesses (use "min" for minimizing them)
objective_sense="max",
# The distribution parameters "mu" and "sigma" are to be passed
# as arguments every time we call it as a function.
required_parameters=["mu", "sigma"],
# The fitnesses will be ranked according to this method:
ranking_method="centered", # the default is "raw"
)
Note that without being bound to any fitness function, estimate_grad
will ask us to provide the samples and the fitnesses. A practical way
of obtaining such samples is to have a functional sampler:
get_samples = make_functional_sampler(
SymmetricSeparableGaussian,
required_parameters=["mu", "sigma"],
)
Now we are ready to use our sampler and our estimator:
current_mu = ... # mu parameter vector
current_sigma = ... # sigma parameter vector
num_samples = ... # number of samples (temporary population size)
samples = get_samples(num_samples, current_mu, current_sigma)
# or, alternatively:
# samples = get_samples(num_samples, mu=current_mu, sigma=current_sigma)
fitnesses = ... # code to compute fitnesses from the samples goes here
gradients = estimate_grads(samples, fitnesses, current_mu, current_sigma)
# or, alternatively:
# gradients = estimate_grads(
# samples, fitnesses, mu=current_mu, sigma=current_sigma
# )
# At this point, we have our `gradients`, which is in the form of a
# dictionary. Gradients for the parameters mu and sigma can be obtained
# from this dictionary like this:
grad_for_mu = gradients["mu"]
grad_for_sigma = gradients["sigma"]
Batched gradient estimation.
The function returned by make_functional_grad_estimator is compatible
with vmap. If the estimator is bound to a specific fitness function,
that fitness function should also be compatible with vmap. If the
fitness function is not vmap-compatible, or if its behavior is
unexpected in the presence of vmap, then, consider instantiating
the gradient estimator without binding it to a fitness function.
As an alternative to vmap, a functional sampler has built-in support for
batched sampling (which actually still uses vmap internally).
Let us again consider our example estimator, estimate_grads.
In this example, the parameters current_mu and current_sigma would be
1-dimensional tensors in the non-batched case (because the distribution
SymmetricSeparableGaussian expects the parameters mu and sigma as
1-dimensional tensors, as can be observed from the class attribute
SymmetricSeparableGaussian.PARAMETER_NDIMS).
If estimate_grads is given mu and/or sigma with more than 1
dimensions, those extra leftmost dimensions will be considered as batch
dimensions, and therefore the resulting sample tensor will have extra
leftmost dimensions too.
Declaring fixed parameters.
A functional gradient estimator can be created in such a way that some of
the parameters are pre-defined and only a subset of the mandatory parameters
are expected via arguments. For example, a gradient estimator that samples
from SymmetricSeparableGaussian with a fixed sigma can be defined like
this:
predefined_sigma = ... # The constant sigma goes here
fgrad2 = make_functional_sampler(
SymmetricSeparableGaussian,
function=f,
objective_sense="max",
required_parameters=["mu"],
fixed_parameters={"sigma": predefined_sigma},
ranking_method="centered",
)
The function fgrad2 can be called like this:
gradients = fgrad2(num_samples, current_mu)
# or, alternatively:
# gradients = fgrad2(num_samples, mu=current_mu)
Specifying objective_sense and/or ranking_method later.
One can omit objective_sense and ranking_method while making the
functional gradient estimator, and specify them later at the moment of
estimation. For example:
fgrad3 = make_functional_sampler(
SymmetricSeparableGaussian,
function=f,
required_parameters=["mu", "sigma"],
# Notice: `objective_sense` and `ranking_method` are omitted
)
...
mu = ...
sigma = ...
gradients = fgrad3(
num_samples,
mu=mu,
sigma=sigma,
objective_sense="max",
ranking_method="centered",
)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
function |
Optional[Callable] |
The fitness function that will be called for estimating
the gradients. If provided, the first positional argument of the
returned gradient estimator will be the number of solutions.
If omitted, the first and second positional arguments of the
returned gradient estimator will be the samples (solutions)
and fitnesses.
Please note that this |
None |
objective_sense |
Optional[str] |
Specify this as "max" if a higher fitness value means
better solution. Specify this as "min" if a higher fitness value
means worse solution. Please note that, if |
None |
required_parameters |
Iterable |
A list of strings, each string being the name of a distribution parameter. The order of this list determines the order of parameter-related positional arguments in the returned callable object. |
required |
fixed_parameters |
Optional[dict] |
A dictionary where the keys are parameter names (as strings), and the values are pre-defined parameter values. |
None |
ranking_method |
Optional[str] |
Give a string here if you would like the fitnesses to be ranked first. Possible values are "centered", "linear", "raw". |
None |
return_samples |
bool |
Set this as True if you would like the gradient estimator to return not just the gradients, but also the samples (solutions) that were used for estimating the gradients. |
False |
return_fitnesses |
bool |
Set this as True if you would like the gradient estimator to return not just the gradients, but also the fitnesses that were used for estimating the gradients. |
False |
Returns:
| Type | Description |
|---|---|
Callable |
A callable object whose function is to estimate gradients. |
Source code in evotorch/distributions.py
def make_functional_grad_estimator(
distribution_class: Type,
*,
required_parameters: Iterable,
function: Optional[Callable] = None,
objective_sense: Optional[str] = None,
fixed_parameters: Optional[dict] = None,
ranking_method: Optional[str] = None,
return_samples: bool = False,
return_fitnesses: bool = False,
) -> Callable:
"""
Make a stateless gradient estimator function.
The returned function estimates gradients for the parameters of the
specified distribution, either with the help of a fitness function,
or with the help of a pair of tensors representing the samples
(or solutions) and their associated fitnesses.
**Usage 1: with the help of a fitness function.**
Let us assume that we have a fitness function `f`, which receives a
matrix (i.e. 2-dimensional tensor) and returns a vector (i.e. a
1-dimensional tensor), where each the i-th row of the matrix represents
the i-th solution, and i-th element of the returned vector represents
the fitness of the i-th solution.
A functional gradient estimator for this function can be created like
this:
```python
from evotorch.distributions import (
SymmetricSeparableGaussian,
make_functional_grad_estimator,
)
def f(x: torch.Tensor) -> torch.Tensor: ...
fgrad = make_functional_grad_estimator(
# The gradient estimator will use this distribution:
SymmetricSeparableGaussian,
# The gradient estimator will be bound to this fitness function:
function=f,
# We want to maximize the fitnesses returned by `f`
# (use "min" for minimizing them)
objective_sense="max",
# The distribution parameters "mu" and "sigma" are to be passed
# as arguments every time we call it as a function.
required_parameters=["mu", "sigma"],
# The fitnesses will be ranked according to this method:
ranking_method="centered", # the default is "raw"
)
```
Now that we have our gradient estimator `fgrad`, we can use it as a
function:
```python
current_mu = ... # mu parameter vector
current_sigma = ... # sigma parameter vector
num_samples = ... # number of samples (temporary population size)
gradients = fgrad(num_samples, current_mu, current_sigma)
# or, alternatively:
# gradients = fgrad(num_samples, mu=current_mu, sigma=current_sigma)
# At this point, we have our `gradients`, which is in the form of a
# dictionary. Gradients for the parameters mu and sigma can be obtained
# from this dictionary like this:
grad_for_mu = gradients["mu"]
grad_for_sigma = gradients["sigma"]
```
**Usage 2: without an explicit fitness function.**
Let us imagine a scenario where the procedure of computing the fitnesses
is not so straightforward and therefore it is not possible to wrap it
within a single fitness function. For such cases, we can create and use a
gradient estimator that is not bound to any such fitness function:
```python
from evotorch.distributions import (
SymmetricSeparableGaussian,
make_functional_sampler,
make_functional_grad_estimator,
)
estimate_grads = make_functional_grad_estimator(
# The gradient estimator will use this distribution:
SymmetricSeparableGaussian,
# We want to maximize the fitnesses (use "min" for minimizing them)
objective_sense="max",
# The distribution parameters "mu" and "sigma" are to be passed
# as arguments every time we call it as a function.
required_parameters=["mu", "sigma"],
# The fitnesses will be ranked according to this method:
ranking_method="centered", # the default is "raw"
)
```
Note that without being bound to any fitness function, `estimate_grad`
will ask us to provide the samples and the fitnesses. A practical way
of obtaining such samples is to have a functional sampler:
```python
get_samples = make_functional_sampler(
SymmetricSeparableGaussian,
required_parameters=["mu", "sigma"],
)
```
Now we are ready to use our sampler and our estimator:
```python
current_mu = ... # mu parameter vector
current_sigma = ... # sigma parameter vector
num_samples = ... # number of samples (temporary population size)
samples = get_samples(num_samples, current_mu, current_sigma)
# or, alternatively:
# samples = get_samples(num_samples, mu=current_mu, sigma=current_sigma)
fitnesses = ... # code to compute fitnesses from the samples goes here
gradients = estimate_grads(samples, fitnesses, current_mu, current_sigma)
# or, alternatively:
# gradients = estimate_grads(
# samples, fitnesses, mu=current_mu, sigma=current_sigma
# )
# At this point, we have our `gradients`, which is in the form of a
# dictionary. Gradients for the parameters mu and sigma can be obtained
# from this dictionary like this:
grad_for_mu = gradients["mu"]
grad_for_sigma = gradients["sigma"]
```
**Batched gradient estimation.**
The function returned by `make_functional_grad_estimator` is compatible
with `vmap`. If the estimator is bound to a specific fitness function,
that fitness function should also be compatible with `vmap`. If the
fitness function is not `vmap`-compatible, or if its behavior is
unexpected in the presence of `vmap`, then, consider instantiating
the gradient estimator without binding it to a fitness function.
As an alternative to `vmap`, a functional sampler has built-in support for
batched sampling (which actually still uses `vmap` internally).
Let us again consider our example estimator, `estimate_grads`.
In this example, the parameters `current_mu` and `current_sigma` would be
1-dimensional tensors in the non-batched case (because the distribution
`SymmetricSeparableGaussian` expects the parameters `mu` and `sigma` as
1-dimensional tensors, as can be observed from the class attribute
`SymmetricSeparableGaussian.PARAMETER_NDIMS`).
If `estimate_grads` is given `mu` and/or `sigma` with more than 1
dimensions, those extra leftmost dimensions will be considered as batch
dimensions, and therefore the resulting sample tensor will have extra
leftmost dimensions too.
**Declaring fixed parameters.**
A functional gradient estimator can be created in such a way that some of
the parameters are pre-defined and only a subset of the mandatory parameters
are expected via arguments. For example, a gradient estimator that samples
from `SymmetricSeparableGaussian` with a fixed sigma can be defined like
this:
```python
predefined_sigma = ... # The constant sigma goes here
fgrad2 = make_functional_sampler(
SymmetricSeparableGaussian,
function=f,
objective_sense="max",
required_parameters=["mu"],
fixed_parameters={"sigma": predefined_sigma},
ranking_method="centered",
)
```
The function `fgrad2` can be called like this:
```python
gradients = fgrad2(num_samples, current_mu)
# or, alternatively:
# gradients = fgrad2(num_samples, mu=current_mu)
```
**Specifying `objective_sense` and/or `ranking_method` later.**
One can omit `objective_sense` and `ranking_method` while making the
functional gradient estimator, and specify them later at the moment of
estimation. For example:
```python
fgrad3 = make_functional_sampler(
SymmetricSeparableGaussian,
function=f,
required_parameters=["mu", "sigma"],
# Notice: `objective_sense` and `ranking_method` are omitted
)
...
mu = ...
sigma = ...
gradients = fgrad3(
num_samples,
mu=mu,
sigma=sigma,
objective_sense="max",
ranking_method="centered",
)
```
Args:
function: The fitness function that will be called for estimating
the gradients. If provided, the first positional argument of the
returned gradient estimator will be the number of solutions.
If omitted, the first and second positional arguments of the
returned gradient estimator will be the samples (solutions)
and fitnesses.
Please note that this `function` is expected to receive a
2-dimensional tensor (representing the population, where each
row of the 2-dimensional tensor is a solution) and is expected
to return a 1-dimensional tensor (where each element is a
scalar fitness).
For batching and/or `vmap` to work, this `function` itself should
be `vmap`-compatible.
objective_sense: Specify this as "max" if a higher fitness value means
better solution. Specify this as "min" if a higher fitness value
means worse solution. Please note that, if `objective_sense` is not
provided at the moment of its making, one will have to specify it
later every time the estimator is called.
required_parameters: A list of strings, each string being the name
of a distribution parameter. The order of this list determines
the order of parameter-related positional arguments in the
returned callable object.
fixed_parameters: A dictionary where the keys are parameter names
(as strings), and the values are pre-defined parameter values.
ranking_method: Give a string here if you would like the fitnesses
to be ranked first. Possible values are "centered", "linear",
"raw".
return_samples: Set this as True if you would like the gradient
estimator to return not just the gradients, but also the samples
(solutions) that were used for estimating the gradients.
return_fitnesses: Set this as True if you would like the gradient
estimator to return not just the gradients, but also the fitnesses
that were used for estimating the gradients.
Returns:
A callable object whose function is to estimate gradients.
"""
return FunctionalGradEstimator(
distribution_class,
function=function,
objective_sense=objective_sense,
required_parameters=required_parameters,
fixed_parameters=fixed_parameters,
ranking_method=ranking_method,
return_samples=return_samples,
return_fitnesses=return_fitnesses,
)
make_functional_sampler(distribution_class, *, required_parameters, fixed_parameters=None)
¶
Make a stateless function that samples from a distribution.
This function is meant to be used when one wants to follow the functional programming paradigm.
As an example, let us imagine that we are interested in sampling from the
distribution SymmetricSeparableGaussian. A sampler function out of this
distribution can be created like this:
from evotorch.distributions import SymmetricSeparableGaussian, make_functional_sampler
get_samples = make_functional_sampler(
SymmetricSeparableGaussian,
required_parameters=["mu", "sigma"],
)
Now we have a function get_samples(), which can be used like this:
number_of_samples = ... # an integer representing the desired number of samples
mu = ... # a one-dimensional tensor
sigma = ... # a one-dimensional tensor
my_samples = get_samples(number_of_samples, mu, sigma)
Alternatively, the parameters of the distribution can be specified via keyword arguments, like this:
Batched sampling.
A functional sampler can be further transformed via torch.func.vmap(...)
for creating batched samples.
As an alternative to vmap, a functional sampler has built-in support for
batched sampling (which actually still uses vmap internally).
Let us again consider our example sampler get_samples.
In this example, the parameters mu and sigma would be 1-dimensional
tensors in the non-batched case (because the distribution
SymmetricSeparableGaussian expects the parameters mu and sigma as
1-dimensional tensors, as can be observed from the class attribute
SymmetricSeparableGaussian.PARAMETER_NDIMS).
If get_samples is given mu and/or sigma with more than 1 dimensions,
those extra leftmost dimensions will be considered as batch dimensions,
and therefore the resulting sample tensor will have extra leftmost
dimensions too.
Declaring fixed parameters.
A functional sampler can be created in such a way that some of the
parameters are pre-defined and only a subset of the mandatory parameters
are expected via arguments. For example, a sampler that samples from
SymmetricSeparableGaussian with a fixed sigma can be defined like this:
predefined_sigma = ... # The constant sigma goes here
get_samples2 = make_functional_sampler(
SymmetricSeparableGaussian,
required_parameters=["mu"],
fixed_parameters={"sigma": predefined_sigma},
)
The function get_samples2 can be called like this:
number_of_samples = ... # an integer, representing the desired number of samples
mu = ... # a one-dimensional tensor
# or like this:
# my_samples2 = get_samples2(number_of_samples, mu=mu)
How the functional sampler uses its wrapped distribution.
If the wrapped distribution class has a static method with the signature
functional_sample(num_solutions: int, parameters: dict) -> torch.Tensor
(which expects num_solutions as the number of solutions/samples
and parameters as the parameter dictionary), this functional sampler
will use that static method to obtain and return the samples.
On the other hand, if the wrapped distribution class declares its class
attribute functional_sample = NotImplemented, then, the wrapped
distribution class will be temporarily instantiated, and then, the
sample() method of this instance will be used to generate and return
the samples.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
distribution_class |
A class that inherits from the base class Distribution. |
required | |
required_parameters |
Iterable |
A list of strings, each string being the name of a distribution parameter. The order of this list determines the order of parameter-related positional arguments in the returned callable object. |
required |
fixed_parameters |
Optional[dict] |
A dictionary where the keys are parameter names (as strings), and the values are pre-defined parameter values. |
None |
Returns:
| Type | Description |
|---|---|
Callable |
A callable object whose function is to return samples from the specified distribution. |
Source code in evotorch/distributions.py
def make_functional_sampler(
distribution_class, *, required_parameters: Iterable, fixed_parameters: Optional[dict] = None
) -> Callable:
"""
Make a stateless function that samples from a distribution.
This function is meant to be used when one wants to follow the functional
programming paradigm.
As an example, let us imagine that we are interested in sampling from the
distribution `SymmetricSeparableGaussian`. A sampler function out of this
distribution can be created like this:
```python
from evotorch.distributions import SymmetricSeparableGaussian, make_functional_sampler
get_samples = make_functional_sampler(
SymmetricSeparableGaussian,
required_parameters=["mu", "sigma"],
)
```
Now we have a function `get_samples()`, which can be used like this:
```python
number_of_samples = ... # an integer representing the desired number of samples
mu = ... # a one-dimensional tensor
sigma = ... # a one-dimensional tensor
my_samples = get_samples(number_of_samples, mu, sigma)
```
Alternatively, the parameters of the distribution can be specified via
keyword arguments, like this:
```python
my_samples = get_samples(number_of_samples, mu=mu, sigma=sigma)
```
**Batched sampling.**
A functional sampler can be further transformed via `torch.func.vmap(...)`
for creating batched samples.
As an alternative to `vmap`, a functional sampler has built-in support for
batched sampling (which actually still uses `vmap` internally).
Let us again consider our example sampler `get_samples`.
In this example, the parameters `mu` and `sigma` would be 1-dimensional
tensors in the non-batched case (because the distribution
`SymmetricSeparableGaussian` expects the parameters `mu` and `sigma` as
1-dimensional tensors, as can be observed from the class attribute
`SymmetricSeparableGaussian.PARAMETER_NDIMS`).
If `get_samples` is given `mu` and/or `sigma` with more than 1 dimensions,
those extra leftmost dimensions will be considered as batch dimensions,
and therefore the resulting sample tensor will have extra leftmost
dimensions too.
**Declaring fixed parameters.**
A functional sampler can be created in such a way that some of the
parameters are pre-defined and only a subset of the mandatory parameters
are expected via arguments. For example, a sampler that samples from
`SymmetricSeparableGaussian` with a fixed sigma can be defined like this:
```python
predefined_sigma = ... # The constant sigma goes here
get_samples2 = make_functional_sampler(
SymmetricSeparableGaussian,
required_parameters=["mu"],
fixed_parameters={"sigma": predefined_sigma},
)
```
The function `get_samples2` can be called like this:
```python
number_of_samples = ... # an integer, representing the desired number of samples
mu = ... # a one-dimensional tensor
# or like this:
# my_samples2 = get_samples2(number_of_samples, mu=mu)
```
**How the functional sampler uses its wrapped distribution.**
If the wrapped distribution class has a static method with the signature
`functional_sample(num_solutions: int, parameters: dict) -> torch.Tensor`
(which expects `num_solutions` as the number of solutions/samples
and `parameters` as the parameter dictionary), this functional sampler
will use that static method to obtain and return the samples.
On the other hand, if the wrapped distribution class declares its class
attribute `functional_sample = NotImplemented`, then, the wrapped
distribution class will be temporarily instantiated, and then, the
`sample()` method of this instance will be used to generate and return
the samples.
Args:
distribution_class: A class that inherits from the base class
[Distribution][evotorch.distributions.Distribution].
required_parameters: A list of strings, each string being the name
of a distribution parameter. The order of this list determines
the order of parameter-related positional arguments in the
returned callable object.
fixed_parameters: A dictionary where the keys are parameter names
(as strings), and the values are pre-defined parameter values.
Returns:
A callable object whose function is to return samples from the
specified distribution.
"""
return FunctionalSampler(
distribution_class, required_parameters=required_parameters, fixed_parameters=fixed_parameters
)
logging
¶
This module contains logging utilities.
Logger
¶
Base class for all logging classes.
Source code in evotorch/logging.py
class Logger:
"""Base class for all logging classes."""
def __init__(self, searcher: SearchAlgorithm, *, interval: int = 1, after_first_step: bool = False):
"""`__init__(...)`: Initialize the Logger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and so on.
On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31, and so
on.
"""
searcher.log_hook.append(self)
self._interval = int(interval)
self._after_first_step = bool(after_first_step)
self._steps_count = 0
def __call__(self, status: dict):
if self._after_first_step:
n = self._steps_count
self._steps_count += 1
else:
self._steps_count += 1
n = self._steps_count
if (n % self._interval) == 0:
self._log(self._filter(status))
def _filter(self, status: dict) -> dict:
return status
def _log(self, status: dict):
raise NotImplementedError
__init__(self, searcher, *, interval=1, after_first_step=False)
special
¶
__init__(...): Initialize the Logger.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
searcher |
SearchAlgorithm |
The evolutionary algorithm instance whose progress is to be logged. |
required |
interval |
int |
Expected as an integer n. Logging is to be done at every n iterations. |
1 |
after_first_step |
bool |
Expected as a boolean. Meaningful only if interval is set as an integer greater than 1. Let us suppose that interval is set as 10. If after_first_step is False (which is the default), then the logging will be done at steps 10, 20, 30, and so on. On the other hand, if after_first_step is True, then the logging will be done at steps 1, 11, 21, 31, and so on. |
False |
Source code in evotorch/logging.py
def __init__(self, searcher: SearchAlgorithm, *, interval: int = 1, after_first_step: bool = False):
"""`__init__(...)`: Initialize the Logger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and so on.
On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31, and so
on.
"""
searcher.log_hook.append(self)
self._interval = int(interval)
self._after_first_step = bool(after_first_step)
self._steps_count = 0
MlflowLogger (ScalarLogger)
¶
A logger which stores the status via Mlflow.
Source code in evotorch/logging.py
class MlflowLogger(ScalarLogger):
"""A logger which stores the status via Mlflow."""
def __init__(
self,
searcher: SearchAlgorithm,
client: Optional[mlflow.tracking.MlflowClient] = None,
run: Union[mlflow.entities.Run, Optional[MlflowID]] = None,
*,
interval: int = 1,
after_first_step: bool = False,
):
"""`__init__(...)`: Initialize the MlflowLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
client: The MlflowClient object whose log_metric() method
will be used for logging. This can be passed as None,
in which case mlflow.log_metrics() will be used instead.
Please note that, if a client is provided, the `run`
argument is required as well.
run: Expected only if a client is provided.
This is the mlflow Run object (an instance of
mlflow.entities.Run), or the ID of the mlflow run.
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and
so on. On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31,
and so on.
"""
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
self._client = client
self._run_id: Optional[MlflowID] = None
if self._client is None:
if run is not None:
raise ValueError("Received `run`, but `client` is missing")
else:
if run is None:
raise ValueError("Received `client`, but `run` is missing")
if isinstance(run, mlflow.entities.Run):
self._run_id = run.info.run_id
else:
self._run_id = run
def _log(self, status: dict):
if self._client is None:
mlflow.log_metrics(status, step=self._steps_count)
else:
for k, v in status.items():
self._client.log_metric(self._run_id, k, v, step=self._steps_count)
__init__(self, searcher, client=None, run=None, *, interval=1, after_first_step=False)
special
¶
__init__(...): Initialize the MlflowLogger.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
searcher |
SearchAlgorithm |
The evolutionary algorithm instance whose progress is to be logged. |
required |
client |
Optional[mlflow.tracking.client.MlflowClient] |
The MlflowClient object whose log_metric() method
will be used for logging. This can be passed as None,
in which case mlflow.log_metrics() will be used instead.
Please note that, if a client is provided, the |
None |
run |
Union[mlflow.entities.run.Run, str, bytes, int] |
Expected only if a client is provided. This is the mlflow Run object (an instance of mlflow.entities.Run), or the ID of the mlflow run. |
None |
interval |
int |
Expected as an integer n. Logging is to be done at every n iterations. |
1 |
after_first_step |
bool |
Expected as a boolean. Meaningful only if interval is set as an integer greater than 1. Let us suppose that interval is set as 10. If after_first_step is False (which is the default), then the logging will be done at steps 10, 20, 30, and so on. On the other hand, if after_first_step is True, then the logging will be done at steps 1, 11, 21, 31, and so on. |
False |
Source code in evotorch/logging.py
def __init__(
self,
searcher: SearchAlgorithm,
client: Optional[mlflow.tracking.MlflowClient] = None,
run: Union[mlflow.entities.Run, Optional[MlflowID]] = None,
*,
interval: int = 1,
after_first_step: bool = False,
):
"""`__init__(...)`: Initialize the MlflowLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
client: The MlflowClient object whose log_metric() method
will be used for logging. This can be passed as None,
in which case mlflow.log_metrics() will be used instead.
Please note that, if a client is provided, the `run`
argument is required as well.
run: Expected only if a client is provided.
This is the mlflow Run object (an instance of
mlflow.entities.Run), or the ID of the mlflow run.
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and
so on. On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31,
and so on.
"""
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
self._client = client
self._run_id: Optional[MlflowID] = None
if self._client is None:
if run is not None:
raise ValueError("Received `run`, but `client` is missing")
else:
if run is None:
raise ValueError("Received `client`, but `run` is missing")
if isinstance(run, mlflow.entities.Run):
self._run_id = run.info.run_id
else:
self._run_id = run
NeptuneLogger (ScalarLogger)
¶
A logger which stores the status via neptune.
Source code in evotorch/logging.py
class NeptuneLogger(ScalarLogger):
"""A logger which stores the status via neptune."""
def __init__(
self,
searcher: SearchAlgorithm,
run: Optional[neptune.Run] = None,
*,
interval: int = 1,
after_first_step: bool = False,
group: Optional[str] = None,
**neptune_kwargs,
):
"""`__init__(...)`: Initialize the NeptuneLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
run: A `neptune.new.run.Run` instance using which the status
will be logged. If None, then a new run will be created.
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and so on.
On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31, and so
on.
group: Into which group will the metrics be stored.
For example, if the status keys to be logged are "score" and
"elapsed", and `group` is set as "training", then the metrics
will be sent to neptune with the keys "training/score" and
"training/elapsed". `group` can also be left as None,
in which case the status will be sent to neptune with the
key names unchanged.
**neptune_kwargs: Any additional keyword arguments to be passed
to `neptune.init_run()` when creating a new run.
For example, `project="my-project"` or `tags=["my-tag"]`.
"""
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
self._group = group
if run is None:
self._run = neptune.init_run(**neptune_kwargs)
else:
self._run = run
@property
def run(self) -> neptune.Run:
return self._run
def _log(self, status: dict):
for k, v in status.items():
target_key = k if self._group is None else self._group + "/" + k
self._run[target_key].log(v)
__init__(self, searcher, run=None, *, interval=1, after_first_step=False, group=None, **neptune_kwargs)
special
¶
__init__(...): Initialize the NeptuneLogger.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
searcher |
SearchAlgorithm |
The evolutionary algorithm instance whose progress is to be logged. |
required |
run |
Optional[neptune.metadata_containers.run.Run] |
A |
None |
interval |
int |
Expected as an integer n. Logging is to be done at every n iterations. |
1 |
after_first_step |
bool |
Expected as a boolean. Meaningful only if interval is set as an integer greater than 1. Let us suppose that interval is set as 10. If after_first_step is False (which is the default), then the logging will be done at steps 10, 20, 30, and so on. On the other hand, if after_first_step is True, then the logging will be done at steps 1, 11, 21, 31, and so on. |
False |
group |
Optional[str] |
Into which group will the metrics be stored.
For example, if the status keys to be logged are "score" and
"elapsed", and |
None |
**neptune_kwargs |
Any additional keyword arguments to be passed
to |
{} |
Source code in evotorch/logging.py
def __init__(
self,
searcher: SearchAlgorithm,
run: Optional[neptune.Run] = None,
*,
interval: int = 1,
after_first_step: bool = False,
group: Optional[str] = None,
**neptune_kwargs,
):
"""`__init__(...)`: Initialize the NeptuneLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
run: A `neptune.new.run.Run` instance using which the status
will be logged. If None, then a new run will be created.
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and so on.
On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31, and so
on.
group: Into which group will the metrics be stored.
For example, if the status keys to be logged are "score" and
"elapsed", and `group` is set as "training", then the metrics
will be sent to neptune with the keys "training/score" and
"training/elapsed". `group` can also be left as None,
in which case the status will be sent to neptune with the
key names unchanged.
**neptune_kwargs: Any additional keyword arguments to be passed
to `neptune.init_run()` when creating a new run.
For example, `project="my-project"` or `tags=["my-tag"]`.
"""
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
self._group = group
if run is None:
self._run = neptune.init_run(**neptune_kwargs)
else:
self._run = run
PandasLogger (ScalarLogger)
¶
A logger which collects status information and generates a pandas.DataFrame at the end.
Source code in evotorch/logging.py
class PandasLogger(ScalarLogger):
"""A logger which collects status information and
generates a pandas.DataFrame at the end.
"""
def __init__(self, searcher: SearchAlgorithm, *, interval: int = 1, after_first_step: bool = False):
"""`__init__(...)`: Initialize the PandasLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and
so on. On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31, and
so on.
"""
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
self._data = []
def _log(self, status: dict):
self._data.append(deepcopy(status))
def to_dataframe(self, *, index: Optional[str] = "iter") -> pandas.DataFrame:
"""Generate a pandas.DataFrame from the collected
status information.
Args:
index: The column to be set as the index.
If passed as None, then no index will be set.
The default is "iter".
"""
result = pandas.DataFrame(self._data)
if index is not None:
result.set_index(index, inplace=True)
return result
__init__(self, searcher, *, interval=1, after_first_step=False)
special
¶
__init__(...): Initialize the PandasLogger.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
searcher |
SearchAlgorithm |
The evolutionary algorithm instance whose progress is to be logged. |
required |
interval |
int |
Expected as an integer n. Logging is to be done at every n iterations. |
1 |
after_first_step |
bool |
Expected as a boolean. Meaningful only if interval is set as an integer greater than 1. Let us suppose that interval is set as 10. If after_first_step is False (which is the default), then the logging will be done at steps 10, 20, 30, and so on. On the other hand, if after_first_step is True, then the logging will be done at steps 1, 11, 21, 31, and so on. |
False |
Source code in evotorch/logging.py
def __init__(self, searcher: SearchAlgorithm, *, interval: int = 1, after_first_step: bool = False):
"""`__init__(...)`: Initialize the PandasLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and
so on. On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31, and
so on.
"""
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
self._data = []
to_dataframe(self, *, index='iter')
¶
Generate a pandas.DataFrame from the collected status information.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
index |
Optional[str] |
The column to be set as the index. If passed as None, then no index will be set. The default is "iter". |
'iter' |
Source code in evotorch/logging.py
def to_dataframe(self, *, index: Optional[str] = "iter") -> pandas.DataFrame:
"""Generate a pandas.DataFrame from the collected
status information.
Args:
index: The column to be set as the index.
If passed as None, then no index will be set.
The default is "iter".
"""
result = pandas.DataFrame(self._data)
if index is not None:
result.set_index(index, inplace=True)
return result
PicklingLogger (Logger)
¶
A logger which periodically pickles the current result of the search.
The pickled data includes the current center solution and the best solution (if available). If the problem being solved is a reinforcement learning task, then the pickled data also includes the observation normalization data and the policy.
Source code in evotorch/logging.py
class PicklingLogger(Logger):
"""
A logger which periodically pickles the current result of the search.
The pickled data includes the current center solution and the best solution
(if available). If the problem being solved is a reinforcement learning
task, then the pickled data also includes the observation normalization
data and the policy.
"""
def __init__(
self,
searcher: SearchAlgorithm,
*,
interval: int,
directory: Optional[Union[str, pathlib.Path]] = None,
prefix: Optional[str] = None,
zfill: int = 6,
items_to_save: Union[str, Iterable] = (
"center",
"best",
"pop_best",
"median_eval",
"mean_eval",
"pop_best_eval",
),
make_policy_from: Optional[str] = None,
after_first_step: bool = False,
verbose: bool = True,
):
"""
`__init__(...)`: Initialize the PicklingLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be pickled.
interval: Expected as an integer n.
Pickling is to be done at every n iterations.
directory: The directory into which the pickle files will be
placed. If given as None, the current directory will be
used.
prefix: The prefix to be used when naming the pickle file.
If left as None, then the prefix will contain the date,
time, name of the class of the problem object, and the PID
of the current process.
zfill: When naming the pickle file, the generation number will
also be used. This `zfill` argument is used to determine
the length of the number of generations string.
For example, if the current number of generations is
150, and zfill is 6 (which is the default), then the
number of generations part of the file name will look
like this: 'generation000150', the numeric part of the
string taking 6 characters.
items_to_save: Items to save from the status dictionary.
This can be a string, or a sequence of strings.
make_policy_from: If given as a string, then the solution
represented by this key in the status dictionary will be
taken, and converted a PyTorch module with the help of the
problem's `to_policy(...)` method, and then will be added
into the pickle file. For example, if this argument is
given as "center", then the center solution will be converted
to a policy and saved.
If this argument is left as None, then, this logger will
first try to obtain the center solution. If there is no
center solution, then the logger will try to obtain the
'pop_best' solution. If that one does not exist either, then
an error will be raised, and the user will be requested to
explicitly state a value for the argument `make_policy_from`.
If the problem object does not provide a `to_policy(...)`,
method, then this configuration will be ignored, and no
policy will be saved.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and so on.
On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31, and so
on.
verbose: If set as True, then a message will be printed out to
the standard output every time the pickling happens.
"""
# Call the initializer of the superclass.
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
# Each Logger register itself to the search algorithm's log hook. Additionally, in the case of PicklingLogger,
# we add this object's final saving method to the search algorithm's end-of-run hook.
# This is to make sure that the latest generation's result is saved, even when the last generation number
# does not coincide with the saving period.
searcher.end_of_run_hook.append(self._final_save)
# Store a weak reference to the search algorithm.
self._searcher_ref = weakref.ref(searcher)
# Store the item keys as a tuple of strings.
if isinstance(items_to_save, str):
self._items_to_save = (items_to_save,)
else:
self._items_to_save = tuple(items_to_save)
# Store the status key that will be used to get the current solution for making the policy.
self._make_policy_from = None if make_policy_from is None else str(make_policy_from)
if prefix is None:
# If a file name prefix is not given by the user, then we prepare one using the current date and time,
# name of the problem type, and the PID of the current process.
strnow = datetime.now().strftime("%Y-%m-%d-%H.%M.%S")
probname = type(searcher.problem).__name__
strpid = str(os.getpid())
self._prefix = f"{probname}_{strnow}_{strpid}"
else:
# If there is a file name prefix given by the user, then we use that name.
self._prefix = str(prefix)
if directory is None:
# If a directory name is not given by the user, then we store the directory name as None.
self._directory = None
else:
# If a directory name is given by the user, then we store it as a string, and make sure that it exists.
self._directory = str(directory)
os.makedirs(self._directory, exist_ok=True)
self._verbose = bool(verbose)
self._zfill = int(zfill)
self._last_generation: Optional[int] = None
self._last_file_name: Optional[str] = None
def _within_dir(self, fname: Union[str, pathlib.Path]) -> str:
fname = str(fname)
if self._directory is not None:
fname = os.path.join(self._directory, fname)
return fname
@classmethod
def _as_cpu_tensor(cls, x: Any) -> Any:
if isinstance(x, Solution):
x = cls._as_cpu_tensor(x.values)
elif isinstance(x, torch.Tensor):
with torch.no_grad():
x = x.cpu().clone()
if isinstance(x, ReadOnlyTensor):
x = x.as_subclass(torch.Tensor)
return x
def _log(self, status: dict):
self.save()
def _final_save(self, status: dict):
# Get the stored search algorithm
searcher: Optional[SearchAlgorithm] = self._searcher_ref()
if searcher is not None:
# If the search algorithm object is still alive, then we check its generation number
current_gen = searcher.step_count
if (self._last_generation is None) or (current_gen > self._last_generation):
# If there was not a save at all, or the latest save was for a previous generation,
# then save the current status of the search.
self.save()
def save(self, fname: Optional[Union[str, pathlib.Path]] = None) -> str:
"""
Pickle the current status of the evolutionary search.
If this PicklingLogger was initialized with a `directory` argument,
then the pickle file will be put into that directory.
Args:
fname: The name of the pickle file to be created.
This can be left as None if the user wishes this PicklingLogger
to determine the name of the file.
Alternatively, an explicit name for this file can be specified
via this argument.
Returns:
The name of the pickle file that was generated, as a string.
"""
# Get the stored search algorithm
searcher: Optional[SearchAlgorithm] = self._searcher_ref()
if searcher is not None:
# If the search algorithm object is still alive, then take the problem object from it.
problem: Problem = searcher.problem
# Initialize the data dictionary that will contain the objects to be put into the pickle file.
data = {}
for item_to_save in self._items_to_save:
# For each item key, get the object with that key from the algorithm's status dictionary, and then put
# that object into our data dictionary.
if item_to_save in searcher.status:
data[item_to_save] = self._as_cpu_tensor(searcher.status[item_to_save])
if (
hasattr(problem, "observation_normalization")
and hasattr(problem, "get_observation_stats")
and problem.observation_normalization
):
# If the problem object has observation normalization, then put the normalizer object to the data
# dictionary.
data["obs_stats"] = problem.get_observation_stats().to("cpu")
if hasattr(problem, "to_policy"):
# If the problem object has the method `to_policy(...)`, then we will generate a policy from the
# current solution, and put that policy into the data dictionary.
if self._make_policy_from is None:
# If the user did not specify the status key of the solution that will be converted to a policy,
# then, we first check if the search algorithm has a "center" item, and if not, we check if it
# has a "pop_best" item.
if "center" in searcher.status:
# The status key of the search algorithm has "center". We declare "center" as the key of the
# current solution.
policy_key = "center"
elif "pop_best" in searcher.status:
# The status key of the search algorithm has "pop_best". We declare "pop_best" as the key of
# the current solution.
policy_key = "pop_best"
else:
# The status key of the search algorithm contains neither a "center" solution and nor a
# "pop_best" solution. So, we raise an error.
raise ValueError(
"PicklingLogger did not receive an explicit value for its `make_policy_from` argument."
" The status dictionary of the search algorithm has neither 'center' nor 'pop_best'."
" Therefore, it is not clear which status item is to be used for making a policy."
" Please try instantiating a PicklingLogger with an explicit `make_policy_from` value."
)
else:
# This is the case where the user explicitly specified (via a status key) which solution will be
# used for making the current policy. We declare that key as the key to use.
policy_key = self._make_policy_from
# Get the solution.
policy_solution = searcher.status[policy_key]
# Make a policy from the solution
policy = problem.to_policy(policy_solution)
if isinstance(policy, nn.Module) and (device_of_module(policy) != torch.device("cpu")):
# If the created policy is a PyTorch module, and this module is not on the cpu, then we make
# a copy of this module on the cpu.
policy = clone(policy).to("cpu")
# We put this policy into our data dictionary.
data["policy"] = policy
# We put the datetime-related information
first_step_datetime = searcher.first_step_datetime
if first_step_datetime is not None:
now = datetime.now()
elapsed = now - first_step_datetime
data["beginning_time"] = first_step_datetime
data["now"] = now
data["elapsed"] = elapsed
if fname is None:
# If a file name was not given, then we generate one.
num_gens = str(searcher.step_count).zfill(self._zfill)
fname = f"{self._prefix}_generation{num_gens}.pickle"
# We prepend the specified directory name, if given.
fname = self._within_dir(fname)
# Here, the pickle file is created and the data is saved.
with open(fname, "wb") as f:
pickle.dump(data, f)
# Store the most recently saved generation number
self._last_generation = searcher.step_count
# Store the name of the last pickle file
self._last_file_name = fname
# Report to the user that the save was successful
if self._verbose:
print("Saved to", fname)
# Return the file name
return fname
else:
return None
@property
def last_generation(self) -> Optional[int]:
"""
Get the last generation for which a pickle file was created.
If no pickle file is created yet, the result will be None.
"""
return self._last_generation
@property
def last_file_name(self) -> Optional[str]:
"""
Get the name of the last pickle file.
If no pickle file is created yet, the result will be None.
"""
return self._last_file_name
def unpickle_last_file(self) -> dict:
"""
Unpickle the most recently made pickle file.
The file itself will not be modified.
Its contents will be returned.
"""
with open(self._last_file_name, "rb") as f:
return pickle.load(f)
last_file_name: Optional[str]
property
readonly
¶
Get the name of the last pickle file. If no pickle file is created yet, the result will be None.
last_generation: Optional[int]
property
readonly
¶
Get the last generation for which a pickle file was created. If no pickle file is created yet, the result will be None.
__init__(self, searcher, *, interval, directory=None, prefix=None, zfill=6, items_to_save=('center', 'best', 'pop_best', 'median_eval', 'mean_eval', 'pop_best_eval'), make_policy_from=None, after_first_step=False, verbose=True)
special
¶
__init__(...): Initialize the PicklingLogger.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
searcher |
SearchAlgorithm |
The evolutionary algorithm instance whose progress is to be pickled. |
required |
interval |
int |
Expected as an integer n. Pickling is to be done at every n iterations. |
required |
directory |
Union[str, pathlib.Path] |
The directory into which the pickle files will be placed. If given as None, the current directory will be used. |
None |
prefix |
Optional[str] |
The prefix to be used when naming the pickle file. If left as None, then the prefix will contain the date, time, name of the class of the problem object, and the PID of the current process. |
None |
zfill |
int |
When naming the pickle file, the generation number will
also be used. This |
6 |
items_to_save |
Union[str, Iterable] |
Items to save from the status dictionary. This can be a string, or a sequence of strings. |
('center', 'best', 'pop_best', 'median_eval', 'mean_eval', 'pop_best_eval') |
make_policy_from |
Optional[str] |
If given as a string, then the solution
represented by this key in the status dictionary will be
taken, and converted a PyTorch module with the help of the
problem's |
None |
after_first_step |
bool |
Expected as a boolean. Meaningful only if interval is set as an integer greater than 1. Let us suppose that interval is set as 10. If after_first_step is False (which is the default), then the logging will be done at steps 10, 20, 30, and so on. On the other hand, if after_first_step is True, then the logging will be done at steps 1, 11, 21, 31, and so on. |
False |
verbose |
bool |
If set as True, then a message will be printed out to the standard output every time the pickling happens. |
True |
Source code in evotorch/logging.py
def __init__(
self,
searcher: SearchAlgorithm,
*,
interval: int,
directory: Optional[Union[str, pathlib.Path]] = None,
prefix: Optional[str] = None,
zfill: int = 6,
items_to_save: Union[str, Iterable] = (
"center",
"best",
"pop_best",
"median_eval",
"mean_eval",
"pop_best_eval",
),
make_policy_from: Optional[str] = None,
after_first_step: bool = False,
verbose: bool = True,
):
"""
`__init__(...)`: Initialize the PicklingLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be pickled.
interval: Expected as an integer n.
Pickling is to be done at every n iterations.
directory: The directory into which the pickle files will be
placed. If given as None, the current directory will be
used.
prefix: The prefix to be used when naming the pickle file.
If left as None, then the prefix will contain the date,
time, name of the class of the problem object, and the PID
of the current process.
zfill: When naming the pickle file, the generation number will
also be used. This `zfill` argument is used to determine
the length of the number of generations string.
For example, if the current number of generations is
150, and zfill is 6 (which is the default), then the
number of generations part of the file name will look
like this: 'generation000150', the numeric part of the
string taking 6 characters.
items_to_save: Items to save from the status dictionary.
This can be a string, or a sequence of strings.
make_policy_from: If given as a string, then the solution
represented by this key in the status dictionary will be
taken, and converted a PyTorch module with the help of the
problem's `to_policy(...)` method, and then will be added
into the pickle file. For example, if this argument is
given as "center", then the center solution will be converted
to a policy and saved.
If this argument is left as None, then, this logger will
first try to obtain the center solution. If there is no
center solution, then the logger will try to obtain the
'pop_best' solution. If that one does not exist either, then
an error will be raised, and the user will be requested to
explicitly state a value for the argument `make_policy_from`.
If the problem object does not provide a `to_policy(...)`,
method, then this configuration will be ignored, and no
policy will be saved.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and so on.
On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31, and so
on.
verbose: If set as True, then a message will be printed out to
the standard output every time the pickling happens.
"""
# Call the initializer of the superclass.
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
# Each Logger register itself to the search algorithm's log hook. Additionally, in the case of PicklingLogger,
# we add this object's final saving method to the search algorithm's end-of-run hook.
# This is to make sure that the latest generation's result is saved, even when the last generation number
# does not coincide with the saving period.
searcher.end_of_run_hook.append(self._final_save)
# Store a weak reference to the search algorithm.
self._searcher_ref = weakref.ref(searcher)
# Store the item keys as a tuple of strings.
if isinstance(items_to_save, str):
self._items_to_save = (items_to_save,)
else:
self._items_to_save = tuple(items_to_save)
# Store the status key that will be used to get the current solution for making the policy.
self._make_policy_from = None if make_policy_from is None else str(make_policy_from)
if prefix is None:
# If a file name prefix is not given by the user, then we prepare one using the current date and time,
# name of the problem type, and the PID of the current process.
strnow = datetime.now().strftime("%Y-%m-%d-%H.%M.%S")
probname = type(searcher.problem).__name__
strpid = str(os.getpid())
self._prefix = f"{probname}_{strnow}_{strpid}"
else:
# If there is a file name prefix given by the user, then we use that name.
self._prefix = str(prefix)
if directory is None:
# If a directory name is not given by the user, then we store the directory name as None.
self._directory = None
else:
# If a directory name is given by the user, then we store it as a string, and make sure that it exists.
self._directory = str(directory)
os.makedirs(self._directory, exist_ok=True)
self._verbose = bool(verbose)
self._zfill = int(zfill)
self._last_generation: Optional[int] = None
self._last_file_name: Optional[str] = None
save(self, fname=None)
¶
Pickle the current status of the evolutionary search.
If this PicklingLogger was initialized with a directory argument,
then the pickle file will be put into that directory.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fname |
Union[str, pathlib.Path] |
The name of the pickle file to be created. This can be left as None if the user wishes this PicklingLogger to determine the name of the file. Alternatively, an explicit name for this file can be specified via this argument. |
None |
Returns:
| Type | Description |
|---|---|
str |
The name of the pickle file that was generated, as a string. |
Source code in evotorch/logging.py
def save(self, fname: Optional[Union[str, pathlib.Path]] = None) -> str:
"""
Pickle the current status of the evolutionary search.
If this PicklingLogger was initialized with a `directory` argument,
then the pickle file will be put into that directory.
Args:
fname: The name of the pickle file to be created.
This can be left as None if the user wishes this PicklingLogger
to determine the name of the file.
Alternatively, an explicit name for this file can be specified
via this argument.
Returns:
The name of the pickle file that was generated, as a string.
"""
# Get the stored search algorithm
searcher: Optional[SearchAlgorithm] = self._searcher_ref()
if searcher is not None:
# If the search algorithm object is still alive, then take the problem object from it.
problem: Problem = searcher.problem
# Initialize the data dictionary that will contain the objects to be put into the pickle file.
data = {}
for item_to_save in self._items_to_save:
# For each item key, get the object with that key from the algorithm's status dictionary, and then put
# that object into our data dictionary.
if item_to_save in searcher.status:
data[item_to_save] = self._as_cpu_tensor(searcher.status[item_to_save])
if (
hasattr(problem, "observation_normalization")
and hasattr(problem, "get_observation_stats")
and problem.observation_normalization
):
# If the problem object has observation normalization, then put the normalizer object to the data
# dictionary.
data["obs_stats"] = problem.get_observation_stats().to("cpu")
if hasattr(problem, "to_policy"):
# If the problem object has the method `to_policy(...)`, then we will generate a policy from the
# current solution, and put that policy into the data dictionary.
if self._make_policy_from is None:
# If the user did not specify the status key of the solution that will be converted to a policy,
# then, we first check if the search algorithm has a "center" item, and if not, we check if it
# has a "pop_best" item.
if "center" in searcher.status:
# The status key of the search algorithm has "center". We declare "center" as the key of the
# current solution.
policy_key = "center"
elif "pop_best" in searcher.status:
# The status key of the search algorithm has "pop_best". We declare "pop_best" as the key of
# the current solution.
policy_key = "pop_best"
else:
# The status key of the search algorithm contains neither a "center" solution and nor a
# "pop_best" solution. So, we raise an error.
raise ValueError(
"PicklingLogger did not receive an explicit value for its `make_policy_from` argument."
" The status dictionary of the search algorithm has neither 'center' nor 'pop_best'."
" Therefore, it is not clear which status item is to be used for making a policy."
" Please try instantiating a PicklingLogger with an explicit `make_policy_from` value."
)
else:
# This is the case where the user explicitly specified (via a status key) which solution will be
# used for making the current policy. We declare that key as the key to use.
policy_key = self._make_policy_from
# Get the solution.
policy_solution = searcher.status[policy_key]
# Make a policy from the solution
policy = problem.to_policy(policy_solution)
if isinstance(policy, nn.Module) and (device_of_module(policy) != torch.device("cpu")):
# If the created policy is a PyTorch module, and this module is not on the cpu, then we make
# a copy of this module on the cpu.
policy = clone(policy).to("cpu")
# We put this policy into our data dictionary.
data["policy"] = policy
# We put the datetime-related information
first_step_datetime = searcher.first_step_datetime
if first_step_datetime is not None:
now = datetime.now()
elapsed = now - first_step_datetime
data["beginning_time"] = first_step_datetime
data["now"] = now
data["elapsed"] = elapsed
if fname is None:
# If a file name was not given, then we generate one.
num_gens = str(searcher.step_count).zfill(self._zfill)
fname = f"{self._prefix}_generation{num_gens}.pickle"
# We prepend the specified directory name, if given.
fname = self._within_dir(fname)
# Here, the pickle file is created and the data is saved.
with open(fname, "wb") as f:
pickle.dump(data, f)
# Store the most recently saved generation number
self._last_generation = searcher.step_count
# Store the name of the last pickle file
self._last_file_name = fname
# Report to the user that the save was successful
if self._verbose:
print("Saved to", fname)
# Return the file name
return fname
else:
return None
unpickle_last_file(self)
¶
Unpickle the most recently made pickle file. The file itself will not be modified. Its contents will be returned.
SacredLogger (ScalarLogger)
¶
A logger which stores the status via the Run object of sacred.
Source code in evotorch/logging.py
class SacredLogger(ScalarLogger):
"""A logger which stores the status via the Run object of sacred."""
def __init__(
self,
searcher: SearchAlgorithm,
run: ExpOrRun,
result: Optional[str] = None,
*,
interval: int = 1,
after_first_step: bool = False,
):
"""`__init__(...)`: Initialize the SacredLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
run: An instance of `sacred.run.Run` or `sacred.Experiment`,
using which the progress will be logged.
result: The key in the status dictionary whose associated
value will be registered as the current result
of the experiment.
If left as None, no result will be registered.
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and
so on. On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31,
and so on.
"""
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
self._result = result
self._run = run
def _log(self, status: dict):
for k, v in status.items():
self._run.log_scalar(k, v)
if self._result is not None:
self._run.result = status[self._result]
__init__(self, searcher, run, result=None, *, interval=1, after_first_step=False)
special
¶
__init__(...): Initialize the SacredLogger.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
searcher |
SearchAlgorithm |
The evolutionary algorithm instance whose progress is to be logged. |
required |
run |
Union[sacred.experiment.Experiment, sacred.run.Run] |
An instance of |
required |
result |
Optional[str] |
The key in the status dictionary whose associated value will be registered as the current result of the experiment. If left as None, no result will be registered. |
None |
interval |
int |
Expected as an integer n. Logging is to be done at every n iterations. |
1 |
after_first_step |
bool |
Expected as a boolean. Meaningful only if interval is set as an integer greater than 1. Let us suppose that interval is set as 10. If after_first_step is False (which is the default), then the logging will be done at steps 10, 20, 30, and so on. On the other hand, if after_first_step is True, then the logging will be done at steps 1, 11, 21, 31, and so on. |
False |
Source code in evotorch/logging.py
def __init__(
self,
searcher: SearchAlgorithm,
run: ExpOrRun,
result: Optional[str] = None,
*,
interval: int = 1,
after_first_step: bool = False,
):
"""`__init__(...)`: Initialize the SacredLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
run: An instance of `sacred.run.Run` or `sacred.Experiment`,
using which the progress will be logged.
result: The key in the status dictionary whose associated
value will be registered as the current result
of the experiment.
If left as None, no result will be registered.
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and
so on. On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31,
and so on.
"""
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
self._result = result
self._run = run
StdOutLogger (ScalarLogger)
¶
A logger which prints the status into the screen.
Source code in evotorch/logging.py
class StdOutLogger(ScalarLogger):
"""A logger which prints the status into the screen."""
def __init__(
self,
searcher: SearchAlgorithm,
*,
interval: int = 1,
after_first_step: bool = False,
leading_keys: Iterable[str] = ("iter",),
):
"""`__init__(...)`: Initialize the StdOutLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and so on.
On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31, and so
on.
leading_keys: A sequence of strings where each string is a status
key. When printing the status, these keys will be shown first.
"""
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
self._leading_keys = list(leading_keys)
self._leading_keys_set = set(self._leading_keys)
def _log(self, status: dict):
max_key_length = max([len(str(k)) for k in status.keys()])
def report(k, v):
nonlocal max_key_length
print(str(k).rjust(max_key_length), ":", v)
for k in self._leading_keys:
if k in status:
v = status[k]
report(k, v)
for k, v in status.items():
if k not in self._leading_keys_set:
report(k, v)
print()
__init__(self, searcher, *, interval=1, after_first_step=False, leading_keys=('iter',))
special
¶
__init__(...): Initialize the StdOutLogger.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
searcher |
SearchAlgorithm |
The evolutionary algorithm instance whose progress is to be logged. |
required |
interval |
int |
Expected as an integer n. Logging is to be done at every n iterations. |
1 |
after_first_step |
bool |
Expected as a boolean. Meaningful only if interval is set as an integer greater than 1. Let us suppose that interval is set as 10. If after_first_step is False (which is the default), then the logging will be done at steps 10, 20, 30, and so on. On the other hand, if after_first_step is True, then the logging will be done at steps 1, 11, 21, 31, and so on. |
False |
leading_keys |
Iterable[str] |
A sequence of strings where each string is a status key. When printing the status, these keys will be shown first. |
('iter',) |
Source code in evotorch/logging.py
def __init__(
self,
searcher: SearchAlgorithm,
*,
interval: int = 1,
after_first_step: bool = False,
leading_keys: Iterable[str] = ("iter",),
):
"""`__init__(...)`: Initialize the StdOutLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and so on.
On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31, and so
on.
leading_keys: A sequence of strings where each string is a status
key. When printing the status, these keys will be shown first.
"""
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
self._leading_keys = list(leading_keys)
self._leading_keys_set = set(self._leading_keys)
WandbLogger (ScalarLogger)
¶
A logger which stores the status to Weights & Biases.
Source code in evotorch/logging.py
class WandbLogger(ScalarLogger):
"""A logger which stores the status to Weights & Biases."""
def __init__(
self,
searcher: SearchAlgorithm,
init: Optional[bool] = True,
*,
interval: int = 1,
after_first_step: bool = False,
group: Optional[str] = None,
**wandb_kwargs,
):
"""`__init__(...)`: Initialize the WandbLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
init: Run `wandb.init()` in the logger initialization
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and so on.
On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31, and so
on.
group: Into which group will the metrics be stored.
For example, if the status keys to be logged are "score" and
"elapsed", and `group` is set as "training", then the metrics
will be sent to W&B with the keys "training/score" and
"training/elapsed". `group` can also be left as None,
in which case the status will be sent to W&B with the
key names unchanged.
**wandb_kwargs: If `init` is `True` any additional keyword argument
will be passed to `wandb.init()`.
For example, WandbLogger(searcher, project=my-project, entity=my-organization)
will result in calling `wandb.init(project=my-project, entity=my-organization)`
"""
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
self._group = group
if init:
wandb.init(**wandb_kwargs)
def _log(self, status: dict):
log_status = dict()
for k, v in status.items():
target_key = k if self._group is None else self._group + "/" + k
log_status[target_key] = v
wandb.log(log_status)
__init__(self, searcher, init=True, *, interval=1, after_first_step=False, group=None, **wandb_kwargs)
special
¶
__init__(...): Initialize the WandbLogger.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
searcher |
SearchAlgorithm |
The evolutionary algorithm instance whose progress is to be logged. |
required |
init |
Optional[bool] |
Run |
True |
interval |
int |
Expected as an integer n. Logging is to be done at every n iterations. |
1 |
after_first_step |
bool |
Expected as a boolean. Meaningful only if interval is set as an integer greater than 1. Let us suppose that interval is set as 10. If after_first_step is False (which is the default), then the logging will be done at steps 10, 20, 30, and so on. On the other hand, if after_first_step is True, then the logging will be done at steps 1, 11, 21, 31, and so on. |
False |
group |
Optional[str] |
Into which group will the metrics be stored.
For example, if the status keys to be logged are "score" and
"elapsed", and |
None |
**wandb_kwargs |
If |
{} |
Source code in evotorch/logging.py
def __init__(
self,
searcher: SearchAlgorithm,
init: Optional[bool] = True,
*,
interval: int = 1,
after_first_step: bool = False,
group: Optional[str] = None,
**wandb_kwargs,
):
"""`__init__(...)`: Initialize the WandbLogger.
Args:
searcher: The evolutionary algorithm instance whose progress
is to be logged.
init: Run `wandb.init()` in the logger initialization
interval: Expected as an integer n.
Logging is to be done at every n iterations.
after_first_step: Expected as a boolean.
Meaningful only if interval is set as an integer greater
than 1. Let us suppose that interval is set as 10.
If after_first_step is False (which is the default),
then the logging will be done at steps 10, 20, 30, and so on.
On the other hand, if after_first_step is True,
then the logging will be done at steps 1, 11, 21, 31, and so
on.
group: Into which group will the metrics be stored.
For example, if the status keys to be logged are "score" and
"elapsed", and `group` is set as "training", then the metrics
will be sent to W&B with the keys "training/score" and
"training/elapsed". `group` can also be left as None,
in which case the status will be sent to W&B with the
key names unchanged.
**wandb_kwargs: If `init` is `True` any additional keyword argument
will be passed to `wandb.init()`.
For example, WandbLogger(searcher, project=my-project, entity=my-organization)
will result in calling `wandb.init(project=my-project, entity=my-organization)`
"""
super().__init__(searcher, interval=interval, after_first_step=after_first_step)
self._group = group
if init:
wandb.init(**wandb_kwargs)
neuroevolution
special
¶
Problem types for neuroevolution
baseneproblem
¶
gymne
¶
This namespace contains the GymNE class.
GymNE (NEProblem)
¶
Representation of a NEProblem where the goal is to maximize
the total reward obtained in a gym environment.
Source code in evotorch/neuroevolution/gymne.py
class GymNE(NEProblem):
"""
Representation of a NEProblem where the goal is to maximize
the total reward obtained in a `gym` environment.
"""
def __init__(
self,
env: Optional[Union[str, Callable]] = None,
network: Optional[Union[str, nn.Module, Callable[[], nn.Module]]] = None,
*,
env_name: Optional[Union[str, Callable]] = None,
network_args: Optional[dict] = None,
env_config: Optional[Mapping] = None,
observation_normalization: bool = False,
num_episodes: int = 1,
episode_length: Optional[int] = None,
decrease_rewards_by: Optional[float] = None,
alive_bonus_schedule: Optional[tuple] = None,
action_noise_stdev: Optional[float] = None,
num_actors: Optional[Union[int, str]] = None,
actor_config: Optional[dict] = None,
num_subbatches: Optional[int] = None,
subbatch_size: Optional[int] = None,
initial_bounds: Optional[BoundsPairLike] = (-0.00001, 0.00001),
):
"""
`__init__(...)`: Initialize the GymNE.
Args:
env: The gym environment to solve. Expected as a Callable
(maybe a function returning a gym.Env, or maybe a gym.Env
subclass), or as a string referring to a gym environment
ID (e.g. "Ant-v4", "Humanoid-v4", etc.).
network: A network structure string, or a Callable (which can be
a class inheriting from `torch.nn.Module`, or a function
which returns a `torch.nn.Module` instance), or an instance
of `torch.nn.Module`.
The object provided here determines the structure of the
neural network policy whose parameters will be evolved.
A network structure string is a string which can be processed
by `evotorch.neuroevolution.net.str_to_net(...)`.
Please see the documentation of the function
`evotorch.neuroevolution.net.str_to_net(...)` to see how such
a neural network structure string looks like.
Note that this network can be a recurrent network.
When the network's `forward(...)` method can optionally accept
an additional positional argument for the hidden state of the
network and returns an additional value for its next state,
then the policy is treated as a recurrent one.
When the network is given as a callable object (e.g.
a subclass of `nn.Module` or a function) and this callable
object is decorated via `evotorch.decorators.pass_info`,
the following keyword arguments will be passed:
(i) `obs_length` (the length of the observation vector),
(ii) `act_length` (the length of the action vector),
(iii) `obs_shape` (the shape tuple of the observation space),
(iv) `act_shape` (the shape tuple of the action space),
(v) `obs_space` (the Box object specifying the observation
space, and
(vi) `act_space` (the Box object specifying the action
space). Note that `act_space` will always be given as a
`gym.spaces.Box` instance, even when the actual gym
environment has a discrete action space. This because `GymNE`
always expects the neural network to return a tensor of
floating-point numbers.
env_name: Deprecated alias for the keyword argument `env`.
It is recommended to use the argument `env` instead.
network_args: Optionally a dict-like object, storing keyword
arguments to be passed to the network while instantiating it.
env_config: Keyword arguments to pass to `gym.make(...)` while
creating the `gym` environment.
observation_normalization: Whether or not to do online observation
normalization.
num_episodes: Number of episodes over which a single solution will
be evaluated.
episode_length: Maximum amount of simulator interactions allowed
in a single episode. If left as None, whether or not an episode
is terminated is determined only by the `gym` environment
itself.
decrease_rewards_by: Some gym env.s are defined in such a way that
the agent gets a constant reward for each timestep
it survives. This constant reward can also be called
"survival bonus". Such a rewarding scheme can lead the
evolution to local optima where the agent does nothing
but does not die either, just to collect the survival
bonuses. To prevent this, it can be desired to
remove the survival bonuses from each reward obtained.
If this is the case with the problem at hand,
the user can set the argument `decrease_rewards_by`
to a positive float number, and that number will
be subtracted from each reward.
alive_bonus_schedule: Use this to add a customized amount of
alive bonus.
If left as None (which is the default), additional alive
bonus will not be added.
If given as a tuple `(t, b)`, an alive bonus `b` will be
added onto all the rewards beyond the timestep `t`.
If given as a tuple `(t0, t1, b)`, a partial (linearly
increasing towards `b`) alive bonus will be added onto
all the rewards between the timesteps `t0` and `t1`,
and a full alive bonus (which equals to `b`) will be added
onto all the rewards beyond the timestep `t1`.
action_noise_stdev: If given as a real number `s`, then, for
each generated action, Gaussian noise with standard
deviation `s` will be sampled, and then this sampled noise
will be added onto the action.
If action noise is not desired, then this argument can be
left as None.
num_actors: Number of actors to create for parallelized
evaluation of the solutions.
One can also set this as "max", which means that
an actor will be created on each available CPU.
When the parallelization is enabled each actor will have its
own instance of the `gym` environment.
actor_config: A dictionary, representing the keyword arguments
to be passed to the options(...) used when creating the
ray actor objects. To be used for explicitly allocating
resources per each actor.
For example, for declaring that each actor is to use a GPU,
one can pass `actor_config=dict(num_gpus=1)`.
Can also be given as None (which is the default),
if no such options are to be passed.
num_subbatches: If `num_subbatches` is None (assuming that
`subbatch_size` is also None), then, when evaluating a
population, the population will be split into n pieces, `n`
being the number of actors, and each actor will evaluate
its assigned piece. If `num_subbatches` is an integer `m`,
then the population will be split into `m` pieces,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
subbatch_size: If `subbatch_size` is None (assuming that
`num_subbatches` is also None), then, when evaluating a
population, the population will be split into `n` pieces, `n`
being the number of actors, and each actor will evaluate its
assigned piece. If `subbatch_size` is an integer `m`,
then the population will be split into pieces of size `m`,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
When there can be significant difference across the solutions
in terms of computational requirements, specifying a
`subbatch_size` can be beneficial, because, while one
actor is busy with a subbatch containing computationally
challenging solutions, other actors can accept more
tasks and save time.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
initial_bounds: Specifies an interval from which the values of the
initial policy parameters will be drawn.
"""
# Store various environment information
if (env is not None) and (env_name is None):
self._env_maker = env
elif (env is None) and (env_name is not None):
self._env_maker = env_name
elif (env is not None) and (env_name is not None):
raise ValueError(
f"Received values for both `env` ({repr(env)}) and `env_name` ({repr(env_name)})."
f" Please specify the environment to solve via only one of these arguments, not both."
)
else:
raise ValueError("Environment name is missing. Please specify it via the argument `env`.")
# Make sure that the network argument is not missing.
if network is None:
raise ValueError(
"Received None via the argument `network`."
"Please provide the network as a string, or as a `Callable`, or as a `torch.nn.Module` instance."
)
# Store various environment information
self._env_config = {} if env_config is None else deepcopy(dict(env_config))
self._decrease_rewards_by = 0.0 if decrease_rewards_by is None else float(decrease_rewards_by)
self._alive_bonus_schedule = alive_bonus_schedule
self._action_noise_stdev = None if action_noise_stdev is None else float(action_noise_stdev)
self._observation_normalization = bool(observation_normalization)
self._num_episodes = int(num_episodes)
self._episode_length = None if episode_length is None else int(episode_length)
self._info_keys = dict(cumulative_reward="avg", interaction_count="sum")
self._env: Optional[gym.Env] = None
self._obs_stats: Optional[RunningStat] = None
self._collected_stats: Optional[RunningStat] = None
# Create a temporary environment to read its dimensions
tmp_env = _make_env(self._env_maker, **(self._env_config))
# Store the temporary environment's dimensions
self._obs_length = len(tmp_env.observation_space.low)
if isinstance(tmp_env.action_space, gym.spaces.Discrete):
self._act_length = tmp_env.action_space.n
self._box_act_space = gym.spaces.Box(low=float("-inf"), high=float("inf"), shape=(self._act_length,))
else:
self._act_length = len(tmp_env.action_space.low)
self._box_act_space = tmp_env.action_space
self._act_space = tmp_env.action_space
self._obs_space = tmp_env.observation_space
self._obs_shape = tmp_env.observation_space.low.shape
# Validate the space types of the environment
ensure_space_types(tmp_env)
if self._observation_normalization:
self._obs_stats = RunningStat()
self._collected_stats = RunningStat()
else:
self._obs_stats = None
self._collected_stats = None
self._interaction_count: int = 0
self._episode_count: int = 0
super().__init__(
objective_sense="max", # RL is maximization
network=network, # Using the policy as the network
network_args=network_args,
initial_bounds=initial_bounds,
num_actors=num_actors,
actor_config=actor_config,
subbatch_size=subbatch_size,
device="cpu",
)
self.after_eval_hook.append(self._extra_status)
@property
def _network_constants(self) -> dict:
return {
"obs_length": self._obs_length,
"act_length": self._act_length,
"obs_space": self._obs_space,
"act_space": self._box_act_space,
"obs_shape": self._obs_space.shape,
"act_shape": self._box_act_space.shape,
}
@property
def _str_network_constants(self) -> dict:
return {
"obs_space": self._obs_space.shape,
"act_space": self._box_act_space.shape,
}
def _instantiate_new_env(self, **kwargs) -> gym.Env:
env_config = {**kwargs, **(self._env_config)}
env = _make_env(self._env_maker, **env_config)
if self._alive_bonus_schedule is not None:
env = AliveBonusScheduleWrapper(env, self._alive_bonus_schedule)
return env
def _get_env(self) -> gym.Env:
if self._env is None:
self._env = self._instantiate_new_env()
return self._env
def _normalize_observation(self, observation: Iterable, *, update_stats: bool = True) -> Iterable:
observation = np.asarray(observation, dtype="float32")
if self.observation_normalization:
if update_stats:
self._obs_stats.update(observation)
self._collected_stats.update(observation)
return self._obs_stats.normalize(observation)
else:
return observation
def _use_policy(self, observation: Iterable, policy: nn.Module) -> Iterable:
with torch.no_grad():
result = policy(torch.as_tensor(observation, dtype=torch.float32, device="cpu")).numpy()
if self._action_noise_stdev is not None:
result = (
result
+ self.make_gaussian(len(result), center=0.0, stdev=self._action_noise_stdev, device="cpu").numpy()
)
env = self._get_env()
if isinstance(env.action_space, gym.spaces.Discrete):
result = np.argmax(result)
elif isinstance(env.action_space, gym.spaces.Box):
result = np.clip(result, env.action_space.low, env.action_space.high)
return result
def _prepare(self) -> None:
super()._prepare()
self._get_env()
@property
def network_device(self) -> Device:
"""The device on which the problem should place data e.g. the network
In the case of GymNE, supported Gym environments return numpy arrays on CPU which are converted to Tensors
Therefore, it is almost always optimal to place the network on CPU
"""
return torch.device("cpu")
def _rollout(
self,
*,
policy: nn.Module,
update_stats: bool = True,
visualize: bool = False,
decrease_rewards_by: Optional[float] = None,
) -> dict:
"""Peform a rollout of a network"""
if decrease_rewards_by is None:
decrease_rewards_by = self._decrease_rewards_by
else:
decrease_rewards_by = float(decrease_rewards_by)
policy = ensure_stateful(policy)
policy.reset()
if visualize:
env = self._instantiate_new_env(render_mode="human")
else:
env = self._get_env()
observation = self._normalize_observation(reset_env(env), update_stats=update_stats)
if visualize:
env.render()
t = 0
cumulative_reward = 0.0
while True:
observation, raw_reward, done, info = take_step_in_env(env, self._use_policy(observation, policy))
reward = raw_reward - decrease_rewards_by
t += 1
if update_stats:
self._interaction_count += 1
if visualize:
env.render()
observation = self._normalize_observation(observation, update_stats=update_stats)
cumulative_reward += reward
if done or ((self._episode_length is not None) and (t >= self._episode_length)):
if update_stats:
self._episode_count += 1
final_info = dict(cumulative_reward=cumulative_reward, interaction_count=t)
for k in self._info_keys:
if k not in final_info:
final_info[k] = info[k]
return final_info
@property
def _nonserialized_attribs(self) -> List[str]:
return super()._nonserialized_attribs + ["_env"]
def run(
self,
policy: Union[nn.Module, Iterable],
*,
update_stats: bool = False,
visualize: bool = False,
num_episodes: Optional[int] = None,
decrease_rewards_by: Optional[float] = None,
) -> dict:
"""
Evaluate the policy on the gym environment.
Args:
policy: The policy to be evaluated. This can be a torch module
or a sequence of real numbers representing the parameters
of a policy network.
update_stats: Whether or not to update the observation
normalization data while running the policy. If observation
normalization is not enabled, then this argument will be
ignored.
visualize: Whether or not to render the environment while running
the policy.
num_episodes: Over how many episodes will the policy be evaluated.
Expected as None (which is the default), or as an integer.
If given as None, then the `num_episodes` value that was given
while initializing this GymNE will be used.
decrease_rewards_by: How much each reward value should be
decreased. If left as None, the `decrease_rewards_by` value
value that was given while initializing this GymNE will be
used.
Returns:
A dictionary containing the score and the timestep count.
"""
if not isinstance(policy, nn.Module):
policy = self.make_net(policy)
if num_episodes is None:
num_episodes = self._num_episodes
try:
policy.eval()
episode_results = [
self._rollout(
policy=policy,
update_stats=update_stats,
visualize=visualize,
decrease_rewards_by=decrease_rewards_by,
)
for _ in range(num_episodes)
]
results = _accumulate_all_across_dicts(episode_results, self._info_keys)
return results
finally:
policy.train()
def visualize(
self,
policy: Union[nn.Module, Iterable],
*,
update_stats: bool = False,
num_episodes: Optional[int] = 1,
decrease_rewards_by: Optional[float] = None,
) -> dict:
"""
Evaluate the policy and render its actions in the environment.
Args:
policy: The policy to be evaluated. This can be a torch module
or a sequence of real numbers representing the parameters
of a policy network.
update_stats: Whether or not to update the observation
normalization data while running the policy. If observation
normalization is not enabled, then this argument will be
ignored.
num_episodes: Over how many episodes will the policy be evaluated.
Expected as None (which is the default), or as an integer.
If given as None, then the `num_episodes` value that was given
while initializing this GymNE will be used.
decrease_rewards_by: How much each reward value should be
decreased. If left as None, the `decrease_rewards_by` value
value that was given while initializing this GymNE will be
used.
Returns:
A dictionary containing the score and the timestep count.
"""
return self.run(
policy=policy,
update_stats=update_stats,
visualize=True,
num_episodes=num_episodes,
decrease_rewards_by=decrease_rewards_by,
)
def _ensure_obsnorm(self):
if not self.observation_normalization:
raise ValueError("This feature can only be used when observation_normalization=True.")
def get_observation_stats(self) -> RunningStat:
"""Get the observation stats"""
self._ensure_obsnorm()
return self._obs_stats
def _make_sync_data_for_actors(self) -> Any:
if self.observation_normalization:
return dict(obs_stats=self.get_observation_stats())
else:
return None
def set_observation_stats(self, rs: RunningStat):
"""Set the observation stats"""
self._ensure_obsnorm()
self._obs_stats.reset()
self._obs_stats.update(rs)
def _use_sync_data_from_main(self, received: dict):
for k, v in received.items():
if k == "obs_stats":
self.set_observation_stats(v)
def pop_observation_stats(self) -> RunningStat:
"""Get and clear the collected observation stats"""
self._ensure_obsnorm()
result = self._collected_stats
self._collected_stats = RunningStat()
return result
def _make_sync_data_for_main(self) -> Any:
result = dict(episode_count=self.episode_count, interaction_count=self.interaction_count)
if self.observation_normalization:
result["obs_stats_delta"] = self.pop_observation_stats()
return result
def update_observation_stats(self, rs: RunningStat):
"""Update the observation stats via another RunningStat instance"""
self._ensure_obsnorm()
self._obs_stats.update(rs)
def _use_sync_data_from_actors(self, received: list):
total_episode_count = 0
total_interaction_count = 0
for data in received:
data: dict
total_episode_count += data["episode_count"]
total_interaction_count += data["interaction_count"]
if self.observation_normalization:
self.update_observation_stats(data["obs_stats_delta"])
self.set_episode_count(total_episode_count)
self.set_interaction_count(total_interaction_count)
def _make_pickle_data_for_main(self) -> dict:
# For when the main Problem object (the non-remote one) gets pickled,
# this function returns the counters of this remote Problem instance,
# to be sent to the main one.
return dict(interaction_count=self.interaction_count, episode_count=self.episode_count)
def _use_pickle_data_from_main(self, state: dict):
# For when a newly unpickled Problem object gets (re)parallelized,
# this function restores the inner states specific to this remote
# worker. In the case of GymNE, those inner states are episode
# and interaction counters.
for k, v in state.items():
if k == "episode_count":
self.set_episode_count(v)
elif k == "interaction_count":
self.set_interaction_count(v)
else:
raise ValueError(f"When restoring the inner state of a remote worker, unrecognized state key: {k}")
def _extra_status(self, batch: SolutionBatch):
return dict(total_interaction_count=self.interaction_count, total_episode_count=self.episode_count)
@property
def observation_normalization(self) -> bool:
"""
Get whether or not observation normalization is enabled.
"""
return self._observation_normalization
def set_episode_count(self, n: int):
"""
Set the episode count manually.
"""
self._episode_count = int(n)
def set_interaction_count(self, n: int):
"""
Set the interaction count manually.
"""
self._interaction_count = int(n)
@property
def interaction_count(self) -> int:
"""
Get the total number of simulator interactions made.
"""
return self._interaction_count
@property
def episode_count(self) -> int:
"""
Get the total number of episodes completed.
"""
return self._episode_count
def _get_local_episode_count(self) -> int:
return self.episode_count
def _get_local_interaction_count(self) -> int:
return self.interaction_count
def _evaluate_network(self, policy: nn.Module) -> Union[float, torch.Tensor]:
result = self.run(
policy,
update_stats=True,
visualize=False,
num_episodes=self._num_episodes,
decrease_rewards_by=self._decrease_rewards_by,
)
return result["cumulative_reward"]
def to_policy(self, x: Iterable, *, clip_actions: bool = True) -> nn.Module:
"""
Convert the given parameter vector to a policy as a PyTorch module.
If the problem is configured to have observation normalization,
the PyTorch module also contains an additional normalization layer.
Args:
x: An sequence of real numbers, containing the parameters
of a policy. Can be a PyTorch tensor, a numpy array,
or a Solution.
clip_actions: Whether or not to add an action clipping layer so
that the generated actions will always be within an
acceptable range for the environment.
Returns:
The policy expressed by the parameters.
"""
policy = self.make_net(x)
if self.observation_normalization and (self._obs_stats.count > 0):
policy = ObsNormWrapperModule(policy, self._obs_stats)
if clip_actions and isinstance(self._get_env().action_space, gym.spaces.Box):
policy = ActClipWrapperModule(policy, self._get_env().action_space)
return policy
def save_solution(self, solution: Iterable, fname: Union[str, Path]):
"""
Save the solution into a pickle file.
Among the saved data within the pickle file are the solution
(as a PyTorch tensor), the policy (as a `torch.nn.Module` instance),
and observation stats (if any).
Args:
solution: The solution to be saved. This can be a PyTorch tensor,
a `Solution` instance, or any `Iterable`.
fname: The file name of the pickle file to be created.
"""
# Convert the solution to a PyTorch tensor on the cpu.
if isinstance(solution, torch.Tensor):
solution = solution.to("cpu")
elif isinstance(solution, Solution):
solution = solution.values.clone().to("cpu")
else:
solution = torch.as_tensor(solution, dtype=torch.float32, device="cpu")
if isinstance(solution, ReadOnlyTensor):
solution = solution.as_subclass(torch.Tensor)
policy = self.to_policy(solution).to("cpu")
# Store the solution and the policy.
result = {
"solution": solution,
"policy": policy,
}
# If available, store the observation stats.
if self.observation_normalization and (self._obs_stats is not None):
result["obs_mean"] = torch.as_tensor(self._obs_stats.mean)
result["obs_stdev"] = torch.as_tensor(self._obs_stats.stdev)
result["obs_sum"] = torch.as_tensor(self._obs_stats.sum)
result["obs_sum_of_squares"] = torch.as_tensor(self._obs_stats.sum_of_squares)
# Some additional data.
result["interaction_count"] = self.interaction_count
result["episode_count"] = self.episode_count
result["time"] = datetime.now()
# If the environment is specified via a string ID, then store that ID.
if isinstance(self._env_maker, str):
result["env"] = self._env_maker
# Save the dictionary which stores the data.
with open(fname, "wb") as f:
pickle.dump(result, f)
def get_env(self) -> gym.Env:
"""
Get the gym environment stored by this GymNE instance
"""
return self._get_env()
episode_count: int
property
readonly
¶
Get the total number of episodes completed.
interaction_count: int
property
readonly
¶
Get the total number of simulator interactions made.
network_device: Union[str, torch.device]
property
readonly
¶
The device on which the problem should place data e.g. the network In the case of GymNE, supported Gym environments return numpy arrays on CPU which are converted to Tensors Therefore, it is almost always optimal to place the network on CPU
observation_normalization: bool
property
readonly
¶
Get whether or not observation normalization is enabled.
__init__(self, env=None, network=None, *, env_name=None, network_args=None, env_config=None, observation_normalization=False, num_episodes=1, episode_length=None, decrease_rewards_by=None, alive_bonus_schedule=None, action_noise_stdev=None, num_actors=None, actor_config=None, num_subbatches=None, subbatch_size=None, initial_bounds=(-1e-05, 1e-05))
special
¶
__init__(...): Initialize the GymNE.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
env |
Union[str, Callable] |
The gym environment to solve. Expected as a Callable (maybe a function returning a gym.Env, or maybe a gym.Env subclass), or as a string referring to a gym environment ID (e.g. "Ant-v4", "Humanoid-v4", etc.). |
None |
network |
Union[str, torch.nn.modules.module.Module, Callable[[], torch.nn.modules.module.Module]] |
A network structure string, or a Callable (which can be
a class inheriting from |
None |
env_name |
Union[str, Callable] |
Deprecated alias for the keyword argument |
None |
network_args |
Optional[dict] |
Optionally a dict-like object, storing keyword arguments to be passed to the network while instantiating it. |
None |
env_config |
Optional[collections.abc.Mapping] |
Keyword arguments to pass to |
None |
observation_normalization |
bool |
Whether or not to do online observation normalization. |
False |
num_episodes |
int |
Number of episodes over which a single solution will be evaluated. |
1 |
episode_length |
Optional[int] |
Maximum amount of simulator interactions allowed
in a single episode. If left as None, whether or not an episode
is terminated is determined only by the |
None |
decrease_rewards_by |
Optional[float] |
Some gym env.s are defined in such a way that
the agent gets a constant reward for each timestep
it survives. This constant reward can also be called
"survival bonus". Such a rewarding scheme can lead the
evolution to local optima where the agent does nothing
but does not die either, just to collect the survival
bonuses. To prevent this, it can be desired to
remove the survival bonuses from each reward obtained.
If this is the case with the problem at hand,
the user can set the argument |
None |
alive_bonus_schedule |
Optional[tuple] |
Use this to add a customized amount of
alive bonus.
If left as None (which is the default), additional alive
bonus will not be added.
If given as a tuple |
None |
action_noise_stdev |
Optional[float] |
If given as a real number |
None |
num_actors |
Union[int, str] |
Number of actors to create for parallelized
evaluation of the solutions.
One can also set this as "max", which means that
an actor will be created on each available CPU.
When the parallelization is enabled each actor will have its
own instance of the |
None |
actor_config |
Optional[dict] |
A dictionary, representing the keyword arguments
to be passed to the options(...) used when creating the
ray actor objects. To be used for explicitly allocating
resources per each actor.
For example, for declaring that each actor is to use a GPU,
one can pass |
None |
num_subbatches |
Optional[int] |
If |
None |
subbatch_size |
Optional[int] |
If |
None |
initial_bounds |
Union[Iterable[Union[float, Iterable[float], torch.Tensor]], evotorch.core.BoundsPair] |
Specifies an interval from which the values of the initial policy parameters will be drawn. |
(-1e-05, 1e-05) |
Source code in evotorch/neuroevolution/gymne.py
def __init__(
self,
env: Optional[Union[str, Callable]] = None,
network: Optional[Union[str, nn.Module, Callable[[], nn.Module]]] = None,
*,
env_name: Optional[Union[str, Callable]] = None,
network_args: Optional[dict] = None,
env_config: Optional[Mapping] = None,
observation_normalization: bool = False,
num_episodes: int = 1,
episode_length: Optional[int] = None,
decrease_rewards_by: Optional[float] = None,
alive_bonus_schedule: Optional[tuple] = None,
action_noise_stdev: Optional[float] = None,
num_actors: Optional[Union[int, str]] = None,
actor_config: Optional[dict] = None,
num_subbatches: Optional[int] = None,
subbatch_size: Optional[int] = None,
initial_bounds: Optional[BoundsPairLike] = (-0.00001, 0.00001),
):
"""
`__init__(...)`: Initialize the GymNE.
Args:
env: The gym environment to solve. Expected as a Callable
(maybe a function returning a gym.Env, or maybe a gym.Env
subclass), or as a string referring to a gym environment
ID (e.g. "Ant-v4", "Humanoid-v4", etc.).
network: A network structure string, or a Callable (which can be
a class inheriting from `torch.nn.Module`, or a function
which returns a `torch.nn.Module` instance), or an instance
of `torch.nn.Module`.
The object provided here determines the structure of the
neural network policy whose parameters will be evolved.
A network structure string is a string which can be processed
by `evotorch.neuroevolution.net.str_to_net(...)`.
Please see the documentation of the function
`evotorch.neuroevolution.net.str_to_net(...)` to see how such
a neural network structure string looks like.
Note that this network can be a recurrent network.
When the network's `forward(...)` method can optionally accept
an additional positional argument for the hidden state of the
network and returns an additional value for its next state,
then the policy is treated as a recurrent one.
When the network is given as a callable object (e.g.
a subclass of `nn.Module` or a function) and this callable
object is decorated via `evotorch.decorators.pass_info`,
the following keyword arguments will be passed:
(i) `obs_length` (the length of the observation vector),
(ii) `act_length` (the length of the action vector),
(iii) `obs_shape` (the shape tuple of the observation space),
(iv) `act_shape` (the shape tuple of the action space),
(v) `obs_space` (the Box object specifying the observation
space, and
(vi) `act_space` (the Box object specifying the action
space). Note that `act_space` will always be given as a
`gym.spaces.Box` instance, even when the actual gym
environment has a discrete action space. This because `GymNE`
always expects the neural network to return a tensor of
floating-point numbers.
env_name: Deprecated alias for the keyword argument `env`.
It is recommended to use the argument `env` instead.
network_args: Optionally a dict-like object, storing keyword
arguments to be passed to the network while instantiating it.
env_config: Keyword arguments to pass to `gym.make(...)` while
creating the `gym` environment.
observation_normalization: Whether or not to do online observation
normalization.
num_episodes: Number of episodes over which a single solution will
be evaluated.
episode_length: Maximum amount of simulator interactions allowed
in a single episode. If left as None, whether or not an episode
is terminated is determined only by the `gym` environment
itself.
decrease_rewards_by: Some gym env.s are defined in such a way that
the agent gets a constant reward for each timestep
it survives. This constant reward can also be called
"survival bonus". Such a rewarding scheme can lead the
evolution to local optima where the agent does nothing
but does not die either, just to collect the survival
bonuses. To prevent this, it can be desired to
remove the survival bonuses from each reward obtained.
If this is the case with the problem at hand,
the user can set the argument `decrease_rewards_by`
to a positive float number, and that number will
be subtracted from each reward.
alive_bonus_schedule: Use this to add a customized amount of
alive bonus.
If left as None (which is the default), additional alive
bonus will not be added.
If given as a tuple `(t, b)`, an alive bonus `b` will be
added onto all the rewards beyond the timestep `t`.
If given as a tuple `(t0, t1, b)`, a partial (linearly
increasing towards `b`) alive bonus will be added onto
all the rewards between the timesteps `t0` and `t1`,
and a full alive bonus (which equals to `b`) will be added
onto all the rewards beyond the timestep `t1`.
action_noise_stdev: If given as a real number `s`, then, for
each generated action, Gaussian noise with standard
deviation `s` will be sampled, and then this sampled noise
will be added onto the action.
If action noise is not desired, then this argument can be
left as None.
num_actors: Number of actors to create for parallelized
evaluation of the solutions.
One can also set this as "max", which means that
an actor will be created on each available CPU.
When the parallelization is enabled each actor will have its
own instance of the `gym` environment.
actor_config: A dictionary, representing the keyword arguments
to be passed to the options(...) used when creating the
ray actor objects. To be used for explicitly allocating
resources per each actor.
For example, for declaring that each actor is to use a GPU,
one can pass `actor_config=dict(num_gpus=1)`.
Can also be given as None (which is the default),
if no such options are to be passed.
num_subbatches: If `num_subbatches` is None (assuming that
`subbatch_size` is also None), then, when evaluating a
population, the population will be split into n pieces, `n`
being the number of actors, and each actor will evaluate
its assigned piece. If `num_subbatches` is an integer `m`,
then the population will be split into `m` pieces,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
subbatch_size: If `subbatch_size` is None (assuming that
`num_subbatches` is also None), then, when evaluating a
population, the population will be split into `n` pieces, `n`
being the number of actors, and each actor will evaluate its
assigned piece. If `subbatch_size` is an integer `m`,
then the population will be split into pieces of size `m`,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
When there can be significant difference across the solutions
in terms of computational requirements, specifying a
`subbatch_size` can be beneficial, because, while one
actor is busy with a subbatch containing computationally
challenging solutions, other actors can accept more
tasks and save time.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
initial_bounds: Specifies an interval from which the values of the
initial policy parameters will be drawn.
"""
# Store various environment information
if (env is not None) and (env_name is None):
self._env_maker = env
elif (env is None) and (env_name is not None):
self._env_maker = env_name
elif (env is not None) and (env_name is not None):
raise ValueError(
f"Received values for both `env` ({repr(env)}) and `env_name` ({repr(env_name)})."
f" Please specify the environment to solve via only one of these arguments, not both."
)
else:
raise ValueError("Environment name is missing. Please specify it via the argument `env`.")
# Make sure that the network argument is not missing.
if network is None:
raise ValueError(
"Received None via the argument `network`."
"Please provide the network as a string, or as a `Callable`, or as a `torch.nn.Module` instance."
)
# Store various environment information
self._env_config = {} if env_config is None else deepcopy(dict(env_config))
self._decrease_rewards_by = 0.0 if decrease_rewards_by is None else float(decrease_rewards_by)
self._alive_bonus_schedule = alive_bonus_schedule
self._action_noise_stdev = None if action_noise_stdev is None else float(action_noise_stdev)
self._observation_normalization = bool(observation_normalization)
self._num_episodes = int(num_episodes)
self._episode_length = None if episode_length is None else int(episode_length)
self._info_keys = dict(cumulative_reward="avg", interaction_count="sum")
self._env: Optional[gym.Env] = None
self._obs_stats: Optional[RunningStat] = None
self._collected_stats: Optional[RunningStat] = None
# Create a temporary environment to read its dimensions
tmp_env = _make_env(self._env_maker, **(self._env_config))
# Store the temporary environment's dimensions
self._obs_length = len(tmp_env.observation_space.low)
if isinstance(tmp_env.action_space, gym.spaces.Discrete):
self._act_length = tmp_env.action_space.n
self._box_act_space = gym.spaces.Box(low=float("-inf"), high=float("inf"), shape=(self._act_length,))
else:
self._act_length = len(tmp_env.action_space.low)
self._box_act_space = tmp_env.action_space
self._act_space = tmp_env.action_space
self._obs_space = tmp_env.observation_space
self._obs_shape = tmp_env.observation_space.low.shape
# Validate the space types of the environment
ensure_space_types(tmp_env)
if self._observation_normalization:
self._obs_stats = RunningStat()
self._collected_stats = RunningStat()
else:
self._obs_stats = None
self._collected_stats = None
self._interaction_count: int = 0
self._episode_count: int = 0
super().__init__(
objective_sense="max", # RL is maximization
network=network, # Using the policy as the network
network_args=network_args,
initial_bounds=initial_bounds,
num_actors=num_actors,
actor_config=actor_config,
subbatch_size=subbatch_size,
device="cpu",
)
self.after_eval_hook.append(self._extra_status)
get_env(self)
¶
get_observation_stats(self)
¶
pop_observation_stats(self)
¶
run(self, policy, *, update_stats=False, visualize=False, num_episodes=None, decrease_rewards_by=None)
¶
Evaluate the policy on the gym environment.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
policy |
Union[torch.nn.modules.module.Module, Iterable] |
The policy to be evaluated. This can be a torch module or a sequence of real numbers representing the parameters of a policy network. |
required |
update_stats |
bool |
Whether or not to update the observation normalization data while running the policy. If observation normalization is not enabled, then this argument will be ignored. |
False |
visualize |
bool |
Whether or not to render the environment while running the policy. |
False |
num_episodes |
Optional[int] |
Over how many episodes will the policy be evaluated.
Expected as None (which is the default), or as an integer.
If given as None, then the |
None |
decrease_rewards_by |
Optional[float] |
How much each reward value should be
decreased. If left as None, the |
None |
Returns:
| Type | Description |
|---|---|
dict |
A dictionary containing the score and the timestep count. |
Source code in evotorch/neuroevolution/gymne.py
def run(
self,
policy: Union[nn.Module, Iterable],
*,
update_stats: bool = False,
visualize: bool = False,
num_episodes: Optional[int] = None,
decrease_rewards_by: Optional[float] = None,
) -> dict:
"""
Evaluate the policy on the gym environment.
Args:
policy: The policy to be evaluated. This can be a torch module
or a sequence of real numbers representing the parameters
of a policy network.
update_stats: Whether or not to update the observation
normalization data while running the policy. If observation
normalization is not enabled, then this argument will be
ignored.
visualize: Whether or not to render the environment while running
the policy.
num_episodes: Over how many episodes will the policy be evaluated.
Expected as None (which is the default), or as an integer.
If given as None, then the `num_episodes` value that was given
while initializing this GymNE will be used.
decrease_rewards_by: How much each reward value should be
decreased. If left as None, the `decrease_rewards_by` value
value that was given while initializing this GymNE will be
used.
Returns:
A dictionary containing the score and the timestep count.
"""
if not isinstance(policy, nn.Module):
policy = self.make_net(policy)
if num_episodes is None:
num_episodes = self._num_episodes
try:
policy.eval()
episode_results = [
self._rollout(
policy=policy,
update_stats=update_stats,
visualize=visualize,
decrease_rewards_by=decrease_rewards_by,
)
for _ in range(num_episodes)
]
results = _accumulate_all_across_dicts(episode_results, self._info_keys)
return results
finally:
policy.train()
save_solution(self, solution, fname)
¶
Save the solution into a pickle file.
Among the saved data within the pickle file are the solution
(as a PyTorch tensor), the policy (as a torch.nn.Module instance),
and observation stats (if any).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
solution |
Iterable |
The solution to be saved. This can be a PyTorch tensor,
a |
required |
fname |
Union[str, pathlib.Path] |
The file name of the pickle file to be created. |
required |
Source code in evotorch/neuroevolution/gymne.py
def save_solution(self, solution: Iterable, fname: Union[str, Path]):
"""
Save the solution into a pickle file.
Among the saved data within the pickle file are the solution
(as a PyTorch tensor), the policy (as a `torch.nn.Module` instance),
and observation stats (if any).
Args:
solution: The solution to be saved. This can be a PyTorch tensor,
a `Solution` instance, or any `Iterable`.
fname: The file name of the pickle file to be created.
"""
# Convert the solution to a PyTorch tensor on the cpu.
if isinstance(solution, torch.Tensor):
solution = solution.to("cpu")
elif isinstance(solution, Solution):
solution = solution.values.clone().to("cpu")
else:
solution = torch.as_tensor(solution, dtype=torch.float32, device="cpu")
if isinstance(solution, ReadOnlyTensor):
solution = solution.as_subclass(torch.Tensor)
policy = self.to_policy(solution).to("cpu")
# Store the solution and the policy.
result = {
"solution": solution,
"policy": policy,
}
# If available, store the observation stats.
if self.observation_normalization and (self._obs_stats is not None):
result["obs_mean"] = torch.as_tensor(self._obs_stats.mean)
result["obs_stdev"] = torch.as_tensor(self._obs_stats.stdev)
result["obs_sum"] = torch.as_tensor(self._obs_stats.sum)
result["obs_sum_of_squares"] = torch.as_tensor(self._obs_stats.sum_of_squares)
# Some additional data.
result["interaction_count"] = self.interaction_count
result["episode_count"] = self.episode_count
result["time"] = datetime.now()
# If the environment is specified via a string ID, then store that ID.
if isinstance(self._env_maker, str):
result["env"] = self._env_maker
# Save the dictionary which stores the data.
with open(fname, "wb") as f:
pickle.dump(result, f)
set_episode_count(self, n)
¶
set_interaction_count(self, n)
¶
set_observation_stats(self, rs)
¶
to_policy(self, x, *, clip_actions=True)
¶
Convert the given parameter vector to a policy as a PyTorch module.
If the problem is configured to have observation normalization, the PyTorch module also contains an additional normalization layer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Iterable |
An sequence of real numbers, containing the parameters of a policy. Can be a PyTorch tensor, a numpy array, or a Solution. |
required |
clip_actions |
bool |
Whether or not to add an action clipping layer so that the generated actions will always be within an acceptable range for the environment. |
True |
Returns:
| Type | Description |
|---|---|
Module |
The policy expressed by the parameters. |
Source code in evotorch/neuroevolution/gymne.py
def to_policy(self, x: Iterable, *, clip_actions: bool = True) -> nn.Module:
"""
Convert the given parameter vector to a policy as a PyTorch module.
If the problem is configured to have observation normalization,
the PyTorch module also contains an additional normalization layer.
Args:
x: An sequence of real numbers, containing the parameters
of a policy. Can be a PyTorch tensor, a numpy array,
or a Solution.
clip_actions: Whether or not to add an action clipping layer so
that the generated actions will always be within an
acceptable range for the environment.
Returns:
The policy expressed by the parameters.
"""
policy = self.make_net(x)
if self.observation_normalization and (self._obs_stats.count > 0):
policy = ObsNormWrapperModule(policy, self._obs_stats)
if clip_actions and isinstance(self._get_env().action_space, gym.spaces.Box):
policy = ActClipWrapperModule(policy, self._get_env().action_space)
return policy
update_observation_stats(self, rs)
¶
visualize(self, policy, *, update_stats=False, num_episodes=1, decrease_rewards_by=None)
¶
Evaluate the policy and render its actions in the environment.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
policy |
Union[torch.nn.modules.module.Module, Iterable] |
The policy to be evaluated. This can be a torch module or a sequence of real numbers representing the parameters of a policy network. |
required |
update_stats |
bool |
Whether or not to update the observation normalization data while running the policy. If observation normalization is not enabled, then this argument will be ignored. |
False |
num_episodes |
Optional[int] |
Over how many episodes will the policy be evaluated.
Expected as None (which is the default), or as an integer.
If given as None, then the |
1 |
decrease_rewards_by |
Optional[float] |
How much each reward value should be
decreased. If left as None, the |
None |
Returns:
| Type | Description |
|---|---|
dict |
A dictionary containing the score and the timestep count. |
Source code in evotorch/neuroevolution/gymne.py
def visualize(
self,
policy: Union[nn.Module, Iterable],
*,
update_stats: bool = False,
num_episodes: Optional[int] = 1,
decrease_rewards_by: Optional[float] = None,
) -> dict:
"""
Evaluate the policy and render its actions in the environment.
Args:
policy: The policy to be evaluated. This can be a torch module
or a sequence of real numbers representing the parameters
of a policy network.
update_stats: Whether or not to update the observation
normalization data while running the policy. If observation
normalization is not enabled, then this argument will be
ignored.
num_episodes: Over how many episodes will the policy be evaluated.
Expected as None (which is the default), or as an integer.
If given as None, then the `num_episodes` value that was given
while initializing this GymNE will be used.
decrease_rewards_by: How much each reward value should be
decreased. If left as None, the `decrease_rewards_by` value
value that was given while initializing this GymNE will be
used.
Returns:
A dictionary containing the score and the timestep count.
"""
return self.run(
policy=policy,
update_stats=update_stats,
visualize=True,
num_episodes=num_episodes,
decrease_rewards_by=decrease_rewards_by,
)
neproblem
¶
This namespace contains the NeuroevolutionProblem class.
NEProblem (BaseNEProblem)
¶
Base class for neuro-evolution problems where the goal is to optimize the parameters of a neural network represented as a PyTorch module.
Any problem inheriting from this class is expected to override the method
_evaluate_network(self, net: torch.nn.Module) -> Union[torch.Tensor, float]
where net is the neural network to be evaluated, and the return value
is a scalar or a vector (for multi-objective cases) expressing the
fitness value(s).
Alternatively, this class can be directly instantiated in the following way:
def f(module: MyTorchModuleClass) -> Union[float, torch.Tensor, tuple]:
# Evaluate the given PyTorch module here
fitness = ...
return fitness
problem = NEProblem("min", MyTorchModuleClass, f, ...)
which specifies that the problem's goal is to minimize the return of the
function f.
For multi-objective cases, the fitness returned by f is expected as a
1-dimensional tensor. For when the problem has additional evaluation data,
a two-element tuple can be returned by f instead, where the first
element is the fitness value(s) and the second element is a 1-dimensional
tensor storing the additional data.
Source code in evotorch/neuroevolution/neproblem.py
class NEProblem(BaseNEProblem):
"""
Base class for neuro-evolution problems where the goal is to optimize the
parameters of a neural network represented as a PyTorch module.
Any problem inheriting from this class is expected to override the method
`_evaluate_network(self, net: torch.nn.Module) -> Union[torch.Tensor, float]`
where `net` is the neural network to be evaluated, and the return value
is a scalar or a vector (for multi-objective cases) expressing the
fitness value(s).
Alternatively, this class can be directly instantiated in the following
way:
```python
def f(module: MyTorchModuleClass) -> Union[float, torch.Tensor, tuple]:
# Evaluate the given PyTorch module here
fitness = ...
return fitness
problem = NEProblem("min", MyTorchModuleClass, f, ...)
```
which specifies that the problem's goal is to minimize the return of the
function `f`.
For multi-objective cases, the fitness returned by `f` is expected as a
1-dimensional tensor. For when the problem has additional evaluation data,
a two-element tuple can be returned by `f` instead, where the first
element is the fitness value(s) and the second element is a 1-dimensional
tensor storing the additional data.
"""
def __init__(
self,
objective_sense: ObjectiveSense,
network: Union[str, nn.Module, Callable[[], nn.Module]],
network_eval_func: Optional[Callable] = None,
*,
network_args: Optional[dict] = None,
initial_bounds: Optional[BoundsPairLike] = (-0.00001, 0.00001),
eval_dtype: Optional[DType] = None,
eval_data_length: int = 0,
seed: Optional[int] = None,
num_actors: Optional[Union[int, str]] = None,
actor_config: Optional[dict] = None,
num_gpus_per_actor: Optional[Union[int, float, str]] = None,
num_subbatches: Optional[int] = None,
subbatch_size: Optional[int] = None,
device: Optional[Device] = None,
):
"""
`__init__(...)`: Initialize the NEProblem.
Args:
objective_sense: The objective sense, expected as "min" or "max"
for single-objective cases, or as a sequence of strings
(each string being "min" or "max") for multi-objective cases.
network: A network structure string, or a Callable (which can be
a class inheriting from `torch.nn.Module`, or a function
which returns a `torch.nn.Module` instance), or an instance
of `torch.nn.Module`.
The object provided here determines the structure of the
neural network whose parameters will be evolved.
A network structure string is a string which can be processed
by `evotorch.neuroevolution.net.str_to_net(...)`.
Please see the documentation of the function
`evotorch.neuroevolution.net.str_to_net(...)` to see how such
a neural network structure string looks like.
network_eval_func: Optionally a function (or any Callable object)
which receives a PyTorch module as its argument, and returns
either a fitness, or a two-element tuple containing the fitness
and the additional evaluation data. The fitness can be a scalar
(for single-objective cases) or a 1-dimensional tensor (for
multi-objective cases). The additional evaluation data is
expected as a 1-dimensional tensor.
If this argument is left as None, it will be expected that
the method `_evaluate_network(...)` is overriden by the
inheriting class.
network_args: Optionally a dict-like object, storing keyword
arguments to be passed to the network while instantiating it.
initial_bounds: Specifies an interval from which the values of the
initial neural network parameters will be drawn.
eval_dtype: dtype to be used for fitnesses. If not specified, then
`eval_dtype` will be inferred from the dtype of the parameters
of the neural network.
In more details, if the neural network's parameters have a
float dtype, `eval_dtype` will be a compatible float.
Otherwise, it will be "float32".
eval_data_length: Length of the extra evaluation data.
seed: Random number seed. If left as None, this NEProblem instance
will not have its own random generator, and the global random
generator of PyTorch will be used instead.
num_actors: Number of actors to create for parallelized
evaluation of the solutions.
Certain string values are also accepted.
When given as "max" or as "num_cpus", the number of actors
will be equal to the number of all available CPUs in the ray
cluster.
When given as "num_gpus", the number of actors will be
equal to the number of all available GPUs in the ray
cluster, and each actor will be assigned a GPU.
When given as "num_devices", the number of actors will be
equal to the minimum among the number of CPUs and the number
of GPUs available in the cluster (or will be equal to the
number of CPUs if there is no GPU), and each actor will be
assigned a GPU (if available).
If `num_actors` is given as "num_gpus" or "num_devices",
the argument `num_gpus_per_actor` must not be used,
and the `actor_config` dictionary must not contain the
key "num_gpus".
If `num_actors` is given as something other than "num_gpus"
or "num_devices", and if you wish to assign GPUs to each
actor, then please see the argument `num_gpus_per_actor`.
actor_config: A dictionary, representing the keyword arguments
to be passed to the options(...) used when creating the
ray actor objects. To be used for explicitly allocating
resources per each actor.
For example, for declaring that each actor is to use a GPU,
one can pass `actor_config=dict(num_gpus=1)`.
Can also be given as None (which is the default),
if no such options are to be passed.
num_gpus_per_actor: Number of GPUs to be allocated by each
remote actor.
The default behavior is to NOT allocate any GPU at all
(which is the default behavior of the ray library as well).
When given as a number `n`, each actor will be given
`n` GPUs (where `n` can be an integer, or can be a `float`
for fractional allocation).
When given as a string "max", then the available GPUs
across the entire ray cluster (or within the local computer
in the simplest cases) will be equally distributed among
the actors.
When given as a string "all", then each actor will have
access to all the GPUs (this will be achieved by suppressing
the environment variable `CUDA_VISIBLE_DEVICES` for each
actor).
When the problem is not distributed (i.e. when there are
no actors), this argument is expected to be left as None.
num_subbatches: If `num_subbatches` is None (assuming that
`subbatch_size` is also None), then, when evaluating a
population, the population will be split into n pieces, `n`
being the number of actors, and each actor will evaluate
its assigned piece. If `num_subbatches` is an integer `m`,
then the population will be split into `m` pieces,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
While using a distributed algorithm, this argument determines
how many sub-batches will be generated, and therefore,
how many gradients will be computed by the remote actors.
subbatch_size: If `subbatch_size` is None (assuming that
`num_subbatches` is also None), then, when evaluating a
population, the population will be split into `n` pieces, `n`
being the number of actors, and each actor will evaluate its
assigned piece. If `subbatch_size` is an integer `m`,
then the population will be split into pieces of size `m`,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
When there can be significant difference across the solutions
in terms of computational requirements, specifying a
`subbatch_size` can be beneficial, because, while one
actor is busy with a subbatch containing computationally
challenging solutions, other actors can accept more
tasks and save time.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
While using a distributed algorithm, this argument determines
the size of a sub-batch (or sub-population) sampled by a
remote actor for computing a gradient.
In distributed mode, it is expected that the population size
is divisible by `subbatch_size`.
device: Default device in which a new population will be generated
and the neural networks will operate.
If not specified, "cpu" will be used.
"""
# Set the main device of the problem
# Although the operation of setting the main device is done by the main Problem class,
# here we need this at an earlier stage.
if device is None:
device = "cpu"
self._device = torch.device(device)
# Set the network
self._original_network = network
self._network_args = {} if network_args is None else deepcopy(network_args)
if isinstance(self._original_network, nn.Module):
self._original_network = self._original_network.cpu()
# Store the function that will evaluate the network, if available
self._network_eval_func: Optional[Callable] = network_eval_func
self.instantiated_network: nn.Module = None
# Create temporary network
temp_network = self._instantiate_net(self._original_network, device="cpu")
super().__init__(
objective_sense=objective_sense,
initial_bounds=initial_bounds,
bounds=None, # Neuroevolution is an unbounded problem
solution_length=count_parameters(temp_network), # The solution length is inherited from the network passed
dtype=next(temp_network.parameters()).dtype, # The datatype is inherited from the network passed
eval_dtype=eval_dtype,
device=device,
eval_data_length=eval_data_length,
seed=seed,
num_actors=num_actors,
num_gpus_per_actor=num_gpus_per_actor,
actor_config=actor_config,
num_subbatches=num_subbatches,
subbatch_size=subbatch_size,
store_solution_stats=None,
)
@property
def network_device(self) -> Device:
"""The device on which the problem should place data e.g. the network"""
cpu_device = torch.device("cpu")
if self.is_main:
# This is the case where this is the main process (not a remote actor)
if self.device == cpu_device:
# If the main device of the problem is "cpu", then we assume that the network is going to be on the cpu as well
return cpu_device
else:
# If the main device of the problem is some other device, then it is that device into which the network will be put
return self.device
else:
# If this is a remote actor, then the network will be put into the auxiliary device allocated for that actor
return self.aux_device
@property
def _str_network_constants(self) -> dict:
"""
Named constants which will be passed to `str_to_net`.
To be overridden by the user for custom fixed constants for a problem.
"""
return {}
@property
def _network_constants(self) -> dict:
"""
Named constants which will be passed to the network instantiation.
To be overridden by the user for custom fixed constants for a problem.
"""
return {}
def network_constants(self) -> dict:
"""Named constants which can be passed to the network instantiation"""
constants = {}
constants.update(self._network_constants)
constants.update(self._network_args)
return constants
@property
def _nonserialized_attribs(self) -> List[str]:
return ["instantiated_network"]
def _instantiate_net(self, network: Union[str, nn.Module, dict], device: Optional[Device] = None) -> nn.Module:
"""Instantiate the network on the target device, to be overridden by the user for custom behaviour
Returns:
instantiated_network (nn.Module): The network instantiated on the target device
"""
# Branching point determines instantiation of network
if isinstance(network, str):
# Passed argument was a string representation of a torch module
net_consts = {}
net_consts.update(self.network_constants())
net_consts.update(self._str_network_constants)
instantiated_network = str_to_net(network, **net_consts)
elif isinstance(network, nn.Module):
# Passed argument was directly a torch module
instantiated_network = network
else:
# Passed argument was callable yielding network
instantiated_network = pass_info_if_needed(network, self._network_constants)(**self._network_args)
# Map to device
device = self.network_device if device is None else device
instantiated_network = instantiated_network.to(device)
return instantiated_network
def _prepare(self) -> None:
"""Instantiate the network on the target device, if not already done"""
self.instantiated_network = self._instantiate_net(self._original_network)
# Clear reference to original network
self._original_network = None
def make_net(self, parameters: Iterable) -> nn.Module:
"""
Make a new network filled with the provided parameters.
Args:
parameters: Parameters to be used as weights within the network.
Can be a Solution, or any 1-dimensional Iterable that can be
converted to a PyTorch tensor.
Returns:
A new network, as a `torch.Module` instance.
"""
if isinstance(parameters, Solution):
parameters = parameters.access_values(keep_evals=True)
else:
parameters = self.as_tensor(parameters)
with torch.no_grad():
net = deepcopy(self.parameterize_net(parameters))
return net
def parameterize_net(self, parameters: torch.Tensor) -> nn.Module:
"""Parameterize the network with a given set of parameters.
Args:
parameters (torch.Tensor): The parameters with which to instantiate the network
Returns:
instantiated_network (nn.Module): The network instantiated with the parameters
"""
# Check if network exists
if self.instantiated_network is None:
self.instantiated_network = self._instantiate_net(self._original_network)
network = self.instantiated_network
# Move the parameters if needed
if parameters.device != self.network_device:
parameters = parameters.to(self.network_device)
# Fill the network with the parameters
fill_parameters(network, parameters)
# Return the network
return network
@property
def _grad_device(self) -> Device:
"""
Get the device in which new solutions will be made in distributed mode.
In more details, in distributed mode, each actor creates its own
sub-populations, evaluates them, and computes its own gradient
(all such actor gradients eventually being collected by the
distribution-based search algorithm in the main process).
For some problem types, it can make sense for the remote actors to
create their temporary sub-populations on another device
(e.g. on the GPU that is allocated specifically for them).
For such situations, one is encouraged to override this property
and make it return whatever device is to be used.
In the case of NEProblem, this property returns whatever device
is specified by the property `network_device`.
"""
return self.network_device
def _evaluate_network(self, network: nn.Module) -> Union[float, torch.Tensor, tuple]:
"""
Evaluate a network and return the evaluation result(s).
In the case where the `__init__` of `NEProblem` was not given
a network evaluator function (via the argument `network_eval_func`),
it will be expected that the inheriting class overrides this
method and defines how a network should be evaluated.
Args:
network (nn.Module): The network to evaluate
Returns:
fitness: The networks' fitness value(s), as a scalar for
single-objective cases, or as a 1-dimensional tensor
for multi-objective cases. The returned value can also
be a two-element tuple where the first element is the
fitness (as a scalar or as a vector) and the second
element is a 1-dimensional vector storing the extra
evaluation data.
"""
raise NotImplementedError
def _evaluate(self, solution: Solution):
"""
Evaluate a single solution.
This is achieved by parameterising the problem's attribute
named `instantiated_network`, and then evaluating the network
with the method `_evaluate_network(...)`.
Args:
solution (Solution): The solution to evaluate.
"""
parameters = solution.values
if self._network_eval_func is None:
evaluator = self._evaluate_network
else:
evaluator = self._network_eval_func
fitnesses = evaluator(self.parameterize_net(parameters))
if isinstance(fitnesses, tuple):
solution.set_evals(*fitnesses)
else:
solution.set_evals(fitnesses)
network_device: Union[str, torch.device]
property
readonly
¶
The device on which the problem should place data e.g. the network
__init__(self, objective_sense, network, network_eval_func=None, *, network_args=None, initial_bounds=(-1e-05, 1e-05), eval_dtype=None, eval_data_length=0, seed=None, num_actors=None, actor_config=None, num_gpus_per_actor=None, num_subbatches=None, subbatch_size=None, device=None)
special
¶
__init__(...): Initialize the NEProblem.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
objective_sense |
Union[str, Iterable[str]] |
The objective sense, expected as "min" or "max" for single-objective cases, or as a sequence of strings (each string being "min" or "max") for multi-objective cases. |
required |
network |
Union[str, torch.nn.modules.module.Module, Callable[[], torch.nn.modules.module.Module]] |
A network structure string, or a Callable (which can be
a class inheriting from |
required |
network_eval_func |
Optional[Callable] |
Optionally a function (or any Callable object)
which receives a PyTorch module as its argument, and returns
either a fitness, or a two-element tuple containing the fitness
and the additional evaluation data. The fitness can be a scalar
(for single-objective cases) or a 1-dimensional tensor (for
multi-objective cases). The additional evaluation data is
expected as a 1-dimensional tensor.
If this argument is left as None, it will be expected that
the method |
None |
network_args |
Optional[dict] |
Optionally a dict-like object, storing keyword arguments to be passed to the network while instantiating it. |
None |
initial_bounds |
Union[Iterable[Union[float, Iterable[float], torch.Tensor]], evotorch.core.BoundsPair] |
Specifies an interval from which the values of the initial neural network parameters will be drawn. |
(-1e-05, 1e-05) |
eval_dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
dtype to be used for fitnesses. If not specified, then
|
None |
eval_data_length |
int |
Length of the extra evaluation data. |
0 |
seed |
Optional[int] |
Random number seed. If left as None, this NEProblem instance will not have its own random generator, and the global random generator of PyTorch will be used instead. |
None |
num_actors |
Union[int, str] |
Number of actors to create for parallelized
evaluation of the solutions.
Certain string values are also accepted.
When given as "max" or as "num_cpus", the number of actors
will be equal to the number of all available CPUs in the ray
cluster.
When given as "num_gpus", the number of actors will be
equal to the number of all available GPUs in the ray
cluster, and each actor will be assigned a GPU.
When given as "num_devices", the number of actors will be
equal to the minimum among the number of CPUs and the number
of GPUs available in the cluster (or will be equal to the
number of CPUs if there is no GPU), and each actor will be
assigned a GPU (if available).
If |
None |
actor_config |
Optional[dict] |
A dictionary, representing the keyword arguments
to be passed to the options(...) used when creating the
ray actor objects. To be used for explicitly allocating
resources per each actor.
For example, for declaring that each actor is to use a GPU,
one can pass |
None |
num_gpus_per_actor |
Union[int, float, str] |
Number of GPUs to be allocated by each
remote actor.
The default behavior is to NOT allocate any GPU at all
(which is the default behavior of the ray library as well).
When given as a number |
None |
num_subbatches |
Optional[int] |
If |
None |
subbatch_size |
Optional[int] |
If |
None |
device |
Union[str, torch.device] |
Default device in which a new population will be generated and the neural networks will operate. If not specified, "cpu" will be used. |
None |
Source code in evotorch/neuroevolution/neproblem.py
def __init__(
self,
objective_sense: ObjectiveSense,
network: Union[str, nn.Module, Callable[[], nn.Module]],
network_eval_func: Optional[Callable] = None,
*,
network_args: Optional[dict] = None,
initial_bounds: Optional[BoundsPairLike] = (-0.00001, 0.00001),
eval_dtype: Optional[DType] = None,
eval_data_length: int = 0,
seed: Optional[int] = None,
num_actors: Optional[Union[int, str]] = None,
actor_config: Optional[dict] = None,
num_gpus_per_actor: Optional[Union[int, float, str]] = None,
num_subbatches: Optional[int] = None,
subbatch_size: Optional[int] = None,
device: Optional[Device] = None,
):
"""
`__init__(...)`: Initialize the NEProblem.
Args:
objective_sense: The objective sense, expected as "min" or "max"
for single-objective cases, or as a sequence of strings
(each string being "min" or "max") for multi-objective cases.
network: A network structure string, or a Callable (which can be
a class inheriting from `torch.nn.Module`, or a function
which returns a `torch.nn.Module` instance), or an instance
of `torch.nn.Module`.
The object provided here determines the structure of the
neural network whose parameters will be evolved.
A network structure string is a string which can be processed
by `evotorch.neuroevolution.net.str_to_net(...)`.
Please see the documentation of the function
`evotorch.neuroevolution.net.str_to_net(...)` to see how such
a neural network structure string looks like.
network_eval_func: Optionally a function (or any Callable object)
which receives a PyTorch module as its argument, and returns
either a fitness, or a two-element tuple containing the fitness
and the additional evaluation data. The fitness can be a scalar
(for single-objective cases) or a 1-dimensional tensor (for
multi-objective cases). The additional evaluation data is
expected as a 1-dimensional tensor.
If this argument is left as None, it will be expected that
the method `_evaluate_network(...)` is overriden by the
inheriting class.
network_args: Optionally a dict-like object, storing keyword
arguments to be passed to the network while instantiating it.
initial_bounds: Specifies an interval from which the values of the
initial neural network parameters will be drawn.
eval_dtype: dtype to be used for fitnesses. If not specified, then
`eval_dtype` will be inferred from the dtype of the parameters
of the neural network.
In more details, if the neural network's parameters have a
float dtype, `eval_dtype` will be a compatible float.
Otherwise, it will be "float32".
eval_data_length: Length of the extra evaluation data.
seed: Random number seed. If left as None, this NEProblem instance
will not have its own random generator, and the global random
generator of PyTorch will be used instead.
num_actors: Number of actors to create for parallelized
evaluation of the solutions.
Certain string values are also accepted.
When given as "max" or as "num_cpus", the number of actors
will be equal to the number of all available CPUs in the ray
cluster.
When given as "num_gpus", the number of actors will be
equal to the number of all available GPUs in the ray
cluster, and each actor will be assigned a GPU.
When given as "num_devices", the number of actors will be
equal to the minimum among the number of CPUs and the number
of GPUs available in the cluster (or will be equal to the
number of CPUs if there is no GPU), and each actor will be
assigned a GPU (if available).
If `num_actors` is given as "num_gpus" or "num_devices",
the argument `num_gpus_per_actor` must not be used,
and the `actor_config` dictionary must not contain the
key "num_gpus".
If `num_actors` is given as something other than "num_gpus"
or "num_devices", and if you wish to assign GPUs to each
actor, then please see the argument `num_gpus_per_actor`.
actor_config: A dictionary, representing the keyword arguments
to be passed to the options(...) used when creating the
ray actor objects. To be used for explicitly allocating
resources per each actor.
For example, for declaring that each actor is to use a GPU,
one can pass `actor_config=dict(num_gpus=1)`.
Can also be given as None (which is the default),
if no such options are to be passed.
num_gpus_per_actor: Number of GPUs to be allocated by each
remote actor.
The default behavior is to NOT allocate any GPU at all
(which is the default behavior of the ray library as well).
When given as a number `n`, each actor will be given
`n` GPUs (where `n` can be an integer, or can be a `float`
for fractional allocation).
When given as a string "max", then the available GPUs
across the entire ray cluster (or within the local computer
in the simplest cases) will be equally distributed among
the actors.
When given as a string "all", then each actor will have
access to all the GPUs (this will be achieved by suppressing
the environment variable `CUDA_VISIBLE_DEVICES` for each
actor).
When the problem is not distributed (i.e. when there are
no actors), this argument is expected to be left as None.
num_subbatches: If `num_subbatches` is None (assuming that
`subbatch_size` is also None), then, when evaluating a
population, the population will be split into n pieces, `n`
being the number of actors, and each actor will evaluate
its assigned piece. If `num_subbatches` is an integer `m`,
then the population will be split into `m` pieces,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
While using a distributed algorithm, this argument determines
how many sub-batches will be generated, and therefore,
how many gradients will be computed by the remote actors.
subbatch_size: If `subbatch_size` is None (assuming that
`num_subbatches` is also None), then, when evaluating a
population, the population will be split into `n` pieces, `n`
being the number of actors, and each actor will evaluate its
assigned piece. If `subbatch_size` is an integer `m`,
then the population will be split into pieces of size `m`,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
When there can be significant difference across the solutions
in terms of computational requirements, specifying a
`subbatch_size` can be beneficial, because, while one
actor is busy with a subbatch containing computationally
challenging solutions, other actors can accept more
tasks and save time.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
While using a distributed algorithm, this argument determines
the size of a sub-batch (or sub-population) sampled by a
remote actor for computing a gradient.
In distributed mode, it is expected that the population size
is divisible by `subbatch_size`.
device: Default device in which a new population will be generated
and the neural networks will operate.
If not specified, "cpu" will be used.
"""
# Set the main device of the problem
# Although the operation of setting the main device is done by the main Problem class,
# here we need this at an earlier stage.
if device is None:
device = "cpu"
self._device = torch.device(device)
# Set the network
self._original_network = network
self._network_args = {} if network_args is None else deepcopy(network_args)
if isinstance(self._original_network, nn.Module):
self._original_network = self._original_network.cpu()
# Store the function that will evaluate the network, if available
self._network_eval_func: Optional[Callable] = network_eval_func
self.instantiated_network: nn.Module = None
# Create temporary network
temp_network = self._instantiate_net(self._original_network, device="cpu")
super().__init__(
objective_sense=objective_sense,
initial_bounds=initial_bounds,
bounds=None, # Neuroevolution is an unbounded problem
solution_length=count_parameters(temp_network), # The solution length is inherited from the network passed
dtype=next(temp_network.parameters()).dtype, # The datatype is inherited from the network passed
eval_dtype=eval_dtype,
device=device,
eval_data_length=eval_data_length,
seed=seed,
num_actors=num_actors,
num_gpus_per_actor=num_gpus_per_actor,
actor_config=actor_config,
num_subbatches=num_subbatches,
subbatch_size=subbatch_size,
store_solution_stats=None,
)
make_net(self, parameters)
¶
Make a new network filled with the provided parameters.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
parameters |
Iterable |
Parameters to be used as weights within the network. Can be a Solution, or any 1-dimensional Iterable that can be converted to a PyTorch tensor. |
required |
Returns:
| Type | Description |
|---|---|
Module |
A new network, as a |
Source code in evotorch/neuroevolution/neproblem.py
def make_net(self, parameters: Iterable) -> nn.Module:
"""
Make a new network filled with the provided parameters.
Args:
parameters: Parameters to be used as weights within the network.
Can be a Solution, or any 1-dimensional Iterable that can be
converted to a PyTorch tensor.
Returns:
A new network, as a `torch.Module` instance.
"""
if isinstance(parameters, Solution):
parameters = parameters.access_values(keep_evals=True)
else:
parameters = self.as_tensor(parameters)
with torch.no_grad():
net = deepcopy(self.parameterize_net(parameters))
return net
network_constants(self)
¶
Named constants which can be passed to the network instantiation
parameterize_net(self, parameters)
¶
Parameterize the network with a given set of parameters.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
parameters |
torch.Tensor |
The parameters with which to instantiate the network |
required |
Returns:
| Type | Description |
|---|---|
instantiated_network (nn.Module) |
The network instantiated with the parameters |
Source code in evotorch/neuroevolution/neproblem.py
def parameterize_net(self, parameters: torch.Tensor) -> nn.Module:
"""Parameterize the network with a given set of parameters.
Args:
parameters (torch.Tensor): The parameters with which to instantiate the network
Returns:
instantiated_network (nn.Module): The network instantiated with the parameters
"""
# Check if network exists
if self.instantiated_network is None:
self.instantiated_network = self._instantiate_net(self._original_network)
network = self.instantiated_network
# Move the parameters if needed
if parameters.device != self.network_device:
parameters = parameters.to(self.network_device)
# Fill the network with the parameters
fill_parameters(network, parameters)
# Return the network
return network
net
special
¶
Utility classes and functions for neural networks
functional
¶
ModuleExpectingFlatParameters
¶
A wrapper which brings a functional interface around a torch module.
Similar to functorch.FunctionalModule, ModuleExpectingFlatParameters
turns a torch.nn.Module instance to a function which expects a new
leftmost argument representing the parameters of the network.
Unlike functorch.FunctionalModule, a ModuleExpectingFlatParameters
instance, as its name suggests, expects the network parameters to be
given as a 1-dimensional (i.e. flattened) tensor.
Also, unlike functorch.FunctionalModule, an instance of
ModuleExpectingFlatParameters is NOT an instance of torch.nn.Module.
PyTorch modules with buffers can be wrapped by this class, but it is assumed that those buffers are constant. If the wrapped module changes the value(s) of its buffer(s) during its forward passes, most probably things will NOT work right.
As an example, let us consider the following linear layer.
The functional counterpart of net can be obtained via:
from evotorch.neuroevolution.net import ModuleExpectingFlatParameters
fnet = ModuleExpectingFlatParameters(net)
Now, fnet is a callable object which expects network parameters
and network inputs. Let us call fnet with randomly generated network
parameters and with a randomly generated input tensor.
param_length = fnet.parameter_length
random_parameters = torch.randn(param_length)
random_input = torch.randn(3)
result = fnet(random_parameters, random_input)
Source code in evotorch/neuroevolution/net/functional.py
class ModuleExpectingFlatParameters:
"""
A wrapper which brings a functional interface around a torch module.
Similar to `functorch.FunctionalModule`, `ModuleExpectingFlatParameters`
turns a `torch.nn.Module` instance to a function which expects a new
leftmost argument representing the parameters of the network.
Unlike `functorch.FunctionalModule`, a `ModuleExpectingFlatParameters`
instance, as its name suggests, expects the network parameters to be
given as a 1-dimensional (i.e. flattened) tensor.
Also, unlike `functorch.FunctionalModule`, an instance of
`ModuleExpectingFlatParameters` is NOT an instance of `torch.nn.Module`.
PyTorch modules with buffers can be wrapped by this class, but it is
assumed that those buffers are constant. If the wrapped module changes
the value(s) of its buffer(s) during its forward passes, most probably
things will NOT work right.
As an example, let us consider the following linear layer.
```python
import torch
from torch import nn
net = nn.Linear(3, 8)
```
The functional counterpart of `net` can be obtained via:
```python
from evotorch.neuroevolution.net import ModuleExpectingFlatParameters
fnet = ModuleExpectingFlatParameters(net)
```
Now, `fnet` is a callable object which expects network parameters
and network inputs. Let us call `fnet` with randomly generated network
parameters and with a randomly generated input tensor.
```python
param_length = fnet.parameter_length
random_parameters = torch.randn(param_length)
random_input = torch.randn(3)
result = fnet(random_parameters, random_input)
```
"""
@torch.no_grad()
def __init__(self, net: nn.Module, *, disable_autograd_tracking: bool = False):
"""
`__init__(...)`: Initialize the `ModuleExpectingFlatParameters` instance.
Args:
net: The module that is to be wrapped by a functional interface.
disable_autograd_tracking: If given as True, all operations
regarding the wrapped module will be performed in the context
`torch.no_grad()`, forcefully disabling the autograd.
If given as False, autograd will not be affected.
The default is False.
"""
# Declare the variables which will store information regarding the parameters of the module.
self.__param_names = []
self.__param_shapes = []
self.__param_length = 0
self.__param_slices = []
self.__num_params = 0
# Iterate over the parameters of the module and fill the related information.
i = 0
j = 0
for pname, p in net.named_parameters():
self.__param_names.append(pname)
shape = p.shape
self.__param_shapes.append(shape)
length = _shape_length(shape)
self.__param_length += length
j = i + length
self.__param_slices.append(slice(i, j))
i = j
self.__num_params += 1
self.__buffer_dict = {bname: b.clone() for bname, b in net.named_buffers()}
self.__net = deepcopy(net)
self.__net.to("meta")
self.__disable_autograd_tracking = bool(disable_autograd_tracking)
def __transfer_buffers(self, x: torch.Tensor):
"""
Transfer the buffer tensors to the device of the given tensor.
Args:
x: The tensor whose device will also store the buffer tensors.
"""
for bname in self.__buffer_dict.keys():
self.__buffer_dict[bname] = torch.as_tensor(self.__buffer_dict[bname], device=x.device)
@property
def buffers(self) -> tuple:
"""Get the stored buffers"""
return tuple(self.__buffer_dict)
@property
def parameter_length(self) -> int:
return self.__param_length
def __call__(self, parameter_vector: torch.Tensor, x: torch.Tensor, h: Any = None) -> Any:
"""
Call the wrapped module's forward pass procedure.
Args:
parameter_vector: A 1-dimensional tensor which represents the
parameters of the tensor.
x: The inputs.
h: Hidden state(s), in case this is a recurrent network.
Returns:
The result of the forward pass.
"""
if parameter_vector.ndim != 1:
raise ValueError(
f"Expected the parameters as 1 dimensional,"
f" but the received parameter vector has {parameter_vector.ndim} dimensions"
)
if len(parameter_vector) != self.__param_length:
raise ValueError(
f"Expected a parameter vector of length {self.__param_length},"
f" but the received parameter vector's length is {len(parameter_vector)}."
)
state_args = [] if h is None else [h]
params_and_buffers = {}
for i, pname in enumerate(self.__param_names):
param_slice = self.__param_slices[i]
param_shape = self.__param_shapes[i]
param = parameter_vector[param_slice].reshape(param_shape)
params_and_buffers[pname] = param
# Make sure that the buffer tensors are in the same device with x
self.__transfer_buffers(x)
# Add the buffer tensors to the dictionary `params_and_buffers`
params_and_buffers.update(self.__buffer_dict)
# Prepare the no-gradient context if gradient tracking is disabled
context = torch.no_grad() if self.__disable_autograd_tracking else nullcontext()
# Run the module and return the results
with context:
return functional_call(self.__net, params_and_buffers, tuple([x, *state_args]))
buffers: tuple
property
readonly
¶
Get the stored buffers
__call__(self, parameter_vector, x, h=None)
special
¶
Call the wrapped module's forward pass procedure.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
parameter_vector |
Tensor |
A 1-dimensional tensor which represents the parameters of the tensor. |
required |
x |
Tensor |
The inputs. |
required |
h |
Any |
Hidden state(s), in case this is a recurrent network. |
None |
Returns:
| Type | Description |
|---|---|
Any |
The result of the forward pass. |
Source code in evotorch/neuroevolution/net/functional.py
def __call__(self, parameter_vector: torch.Tensor, x: torch.Tensor, h: Any = None) -> Any:
"""
Call the wrapped module's forward pass procedure.
Args:
parameter_vector: A 1-dimensional tensor which represents the
parameters of the tensor.
x: The inputs.
h: Hidden state(s), in case this is a recurrent network.
Returns:
The result of the forward pass.
"""
if parameter_vector.ndim != 1:
raise ValueError(
f"Expected the parameters as 1 dimensional,"
f" but the received parameter vector has {parameter_vector.ndim} dimensions"
)
if len(parameter_vector) != self.__param_length:
raise ValueError(
f"Expected a parameter vector of length {self.__param_length},"
f" but the received parameter vector's length is {len(parameter_vector)}."
)
state_args = [] if h is None else [h]
params_and_buffers = {}
for i, pname in enumerate(self.__param_names):
param_slice = self.__param_slices[i]
param_shape = self.__param_shapes[i]
param = parameter_vector[param_slice].reshape(param_shape)
params_and_buffers[pname] = param
# Make sure that the buffer tensors are in the same device with x
self.__transfer_buffers(x)
# Add the buffer tensors to the dictionary `params_and_buffers`
params_and_buffers.update(self.__buffer_dict)
# Prepare the no-gradient context if gradient tracking is disabled
context = torch.no_grad() if self.__disable_autograd_tracking else nullcontext()
# Run the module and return the results
with context:
return functional_call(self.__net, params_and_buffers, tuple([x, *state_args]))
__init__(self, net, *, disable_autograd_tracking=False)
special
¶
__init__(...): Initialize the ModuleExpectingFlatParameters instance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
net |
Module |
The module that is to be wrapped by a functional interface. |
required |
disable_autograd_tracking |
bool |
If given as True, all operations
regarding the wrapped module will be performed in the context
|
False |
Source code in evotorch/neuroevolution/net/functional.py
@torch.no_grad()
def __init__(self, net: nn.Module, *, disable_autograd_tracking: bool = False):
"""
`__init__(...)`: Initialize the `ModuleExpectingFlatParameters` instance.
Args:
net: The module that is to be wrapped by a functional interface.
disable_autograd_tracking: If given as True, all operations
regarding the wrapped module will be performed in the context
`torch.no_grad()`, forcefully disabling the autograd.
If given as False, autograd will not be affected.
The default is False.
"""
# Declare the variables which will store information regarding the parameters of the module.
self.__param_names = []
self.__param_shapes = []
self.__param_length = 0
self.__param_slices = []
self.__num_params = 0
# Iterate over the parameters of the module and fill the related information.
i = 0
j = 0
for pname, p in net.named_parameters():
self.__param_names.append(pname)
shape = p.shape
self.__param_shapes.append(shape)
length = _shape_length(shape)
self.__param_length += length
j = i + length
self.__param_slices.append(slice(i, j))
i = j
self.__num_params += 1
self.__buffer_dict = {bname: b.clone() for bname, b in net.named_buffers()}
self.__net = deepcopy(net)
self.__net.to("meta")
self.__disable_autograd_tracking = bool(disable_autograd_tracking)
make_functional_module(net, *, disable_autograd_tracking=False)
¶
Wrap a torch module so that it has a functional interface.
Similar to functorch.make_functional(...), this function turns a
torch.nn.Module instance to a function which expects a new leftmost
argument representing the parameters of the network.
Unlike with functorch.make_functional(...), the parameters of the
network are expected in a 1-dimensional (i.e. flattened) tensor.
PyTorch modules with buffers can be wrapped by this class, but it is assumed that those buffers are constant. If the wrapped module changes the value(s) of its buffer(s) during its forward passes, most probably things will NOT work right.
As an example, let us consider the following linear layer.
The functional counterpart of net can be obtained via:
Now, fnet is a callable object which expects network parameters
and network inputs. Let us call fnet with randomly generated network
parameters and with a randomly generated input tensor.
param_length = fnet.parameter_length
random_parameters = torch.randn(param_length)
random_input = torch.randn(3)
result = fnet(random_parameters, random_input)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
net |
Module |
The |
required |
disable_autograd_tracking |
bool |
If given as True, all operations
regarding the wrapped module will be performed in the context
|
False |
Returns:
| Type | Description |
|---|---|
ModuleExpectingFlatParameters |
The functional wrapper, as an instance of
|
Source code in evotorch/neuroevolution/net/functional.py
def make_functional_module(net: nn.Module, *, disable_autograd_tracking: bool = False) -> ModuleExpectingFlatParameters:
"""
Wrap a torch module so that it has a functional interface.
Similar to `functorch.make_functional(...)`, this function turns a
`torch.nn.Module` instance to a function which expects a new leftmost
argument representing the parameters of the network.
Unlike with `functorch.make_functional(...)`, the parameters of the
network are expected in a 1-dimensional (i.e. flattened) tensor.
PyTorch modules with buffers can be wrapped by this class, but it is
assumed that those buffers are constant. If the wrapped module changes
the value(s) of its buffer(s) during its forward passes, most probably
things will NOT work right.
As an example, let us consider the following linear layer.
```python
import torch
from torch import nn
net = nn.Linear(3, 8)
```
The functional counterpart of `net` can be obtained via:
```python
from evotorch.neuroevolution.net import make_functional_module
fnet = make_functional_module(net)
```
Now, `fnet` is a callable object which expects network parameters
and network inputs. Let us call `fnet` with randomly generated network
parameters and with a randomly generated input tensor.
```python
param_length = fnet.parameter_length
random_parameters = torch.randn(param_length)
random_input = torch.randn(3)
result = fnet(random_parameters, random_input)
```
Args:
net: The `torch.nn.Module` instance to be wrapped by a functional
interface.
disable_autograd_tracking: If given as True, all operations
regarding the wrapped module will be performed in the context
`torch.no_grad()`, forcefully disabling the autograd.
If given as False, autograd will not be affected.
The default is False.
Returns:
The functional wrapper, as an instance of
`evotorch.neuroevolution.net.ModuleExpectingFlatParameters`.
"""
return ModuleExpectingFlatParameters(net, disable_autograd_tracking=disable_autograd_tracking)
layers
¶
Various neural network layer types
Apply (Module)
¶
A torch module for applying an arithmetic operator on an input tensor
Source code in evotorch/neuroevolution/net/layers.py
class Apply(nn.Module):
"""A torch module for applying an arithmetic operator on an input tensor"""
def __init__(self, operator: str, argument: float):
"""`__init__(...)`: Initialize the Apply module.
Args:
operator: Must be '+', '-', '*', '/', or '**'.
Indicates which operation will be done
on the input tensor.
argument: Expected as a float, represents
the right-argument of the operation
(the left-argument being the input
tensor).
"""
nn.Module.__init__(self)
self._operator = str(operator)
assert self._operator in ("+", "-", "*", "/", "**")
self._argument = float(argument)
def forward(self, x):
op = self._operator
arg = self._argument
if op == "+":
return x + arg
elif op == "-":
return x - arg
elif op == "*":
return x * arg
elif op == "/":
return x / arg
elif op == "**":
return x**arg
else:
raise ValueError("Unknown operator:" + repr(op))
def extra_repr(self):
return "operator={}, argument={}".format(repr(self._operator), self._argument)
__init__(self, operator, argument)
special
¶
__init__(...): Initialize the Apply module.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
operator |
str |
Must be '+', '-', '', '/', or '*'. Indicates which operation will be done on the input tensor. |
required |
argument |
float |
Expected as a float, represents the right-argument of the operation (the left-argument being the input tensor). |
required |
Source code in evotorch/neuroevolution/net/layers.py
def __init__(self, operator: str, argument: float):
"""`__init__(...)`: Initialize the Apply module.
Args:
operator: Must be '+', '-', '*', '/', or '**'.
Indicates which operation will be done
on the input tensor.
argument: Expected as a float, represents
the right-argument of the operation
(the left-argument being the input
tensor).
"""
nn.Module.__init__(self)
self._operator = str(operator)
assert self._operator in ("+", "-", "*", "/", "**")
self._argument = float(argument)
extra_repr(self)
¶
Set the extra representation of the module.
To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.
forward(self, x)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
Source code in evotorch/neuroevolution/net/layers.py
Bin (Module)
¶
A small torch module for binning the values of tensors.
In more details, considering a lower bound value lb, an upper bound value ub, and an input tensor x, each value within x closer to lb will be converted to lb and each value within x closer to ub will be converted to ub.
Source code in evotorch/neuroevolution/net/layers.py
class Bin(nn.Module):
"""A small torch module for binning the values of tensors.
In more details, considering a lower bound value lb,
an upper bound value ub, and an input tensor x,
each value within x closer to lb will be converted to lb
and each value within x closer to ub will be converted to ub.
"""
def __init__(self, lb: float, ub: float):
"""`__init__(...)`: Initialize the Clip operator.
Args:
lb: Lower bound
ub: Upper bound
"""
nn.Module.__init__(self)
self._lb = float(lb)
self._ub = float(ub)
self._interval_size = self._ub - self._lb
self._shrink_amount = self._interval_size / 2.0
self._shift_amount = (self._ub + self._lb) / 2.0
def forward(self, x: torch.Tensor):
x = x - self._shift_amount
x = x / self._shrink_amount
x = torch.sign(x)
x = x * self._shrink_amount
x = x + self._shift_amount
return x
def extra_repr(self):
return "lb={}, ub={}".format(self._lb, self._ub)
__init__(self, lb, ub)
special
¶
__init__(...): Initialize the Clip operator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
lb |
float |
Lower bound |
required |
ub |
float |
Upper bound |
required |
Source code in evotorch/neuroevolution/net/layers.py
def __init__(self, lb: float, ub: float):
"""`__init__(...)`: Initialize the Clip operator.
Args:
lb: Lower bound
ub: Upper bound
"""
nn.Module.__init__(self)
self._lb = float(lb)
self._ub = float(ub)
self._interval_size = self._ub - self._lb
self._shrink_amount = self._interval_size / 2.0
self._shift_amount = (self._ub + self._lb) / 2.0
extra_repr(self)
¶
Set the extra representation of the module.
To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.
forward(self, x)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
Clip (Module)
¶
A small torch module for clipping the values of tensors
Source code in evotorch/neuroevolution/net/layers.py
class Clip(nn.Module):
"""A small torch module for clipping the values of tensors"""
def __init__(self, lb: float, ub: float):
"""`__init__(...)`: Initialize the Clip operator.
Args:
lb: Lower bound. Values less than this will be clipped.
ub: Upper bound. Values greater than this will be clipped.
"""
nn.Module.__init__(self)
self._lb = float(lb)
self._ub = float(ub)
def forward(self, x: torch.Tensor):
return x.clamp(self._lb, self._ub)
def extra_repr(self):
return "lb={}, ub={}".format(self._lb, self._ub)
__init__(self, lb, ub)
special
¶
__init__(...): Initialize the Clip operator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
lb |
float |
Lower bound. Values less than this will be clipped. |
required |
ub |
float |
Upper bound. Values greater than this will be clipped. |
required |
Source code in evotorch/neuroevolution/net/layers.py
extra_repr(self)
¶
Set the extra representation of the module.
To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.
forward(self, x)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
FeedForwardNet (Module)
¶
Representation of a feed forward neural network as a torch Module.
An example initialization of a FeedForwardNet is as follows:
net = drt.FeedForwardNet(4, [(8, 'tanh'), (6, 'tanh')])
which means that we would like to have a network which expects an input vector of length 4 and passes its input through 2 tanh-activated hidden layers (with neurons count 8 and 6, respectively). The output of the last hidden layer (of length 6) is the final output vector.
The string representation of the module obtained via the example above is:
FeedForwardNet(
(layer_0): Linear(in_features=4, out_features=8, bias=True)
(actfunc_0): Tanh()
(layer_1): Linear(in_features=8, out_features=6, bias=True)
(actfunc_1): Tanh()
)
Source code in evotorch/neuroevolution/net/layers.py
class FeedForwardNet(nn.Module):
"""
Representation of a feed forward neural network as a torch Module.
An example initialization of a FeedForwardNet is as follows:
net = drt.FeedForwardNet(4, [(8, 'tanh'), (6, 'tanh')])
which means that we would like to have a network which expects an input
vector of length 4 and passes its input through 2 tanh-activated hidden
layers (with neurons count 8 and 6, respectively).
The output of the last hidden layer (of length 6) is the final
output vector.
The string representation of the module obtained via the example above
is:
FeedForwardNet(
(layer_0): Linear(in_features=4, out_features=8, bias=True)
(actfunc_0): Tanh()
(layer_1): Linear(in_features=8, out_features=6, bias=True)
(actfunc_1): Tanh()
)
"""
LengthActTuple = Tuple[int, Union[str, Callable]]
LengthActBiasTuple = Tuple[int, Union[str, Callable], Union[bool]]
def __init__(self, input_size: int, layers: List[Union[LengthActTuple, LengthActBiasTuple]]):
"""`__init__(...)`: Initialize the FeedForward network.
Args:
input_size: Input size of the network, expected as an int.
layers: Expected as a list of tuples,
where each tuple is either of the form
`(layer_size, activation_function)`
or of the form
`(layer_size, activation_function, bias)`
in which
(i) `layer_size` is an int, specifying the number of neurons;
(ii) `activation_function` is None, or a callable object,
or a string containing the name of the activation function
('relu', 'selu', 'elu', 'tanh', 'hardtanh', or 'sigmoid');
(iii) `bias` is a boolean, specifying whether the layer
is to have a bias or not.
When omitted, bias is set to True.
"""
nn.Module.__init__(self)
for i, layer in enumerate(layers):
if len(layer) == 2:
size, actfunc = layer
bias = True
elif len(layer) == 3:
size, actfunc, bias = layer
else:
assert False, "A layer tuple of invalid size is encountered"
setattr(self, "layer_" + str(i), nn.Linear(input_size, size, bias=bias))
if isinstance(actfunc, str):
if actfunc == "relu":
actfunc = nn.ReLU()
elif actfunc == "selu":
actfunc = nn.SELU()
elif actfunc == "elu":
actfunc = nn.ELU()
elif actfunc == "tanh":
actfunc = nn.Tanh()
elif actfunc == "hardtanh":
actfunc = nn.Hardtanh()
elif actfunc == "sigmoid":
actfunc = nn.Sigmoid()
elif actfunc == "round":
actfunc = Round()
else:
raise ValueError("Unknown activation function: " + repr(actfunc))
setattr(self, "actfunc_" + str(i), actfunc)
input_size = size
def forward(self, x):
i = 0
while hasattr(self, "layer_" + str(i)):
x = getattr(self, "layer_" + str(i))(x)
f = getattr(self, "actfunc_" + str(i))
if f is not None:
x = f(x)
i += 1
return x
__init__(self, input_size, layers)
special
¶
__init__(...): Initialize the FeedForward network.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input_size |
int |
Input size of the network, expected as an int. |
required |
layers |
List[Union[Tuple[int, Union[str, Callable]], Tuple[int, Union[str, Callable], bool]]] |
Expected as a list of tuples,
where each tuple is either of the form
|
required |
Source code in evotorch/neuroevolution/net/layers.py
def __init__(self, input_size: int, layers: List[Union[LengthActTuple, LengthActBiasTuple]]):
"""`__init__(...)`: Initialize the FeedForward network.
Args:
input_size: Input size of the network, expected as an int.
layers: Expected as a list of tuples,
where each tuple is either of the form
`(layer_size, activation_function)`
or of the form
`(layer_size, activation_function, bias)`
in which
(i) `layer_size` is an int, specifying the number of neurons;
(ii) `activation_function` is None, or a callable object,
or a string containing the name of the activation function
('relu', 'selu', 'elu', 'tanh', 'hardtanh', or 'sigmoid');
(iii) `bias` is a boolean, specifying whether the layer
is to have a bias or not.
When omitted, bias is set to True.
"""
nn.Module.__init__(self)
for i, layer in enumerate(layers):
if len(layer) == 2:
size, actfunc = layer
bias = True
elif len(layer) == 3:
size, actfunc, bias = layer
else:
assert False, "A layer tuple of invalid size is encountered"
setattr(self, "layer_" + str(i), nn.Linear(input_size, size, bias=bias))
if isinstance(actfunc, str):
if actfunc == "relu":
actfunc = nn.ReLU()
elif actfunc == "selu":
actfunc = nn.SELU()
elif actfunc == "elu":
actfunc = nn.ELU()
elif actfunc == "tanh":
actfunc = nn.Tanh()
elif actfunc == "hardtanh":
actfunc = nn.Hardtanh()
elif actfunc == "sigmoid":
actfunc = nn.Sigmoid()
elif actfunc == "round":
actfunc = Round()
else:
raise ValueError("Unknown activation function: " + repr(actfunc))
setattr(self, "actfunc_" + str(i), actfunc)
input_size = size
forward(self, x)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
LSTM (Module)
¶
Source code in evotorch/neuroevolution/net/layers.py
class LSTM(nn.Module):
def __init__(
self,
input_size: int,
hidden_size: int,
*,
dtype: torch.dtype = torch.float32,
device: Union[str, torch.device] = "cpu",
):
super().__init__()
input_size = int(input_size)
hidden_size = int(hidden_size)
self.input_size = input_size
self.hidden_size = hidden_size
def input_weight():
return nn.Parameter(torch.randn(self.hidden_size, self.input_size, dtype=dtype, device=device))
def weight():
return nn.Parameter(torch.randn(self.hidden_size, self.hidden_size, dtype=dtype, device=device))
def bias():
return nn.Parameter(torch.zeros(self.hidden_size, dtype=dtype, device=device))
self.W_ii = input_weight()
self.W_if = input_weight()
self.W_ig = input_weight()
self.W_io = input_weight()
self.W_hi = weight()
self.W_hf = weight()
self.W_hg = weight()
self.W_ho = weight()
self.b_ii = bias()
self.b_if = bias()
self.b_ig = bias()
self.b_io = bias()
self.b_hi = bias()
self.b_hf = bias()
self.b_hg = bias()
self.b_ho = bias()
def forward(self, x: torch.Tensor, hidden=None) -> tuple:
sigm = torch.sigmoid
tanh = torch.tanh
if hidden is None:
h_prev = torch.zeros(self.hidden_size, dtype=x.dtype, device=x.device)
c_prev = torch.zeros(self.hidden_size, dtype=x.dtype, device=x.device)
else:
h_prev, c_prev = hidden
i_t = sigm(self.W_ii @ x + self.b_ii + self.W_hi @ h_prev + self.b_hi)
f_t = sigm(self.W_if @ x + self.b_if + self.W_hf @ h_prev + self.b_hf)
g_t = tanh(self.W_ig @ x + self.b_ig + self.W_hg @ h_prev + self.b_hg)
o_t = sigm(self.W_io @ x + self.b_io + self.W_ho @ h_prev + self.b_ho)
c_t = f_t * c_prev + i_t * g_t
h_t = o_t * tanh(c_t)
return h_t, (h_t, c_t)
def __repr__(self) -> str:
clsname = type(self).__name__
return f"{clsname}(input_size={self.input_size}, hidden_size={self.hidden_size})"
forward(self, x, hidden=None)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
Source code in evotorch/neuroevolution/net/layers.py
def forward(self, x: torch.Tensor, hidden=None) -> tuple:
sigm = torch.sigmoid
tanh = torch.tanh
if hidden is None:
h_prev = torch.zeros(self.hidden_size, dtype=x.dtype, device=x.device)
c_prev = torch.zeros(self.hidden_size, dtype=x.dtype, device=x.device)
else:
h_prev, c_prev = hidden
i_t = sigm(self.W_ii @ x + self.b_ii + self.W_hi @ h_prev + self.b_hi)
f_t = sigm(self.W_if @ x + self.b_if + self.W_hf @ h_prev + self.b_hf)
g_t = tanh(self.W_ig @ x + self.b_ig + self.W_hg @ h_prev + self.b_hg)
o_t = sigm(self.W_io @ x + self.b_io + self.W_ho @ h_prev + self.b_ho)
c_t = f_t * c_prev + i_t * g_t
h_t = o_t * tanh(c_t)
return h_t, (h_t, c_t)
LSTMNet (Module)
¶
Source code in evotorch/neuroevolution/net/layers.py
class LSTM(nn.Module):
def __init__(
self,
input_size: int,
hidden_size: int,
*,
dtype: torch.dtype = torch.float32,
device: Union[str, torch.device] = "cpu",
):
super().__init__()
input_size = int(input_size)
hidden_size = int(hidden_size)
self.input_size = input_size
self.hidden_size = hidden_size
def input_weight():
return nn.Parameter(torch.randn(self.hidden_size, self.input_size, dtype=dtype, device=device))
def weight():
return nn.Parameter(torch.randn(self.hidden_size, self.hidden_size, dtype=dtype, device=device))
def bias():
return nn.Parameter(torch.zeros(self.hidden_size, dtype=dtype, device=device))
self.W_ii = input_weight()
self.W_if = input_weight()
self.W_ig = input_weight()
self.W_io = input_weight()
self.W_hi = weight()
self.W_hf = weight()
self.W_hg = weight()
self.W_ho = weight()
self.b_ii = bias()
self.b_if = bias()
self.b_ig = bias()
self.b_io = bias()
self.b_hi = bias()
self.b_hf = bias()
self.b_hg = bias()
self.b_ho = bias()
def forward(self, x: torch.Tensor, hidden=None) -> tuple:
sigm = torch.sigmoid
tanh = torch.tanh
if hidden is None:
h_prev = torch.zeros(self.hidden_size, dtype=x.dtype, device=x.device)
c_prev = torch.zeros(self.hidden_size, dtype=x.dtype, device=x.device)
else:
h_prev, c_prev = hidden
i_t = sigm(self.W_ii @ x + self.b_ii + self.W_hi @ h_prev + self.b_hi)
f_t = sigm(self.W_if @ x + self.b_if + self.W_hf @ h_prev + self.b_hf)
g_t = tanh(self.W_ig @ x + self.b_ig + self.W_hg @ h_prev + self.b_hg)
o_t = sigm(self.W_io @ x + self.b_io + self.W_ho @ h_prev + self.b_ho)
c_t = f_t * c_prev + i_t * g_t
h_t = o_t * tanh(c_t)
return h_t, (h_t, c_t)
def __repr__(self) -> str:
clsname = type(self).__name__
return f"{clsname}(input_size={self.input_size}, hidden_size={self.hidden_size})"
forward(self, x, hidden=None)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
Source code in evotorch/neuroevolution/net/layers.py
def forward(self, x: torch.Tensor, hidden=None) -> tuple:
sigm = torch.sigmoid
tanh = torch.tanh
if hidden is None:
h_prev = torch.zeros(self.hidden_size, dtype=x.dtype, device=x.device)
c_prev = torch.zeros(self.hidden_size, dtype=x.dtype, device=x.device)
else:
h_prev, c_prev = hidden
i_t = sigm(self.W_ii @ x + self.b_ii + self.W_hi @ h_prev + self.b_hi)
f_t = sigm(self.W_if @ x + self.b_if + self.W_hf @ h_prev + self.b_hf)
g_t = tanh(self.W_ig @ x + self.b_ig + self.W_hg @ h_prev + self.b_hg)
o_t = sigm(self.W_io @ x + self.b_io + self.W_ho @ h_prev + self.b_ho)
c_t = f_t * c_prev + i_t * g_t
h_t = o_t * tanh(c_t)
return h_t, (h_t, c_t)
LocomotorNet (Module)
¶
This is a control network which consists of two components: one linear, and one non-linear. The non-linear component is an input-independent set of sinusoidals waves whose amplitudes, frequencies and phases are trainable. Upon execution of a forward pass, the output of the non-linear component is the sum of all these sinusoidal waves. The linear component is a linear layer (optionally with bias) whose weights (and biases) are trainable. The final output of the LocomotorNet at the end of a forward pass is the sum of the linear and the non-linear components.
Note that this is a stateful network, where the only state
is the timestep t, which starts from 0 and gets incremented by 1
at the end of each forward pass. The reset() method resets
t back to 0.
Reference
Mario Srouji, Jian Zhang, Ruslan Salakhutdinov (2018). Structured Control Nets for Deep Reinforcement Learning.
Source code in evotorch/neuroevolution/net/layers.py
class LocomotorNet(nn.Module):
"""LocomotorNet: A locomotion-specific structured control net.
This is a control network which consists of two components:
one linear, and one non-linear. The non-linear component
is an input-independent set of sinusoidals waves whose
amplitudes, frequencies and phases are trainable.
Upon execution of a forward pass, the output of the non-linear
component is the sum of all these sinusoidal waves.
The linear component is a linear layer (optionally with bias)
whose weights (and biases) are trainable.
The final output of the LocomotorNet at the end of a forward pass
is the sum of the linear and the non-linear components.
Note that this is a stateful network, where the only state
is the timestep t, which starts from 0 and gets incremented by 1
at the end of each forward pass. The `reset()` method resets
t back to 0.
Reference:
Mario Srouji, Jian Zhang, Ruslan Salakhutdinov (2018).
Structured Control Nets for Deep Reinforcement Learning.
"""
def __init__(self, *, in_features: int, out_features: int, bias: bool = True, num_sinusoids=16):
"""`__init__(...)`: Initialize the LocomotorNet.
Args:
in_features: Length of the input vector
out_features: Length of the output vector
bias: Whether or not the linear component is to have a bias
num_sinusoids: Number of sinusoidal waves
"""
nn.Module.__init__(self)
self._in_features = in_features
self._out_features = out_features
self._bias = bias
self._num_sinusoids = num_sinusoids
self._linear_component = nn.Linear(
in_features=self._in_features, out_features=self._out_features, bias=self._bias
)
self._amplitudes = nn.ParameterList()
self._frequencies = nn.ParameterList()
self._phases = nn.ParameterList()
for _ in range(self._num_sinusoids):
for paramlist in (self._amplitudes, self._frequencies, self._phases):
paramlist.append(nn.Parameter(torch.randn(self._out_features, dtype=torch.float32)))
self.reset()
def reset(self):
"""Set the timestep t to 0"""
self._t = 0
@property
def t(self) -> int:
"""The current timestep t"""
return self._t
@property
def in_features(self) -> int:
"""Get the length of the input vector"""
return self._in_features
@property
def out_features(self) -> int:
"""Get the length of the output vector"""
return self._out_features
@property
def num_sinusoids(self) -> int:
"""Get the number of sinusoidal waves of the non-linear component"""
return self._num_sinusoids
@property
def bias(self) -> bool:
"""Get whether or not the linear component has bias"""
return self._bias
def forward(self, x: torch.Tensor) -> torch.Tensor:
"""Execute a forward pass"""
u_linear = self._linear_component(x)
t = self._t
u_nonlinear = torch.zeros(self._out_features)
for i in range(self._num_sinusoids):
A = self._amplitudes[i]
w = self._frequencies[i]
phi = self._phases[i]
u_nonlinear = u_nonlinear + (A * torch.sin(w * t + phi))
self._t += 1
return u_linear + u_nonlinear
bias: bool
property
readonly
¶
Get whether or not the linear component has bias
in_features: int
property
readonly
¶
Get the length of the input vector
num_sinusoids: int
property
readonly
¶
Get the number of sinusoidal waves of the non-linear component
out_features: int
property
readonly
¶
Get the length of the output vector
t: int
property
readonly
¶
The current timestep t
__init__(self, *, in_features, out_features, bias=True, num_sinusoids=16)
special
¶
__init__(...): Initialize the LocomotorNet.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
in_features |
int |
Length of the input vector |
required |
out_features |
int |
Length of the output vector |
required |
bias |
bool |
Whether or not the linear component is to have a bias |
True |
num_sinusoids |
Number of sinusoidal waves |
16 |
Source code in evotorch/neuroevolution/net/layers.py
def __init__(self, *, in_features: int, out_features: int, bias: bool = True, num_sinusoids=16):
"""`__init__(...)`: Initialize the LocomotorNet.
Args:
in_features: Length of the input vector
out_features: Length of the output vector
bias: Whether or not the linear component is to have a bias
num_sinusoids: Number of sinusoidal waves
"""
nn.Module.__init__(self)
self._in_features = in_features
self._out_features = out_features
self._bias = bias
self._num_sinusoids = num_sinusoids
self._linear_component = nn.Linear(
in_features=self._in_features, out_features=self._out_features, bias=self._bias
)
self._amplitudes = nn.ParameterList()
self._frequencies = nn.ParameterList()
self._phases = nn.ParameterList()
for _ in range(self._num_sinusoids):
for paramlist in (self._amplitudes, self._frequencies, self._phases):
paramlist.append(nn.Parameter(torch.randn(self._out_features, dtype=torch.float32)))
self.reset()
forward(self, x)
¶
Execute a forward pass
Source code in evotorch/neuroevolution/net/layers.py
def forward(self, x: torch.Tensor) -> torch.Tensor:
"""Execute a forward pass"""
u_linear = self._linear_component(x)
t = self._t
u_nonlinear = torch.zeros(self._out_features)
for i in range(self._num_sinusoids):
A = self._amplitudes[i]
w = self._frequencies[i]
phi = self._phases[i]
u_nonlinear = u_nonlinear + (A * torch.sin(w * t + phi))
self._t += 1
return u_linear + u_nonlinear
reset(self)
¶
RNN (Module)
¶
Source code in evotorch/neuroevolution/net/layers.py
class RNN(nn.Module):
def __init__(
self,
input_size: int,
hidden_size: int,
nonlinearity: str = "tanh",
*,
dtype: torch.dtype = torch.float32,
device: Union[str, torch.device] = "cpu",
):
super().__init__()
input_size = int(input_size)
hidden_size = int(hidden_size)
nonlinearity = str(nonlinearity)
self.W1 = nn.Parameter(torch.randn(hidden_size, input_size, dtype=dtype, device=device))
self.W2 = nn.Parameter(torch.randn(hidden_size, hidden_size, dtype=dtype, device=device))
self.b1 = nn.Parameter(torch.zeros(hidden_size, dtype=dtype, device=device))
self.b2 = nn.Parameter(torch.zeros(hidden_size, dtype=dtype, device=device))
if nonlinearity == "tanh":
self.actfunc = torch.tanh
else:
self.actfunc = getattr(nnf, nonlinearity)
self.nonlinearity = nonlinearity
self.input_size = input_size
self.hidden_size = hidden_size
def forward(self, x: torch.Tensor, h: Optional[torch.Tensor] = None) -> tuple:
if h is None:
h = torch.zeros(self.hidden_size, dtype=x.dtype, device=x.device)
act = self.actfunc
W1 = self.W1
W2 = self.W2
b1 = self.b1.unsqueeze(-1)
b2 = self.b2.unsqueeze(-1)
x = x.unsqueeze(-1)
h = h.unsqueeze(-1)
y = act(((W1 @ x) + b1) + ((W2 @ h) + b2))
y = y.squeeze(-1)
return y, y
def __repr__(self) -> str:
clsname = type(self).__name__
return f"{clsname}(input_size={self.input_size}, hidden_size={self.hidden_size}, nonlinearity={repr(self.nonlinearity)})"
forward(self, x, h=None)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
Source code in evotorch/neuroevolution/net/layers.py
def forward(self, x: torch.Tensor, h: Optional[torch.Tensor] = None) -> tuple:
if h is None:
h = torch.zeros(self.hidden_size, dtype=x.dtype, device=x.device)
act = self.actfunc
W1 = self.W1
W2 = self.W2
b1 = self.b1.unsqueeze(-1)
b2 = self.b2.unsqueeze(-1)
x = x.unsqueeze(-1)
h = h.unsqueeze(-1)
y = act(((W1 @ x) + b1) + ((W2 @ h) + b2))
y = y.squeeze(-1)
return y, y
RecurrentNet (Module)
¶
Source code in evotorch/neuroevolution/net/layers.py
class RNN(nn.Module):
def __init__(
self,
input_size: int,
hidden_size: int,
nonlinearity: str = "tanh",
*,
dtype: torch.dtype = torch.float32,
device: Union[str, torch.device] = "cpu",
):
super().__init__()
input_size = int(input_size)
hidden_size = int(hidden_size)
nonlinearity = str(nonlinearity)
self.W1 = nn.Parameter(torch.randn(hidden_size, input_size, dtype=dtype, device=device))
self.W2 = nn.Parameter(torch.randn(hidden_size, hidden_size, dtype=dtype, device=device))
self.b1 = nn.Parameter(torch.zeros(hidden_size, dtype=dtype, device=device))
self.b2 = nn.Parameter(torch.zeros(hidden_size, dtype=dtype, device=device))
if nonlinearity == "tanh":
self.actfunc = torch.tanh
else:
self.actfunc = getattr(nnf, nonlinearity)
self.nonlinearity = nonlinearity
self.input_size = input_size
self.hidden_size = hidden_size
def forward(self, x: torch.Tensor, h: Optional[torch.Tensor] = None) -> tuple:
if h is None:
h = torch.zeros(self.hidden_size, dtype=x.dtype, device=x.device)
act = self.actfunc
W1 = self.W1
W2 = self.W2
b1 = self.b1.unsqueeze(-1)
b2 = self.b2.unsqueeze(-1)
x = x.unsqueeze(-1)
h = h.unsqueeze(-1)
y = act(((W1 @ x) + b1) + ((W2 @ h) + b2))
y = y.squeeze(-1)
return y, y
def __repr__(self) -> str:
clsname = type(self).__name__
return f"{clsname}(input_size={self.input_size}, hidden_size={self.hidden_size}, nonlinearity={repr(self.nonlinearity)})"
forward(self, x, h=None)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
Source code in evotorch/neuroevolution/net/layers.py
def forward(self, x: torch.Tensor, h: Optional[torch.Tensor] = None) -> tuple:
if h is None:
h = torch.zeros(self.hidden_size, dtype=x.dtype, device=x.device)
act = self.actfunc
W1 = self.W1
W2 = self.W2
b1 = self.b1.unsqueeze(-1)
b2 = self.b2.unsqueeze(-1)
x = x.unsqueeze(-1)
h = h.unsqueeze(-1)
y = act(((W1 @ x) + b1) + ((W2 @ h) + b2))
y = y.squeeze(-1)
return y, y
Round (Module)
¶
A small torch module for rounding the values of an input tensor
Source code in evotorch/neuroevolution/net/layers.py
class Round(nn.Module):
"""A small torch module for rounding the values of an input tensor"""
def __init__(self, ndigits: int = 0):
nn.Module.__init__(self)
self._ndigits = int(ndigits)
self._q = 10.0**self._ndigits
def forward(self, x):
x = x * self._q
x = torch.round(x)
x = x / self._q
return x
def extra_repr(self):
return "ndigits=" + str(self._ndigits)
extra_repr(self)
¶
Set the extra representation of the module.
To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.
forward(self, x)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
Slice (Module)
¶
A small torch module for getting the slice of an input tensor
Source code in evotorch/neuroevolution/net/layers.py
class Slice(nn.Module):
"""A small torch module for getting the slice of an input tensor"""
def __init__(self, from_index: int, to_index: int):
"""`__init__(...)`: Initialize the Slice operator.
Args:
from_index: The index from which the slice begins.
to_index: The exclusive index at which the slice ends.
"""
nn.Module.__init__(self)
self._from_index = from_index
self._to_index = to_index
def forward(self, x):
return x[self._from_index : self._to_index]
def extra_repr(self):
return "from_index={}, to_index={}".format(self._from_index, self._to_index)
__init__(self, from_index, to_index)
special
¶
__init__(...): Initialize the Slice operator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
from_index |
int |
The index from which the slice begins. |
required |
to_index |
int |
The exclusive index at which the slice ends. |
required |
Source code in evotorch/neuroevolution/net/layers.py
extra_repr(self)
¶
Set the extra representation of the module.
To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.
forward(self, x)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
StructuredControlNet (Module)
¶
Structured Control Net.
This is a control network consisting of two components: (i) a non-linear component, which is a feed-forward network; and (ii) a linear component, which is a linear layer. Both components take the input vector provided to the structured control network. The final output is the sum of the outputs of both components.
Reference
Mario Srouji, Jian Zhang, Ruslan Salakhutdinov (2018). Structured Control Nets for Deep Reinforcement Learning.
Source code in evotorch/neuroevolution/net/layers.py
class StructuredControlNet(nn.Module):
"""Structured Control Net.
This is a control network consisting of two components:
(i) a non-linear component, which is a feed-forward network; and
(ii) a linear component, which is a linear layer.
Both components take the input vector provided to the
structured control network.
The final output is the sum of the outputs of both components.
Reference:
Mario Srouji, Jian Zhang, Ruslan Salakhutdinov (2018).
Structured Control Nets for Deep Reinforcement Learning.
"""
def __init__(
self,
*,
in_features: int,
out_features: int,
num_layers: int,
hidden_size: int,
bias: bool = True,
nonlinearity: Union[str, Callable] = "tanh",
):
"""`__init__(...)`: Initialize the structured control net.
Args:
in_features: Length of the input vector
out_features: Length of the output vector
num_layers: Number of hidden layers for the non-linear component
hidden_size: Number of neurons in a hidden layer of the
non-linear component
bias: Whether or not the linear component is to have bias
nonlinearity: Activation function
"""
nn.Module.__init__(self)
self._in_features = in_features
self._out_features = out_features
self._num_layers = num_layers
self._hidden_size = hidden_size
self._bias = bias
self._nonlinearity = nonlinearity
self._linear_component = nn.Linear(
in_features=self._in_features, out_features=self._out_features, bias=self._bias
)
self._nonlinear_component = FeedForwardNet(
input_size=self._in_features,
layers=(
list((self._hidden_size, self._nonlinearity) for _ in range(self._num_layers))
+ [(self._out_features, self._nonlinearity)]
),
)
def forward(self, x: torch.Tensor) -> torch.Tensor:
"""TODO: documentation"""
return self._linear_component(x) + self._nonlinear_component(x)
@property
def in_features(self):
"""TODO: documentation"""
return self._in_features
@property
def out_features(self):
"""TODO: documentation"""
return self._out_features
@property
def num_layers(self):
"""TODO: documentation"""
return self._num_layers
@property
def hidden_size(self):
"""TODO: documentation"""
return self._hidden_size
@property
def bias(self):
"""TODO: documentation"""
return self._bias
@property
def nonlinearity(self):
"""TODO: documentation"""
return self._nonlinearity
bias
property
readonly
¶
hidden_size
property
readonly
¶
in_features
property
readonly
¶
nonlinearity
property
readonly
¶
num_layers
property
readonly
¶
out_features
property
readonly
¶
__init__(self, *, in_features, out_features, num_layers, hidden_size, bias=True, nonlinearity='tanh')
special
¶
__init__(...): Initialize the structured control net.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
in_features |
int |
Length of the input vector |
required |
out_features |
int |
Length of the output vector |
required |
num_layers |
int |
Number of hidden layers for the non-linear component |
required |
hidden_size |
int |
Number of neurons in a hidden layer of the non-linear component |
required |
bias |
bool |
Whether or not the linear component is to have bias |
True |
nonlinearity |
Union[str, Callable] |
Activation function |
'tanh' |
Source code in evotorch/neuroevolution/net/layers.py
def __init__(
self,
*,
in_features: int,
out_features: int,
num_layers: int,
hidden_size: int,
bias: bool = True,
nonlinearity: Union[str, Callable] = "tanh",
):
"""`__init__(...)`: Initialize the structured control net.
Args:
in_features: Length of the input vector
out_features: Length of the output vector
num_layers: Number of hidden layers for the non-linear component
hidden_size: Number of neurons in a hidden layer of the
non-linear component
bias: Whether or not the linear component is to have bias
nonlinearity: Activation function
"""
nn.Module.__init__(self)
self._in_features = in_features
self._out_features = out_features
self._num_layers = num_layers
self._hidden_size = hidden_size
self._bias = bias
self._nonlinearity = nonlinearity
self._linear_component = nn.Linear(
in_features=self._in_features, out_features=self._out_features, bias=self._bias
)
self._nonlinear_component = FeedForwardNet(
input_size=self._in_features,
layers=(
list((self._hidden_size, self._nonlinearity) for _ in range(self._num_layers))
+ [(self._out_features, self._nonlinearity)]
),
)
forward(self, x)
¶
misc
¶
Utilities for reading and for writing neural network parameters
count_parameters(net)
¶
Get the number of parameters the network.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
net |
Module |
The torch module whose parameters will be counted. |
required |
Returns:
| Type | Description |
|---|---|
int |
The number of parameters, as an integer. |
Source code in evotorch/neuroevolution/net/misc.py
device_of_module(m, default=None)
¶
Get the device in which the module exists.
This function looks at the first parameter of the module, and returns its device. This function is not meant to be used on modules whose parameters exist on different devices.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
m |
Module |
The module whose device is being queried. |
required |
default |
Union[str, torch.device] |
The fallback device to return if the module has no parameters. If this is left as None, the fallback device is assumed to be "cpu". |
None |
Returns:
| Type | Description |
|---|---|
device |
The device of the module, determined from its first parameter. |
Source code in evotorch/neuroevolution/net/misc.py
def device_of_module(m: nn.Module, default: Optional[Union[str, torch.device]] = None) -> torch.device:
"""
Get the device in which the module exists.
This function looks at the first parameter of the module, and returns
its device. This function is not meant to be used on modules whose
parameters exist on different devices.
Args:
m: The module whose device is being queried.
default: The fallback device to return if the module has no
parameters. If this is left as None, the fallback device
is assumed to be "cpu".
Returns:
The device of the module, determined from its first parameter.
"""
if default is None:
default = torch.device("cpu")
device = default
for p in m.parameters():
device = p.device
break
return device
fill_parameters(net, vector)
¶
Fill the parameters of a torch module (net) from a vector.
No gradient information is kept.
The vector's length must be exactly the same with the number of parameters of the PyTorch module.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
net |
Module |
The torch module whose parameter values will be filled. |
required |
vector |
Tensor |
A 1-D torch tensor which stores the parameter values. |
required |
Source code in evotorch/neuroevolution/net/misc.py
@torch.no_grad()
def fill_parameters(net: nn.Module, vector: torch.Tensor):
"""Fill the parameters of a torch module (net) from a vector.
No gradient information is kept.
The vector's length must be exactly the same with the number
of parameters of the PyTorch module.
Args:
net: The torch module whose parameter values will be filled.
vector: A 1-D torch tensor which stores the parameter values.
"""
address = 0
for p in net.parameters():
d = p.data.view(-1)
n = len(d)
d[:] = torch.as_tensor(vector[address : address + n], device=d.device)
address += n
if address != len(vector):
raise IndexError("The parameter vector is larger than expected")
parameter_vector(net, *, device=None)
¶
Get all the parameters of a torch module (net) into a vector
No gradient information is kept.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
net |
Module |
The torch module whose parameters will be extracted. |
required |
device |
Union[str, torch.device] |
The device in which the parameter vector will be constructed. If the network has parameter across multiple devices, you can specify this argument so that concatenation of all the parameters will be successful. |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The parameters of the module in a 1-D tensor. |
Source code in evotorch/neuroevolution/net/misc.py
@torch.no_grad()
def parameter_vector(net: nn.Module, *, device: Optional[Device] = None) -> torch.Tensor:
"""Get all the parameters of a torch module (net) into a vector
No gradient information is kept.
Args:
net: The torch module whose parameters will be extracted.
device: The device in which the parameter vector will be constructed.
If the network has parameter across multiple devices,
you can specify this argument so that concatenation of all the
parameters will be successful.
Returns:
The parameters of the module in a 1-D tensor.
"""
dev_kwarg = {} if device is None else {"device": device}
all_vectors = []
for p in net.parameters():
all_vectors.append(torch.as_tensor(p.data.view(-1), **dev_kwarg))
return torch.cat(all_vectors)
multilayered
¶
MultiLayered (Module)
¶
Source code in evotorch/neuroevolution/net/multilayered.py
class MultiLayered(nn.Module):
def __init__(self, *layers: nn.Module):
super().__init__()
self._submodules = nn.ModuleList(layers)
def forward(self, x: torch.Tensor, h: Optional[dict] = None):
if h is None:
h = {}
new_h = {}
for i, layer in enumerate(self._submodules):
layer_h = h.get(i, None)
if layer_h is None:
layer_result = layer(x)
else:
layer_result = layer(x, h[i])
if isinstance(layer_result, tuple):
if len(layer_result) == 2:
x, layer_new_h = layer_result
else:
raise ValueError(
f"The layer number {i} returned a tuple of length {len(layer_result)}."
f" A tensor or a tuple of two elements was expected."
)
elif isinstance(layer_result, torch.Tensor):
x = layer_result
layer_new_h = None
else:
raise TypeError(
f"The layer number {i} returned an object of type {type(layer_result)}."
f" A tensor or a tuple of two elements was expected."
)
if layer_new_h is not None:
new_h[i] = layer_new_h
if len(new_h) == 0:
return x
else:
return x, new_h
def __iter__(self):
return self._submodules.__iter__()
def __getitem__(self, i):
return self._submodules[i]
def __len__(self):
return len(self._submodules)
def append(self, module: nn.Module):
self._submodules.append(module)
forward(self, x, h=None)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
Source code in evotorch/neuroevolution/net/multilayered.py
def forward(self, x: torch.Tensor, h: Optional[dict] = None):
if h is None:
h = {}
new_h = {}
for i, layer in enumerate(self._submodules):
layer_h = h.get(i, None)
if layer_h is None:
layer_result = layer(x)
else:
layer_result = layer(x, h[i])
if isinstance(layer_result, tuple):
if len(layer_result) == 2:
x, layer_new_h = layer_result
else:
raise ValueError(
f"The layer number {i} returned a tuple of length {len(layer_result)}."
f" A tensor or a tuple of two elements was expected."
)
elif isinstance(layer_result, torch.Tensor):
x = layer_result
layer_new_h = None
else:
raise TypeError(
f"The layer number {i} returned an object of type {type(layer_result)}."
f" A tensor or a tuple of two elements was expected."
)
if layer_new_h is not None:
new_h[i] = layer_new_h
if len(new_h) == 0:
return x
else:
return x, new_h
parser
¶
Utilities for parsing string representations of neural net policies
NetParsingError (Exception)
¶
Representation of a parsing error
Source code in evotorch/neuroevolution/net/parser.py
class NetParsingError(Exception):
"""
Representation of a parsing error
"""
def __init__(
self,
message: str,
lineno: Optional[int] = None,
col_offset: Optional[int] = None,
original_error: Optional[Exception] = None,
):
"""
`__init__(...)`: Initialize the NetParsingError.
Args:
message: Error message, as string.
lineno: Erroneous line number in the string representation of the
neural network structure.
col_offset: Erroneous column number in the string representation
of the neural network structure.
original_error: If another error caused this parsing error,
that original error can be attached to this `NetParsingError`
instance via this argument.
"""
super().__init__()
self.message = message
self.lineno = lineno
self.col_offset = col_offset
self.original_error = original_error
def _to_string(self) -> str:
parts = []
parts.append(type(self).__name__)
if self.lineno is not None:
parts.append(" at line(")
parts.append(str(self.lineno - 1))
parts.append(")")
if self.col_offset is not None:
parts.append(" at column(")
parts.append(str(self.col_offset + 1))
parts.append(")")
parts.append(": ")
parts.append(self.message)
return "".join(parts)
def __str__(self) -> str:
return self._to_string()
def __repr__(self) -> str:
return self._to_string()
__init__(self, message, lineno=None, col_offset=None, original_error=None)
special
¶
__init__(...): Initialize the NetParsingError.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
message |
str |
Error message, as string. |
required |
lineno |
Optional[int] |
Erroneous line number in the string representation of the neural network structure. |
None |
col_offset |
Optional[int] |
Erroneous column number in the string representation of the neural network structure. |
None |
original_error |
Optional[Exception] |
If another error caused this parsing error,
that original error can be attached to this |
None |
Source code in evotorch/neuroevolution/net/parser.py
def __init__(
self,
message: str,
lineno: Optional[int] = None,
col_offset: Optional[int] = None,
original_error: Optional[Exception] = None,
):
"""
`__init__(...)`: Initialize the NetParsingError.
Args:
message: Error message, as string.
lineno: Erroneous line number in the string representation of the
neural network structure.
col_offset: Erroneous column number in the string representation
of the neural network structure.
original_error: If another error caused this parsing error,
that original error can be attached to this `NetParsingError`
instance via this argument.
"""
super().__init__()
self.message = message
self.lineno = lineno
self.col_offset = col_offset
self.original_error = original_error
str_to_net(s, **constants)
¶
Read a string representation of a neural net structure,
and return a torch.nn.Module instance out of it.
Let us imagine that one wants to describe the following neural network structure:
from torch import nn
from evotorch.neuroevolution.net import MultiLayered
net = MultiLayered(nn.Linear(8, 16), nn.Tanh(), nn.Linear(16, 4, bias=False), nn.ReLU())
By using str_to_net(...) one can construct an equivalent
module via:
from evotorch.neuroevolution.net import str_to_net
net = str_to_net("Linear(8, 16) >> Tanh() >> Linear(16, 4, bias=False) >> ReLU()")
The string can also be multi-line:
One can also define constants for using them in strings:
net = str_to_net(
'''
Linear(input_size, hidden_size)
>> Tanh()
>> Linear(hidden_size, output_size, bias=False)
>> ReLU()
''',
input_size=8,
hidden_size=16,
output_size=4,
)
In the neural net structure string, when one refers to a module type,
say, Linear, first the name Linear is searched for in the namespace
evotorch.neuroevolution.net.layers, and then in the namespace torch.nn.
In the case of Linear, the searched name exists in torch.nn,
and therefore, the layer type to be instantiated is accepted as
torch.nn.Linear.
Instead of Linear, if one had used the name, say,
StructuredControlNet, then, the layer type to be instantiated
would be evotorch.neuroevolution.net.layers.StructuredControlNet.
The namespace evotorch.neuroevolution.net.layers contains its own
implementations for RNN and LSTM. These recurrent layer implementations
work similarly to their counterparts torch.nn.RNN and torch.nn.LSTM,
except that EvoTorch's implementations do not expect the data with extra
leftmost dimensions for batching and for timesteps. Instead, they expect
to receive a single input and a single current hidden state, and produce
a single output and a single new hidden state. These recurrent layer
implementations of EvoTorch can be used within a neural net structure
string. Therefore, the following examples are valid:
rnn1 = str_to_net("RNN(4, 8) >> Linear(8, 2)")
rnn2 = str_to_net(
'''
Linear(4, 10)
>> Tanh()
>> RNN(input_size=10, hidden_size=24, nonlinearity='tanh'
>> Linear(24, 2)
'''
)
lstm1 = str_to_net("LSTM(4, 32) >> Linear(32, 2)")
lstm2 = str_to_net("LSTM(input_size=4, hidden_size=32) >> Linear(32, 2)")
Notes regarding usage with evotorch.neuroevolution.GymNE
or with evotorch.neuroevolution.VecGymNE:
While instantiating a GymNE or a VecGymNE, one can specify a neural
net structure string as the policy. Therefore, while filling the policy
string for a GymNE, all these rules mentioned above apply. Additionally,
while using str_to_net(...) internally, GymNE and VecGymNE define
these extra constants:
obs_length (length of the observation vector),
act_length (length of the action vector for continuous-action
environments, or number of actions for discrete-action
environments), and
obs_shape (shape of the observation as a tuple, assuming that the
observation space is of type gym.spaces.Box, usable within the string
like obs_shape[0], obs_shape[1], etc., or simply obs_shape to refer
to the entire tuple).
Therefore, while instantiating a GymNE or a VecGymNE, one can define a
single-hidden-layered policy via this string:
In the policy string above, one might choose to omit the last Tanh(), as
GymNE and VecGymNE will clip the final output of the policy to conform
to the action boundaries defined by the target reinforcement learning
environment, and such a clipping operation might be seen as using an
activation function similar to hard-tanh anyway.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
s |
str |
The string which expresses the neural net structure. |
required |
Returns:
| Type | Description |
|---|---|
Module |
The PyTorch module of the specified structure. |
Source code in evotorch/neuroevolution/net/parser.py
def str_to_net(s: str, **constants) -> nn.Module:
"""
Read a string representation of a neural net structure,
and return a `torch.nn.Module` instance out of it.
Let us imagine that one wants to describe the following
neural network structure:
```python
from torch import nn
from evotorch.neuroevolution.net import MultiLayered
net = MultiLayered(nn.Linear(8, 16), nn.Tanh(), nn.Linear(16, 4, bias=False), nn.ReLU())
```
By using `str_to_net(...)` one can construct an equivalent
module via:
```python
from evotorch.neuroevolution.net import str_to_net
net = str_to_net("Linear(8, 16) >> Tanh() >> Linear(16, 4, bias=False) >> ReLU()")
```
The string can also be multi-line:
```python
net = str_to_net(
'''
Linear(8, 16)
>> Tanh()
>> Linear(16, 4, bias=False)
>> ReLU()
'''
)
```
One can also define constants for using them in strings:
```python
net = str_to_net(
'''
Linear(input_size, hidden_size)
>> Tanh()
>> Linear(hidden_size, output_size, bias=False)
>> ReLU()
''',
input_size=8,
hidden_size=16,
output_size=4,
)
```
In the neural net structure string, when one refers to a module type,
say, `Linear`, first the name `Linear` is searched for in the namespace
`evotorch.neuroevolution.net.layers`, and then in the namespace `torch.nn`.
In the case of `Linear`, the searched name exists in `torch.nn`,
and therefore, the layer type to be instantiated is accepted as
`torch.nn.Linear`.
Instead of `Linear`, if one had used the name, say,
`StructuredControlNet`, then, the layer type to be instantiated
would be `evotorch.neuroevolution.net.layers.StructuredControlNet`.
The namespace `evotorch.neuroevolution.net.layers` contains its own
implementations for RNN and LSTM. These recurrent layer implementations
work similarly to their counterparts `torch.nn.RNN` and `torch.nn.LSTM`,
except that EvoTorch's implementations do not expect the data with extra
leftmost dimensions for batching and for timesteps. Instead, they expect
to receive a single input and a single current hidden state, and produce
a single output and a single new hidden state. These recurrent layer
implementations of EvoTorch can be used within a neural net structure
string. Therefore, the following examples are valid:
```python
rnn1 = str_to_net("RNN(4, 8) >> Linear(8, 2)")
rnn2 = str_to_net(
'''
Linear(4, 10)
>> Tanh()
>> RNN(input_size=10, hidden_size=24, nonlinearity='tanh'
>> Linear(24, 2)
'''
)
lstm1 = str_to_net("LSTM(4, 32) >> Linear(32, 2)")
lstm2 = str_to_net("LSTM(input_size=4, hidden_size=32) >> Linear(32, 2)")
```
**Notes regarding usage with `evotorch.neuroevolution.GymNE`
or with `evotorch.neuroevolution.VecGymNE`:**
While instantiating a `GymNE` or a `VecGymNE`, one can specify a neural
net structure string as the policy. Therefore, while filling the policy
string for a `GymNE`, all these rules mentioned above apply. Additionally,
while using `str_to_net(...)` internally, `GymNE` and `VecGymNE` define
these extra constants:
`obs_length` (length of the observation vector),
`act_length` (length of the action vector for continuous-action
environments, or number of actions for discrete-action
environments), and
`obs_shape` (shape of the observation as a tuple, assuming that the
observation space is of type `gym.spaces.Box`, usable within the string
like `obs_shape[0]`, `obs_shape[1]`, etc., or simply `obs_shape` to refer
to the entire tuple).
Therefore, while instantiating a `GymNE` or a `VecGymNE`, one can define a
single-hidden-layered policy via this string:
```
"Linear(obs_length, 16) >> Tanh() >> Linear(16, act_length) >> Tanh()"
```
In the policy string above, one might choose to omit the last `Tanh()`, as
`GymNE` and `VecGymNE` will clip the final output of the policy to conform
to the action boundaries defined by the target reinforcement learning
environment, and such a clipping operation might be seen as using an
activation function similar to hard-tanh anyway.
Args:
s: The string which expresses the neural net structure.
Returns:
The PyTorch module of the specified structure.
"""
s = f"(\n{s}\n)"
return _process_expr(ast.parse(s, mode="eval").body, constants=constants)
rl
¶
This namespace provides various reinforcement learning utilities.
ActClipWrapperModule (Module)
¶
Source code in evotorch/neuroevolution/net/rl.py
class ActClipWrapperModule(nn.Module):
def __init__(self, wrapped_module: nn.Module, obs_space: Box):
super().__init__()
device = device_of_module(wrapped_module)
if not isinstance(obs_space, Box):
raise TypeError(f"Unrecognized observation space: {obs_space}")
self.wrapped_module = wrapped_module
self.register_buffer("_low", torch.from_numpy(obs_space.low).to(device))
self.register_buffer("_high", torch.from_numpy(obs_space.high).to(device))
def forward(self, x: torch.Tensor, h: Any = None) -> Union[torch.Tensor, tuple]:
if h is None:
result = self.wrapped_module(x)
else:
result = self.wrapped_module(x, h)
if isinstance(result, tuple):
x, h = result
got_h = True
else:
x = result
h = None
got_h = False
x = torch.max(x, self._low)
x = torch.min(x, self._high)
if got_h:
return x, h
else:
return x
forward(self, x, h=None)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
Source code in evotorch/neuroevolution/net/rl.py
def forward(self, x: torch.Tensor, h: Any = None) -> Union[torch.Tensor, tuple]:
if h is None:
result = self.wrapped_module(x)
else:
result = self.wrapped_module(x, h)
if isinstance(result, tuple):
x, h = result
got_h = True
else:
x = result
h = None
got_h = False
x = torch.max(x, self._low)
x = torch.min(x, self._high)
if got_h:
return x, h
else:
return x
AliveBonusScheduleWrapper (Wrapper)
¶
A Wrapper which awards the agent for being alive in a scheduled manner This wrapper is meant to be used for non-vectorized environments.
Source code in evotorch/neuroevolution/net/rl.py
class AliveBonusScheduleWrapper(gym.Wrapper):
"""
A Wrapper which awards the agent for being alive in a scheduled manner
This wrapper is meant to be used for non-vectorized environments.
"""
def __init__(self, env: gym.Env, alive_bonus_schedule: tuple, **kwargs):
"""
`__init__(...)`: Initialize the AliveBonusScheduleWrapper.
Args:
env: Environment to wrap.
alive_bonus_schedule: If given as a tuple `(t, b)`, an alive
bonus `b` will be added onto all the rewards beyond the
timestep `t`.
If given as a tuple `(t0, t1, b)`, a partial (linearly
increasing towards `b`) alive bonus will be added onto
all the rewards between the timesteps `t0` and `t1`,
and a full alive bonus (which equals to `b`) will be added
onto all the rewards beyond the timestep `t1`.
kwargs: Expected in the form of additional keyword arguments,
these will be passed to the initialization method of the
superclass.
"""
super().__init__(env, **kwargs)
self.__t: Optional[int] = None
if len(alive_bonus_schedule) == 3:
self.__t0, self.__t1, self.__bonus = (
int(alive_bonus_schedule[0]),
int(alive_bonus_schedule[1]),
float(alive_bonus_schedule[2]),
)
elif len(alive_bonus_schedule) == 2:
self.__t0, self.__t1, self.__bonus = (
int(alive_bonus_schedule[0]),
int(alive_bonus_schedule[0]),
float(alive_bonus_schedule[1]),
)
else:
raise ValueError(
f"The argument `alive_bonus_schedule` was expected to have 2 or 3 elements."
f" However, its value is {repr(alive_bonus_schedule)} (having {len(alive_bonus_schedule)} elements)."
)
if self.__t1 > self.__t0:
self.__gap = self.__t1 - self.__t0
else:
self.__gap = None
def reset(self, *args, **kwargs):
self.__t = 0
return self.env.reset(*args, **kwargs)
def step(self, action) -> tuple:
step_result = self.env.step(action)
self.__t += 1
observation = step_result[0]
reward = step_result[1]
rest = step_result[2:]
if self.__t >= self.__t1:
reward = reward + self.__bonus
elif (self.__gap is not None) and (self.__t >= self.__t0):
reward = reward + ((self.__t - self.__t0) / self.__gap) * self.__bonus
return (observation, reward) + rest
__init__(self, env, alive_bonus_schedule, **kwargs)
special
¶
__init__(...): Initialize the AliveBonusScheduleWrapper.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
env |
Env |
Environment to wrap. |
required |
alive_bonus_schedule |
tuple |
If given as a tuple |
required |
kwargs |
Expected in the form of additional keyword arguments, these will be passed to the initialization method of the superclass. |
{} |
Source code in evotorch/neuroevolution/net/rl.py
def __init__(self, env: gym.Env, alive_bonus_schedule: tuple, **kwargs):
"""
`__init__(...)`: Initialize the AliveBonusScheduleWrapper.
Args:
env: Environment to wrap.
alive_bonus_schedule: If given as a tuple `(t, b)`, an alive
bonus `b` will be added onto all the rewards beyond the
timestep `t`.
If given as a tuple `(t0, t1, b)`, a partial (linearly
increasing towards `b`) alive bonus will be added onto
all the rewards between the timesteps `t0` and `t1`,
and a full alive bonus (which equals to `b`) will be added
onto all the rewards beyond the timestep `t1`.
kwargs: Expected in the form of additional keyword arguments,
these will be passed to the initialization method of the
superclass.
"""
super().__init__(env, **kwargs)
self.__t: Optional[int] = None
if len(alive_bonus_schedule) == 3:
self.__t0, self.__t1, self.__bonus = (
int(alive_bonus_schedule[0]),
int(alive_bonus_schedule[1]),
float(alive_bonus_schedule[2]),
)
elif len(alive_bonus_schedule) == 2:
self.__t0, self.__t1, self.__bonus = (
int(alive_bonus_schedule[0]),
int(alive_bonus_schedule[0]),
float(alive_bonus_schedule[1]),
)
else:
raise ValueError(
f"The argument `alive_bonus_schedule` was expected to have 2 or 3 elements."
f" However, its value is {repr(alive_bonus_schedule)} (having {len(alive_bonus_schedule)} elements)."
)
if self.__t1 > self.__t0:
self.__gap = self.__t1 - self.__t0
else:
self.__gap = None
reset(self, *args, **kwargs)
¶
step(self, action)
¶
Uses the :meth:step of the :attr:env that can be overwritten to change the returned data.
Source code in evotorch/neuroevolution/net/rl.py
def step(self, action) -> tuple:
step_result = self.env.step(action)
self.__t += 1
observation = step_result[0]
reward = step_result[1]
rest = step_result[2:]
if self.__t >= self.__t1:
reward = reward + self.__bonus
elif (self.__gap is not None) and (self.__t >= self.__t0):
reward = reward + ((self.__t - self.__t0) / self.__gap) * self.__bonus
return (observation, reward) + rest
ObsNormWrapperModule (Module)
¶
Source code in evotorch/neuroevolution/net/rl.py
class ObsNormWrapperModule(nn.Module):
def __init__(self, wrapped_module: nn.Module, rn: Union[RunningStat, RunningNorm]):
super().__init__()
device = device_of_module(wrapped_module)
self.wrapped_module = wrapped_module
with torch.no_grad():
normalizer = deepcopy(rn.to_layer()).to(device)
self.normalizer = normalizer
def forward(self, x: torch.Tensor, h: Any = None) -> Union[torch.Tensor, tuple]:
x = self.normalizer(x)
if h is None:
result = self.wrapped_module(x)
else:
result = self.wrapped_module(x, h)
if isinstance(result, tuple):
x, h = result
got_h = True
else:
x = result
h = None
got_h = False
if got_h:
return x, h
else:
return x
forward(self, x, h=None)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
Source code in evotorch/neuroevolution/net/rl.py
def forward(self, x: torch.Tensor, h: Any = None) -> Union[torch.Tensor, tuple]:
x = self.normalizer(x)
if h is None:
result = self.wrapped_module(x)
else:
result = self.wrapped_module(x, h)
if isinstance(result, tuple):
x, h = result
got_h = True
else:
x = result
h = None
got_h = False
if got_h:
return x, h
else:
return x
reset_env(env)
¶
Reset a gymnasium environment.
Even though the gymnasium library switched to a new API where the
reset() method returns a tuple (observation, info), this function
follows the conventions of the classical gym library and returns
only the observation of the newly reset environment.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
env |
Env |
The gymnasium environment which will be reset. |
required |
Returns:
| Type | Description |
|---|---|
Iterable |
The initial observation |
Source code in evotorch/neuroevolution/net/rl.py
def reset_env(env: gym.Env) -> Iterable:
"""
Reset a gymnasium environment.
Even though the `gymnasium` library switched to a new API where the
`reset()` method returns a tuple `(observation, info)`, this function
follows the conventions of the classical `gym` library and returns
only the observation of the newly reset environment.
Args:
env: The gymnasium environment which will be reset.
Returns:
The initial observation
"""
result = env.reset()
if isinstance(result, tuple) and (len(result) == 2):
result = result[0]
return result
take_step_in_env(env, action)
¶
Take a step in the gymnasium environment. Taking a step means performing the action provided via the arguments.
Even though the gymnasium library switched to a new API where the
step() method returns a 5-element tuple of the form
(observation, reward, terminated, truncated, info), this function
follows the conventions of the classical gym library and returns
a 4-element tuple (observation, reward, done, info).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
env |
Env |
The gymnasium environment in which the action will be performed. |
required |
action |
Iterable |
The action to be performed. |
required |
Returns:
| Type | Description |
|---|---|
tuple |
A tuple in the form |
Source code in evotorch/neuroevolution/net/rl.py
def take_step_in_env(env: gym.Env, action: Iterable) -> tuple:
"""
Take a step in the gymnasium environment.
Taking a step means performing the action provided via the arguments.
Even though the `gymnasium` library switched to a new API where the
`step()` method returns a 5-element tuple of the form
`(observation, reward, terminated, truncated, info)`, this function
follows the conventions of the classical `gym` library and returns
a 4-element tuple `(observation, reward, done, info)`.
Args:
env: The gymnasium environment in which the action will be performed.
action: The action to be performed.
Returns:
A tuple in the form `(observation, reward, done, info)` where
`observation` is the observation received after performing the action,
`reward` is the amount of reward gained,
`done` is a boolean value indicating whether or not the episode has
ended, and
`info` is additional information (usually as a dictionary).
"""
result = env.step(action)
if isinstance(result, tuple):
n = len(result)
if n == 4:
observation, reward, done, info = result
elif n == 5:
observation, reward, terminated, truncated, info = result
done = terminated or truncated
else:
raise ValueError(
f"The result of the `step(...)` method of the gym environment"
f" was expected as a tuple of length 4 or 5."
f" However, the received result is {repr(result)}, which is"
f" of length {len(result)}."
)
else:
raise TypeError(
f"The result of the `step(...)` method of the gym environment"
f" was expected as a tuple of length 4 or 5."
f" However, the received result is {repr(result)}, which is"
f" of type {type(result)}."
)
return observation, reward, done, info
runningnorm
¶
CollectedStats (tuple)
¶
ObsNormLayer (Module)
¶
An observation normalizer which behaves as a PyTorch Module.
Source code in evotorch/neuroevolution/net/runningnorm.py
class ObsNormLayer(nn.Module):
"""
An observation normalizer which behaves as a PyTorch Module.
"""
def __init__(
self, mean: torch.Tensor, stdev: torch.Tensor, low: Optional[float] = None, high: Optional[float] = None
) -> None:
"""
`__init__(...)`: Initialize the ObsNormLayer.
Args:
mean: The mean according to which the observations are to be
normalized.
stdev: The standard deviation according to which the observations
are to be normalized.
low: Optionally a real number if the result of the normalization
is to be clipped. Represents the lower bound for the clipping
operation.
high: Optionally a real number if the result of the normalization
is to be clipped. Represents the upper bound for the clipping
operation.
"""
super().__init__()
self.register_buffer("_mean", mean)
self.register_buffer("_stdev", stdev)
self._lb = None if low is None else float(low)
self._ub = None if high is None else float(high)
def forward(self, x: torch.Tensor) -> torch.Tensor:
"""
Normalize an observation or a batch of observations.
Args:
x: The observation(s).
Returns:
The normalized counterpart of the observation(s).
"""
return _clamp((x - self._mean) / self._stdev, self._lb, self._ub)
__init__(self, mean, stdev, low=None, high=None)
special
¶
__init__(...): Initialize the ObsNormLayer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
mean |
Tensor |
The mean according to which the observations are to be normalized. |
required |
stdev |
Tensor |
The standard deviation according to which the observations are to be normalized. |
required |
low |
Optional[float] |
Optionally a real number if the result of the normalization is to be clipped. Represents the lower bound for the clipping operation. |
None |
high |
Optional[float] |
Optionally a real number if the result of the normalization is to be clipped. Represents the upper bound for the clipping operation. |
None |
Source code in evotorch/neuroevolution/net/runningnorm.py
def __init__(
self, mean: torch.Tensor, stdev: torch.Tensor, low: Optional[float] = None, high: Optional[float] = None
) -> None:
"""
`__init__(...)`: Initialize the ObsNormLayer.
Args:
mean: The mean according to which the observations are to be
normalized.
stdev: The standard deviation according to which the observations
are to be normalized.
low: Optionally a real number if the result of the normalization
is to be clipped. Represents the lower bound for the clipping
operation.
high: Optionally a real number if the result of the normalization
is to be clipped. Represents the upper bound for the clipping
operation.
"""
super().__init__()
self.register_buffer("_mean", mean)
self.register_buffer("_stdev", stdev)
self._lb = None if low is None else float(low)
self._ub = None if high is None else float(high)
forward(self, x)
¶
Normalize an observation or a batch of observations.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Tensor |
The observation(s). |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The normalized counterpart of the observation(s). |
Source code in evotorch/neuroevolution/net/runningnorm.py
RunningNorm
¶
An online observation normalization tool
Source code in evotorch/neuroevolution/net/runningnorm.py
class RunningNorm:
"""
An online observation normalization tool
"""
def __init__(
self,
*,
shape: Union[tuple, int],
dtype: DType,
device: Optional[Device] = None,
min_variance: float = 1e-2,
clip: Optional[tuple] = None,
) -> None:
"""
`__init__(...)`: Initialize the RunningNorm
Args:
shape: Observation shape. Can be an integer or a tuple.
dtype: The dtype of the observations.
device: The device in which the observation stats are held.
If left as None, the device is assumed to be "cpu".
min_variance: A lower bound for the variance to be used in
the normalization computations.
In other words, if the computed variance according to the
collected observations ends up lower than `min_variance`,
this `min_variance` will be used instead (in an elementwise
manner) while computing the normalized observations.
As in Salimans et al. (2017), the default is 1e-2.
clip: Can be left as None (which is the default), or can be
given as a pair of real numbers.
This is used for clipping the observations after the
normalization operation.
In Salimans et al. (2017), (-5.0, +5.0) was used.
"""
# Make sure that the shape is stored as a torch.Size object.
if isinstance(shape, Iterable):
self._shape = torch.Size(shape)
else:
self._shape = torch.Size([int(shape)])
# Store the number of dimensions
self._ndim = len(self._shape)
# Store the dtype and the device
self._dtype = to_torch_dtype(dtype)
self._device = "cpu" if device is None else device
# Initialize the internally stored data as empty
self._sum: Optional[torch.Tensor] = None
self._sum_of_squares: Optional[torch.Tensor] = None
self._count: int = 0
# Store the minimum variance
self._min_variance = float(min_variance)
if clip is not None:
# If a clip tuple was provided, store the specified lower and upper bounds
lb, ub = clip
self._lb = float(lb)
self._ub = float(ub)
else:
# If a clip tuple was not provided the bounds are stored as None
self._lb = None
self._ub = None
def to(self, device: Device) -> "RunningNorm":
"""
If the target device is a different device, then make a copy of this
RunningNorm instance on the target device.
If the target device is the same with this RunningNorm's device, then
return this RunningNorm itself.
Args:
device: The target device.
Returns:
The RunningNorm on the target device. This can be a copy, or the
original RunningNorm instance itself.
"""
if torch.device(device) == torch.device(self.device):
return self
else:
new_running_norm = object.__new__(type(self))
already_handled = {"_sum", "_sum_of_squares", "_device"}
new_running_norm._sum = self._sum.to(device)
new_running_norm._sum_of_squares = self._sum_of_squares.to(device)
new_running_norm._device = device
for k, v in self.__dict__.items():
if k not in already_handled:
setattr(new_running_norm, k, deepcopy(v))
return new_running_norm
@property
def device(self) -> Device:
"""
The device in which the observation stats are held
"""
return self._device
@property
def dtype(self) -> DType:
"""
The dtype of the stored observation stats
"""
return self._dtype
@property
def shape(self) -> tuple:
"""
Observation shape
"""
return self._shape
@property
def min_variance(self) -> float:
"""
Minimum variance
"""
return self._min_variance
@property
def low(self) -> Optional[float]:
"""
The lower component of the bounds given in the `clip` tuple.
If `clip` was initialized as None, this is also None.
"""
return self._lb
@property
def high(self) -> Optional[float]:
"""
The higher (upper) component of the bounds given in the `clip` tuple.
If `clip` was initialized as None, this is also None.
"""
return self._ub
def _like_its_own(self, x: Iterable) -> torch.Tensor:
return torch.as_tensor(x, dtype=self._dtype, device=self._device)
def _verify(self, x: Iterable) -> torch.Tensor:
x = self._like_its_own(x)
if x.ndim == self._ndim:
if x.shape != self._shape:
raise ValueError(
f"This RunningNorm instance was initialized with shape: {self._shape}."
f" However, the provided tensor has an incompatible shape: {x._shape}."
)
elif x.ndim == (self._ndim + 1):
if x.shape[1:] != self._shape:
raise ValueError(
f"This RunningNorm instance was initialized with shape: {self._shape}."
f" The provided tensor is shaped {x.shape}."
f" Accepting the tensor's leftmost dimension as the batch size,"
f" the remaining shape is incompatible: {x.shape[1:]}"
)
else:
raise ValueError(
f"This RunningNorm instance was initialized with shape: {self._shape}."
f" The provided tensor is shaped {x.shape}."
f" The number of dimensions of the given tensor is incompatible."
)
return x
def _has_no_data(self) -> bool:
return (self._sum is None) and (self._sum_of_squares is None) and (self._count == 0)
def _has_data(self) -> bool:
return (self._sum is not None) and (self._sum_of_squares is not None) and (self._count > 0)
def reset(self):
"""
Remove all the collected observation data.
"""
self._sum = None
self._sum_of_squares = None
self._count = 0
@torch.no_grad()
def update(self, x: Union[Iterable, "RunningNorm"], mask: Optional[Iterable] = None, *, verify: bool = True):
"""
Update the stored stats with new observation data.
Args:
x: The new observation(s), as a PyTorch tensor, or any Iterable
that can be converted to a PyTorch tensor, or another
RunningNorm instance.
If given as a tensor or as an Iterable, the shape of `x` can
be the same with observation shape, or it can be augmented
with an extra leftmost dimension.
In the case of augmented dimension, `x` is interpreted not as
a single observation, but as a batch of observations.
If `x` is another RunningNorm instance, the stats stored by
this RunningNorm instance will be updated with all the data
stored by `x`.
mask: Can be given as a 1-dimensional Iterable of booleans ONLY
if `x` represents a batch of observations.
If a `mask` is provided, the i-th observation within the
observation batch `x` will be taken into account only if
the i-th item of the `mask` is True.
verify: Whether or not to verify the shape of the given Iterable
objects. The default is True.
"""
if isinstance(x, RunningNorm):
# If we are to update our stats according to another RunningNorm instance
if x._count > 0:
# We bother only if x is non-empty
if mask is not None:
# We were given another RunningNorm, not a batch of observations.
# So, we do not expect to receive a mask tensor.
# If a mask was provided, then this is an unexpected way of calling this function.
# We therefore raise an error.
raise ValueError(
"The `mask` argument is expected as None if the first argument is a RunningNorm."
" However, `mask` is found as something other than None."
)
if self._shape != x._shape:
# If the shapes of this RunningNorm and of the other RunningNorm
# do not match, then we cannot use `x` for updating our stats.
# It might be the case that `x` was initialized for another
# task, with differently sized observations.
# We therefore raise an error.
raise ValueError(
f"The RunningNorm to be updated has the shape {self._shape}"
f" The other RunningNorm has the shape {self._shape}"
f" These shapes are incompatible."
)
if self._has_no_data():
# If this RunningNorm has no data at all, then we clone the
# data of x.
self._sum = self._like_its_own(x._sum.clone())
self._sum_of_squares = self._like_its_own(x._sum_of_squares.clone())
self._count = x._count
elif self._has_data():
# If this RunningNorm has its own data, then we update the
# stored data with the data stored by x.
self._sum += self._like_its_own(x._sum)
self._sum_of_squares += self._like_its_own(x._sum_of_squares)
self._count += x._count
else:
assert False, "RunningNorm is in an invalid state! This might be a bug."
else:
# This is the case where the received argument x is not a
# RunningNorm object, but an Iterable.
if verify:
# If we have the `verify` flag, then we make sure that
# x is a tensor of the correct shape
x = self._verify(x)
if x.ndim == self._ndim:
# If the shape of x is exactly the same with the observation shape
# then we assume that x represents a single observation, and not a
# batch of observations.
if mask is not None:
# Since we are dealing with a single observation,
# we do not expect to receive a mask argument.
# If the mask argument was provided, then this is an unexpected
# usage of this function.
# We therefore raise an error.
raise ValueError(
"The `mask` argument is expected as None if the first argument is a single observation"
" (i.e. not a batch of observations, with an extra leftmost dimension)."
" However, `mask` is found as something other than None."
)
# Since x is a single observation,
# the sum of observations extracted from x is x itself,
# and the sum of squared observations extracted from x is
# the square of x itself.
sum_of_x = x
sum_of_x_squared = x.square()
# We extracted a single observation from x
n = 1
elif x.ndim == (self._ndim + 1):
# If the number of dimensions of x is one more than the number
# of dimensions of this RunningNorm, then we assume that x is a batch
# of observations.
if mask is not None:
# If a mask is provided, then we first make sure that it is a tensor
# of dtype bool in the correct device.
mask = torch.as_tensor(mask, dtype=torch.bool, device=self._device)
if mask.ndim != 1:
# We expect the mask to be 1-dimensional.
# If not, we raise an error.
raise ValueError(
f"The `mask` tensor was expected as a 1-dimensional tensor."
f" However, its shape is {mask.shape}."
)
if len(mask) != x.shape[0]:
# If the length of the mask is not the batch size of x,
# then there is a mismatch.
# We therefore raise an error.
raise ValueError(
f"The shape of the given tensor is {x.shape}."
f" Therefore, the batch size of observations is {x.shape[0]}."
f" However, the given `mask` tensor does not has an incompatible length: {len(mask)}."
)
# We compute how many True items we have in the mask.
# This integer gives us how many observations we extract from x.
n = int(torch.sum(torch.as_tensor(mask, dtype=torch.int64, device=self._device)))
# We now re-cast the mask as the observation dtype (so that True items turn to 1.0
# and False items turn to 0.0), and then increase its number of dimensions so that
# it can operate directly with x.
mask = self._like_its_own(mask).reshape(torch.Size([x.shape[0]] + ([1] * (x.ndim - 1))))
# Finally, we multiply x with the mask. This means that the observations with corresponding
# mask values as False are zeroed out.
x = x * mask
else:
# This is the case where we did not receive a mask.
# We can simply say that the number of observations to extract from x
# is the size of its leftmost dimension, i.e. the batch size.
n = x.shape[0]
# With or without a mask, we are now ready to extract the sum and sum of squares
# from x.
sum_of_x = torch.sum(x, dim=0)
sum_of_x_squared = torch.sum(x.square(), dim=0)
else:
# This is the case where the number of dimensions of x is unrecognized.
# This case is actually already checked by the _verify(...) method earlier.
# This defensive fallback case is only for when verify=False and it turned out
# that the ndim is invalid.
raise ValueError(f"Invalid shape: {x.shape}")
# At this point, we handled all the valid cases regarding the Iterable x,
# and we have our sum_of_x (sum of all observations), sum_of_squares
# (sum of all squared observations), and n (number of observations extracted
# from x).
if self._has_no_data():
# If our RunningNorm is empty, the observation data we extracted from x
# become our RunningNorm's new data.
self._sum = sum_of_x
self._sum_of_squares = sum_of_x_squared
self._count = n
elif self._has_data():
# If our RunningNorm is not empty, the stored data is updated with the
# data extracted from x.
self._sum += sum_of_x
self._sum_of_squares += sum_of_x_squared
self._count += n
else:
# This is an erroneous state where the internal data looks neither
# existent nor completely empty.
# This might be the result of a bug, or maybe this instance's
# protected variables were tempered with from the outside.
assert False, "RunningNorm is in an invalid state! This might be a bug."
@property
@torch.no_grad()
def stats(self) -> CollectedStats:
"""
The collected data's mean and standard deviation (stdev) in a tuple
"""
# Using the internally stored sum, sum_of_squares, and count,
# compute E[x] and E[x^2]
E_x = self._sum / self._count
E_x2 = self._sum_of_squares / self._count
# The mean is E[x]
mean = E_x
# The variance is E[x^2] - (E[x])^2, elementwise clipped such that
# it cannot go below min_variance
variance = _clamp(E_x2 - E_x.square(), self._min_variance, None)
# Standard deviation is finally computed as the square root of the variance
stdev = torch.sqrt(variance)
# Return the stats in a named tuple
return CollectedStats(mean=mean, stdev=stdev)
@property
def mean(self) -> torch.Tensor:
"""
The collected data's mean
"""
return self._sum / self._count
@property
def stdev(self) -> torch.Tensor:
"""
The collected data's standard deviation
"""
return self.stats.stdev
@property
def sum(self) -> torch.Tensor:
"""
The collected data's sum
"""
return self._sum
@property
def sum_of_squares(self) -> torch.Tensor:
"""
Sum of squares of the collected data
"""
return self._sum_of_squares
@property
def count(self) -> int:
"""
Number of observations encountered
"""
return self._count
@torch.no_grad()
def normalize(self, x: Iterable, *, result_as_numpy: Optional[bool] = None, verify: bool = True) -> Iterable:
"""
Normalize the given observation x.
Args:
x: The observation(s), as a PyTorch tensor, or any Iterable
that is convertable to a PyTorch tensor.
`x` can be a single observation, or it can be a batch
of observations (with an extra leftmost dimension).
result_as_numpy: Whether or not to return the normalized
observation as a numpy array.
If left as None (which is the default), then the returned
type depends on x: a PyTorch tensor is returned if x is a
PyTorch tensor, and a numpy array is returned otherwise.
If True, the result is always a numpy array.
If False, the result is always a PyTorch tensor.
verify: Whether or not to check the type and dimensions of x.
This is True by default.
Note that, if `verify` is False, this function will not
properly check the type of `x` and will assume that `x`
is a PyTorch tensor.
Returns:
The normalized observation, as a PyTorch tensor or a numpy array.
"""
if self._count == 0:
# If this RunningNorm instance has no data yet,
# then we do not know how to do the normalization.
# We therefore raise an error.
raise ValueError("Cannot do normalization because no data is collected yet.")
if verify:
# Here we verify the type and shape of x.
if result_as_numpy is None:
# If there is not an explicit request about the return type,
# we infer the return type from the type of x:
# if x is a tensor, we return a tensor;
# otherwise, we assume x to be a CPU-bound iterable, and
# therefore we return a numpy array.
result_as_numpy = not isinstance(x, torch.Tensor)
else:
result_as_numpy = bool(result_as_numpy)
# We call _verify() to make sure that x is of correct shape
# and is properly converted to a PyTorch tensor.
x = self._verify(x)
# We get the mean and stdev of the collected data
mean, stdev = self.stats
# Now we compute the normalized observation, clipped according to the
# lower and upper bounds expressed by the `clip` tuple, if exists.
result = _clamp((x - mean) / stdev, self._lb, self._ub)
if result_as_numpy:
# If we are to return the result as a numpy array, we do the
# necessary conversion.
result = result.cpu().numpy()
# Finally, return the result
return result
@torch.no_grad()
def update_and_normalize(self, x: Iterable, mask: Optional[Iterable] = None) -> Iterable:
"""
Update the observation stats according to x, then normalize x.
Args:
x: The observation(s), as a PyTorch tensor, or as an Iterable
which can be converted to a PyTorch tensor.
The shape of x can be the same with the observaiton shape,
or it can be augmented with an extra leftmost dimension
to express a batch of observations.
mask: Can be given as a 1-dimensional Iterable of booleans ONLY
if `x` represents a batch of observations.
If a `mask` is provided, the i-th observation within the
observation batch `x` will be taken into account only if
the the i-th item of the `mask` is True.
Returns:
The normalized counterpart of the observation(s) expressed by x.
"""
result_as_numpy = not isinstance(x, torch.Tensor)
x = self._verify(x)
self.update(x, mask, verify=False)
result = self.normalize(x, verify=False)
if result_as_numpy:
result = result.cpu().numpy()
return result
def to_layer(self) -> "ObsNormLayer":
"""
Make a PyTorch module which normalizes the its inputs.
Returns:
An ObsNormLayer instance.
"""
mean, stdev = self.stats
low = self.low
high = self.high
return ObsNormLayer(mean=mean, stdev=stdev, low=low, high=high)
def __repr__(self) -> str:
return f"<{self.__class__.__name__}, count: {self.count}>"
def __copy__(self) -> "RunningNorm":
return deepcopy(self)
count: int
property
readonly
¶
Number of observations encountered
device: Union[str, torch.device]
property
readonly
¶
The device in which the observation stats are held
dtype: Union[str, torch.dtype, numpy.dtype, Type]
property
readonly
¶
The dtype of the stored observation stats
high: Optional[float]
property
readonly
¶
The higher (upper) component of the bounds given in the clip tuple.
If clip was initialized as None, this is also None.
low: Optional[float]
property
readonly
¶
The lower component of the bounds given in the clip tuple.
If clip was initialized as None, this is also None.
mean: Tensor
property
readonly
¶
The collected data's mean
min_variance: float
property
readonly
¶
Minimum variance
shape: tuple
property
readonly
¶
Observation shape
stats: CollectedStats
property
readonly
¶
The collected data's mean and standard deviation (stdev) in a tuple
stdev: Tensor
property
readonly
¶
The collected data's standard deviation
sum: Tensor
property
readonly
¶
The collected data's sum
sum_of_squares: Tensor
property
readonly
¶
Sum of squares of the collected data
__init__(self, *, shape, dtype, device=None, min_variance=0.01, clip=None)
special
¶
__init__(...): Initialize the RunningNorm
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
shape |
Union[tuple, int] |
Observation shape. Can be an integer or a tuple. |
required |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
The dtype of the observations. |
required |
device |
Union[str, torch.device] |
The device in which the observation stats are held. If left as None, the device is assumed to be "cpu". |
None |
min_variance |
float |
A lower bound for the variance to be used in
the normalization computations.
In other words, if the computed variance according to the
collected observations ends up lower than |
0.01 |
clip |
Optional[tuple] |
Can be left as None (which is the default), or can be given as a pair of real numbers. This is used for clipping the observations after the normalization operation. In Salimans et al. (2017), (-5.0, +5.0) was used. |
None |
Source code in evotorch/neuroevolution/net/runningnorm.py
def __init__(
self,
*,
shape: Union[tuple, int],
dtype: DType,
device: Optional[Device] = None,
min_variance: float = 1e-2,
clip: Optional[tuple] = None,
) -> None:
"""
`__init__(...)`: Initialize the RunningNorm
Args:
shape: Observation shape. Can be an integer or a tuple.
dtype: The dtype of the observations.
device: The device in which the observation stats are held.
If left as None, the device is assumed to be "cpu".
min_variance: A lower bound for the variance to be used in
the normalization computations.
In other words, if the computed variance according to the
collected observations ends up lower than `min_variance`,
this `min_variance` will be used instead (in an elementwise
manner) while computing the normalized observations.
As in Salimans et al. (2017), the default is 1e-2.
clip: Can be left as None (which is the default), or can be
given as a pair of real numbers.
This is used for clipping the observations after the
normalization operation.
In Salimans et al. (2017), (-5.0, +5.0) was used.
"""
# Make sure that the shape is stored as a torch.Size object.
if isinstance(shape, Iterable):
self._shape = torch.Size(shape)
else:
self._shape = torch.Size([int(shape)])
# Store the number of dimensions
self._ndim = len(self._shape)
# Store the dtype and the device
self._dtype = to_torch_dtype(dtype)
self._device = "cpu" if device is None else device
# Initialize the internally stored data as empty
self._sum: Optional[torch.Tensor] = None
self._sum_of_squares: Optional[torch.Tensor] = None
self._count: int = 0
# Store the minimum variance
self._min_variance = float(min_variance)
if clip is not None:
# If a clip tuple was provided, store the specified lower and upper bounds
lb, ub = clip
self._lb = float(lb)
self._ub = float(ub)
else:
# If a clip tuple was not provided the bounds are stored as None
self._lb = None
self._ub = None
normalize(self, x, *, result_as_numpy=None, verify=True)
¶
Normalize the given observation x.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Iterable |
The observation(s), as a PyTorch tensor, or any Iterable
that is convertable to a PyTorch tensor.
|
required |
result_as_numpy |
Optional[bool] |
Whether or not to return the normalized observation as a numpy array. If left as None (which is the default), then the returned type depends on x: a PyTorch tensor is returned if x is a PyTorch tensor, and a numpy array is returned otherwise. If True, the result is always a numpy array. If False, the result is always a PyTorch tensor. |
None |
verify |
bool |
Whether or not to check the type and dimensions of x.
This is True by default.
Note that, if |
True |
Returns:
| Type | Description |
|---|---|
Iterable |
The normalized observation, as a PyTorch tensor or a numpy array. |
Source code in evotorch/neuroevolution/net/runningnorm.py
@torch.no_grad()
def normalize(self, x: Iterable, *, result_as_numpy: Optional[bool] = None, verify: bool = True) -> Iterable:
"""
Normalize the given observation x.
Args:
x: The observation(s), as a PyTorch tensor, or any Iterable
that is convertable to a PyTorch tensor.
`x` can be a single observation, or it can be a batch
of observations (with an extra leftmost dimension).
result_as_numpy: Whether or not to return the normalized
observation as a numpy array.
If left as None (which is the default), then the returned
type depends on x: a PyTorch tensor is returned if x is a
PyTorch tensor, and a numpy array is returned otherwise.
If True, the result is always a numpy array.
If False, the result is always a PyTorch tensor.
verify: Whether or not to check the type and dimensions of x.
This is True by default.
Note that, if `verify` is False, this function will not
properly check the type of `x` and will assume that `x`
is a PyTorch tensor.
Returns:
The normalized observation, as a PyTorch tensor or a numpy array.
"""
if self._count == 0:
# If this RunningNorm instance has no data yet,
# then we do not know how to do the normalization.
# We therefore raise an error.
raise ValueError("Cannot do normalization because no data is collected yet.")
if verify:
# Here we verify the type and shape of x.
if result_as_numpy is None:
# If there is not an explicit request about the return type,
# we infer the return type from the type of x:
# if x is a tensor, we return a tensor;
# otherwise, we assume x to be a CPU-bound iterable, and
# therefore we return a numpy array.
result_as_numpy = not isinstance(x, torch.Tensor)
else:
result_as_numpy = bool(result_as_numpy)
# We call _verify() to make sure that x is of correct shape
# and is properly converted to a PyTorch tensor.
x = self._verify(x)
# We get the mean and stdev of the collected data
mean, stdev = self.stats
# Now we compute the normalized observation, clipped according to the
# lower and upper bounds expressed by the `clip` tuple, if exists.
result = _clamp((x - mean) / stdev, self._lb, self._ub)
if result_as_numpy:
# If we are to return the result as a numpy array, we do the
# necessary conversion.
result = result.cpu().numpy()
# Finally, return the result
return result
reset(self)
¶
to(self, device)
¶
If the target device is a different device, then make a copy of this RunningNorm instance on the target device. If the target device is the same with this RunningNorm's device, then return this RunningNorm itself.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
device |
Union[str, torch.device] |
The target device. |
required |
Returns:
| Type | Description |
|---|---|
RunningNorm |
The RunningNorm on the target device. This can be a copy, or the original RunningNorm instance itself. |
Source code in evotorch/neuroevolution/net/runningnorm.py
def to(self, device: Device) -> "RunningNorm":
"""
If the target device is a different device, then make a copy of this
RunningNorm instance on the target device.
If the target device is the same with this RunningNorm's device, then
return this RunningNorm itself.
Args:
device: The target device.
Returns:
The RunningNorm on the target device. This can be a copy, or the
original RunningNorm instance itself.
"""
if torch.device(device) == torch.device(self.device):
return self
else:
new_running_norm = object.__new__(type(self))
already_handled = {"_sum", "_sum_of_squares", "_device"}
new_running_norm._sum = self._sum.to(device)
new_running_norm._sum_of_squares = self._sum_of_squares.to(device)
new_running_norm._device = device
for k, v in self.__dict__.items():
if k not in already_handled:
setattr(new_running_norm, k, deepcopy(v))
return new_running_norm
to_layer(self)
¶
Make a PyTorch module which normalizes the its inputs.
Returns:
| Type | Description |
|---|---|
ObsNormLayer |
An ObsNormLayer instance. |
Source code in evotorch/neuroevolution/net/runningnorm.py
update(self, x, mask=None, *, verify=True)
¶
Update the stored stats with new observation data.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Union[Iterable, RunningNorm] |
The new observation(s), as a PyTorch tensor, or any Iterable
that can be converted to a PyTorch tensor, or another
RunningNorm instance.
If given as a tensor or as an Iterable, the shape of |
required |
mask |
Optional[Iterable] |
Can be given as a 1-dimensional Iterable of booleans ONLY
if |
None |
verify |
bool |
Whether or not to verify the shape of the given Iterable objects. The default is True. |
True |
Source code in evotorch/neuroevolution/net/runningnorm.py
@torch.no_grad()
def update(self, x: Union[Iterable, "RunningNorm"], mask: Optional[Iterable] = None, *, verify: bool = True):
"""
Update the stored stats with new observation data.
Args:
x: The new observation(s), as a PyTorch tensor, or any Iterable
that can be converted to a PyTorch tensor, or another
RunningNorm instance.
If given as a tensor or as an Iterable, the shape of `x` can
be the same with observation shape, or it can be augmented
with an extra leftmost dimension.
In the case of augmented dimension, `x` is interpreted not as
a single observation, but as a batch of observations.
If `x` is another RunningNorm instance, the stats stored by
this RunningNorm instance will be updated with all the data
stored by `x`.
mask: Can be given as a 1-dimensional Iterable of booleans ONLY
if `x` represents a batch of observations.
If a `mask` is provided, the i-th observation within the
observation batch `x` will be taken into account only if
the i-th item of the `mask` is True.
verify: Whether or not to verify the shape of the given Iterable
objects. The default is True.
"""
if isinstance(x, RunningNorm):
# If we are to update our stats according to another RunningNorm instance
if x._count > 0:
# We bother only if x is non-empty
if mask is not None:
# We were given another RunningNorm, not a batch of observations.
# So, we do not expect to receive a mask tensor.
# If a mask was provided, then this is an unexpected way of calling this function.
# We therefore raise an error.
raise ValueError(
"The `mask` argument is expected as None if the first argument is a RunningNorm."
" However, `mask` is found as something other than None."
)
if self._shape != x._shape:
# If the shapes of this RunningNorm and of the other RunningNorm
# do not match, then we cannot use `x` for updating our stats.
# It might be the case that `x` was initialized for another
# task, with differently sized observations.
# We therefore raise an error.
raise ValueError(
f"The RunningNorm to be updated has the shape {self._shape}"
f" The other RunningNorm has the shape {self._shape}"
f" These shapes are incompatible."
)
if self._has_no_data():
# If this RunningNorm has no data at all, then we clone the
# data of x.
self._sum = self._like_its_own(x._sum.clone())
self._sum_of_squares = self._like_its_own(x._sum_of_squares.clone())
self._count = x._count
elif self._has_data():
# If this RunningNorm has its own data, then we update the
# stored data with the data stored by x.
self._sum += self._like_its_own(x._sum)
self._sum_of_squares += self._like_its_own(x._sum_of_squares)
self._count += x._count
else:
assert False, "RunningNorm is in an invalid state! This might be a bug."
else:
# This is the case where the received argument x is not a
# RunningNorm object, but an Iterable.
if verify:
# If we have the `verify` flag, then we make sure that
# x is a tensor of the correct shape
x = self._verify(x)
if x.ndim == self._ndim:
# If the shape of x is exactly the same with the observation shape
# then we assume that x represents a single observation, and not a
# batch of observations.
if mask is not None:
# Since we are dealing with a single observation,
# we do not expect to receive a mask argument.
# If the mask argument was provided, then this is an unexpected
# usage of this function.
# We therefore raise an error.
raise ValueError(
"The `mask` argument is expected as None if the first argument is a single observation"
" (i.e. not a batch of observations, with an extra leftmost dimension)."
" However, `mask` is found as something other than None."
)
# Since x is a single observation,
# the sum of observations extracted from x is x itself,
# and the sum of squared observations extracted from x is
# the square of x itself.
sum_of_x = x
sum_of_x_squared = x.square()
# We extracted a single observation from x
n = 1
elif x.ndim == (self._ndim + 1):
# If the number of dimensions of x is one more than the number
# of dimensions of this RunningNorm, then we assume that x is a batch
# of observations.
if mask is not None:
# If a mask is provided, then we first make sure that it is a tensor
# of dtype bool in the correct device.
mask = torch.as_tensor(mask, dtype=torch.bool, device=self._device)
if mask.ndim != 1:
# We expect the mask to be 1-dimensional.
# If not, we raise an error.
raise ValueError(
f"The `mask` tensor was expected as a 1-dimensional tensor."
f" However, its shape is {mask.shape}."
)
if len(mask) != x.shape[0]:
# If the length of the mask is not the batch size of x,
# then there is a mismatch.
# We therefore raise an error.
raise ValueError(
f"The shape of the given tensor is {x.shape}."
f" Therefore, the batch size of observations is {x.shape[0]}."
f" However, the given `mask` tensor does not has an incompatible length: {len(mask)}."
)
# We compute how many True items we have in the mask.
# This integer gives us how many observations we extract from x.
n = int(torch.sum(torch.as_tensor(mask, dtype=torch.int64, device=self._device)))
# We now re-cast the mask as the observation dtype (so that True items turn to 1.0
# and False items turn to 0.0), and then increase its number of dimensions so that
# it can operate directly with x.
mask = self._like_its_own(mask).reshape(torch.Size([x.shape[0]] + ([1] * (x.ndim - 1))))
# Finally, we multiply x with the mask. This means that the observations with corresponding
# mask values as False are zeroed out.
x = x * mask
else:
# This is the case where we did not receive a mask.
# We can simply say that the number of observations to extract from x
# is the size of its leftmost dimension, i.e. the batch size.
n = x.shape[0]
# With or without a mask, we are now ready to extract the sum and sum of squares
# from x.
sum_of_x = torch.sum(x, dim=0)
sum_of_x_squared = torch.sum(x.square(), dim=0)
else:
# This is the case where the number of dimensions of x is unrecognized.
# This case is actually already checked by the _verify(...) method earlier.
# This defensive fallback case is only for when verify=False and it turned out
# that the ndim is invalid.
raise ValueError(f"Invalid shape: {x.shape}")
# At this point, we handled all the valid cases regarding the Iterable x,
# and we have our sum_of_x (sum of all observations), sum_of_squares
# (sum of all squared observations), and n (number of observations extracted
# from x).
if self._has_no_data():
# If our RunningNorm is empty, the observation data we extracted from x
# become our RunningNorm's new data.
self._sum = sum_of_x
self._sum_of_squares = sum_of_x_squared
self._count = n
elif self._has_data():
# If our RunningNorm is not empty, the stored data is updated with the
# data extracted from x.
self._sum += sum_of_x
self._sum_of_squares += sum_of_x_squared
self._count += n
else:
# This is an erroneous state where the internal data looks neither
# existent nor completely empty.
# This might be the result of a bug, or maybe this instance's
# protected variables were tempered with from the outside.
assert False, "RunningNorm is in an invalid state! This might be a bug."
update_and_normalize(self, x, mask=None)
¶
Update the observation stats according to x, then normalize x.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Iterable |
The observation(s), as a PyTorch tensor, or as an Iterable which can be converted to a PyTorch tensor. The shape of x can be the same with the observaiton shape, or it can be augmented with an extra leftmost dimension to express a batch of observations. |
required |
mask |
Optional[Iterable] |
Can be given as a 1-dimensional Iterable of booleans ONLY
if |
None |
Returns:
| Type | Description |
|---|---|
Iterable |
The normalized counterpart of the observation(s) expressed by x. |
Source code in evotorch/neuroevolution/net/runningnorm.py
@torch.no_grad()
def update_and_normalize(self, x: Iterable, mask: Optional[Iterable] = None) -> Iterable:
"""
Update the observation stats according to x, then normalize x.
Args:
x: The observation(s), as a PyTorch tensor, or as an Iterable
which can be converted to a PyTorch tensor.
The shape of x can be the same with the observaiton shape,
or it can be augmented with an extra leftmost dimension
to express a batch of observations.
mask: Can be given as a 1-dimensional Iterable of booleans ONLY
if `x` represents a batch of observations.
If a `mask` is provided, the i-th observation within the
observation batch `x` will be taken into account only if
the the i-th item of the `mask` is True.
Returns:
The normalized counterpart of the observation(s) expressed by x.
"""
result_as_numpy = not isinstance(x, torch.Tensor)
x = self._verify(x)
self.update(x, mask, verify=False)
result = self.normalize(x, verify=False)
if result_as_numpy:
result = result.cpu().numpy()
return result
runningstat
¶
RunningStat
¶
Tool for efficiently computing the mean and stdev of arrays. The arrays themselves are not stored separately, instead, they are accumulated.
This RunningStat is implemented as a wrapper around RunningNorm. The difference is that the interface of RunningStat is simplified to expect only numpy arrays, and expect only non-vectorized observations. With this simplified interface, RunningStat is meant to be used by GymNE, on classical non-vectorized gym tasks.
Source code in evotorch/neuroevolution/net/runningstat.py
class RunningStat:
"""
Tool for efficiently computing the mean and stdev of arrays.
The arrays themselves are not stored separately,
instead, they are accumulated.
This RunningStat is implemented as a wrapper around RunningNorm.
The difference is that the interface of RunningStat is simplified
to expect only numpy arrays, and expect only non-vectorized
observations.
With this simplified interface, RunningStat is meant to be used
by GymNE, on classical non-vectorized gym tasks.
"""
def __init__(self):
"""
`__init__(...)`: Initialize the RunningStat.
"""
self._rn: Optional[RunningNorm] = None
self.reset()
def reset(self):
"""
Reset the RunningStat to its initial state.
"""
self._rn = None
@property
def count(self) -> int:
"""
Get the number of arrays accumulated.
"""
if self._rn is None:
return 0
else:
return self._rn.count
@property
def sum(self) -> np.ndarray:
"""
Get the sum of all accumulated arrays.
"""
return self._rn.sum.numpy()
@property
def sum_of_squares(self) -> np.ndarray:
"""
Get the sum of squares of all accumulated arrays.
"""
return self._rn.sum_of_squares.numpy()
@property
def mean(self) -> np.ndarray:
"""
Get the mean of all accumulated arrays.
"""
return self._rn.mean.numpy()
@property
def stdev(self) -> np.ndarray:
"""
Get the standard deviation of all accumulated arrays.
"""
return self._rn.stdev.numpy()
def update(self, x: Union[np.ndarray, "RunningStat"]):
"""
Accumulate more data into the RunningStat object.
If the argument is an array, that array is added
as one more data element.
If the argument is another RunningStat instance,
all the stats accumulated by that RunningStat object
are added into this RunningStat object.
"""
if isinstance(x, RunningStat):
if x.count > 0:
if self._rn is None:
self._rn = deepcopy(x._rn)
else:
self._rn.update(x._rn)
else:
if self._rn is None:
x = np.array(x, dtype="float32")
self._rn = RunningNorm(shape=x.shape, dtype="float32", device="cpu")
self._rn.update(x)
def normalize(self, x: Union[np.ndarray, list]) -> np.ndarray:
"""
Normalize the array x according to the accumulated stats.
"""
if self._rn is None:
return x
else:
x = np.array(x, dtype="float32")
return self._rn.normalize(x)
def __copy__(self):
return deepcopy(self)
def __repr__(self) -> str:
return f"<{self.__class__.__name__}, count: {self.count}>"
def to(self, device: Union[str, torch.device]) -> "RunningStat":
"""
If the target device is cpu, return this RunningStat instance itself.
A RunningStat object is meant to work with numpy arrays. Therefore,
any device other than the cpu will trigger an error.
Args:
device: The target device. Only cpu is supported.
Returns:
The original RunningStat.
"""
if torch.device(device) == torch.device("cpu"):
return self
else:
raise ValueError(
f"The received target device is {repr(device)}. However, RunningStat can only work on a cpu."
)
def to_layer(self) -> nn.Module:
"""
Make a PyTorch module which normalizes the its inputs.
Returns:
An ObsNormLayer instance.
"""
return self._rn.to_layer()
count: int
property
readonly
¶
Get the number of arrays accumulated.
mean: ndarray
property
readonly
¶
Get the mean of all accumulated arrays.
stdev: ndarray
property
readonly
¶
Get the standard deviation of all accumulated arrays.
sum: ndarray
property
readonly
¶
Get the sum of all accumulated arrays.
sum_of_squares: ndarray
property
readonly
¶
Get the sum of squares of all accumulated arrays.
__init__(self)
special
¶
normalize(self, x)
¶
Normalize the array x according to the accumulated stats.
reset(self)
¶
to(self, device)
¶
If the target device is cpu, return this RunningStat instance itself. A RunningStat object is meant to work with numpy arrays. Therefore, any device other than the cpu will trigger an error.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
device |
Union[str, torch.device] |
The target device. Only cpu is supported. |
required |
Returns:
| Type | Description |
|---|---|
RunningStat |
The original RunningStat. |
Source code in evotorch/neuroevolution/net/runningstat.py
def to(self, device: Union[str, torch.device]) -> "RunningStat":
"""
If the target device is cpu, return this RunningStat instance itself.
A RunningStat object is meant to work with numpy arrays. Therefore,
any device other than the cpu will trigger an error.
Args:
device: The target device. Only cpu is supported.
Returns:
The original RunningStat.
"""
if torch.device(device) == torch.device("cpu"):
return self
else:
raise ValueError(
f"The received target device is {repr(device)}. However, RunningStat can only work on a cpu."
)
to_layer(self)
¶
Make a PyTorch module which normalizes the its inputs.
Returns:
| Type | Description |
|---|---|
Module |
An ObsNormLayer instance. |
update(self, x)
¶
Accumulate more data into the RunningStat object. If the argument is an array, that array is added as one more data element. If the argument is another RunningStat instance, all the stats accumulated by that RunningStat object are added into this RunningStat object.
Source code in evotorch/neuroevolution/net/runningstat.py
def update(self, x: Union[np.ndarray, "RunningStat"]):
"""
Accumulate more data into the RunningStat object.
If the argument is an array, that array is added
as one more data element.
If the argument is another RunningStat instance,
all the stats accumulated by that RunningStat object
are added into this RunningStat object.
"""
if isinstance(x, RunningStat):
if x.count > 0:
if self._rn is None:
self._rn = deepcopy(x._rn)
else:
self._rn.update(x._rn)
else:
if self._rn is None:
x = np.array(x, dtype="float32")
self._rn = RunningNorm(shape=x.shape, dtype="float32", device="cpu")
self._rn.update(x)
statefulmodule
¶
StatefulModule (Module)
¶
A wrapper that provides a stateful interface for recurrent torch modules.
If the torch module to be wrapped is non-recurrent and its forward method has a single input (the input tensor) and a single output (the output tensor), then this wrapper module acts as a no-op wrapper.
If the torch module to be wrapped is recurrent and its forward method has
two inputs (the input tensor and an optional second argument for the hidden
state) and two outputs (the output tensor and the new hidden state), then
this wrapper brings a new forward-passing interface. In this new interface,
the forward method has a single input (the input tensor) and a single
output (the output tensor). The hidden states, instead of being
explicitly requested via a second argument and returned as a second
result, are stored and used by the wrapper.
When a new series of inputs is to be used, one has to call the reset()
method of this wrapper.
Source code in evotorch/neuroevolution/net/statefulmodule.py
class StatefulModule(nn.Module):
"""
A wrapper that provides a stateful interface for recurrent torch modules.
If the torch module to be wrapped is non-recurrent and its forward method
has a single input (the input tensor) and a single output (the output
tensor), then this wrapper module acts as a no-op wrapper.
If the torch module to be wrapped is recurrent and its forward method has
two inputs (the input tensor and an optional second argument for the hidden
state) and two outputs (the output tensor and the new hidden state), then
this wrapper brings a new forward-passing interface. In this new interface,
the forward method has a single input (the input tensor) and a single
output (the output tensor). The hidden states, instead of being
explicitly requested via a second argument and returned as a second
result, are stored and used by the wrapper.
When a new series of inputs is to be used, one has to call the `reset()`
method of this wrapper.
"""
def __init__(self, wrapped_module: nn.Module):
"""
`__init__(...)`: Initialize the StatefulModule.
Args:
wrapped_module: The `torch.nn.Module` instance to wrap.
"""
super().__init__()
# Declare the variable that will store the hidden state of wrapped_module, if any.
self._hidden: Any = None
# Store the module that is wrapped.
self.wrapped_module = wrapped_module
def forward(self, x: torch.Tensor) -> torch.Tensor:
if self._hidden is None:
# If there is no stored hidden state, then only pass the input tensor to the wrapped module.
out = self.wrapped_module(x)
else:
# If there is a hidden state saved from the previous call to this `forward(...)` method, then pass the
# input tensor and this stored hidden state.
out = self.wrapped_module(x, self._hidden)
if isinstance(out, tuple):
# If the result of the wrapped module is a tuple, then we assume that the wrapped module returned an
# output tensor and a hidden state. We assume the first element of this tuple as the output tensor,
# and the second element as the new hidden state.
# We set the variable y to the output tensor, and we store the new hidden state via the attribute
# `_hidden`.
y, self._hidden = out
else:
# If the result of the wrapped module is not a tuple, then we assume that the wrapped module returned
# only the output tensor. We set the variable y to the output tensor, and set the attribute `_hidden`
# as None to indicate that there was no hidden state received.
y = out
self._hidden = None
# We return y, which stores the output received by the wrapped module.
return y
def reset(self):
"""
Reset the hidden state, if any.
"""
self._hidden = None
__init__(self, wrapped_module)
special
¶
__init__(...): Initialize the StatefulModule.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
wrapped_module |
Module |
The |
required |
Source code in evotorch/neuroevolution/net/statefulmodule.py
def __init__(self, wrapped_module: nn.Module):
"""
`__init__(...)`: Initialize the StatefulModule.
Args:
wrapped_module: The `torch.nn.Module` instance to wrap.
"""
super().__init__()
# Declare the variable that will store the hidden state of wrapped_module, if any.
self._hidden: Any = None
# Store the module that is wrapped.
self.wrapped_module = wrapped_module
forward(self, x)
¶
Define the computation performed at every call.
Should be overridden by all subclasses.
.. note::
Although the recipe for forward pass needs to be defined within
this function, one should call the :class:Module instance afterwards
instead of this since the former takes care of running the
registered hooks while the latter silently ignores them.
Source code in evotorch/neuroevolution/net/statefulmodule.py
def forward(self, x: torch.Tensor) -> torch.Tensor:
if self._hidden is None:
# If there is no stored hidden state, then only pass the input tensor to the wrapped module.
out = self.wrapped_module(x)
else:
# If there is a hidden state saved from the previous call to this `forward(...)` method, then pass the
# input tensor and this stored hidden state.
out = self.wrapped_module(x, self._hidden)
if isinstance(out, tuple):
# If the result of the wrapped module is a tuple, then we assume that the wrapped module returned an
# output tensor and a hidden state. We assume the first element of this tuple as the output tensor,
# and the second element as the new hidden state.
# We set the variable y to the output tensor, and we store the new hidden state via the attribute
# `_hidden`.
y, self._hidden = out
else:
# If the result of the wrapped module is not a tuple, then we assume that the wrapped module returned
# only the output tensor. We set the variable y to the output tensor, and set the attribute `_hidden`
# as None to indicate that there was no hidden state received.
y = out
self._hidden = None
# We return y, which stores the output received by the wrapped module.
return y
reset(self)
¶
ensure_stateful(net)
¶
Ensure that a module is wrapped by StatefulModule.
If the given module is already wrapped by StatefulModule, then the module itself is returned. If the given module is not wrapped by StatefulModule, then this function first wraps the module via a new StatefulModule instance, and then this new wrapper is returned.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
net |
Module |
The |
required |
Returns:
| Type | Description |
|---|---|
StatefulModule |
The module |
Source code in evotorch/neuroevolution/net/statefulmodule.py
def ensure_stateful(net: nn.Module) -> StatefulModule:
"""
Ensure that a module is wrapped by StatefulModule.
If the given module is already wrapped by StatefulModule, then the
module itself is returned.
If the given module is not wrapped by StatefulModule, then this function
first wraps the module via a new StatefulModule instance, and then this
new wrapper is returned.
Args:
net: The `torch.nn.Module` to be wrapped by StatefulModule (if it is
not already wrapped by it).
Returns:
The module `net`, wrapped by StatefulModule.
"""
if not isinstance(net, StatefulModule):
return StatefulModule(net)
return net
vecrl
¶
This namespace provides various vectorized reinforcement learning utilities.
BaseVectorEnv (VectorEnv)
¶
A base class for vectorized gymnasium environments.
In gymnasium 0.29.x, the __init__(...) method of the base class
gymnasium.vector.VectorEnv expects the arguments num_envs,
observation_space, and action_space, and then prepares the instance
attributes num_envs, single_observation_space, single_action_space,
observation_space, and action_space according to the initialization
arguments it receives.
It appears that with gymnasium 1.x, this API is changing, and
gymnasium.vector.VectorEnv strictly expects no positional arguments.
This BaseVectorEnv class is meant as a base class which preserves
the behavior of gymnasium 0.29.x, meaning that it will expects the
arguments, and prepare the attributes mentioned above.
Please note, however, that this BaseVectorEnv implementation
can only work with environments whose single observation and single
action spaces are either Box or Discrete.
Source code in evotorch/neuroevolution/net/vecrl.py
class BaseVectorEnv(gym.vector.VectorEnv):
"""
A base class for vectorized gymnasium environments.
In gymnasium 0.29.x, the `__init__(...)` method of the base class
`gymnasium.vector.VectorEnv` expects the arguments `num_envs`,
`observation_space`, and `action_space`, and then prepares the instance
attributes `num_envs`, `single_observation_space`, `single_action_space`,
`observation_space`, and `action_space` according to the initialization
arguments it receives.
It appears that with gymnasium 1.x, this API is changing, and
`gymnasium.vector.VectorEnv` strictly expects no positional arguments.
This `BaseVectorEnv` class is meant as a base class which preserves
the behavior of gymnasium 0.29.x, meaning that it will expects the
arguments, and prepare the attributes mentioned above.
Please note, however, that this `BaseVectorEnv` implementation
can only work with environments whose single observation and single
action spaces are either `Box` or `Discrete`.
"""
def __init__(self, num_envs: int, observation_space: Space, action_space: Space):
"""
`__init__(...)`: Initialize the vectorized environment.
Args:
num_envs: Number of sub-environments handled by this `BaseVectorEnv`.
observation_space: Observation space of a single sub-environment.
This can only be given as an instance of type
`gymnasium.spaces.Box` or `gymnasium.spaces.Discrete`.
action_space: Action space of a single sub-environment.
This can only be given as an instance of type
`gymnasium.spaces.Box` or `gymnasium.spaces.Discrete`.
"""
super().__init__()
self.num_envs = int(num_envs)
self.single_observation_space = observation_space
self.single_action_space = action_space
self.observation_space = _batch_space(self.single_observation_space, self.num_envs)
self.action_space = _batch_space(self.single_action_space, self.num_envs)
__init__(self, num_envs, observation_space, action_space)
special
¶
__init__(...): Initialize the vectorized environment.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
num_envs |
int |
Number of sub-environments handled by this |
required |
observation_space |
Space |
Observation space of a single sub-environment.
This can only be given as an instance of type
|
required |
action_space |
Space |
Action space of a single sub-environment.
This can only be given as an instance of type
|
required |
Source code in evotorch/neuroevolution/net/vecrl.py
def __init__(self, num_envs: int, observation_space: Space, action_space: Space):
"""
`__init__(...)`: Initialize the vectorized environment.
Args:
num_envs: Number of sub-environments handled by this `BaseVectorEnv`.
observation_space: Observation space of a single sub-environment.
This can only be given as an instance of type
`gymnasium.spaces.Box` or `gymnasium.spaces.Discrete`.
action_space: Action space of a single sub-environment.
This can only be given as an instance of type
`gymnasium.spaces.Box` or `gymnasium.spaces.Discrete`.
"""
super().__init__()
self.num_envs = int(num_envs)
self.single_observation_space = observation_space
self.single_action_space = action_space
self.observation_space = _batch_space(self.single_observation_space, self.num_envs)
self.action_space = _batch_space(self.single_action_space, self.num_envs)
Policy
¶
A Policy for deciding the actions for a reinforcement learning environment.
This can be seen as a stateful wrapper around a PyTorch module.
Let us assume that we have the following PyTorch module:
which has 48 parameters (when all the parameters are flattened).
Let us randomly generate a parameter vector for our module net:
We can now prepare a policy:
If we generate a random observation:
We can receive our action as follows:
If the PyTorch module that we wish to wrap is a recurrent network (i.e. a network which expects an optional second argument for the hidden state, and returns a second value which represents the updated hidden state), then, the hidden state is automatically managed by the Policy instance.
Let us assume that we have a recurrent network named recnet.
In this case, because the hidden state of the network is internally managed, the usage is still the same with our previous non-recurrent
Examples:
When using a recurrent module on multiple episodes, it is important to reset the hidden state of the network. This is achieved by the reset method:
policy.reset()
action1 = policy(observation1)
# action2 will be computed with the hidden state generated by the
# previous forward-pass.
action2 = policy(observation2)
policy.reset()
# action3 will be computed according to the renewed hidden state.
action3 = policy(observation3)
Both for non-recurrent and recurrent networks, it is possible to perform vectorized operations. For now, let us return to our first non-recurrent example:
Instead of generating only one parameter vector, we now generate a batch of parameter vectors. Let us say that our batch size is 10:
Like we did in the non-batched examples, we can do:
Because we are now in the batched mode, policy now expects a batch
of observations and will return a batch of actions:
When doing vectorized reinforcement learning with a recurrent module,
it can be the case that only some of the environments are finished,
and therefore it is necessary to reset the hidden states associated
with those environments only. The reset(...) method of Policy
has a second argument to specify which of the recurrent network
instances are to be reset. For example, if the episodes of the
environments with indices 2 and 5 are about to restart (and therefore
we wish to reset the states of the networks with indices 2 and 5),
then, we can do:
Source code in evotorch/neuroevolution/net/vecrl.py
class Policy:
"""
A Policy for deciding the actions for a reinforcement learning environment.
This can be seen as a stateful wrapper around a PyTorch module.
Let us assume that we have the following PyTorch module:
```python
from torch import nn
net = nn.Linear(5, 8)
```
which has 48 parameters (when all the parameters are flattened).
Let us randomly generate a parameter vector for our module `net`:
```python
parameters = torch.randn(48)
```
We can now prepare a policy:
```python
policy = Policy(net)
policy.set_parameters(parameters)
```
If we generate a random observation:
```python
observation = torch.randn(5)
```
We can receive our action as follows:
```python
action = policy(observation)
```
If the PyTorch module that we wish to wrap is a recurrent network (i.e.
a network which expects an optional second argument for the hidden state,
and returns a second value which represents the updated hidden state),
then, the hidden state is automatically managed by the Policy instance.
Let us assume that we have a recurrent network named `recnet`.
```python
policy = Policy(recnet)
policy.set_parameters(parameters_of_recnet)
```
In this case, because the hidden state of the network is internally
managed, the usage is still the same with our previous non-recurrent
example:
```python
action = policy(observation)
```
When using a recurrent module on multiple episodes, it is important
to reset the hidden state of the network. This is achieved by the
reset method:
```python
policy.reset()
action1 = policy(observation1)
# action2 will be computed with the hidden state generated by the
# previous forward-pass.
action2 = policy(observation2)
policy.reset()
# action3 will be computed according to the renewed hidden state.
action3 = policy(observation3)
```
Both for non-recurrent and recurrent networks, it is possible to
perform vectorized operations. For now, let us return to our
first non-recurrent example:
```python
net = nn.Linear(5, 8)
```
Instead of generating only one parameter vector, we now generate
a batch of parameter vectors. Let us say that our batch size is 10:
```python
batch_of_parameters = torch.randn(10, 48)
```
Like we did in the non-batched examples, we can do:
```python
policy = Policy(net)
policy.set_parameters(batch_of_parameters)
```
Because we are now in the batched mode, `policy` now expects a batch
of observations and will return a batch of actions:
```python
batch_of_observations = torch.randn(10, 5)
batch_of_actions = policy(batch_of_observations)
```
When doing vectorized reinforcement learning with a recurrent module,
it can be the case that only some of the environments are finished,
and therefore it is necessary to reset the hidden states associated
with those environments only. The `reset(...)` method of Policy
has a second argument to specify which of the recurrent network
instances are to be reset. For example, if the episodes of the
environments with indices 2 and 5 are about to restart (and therefore
we wish to reset the states of the networks with indices 2 and 5),
then, we can do:
```python
policy.reset(torch.tensor([2, 5]))
```
"""
def __init__(self, net: Union[str, Callable, nn.Module], **kwargs):
"""
`__init__(...)`: Initialize the Policy.
Args:
net: The network to be wrapped by the Policy object.
This can be a string, a Callable (e.g. a `torch.nn.Module`
subclass), or a `torch.nn.Module` instance.
When this argument is a string, the network will be
created with the help of the function
`evotorch.neuroevolution.net.str_to_net(...)` and then
wrapped. Please see the `str_to_net(...)` function's
documentation for details regarding how a network structure
can be expressed via strings.
kwargs: Expected in the form of additional keyword arguments,
these keyword arguments will be passed to the provided
Callable object (if the argument `net` is a Callable)
or to `str_to_net(...)` (if the argument `net` is a string)
at the moment of generating the network.
If the argument `net` is a `torch.nn.Module` instance,
having any additional keyword arguments will trigger an
error, because the network is already instantiated and
therefore, it is not possible to pass these keyword arguments.
"""
from ..net import str_to_net
from ..net.functional import ModuleExpectingFlatParameters, make_functional_module
if isinstance(net, str):
self.__module = str_to_net(net, **kwargs)
elif isinstance(net, nn.Module):
if len(kwargs) > 0:
raise ValueError(
f"When the network is given as an `nn.Module` instance, extra network arguments cannot be used"
f" (because the network is already instantiated)."
f" However, these extra keyword arguments were received: {kwargs}."
)
self.__module = net
elif isinstance(net, Callable):
self.__module = net(**kwargs)
else:
raise TypeError(
f"The class `Policy` expected a string or an `nn.Module` instance, or a Callable, but received {net}"
f" (whose type is {type(net)})."
)
self.__fmodule: ModuleExpectingFlatParameters = make_functional_module(self.__module)
self.__state: Any = None
self.__parameters: Optional[torch.Tensor] = None
def set_parameters(self, parameters: torch.Tensor, indices: Optional[MaskOrIndices] = None, *, reset: bool = True):
"""
Set the parameters of the policy.
Args:
parameters: A 1-dimensional or a 2-dimensional tensor containing
the flattened parameters to be used with the neural network.
If the given parameters are two-dimensional, then, given that
the leftmost size of the parameter tensor is `n`, the
observations will be expected in a batch with leftmost size
`n`, and the returned actions will also be in a batch,
again with the leftmost size `n`.
indices: For when the parameters were previously given via a
2-dimensional tensor, provide this argument if you would like
to change only some rows of the previously given parameters.
For example, if `indices` is given as `torch.tensor([2, 4])`
and the argument `parameters` is given as a 2-dimensional
tensor with leftmost size 2, then the rows with indices
2 and 4 will be replaced by these new parameters provided
via the argument `parameters`.
reset: If given as True, the hidden states of the networks whose
parameters just changed will be reset. If `indices` was not
provided at all, then this means that the parameters of all
networks are modified, in which case, all the hidden states
will be reset.
If given as False, no such resetting will be done.
"""
if self.__parameters is None:
if indices is not None:
raise ValueError(
"The argument `indices` can be used only if network parameters were previously specified."
" However, it seems that the method `set_parameters(...)` was not called before."
)
self.__parameters = parameters
else:
if indices is None:
self.__parameters = parameters
else:
self.__parameters[indices] = parameters
if reset:
self.reset(indices)
def __call__(self, x: torch.Tensor) -> torch.Tensor:
"""
Pass the given observations through the network.
Args:
x: The observations, as a PyTorch tensor.
If the parameters were given (via the method
`set_parameters(...)`) as a 1-dimensional tensor, then this
argument is expected to store a single observation.
If the parameters were given as a 2-dimensional tensor,
then, this argument is expected to store a batch of
observations, and the leftmost size of this observation
tensor must match with the leftmost size of the parameter
tensor.
Returns:
The output tensor, which represents the action to take.
"""
if self.__parameters is None:
raise ValueError("Please use the method `set_parameters(...)` before calling the policy.")
if self.__state is None:
further_args = (x,)
else:
further_args = (x, self.__state)
parameters = self.__parameters
ndim = parameters.ndim
if ndim == 1:
result = self.__fmodule(parameters, *further_args)
elif ndim == 2:
vmapped = vmap(self.__fmodule)
result = vmapped(parameters, *further_args)
else:
raise ValueError(
f"Expected the parameters as a 1 or 2 dimensional tensor."
f" However, the received parameters tensor has {ndim} dimensions."
)
if isinstance(result, torch.Tensor):
return result
elif isinstance(result, tuple):
result, state = result
self.__state = state
return result
else:
raise TypeError(f"The torch module used by the Policy returned an unexpected object: {result}")
def reset(self, indices: Optional[MaskOrIndices] = None, *, copy: bool = True):
"""
Reset the hidden states, if the contained module is a recurrent network.
Args:
indices: Optionally a sequence of integers or a sequence of
booleans, specifying which networks' states will be
reset. If left as None, then the states of all the networks
will be reset.
copy: When `indices` is given as something other than None,
if `copy` is given as True, then the resetting will NOT
be done in-place. Instead, a new copy of the hidden state
will first be created, and then the specified regions
of this new copy will be cleared, and then finally this
modified copy will be declared as the new hidden state.
It is a common practice for recurrent neural network
implementations to return the same tensor both as its
output and as (part of) its hidden state. With `copy=False`,
the resetting would be done in-place, and the action
tensor could be involuntarily reset as well.
This in-place modification could cause silent bugs
if the unintended modification on the action tensor
happens BEFORE the action is sent to the reinforcement
learning environment.
To prevent such situations, the default value for the argument
`copy` is True.
"""
if indices is None:
self.__state = None
else:
if self.__state is not None:
with torch.no_grad():
if copy:
self.__state = deepcopy(self.__state)
reset_tensors(self.__state, indices)
@property
def parameters(self) -> torch.Tensor:
"""
The currently used parameters.
"""
return self.__parameters
@property
def h(self) -> Optional[torch.Tensor]:
"""
The hidden state of the contained recurrent network, if any.
If the contained recurrent network did not generate a hidden state
yet, or if the contained network is not recurrent, then the result
will be None.
"""
return self.__state
@property
def parameter_length(self) -> int:
"""
Length of the parameter tensor.
"""
return self.__fmodule.parameter_length
@property
def wrapped_module(self) -> nn.Module:
"""
The wrapped `torch.nn.Module` instance.
"""
return self.__module
def to_torch_module(self, parameter_vector: torch.Tensor) -> nn.Module:
"""
Get a copy of the contained network, parameterized as specified.
Args:
parameter_vector: The parameters to be used by the new network.
Returns:
Copy of the contained network, as a `torch.nn.Module` instance.
"""
with torch.no_grad():
net = deepcopy(self.__module).to(parameter_vector.device)
nnu.vector_to_parameters(parameter_vector, net.parameters())
return net
h: Optional[torch.Tensor]
property
readonly
¶
The hidden state of the contained recurrent network, if any.
If the contained recurrent network did not generate a hidden state yet, or if the contained network is not recurrent, then the result will be None.
parameter_length: int
property
readonly
¶
Length of the parameter tensor.
parameters: Tensor
property
readonly
¶
The currently used parameters.
wrapped_module: Module
property
readonly
¶
The wrapped torch.nn.Module instance.
__call__(self, x)
special
¶
Pass the given observations through the network.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Tensor |
The observations, as a PyTorch tensor.
If the parameters were given (via the method
|
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The output tensor, which represents the action to take. |
Source code in evotorch/neuroevolution/net/vecrl.py
def __call__(self, x: torch.Tensor) -> torch.Tensor:
"""
Pass the given observations through the network.
Args:
x: The observations, as a PyTorch tensor.
If the parameters were given (via the method
`set_parameters(...)`) as a 1-dimensional tensor, then this
argument is expected to store a single observation.
If the parameters were given as a 2-dimensional tensor,
then, this argument is expected to store a batch of
observations, and the leftmost size of this observation
tensor must match with the leftmost size of the parameter
tensor.
Returns:
The output tensor, which represents the action to take.
"""
if self.__parameters is None:
raise ValueError("Please use the method `set_parameters(...)` before calling the policy.")
if self.__state is None:
further_args = (x,)
else:
further_args = (x, self.__state)
parameters = self.__parameters
ndim = parameters.ndim
if ndim == 1:
result = self.__fmodule(parameters, *further_args)
elif ndim == 2:
vmapped = vmap(self.__fmodule)
result = vmapped(parameters, *further_args)
else:
raise ValueError(
f"Expected the parameters as a 1 or 2 dimensional tensor."
f" However, the received parameters tensor has {ndim} dimensions."
)
if isinstance(result, torch.Tensor):
return result
elif isinstance(result, tuple):
result, state = result
self.__state = state
return result
else:
raise TypeError(f"The torch module used by the Policy returned an unexpected object: {result}")
__init__(self, net, **kwargs)
special
¶
__init__(...): Initialize the Policy.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
net |
Union[str, Callable, torch.nn.modules.module.Module] |
The network to be wrapped by the Policy object.
This can be a string, a Callable (e.g. a |
required |
kwargs |
Expected in the form of additional keyword arguments,
these keyword arguments will be passed to the provided
Callable object (if the argument |
{} |
Source code in evotorch/neuroevolution/net/vecrl.py
def __init__(self, net: Union[str, Callable, nn.Module], **kwargs):
"""
`__init__(...)`: Initialize the Policy.
Args:
net: The network to be wrapped by the Policy object.
This can be a string, a Callable (e.g. a `torch.nn.Module`
subclass), or a `torch.nn.Module` instance.
When this argument is a string, the network will be
created with the help of the function
`evotorch.neuroevolution.net.str_to_net(...)` and then
wrapped. Please see the `str_to_net(...)` function's
documentation for details regarding how a network structure
can be expressed via strings.
kwargs: Expected in the form of additional keyword arguments,
these keyword arguments will be passed to the provided
Callable object (if the argument `net` is a Callable)
or to `str_to_net(...)` (if the argument `net` is a string)
at the moment of generating the network.
If the argument `net` is a `torch.nn.Module` instance,
having any additional keyword arguments will trigger an
error, because the network is already instantiated and
therefore, it is not possible to pass these keyword arguments.
"""
from ..net import str_to_net
from ..net.functional import ModuleExpectingFlatParameters, make_functional_module
if isinstance(net, str):
self.__module = str_to_net(net, **kwargs)
elif isinstance(net, nn.Module):
if len(kwargs) > 0:
raise ValueError(
f"When the network is given as an `nn.Module` instance, extra network arguments cannot be used"
f" (because the network is already instantiated)."
f" However, these extra keyword arguments were received: {kwargs}."
)
self.__module = net
elif isinstance(net, Callable):
self.__module = net(**kwargs)
else:
raise TypeError(
f"The class `Policy` expected a string or an `nn.Module` instance, or a Callable, but received {net}"
f" (whose type is {type(net)})."
)
self.__fmodule: ModuleExpectingFlatParameters = make_functional_module(self.__module)
self.__state: Any = None
self.__parameters: Optional[torch.Tensor] = None
reset(self, indices=None, *, copy=True)
¶
Reset the hidden states, if the contained module is a recurrent network.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
indices |
Union[int, Iterable] |
Optionally a sequence of integers or a sequence of booleans, specifying which networks' states will be reset. If left as None, then the states of all the networks will be reset. |
None |
copy |
bool |
When |
True |
Source code in evotorch/neuroevolution/net/vecrl.py
def reset(self, indices: Optional[MaskOrIndices] = None, *, copy: bool = True):
"""
Reset the hidden states, if the contained module is a recurrent network.
Args:
indices: Optionally a sequence of integers or a sequence of
booleans, specifying which networks' states will be
reset. If left as None, then the states of all the networks
will be reset.
copy: When `indices` is given as something other than None,
if `copy` is given as True, then the resetting will NOT
be done in-place. Instead, a new copy of the hidden state
will first be created, and then the specified regions
of this new copy will be cleared, and then finally this
modified copy will be declared as the new hidden state.
It is a common practice for recurrent neural network
implementations to return the same tensor both as its
output and as (part of) its hidden state. With `copy=False`,
the resetting would be done in-place, and the action
tensor could be involuntarily reset as well.
This in-place modification could cause silent bugs
if the unintended modification on the action tensor
happens BEFORE the action is sent to the reinforcement
learning environment.
To prevent such situations, the default value for the argument
`copy` is True.
"""
if indices is None:
self.__state = None
else:
if self.__state is not None:
with torch.no_grad():
if copy:
self.__state = deepcopy(self.__state)
reset_tensors(self.__state, indices)
set_parameters(self, parameters, indices=None, *, reset=True)
¶
Set the parameters of the policy.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
parameters |
Tensor |
A 1-dimensional or a 2-dimensional tensor containing
the flattened parameters to be used with the neural network.
If the given parameters are two-dimensional, then, given that
the leftmost size of the parameter tensor is |
required |
indices |
Union[int, Iterable] |
For when the parameters were previously given via a
2-dimensional tensor, provide this argument if you would like
to change only some rows of the previously given parameters.
For example, if |
None |
reset |
bool |
If given as True, the hidden states of the networks whose
parameters just changed will be reset. If |
True |
Source code in evotorch/neuroevolution/net/vecrl.py
def set_parameters(self, parameters: torch.Tensor, indices: Optional[MaskOrIndices] = None, *, reset: bool = True):
"""
Set the parameters of the policy.
Args:
parameters: A 1-dimensional or a 2-dimensional tensor containing
the flattened parameters to be used with the neural network.
If the given parameters are two-dimensional, then, given that
the leftmost size of the parameter tensor is `n`, the
observations will be expected in a batch with leftmost size
`n`, and the returned actions will also be in a batch,
again with the leftmost size `n`.
indices: For when the parameters were previously given via a
2-dimensional tensor, provide this argument if you would like
to change only some rows of the previously given parameters.
For example, if `indices` is given as `torch.tensor([2, 4])`
and the argument `parameters` is given as a 2-dimensional
tensor with leftmost size 2, then the rows with indices
2 and 4 will be replaced by these new parameters provided
via the argument `parameters`.
reset: If given as True, the hidden states of the networks whose
parameters just changed will be reset. If `indices` was not
provided at all, then this means that the parameters of all
networks are modified, in which case, all the hidden states
will be reset.
If given as False, no such resetting will be done.
"""
if self.__parameters is None:
if indices is not None:
raise ValueError(
"The argument `indices` can be used only if network parameters were previously specified."
" However, it seems that the method `set_parameters(...)` was not called before."
)
self.__parameters = parameters
else:
if indices is None:
self.__parameters = parameters
else:
self.__parameters[indices] = parameters
if reset:
self.reset(indices)
to_torch_module(self, parameter_vector)
¶
Get a copy of the contained network, parameterized as specified.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
parameter_vector |
Tensor |
The parameters to be used by the new network. |
required |
Returns:
| Type | Description |
|---|---|
Module |
Copy of the contained network, as a |
Source code in evotorch/neuroevolution/net/vecrl.py
def to_torch_module(self, parameter_vector: torch.Tensor) -> nn.Module:
"""
Get a copy of the contained network, parameterized as specified.
Args:
parameter_vector: The parameters to be used by the new network.
Returns:
Copy of the contained network, as a `torch.nn.Module` instance.
"""
with torch.no_grad():
net = deepcopy(self.__module).to(parameter_vector.device)
nnu.vector_to_parameters(parameter_vector, net.parameters())
return net
SyncVectorEnv (BaseVectorEnv)
¶
A vectorized gymnasium environment for handling multiple sub-environments.
This is an alternative implementation to the class gymnasium.vector.SyncVectorEnv.
This alternative SyncVectorEnv implementation has eager auto-reset.
After taking a step(), any sub-environment whose terminated or truncated signal is True will be immediately subject to resetting, and the returned observation and info will immediately reflect the first state of the new episode. This is compatible with the auto-reset behavior of gymnasium 0.29.x, and is different from the auto-reset behavior introduced in gymnasium 1.x.
Source code in evotorch/neuroevolution/net/vecrl.py
class SyncVectorEnv(BaseVectorEnv):
"""
A vectorized gymnasium environment for handling multiple sub-environments.
This is an alternative implementation to the class `gymnasium.vector.SyncVectorEnv`.
This alternative SyncVectorEnv implementation has _eager_ auto-reset.
After taking a step(), any sub-environment whose terminated or truncated
signal is True will be immediately subject to resetting, and the returned
observation and info will immediately reflect the first state of the new
episode. This is compatible with the auto-reset behavior of gymnasium 0.29.x,
and is different from the auto-reset behavior introduced in gymnasium 1.x.
"""
def __init__(
self,
env_makers: Iterable[gym.Env],
*,
empty_info: bool = False,
num_episodes: Optional[int] = None,
device: Optional[Union[str, torch.device]] = None,
):
"""
`__init__(...)`: Initialize the `SyncVectorEnv`.
Args:
env_makers: An iterable object which stores functions that make
the sub-environments to be managed by this `SyncVectorEnv`.
The number of functions within this iterable object
determines the number of sub-environments that will be
managed.
empty_info: Whether or not to ignore the actual `info` dictionaries
of the sub-environments and report empty `info` dictionaries
instead. The default is False. Set this as True if you are not
interested in additional `info`s, and if you wish to save some
computational cycles by not merging the separate `info`
dictionaries into a single dictionary.
num_episodes: Optionally an integer which represents the number
of episodes one wishes to run for each sub-environment.
If this `num_episodes` is given as a positive integer `n`,
each sub-environment will be subject to auto-reset `n-1` times.
After its number of environments is run out, a sub-environment
will keep reporting that it is both terminated and truncated,
its observations will consist of dummy values (`nan` for
`float`-typed observations, 0 for `int`-typed observations),
and its rewards will be `nan`. The internal episode counter
for the sub-environments will be reset when the `reset()`
method of `SyncVectorEnv` is called.
If `num_episodes` is left as None, auto-reset behavior will
be enabled indefinitely.
device: Optionally the device on which the observations, rewards,
terminated and truncated booleans and info arrays will be
reported. Please note that the sub-environments are always
expected with a numpy interface. This argument is used only for
optionally converting the sub-environments' state arrays to
PyTorch tensors on the target device. If this is left as None,
the reported arrays will be numpy arrays. If this is given as a
string or as a `torch.device`, the reported arrays will be
PyTorch tensors on the specified device.
"""
self.__envs: Sequence[gym.Env] = [env_maker() for env_maker in env_makers]
num_envs = len(self.__envs)
if num_envs == 0:
raise ValueError(
"At least one sub-environment was expected, but got an empty collection of sub-environments."
)
self.__empty_info = bool(empty_info)
self.__device = device
single_observation_space = None
single_action_space = None
for i_env, env in enumerate(self.__envs):
if i_env == 0:
single_observation_space = env.observation_space
if not isinstance(single_observation_space, Box):
raise TypeError(
f"Expected a Box-typed observation space, but encountered {single_observation_space}."
)
single_action_space = env.action_space
_must_be_supported_space(single_action_space)
else:
if env.observation_space.shape != single_observation_space.shape:
raise ValueError("The observation shapes of the sub-environments do not match")
if isinstance(env.action_space, Discrete):
if not isinstance(single_action_space, Discrete):
raise TypeError("The action space types of the sub-environments do not match")
if env.action_space.n != single_action_space.n:
raise ValueError("The discrete numbers of actions of the sub-environments do not match")
elif isinstance(env.action_space, Box):
if not isinstance(single_action_space, Box):
raise TypeError("The action space types of the sub-environments do not match")
if env.observation_space.shape != single_observation_space.shape:
raise ValueError("The action space shapes of the sub-environments do not match")
else:
assert False, "Code execution should not have reached here. This is most probably a bug."
self.__batched_obs_shape = (num_envs,) + single_observation_space.shape
self.__batched_obs_dtype = single_observation_space.dtype
self.__random_state: Optional[np.random.RandomState] = None
if num_episodes is None:
self.__num_episodes = None
self.__num_episodes_counter = None
self.__dummy_observation = None
else:
self.__num_episodes = int(num_episodes)
if self.__num_episodes <= 0:
raise ValueError(f"Expected `num_episodes` as a positive integer, but its value is {num_episodes}")
self.__dummy_observation = np.zeros(single_observation_space.shape, dtype=single_observation_space.dtype)
if "float" in str(self.__dummy_observation.dtype):
self.__dummy_observation[:] = float("nan")
self.__num_episodes_counter = np.ones(num_envs, dtype=int)
super().__init__(num_envs, single_observation_space, single_action_space)
def __pop_seed_kwargs(self) -> list:
if self.__random_state is None:
return [{} for _ in range(self.num_envs)]
else:
seeds = self.__random_state.randint(0, 2**32, self.num_envs)
result = [{"seed": int(seed_integer)} for seed_integer in seeds]
self.__random_state = None
return result
def __move_to_target_device(
self,
data: Union[np.ndarray, torch.Tensor, dict],
) -> Union[np.ndarray, torch.Tensor, dict]:
from numbers import Real
if self.__device is None:
return data
def move(x: object) -> object:
if isinstance(x, (Real, bool, np.bool_, torch.Tensor, np.ndarray)):
return torch.as_tensor(x, device=self.__device)
else:
return x
if isinstance(data, dict):
return {k: move(v) for k, v in data.items()}
else:
return move(data)
def __move_each_to_target_device(self, *args) -> tuple:
return tuple(self.__move_to_target_device(x) for x in args)
def seed(self, seed_integer: Optional[int] = None):
"""
Prepare an internal random number generator to be used by the next `reset()`.
In more details, if an integer is given via the argument `seed_integer`,
an internal random number generator (of type `numpy.random.RandomState`)
will be instantiated with `seed_integer` as its seed. Then, the next time
`reset()` is called, each sub-environment will be given a sub-seed, each
sub-seed being a new integer generated from this internal random number
generator. Once this operation is complete, the internal random generator
is destroyed, so that the remaining reset operations will continue to
be randomized according to the sub-environment-specific generators.
On the other hand, if the argument `seed_integer` is given as `None`,
the internal random number generator will be destroyed, meaning that the
next call to `reset()` will reset each sub-environment without specifying
any sub-seed at all.
As an alternative, one can also provide a seed as a positional argument
to `reset()`. The following two usages are equivalent:
```python
vec_env = SyncVectorEnv(
[function_to_make_a_single_env() for _ in range(number_of_sub_envs)]
)
# Usage 1 (calling seed and reset separately):
vec_env.seed(an_integer)
vec_env.reset()
# Usage 2 (calling reset with a seed argument):
vec_env.reset(seed=an_integer)
```
Args:
seed_integer: An integer if you wish each sub-environment to be
randomized via a pseudo-random generator seeded by this given
integer. Otherwise, this can be left as None.
"""
if seed_integer is None:
self.__random_state = None
else:
self.__random_state = np.random.RandomState(seed_integer)
def reset(self, **kwargs) -> tuple:
"""
Reset each sub-environment.
Any keyword argument other than `seed` will be sent directly to the
`reset(...)` methods of the underlying sub-environments.
If, among the keyword arguments, there is `seed`, the value for this
`seed` keyword argument will be expected either as None, or as an integer.
The setting `seed=None` can be used if the user wishes to ensure that
there will be no explicit seeding when resetting the sub-environments
(even when the `seed(...)` method of `SyncVectorEnv` was called
previously with an explicit seed integer).
The setting `seed=S`, where `S` is an integer, causes the following
steps to be executed:
(i) prepare a temporary random number generator with seed `S`;
(ii) from the temporary random number generator, generate `N` sub-seed
integers where `N` is the number of sub-environments;
(iii) reset each sub-environment with a sub-seed;
(iv) destroy the temporary random number generator.
Args:
kwargs: Keyword arguments to be passed to the `reset()` methods
of the underlying sub-environments. The keyword `seed` will be
intercepted and treated specially.
Returns:
A tuple of the form `(observation, info)`, where `observation` is
a numpy array storing the observations of all the sub-environments
(where the leftmost dimension is the batch dimension), and `info`
is the `info` dictionary. If possible, the values within the
`info` dictionary will be combined to single numpy arrays as well.
If this `SyncVectorEnv` was initialized with a `device`, the
results will be in the form of PyTorch tensors on the specified device.
"""
if "seed" in kwargs:
self.seed(kwargs["seed"])
remaining_kwargs = {k: v for k, v in kwargs.items() if k != "seed"}
else:
remaining_kwargs = kwargs
if self.__num_episodes is not None:
self.__num_episodes_counter[:] = self.__num_episodes
seed_kwargs_list = self.__pop_seed_kwargs()
observations = []
infos = []
for env, seed_kwargs in zip(self.__envs, seed_kwargs_list):
observation, info = env.reset(**seed_kwargs, **remaining_kwargs)
observations.append(observation)
if not self.__empty_info:
infos.append(info)
if self.__empty_info:
batched_info = {}
else:
batched_info = _batch_info_dicts(infos)
return self.__move_each_to_target_device(np.stack(observations), batched_info)
def step(self, action: Union[torch.Tensor, np.ndarray]) -> tuple: # noqa: C901
"""
Take a step within each sub-environment.
Args:
action: A numpy array or a PyTorch tensor that contains the action.
The size of the leftmost dimension of this array or tensor
is expected to be equal to the number of sub-environments.
Returns:
A tuple of the form (`observation`, `reward`, `terminated`,
`truncated`, `info`) where `observation` is an array or tensor
storing the observations of the sub-environments, `reward`
is an array or tensor storing the rewards, `terminated` is an
array or tensor of booleans stating whether or not the
sub-environments got reset because of termination,
`truncated` is an array or tensor of booleans stating whether or
not the sub-environments got reset because of truncation, and
`info` is a dictionary storing any additional information
regarding the states of the sub-environments.
If this `SyncVectorEnv` was initialized with a `device`, the
results will be in the form of PyTorch tensors on the specified
device.
"""
if isinstance(action, torch.Tensor):
action = action.cpu().numpy()
else:
action = np.asarray(action)
if action.ndim == 0:
raise ValueError("The action array must be at least 1-dimensional")
batch_size = action.shape[0]
if batch_size != self.num_envs:
raise ValueError("The leftmost dimension of the action array does not match the number of sub-environments")
batched_obs_shape = self.__batched_obs_shape
batched_obs_dtype = self.__batched_obs_dtype
num_envs = self.num_envs
if self.__empty_info:
initialized_info = {}
else:
initialized_info = [None for _ in range(num_envs)]
class per_env:
observation = np.zeros(batched_obs_shape, dtype=batched_obs_dtype)
reward = np.zeros(num_envs, dtype=float)
terminated = np.zeros(num_envs, dtype=bool)
truncated = np.zeros(num_envs, dtype=bool)
info = initialized_info
def is_active_env(env_index: int) -> bool:
if self.__num_episodes is None:
return True
return self.__num_episodes_counter[env_index] > 0
def is_last_episode(env_index: int) -> bool:
if self.__num_episodes is None:
return False
return self.__num_episodes_counter[env_index] == 1
def decrement_episode_counter(env_index: int):
if self.__num_episodes is None:
return
self.__num_episodes_counter[env_index] -= 1
def apply_step(env_index: int, single_action: Union[np.ndarray, np.generic, Number, bool]) -> tuple:
if not is_active_env(env_index):
return self.__dummy_observation, float("nan"), True, True, {}
env = self.__envs[env_index]
observation, reward, terminated, truncated, info = env.step(single_action)
if terminated or truncated:
was_last_episode = is_last_episode(env_index)
decrement_episode_counter(env_index)
obs_after_reset, info_after_reset = env.reset()
if not was_last_episode:
observation = obs_after_reset
info = info_after_reset
return observation, reward, terminated, truncated, info
for i_env in range(len(self.__envs)):
# observation, reward, terminated, truncated, info = self.__envs[i_env].step(action[i_env])
# done = terminated | truncated
# if done:
# observation, info = self.__envs[i_env].reset()
observation, reward, terminated, truncated, info = apply_step(i_env, action[i_env])
per_env.observation[i_env] = observation
per_env.reward[i_env] = reward
per_env.terminated[i_env] = terminated
per_env.truncated[i_env] = truncated
if not self.__empty_info:
per_env.info[i_env] = info
if not self.__empty_info:
per_env.info = _batch_info_dicts(per_env.info)
return self.__move_each_to_target_device(
per_env.observation,
per_env.reward,
per_env.terminated,
per_env.truncated,
per_env.info,
)
def render(self, *args, **kwargs):
"""
Does not do anything, ignores its arguments, and returns None.
"""
pass
def close(self):
"""
Close each sub-environment.
"""
for env in self.__envs:
env.close()
__init__(self, env_makers, *, empty_info=False, num_episodes=None, device=None)
special
¶
__init__(...): Initialize the SyncVectorEnv.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
env_makers |
Iterable[gymnasium.core.Env] |
An iterable object which stores functions that make
the sub-environments to be managed by this |
required |
empty_info |
bool |
Whether or not to ignore the actual |
False |
num_episodes |
Optional[int] |
Optionally an integer which represents the number
of episodes one wishes to run for each sub-environment.
If this |
None |
device |
Union[str, torch.device] |
Optionally the device on which the observations, rewards,
terminated and truncated booleans and info arrays will be
reported. Please note that the sub-environments are always
expected with a numpy interface. This argument is used only for
optionally converting the sub-environments' state arrays to
PyTorch tensors on the target device. If this is left as None,
the reported arrays will be numpy arrays. If this is given as a
string or as a |
None |
Source code in evotorch/neuroevolution/net/vecrl.py
def __init__(
self,
env_makers: Iterable[gym.Env],
*,
empty_info: bool = False,
num_episodes: Optional[int] = None,
device: Optional[Union[str, torch.device]] = None,
):
"""
`__init__(...)`: Initialize the `SyncVectorEnv`.
Args:
env_makers: An iterable object which stores functions that make
the sub-environments to be managed by this `SyncVectorEnv`.
The number of functions within this iterable object
determines the number of sub-environments that will be
managed.
empty_info: Whether or not to ignore the actual `info` dictionaries
of the sub-environments and report empty `info` dictionaries
instead. The default is False. Set this as True if you are not
interested in additional `info`s, and if you wish to save some
computational cycles by not merging the separate `info`
dictionaries into a single dictionary.
num_episodes: Optionally an integer which represents the number
of episodes one wishes to run for each sub-environment.
If this `num_episodes` is given as a positive integer `n`,
each sub-environment will be subject to auto-reset `n-1` times.
After its number of environments is run out, a sub-environment
will keep reporting that it is both terminated and truncated,
its observations will consist of dummy values (`nan` for
`float`-typed observations, 0 for `int`-typed observations),
and its rewards will be `nan`. The internal episode counter
for the sub-environments will be reset when the `reset()`
method of `SyncVectorEnv` is called.
If `num_episodes` is left as None, auto-reset behavior will
be enabled indefinitely.
device: Optionally the device on which the observations, rewards,
terminated and truncated booleans and info arrays will be
reported. Please note that the sub-environments are always
expected with a numpy interface. This argument is used only for
optionally converting the sub-environments' state arrays to
PyTorch tensors on the target device. If this is left as None,
the reported arrays will be numpy arrays. If this is given as a
string or as a `torch.device`, the reported arrays will be
PyTorch tensors on the specified device.
"""
self.__envs: Sequence[gym.Env] = [env_maker() for env_maker in env_makers]
num_envs = len(self.__envs)
if num_envs == 0:
raise ValueError(
"At least one sub-environment was expected, but got an empty collection of sub-environments."
)
self.__empty_info = bool(empty_info)
self.__device = device
single_observation_space = None
single_action_space = None
for i_env, env in enumerate(self.__envs):
if i_env == 0:
single_observation_space = env.observation_space
if not isinstance(single_observation_space, Box):
raise TypeError(
f"Expected a Box-typed observation space, but encountered {single_observation_space}."
)
single_action_space = env.action_space
_must_be_supported_space(single_action_space)
else:
if env.observation_space.shape != single_observation_space.shape:
raise ValueError("The observation shapes of the sub-environments do not match")
if isinstance(env.action_space, Discrete):
if not isinstance(single_action_space, Discrete):
raise TypeError("The action space types of the sub-environments do not match")
if env.action_space.n != single_action_space.n:
raise ValueError("The discrete numbers of actions of the sub-environments do not match")
elif isinstance(env.action_space, Box):
if not isinstance(single_action_space, Box):
raise TypeError("The action space types of the sub-environments do not match")
if env.observation_space.shape != single_observation_space.shape:
raise ValueError("The action space shapes of the sub-environments do not match")
else:
assert False, "Code execution should not have reached here. This is most probably a bug."
self.__batched_obs_shape = (num_envs,) + single_observation_space.shape
self.__batched_obs_dtype = single_observation_space.dtype
self.__random_state: Optional[np.random.RandomState] = None
if num_episodes is None:
self.__num_episodes = None
self.__num_episodes_counter = None
self.__dummy_observation = None
else:
self.__num_episodes = int(num_episodes)
if self.__num_episodes <= 0:
raise ValueError(f"Expected `num_episodes` as a positive integer, but its value is {num_episodes}")
self.__dummy_observation = np.zeros(single_observation_space.shape, dtype=single_observation_space.dtype)
if "float" in str(self.__dummy_observation.dtype):
self.__dummy_observation[:] = float("nan")
self.__num_episodes_counter = np.ones(num_envs, dtype=int)
super().__init__(num_envs, single_observation_space, single_action_space)
close(self)
¶
render(self, *args, **kwargs)
¶
reset(self, **kwargs)
¶
Reset each sub-environment.
Any keyword argument other than seed will be sent directly to the
reset(...) methods of the underlying sub-environments.
If, among the keyword arguments, there is seed, the value for this
seed keyword argument will be expected either as None, or as an integer.
The setting seed=None can be used if the user wishes to ensure that
there will be no explicit seeding when resetting the sub-environments
(even when the seed(...) method of SyncVectorEnv was called
previously with an explicit seed integer).
The setting seed=S, where S is an integer, causes the following
steps to be executed:
(i) prepare a temporary random number generator with seed S;
(ii) from the temporary random number generator, generate N sub-seed
integers where N is the number of sub-environments;
(iii) reset each sub-environment with a sub-seed;
(iv) destroy the temporary random number generator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
kwargs |
Keyword arguments to be passed to the |
{} |
Returns:
| Type | Description |
|---|---|
tuple |
A tuple of the form |
Source code in evotorch/neuroevolution/net/vecrl.py
def reset(self, **kwargs) -> tuple:
"""
Reset each sub-environment.
Any keyword argument other than `seed` will be sent directly to the
`reset(...)` methods of the underlying sub-environments.
If, among the keyword arguments, there is `seed`, the value for this
`seed` keyword argument will be expected either as None, or as an integer.
The setting `seed=None` can be used if the user wishes to ensure that
there will be no explicit seeding when resetting the sub-environments
(even when the `seed(...)` method of `SyncVectorEnv` was called
previously with an explicit seed integer).
The setting `seed=S`, where `S` is an integer, causes the following
steps to be executed:
(i) prepare a temporary random number generator with seed `S`;
(ii) from the temporary random number generator, generate `N` sub-seed
integers where `N` is the number of sub-environments;
(iii) reset each sub-environment with a sub-seed;
(iv) destroy the temporary random number generator.
Args:
kwargs: Keyword arguments to be passed to the `reset()` methods
of the underlying sub-environments. The keyword `seed` will be
intercepted and treated specially.
Returns:
A tuple of the form `(observation, info)`, where `observation` is
a numpy array storing the observations of all the sub-environments
(where the leftmost dimension is the batch dimension), and `info`
is the `info` dictionary. If possible, the values within the
`info` dictionary will be combined to single numpy arrays as well.
If this `SyncVectorEnv` was initialized with a `device`, the
results will be in the form of PyTorch tensors on the specified device.
"""
if "seed" in kwargs:
self.seed(kwargs["seed"])
remaining_kwargs = {k: v for k, v in kwargs.items() if k != "seed"}
else:
remaining_kwargs = kwargs
if self.__num_episodes is not None:
self.__num_episodes_counter[:] = self.__num_episodes
seed_kwargs_list = self.__pop_seed_kwargs()
observations = []
infos = []
for env, seed_kwargs in zip(self.__envs, seed_kwargs_list):
observation, info = env.reset(**seed_kwargs, **remaining_kwargs)
observations.append(observation)
if not self.__empty_info:
infos.append(info)
if self.__empty_info:
batched_info = {}
else:
batched_info = _batch_info_dicts(infos)
return self.__move_each_to_target_device(np.stack(observations), batched_info)
seed(self, seed_integer=None)
¶
Prepare an internal random number generator to be used by the next reset().
In more details, if an integer is given via the argument seed_integer,
an internal random number generator (of type numpy.random.RandomState)
will be instantiated with seed_integer as its seed. Then, the next time
reset() is called, each sub-environment will be given a sub-seed, each
sub-seed being a new integer generated from this internal random number
generator. Once this operation is complete, the internal random generator
is destroyed, so that the remaining reset operations will continue to
be randomized according to the sub-environment-specific generators.
On the other hand, if the argument seed_integer is given as None,
the internal random number generator will be destroyed, meaning that the
next call to reset() will reset each sub-environment without specifying
any sub-seed at all.
As an alternative, one can also provide a seed as a positional argument
to reset(). The following two usages are equivalent:
vec_env = SyncVectorEnv(
[function_to_make_a_single_env() for _ in range(number_of_sub_envs)]
)
# Usage 1 (calling seed and reset separately):
vec_env.seed(an_integer)
vec_env.reset()
# Usage 2 (calling reset with a seed argument):
vec_env.reset(seed=an_integer)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
seed_integer |
Optional[int] |
An integer if you wish each sub-environment to be randomized via a pseudo-random generator seeded by this given integer. Otherwise, this can be left as None. |
None |
Source code in evotorch/neuroevolution/net/vecrl.py
def seed(self, seed_integer: Optional[int] = None):
"""
Prepare an internal random number generator to be used by the next `reset()`.
In more details, if an integer is given via the argument `seed_integer`,
an internal random number generator (of type `numpy.random.RandomState`)
will be instantiated with `seed_integer` as its seed. Then, the next time
`reset()` is called, each sub-environment will be given a sub-seed, each
sub-seed being a new integer generated from this internal random number
generator. Once this operation is complete, the internal random generator
is destroyed, so that the remaining reset operations will continue to
be randomized according to the sub-environment-specific generators.
On the other hand, if the argument `seed_integer` is given as `None`,
the internal random number generator will be destroyed, meaning that the
next call to `reset()` will reset each sub-environment without specifying
any sub-seed at all.
As an alternative, one can also provide a seed as a positional argument
to `reset()`. The following two usages are equivalent:
```python
vec_env = SyncVectorEnv(
[function_to_make_a_single_env() for _ in range(number_of_sub_envs)]
)
# Usage 1 (calling seed and reset separately):
vec_env.seed(an_integer)
vec_env.reset()
# Usage 2 (calling reset with a seed argument):
vec_env.reset(seed=an_integer)
```
Args:
seed_integer: An integer if you wish each sub-environment to be
randomized via a pseudo-random generator seeded by this given
integer. Otherwise, this can be left as None.
"""
if seed_integer is None:
self.__random_state = None
else:
self.__random_state = np.random.RandomState(seed_integer)
step(self, action)
¶
Take a step within each sub-environment.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
action |
Union[torch.Tensor, numpy.ndarray] |
A numpy array or a PyTorch tensor that contains the action. The size of the leftmost dimension of this array or tensor is expected to be equal to the number of sub-environments. |
required |
Returns:
| Type | Description |
|---|---|
tuple |
A tuple of the form ( |
Source code in evotorch/neuroevolution/net/vecrl.py
def step(self, action: Union[torch.Tensor, np.ndarray]) -> tuple: # noqa: C901
"""
Take a step within each sub-environment.
Args:
action: A numpy array or a PyTorch tensor that contains the action.
The size of the leftmost dimension of this array or tensor
is expected to be equal to the number of sub-environments.
Returns:
A tuple of the form (`observation`, `reward`, `terminated`,
`truncated`, `info`) where `observation` is an array or tensor
storing the observations of the sub-environments, `reward`
is an array or tensor storing the rewards, `terminated` is an
array or tensor of booleans stating whether or not the
sub-environments got reset because of termination,
`truncated` is an array or tensor of booleans stating whether or
not the sub-environments got reset because of truncation, and
`info` is a dictionary storing any additional information
regarding the states of the sub-environments.
If this `SyncVectorEnv` was initialized with a `device`, the
results will be in the form of PyTorch tensors on the specified
device.
"""
if isinstance(action, torch.Tensor):
action = action.cpu().numpy()
else:
action = np.asarray(action)
if action.ndim == 0:
raise ValueError("The action array must be at least 1-dimensional")
batch_size = action.shape[0]
if batch_size != self.num_envs:
raise ValueError("The leftmost dimension of the action array does not match the number of sub-environments")
batched_obs_shape = self.__batched_obs_shape
batched_obs_dtype = self.__batched_obs_dtype
num_envs = self.num_envs
if self.__empty_info:
initialized_info = {}
else:
initialized_info = [None for _ in range(num_envs)]
class per_env:
observation = np.zeros(batched_obs_shape, dtype=batched_obs_dtype)
reward = np.zeros(num_envs, dtype=float)
terminated = np.zeros(num_envs, dtype=bool)
truncated = np.zeros(num_envs, dtype=bool)
info = initialized_info
def is_active_env(env_index: int) -> bool:
if self.__num_episodes is None:
return True
return self.__num_episodes_counter[env_index] > 0
def is_last_episode(env_index: int) -> bool:
if self.__num_episodes is None:
return False
return self.__num_episodes_counter[env_index] == 1
def decrement_episode_counter(env_index: int):
if self.__num_episodes is None:
return
self.__num_episodes_counter[env_index] -= 1
def apply_step(env_index: int, single_action: Union[np.ndarray, np.generic, Number, bool]) -> tuple:
if not is_active_env(env_index):
return self.__dummy_observation, float("nan"), True, True, {}
env = self.__envs[env_index]
observation, reward, terminated, truncated, info = env.step(single_action)
if terminated or truncated:
was_last_episode = is_last_episode(env_index)
decrement_episode_counter(env_index)
obs_after_reset, info_after_reset = env.reset()
if not was_last_episode:
observation = obs_after_reset
info = info_after_reset
return observation, reward, terminated, truncated, info
for i_env in range(len(self.__envs)):
# observation, reward, terminated, truncated, info = self.__envs[i_env].step(action[i_env])
# done = terminated | truncated
# if done:
# observation, info = self.__envs[i_env].reset()
observation, reward, terminated, truncated, info = apply_step(i_env, action[i_env])
per_env.observation[i_env] = observation
per_env.reward[i_env] = reward
per_env.terminated[i_env] = terminated
per_env.truncated[i_env] = truncated
if not self.__empty_info:
per_env.info[i_env] = info
if not self.__empty_info:
per_env.info = _batch_info_dicts(per_env.info)
return self.__move_each_to_target_device(
per_env.observation,
per_env.reward,
per_env.terminated,
per_env.truncated,
per_env.info,
)
TorchWrapper
¶
A wrapper for vectorized or non-vectorized gymnasium environments.
This wrapper ensures that the actions, observations, rewards, and the 'done' values are expressed as PyTorch tensors.
Please note that TorchWrapper does not inherit neither from
gymnasium.Wrapper, nor from gymnasium.vector.VectorEnvWrapper.
Once an environment is wrapped via TorchWrapper, it is NOT
recommended to further wrap it via other types of wrappers.
Source code in evotorch/neuroevolution/net/vecrl.py
class TorchWrapper:
"""
A wrapper for vectorized or non-vectorized gymnasium environments.
This wrapper ensures that the actions, observations, rewards, and
the 'done' values are expressed as PyTorch tensors.
Please note that `TorchWrapper` does not inherit neither from
`gymnasium.Wrapper`, nor from `gymnasium.vector.VectorEnvWrapper`.
Once an environment is wrapped via `TorchWrapper`, it is NOT
recommended to further wrap it via other types of wrappers.
"""
def __init__(
self,
env: Union[gym.Env, gym.vector.VectorEnv, "TorchWrapper"],
*,
force_classic_api: bool = False,
discrete_to_continuous_act: bool = False,
clip_actions: bool = False,
# **kwargs,
):
"""
`__init__(...)`: Initialize the TorchWrapper.
Args:
env: The gymnasium environment to be wrapped.
force_classic_api: Set this as True if you would like to enable
the classic API. In the classic API, the `reset(...)` method
returns only the observation and the `step(...)` method
returns 4 elements (not 5).
discrete_to_continuous_act: When this is set as True and the
wrapped environment has a Discrete action space, this wrapper
will transform the action space to Box. A Discrete-action
environment with `n` actions will be converted to a Box-action
environment where the action length is `n`.
The index of the largest value within the action vector will
be applied to the underlying environment.
clip_actions: Set this as True if you would like to clip the given
actions so that they conform to the declared boundaries of the
action space.
"""
# super().__init__(env, **kwargs)
self.env = env
self.observation_space = env.observation_space
self.action_space = env.action_space
# Declare the variable that will store the array type of the underlying environment.
self.__array_type: Optional[str] = None
if hasattr(env.unwrapped, "single_observation_space"):
# If the underlying environment has the attribute "single_observation_space",
# then this is a vectorized environment.
self.__vectorized = True
# Get the observation and action spaces.
obs_space = _unbatch_space(env.observation_space)
act_space = _unbatch_space(env.action_space)
self.single_observation_space = obs_space
self.single_action_space = act_space
self.num_envs = env.unwrapped.num_envs
else:
# If the underlying environment has the attribute "single_observation_space",
# then this is a non-vectorized environment.
self.__vectorized = False
# Get the observation and action spaces.
obs_space = env.observation_space
act_space = env.action_space
# Ensure that the observation and action spaces are supported.
_must_be_supported_space(obs_space)
_must_be_supported_space(act_space)
# Store the choice of the user regarding "force_classic_api".
self.__force_classic_api = bool(force_classic_api)
if isinstance(act_space, Discrete) and discrete_to_continuous_act:
# The underlying action space is Discrete and `discrete_to_continuous_act` is given as True.
# Therefore, we convert the action space to continuous (to Box).
# Take the shape and the dtype of the discrete action space.
single_action_shape = (act_space.n,)
single_action_dtype = torch.from_numpy(np.array([], dtype=act_space.dtype)).dtype
# We store the integer dtype of the environment.
self.__discrete_dtype = single_action_dtype
if self.__vectorized:
# If the environment is vectorized, we declare the new `action_space` and the `single_action_space`
# for the enviornment.
action_shape = (env.num_envs,) + single_action_shape
self.single_action_space = Box(float("-inf"), float("inf"), shape=single_action_shape, dtype=np.float32)
self.action_space = Box(float("-inf"), float("inf"), shape=action_shape, dtype=np.float32)
else:
# If the environment is not vectorized, we declare the new `action_space` for the environment.
self.action_space = Box(float("-inf"), float("inf"), shape=single_action_shape, dtype=np.float32)
else:
# This is the case where we do not transform the action space.
# The discrete dtype will not be used, so, we set it as None.
self.__discrete_dtype = None
if isinstance(act_space, Box) and clip_actions:
# If the action space is Box and the wrapper is configured to clip the actions, then we store the lower
# and the upper bounds for the actions.
self.__act_lb = torch.from_numpy(act_space.low)
self.__act_ub = torch.from_numpy(act_space.high)
else:
# If there will not be any action clipping, then we store the lower and the upper bounds as None.
self.__act_lb = None
self.__act_ub = None
@property
def array_type(self) -> Optional[str]:
"""
Get the array type of the wrapped environment.
This can be "jax", "torch", or "numpy".
"""
return self.__array_type
def __infer_array_type(self, observation):
if self.__array_type is None:
# If the array type is not determined yet, set it as the array type of the received observation.
# If the observation has an unrecognized type, set the array type as "numpy".
self.__array_type = array_type(observation, "numpy")
def reset(self, *args, **kwargs):
"""Reset the environment"""
# Call the reset method of the wrapped environment.
reset_result = self.env.reset(*args, **kwargs)
if isinstance(reset_result, tuple):
# If we received a tuple of two elements, then we assume that this is the new gym API.
# We note that we received an info dictionary.
got_info = True
# We keep the received observation and info.
observation, info = reset_result
else:
# If we did not receive a tuple, then we assume that this is the old gym API.
# We note that we did not receive an info dictionary.
got_info = False
# We keep the received observation.
observation = reset_result
# We did not receive an info dictionary, so, we set it as an empty dictionary.
info = {}
# We understand the array type of the underlying environment from the first observation.
self.__infer_array_type(observation)
# Convert the observation to a PyTorch tensor.
observation = convert_to_torch(observation)
if self.__force_classic_api:
# If the option `force_classic_api` was set as True, then we only return the observation.
return observation
else:
# Here we handle the case where `force_classic_api` was set as False.
if got_info:
# If we got an additional info dictionary, we return it next to the observation.
return observation, info
else:
# If we did not get any info dictionary, we return only the observation.
return observation
def step(self, action, *args, **kwargs):
"""Take a step in the environment"""
if self.__array_type is None:
# If the array type is not known yet, then probably `reset()` has not been called yet.
# We raise an error.
raise ValueError(
"Could not understand what type of array this environment works with."
" Perhaps the `reset()` method has not been called yet?"
)
if self.__discrete_dtype is not None:
# If the wrapped environment is discrete-actioned, then we take the integer counterpart of the action.
action = torch.argmax(action, dim=-1).to(dtype=self.__discrete_dtype)
if self.__act_lb is not None:
# The internal variable `__act_lb` having a value other than None means that the initialization argument
# `clip_actions` was given as True.
# Therefore, we clip the actions.
self.__act_lb = self.__act_lb.to(action.device)
self.__act_ub = self.__act_ub.to(action.device)
action = torch.max(action, self.__act_lb)
action = torch.min(action, self.__act_ub)
# Convert the action tensor to the expected array type of the underlying environment.
action = convert_from_torch(action, self.__array_type)
# Perform the step and get the result.
result = self.env.step(action, *args, **kwargs)
if not isinstance(result, tuple):
# If the `step(...)` method returned anything other than tuple, we raise an error.
raise TypeError(f"Expected a tuple as the result of the `step()` method, but received a {type(result)}")
if len(result) == 5:
# If the result is a tuple of 5 elements, then we note that we are using the new API.
using_new_api = True
# Take the observation, reward, two boolean variables done and done2 indicating that the episode(s)
# has/have ended, and additional info.
# `done` indicates whether or not the episode(s) reached terminal state(s).
# `done2` indicates whether or not the episode(s) got truncated because of the timestep limit.
observation, reward, done, done2, info = result
elif len(result) == 4:
# If the result is a tuple of 4 elements, then we note that we are not using the new API.
using_new_api = False
# Take the observation, reward, the done boolean flag, and additional info.
observation, reward, done, info = result
done2 = None
else:
raise ValueError(f"Unexpected number of elements were returned from step(): {len(result)}")
# Convert the observation, reward, and done variables to PyTorch tensors.
observation = convert_to_torch(observation)
reward = convert_to_torch(reward)
done = convert_to_torch_bool(done)
if done2 is not None:
done2 = convert_to_torch_bool(done2)
if self.__force_classic_api:
# This is the case where the initialization argument `force_classic_api` was set as True.
if done2 is not None:
# We combine the terminal state and truncation signals into a single boolean tensor indicating
# whether or not the episode(s) ended.
done = done | done2
# Return 4 elements, compatible with the classic gym API.
return observation, reward, done, info
else:
# This is the case where the initialization argument `force_classic_api` was set as False.
if using_new_api:
# If we are using the new API, then we return the 5-element result.
return observation, reward, done, done2, info
else:
# If we are using the new API, then we return the 4-element result.
return observation, reward, done, info
def seed(self, *args, **kwargs) -> Any:
return self.env.seed(*args, **kwargs)
def render(self, *args, **kwargs) -> Any:
return self.env.render(*args, **kwargs)
def close(self, *args, **kwargs) -> Any:
return self.env.close(*args, **kwargs)
@property
def unwrapped(self) -> Union[gym.Env, gym.vector.VectorEnv]:
return self.env.unwrapped
array_type: Optional[str]
property
readonly
¶
Get the array type of the wrapped environment. This can be "jax", "torch", or "numpy".
__init__(self, env, *, force_classic_api=False, discrete_to_continuous_act=False, clip_actions=False)
special
¶
__init__(...): Initialize the TorchWrapper.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
env |
Union[gymnasium.core.Env, gymnasium.vector.vector_env.VectorEnv, TorchWrapper] |
The gymnasium environment to be wrapped. |
required |
force_classic_api |
bool |
Set this as True if you would like to enable
the classic API. In the classic API, the |
False |
discrete_to_continuous_act |
bool |
When this is set as True and the
wrapped environment has a Discrete action space, this wrapper
will transform the action space to Box. A Discrete-action
environment with |
False |
clip_actions |
bool |
Set this as True if you would like to clip the given actions so that they conform to the declared boundaries of the action space. |
False |
Source code in evotorch/neuroevolution/net/vecrl.py
def __init__(
self,
env: Union[gym.Env, gym.vector.VectorEnv, "TorchWrapper"],
*,
force_classic_api: bool = False,
discrete_to_continuous_act: bool = False,
clip_actions: bool = False,
# **kwargs,
):
"""
`__init__(...)`: Initialize the TorchWrapper.
Args:
env: The gymnasium environment to be wrapped.
force_classic_api: Set this as True if you would like to enable
the classic API. In the classic API, the `reset(...)` method
returns only the observation and the `step(...)` method
returns 4 elements (not 5).
discrete_to_continuous_act: When this is set as True and the
wrapped environment has a Discrete action space, this wrapper
will transform the action space to Box. A Discrete-action
environment with `n` actions will be converted to a Box-action
environment where the action length is `n`.
The index of the largest value within the action vector will
be applied to the underlying environment.
clip_actions: Set this as True if you would like to clip the given
actions so that they conform to the declared boundaries of the
action space.
"""
# super().__init__(env, **kwargs)
self.env = env
self.observation_space = env.observation_space
self.action_space = env.action_space
# Declare the variable that will store the array type of the underlying environment.
self.__array_type: Optional[str] = None
if hasattr(env.unwrapped, "single_observation_space"):
# If the underlying environment has the attribute "single_observation_space",
# then this is a vectorized environment.
self.__vectorized = True
# Get the observation and action spaces.
obs_space = _unbatch_space(env.observation_space)
act_space = _unbatch_space(env.action_space)
self.single_observation_space = obs_space
self.single_action_space = act_space
self.num_envs = env.unwrapped.num_envs
else:
# If the underlying environment has the attribute "single_observation_space",
# then this is a non-vectorized environment.
self.__vectorized = False
# Get the observation and action spaces.
obs_space = env.observation_space
act_space = env.action_space
# Ensure that the observation and action spaces are supported.
_must_be_supported_space(obs_space)
_must_be_supported_space(act_space)
# Store the choice of the user regarding "force_classic_api".
self.__force_classic_api = bool(force_classic_api)
if isinstance(act_space, Discrete) and discrete_to_continuous_act:
# The underlying action space is Discrete and `discrete_to_continuous_act` is given as True.
# Therefore, we convert the action space to continuous (to Box).
# Take the shape and the dtype of the discrete action space.
single_action_shape = (act_space.n,)
single_action_dtype = torch.from_numpy(np.array([], dtype=act_space.dtype)).dtype
# We store the integer dtype of the environment.
self.__discrete_dtype = single_action_dtype
if self.__vectorized:
# If the environment is vectorized, we declare the new `action_space` and the `single_action_space`
# for the enviornment.
action_shape = (env.num_envs,) + single_action_shape
self.single_action_space = Box(float("-inf"), float("inf"), shape=single_action_shape, dtype=np.float32)
self.action_space = Box(float("-inf"), float("inf"), shape=action_shape, dtype=np.float32)
else:
# If the environment is not vectorized, we declare the new `action_space` for the environment.
self.action_space = Box(float("-inf"), float("inf"), shape=single_action_shape, dtype=np.float32)
else:
# This is the case where we do not transform the action space.
# The discrete dtype will not be used, so, we set it as None.
self.__discrete_dtype = None
if isinstance(act_space, Box) and clip_actions:
# If the action space is Box and the wrapper is configured to clip the actions, then we store the lower
# and the upper bounds for the actions.
self.__act_lb = torch.from_numpy(act_space.low)
self.__act_ub = torch.from_numpy(act_space.high)
else:
# If there will not be any action clipping, then we store the lower and the upper bounds as None.
self.__act_lb = None
self.__act_ub = None
reset(self, *args, **kwargs)
¶
Reset the environment
Source code in evotorch/neuroevolution/net/vecrl.py
def reset(self, *args, **kwargs):
"""Reset the environment"""
# Call the reset method of the wrapped environment.
reset_result = self.env.reset(*args, **kwargs)
if isinstance(reset_result, tuple):
# If we received a tuple of two elements, then we assume that this is the new gym API.
# We note that we received an info dictionary.
got_info = True
# We keep the received observation and info.
observation, info = reset_result
else:
# If we did not receive a tuple, then we assume that this is the old gym API.
# We note that we did not receive an info dictionary.
got_info = False
# We keep the received observation.
observation = reset_result
# We did not receive an info dictionary, so, we set it as an empty dictionary.
info = {}
# We understand the array type of the underlying environment from the first observation.
self.__infer_array_type(observation)
# Convert the observation to a PyTorch tensor.
observation = convert_to_torch(observation)
if self.__force_classic_api:
# If the option `force_classic_api` was set as True, then we only return the observation.
return observation
else:
# Here we handle the case where `force_classic_api` was set as False.
if got_info:
# If we got an additional info dictionary, we return it next to the observation.
return observation, info
else:
# If we did not get any info dictionary, we return only the observation.
return observation
step(self, action, *args, **kwargs)
¶
Take a step in the environment
Source code in evotorch/neuroevolution/net/vecrl.py
def step(self, action, *args, **kwargs):
"""Take a step in the environment"""
if self.__array_type is None:
# If the array type is not known yet, then probably `reset()` has not been called yet.
# We raise an error.
raise ValueError(
"Could not understand what type of array this environment works with."
" Perhaps the `reset()` method has not been called yet?"
)
if self.__discrete_dtype is not None:
# If the wrapped environment is discrete-actioned, then we take the integer counterpart of the action.
action = torch.argmax(action, dim=-1).to(dtype=self.__discrete_dtype)
if self.__act_lb is not None:
# The internal variable `__act_lb` having a value other than None means that the initialization argument
# `clip_actions` was given as True.
# Therefore, we clip the actions.
self.__act_lb = self.__act_lb.to(action.device)
self.__act_ub = self.__act_ub.to(action.device)
action = torch.max(action, self.__act_lb)
action = torch.min(action, self.__act_ub)
# Convert the action tensor to the expected array type of the underlying environment.
action = convert_from_torch(action, self.__array_type)
# Perform the step and get the result.
result = self.env.step(action, *args, **kwargs)
if not isinstance(result, tuple):
# If the `step(...)` method returned anything other than tuple, we raise an error.
raise TypeError(f"Expected a tuple as the result of the `step()` method, but received a {type(result)}")
if len(result) == 5:
# If the result is a tuple of 5 elements, then we note that we are using the new API.
using_new_api = True
# Take the observation, reward, two boolean variables done and done2 indicating that the episode(s)
# has/have ended, and additional info.
# `done` indicates whether or not the episode(s) reached terminal state(s).
# `done2` indicates whether or not the episode(s) got truncated because of the timestep limit.
observation, reward, done, done2, info = result
elif len(result) == 4:
# If the result is a tuple of 4 elements, then we note that we are not using the new API.
using_new_api = False
# Take the observation, reward, the done boolean flag, and additional info.
observation, reward, done, info = result
done2 = None
else:
raise ValueError(f"Unexpected number of elements were returned from step(): {len(result)}")
# Convert the observation, reward, and done variables to PyTorch tensors.
observation = convert_to_torch(observation)
reward = convert_to_torch(reward)
done = convert_to_torch_bool(done)
if done2 is not None:
done2 = convert_to_torch_bool(done2)
if self.__force_classic_api:
# This is the case where the initialization argument `force_classic_api` was set as True.
if done2 is not None:
# We combine the terminal state and truncation signals into a single boolean tensor indicating
# whether or not the episode(s) ended.
done = done | done2
# Return 4 elements, compatible with the classic gym API.
return observation, reward, done, info
else:
# This is the case where the initialization argument `force_classic_api` was set as False.
if using_new_api:
# If we are using the new API, then we return the 5-element result.
return observation, reward, done, done2, info
else:
# If we are using the new API, then we return the 4-element result.
return observation, reward, done, info
array_type(x, fallback=None)
¶
Get the type of an array as a string ("jax", "torch", or "numpy"). If the type of the array cannot be determined and a fallback is provided, then the fallback value will be returned.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
The array whose type will be determined. |
required |
fallback |
Optional[str] |
Fallback value, as a string, which will be returned if the array type cannot be determined. |
None |
Returns:
| Type | Description |
|---|---|
str |
The array type as a string ("jax", "torch", or "numpy"). |
Exceptions:
| Type | Description |
|---|---|
TypeError |
if the array type cannot be determined and a fallback value is not provided. |
Source code in evotorch/neuroevolution/net/vecrl.py
def array_type(x: Any, fallback: Optional[str] = None) -> str:
"""
Get the type of an array as a string ("jax", "torch", or "numpy").
If the type of the array cannot be determined and a fallback is provided,
then the fallback value will be returned.
Args:
x: The array whose type will be determined.
fallback: Fallback value, as a string, which will be returned if the
array type cannot be determined.
Returns:
The array type as a string ("jax", "torch", or "numpy").
Raises:
TypeError: if the array type cannot be determined and a fallback
value is not provided.
"""
if is_jax_array(x):
return "jax"
elif isinstance(x, torch.Tensor):
return "torch"
elif isinstance(x, np.ndarray):
return "numpy"
elif fallback is not None:
return fallback
else:
raise TypeError(f"The object has an unrecognized type: {type(x)}")
convert_from_torch(x, array_type)
¶
Convert the given PyTorch tensor to an array of the specified type.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Tensor |
The PyTorch array that will be converted. |
required |
array_type |
str |
Type to which the PyTorch tensor will be converted. Expected as one of these strings: "jax", "torch", "numpy". |
required |
Returns:
| Type | Description |
|---|---|
Any |
The array of the specified type. Can be a JAX array, a numpy array, or PyTorch tensor. |
Exceptions:
| Type | Description |
|---|---|
ValueError |
if the array type cannot be determined. |
Source code in evotorch/neuroevolution/net/vecrl.py
def convert_from_torch(x: torch.Tensor, array_type: str) -> Any:
"""
Convert the given PyTorch tensor to an array of the specified type.
Args:
x: The PyTorch array that will be converted.
array_type: Type to which the PyTorch tensor will be converted.
Expected as one of these strings: "jax", "torch", "numpy".
Returns:
The array of the specified type. Can be a JAX array, a numpy array,
or PyTorch tensor.
Raises:
ValueError: if the array type cannot be determined.
"""
if array_type == "torch":
return x
elif array_type == "jax":
return torch_to_jax(x)
elif array_type == "numpy":
return x.cpu().numpy()
else:
raise ValueError(f"Unrecognized array type: {array_type}")
convert_to_torch(x)
¶
Convert the given array to PyTorch tensor.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
Array to be converted. Can be a JAX array, a numpy array, a PyTorch tensor (in which case the input tensor will be returned as it is) or any Iterable object. |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The PyTorch counterpart of the given array. |
Source code in evotorch/neuroevolution/net/vecrl.py
def convert_to_torch(x: Any) -> torch.Tensor:
"""
Convert the given array to PyTorch tensor.
Args:
x: Array to be converted. Can be a JAX array, a numpy array,
a PyTorch tensor (in which case the input tensor will be
returned as it is) or any Iterable object.
Returns:
The PyTorch counterpart of the given array.
"""
if isinstance(x, torch.Tensor):
return x
elif is_jax_array(x):
return jax_to_torch(x)
elif isinstance(x, np.ndarray):
return torch.from_numpy(x)
else:
return torch.as_tensor(x)
convert_to_torch_bool(x)
¶
Convert the given array to a PyTorch tensor of bools.
If the given object is an array of floating point numbers, then, values that are near to 0.0 (with a tolerance of 1e-4) will be converted to False, and the others will be converted to True. If the given object is an array of integers, then zero values will be converted to False, and non-zero values will be converted to True. If the given object is an array of booleans, then no change will be made to those boolean values.
The given object can be a JAX array, a numpy array, or a PyTorch tensor. The result will always be a PyTorch tensor.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
Array to be converted. |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The array converted to a PyTorch tensor with its dtype set as bool. |
Source code in evotorch/neuroevolution/net/vecrl.py
def convert_to_torch_bool(x: Any) -> torch.Tensor:
"""
Convert the given array to a PyTorch tensor of bools.
If the given object is an array of floating point numbers, then, values
that are near to 0.0 (with a tolerance of 1e-4) will be converted to
False, and the others will be converted to True.
If the given object is an array of integers, then zero values will be
converted to False, and non-zero values will be converted to True.
If the given object is an array of booleans, then no change will be made
to those boolean values.
The given object can be a JAX array, a numpy array, or a PyTorch tensor.
The result will always be a PyTorch tensor.
Args:
x: Array to be converted.
Returns:
The array converted to a PyTorch tensor with its dtype set as bool.
"""
x = convert_to_torch(x)
if x.dtype == torch.bool:
pass # nothing to do
elif "float" in str(x.dtype):
x = torch.abs(x) > 1e-4
else:
x = torch.as_tensor(x, dtype=torch.bool)
return x
make_brax_env(env_name, *, force_classic_api=False, num_envs=None, discrete_to_continuous_act=False, clip_actions=False, **kwargs)
¶
Make a brax environment and wrap it via TorchWrapper.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
env_name |
str |
Name of the brax environment, as string (e.g. "humanoid").
If the string starts with "old::" (e.g. "old::humanoid", etc.),
then the environment will be made using the namespace |
required |
force_classic_api |
bool |
Whether or not the classic gym API is to be used. |
False |
num_envs |
Optional[int] |
Batch size for the vectorized environment. |
None |
discrete_to_continuous_act |
bool |
Whether or not the the discrete action space of the environment is to be converted to a continuous one. This does nothing if the environment's action space is not discrete. |
False |
clip_actions |
bool |
Whether or not the actions should be explicitly clipped so that they stay within the declared action boundaries. |
False |
kwargs |
Expected in the form of additional keyword arguments, these are passed to the environment. |
{} |
Returns:
| Type | Description |
|---|---|
TorchWrapper |
The brax environment, wrapped by TorchWrapper. |
Source code in evotorch/neuroevolution/net/vecrl.py
def make_brax_env(
env_name: str,
*,
force_classic_api: bool = False,
num_envs: Optional[int] = None,
discrete_to_continuous_act: bool = False,
clip_actions: bool = False,
**kwargs,
) -> TorchWrapper:
"""
Make a brax environment and wrap it via TorchWrapper.
Args:
env_name: Name of the brax environment, as string (e.g. "humanoid").
If the string starts with "old::" (e.g. "old::humanoid", etc.),
then the environment will be made using the namespace `brax.v1`
(which was introduced in brax version 0.9.0 where the updated
implementations of the environments became default and the classical
ones moved into `brax.v1`).
You can use the prefix "old::" for reproducing previous results
that were obtained or reported using an older version of brax.
force_classic_api: Whether or not the classic gym API is to be used.
num_envs: Batch size for the vectorized environment.
discrete_to_continuous_act: Whether or not the the discrete action
space of the environment is to be converted to a continuous one.
This does nothing if the environment's action space is not
discrete.
clip_actions: Whether or not the actions should be explicitly clipped
so that they stay within the declared action boundaries.
kwargs: Expected in the form of additional keyword arguments, these
are passed to the environment.
Returns:
The brax environment, wrapped by TorchWrapper.
"""
if brax is not None:
config = {}
config.update(kwargs)
if num_envs is not None:
config["num_envs"] = num_envs
env = VectorEnvFromBrax(env_name, **config)
env = TorchWrapper(
env,
force_classic_api=force_classic_api,
discrete_to_continuous_act=discrete_to_continuous_act,
clip_actions=clip_actions,
)
return env
else:
_brax_is_missing()
make_gym_env(env_name, *, force_classic_api=False, num_envs=None, discrete_to_continuous_act=False, clip_actions=False, empty_info=False, num_episodes=None, device=None, **kwargs)
¶
Make gymnasium environment(s) and wrap them via a TorchWrapper.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
env_name |
str |
Name of the gymnasium environment, as string (e.g. "Humanoid-v4"). |
required |
force_classic_api |
bool |
Whether or not the classic gym API is to be used. |
False |
num_envs |
Optional[int] |
Optionally a batch size for the vectorized environment.
If given as an integer, the environment will be instantiated multiple
times, and then wrapped via |
None |
discrete_to_continuous_act |
bool |
Whether or not the the discrete action space of the environment is to be converted to a continuous one. This does nothing if the environment's action space is not discrete. |
False |
clip_actions |
bool |
Whether or not the actions should be explicitly clipped so that they stay within the declared action boundaries. |
False |
empty_info |
bool |
Whether or not to ignore the info dictionaries of the
sub-environments and always return an empty dictionary for the
extra info. This feature is only available when |
False |
num_episodes |
Optional[int] |
Optionally an integer which specifies the number of
episodes each sub-environment will run for. Until its number of
episodes run out, each sub-environment will be subject to
auto-reset. Alternatively, |
None |
device |
Union[str, torch.device] |
Optionally the device on which the state(s) of the environment(s)
will be reported. If None, the reported arrays of the underlying
environment(s) will be unchanged. If given as a |
None |
kwargs |
Expected in the form of additional keyword arguments, these are passed to the environment. |
{} |
Returns:
| Type | Description |
|---|---|
TorchWrapper |
The gymnasium environments, wrapped by a TorchWrapper. |
Source code in evotorch/neuroevolution/net/vecrl.py
def make_gym_env(
env_name: str,
*,
force_classic_api: bool = False,
num_envs: Optional[int] = None,
discrete_to_continuous_act: bool = False,
clip_actions: bool = False,
empty_info: bool = False,
num_episodes: Optional[int] = None,
device: Optional[Union[str, torch.device]] = None,
**kwargs,
) -> TorchWrapper:
"""
Make gymnasium environment(s) and wrap them via a TorchWrapper.
Args:
env_name: Name of the gymnasium environment, as string (e.g. "Humanoid-v4").
force_classic_api: Whether or not the classic gym API is to be used.
num_envs: Optionally a batch size for the vectorized environment.
If given as an integer, the environment will be instantiated multiple
times, and then wrapped via `SyncVectorEnv`.
discrete_to_continuous_act: Whether or not the the discrete action
space of the environment is to be converted to a continuous one.
This does nothing if the environment's action space is not
discrete.
clip_actions: Whether or not the actions should be explicitly clipped
so that they stay within the declared action boundaries.
empty_info: Whether or not to ignore the info dictionaries of the
sub-environments and always return an empty dictionary for the
extra info. This feature is only available when `num_envs` is given
as an integer. If `num_envs` is None, `empty_info` should be left as
False.
num_episodes: Optionally an integer which specifies the number of
episodes each sub-environment will run for. Until its number of
episodes run out, each sub-environment will be subject to
auto-reset. Alternatively, `num_episodes` can be left as None,
which means that the sub-environments will be subject to auto-reset
indefinitely.
Please note that this feature can be used only when `num_envs` is
given as an integer (i.e. when we work with a batch of
environments). When `num_envs` is None, `num_episodes` is expected
as None as well.
device: Optionally the device on which the state(s) of the environment(s)
will be reported. If None, the reported arrays of the underlying
environment(s) will be unchanged. If given as a `torch.device` or as
a string, the reported arrays will be converted to PyTorch tensors
and then moved to this specified device.
This feature is only available when `num_envs` is given as an
integer. If `num_envs` is None, `device` should also be None.
kwargs: Expected in the form of additional keyword arguments, these
are passed to the environment.
Returns:
The gymnasium environments, wrapped by a TorchWrapper.
"""
def make_the_env():
return gym.make(env_name, **kwargs)
if num_envs is None:
if empty_info:
raise ValueError(
f"The argument `empty_info` was received as {repr(empty_info)}."
" The `empty_info` behavior can be turned on only when `num_envs` is not None."
" However, `num_envs` was received as None."
)
if num_episodes is not None:
raise ValueError(
f"The argument `num_episodes` was received as {repr(num_episodes)}."
" The `num_episodes` behavior can be turned on only when `num_envs` is not None."
" However, `num_envs` was received as None."
)
if device is not None:
raise ValueError(
f"The argument `device` was received as {repr(device)}."
" Having a target device is supported only when `num_envs` is not None."
" However, `num_envs` was received as None."
)
to_be_wrapped = make_the_env()
else:
to_be_wrapped = SyncVectorEnv(
[make_the_env for _ in range(num_envs)],
empty_info=empty_info,
num_episodes=num_episodes,
device=device,
)
vec_env = TorchWrapper(
to_be_wrapped,
force_classic_api=force_classic_api,
discrete_to_continuous_act=discrete_to_continuous_act,
clip_actions=clip_actions,
)
return vec_env
make_vector_env(env_name, *, force_classic_api=False, num_envs=None, discrete_to_continuous_act=False, clip_actions=False, gym_kwargs=None, brax_kwargs=None, **kwargs)
¶
Make a new vectorized environment and wrap it via TorchWrapper.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
env_name |
str |
Name of the environment, as string.
If the string starts with "gym::" (e.g. "gym::Humanoid-v4", etc.),
then it is assumed that the target environment is a traditional
non-vectorized gymnasium environment. This non-vectorized
will first be duplicated and wrapped via a |
required |
force_classic_api |
bool |
Whether or not the classic gym API is to be used. |
False |
num_envs |
Optional[int] |
Batch size for the vectorized environment. |
None |
discrete_to_continuous_act |
bool |
Whether or not the the discrete action space of the environment is to be converted to a continuous one. This does nothing if the environment's action space is not discrete. |
False |
clip_actions |
bool |
Whether or not the actions should be explicitly clipped so that they stay within the declared action boundaries. |
False |
gym_kwargs |
Optional[dict] |
Keyword arguments to pass only if the environment is a classical gymnasium environment. |
None |
brax_kwargs |
Optional[dict] |
Keyword arguments to pass only if the environment is a brax environment. |
None |
kwargs |
Expected in the form of additional keyword arguments, these are passed to the environment. |
{} |
Returns:
| Type | Description |
|---|---|
TorchWrapper |
The vectorized gymnasium environment, wrapped by TorchWrapper. |
Source code in evotorch/neuroevolution/net/vecrl.py
def make_vector_env(
env_name: str,
*,
force_classic_api: bool = False,
num_envs: Optional[int] = None,
discrete_to_continuous_act: bool = False,
clip_actions: bool = False,
gym_kwargs: Optional[dict] = None,
brax_kwargs: Optional[dict] = None,
**kwargs,
) -> TorchWrapper:
"""
Make a new vectorized environment and wrap it via TorchWrapper.
Args:
env_name: Name of the environment, as string.
If the string starts with "gym::" (e.g. "gym::Humanoid-v4", etc.),
then it is assumed that the target environment is a traditional
non-vectorized gymnasium environment. This non-vectorized
will first be duplicated and wrapped via a `SyncVectorEnv` so that
it gains a vectorized interface, and then, it will be wrapped via
`TorchWrapper`.
If the string starts with "brax::" (e.g. "brax::humanoid", etc.),
then it is assumed that the target environment is a brax
environment which will be wrapped via TorchWrapper.
If the string starts with "brax::old::" (e.g.
"brax::old::humanoid", etc.), then the environment will be made
using the namespace `brax.v1` (which was introduced in brax
version 0.9.0 where the updated implementations of the environments
became default and the classical ones moved into `brax.v1`).
You can use the prefix "brax::old::" for reproducing previous
results that were obtained or reported using an older version of
brax.
If the string does not contain "::" at all (e.g. "Humanoid-v4"),
then it is assumed that the target environment is a gymnasium
environment. Therefore, "gym::Humanoid-v4" and "Humanoid-v4"
are equivalent.
force_classic_api: Whether or not the classic gym API is to be used.
num_envs: Batch size for the vectorized environment.
discrete_to_continuous_act: Whether or not the the discrete action
space of the environment is to be converted to a continuous one.
This does nothing if the environment's action space is not
discrete.
clip_actions: Whether or not the actions should be explicitly clipped
so that they stay within the declared action boundaries.
gym_kwargs: Keyword arguments to pass only if the environment is a
classical gymnasium environment.
brax_kwargs: Keyword arguments to pass only if the environment is a
brax environment.
kwargs: Expected in the form of additional keyword arguments, these
are passed to the environment.
Returns:
The vectorized gymnasium environment, wrapped by TorchWrapper.
"""
env_parts = str(env_name).split("::", maxsplit=1)
if gym_kwargs is None:
gym_kwargs = {}
if brax_kwargs is None:
brax_kwargs = {}
kwargs_to_pass = {}
kwargs_to_pass.update(kwargs)
if len(env_parts) == 0:
raise ValueError(f"Invalid value for `env_name`: {repr(env_name)}")
elif len(env_parts) == 1:
fn = make_gym_env
kwargs_to_pass.update(gym_kwargs)
elif len(env_parts) == 2:
env_name = env_parts[1]
if env_parts[0] == "gym":
fn = make_gym_env
kwargs_to_pass.update(gym_kwargs)
elif env_parts[0] == "brax":
fn = make_brax_env
kwargs_to_pass.update(brax_kwargs)
else:
invalid_value = env_parts[0] + "::"
raise ValueError(
f"The argument `env_name` starts with {repr(invalid_value)}, implying that the environment is stored"
f" in a registry named {repr(env_parts[0])}."
f" However, the registry {repr(env_parts[0])} is not recognized."
f" Supported environment registries are: 'gym', 'brax'."
)
else:
assert False, "Unexpected value received from len(env_parts)"
return fn(
env_name,
force_classic_api=force_classic_api,
num_envs=num_envs,
discrete_to_continuous_act=discrete_to_continuous_act,
clip_actions=clip_actions,
**kwargs_to_pass,
)
reset_tensors(x, indices)
¶
Reset the specified regions of the given tensor(s) as 0.
Note that the resetting is performed in-place, which means, the provided tensors are modified.
The regions are determined by the argument indices, which can be a sequence of booleans (in which case it is
interpreted as a mask), or a sequence of integers (in which case it is interpreted as the list of indices).
For example, let us imagine that we have the following tensor:
import torch
x = torch.tensor(
[
[0, 1, 2, 3],
[4, 5, 6, 7],
[8, 9, 10, 11],
[12, 13, 14, 15],
],
dtype=torch.float32,
)
If we wish to reset the rows with indices 0 and 2, we could use:
The new value of x would then be:
torch.tensor(
[
[0, 0, 0, 0],
[4, 5, 6, 7],
[0, 0, 0, 0],
[12, 13, 14, 15],
],
dtype=torch.float32,
)
The first argument does not have to be a single tensor. Instead, it can be a container (i.e. a dictionary-like object or an iterable) that stores tensors. In this case, each tensor stored by the container will be subject to resetting. In more details, each tensor within the iterable(s) and each tensor within the value part of the dictionary-like object(s) will be reset.
As an example, let us assume that we have the following collection:
a = torch.tensor(
[
[0, 1],
[2, 3],
[4, 5],
],
dtype=torch.float32,
)
b = torch.tensor(
[
[0, 10, 20],
[30, 40, 50],
[60, 70, 80],
],
dtype=torch.float32,
)
c = torch.tensor(
[
[100],
[200],
[300],
],
dtype=torch.float32,
)
d = torch.tensor([-1, -2, -3], dtype=torch.float32)
my_tensors = [a, {"1": b, "2": (c, d)}]
To clear the regions with indices, e.g, (1, 2), we could do:
and the result would be:
>>> print(a)
torch.tensor(
[
[0, 1],
[0, 0],
[0, 0],
],
dtype=torch.float32,
)
>>> print(b)
torch.tensor(
[
[0, 10, 20],
[0, 0, 0],
[0, 0, 0],
],
dtype=torch.float32,
)
>>> print(c)
c = torch.tensor(
[
[100],
[0],
[0],
],
dtype=torch.float32,
)
>>> print(d)
torch.tensor([-1, 0, 0], dtype=torch.float32)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
A tensor or a collection of tensors, whose values are subject to resetting. |
required |
indices |
Union[int, Iterable] |
A sequence of integers or booleans, specifying which regions of the tensor(s) will be reset. |
required |
Source code in evotorch/neuroevolution/net/vecrl.py
def reset_tensors(x: Any, indices: MaskOrIndices):
"""
Reset the specified regions of the given tensor(s) as 0.
Note that the resetting is performed in-place, which means, the provided tensors are modified.
The regions are determined by the argument `indices`, which can be a sequence of booleans (in which case it is
interpreted as a mask), or a sequence of integers (in which case it is interpreted as the list of indices).
For example, let us imagine that we have the following tensor:
```python
import torch
x = torch.tensor(
[
[0, 1, 2, 3],
[4, 5, 6, 7],
[8, 9, 10, 11],
[12, 13, 14, 15],
],
dtype=torch.float32,
)
```
If we wish to reset the rows with indices 0 and 2, we could use:
```python
reset_tensors(x, [0, 2])
```
The new value of `x` would then be:
```
torch.tensor(
[
[0, 0, 0, 0],
[4, 5, 6, 7],
[0, 0, 0, 0],
[12, 13, 14, 15],
],
dtype=torch.float32,
)
```
The first argument does not have to be a single tensor.
Instead, it can be a container (i.e. a dictionary-like object or an iterable) that stores tensors.
In this case, each tensor stored by the container will be subject to resetting.
In more details, each tensor within the iterable(s) and each tensor within the value part of the dictionary-like
object(s) will be reset.
As an example, let us assume that we have the following collection:
```python
a = torch.tensor(
[
[0, 1],
[2, 3],
[4, 5],
],
dtype=torch.float32,
)
b = torch.tensor(
[
[0, 10, 20],
[30, 40, 50],
[60, 70, 80],
],
dtype=torch.float32,
)
c = torch.tensor(
[
[100],
[200],
[300],
],
dtype=torch.float32,
)
d = torch.tensor([-1, -2, -3], dtype=torch.float32)
my_tensors = [a, {"1": b, "2": (c, d)}]
```
To clear the regions with indices, e.g, (1, 2), we could do:
```python
reset_tensors(my_tensors, [1, 2])
```
and the result would be:
```
>>> print(a)
torch.tensor(
[
[0, 1],
[0, 0],
[0, 0],
],
dtype=torch.float32,
)
>>> print(b)
torch.tensor(
[
[0, 10, 20],
[0, 0, 0],
[0, 0, 0],
],
dtype=torch.float32,
)
>>> print(c)
c = torch.tensor(
[
[100],
[0],
[0],
],
dtype=torch.float32,
)
>>> print(d)
torch.tensor([-1, 0, 0], dtype=torch.float32)
```
Args:
x: A tensor or a collection of tensors, whose values are subject to resetting.
indices: A sequence of integers or booleans, specifying which regions of the tensor(s) will be reset.
"""
if isinstance(x, torch.Tensor):
# If the first argument is a tensor, then we clear it according to the indices we received.
x[indices] = 0
elif isinstance(x, (str, bytes, bytearray)):
# str, bytes, and bytearray are the types of `Iterable` that we do not wish to process.
# Therefore, we explicitly add a condition for them here, and explicitly state that nothing should be done
# when instances of them are encountered.
pass
elif isinstance(x, Mapping):
# If the first argument is a Mapping (i.e. a dictionary-like object), then, for each value part of the
# Mapping instance, we call this function itself.
for key, value in x.items():
reset_tensors(value, indices)
elif isinstance(x, Iterable):
# If the first argument is an Iterable (e.g. a list, a tuple, etc.), then, for each value contained by this
# Iterable instance, we call this function itself.
for value in x:
reset_tensors(value, indices)
supervisedne
¶
SupervisedNE (NEProblem)
¶
Representation of a neuro-evolution problem where the goal is to minimize a loss function in a supervised learning setting.
A supervised learning problem can be defined via subclassing this class
and overriding the methods
_loss(y_hat, y) (which is to define how the loss is computed)
and _make_dataloader() (which is to define how a new DataLoader is
created).
Alternatively, this class can be directly instantiated as follows:
def my_loss_function(output_of_network, desired_output):
loss = ... # compute the loss here
return loss
problem = SupervisedNE(
my_dataset, MyTorchModuleClass, my_loss_function, minibatch_size=..., ...
)
Source code in evotorch/neuroevolution/supervisedne.py
class SupervisedNE(NEProblem):
"""
Representation of a neuro-evolution problem where the goal is to minimize
a loss function in a supervised learning setting.
A supervised learning problem can be defined via subclassing this class
and overriding the methods
`_loss(y_hat, y)` (which is to define how the loss is computed)
and `_make_dataloader()` (which is to define how a new DataLoader is
created).
Alternatively, this class can be directly instantiated as follows:
```python
def my_loss_function(output_of_network, desired_output):
loss = ... # compute the loss here
return loss
problem = SupervisedNE(
my_dataset, MyTorchModuleClass, my_loss_function, minibatch_size=..., ...
)
```
"""
def __init__(
self,
dataset: Dataset,
network: Union[str, nn.Module, Callable[[], nn.Module]],
loss_func: Optional[Callable] = None,
*,
network_args: Optional[dict] = None,
initial_bounds: Optional[BoundsPairLike] = (-0.00001, 0.00001),
minibatch_size: Optional[int] = None,
num_minibatches: Optional[int] = None,
num_actors: Optional[Union[int, str]] = None,
common_minibatch: bool = True,
num_gpus_per_actor: Optional[Union[int, float, str]] = None,
actor_config: Optional[dict] = None,
num_subbatches: Optional[int] = None,
subbatch_size: Optional[int] = None,
device: Optional[Device] = None,
):
"""
`__init__(...)`: Initialize the SupervisedNE.
Args:
dataset: The Dataset from which the minibatches will be pulled
network: A network structure string, or a Callable (which can be
a class inheriting from `torch.nn.Module`, or a function
which returns a `torch.nn.Module` instance), or an instance
of `torch.nn.Module`.
The object provided here determines the structure of the
neural network whose parameters will be evolved.
A network structure string is a string which can be processed
by `evotorch.neuroevolution.net.str_to_net(...)`.
Please see the documentation of the function
`evotorch.neuroevolution.net.str_to_net(...)` to see how such
a neural network structure string looks like.
loss_func: Optionally a function (or a Callable object) which
receives `y_hat` (the output generated by the neural network)
and `y` (the desired output), and returns the loss as a
scalar.
This argument can also be left as None, in which case it will
be expected that the method `_loss(self, y_hat, y)` is
overridden by the inheriting class.
network_args: Optionally a dict-like object, storing keyword
arguments to be passed to the network while instantiating it.
initial_bounds: Specifies an interval from which the values of the
initial neural network parameters will be drawn.
minibatch_size: Optionally an integer, describing the size of a
minibatch when pulling data from the dataset.
Can also be left as None, in which case it will be expected
that the inheriting class overrides the method
`_make_dataloader()` and defines how a new DataLoader is to be
made.
num_minibatches: An integer, specifying over how many minibatches
will a single neural network be evaluated.
If not specified, it will be assumed that the desired number
of minibatches per network evaluation is 1.
num_actors: Number of actors to create for parallelized
evaluation of the solutions.
Certain string values are also accepted.
When given as "max" or as "num_cpus", the number of actors
will be equal to the number of all available CPUs in the ray
cluster.
When given as "num_gpus", the number of actors will be
equal to the number of all available GPUs in the ray
cluster, and each actor will be assigned a GPU.
When given as "num_devices", the number of actors will be
equal to the minimum among the number of CPUs and the number
of GPUs available in the cluster (or will be equal to the
number of CPUs if there is no GPU), and each actor will be
assigned a GPU (if available).
If `num_actors` is given as "num_gpus" or "num_devices",
the argument `num_gpus_per_actor` must not be used,
and the `actor_config` dictionary must not contain the
key "num_gpus".
If `num_actors` is given as something other than "num_gpus"
or "num_devices", and if you wish to assign GPUs to each
actor, then please see the argument `num_gpus_per_actor`.
common_minibatch: Whether the same minibatches will be
used when evaluating the solutions or not.
actor_config: A dictionary, representing the keyword arguments
to be passed to the options(...) used when creating the
ray actor objects. To be used for explicitly allocating
resources per each actor.
For example, for declaring that each actor is to use a GPU,
one can pass `actor_config=dict(num_gpus=1)`.
Can also be given as None (which is the default),
if no such options are to be passed.
num_gpus_per_actor: Number of GPUs to be allocated by each
remote actor.
The default behavior is to NOT allocate any GPU at all
(which is the default behavior of the ray library as well).
When given as a number `n`, each actor will be given
`n` GPUs (where `n` can be an integer, or can be a `float`
for fractional allocation).
When given as a string "max", then the available GPUs
across the entire ray cluster (or within the local computer
in the simplest cases) will be equally distributed among
the actors.
When given as a string "all", then each actor will have
access to all the GPUs (this will be achieved by suppressing
the environment variable `CUDA_VISIBLE_DEVICES` for each
actor).
When the problem is not distributed (i.e. when there are
no actors), this argument is expected to be left as None.
num_subbatches: If `num_subbatches` is None (assuming that
`subbatch_size` is also None), then, when evaluating a
population, the population will be split into n pieces, `n`
being the number of actors, and each actor will evaluate
its assigned piece. If `num_subbatches` is an integer `m`,
then the population will be split into `m` pieces,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
While using a distributed algorithm, this argument determines
how many sub-batches will be generated, and therefore,
how many gradients will be computed by the remote actors.
subbatch_size: If `subbatch_size` is None (assuming that
`num_subbatches` is also None), then, when evaluating a
population, the population will be split into `n` pieces, `n`
being the number of actors, and each actor will evaluate its
assigned piece. If `subbatch_size` is an integer `m`,
then the population will be split into pieces of size `m`,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
When there can be significant difference across the solutions
in terms of computational requirements, specifying a
`subbatch_size` can be beneficial, because, while one
actor is busy with a subbatch containing computationally
challenging solutions, other actors can accept more
tasks and save time.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
While using a distributed algorithm, this argument determines
the size of a sub-batch (or sub-population) sampled by a
remote actor for computing a gradient.
In distributed mode, it is expected that the population size
is divisible by `subbatch_size`.
device: Default device in which a new population will be generated
and the neural networks will operate.
If not specified, "cpu" will be used.
"""
super().__init__(
objective_sense="min",
network=network,
network_args=network_args,
initial_bounds=initial_bounds,
num_actors=num_actors,
num_gpus_per_actor=num_gpus_per_actor,
actor_config=actor_config,
num_subbatches=num_subbatches,
subbatch_size=subbatch_size,
device=device,
)
self.dataset = dataset
self.dataloader: DataLoader = None
self.dataloader_iterator = None
self._loss_func = loss_func
self._minibatch_size = None if minibatch_size is None else int(minibatch_size)
self._num_minibatches = 1 if num_minibatches is None else int(num_minibatches)
self._common_minibatch = common_minibatch
self._current_minibatches: Optional[list] = None
def _make_dataloader(self) -> DataLoader:
"""
Make a new DataLoader.
This method, in its default state, does not contain an implementation.
In the case where the `__init__` of `SupervisedNE` is not provided
with a minibatch size, it will be expected that this method is
overridden by the inheriting class and that the operation of creating
a new DataLoader is defined here.
Returns:
The new DataLoader.
"""
raise NotImplementedError
def make_dataloader(self) -> DataLoader:
"""
Make a new DataLoader.
If the `__init__` of `SupervisedNE` was provided with a minibatch size
via the argument `minibatch_size`, then a new DataLoader will be made
with that minibatch size.
Otherwise, it will be expected that the method `_make_dataloader(...)`
was overridden to contain details regarding how the DataLoader should be
created, and that method will be executed.
Returns:
The created DataLoader.
"""
if self._minibatch_size is None:
return self._make_dataloader()
else:
return DataLoader(self.dataset, shuffle=True, batch_size=self._minibatch_size)
def _evaluate_using_minibatch(self, network: nn.Module, batch: Any) -> Union[float, torch.Tensor]:
"""
Pass a minibatch through a network, and compute the loss.
Args:
network: The network using which the loss will be computed.
batch: The minibatch that will be used as data.
Returns:
The loss.
"""
with torch.no_grad():
x, y = batch
yhat = network(x)
return self.loss(yhat, y)
def _loss(self, y_hat: Any, y: Any) -> Union[float, torch.Tensor]:
"""
The loss function.
This method, in its default state, does not contain an implementation.
In the case where `__init__` of `SupervisedNE` class was not given
a loss function via the argument `loss_func`, it will be expected
that this method is overridden by the inheriting class and that the
operation of computing the loss is defined here.
Args:
y_hat: The output estimated by the network
y: The desired output
Returns:
A scalar, representing the loss
"""
raise NotImplementedError
def loss(self, y_hat: Any, y: Any) -> Union[float, torch.Tensor]:
"""
Run the loss function and return the loss.
If the `__init__` of `SupervisedNE` class was given a loss
function via the argument `loss_func`, then that loss function
will be used. Otherwise, it will be expected that the method
`_loss(...)` is overriden with a loss definition, and that method
will be used to compute the loss.
The computed loss will be returned.
Args:
y_hat: The output estimated by the network
y: The desired output
Returns:
A scalar, representing the loss
"""
if self._loss_func is None:
return self._loss(y_hat, y)
else:
return self._loss_func(y_hat, y)
def _prepare(self) -> None:
self.dataloader = self.make_dataloader()
def get_minibatch(self) -> Any:
"""
Get the next minibatch from the DataLoader.
"""
if self.dataloader is None:
self._prepare()
if self.dataloader_iterator is None:
self.dataloader_iterator = iter(self.dataloader)
batch = None
try:
batch = next(self.dataloader_iterator)
except StopIteration:
pass
if batch is None:
self.dataloader_iterator = iter(self.dataloader)
batch = next(self.dataloader_iterator)
# Move batch to device of network
return [var.to(self.network_device) for var in batch]
def _evaluate_network(self, network: nn.Module) -> torch.Tensor:
loss = 0.0
for batch_idx in range(self._num_minibatches):
if not self._common_minibatch:
self._current_minibatch = self.get_minibatch()
else:
self._current_minibatch = self._current_minibatches[batch_idx]
loss += self._evaluate_using_minibatch(network, self._current_minibatch) / self._num_minibatches
return loss
def _evaluate_batch(self, batch: SolutionBatch):
if self._common_minibatch:
# If using a common data batch, generate them now and use them for the entire batch of solutions
self._current_minibatches = [self.get_minibatch() for _ in range(self._num_minibatches)]
return super()._evaluate_batch(batch)
__init__(self, dataset, network, loss_func=None, *, network_args=None, initial_bounds=(-1e-05, 1e-05), minibatch_size=None, num_minibatches=None, num_actors=None, common_minibatch=True, num_gpus_per_actor=None, actor_config=None, num_subbatches=None, subbatch_size=None, device=None)
special
¶
__init__(...): Initialize the SupervisedNE.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset |
Dataset |
The Dataset from which the minibatches will be pulled |
required |
network |
Union[str, torch.nn.modules.module.Module, Callable[[], torch.nn.modules.module.Module]] |
A network structure string, or a Callable (which can be
a class inheriting from |
required |
loss_func |
Optional[Callable] |
Optionally a function (or a Callable object) which
receives |
None |
network_args |
Optional[dict] |
Optionally a dict-like object, storing keyword arguments to be passed to the network while instantiating it. |
None |
initial_bounds |
Union[Iterable[Union[float, Iterable[float], torch.Tensor]], evotorch.core.BoundsPair] |
Specifies an interval from which the values of the initial neural network parameters will be drawn. |
(-1e-05, 1e-05) |
minibatch_size |
Optional[int] |
Optionally an integer, describing the size of a
minibatch when pulling data from the dataset.
Can also be left as None, in which case it will be expected
that the inheriting class overrides the method
|
None |
num_minibatches |
Optional[int] |
An integer, specifying over how many minibatches will a single neural network be evaluated. If not specified, it will be assumed that the desired number of minibatches per network evaluation is 1. |
None |
num_actors |
Union[int, str] |
Number of actors to create for parallelized
evaluation of the solutions.
Certain string values are also accepted.
When given as "max" or as "num_cpus", the number of actors
will be equal to the number of all available CPUs in the ray
cluster.
When given as "num_gpus", the number of actors will be
equal to the number of all available GPUs in the ray
cluster, and each actor will be assigned a GPU.
When given as "num_devices", the number of actors will be
equal to the minimum among the number of CPUs and the number
of GPUs available in the cluster (or will be equal to the
number of CPUs if there is no GPU), and each actor will be
assigned a GPU (if available).
If |
None |
common_minibatch |
bool |
Whether the same minibatches will be used when evaluating the solutions or not. |
True |
actor_config |
Optional[dict] |
A dictionary, representing the keyword arguments
to be passed to the options(...) used when creating the
ray actor objects. To be used for explicitly allocating
resources per each actor.
For example, for declaring that each actor is to use a GPU,
one can pass |
None |
num_gpus_per_actor |
Union[int, float, str] |
Number of GPUs to be allocated by each
remote actor.
The default behavior is to NOT allocate any GPU at all
(which is the default behavior of the ray library as well).
When given as a number |
None |
num_subbatches |
Optional[int] |
If |
None |
subbatch_size |
Optional[int] |
If |
None |
device |
Union[str, torch.device] |
Default device in which a new population will be generated and the neural networks will operate. If not specified, "cpu" will be used. |
None |
Source code in evotorch/neuroevolution/supervisedne.py
def __init__(
self,
dataset: Dataset,
network: Union[str, nn.Module, Callable[[], nn.Module]],
loss_func: Optional[Callable] = None,
*,
network_args: Optional[dict] = None,
initial_bounds: Optional[BoundsPairLike] = (-0.00001, 0.00001),
minibatch_size: Optional[int] = None,
num_minibatches: Optional[int] = None,
num_actors: Optional[Union[int, str]] = None,
common_minibatch: bool = True,
num_gpus_per_actor: Optional[Union[int, float, str]] = None,
actor_config: Optional[dict] = None,
num_subbatches: Optional[int] = None,
subbatch_size: Optional[int] = None,
device: Optional[Device] = None,
):
"""
`__init__(...)`: Initialize the SupervisedNE.
Args:
dataset: The Dataset from which the minibatches will be pulled
network: A network structure string, or a Callable (which can be
a class inheriting from `torch.nn.Module`, or a function
which returns a `torch.nn.Module` instance), or an instance
of `torch.nn.Module`.
The object provided here determines the structure of the
neural network whose parameters will be evolved.
A network structure string is a string which can be processed
by `evotorch.neuroevolution.net.str_to_net(...)`.
Please see the documentation of the function
`evotorch.neuroevolution.net.str_to_net(...)` to see how such
a neural network structure string looks like.
loss_func: Optionally a function (or a Callable object) which
receives `y_hat` (the output generated by the neural network)
and `y` (the desired output), and returns the loss as a
scalar.
This argument can also be left as None, in which case it will
be expected that the method `_loss(self, y_hat, y)` is
overridden by the inheriting class.
network_args: Optionally a dict-like object, storing keyword
arguments to be passed to the network while instantiating it.
initial_bounds: Specifies an interval from which the values of the
initial neural network parameters will be drawn.
minibatch_size: Optionally an integer, describing the size of a
minibatch when pulling data from the dataset.
Can also be left as None, in which case it will be expected
that the inheriting class overrides the method
`_make_dataloader()` and defines how a new DataLoader is to be
made.
num_minibatches: An integer, specifying over how many minibatches
will a single neural network be evaluated.
If not specified, it will be assumed that the desired number
of minibatches per network evaluation is 1.
num_actors: Number of actors to create for parallelized
evaluation of the solutions.
Certain string values are also accepted.
When given as "max" or as "num_cpus", the number of actors
will be equal to the number of all available CPUs in the ray
cluster.
When given as "num_gpus", the number of actors will be
equal to the number of all available GPUs in the ray
cluster, and each actor will be assigned a GPU.
When given as "num_devices", the number of actors will be
equal to the minimum among the number of CPUs and the number
of GPUs available in the cluster (or will be equal to the
number of CPUs if there is no GPU), and each actor will be
assigned a GPU (if available).
If `num_actors` is given as "num_gpus" or "num_devices",
the argument `num_gpus_per_actor` must not be used,
and the `actor_config` dictionary must not contain the
key "num_gpus".
If `num_actors` is given as something other than "num_gpus"
or "num_devices", and if you wish to assign GPUs to each
actor, then please see the argument `num_gpus_per_actor`.
common_minibatch: Whether the same minibatches will be
used when evaluating the solutions or not.
actor_config: A dictionary, representing the keyword arguments
to be passed to the options(...) used when creating the
ray actor objects. To be used for explicitly allocating
resources per each actor.
For example, for declaring that each actor is to use a GPU,
one can pass `actor_config=dict(num_gpus=1)`.
Can also be given as None (which is the default),
if no such options are to be passed.
num_gpus_per_actor: Number of GPUs to be allocated by each
remote actor.
The default behavior is to NOT allocate any GPU at all
(which is the default behavior of the ray library as well).
When given as a number `n`, each actor will be given
`n` GPUs (where `n` can be an integer, or can be a `float`
for fractional allocation).
When given as a string "max", then the available GPUs
across the entire ray cluster (or within the local computer
in the simplest cases) will be equally distributed among
the actors.
When given as a string "all", then each actor will have
access to all the GPUs (this will be achieved by suppressing
the environment variable `CUDA_VISIBLE_DEVICES` for each
actor).
When the problem is not distributed (i.e. when there are
no actors), this argument is expected to be left as None.
num_subbatches: If `num_subbatches` is None (assuming that
`subbatch_size` is also None), then, when evaluating a
population, the population will be split into n pieces, `n`
being the number of actors, and each actor will evaluate
its assigned piece. If `num_subbatches` is an integer `m`,
then the population will be split into `m` pieces,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
While using a distributed algorithm, this argument determines
how many sub-batches will be generated, and therefore,
how many gradients will be computed by the remote actors.
subbatch_size: If `subbatch_size` is None (assuming that
`num_subbatches` is also None), then, when evaluating a
population, the population will be split into `n` pieces, `n`
being the number of actors, and each actor will evaluate its
assigned piece. If `subbatch_size` is an integer `m`,
then the population will be split into pieces of size `m`,
and actors will continually accept the next unevaluated
piece as they finish their current tasks.
When there can be significant difference across the solutions
in terms of computational requirements, specifying a
`subbatch_size` can be beneficial, because, while one
actor is busy with a subbatch containing computationally
challenging solutions, other actors can accept more
tasks and save time.
The arguments `num_subbatches` and `subbatch_size` cannot
be given values other than None at the same time.
While using a distributed algorithm, this argument determines
the size of a sub-batch (or sub-population) sampled by a
remote actor for computing a gradient.
In distributed mode, it is expected that the population size
is divisible by `subbatch_size`.
device: Default device in which a new population will be generated
and the neural networks will operate.
If not specified, "cpu" will be used.
"""
super().__init__(
objective_sense="min",
network=network,
network_args=network_args,
initial_bounds=initial_bounds,
num_actors=num_actors,
num_gpus_per_actor=num_gpus_per_actor,
actor_config=actor_config,
num_subbatches=num_subbatches,
subbatch_size=subbatch_size,
device=device,
)
self.dataset = dataset
self.dataloader: DataLoader = None
self.dataloader_iterator = None
self._loss_func = loss_func
self._minibatch_size = None if minibatch_size is None else int(minibatch_size)
self._num_minibatches = 1 if num_minibatches is None else int(num_minibatches)
self._common_minibatch = common_minibatch
self._current_minibatches: Optional[list] = None
get_minibatch(self)
¶
Get the next minibatch from the DataLoader.
Source code in evotorch/neuroevolution/supervisedne.py
def get_minibatch(self) -> Any:
"""
Get the next minibatch from the DataLoader.
"""
if self.dataloader is None:
self._prepare()
if self.dataloader_iterator is None:
self.dataloader_iterator = iter(self.dataloader)
batch = None
try:
batch = next(self.dataloader_iterator)
except StopIteration:
pass
if batch is None:
self.dataloader_iterator = iter(self.dataloader)
batch = next(self.dataloader_iterator)
# Move batch to device of network
return [var.to(self.network_device) for var in batch]
loss(self, y_hat, y)
¶
Run the loss function and return the loss.
If the __init__ of SupervisedNE class was given a loss
function via the argument loss_func, then that loss function
will be used. Otherwise, it will be expected that the method
_loss(...) is overriden with a loss definition, and that method
will be used to compute the loss.
The computed loss will be returned.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
y_hat |
Any |
The output estimated by the network |
required |
y |
Any |
The desired output |
required |
Returns:
| Type | Description |
|---|---|
Union[float, torch.Tensor] |
A scalar, representing the loss |
Source code in evotorch/neuroevolution/supervisedne.py
def loss(self, y_hat: Any, y: Any) -> Union[float, torch.Tensor]:
"""
Run the loss function and return the loss.
If the `__init__` of `SupervisedNE` class was given a loss
function via the argument `loss_func`, then that loss function
will be used. Otherwise, it will be expected that the method
`_loss(...)` is overriden with a loss definition, and that method
will be used to compute the loss.
The computed loss will be returned.
Args:
y_hat: The output estimated by the network
y: The desired output
Returns:
A scalar, representing the loss
"""
if self._loss_func is None:
return self._loss(y_hat, y)
else:
return self._loss_func(y_hat, y)
make_dataloader(self)
¶
Make a new DataLoader.
If the __init__ of SupervisedNE was provided with a minibatch size
via the argument minibatch_size, then a new DataLoader will be made
with that minibatch size.
Otherwise, it will be expected that the method _make_dataloader(...)
was overridden to contain details regarding how the DataLoader should be
created, and that method will be executed.
Returns:
| Type | Description |
|---|---|
DataLoader |
The created DataLoader. |
Source code in evotorch/neuroevolution/supervisedne.py
def make_dataloader(self) -> DataLoader:
"""
Make a new DataLoader.
If the `__init__` of `SupervisedNE` was provided with a minibatch size
via the argument `minibatch_size`, then a new DataLoader will be made
with that minibatch size.
Otherwise, it will be expected that the method `_make_dataloader(...)`
was overridden to contain details regarding how the DataLoader should be
created, and that method will be executed.
Returns:
The created DataLoader.
"""
if self._minibatch_size is None:
return self._make_dataloader()
else:
return DataLoader(self.dataset, shuffle=True, batch_size=self._minibatch_size)
vecgymne
¶
VecGymNE (BaseNEProblem)
¶
An EvoTorch problem for solving vectorized gym environments
Source code in evotorch/neuroevolution/vecgymne.py
class VecGymNE(BaseNEProblem):
"""
An EvoTorch problem for solving vectorized gym environments
"""
def __init__(
self,
env: Union[str, Callable],
network: Union[str, Callable, nn.Module],
*,
env_config: Optional[Mapping] = None,
max_num_envs: Optional[int] = None,
network_args: Optional[Mapping] = None,
observation_normalization: bool = False,
decrease_rewards_by: Optional[float] = None,
alive_bonus_schedule: Optional[tuple] = None,
action_noise_stdev: Optional[float] = None,
num_episodes: int = 1,
device: Optional[Device] = None,
num_actors: Optional[Union[int, str]] = None,
num_gpus_per_actor: Optional[int] = None,
num_subbatches: Optional[int] = None,
subbatch_size: Optional[int] = None,
actor_config: Optional[Mapping] = None,
):
"""
Initialize the VecGymNE.
Args:
env: Environment to be solved.
If this is given as a string starting with "gym::" (e.g.
"gym::Humanoid-v4", etc.), then it is assumed that the target
environment is a classical gym environment.
If this is given as a string starting with "brax::" (e.g.
"brax::humanoid", etc.), then it is assumed that the target
environment is a brax environment.
If this is given as a string which does not contain "::" at
all (e.g. "Humanoid-v4", etc.), then it is assumed that the
target environment is a classical gym environment. Therefore,
"gym::Humanoid-v4" and "Humanoid-v4" are equivalent.
If this argument is given as a Callable (maybe a function or a
class), then, with the assumption that this Callable expects
a keyword argument `num_envs: int`, this Callable is called
and its result (expected as a `gym.vector.VectorEnv` instance)
is used as the environment.
network: A network structure string, or a Callable (which can be
a class inheriting from `torch.nn.Module`, or a function
which returns a `torch.nn.Module` instance), or an instance
of `torch.nn.Module`.
The object provided here determines the structure of the
neural network whose parameters will be evolved.
A network structure string is a string which can be processed
by `evotorch.neuroevolution.net.str_to_net(...)`.
Please see the documentation of the function
`evotorch.neuroevolution.net.str_to_net(...)` to see how such
a neural network structure string looks like.
Note that this network can be a recurrent network.
When the network's `forward(...)` method can optionally accept
an additional positional argument for the hidden state of the
network and returns an additional value for its next state,
then the policy is treated as a recurrent one.
When the network is given as a callable object (e.g.
a subclass of `nn.Module` or a function) and this callable
object is decorated via `evotorch.decorators.pass_info`,
the following keyword arguments will be passed:
(i) `obs_length` (the length of the observation vector),
(ii) `act_length` (the length of the action vector),
(iii) `obs_shape` (the shape tuple of the observation space),
(iv) `act_shape` (the shape tuple of the action space),
(v) `obs_space` (the Box object specifying the observation
space, and
(vi) `act_space` (the Box object specifying the action
space). Note that `act_space` will always be given as a
`gym.spaces.Box` instance, even when the actual gym
environment has a discrete action space. This because
`VecGymNE` always expects the neural network to return
a tensor of floating-point numbers.
env_config: Keyword arguments to pass to the environment while
it is being created.
max_num_envs: Maximum number of environments to be instantiated.
By default, this is None, which means that the number of
environments can go up to the population size (or up to the
number of solutions that a remote actor receives, if the
problem object is configured to have parallelization).
For situations where the current reinforcement learning task
requires large amount of resources (e.g. memory), allocating
environments as much as the number of solutions might not
be feasible. In such cases, one can set `max_num_envs` as an
integer to bring an upper bound (in total, across all the
remote actors, for when the problem is parallelized) to how
many environments can be allocated.
network_args: Any additional keyword argument to be used when
instantiating the network can be specified via `network_args`
as a dictionary. If there are no such additional keyword
arguments, then `network_args` can be left as None.
Note that the argument `network_args` is expected to be None
when the network is specified as a `torch.nn.Module` instance.
observation_normalization: Whether or not online normalization
will be done on the encountered observations.
decrease_rewards_by: If given as a float, each reward will be
decreased by this amount. For example, if the environment's
reward function has a constant "alive bonus" (i.e. a bonus
that is constantly added onto the reward as long as the
agent is alive), and if you wish to negate this bonus,
you can set `decrease_rewards_by` to this bonus amount,
and the bonus will be nullified.
If you do not wish to affect the rewards in this manner,
keep this as None.
alive_bonus_schedule: Use this to add a customized amount of
alive bonus.
If left as None (which is the default), additional alive
bonus will not be added.
If given as a tuple `(t, b)`, an alive bonus `b` will be
added onto all the rewards beyond the timestep `t`.
If given as a tuple `(t0, t1, b)`, a partial (linearly
increasing towards `b`) alive bonus will be added onto
all the rewards between the timesteps `t0` and `t1`,
and a full alive bonus (which equals to `b`) will be added
onto all the rewards beyond the timestep `t1`.
action_noise_stdev: If given as a real number `s`, then, for
each generated action, Gaussian noise with standard
deviation `s` will be sampled, and then this sampled noise
will be added onto the action.
If action noise is not desired, then this argument can be
left as None.
For sampling the noise, the global random number generator
of PyTorch on the simulator's device will be used.
num_episodes: Number of episodes over which each policy will
be evaluated. The default is 1.
device: The device in which the population will be kept.
If you wish to do a single-GPU evolution, we recommend
to set this as "cuda" (or "cuda:0", or "cuda:1", etc.),
assuming that the simulator will also instantiate itself
on that same device.
Alternatively, if you wish to do a multi-GPU evolution,
we recommend to leave this as None or set this as "cpu",
so that the main population will be kept on the cpu
and the remote actors will perform their evaluations on
the GPUs that are assigned to them.
num_actors: Number of actors to create for parallelized
evaluation of the solutions.
Certain string values are also accepted.
When given as "max" or as "num_cpus", the number of actors
will be equal to the number of all available CPUs in the ray
cluster.
When given as "num_gpus", the number of actors will be
equal to the number of all available GPUs in the ray
cluster, and each actor will be assigned a GPU.
When given as "num_devices", the number of actors will be
equal to the minimum among the number of CPUs and the number
of GPUs available in the cluster (or will be equal to the
number of CPUs if there is no GPU), and each actor will be
assigned a GPU (if available).
If `num_actors` is given as "num_gpus" or "num_devices",
the argument `num_gpus_per_actor` must not be used,
and the `actor_config` dictionary must not contain the
key "num_gpus".
If `num_actors` is given as something other than "num_gpus"
or "num_devices", and if you wish to assign GPUs to each
actor, then please see the argument `num_gpus_per_actor`.
num_gpus_per_actor: Number of GPUs to be assigned to each
actor. This can be an integer or a float (for when you
wish to assign fractional amounts of GPUs to actors).
When `num_actors` has the special value "num_devices",
the argument `num_gpus_per_actor` is expected to be left as
None.
num_subbatches: For when there are multiple actors, you can
set this to an integer n if you wish the population
to be divided exactly into n sub-batches. The actors, as they
finish their currently assigned sub-batch of solutions,
will pick the next un-evaluated sub-batch.
If you specify too large numbers for this argument, then
each sub-batch will be smaller.
When working with vectorized simulators on GPU, having too
many and too small sub-batches can hurt the performance.
This argument can be left as None, in which case, assuming
that `subbatch_size` is also None, the population will be
split to m sub-batches, m being the number of actors.
subbatch_size: For when there are multiple actors, you can
set this to an integer n if you wish the population to be
divided into sub-batches in such a way that each sub-batch
will consist of exactly n solutions. The actors, as they
finish their currently assigned sub-batch of solutions,
will pick the next un-evaluated sub-batch.
If you specify too small numbers for this argument, then
there will be many sub-batches, each sub-batch having a
small number of solutions.
When working with vectorized simulators on GPU, having too
many and too small sub-batches can hurt the performance.
This argument can be left as None, in which case, assuming
that `num_subbatches` is also None, the population will be
split to m sub-batches, m being the number of actors.
actor_config: Additional configuration to be used when creating
each actor with the help of `ray` library.
Can be left as None if additional configuration is not needed.
"""
# Store the string or the Callable that will be used to generate the reinforcement learning environment.
self._env_maker = env
# Declare the variable which will store the environment.
self._env: Optional[TorchWrapper] = None
# Declare the variable which will store the batch size of the vectorized environment.
self._num_envs: Optional[int] = None
# Store the upper bound (if any) regarding how many environments can exist at the same time.
self._max_num_envs: Optional[int] = None if max_num_envs is None else int(max_num_envs)
# Actor-specific upper bound regarding how many environments can exist at the same time.
# This variable will be filled by the `_parallelize(...)` method.
self._actor_max_num_envs: Optional[int] = None
# Declare the variable which stores whether or not we properly initialized the `_actor_max_num_envs` variable.
self._actor_max_num_envs_ready: bool = False
# Store the additional configurations to be used as keyword arguments while instantiating the environment.
self._env_config: dict = {} if env_config is None else dict(env_config)
# Declare the variable that will store the device of the simulator.
# This variable will be filled when the first observation is received from the environment.
# The device of the observation array received from the environment will determine the value of this variable.
self._simulator_device: Optional[torch.device] = None
# Store the neural network architecture (that might be a string or an `nn.Module` instance).
self._architecture = network
if network_args is None:
# If `network_args` is given as None, change it to an empty dictionary
network_args = {}
if isinstance(network, str):
# If the network is given as a string, then we will need the values for the constants `obs_length`,
# `act_length`, and `obs_space`. To obtain those values, we use our helper function
# `_env_constants_for_str_net(...)` which temporarily instantiates the specified environment and returns
# its needed constants.
env_constants = _env_constants_for_str_net(self._env_maker, **(self._env_config))
elif isinstance(network, nn.Module):
# If the network is an already instantiated nn.Module, then we do not prepare any pre-defined constants.
env_constants = {}
else:
# If the network is given as a Callable, then we will need the values for the constants `obs_length`,
# `act_length`, and `obs_space`. To obtain those values, we use our helper function
# `_env_constants_for_callable_net(...)` which temporarily instantiates the specified environment and
# returns its needed constants.
env_constants = _env_constants_for_callable_net(self._env_maker, **(self._env_config))
# Build a `Policy` instance according to the given architecture, and store it.
if isinstance(network, str):
instantiated_net = str_to_net(network, **{**env_constants, **network_args})
elif isinstance(network, nn.Module):
instantiated_net = network
else:
instantiated_net = pass_info_if_needed(network, env_constants)(**network_args)
self._policy = Policy(instantiated_net)
# Store the boolean which indicates whether or not there will be observation normalization.
self._observation_normalization = bool(observation_normalization)
# Declare the variables that will store the observation-related stats if observation normalization is enabled.
self._obs_stats: Optional[RunningNorm] = None
self._collected_stats: Optional[RunningNorm] = None
# Store the number of episodes configuration given by the user.
self._num_episodes = int(num_episodes)
# Store the `decrease_rewards_by` configuration given by the user.
self._decrease_rewards_by = None if decrease_rewards_by is None else float(decrease_rewards_by)
if alive_bonus_schedule is None:
# If `alive_bonus_schedule` argument is None, then we store it as None as well.
self._alive_bonus_schedule = None
else:
# This is the case where the user has specified an `alive_bonus_schedule`.
alive_bonus_schedule = list(alive_bonus_schedule)
alive_bonus_schedule_length = len(alive_bonus_schedule)
if alive_bonus_schedule_length == 2:
# If `alive_bonus_schedule` was given as a 2-element sequence (t, b), then store it as (t, t, b).
# This means that the partial alive bonus time window starts and ends at t, therefore, there will
# be no alive bonus until t, and beginning with t, there will be full alive bonus.
self._alive_bonus_schedule = [
int(alive_bonus_schedule[0]),
int(alive_bonus_schedule[0]),
float(alive_bonus_schedule[1]),
]
elif alive_bonus_schedule_length == 3:
# If `alive_bonus_schedule` was given as a 3-element sequence (t0, t1, b), then store those 3
# elements.
self._alive_bonus_schedule = [
int(alive_bonus_schedule[0]),
int(alive_bonus_schedule[1]),
float(alive_bonus_schedule[2]),
]
else:
# `alive_bonus_schedule` sequences with unrecognized lengths trigger an error.
raise ValueError(
f"Received invalid number elements as the alive bonus schedule."
f" Expected 2 or 3 items, but got these: {self._alive_bonus_schedule}"
f" (having a length of {len(self._alive_bonus_schedule)})."
)
# If `action_noise_stdev` is specified, store it.
self._action_noise_stdev = None if action_noise_stdev is None else float(action_noise_stdev)
# Initialize the counters for the number of simulator interactions and the number of episodes.
self._interaction_count: int = 0
self._episode_count: int = 0
device_is_cpu = (device is None) or (str(device) == "cpu")
actors_use_gpu = (
(num_actors is not None)
and (num_actors > 1)
and (num_gpus_per_actor is not None)
and (num_gpus_per_actor > 0)
)
if not device_is_cpu:
# In the case where the device is something other than the cpu, we tell SyncVectorEnv to use this device.
self._device_for_sync_vector_env = device
self._sync_vector_env_uses_aux_device = False
elif actors_use_gpu:
# In the case where this problem instance is configured to use multiple actors and the actors are
# configured to use the available gpu(s), we tell SyncVectorEnv to use the `aux_device`.
self._device_for_sync_vector_env = None
self._sync_vector_env_uses_aux_device = True
else:
self._device_for_sync_vector_env = None
self._sync_vector_env_uses_aux_device = False
# Call the superclass
super().__init__(
objective_sense="max",
initial_bounds=(-0.00001, 0.00001),
solution_length=self._policy.parameter_length,
device=device,
dtype=torch.float32,
num_actors=num_actors,
num_gpus_per_actor=num_gpus_per_actor,
actor_config=actor_config,
num_subbatches=num_subbatches,
subbatch_size=subbatch_size,
)
self.after_eval_hook.append(self._extra_status)
def _parallelize(self):
super()._parallelize()
if self.is_main:
if not self._actor_max_num_envs_ready:
if self._actors is None:
self._actor_max_num_envs = self._max_num_envs
else:
if self._max_num_envs is not None:
max_num_envs_per_actor = split_workload(self._max_num_envs, len(self._actors))
for i_actor, actor in enumerate(self._actors):
actor.call.remote("_set_actor_max_num_envs", max_num_envs_per_actor[i_actor])
self._actor_max_num_envs_ready = True
def _set_actor_max_num_envs(self, n: int):
self._actor_max_num_envs = n
self._actor_max_num_envs_ready = True
def _extra_status(self, batch: SolutionBatch):
return dict(total_interaction_count=self.interaction_count, total_episode_count=self.episode_count)
@property
def observation_normalization(self) -> bool:
return self._observation_normalization
def set_episode_count(self, n: int):
"""
Set the episode count manually.
"""
self._episode_count = int(n)
def set_interaction_count(self, n: int):
"""
Set the interaction count manually.
"""
self._interaction_count = int(n)
@property
def interaction_count(self) -> int:
"""
Get the total number of simulator interactions made.
"""
return self._interaction_count
@property
def episode_count(self) -> int:
"""
Get the total number of episodes completed.
"""
return self._episode_count
def _get_local_episode_count(self) -> int:
return self.episode_count
def _get_local_interaction_count(self) -> int:
return self.interaction_count
def _get_env(self, num_policies: int) -> TorchWrapper:
# Get the existing environment instance stored by this VecGymNE, after (re)building it if needed.
if (self._env is None) or (num_policies > self._num_envs):
# If this VecGymNE does not have its environment ready yet (i.e. the `_env` attribute is None)
# or it the batch size of the previously instantiated environment is not enough to deal with
# the number of policies (i.e. the `_num_envs` attribute is less than `num_policies`), then
# we (re)build the environment.
# Keyword arguments to pass to the TorchWrapper.
torch_wrapper_cfg = dict(
force_classic_api=True,
discrete_to_continuous_act=True,
clip_actions=True,
)
# Keyword arguments to use only when the underlying environment is a classical gymnasium environment
gym_cfg = dict(empty_info=True, num_episodes=self._num_episodes)
if self._sync_vector_env_uses_aux_device:
gym_cfg["device"] = self.aux_device
elif self._device_for_sync_vector_env is not None:
gym_cfg["device"] = self._device_for_sync_vector_env
if isinstance(self._env_maker, str):
# If the environment is specified via a string, then we use our `make_vector_env` function.
self._env = make_vector_env(
self._env_maker,
num_envs=num_policies,
gym_kwargs=gym_cfg,
**torch_wrapper_cfg,
**(self._env_config),
)
else:
# If the environment is specified via a Callable, then we call it.
# We expect this Callable to accept a keyword argument named `num_envs`, and additionally, we pass
# the environment configuration dictionary as keyword arguments.
self._env = self._env_maker(num_envs=num_policies, **(self._env_config))
if not isinstance(self._env, gym.vector.VectorEnv):
# If what is returned by the Callable is not a vectorized environment, then we trigger an error.
raise TypeError("This is not a vectorized environment")
# We wrap the returned vectorized environment with a TorchWrapper, so that the actions that we send
# and the observations and rewards that we receive are PyTorch tensors.
self._env = TorchWrapper(self._env, **torch_wrapper_cfg)
if self._env.num_envs != num_policies:
# If the finally obtained vectorized environment has a different number of batch size, then we trigger
# an error.
raise ValueError("Incompatible number of environments")
# We update the batch size of the created environment.
self._num_envs = num_policies
if not isinstance(self._env.single_observation_space, Box):
# If the observation space is not Box, then we trigger an error.
raise TypeError(
f"Unsupported observation type: {self._env.single_observation_space}."
f" Only Box-typed observation spaces are supported."
)
try:
# If possible, use the `seed(...)` method to explicitly randomize the environment.
# Although the new gym API removed the seed method, some environments define their own `seed(...)`
# method for randomization.
new_seed = random.randint(0, (2**32) - 1)
self._env.seed(new_seed)
except Exception:
# Our attempt at manually seeding the environment has failed.
# This could be because the environment does not have a `seed(...)` method.
# Nothing to do.
pass
return self._env
@property
def _nonserialized_attribs(self):
# Call the `_nonserialized_attribs` property implementation of the superclass to receive the base list
# of non-serialized attributes, then add "_env" to this base list, and then return the resulting list.
return super()._nonserialized_attribs + ["_env"]
@property
def _grad_device(self) -> torch.device:
# For distributed mode, this property determines the device in which the temporary populations will be made
# for gradient computation.
if self._simulator_device is None:
# If the simulator device is not known yet, then we return the cpu device.
return torch.device("cpu")
else:
# If the simulator device is known, then we return that device.
return self._simulator_device
def _make_running_norm(self, observation: torch.Tensor) -> RunningNorm:
# Make a new RunningNorm instance according to the observation tensor.
# The dtype and the device of the new RunningNorm is taken from the observation.
# This new RunningNorm is empty (i.e. does not contain any stats yet).
return RunningNorm(shape=observation.shape[1:], dtype=observation.dtype, device=observation.device)
def _transfer_running_norm(self, rn: RunningNorm, observation: torch.Tensor) -> RunningNorm:
# Transfer (if necessary) the RunningNorm to the device of the observation tensor.
# The returned RunningNorm may be the RunningNorm itself (if the device did not change)
# or a new copy (if the device did change).
if torch.device(rn.device) != torch.device(observation.device):
rn = rn.to(observation.device)
return rn
def _normalize_observation(
self, observation: torch.Tensor, *, mask: Optional[torch.Tensor] = None, update_stats: bool = True
) -> torch.Tensor:
# This function normalizes the received observation batch.
# If a mask is given (as a tensor of booleans), only observations with corresponding mask value set as True
# will be taken into consideration.
# If `update_stats` is given as True and observation normalization is enabled, then we will update the
# RunningNorm instances as well.
if self._observation_normalization:
# This is the case where observation normalization is enabled.
if self._obs_stats is None:
# If we do not have observation stats yet, we build a new one (according to the dtype and device
# of the observation).
self._obs_stats = self._make_running_norm(observation)
else:
# If we already have observation stats, we make sure that it is in the correct device.
self._obs_stats = self._transfer_running_norm(self._obs_stats, observation)
if update_stats:
# This is the case where the `update_stats` argument was encountered as True.
if self._collected_stats is None:
# If the RunningNorm responsible to collect new stats is not built yet, we build it here
# (according to the dtype and device of the observation).
self._collected_stats = self._make_running_norm(observation)
else:
# If the RunningNorm responsible to collect new stats already exists, then we make sure
# that it is in the correct device.
self._collected_stats = self._transfer_running_norm(self._collected_stats, observation)
# We first update the RunningNorm responsible for collecting the new stats.
self._collected_stats.update(observation, mask)
# We now update the RunningNorm which stores all the stats, and return the normalized observation.
result = self._obs_stats.update_and_normalize(observation, mask)
else:
# This is the case where the `update_stats` argument was encountered as False.
# Here we normalize the observation but do not update our existing RunningNorm instances.
result = self._obs_stats.update(observation, mask)
return result
else:
# This is the case where observation normalization is disabled.
# In this case, we just return the observation as it is.
return observation
def _ensure_obsnorm(self):
if not self.observation_normalization:
raise ValueError("This feature can only be used when observation_normalization=True.")
def get_observation_stats(self) -> RunningNorm:
"""Get the observation stats"""
self._ensure_obsnorm()
return self._obs_stats
def _make_sync_data_for_actors(self) -> Any:
if self.observation_normalization:
obs_stats = self.get_observation_stats()
if obs_stats is not None:
obs_stats = obs_stats.to("cpu")
return dict(obs_stats=obs_stats)
else:
return None
def set_observation_stats(self, rn: RunningNorm):
"""Set the observation stats"""
self._ensure_obsnorm()
self._obs_stats = rn
def _use_sync_data_from_main(self, received: dict):
for k, v in received.items():
if k == "obs_stats":
self.set_observation_stats(v)
def pop_observation_stats(self) -> RunningNorm:
"""Get and clear the collected observation stats"""
self._ensure_obsnorm()
result = self._collected_stats
self._collected_stats = None
return result
def _make_sync_data_for_main(self) -> Any:
result = dict(episode_count=self.episode_count, interaction_count=self.interaction_count)
if self.observation_normalization:
collected = self.pop_observation_stats()
if collected is not None:
collected = collected.to("cpu")
result["obs_stats_delta"] = collected
return result
def update_observation_stats(self, rn: RunningNorm):
"""Update the observation stats via another RunningNorm instance"""
self._ensure_obsnorm()
if self._obs_stats is None:
self._obs_stats = rn
else:
self._obs_stats.update(rn)
def _use_sync_data_from_actors(self, received: list):
total_episode_count = 0
total_interaction_count = 0
for data in received:
data: dict
total_episode_count += data["episode_count"]
total_interaction_count += data["interaction_count"]
if self.observation_normalization:
self.update_observation_stats(data["obs_stats_delta"])
self.set_episode_count(total_episode_count)
self.set_interaction_count(total_interaction_count)
def _make_pickle_data_for_main(self) -> dict:
# For when the main Problem object (the non-remote one) gets pickled,
# this function returns the counters of this remote Problem instance,
# to be sent to the main one.
return dict(interaction_count=self.interaction_count, episode_count=self.episode_count)
def _use_pickle_data_from_main(self, state: dict):
# For when a newly unpickled Problem object gets (re)parallelized,
# this function restores the inner states specific to this remote
# worker. In the case of GymNE, those inner states are episode
# and interaction counters.
for k, v in state.items():
if k == "episode_count":
self.set_episode_count(v)
elif k == "interaction_count":
self.set_interaction_count(v)
else:
raise ValueError(f"When restoring the inner state of a remote worker, unrecognized state key: {k}")
def _evaluate_batch(self, batch: SolutionBatch):
if self._actor_max_num_envs is None:
self._evaluate_subbatch(batch)
else:
subbatches = batch.split(max_size=self._actor_max_num_envs)
for subbatch in subbatches:
self._evaluate_subbatch(subbatch)
def _evaluate_subbatch(self, batch: SolutionBatch):
# Get the number of solutions and the solution batch from the shape of the batch.
num_solutions, solution_length = batch.values_shape
# Get (possibly after (re)building) the environment object.
env = self._get_env(num_solutions)
# Reset the environment and receive the first observation batch.
obs_per_env = env.reset()
# Update the simulator device according to the device of the observation batch received.
self._simulator_device = obs_per_env.device
# Get the number of environments.
num_envs = obs_per_env.shape[0]
# Transfer (if necessary) the solutions (which are the network parameters) to the simulator device.
batch_values = batch.values.to(self._simulator_device)
if num_solutions == num_envs:
# If the number of solutions is equal to the number of environments, then we declare all of the solutions
# as the network parameters, and we declare all of these environments active.
params_per_env = batch_values
active_per_env = torch.ones(num_solutions, dtype=torch.bool, device=self._simulator_device)
elif num_solutions < num_envs:
# If the number of solutions is less than the number of environments, then we allocate a new empty
# tensor to represent the network parameters.
params_per_env = torch.empty((num_envs, solution_length), dtype=batch.dtype, device=self._simulator_device)
# The first `num_solutions` rows of this new parameters tensor is filled with the values of the solutions.
params_per_env[:num_solutions, :] = batch_values
# The remaining parameters become the clones of the first solution.
params_per_env[num_solutions:, :] = batch_values[0]
# At first, all the environments are declared as inactive.
active_per_env = torch.zeros(num_envs, dtype=torch.bool, device=self._simulator_device)
# Now, the first `num_solutions` amount of environments is declared as active.
# The remaining ones remain inactive.
active_per_env[:num_solutions] = True
else:
assert False, "Received incompatible number of environments"
# We get the policy and fill it with the parameters stored by the solutions.
policy = self._policy
policy.set_parameters(params_per_env)
# Declare the counter which stores the total timesteps encountered during this evaluation.
total_timesteps = 0
# Declare the counters (one for each environment) storing the number of episodes completed.
num_eps_per_env = torch.zeros(num_envs, dtype=torch.int64, device=self._simulator_device)
# Declare the scores (one for each environment).
score_per_env = torch.zeros(num_envs, dtype=torch.float32, device=self._simulator_device)
if self._alive_bonus_schedule is not None:
# If an alive_bonus_schedule was provided, then we extract the timesteps.
# bonus_t0 is the timestep where the partial alive bonus will start.
# bonus_t1 is the timestep where the full alive bonus will start.
# alive_bonus is the amount that will be added to reward if the agent is alive.
bonus_t0, bonus_t1, alive_bonus = self._alive_bonus_schedule
if bonus_t1 > bonus_t0:
# If bonus_t1 is bigger than bonus_t0, then we have a partial alive bonus time window.
add_partial_alive_bonus = True
# We compute and store the length of the time window.
bonus_t_gap_as_float = float(bonus_t1 - bonus_t0)
else:
# If bonus_t1 is NOT bigger than bonus_t0, then we do NOT have a partial alive bonus time window.
add_partial_alive_bonus = False
# To properly give the alive bonus for each solution, we need to keep track of the timesteps for all
# the running solutions. So, we declare the following variable.
t_per_env = torch.zeros(num_envs, dtype=torch.int64, device=self._simulator_device)
def normalize(observations: torch.Tensor, *, mask: torch.Tensor) -> torch.Tensor:
original_observations = observations
observations = observations[mask]
if observations.shape[0] == 0:
return observations
else:
normalized = self._normalize_observation(observations)
modified_observations = original_observations.clone()
modified_observations[mask] = normalized
return modified_observations
# We normalize the initial observation.
# obs_per_env = self._normalize_observation(obs_per_env, mask=active_per_env)
obs_per_env = normalize(obs_per_env, mask=active_per_env)
while True:
# Pass the observations through the policy and get the actions to perform.
action_per_env = policy(torch.as_tensor(obs_per_env, dtype=params_per_env.dtype))
if self._action_noise_stdev is not None:
# If we are to apply action noise, we sample from a Gaussian distribution and add the noise onto
# the actions.
action_per_env = action_per_env + (torch.rand_like(action_per_env) * self._action_noise_stdev)
# Apply the actions, get the observations, rewards, and the 'done' flags.
obs_per_env, reward_per_env, done_per_env, _ = env.step(action_per_env)
if self._decrease_rewards_by is not None:
# We decrease the rewards, if we have the configuration to do so.
reward_per_env = reward_per_env - self._decrease_rewards_by
if self._alive_bonus_schedule is not None:
# Here we handle the alive bonus schedule.
# For each environment, increment the timestep.
t_per_env[active_per_env] += 1
# For those who are within the full alive bonus time region, increase the scores by the full amount.
in_full_bonus_t_per_env = active_per_env & (t_per_env >= bonus_t1)
score_per_env[in_full_bonus_t_per_env] += alive_bonus
if add_partial_alive_bonus:
# Here we handle the partial alive bonus time window.
# We first determine which environments are in the partial alive bonus time window.
in_partial_bonus_t_per_env = active_per_env & (t_per_env >= bonus_t0) & (t_per_env < bonus_t1)
# Here we compute the partial alive bonuses and add those bonuses to the scores.
score_per_env[in_partial_bonus_t_per_env] += alive_bonus * (
torch.as_tensor(t_per_env[in_partial_bonus_t_per_env] - bonus_t0, dtype=torch.float32)
/ bonus_t_gap_as_float
)
# Determine which environments just finished their episodes.
just_finished_per_env = active_per_env & done_per_env
# Reset the timestep counters of the environments which are just finished.
t_per_env[just_finished_per_env] = 0
# For each active environment, increase the score by the reward received.
score_per_env[active_per_env] += reward_per_env[active_per_env]
# Update the total timesteps counter.
total_timesteps += int(torch.sum(active_per_env))
# Reset the policies whose episodes are done (so that their hidden states become 0).
policy.reset(done_per_env)
# Update the number of episodes counter for each environment.
num_eps_per_env[done_per_env] += 1
# Solutions with number of completed episodes larger than the number of allowed episodes become inactive.
# active_per_env[:num_solutions] = num_eps_per_env[:num_solutions] < self._num_episodes
active_per_env[:num_solutions] = active_per_env[:num_solutions] & (
num_eps_per_env[:num_solutions] < self._num_episodes
)
if not torch.any(active_per_env[:num_solutions]):
# If there is not a single active solution left, then we exit this loop.
break
# For the next iteration of this loop, we normalize the observation.
# obs_per_env = self._normalize_observation(obs_per_env, mask=active_per_env)
obs_per_env = normalize(obs_per_env, mask=active_per_env)
# Update the interaction count and the episode count stored by this VecGymNE instance.
self._interaction_count += total_timesteps
self._episode_count += num_solutions * self._num_episodes
# Compute the fitnesses
fitnesses = score_per_env[:num_solutions]
if self._num_episodes > 1:
fitnesses /= self._num_episodes
# Assign the scores to the solutions as fitnesses.
batch.set_evals(fitnesses)
def get_env(self) -> Optional[gym.Env]:
"""
Get the gym environment.
Returns:
The gym environment if it is built. If not built yet, None.
"""
return self._env
def to_policy(self, solution: Iterable, *, with_wrapper_modules: bool = True) -> nn.Module:
"""
Convert the given solution to a policy.
Args:
solution: A solution which can be given as a `torch.Tensor`, as a
`Solution`, or as any `Iterable`.
with_wrapper_modules: Whether or not to wrap the policy module
with helper modules so that observations are normalized
and actions are clipped to be within the correct boundaries.
The default and the recommended value is True.
Returns:
The policy, as a `torch.nn.Module` instance.
"""
# Get the gym environment
env = self._get_env(1)
# Get the observation space, its lower and higher bounds.
obs_space = env.single_action_space
low = obs_space.low
high = obs_space.high
# If the lower and higher bounds are not -inf and +inf respectively, then this environment needs clipping.
needs_clipping = _numpy_arrays_specify_bounds(low, high)
# Convert the solution to a PyTorch tensor on cpu.
if isinstance(solution, torch.Tensor):
solution = solution.to("cpu")
elif isinstance(solution, Solution):
solution = solution.values.clone().to("cpu")
else:
solution = torch.as_tensor(solution, dtype=torch.float32, device="cpu")
# Convert the internally stored policy to a PyTorch module.
result = self._policy.to_torch_module(solution)
if with_wrapper_modules:
if self.observation_normalization and (self._obs_stats is not None):
# If observation normalization is needed and there are collected observation stats, then we wrap the
# policy with an ObsNormWrapperModule.
result = ObsNormWrapperModule(result, self._obs_stats)
if needs_clipping:
# If clipping is needed, then we wrap the policy with an ActClipWrapperModule
result = ActClipWrapperModule(result, obs_space)
return result
def save_solution(self, solution: Iterable, fname: Union[str, Path]):
"""
Save the solution into a pickle file.
Among the saved data within the pickle file are the solution
(as a PyTorch tensor), the policy (as a `torch.nn.Module` instance),
and observation stats (if any).
Args:
solution: The solution to be saved. This can be a PyTorch tensor,
a `Solution` instance, or any `Iterable`.
fname: The file name of the pickle file to be created.
"""
# Convert the solution to a PyTorch tensor on the cpu.
if isinstance(solution, torch.Tensor):
solution = solution.to("cpu")
elif isinstance(solution, Solution):
solution = solution.values.clone().to("cpu")
else:
solution = torch.as_tensor(solution, dtype=torch.float32, device="cpu")
if isinstance(solution, ReadOnlyTensor):
solution = solution.as_subclass(torch.Tensor)
# Store the solution and the policy.
result = {
"solution": solution,
"policy": self.to_policy(solution),
}
# If available, store the observation stats.
if self.observation_normalization and (self._obs_stats is not None):
result["obs_mean"] = self._obs_stats.mean.to("cpu")
result["obs_stdev"] = self._obs_stats.stdev.to("cpu")
result["obs_sum"] = self._obs_stats.sum.to("cpu")
result["obs_sum_of_squares"] = self._obs_stats.sum_of_squares.to("cpu")
# Some additional data.
result["interaction_count"] = self.interaction_count
result["episode_count"] = self.episode_count
result["time"] = datetime.now()
if isinstance(self._env_maker, str):
# If the environment was specified via a string, store the string.
result["env"] = self._env_maker
# Store the network architecture.
result["architecture"] = self._architecture
# Save the dictionary which stores the data.
with open(fname, "wb") as f:
pickle.dump(result, f)
@property
def max_num_envs(self) -> Optional[int]:
"""
Maximum number of environments to be allocated.
If a maximum number of environments is not set, then None is returned.
If this problem instance is the main one, then the overall maximum
number of environments is returned.
If this problem instance is a remote one (i.e. is on a remote actor)
then the maximum number of environments for that actor is returned.
"""
if self.is_main:
return self._max_num_envs
else:
return self._actor_max_num_envs
def make_net(self, solution: Iterable) -> nn.Module:
"""
Make a new policy network parameterized by the given solution.
Note that this parameterized network assumes that the observation
is already normalized, and it does not do action clipping to ensure
that the generated actions are within valid bounds.
To have a policy network which has its own observation normalization
and action clipping layers, please see the method `to_policy(...)`.
Args:
solution: The solution which stores the parameters.
This can be a Solution instance, or a 1-dimensional tensor,
or any Iterable of real numbers.
Returns:
The policy network, as a PyTorch module.
"""
return self.to_policy(solution, with_wrapper_modules=False)
@property
def network_device(self) -> Optional[Device]:
"""
The device on which the policy networks will operate.
Specific to VecGymNE, the network device is determined only
after receiving the first observation from the reinforcement
learning environment. Until then, this property has the value
None.
"""
return self._simulator_device
episode_count: int
property
readonly
¶
Get the total number of episodes completed.
interaction_count: int
property
readonly
¶
Get the total number of simulator interactions made.
max_num_envs: Optional[int]
property
readonly
¶
Maximum number of environments to be allocated.
If a maximum number of environments is not set, then None is returned. If this problem instance is the main one, then the overall maximum number of environments is returned. If this problem instance is a remote one (i.e. is on a remote actor) then the maximum number of environments for that actor is returned.
network_device: Union[str, torch.device]
property
readonly
¶
The device on which the policy networks will operate.
Specific to VecGymNE, the network device is determined only after receiving the first observation from the reinforcement learning environment. Until then, this property has the value None.
__init__(self, env, network, *, env_config=None, max_num_envs=None, network_args=None, observation_normalization=False, decrease_rewards_by=None, alive_bonus_schedule=None, action_noise_stdev=None, num_episodes=1, device=None, num_actors=None, num_gpus_per_actor=None, num_subbatches=None, subbatch_size=None, actor_config=None)
special
¶
Initialize the VecGymNE.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
env |
Union[str, Callable] |
Environment to be solved.
If this is given as a string starting with "gym::" (e.g.
"gym::Humanoid-v4", etc.), then it is assumed that the target
environment is a classical gym environment.
If this is given as a string starting with "brax::" (e.g.
"brax::humanoid", etc.), then it is assumed that the target
environment is a brax environment.
If this is given as a string which does not contain "::" at
all (e.g. "Humanoid-v4", etc.), then it is assumed that the
target environment is a classical gym environment. Therefore,
"gym::Humanoid-v4" and "Humanoid-v4" are equivalent.
If this argument is given as a Callable (maybe a function or a
class), then, with the assumption that this Callable expects
a keyword argument |
required |
network |
Union[str, Callable, torch.nn.modules.module.Module] |
A network structure string, or a Callable (which can be
a class inheriting from |
required |
env_config |
Optional[collections.abc.Mapping] |
Keyword arguments to pass to the environment while it is being created. |
None |
max_num_envs |
Optional[int] |
Maximum number of environments to be instantiated.
By default, this is None, which means that the number of
environments can go up to the population size (or up to the
number of solutions that a remote actor receives, if the
problem object is configured to have parallelization).
For situations where the current reinforcement learning task
requires large amount of resources (e.g. memory), allocating
environments as much as the number of solutions might not
be feasible. In such cases, one can set |
None |
network_args |
Optional[collections.abc.Mapping] |
Any additional keyword argument to be used when
instantiating the network can be specified via |
None |
observation_normalization |
bool |
Whether or not online normalization will be done on the encountered observations. |
False |
decrease_rewards_by |
Optional[float] |
If given as a float, each reward will be
decreased by this amount. For example, if the environment's
reward function has a constant "alive bonus" (i.e. a bonus
that is constantly added onto the reward as long as the
agent is alive), and if you wish to negate this bonus,
you can set |
None |
alive_bonus_schedule |
Optional[tuple] |
Use this to add a customized amount of
alive bonus.
If left as None (which is the default), additional alive
bonus will not be added.
If given as a tuple |
None |
action_noise_stdev |
Optional[float] |
If given as a real number |
None |
num_episodes |
int |
Number of episodes over which each policy will be evaluated. The default is 1. |
1 |
device |
Union[str, torch.device] |
The device in which the population will be kept. If you wish to do a single-GPU evolution, we recommend to set this as "cuda" (or "cuda:0", or "cuda:1", etc.), assuming that the simulator will also instantiate itself on that same device. Alternatively, if you wish to do a multi-GPU evolution, we recommend to leave this as None or set this as "cpu", so that the main population will be kept on the cpu and the remote actors will perform their evaluations on the GPUs that are assigned to them. |
None |
num_actors |
Union[int, str] |
Number of actors to create for parallelized
evaluation of the solutions.
Certain string values are also accepted.
When given as "max" or as "num_cpus", the number of actors
will be equal to the number of all available CPUs in the ray
cluster.
When given as "num_gpus", the number of actors will be
equal to the number of all available GPUs in the ray
cluster, and each actor will be assigned a GPU.
When given as "num_devices", the number of actors will be
equal to the minimum among the number of CPUs and the number
of GPUs available in the cluster (or will be equal to the
number of CPUs if there is no GPU), and each actor will be
assigned a GPU (if available).
If |
None |
num_gpus_per_actor |
Optional[int] |
Number of GPUs to be assigned to each
actor. This can be an integer or a float (for when you
wish to assign fractional amounts of GPUs to actors).
When |
None |
num_subbatches |
Optional[int] |
For when there are multiple actors, you can
set this to an integer n if you wish the population
to be divided exactly into n sub-batches. The actors, as they
finish their currently assigned sub-batch of solutions,
will pick the next un-evaluated sub-batch.
If you specify too large numbers for this argument, then
each sub-batch will be smaller.
When working with vectorized simulators on GPU, having too
many and too small sub-batches can hurt the performance.
This argument can be left as None, in which case, assuming
that |
None |
subbatch_size |
Optional[int] |
For when there are multiple actors, you can
set this to an integer n if you wish the population to be
divided into sub-batches in such a way that each sub-batch
will consist of exactly n solutions. The actors, as they
finish their currently assigned sub-batch of solutions,
will pick the next un-evaluated sub-batch.
If you specify too small numbers for this argument, then
there will be many sub-batches, each sub-batch having a
small number of solutions.
When working with vectorized simulators on GPU, having too
many and too small sub-batches can hurt the performance.
This argument can be left as None, in which case, assuming
that |
None |
actor_config |
Optional[collections.abc.Mapping] |
Additional configuration to be used when creating
each actor with the help of |
None |
Source code in evotorch/neuroevolution/vecgymne.py
def __init__(
self,
env: Union[str, Callable],
network: Union[str, Callable, nn.Module],
*,
env_config: Optional[Mapping] = None,
max_num_envs: Optional[int] = None,
network_args: Optional[Mapping] = None,
observation_normalization: bool = False,
decrease_rewards_by: Optional[float] = None,
alive_bonus_schedule: Optional[tuple] = None,
action_noise_stdev: Optional[float] = None,
num_episodes: int = 1,
device: Optional[Device] = None,
num_actors: Optional[Union[int, str]] = None,
num_gpus_per_actor: Optional[int] = None,
num_subbatches: Optional[int] = None,
subbatch_size: Optional[int] = None,
actor_config: Optional[Mapping] = None,
):
"""
Initialize the VecGymNE.
Args:
env: Environment to be solved.
If this is given as a string starting with "gym::" (e.g.
"gym::Humanoid-v4", etc.), then it is assumed that the target
environment is a classical gym environment.
If this is given as a string starting with "brax::" (e.g.
"brax::humanoid", etc.), then it is assumed that the target
environment is a brax environment.
If this is given as a string which does not contain "::" at
all (e.g. "Humanoid-v4", etc.), then it is assumed that the
target environment is a classical gym environment. Therefore,
"gym::Humanoid-v4" and "Humanoid-v4" are equivalent.
If this argument is given as a Callable (maybe a function or a
class), then, with the assumption that this Callable expects
a keyword argument `num_envs: int`, this Callable is called
and its result (expected as a `gym.vector.VectorEnv` instance)
is used as the environment.
network: A network structure string, or a Callable (which can be
a class inheriting from `torch.nn.Module`, or a function
which returns a `torch.nn.Module` instance), or an instance
of `torch.nn.Module`.
The object provided here determines the structure of the
neural network whose parameters will be evolved.
A network structure string is a string which can be processed
by `evotorch.neuroevolution.net.str_to_net(...)`.
Please see the documentation of the function
`evotorch.neuroevolution.net.str_to_net(...)` to see how such
a neural network structure string looks like.
Note that this network can be a recurrent network.
When the network's `forward(...)` method can optionally accept
an additional positional argument for the hidden state of the
network and returns an additional value for its next state,
then the policy is treated as a recurrent one.
When the network is given as a callable object (e.g.
a subclass of `nn.Module` or a function) and this callable
object is decorated via `evotorch.decorators.pass_info`,
the following keyword arguments will be passed:
(i) `obs_length` (the length of the observation vector),
(ii) `act_length` (the length of the action vector),
(iii) `obs_shape` (the shape tuple of the observation space),
(iv) `act_shape` (the shape tuple of the action space),
(v) `obs_space` (the Box object specifying the observation
space, and
(vi) `act_space` (the Box object specifying the action
space). Note that `act_space` will always be given as a
`gym.spaces.Box` instance, even when the actual gym
environment has a discrete action space. This because
`VecGymNE` always expects the neural network to return
a tensor of floating-point numbers.
env_config: Keyword arguments to pass to the environment while
it is being created.
max_num_envs: Maximum number of environments to be instantiated.
By default, this is None, which means that the number of
environments can go up to the population size (or up to the
number of solutions that a remote actor receives, if the
problem object is configured to have parallelization).
For situations where the current reinforcement learning task
requires large amount of resources (e.g. memory), allocating
environments as much as the number of solutions might not
be feasible. In such cases, one can set `max_num_envs` as an
integer to bring an upper bound (in total, across all the
remote actors, for when the problem is parallelized) to how
many environments can be allocated.
network_args: Any additional keyword argument to be used when
instantiating the network can be specified via `network_args`
as a dictionary. If there are no such additional keyword
arguments, then `network_args` can be left as None.
Note that the argument `network_args` is expected to be None
when the network is specified as a `torch.nn.Module` instance.
observation_normalization: Whether or not online normalization
will be done on the encountered observations.
decrease_rewards_by: If given as a float, each reward will be
decreased by this amount. For example, if the environment's
reward function has a constant "alive bonus" (i.e. a bonus
that is constantly added onto the reward as long as the
agent is alive), and if you wish to negate this bonus,
you can set `decrease_rewards_by` to this bonus amount,
and the bonus will be nullified.
If you do not wish to affect the rewards in this manner,
keep this as None.
alive_bonus_schedule: Use this to add a customized amount of
alive bonus.
If left as None (which is the default), additional alive
bonus will not be added.
If given as a tuple `(t, b)`, an alive bonus `b` will be
added onto all the rewards beyond the timestep `t`.
If given as a tuple `(t0, t1, b)`, a partial (linearly
increasing towards `b`) alive bonus will be added onto
all the rewards between the timesteps `t0` and `t1`,
and a full alive bonus (which equals to `b`) will be added
onto all the rewards beyond the timestep `t1`.
action_noise_stdev: If given as a real number `s`, then, for
each generated action, Gaussian noise with standard
deviation `s` will be sampled, and then this sampled noise
will be added onto the action.
If action noise is not desired, then this argument can be
left as None.
For sampling the noise, the global random number generator
of PyTorch on the simulator's device will be used.
num_episodes: Number of episodes over which each policy will
be evaluated. The default is 1.
device: The device in which the population will be kept.
If you wish to do a single-GPU evolution, we recommend
to set this as "cuda" (or "cuda:0", or "cuda:1", etc.),
assuming that the simulator will also instantiate itself
on that same device.
Alternatively, if you wish to do a multi-GPU evolution,
we recommend to leave this as None or set this as "cpu",
so that the main population will be kept on the cpu
and the remote actors will perform their evaluations on
the GPUs that are assigned to them.
num_actors: Number of actors to create for parallelized
evaluation of the solutions.
Certain string values are also accepted.
When given as "max" or as "num_cpus", the number of actors
will be equal to the number of all available CPUs in the ray
cluster.
When given as "num_gpus", the number of actors will be
equal to the number of all available GPUs in the ray
cluster, and each actor will be assigned a GPU.
When given as "num_devices", the number of actors will be
equal to the minimum among the number of CPUs and the number
of GPUs available in the cluster (or will be equal to the
number of CPUs if there is no GPU), and each actor will be
assigned a GPU (if available).
If `num_actors` is given as "num_gpus" or "num_devices",
the argument `num_gpus_per_actor` must not be used,
and the `actor_config` dictionary must not contain the
key "num_gpus".
If `num_actors` is given as something other than "num_gpus"
or "num_devices", and if you wish to assign GPUs to each
actor, then please see the argument `num_gpus_per_actor`.
num_gpus_per_actor: Number of GPUs to be assigned to each
actor. This can be an integer or a float (for when you
wish to assign fractional amounts of GPUs to actors).
When `num_actors` has the special value "num_devices",
the argument `num_gpus_per_actor` is expected to be left as
None.
num_subbatches: For when there are multiple actors, you can
set this to an integer n if you wish the population
to be divided exactly into n sub-batches. The actors, as they
finish their currently assigned sub-batch of solutions,
will pick the next un-evaluated sub-batch.
If you specify too large numbers for this argument, then
each sub-batch will be smaller.
When working with vectorized simulators on GPU, having too
many and too small sub-batches can hurt the performance.
This argument can be left as None, in which case, assuming
that `subbatch_size` is also None, the population will be
split to m sub-batches, m being the number of actors.
subbatch_size: For when there are multiple actors, you can
set this to an integer n if you wish the population to be
divided into sub-batches in such a way that each sub-batch
will consist of exactly n solutions. The actors, as they
finish their currently assigned sub-batch of solutions,
will pick the next un-evaluated sub-batch.
If you specify too small numbers for this argument, then
there will be many sub-batches, each sub-batch having a
small number of solutions.
When working with vectorized simulators on GPU, having too
many and too small sub-batches can hurt the performance.
This argument can be left as None, in which case, assuming
that `num_subbatches` is also None, the population will be
split to m sub-batches, m being the number of actors.
actor_config: Additional configuration to be used when creating
each actor with the help of `ray` library.
Can be left as None if additional configuration is not needed.
"""
# Store the string or the Callable that will be used to generate the reinforcement learning environment.
self._env_maker = env
# Declare the variable which will store the environment.
self._env: Optional[TorchWrapper] = None
# Declare the variable which will store the batch size of the vectorized environment.
self._num_envs: Optional[int] = None
# Store the upper bound (if any) regarding how many environments can exist at the same time.
self._max_num_envs: Optional[int] = None if max_num_envs is None else int(max_num_envs)
# Actor-specific upper bound regarding how many environments can exist at the same time.
# This variable will be filled by the `_parallelize(...)` method.
self._actor_max_num_envs: Optional[int] = None
# Declare the variable which stores whether or not we properly initialized the `_actor_max_num_envs` variable.
self._actor_max_num_envs_ready: bool = False
# Store the additional configurations to be used as keyword arguments while instantiating the environment.
self._env_config: dict = {} if env_config is None else dict(env_config)
# Declare the variable that will store the device of the simulator.
# This variable will be filled when the first observation is received from the environment.
# The device of the observation array received from the environment will determine the value of this variable.
self._simulator_device: Optional[torch.device] = None
# Store the neural network architecture (that might be a string or an `nn.Module` instance).
self._architecture = network
if network_args is None:
# If `network_args` is given as None, change it to an empty dictionary
network_args = {}
if isinstance(network, str):
# If the network is given as a string, then we will need the values for the constants `obs_length`,
# `act_length`, and `obs_space`. To obtain those values, we use our helper function
# `_env_constants_for_str_net(...)` which temporarily instantiates the specified environment and returns
# its needed constants.
env_constants = _env_constants_for_str_net(self._env_maker, **(self._env_config))
elif isinstance(network, nn.Module):
# If the network is an already instantiated nn.Module, then we do not prepare any pre-defined constants.
env_constants = {}
else:
# If the network is given as a Callable, then we will need the values for the constants `obs_length`,
# `act_length`, and `obs_space`. To obtain those values, we use our helper function
# `_env_constants_for_callable_net(...)` which temporarily instantiates the specified environment and
# returns its needed constants.
env_constants = _env_constants_for_callable_net(self._env_maker, **(self._env_config))
# Build a `Policy` instance according to the given architecture, and store it.
if isinstance(network, str):
instantiated_net = str_to_net(network, **{**env_constants, **network_args})
elif isinstance(network, nn.Module):
instantiated_net = network
else:
instantiated_net = pass_info_if_needed(network, env_constants)(**network_args)
self._policy = Policy(instantiated_net)
# Store the boolean which indicates whether or not there will be observation normalization.
self._observation_normalization = bool(observation_normalization)
# Declare the variables that will store the observation-related stats if observation normalization is enabled.
self._obs_stats: Optional[RunningNorm] = None
self._collected_stats: Optional[RunningNorm] = None
# Store the number of episodes configuration given by the user.
self._num_episodes = int(num_episodes)
# Store the `decrease_rewards_by` configuration given by the user.
self._decrease_rewards_by = None if decrease_rewards_by is None else float(decrease_rewards_by)
if alive_bonus_schedule is None:
# If `alive_bonus_schedule` argument is None, then we store it as None as well.
self._alive_bonus_schedule = None
else:
# This is the case where the user has specified an `alive_bonus_schedule`.
alive_bonus_schedule = list(alive_bonus_schedule)
alive_bonus_schedule_length = len(alive_bonus_schedule)
if alive_bonus_schedule_length == 2:
# If `alive_bonus_schedule` was given as a 2-element sequence (t, b), then store it as (t, t, b).
# This means that the partial alive bonus time window starts and ends at t, therefore, there will
# be no alive bonus until t, and beginning with t, there will be full alive bonus.
self._alive_bonus_schedule = [
int(alive_bonus_schedule[0]),
int(alive_bonus_schedule[0]),
float(alive_bonus_schedule[1]),
]
elif alive_bonus_schedule_length == 3:
# If `alive_bonus_schedule` was given as a 3-element sequence (t0, t1, b), then store those 3
# elements.
self._alive_bonus_schedule = [
int(alive_bonus_schedule[0]),
int(alive_bonus_schedule[1]),
float(alive_bonus_schedule[2]),
]
else:
# `alive_bonus_schedule` sequences with unrecognized lengths trigger an error.
raise ValueError(
f"Received invalid number elements as the alive bonus schedule."
f" Expected 2 or 3 items, but got these: {self._alive_bonus_schedule}"
f" (having a length of {len(self._alive_bonus_schedule)})."
)
# If `action_noise_stdev` is specified, store it.
self._action_noise_stdev = None if action_noise_stdev is None else float(action_noise_stdev)
# Initialize the counters for the number of simulator interactions and the number of episodes.
self._interaction_count: int = 0
self._episode_count: int = 0
device_is_cpu = (device is None) or (str(device) == "cpu")
actors_use_gpu = (
(num_actors is not None)
and (num_actors > 1)
and (num_gpus_per_actor is not None)
and (num_gpus_per_actor > 0)
)
if not device_is_cpu:
# In the case where the device is something other than the cpu, we tell SyncVectorEnv to use this device.
self._device_for_sync_vector_env = device
self._sync_vector_env_uses_aux_device = False
elif actors_use_gpu:
# In the case where this problem instance is configured to use multiple actors and the actors are
# configured to use the available gpu(s), we tell SyncVectorEnv to use the `aux_device`.
self._device_for_sync_vector_env = None
self._sync_vector_env_uses_aux_device = True
else:
self._device_for_sync_vector_env = None
self._sync_vector_env_uses_aux_device = False
# Call the superclass
super().__init__(
objective_sense="max",
initial_bounds=(-0.00001, 0.00001),
solution_length=self._policy.parameter_length,
device=device,
dtype=torch.float32,
num_actors=num_actors,
num_gpus_per_actor=num_gpus_per_actor,
actor_config=actor_config,
num_subbatches=num_subbatches,
subbatch_size=subbatch_size,
)
self.after_eval_hook.append(self._extra_status)
get_env(self)
¶
Get the gym environment.
Returns:
| Type | Description |
|---|---|
Optional[gymnasium.core.Env] |
The gym environment if it is built. If not built yet, None. |
get_observation_stats(self)
¶
make_net(self, solution)
¶
Make a new policy network parameterized by the given solution. Note that this parameterized network assumes that the observation is already normalized, and it does not do action clipping to ensure that the generated actions are within valid bounds.
To have a policy network which has its own observation normalization
and action clipping layers, please see the method to_policy(...).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
solution |
Iterable |
The solution which stores the parameters. This can be a Solution instance, or a 1-dimensional tensor, or any Iterable of real numbers. |
required |
Returns:
| Type | Description |
|---|---|
Module |
The policy network, as a PyTorch module. |
Source code in evotorch/neuroevolution/vecgymne.py
def make_net(self, solution: Iterable) -> nn.Module:
"""
Make a new policy network parameterized by the given solution.
Note that this parameterized network assumes that the observation
is already normalized, and it does not do action clipping to ensure
that the generated actions are within valid bounds.
To have a policy network which has its own observation normalization
and action clipping layers, please see the method `to_policy(...)`.
Args:
solution: The solution which stores the parameters.
This can be a Solution instance, or a 1-dimensional tensor,
or any Iterable of real numbers.
Returns:
The policy network, as a PyTorch module.
"""
return self.to_policy(solution, with_wrapper_modules=False)
pop_observation_stats(self)
¶
save_solution(self, solution, fname)
¶
Save the solution into a pickle file.
Among the saved data within the pickle file are the solution
(as a PyTorch tensor), the policy (as a torch.nn.Module instance),
and observation stats (if any).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
solution |
Iterable |
The solution to be saved. This can be a PyTorch tensor,
a |
required |
fname |
Union[str, pathlib.Path] |
The file name of the pickle file to be created. |
required |
Source code in evotorch/neuroevolution/vecgymne.py
def save_solution(self, solution: Iterable, fname: Union[str, Path]):
"""
Save the solution into a pickle file.
Among the saved data within the pickle file are the solution
(as a PyTorch tensor), the policy (as a `torch.nn.Module` instance),
and observation stats (if any).
Args:
solution: The solution to be saved. This can be a PyTorch tensor,
a `Solution` instance, or any `Iterable`.
fname: The file name of the pickle file to be created.
"""
# Convert the solution to a PyTorch tensor on the cpu.
if isinstance(solution, torch.Tensor):
solution = solution.to("cpu")
elif isinstance(solution, Solution):
solution = solution.values.clone().to("cpu")
else:
solution = torch.as_tensor(solution, dtype=torch.float32, device="cpu")
if isinstance(solution, ReadOnlyTensor):
solution = solution.as_subclass(torch.Tensor)
# Store the solution and the policy.
result = {
"solution": solution,
"policy": self.to_policy(solution),
}
# If available, store the observation stats.
if self.observation_normalization and (self._obs_stats is not None):
result["obs_mean"] = self._obs_stats.mean.to("cpu")
result["obs_stdev"] = self._obs_stats.stdev.to("cpu")
result["obs_sum"] = self._obs_stats.sum.to("cpu")
result["obs_sum_of_squares"] = self._obs_stats.sum_of_squares.to("cpu")
# Some additional data.
result["interaction_count"] = self.interaction_count
result["episode_count"] = self.episode_count
result["time"] = datetime.now()
if isinstance(self._env_maker, str):
# If the environment was specified via a string, store the string.
result["env"] = self._env_maker
# Store the network architecture.
result["architecture"] = self._architecture
# Save the dictionary which stores the data.
with open(fname, "wb") as f:
pickle.dump(result, f)
set_episode_count(self, n)
¶
set_interaction_count(self, n)
¶
set_observation_stats(self, rn)
¶
to_policy(self, solution, *, with_wrapper_modules=True)
¶
Convert the given solution to a policy.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
solution |
Iterable |
A solution which can be given as a |
required |
with_wrapper_modules |
bool |
Whether or not to wrap the policy module with helper modules so that observations are normalized and actions are clipped to be within the correct boundaries. The default and the recommended value is True. |
True |
Returns:
| Type | Description |
|---|---|
Module |
The policy, as a |
Source code in evotorch/neuroevolution/vecgymne.py
def to_policy(self, solution: Iterable, *, with_wrapper_modules: bool = True) -> nn.Module:
"""
Convert the given solution to a policy.
Args:
solution: A solution which can be given as a `torch.Tensor`, as a
`Solution`, or as any `Iterable`.
with_wrapper_modules: Whether or not to wrap the policy module
with helper modules so that observations are normalized
and actions are clipped to be within the correct boundaries.
The default and the recommended value is True.
Returns:
The policy, as a `torch.nn.Module` instance.
"""
# Get the gym environment
env = self._get_env(1)
# Get the observation space, its lower and higher bounds.
obs_space = env.single_action_space
low = obs_space.low
high = obs_space.high
# If the lower and higher bounds are not -inf and +inf respectively, then this environment needs clipping.
needs_clipping = _numpy_arrays_specify_bounds(low, high)
# Convert the solution to a PyTorch tensor on cpu.
if isinstance(solution, torch.Tensor):
solution = solution.to("cpu")
elif isinstance(solution, Solution):
solution = solution.values.clone().to("cpu")
else:
solution = torch.as_tensor(solution, dtype=torch.float32, device="cpu")
# Convert the internally stored policy to a PyTorch module.
result = self._policy.to_torch_module(solution)
if with_wrapper_modules:
if self.observation_normalization and (self._obs_stats is not None):
# If observation normalization is needed and there are collected observation stats, then we wrap the
# policy with an ObsNormWrapperModule.
result = ObsNormWrapperModule(result, self._obs_stats)
if needs_clipping:
# If clipping is needed, then we wrap the policy with an ActClipWrapperModule
result = ActClipWrapperModule(result, obs_space)
return result
update_observation_stats(self, rn)
¶
Update the observation stats via another RunningNorm instance
operators
special
¶
This module provides various common operators to be used within evolutionary algorithms.
Each operator is provided as a separate class, which is to be instantiated in this form:
op = OperatorName(
problem, # where problem is a Problem instance
hyperparameter1=...,
hyperparameter2=...,
# ...
)
Each operator has its __call__(...) method overriden so that it can be used
like a function. For example, if the operator op instantiated above were a
mutation operator, it would be used like this:
Please see the documentations of the provided operator classes for details about how to instantiate them, and how to call them.
A common usage for the operators provided here is to use them with GeneticAlgorithm, as shown below:
from evotorch.algorithms import GeneticAlgorithm
from evotorch.operators import SimulatedBinaryCrossOver, GaussianMutation
problem = ... # initialize the Problem
ga = GeneticAlgorithm(
problem,
operators=[
SimulatedBinaryCrossOver(
problem,
tournament_size=...,
cross_over_rate=...,
eta=...,
),
GaussianMutation(
problem,
stdev=...,
),
],
popsize=...,
)
base
¶
Base classes for various operators
CopyingOperator (Operator)
¶
Base class for operators which do not do in-place modifications.
This class does not add any functionality to the Operator class.
Instead, the annotations of the __call__(...) method is
updated so that it makes it clear that a new SolutionBatch is
returned.
One is expected to override the definition of the method _do(...)
in an inheriting subclass to define a custom CopyingOperator.
From outside, a subclass of CopyingOperator is meant to be called like
a function, as follows:
my_new_batch = my_copying_operator_instance(my_batch)
Source code in evotorch/operators/base.py
class CopyingOperator(Operator):
"""
Base class for operators which do not do in-place modifications.
This class does not add any functionality to the Operator class.
Instead, the annotations of the `__call__(...)` method is
updated so that it makes it clear that a new SolutionBatch is
returned.
One is expected to override the definition of the method `_do(...)`
in an inheriting subclass to define a custom `CopyingOperator`.
From outside, a subclass of `CopyingOperator` is meant to be called like
a function, as follows:
my_new_batch = my_copying_operator_instance(my_batch)
"""
def __init__(self, problem: Problem):
"""
`__init__(...)`: Initialize the CopyingOperator.
Args:
problem: The problem object which is being worked on.
"""
super().__init__(problem)
def __call__(self, batch: SolutionBatch) -> SolutionBatch:
return self._do(batch)
def _do(self, batch: SolutionBatch) -> SolutionBatch:
"""The actual definition of the operation on the batch.
Expected to be overriden by a subclass.
"""
raise NotImplementedError
__init__(self, problem)
special
¶
__init__(...): Initialize the CopyingOperator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object which is being worked on. |
required |
CrossOver (CopyingOperator)
¶
Base class for any CrossOver operator.
One is expected to override the definition of the method
_do_cross_over(...) in an inheriting subclass to define a
custom CrossOver.
From outside, a CrossOver instance is meant to be called like this:
child_solution_batch = my_cross_over_instance(population_batch)
which causes the CrossOver instance to select parents from the
population_batch, recombine their values according to what is
instructed in _do_cross_over(...), and return the newly made solutions
in a SolutionBatch.
Source code in evotorch/operators/base.py
class CrossOver(CopyingOperator):
"""
Base class for any CrossOver operator.
One is expected to override the definition of the method
`_do_cross_over(...)` in an inheriting subclass to define a
custom `CrossOver`.
From outside, a `CrossOver` instance is meant to be called like this:
child_solution_batch = my_cross_over_instance(population_batch)
which causes the `CrossOver` instance to select parents from the
`population_batch`, recombine their values according to what is
instructed in `_do_cross_over(...)`, and return the newly made solutions
in a `SolutionBatch`.
"""
def __init__(
self,
problem: Problem,
*,
tournament_size: int,
obj_index: Optional[int] = None,
num_children: Optional[int] = None,
cross_over_rate: Optional[float] = None,
):
"""
`__init__(...)`: Initialize the CrossOver.
Args:
problem: The problem object which is being worked on.
tournament_size: Size of the tournament which will be used for
doing selection.
obj_index: Index of the objective according to which the selection
will be done.
If `obj_index` is None and the problem is single-objective,
then the selection will be done according to that single
objective.
If `obj_index` is None and the problem is multi-objective,
then the selection will be done according to pareto-dominance
and crowding criteria, as done in NSGA-II.
If `obj_index` is an integer `i`, then the selection will be
done according to the i-th objective only, even when the
problem is multi-objective.
num_children: How many children to generate.
Expected as an even number.
Cannot be used together with `cross_over_rate`.
cross_over_rate: Rate of the cross-over operations in comparison
with the population size.
1.0 means that the number of generated children will be equal
to the original population size.
Cannot be used together with `num_children`.
"""
super().__init__(problem)
self._obj_index = None if obj_index is None else problem.normalize_obj_index(obj_index)
self._tournament_size = int(tournament_size)
if num_children is not None and cross_over_rate is not None:
raise ValueError(
"Received both `num_children` and `cross_over_rate` as values other than None."
" It was expected to receive both of them as None, or one of them as None,"
" but not both of them as values other than None."
)
self._num_children = None if num_children is None else int(num_children)
self._cross_over_rate = None if cross_over_rate is None else float(cross_over_rate)
def _compute_num_tournaments(self, batch: SolutionBatch) -> int:
if self._num_children is None and self._cross_over_rate is None:
# return len(batch) * 2
result = len(batch)
if (result % 2) != 0:
result += 1
return result
elif self._num_children is not None:
if (self._num_children % 2) != 0:
raise ValueError(
f"The initialization argument `num_children` was expected as an even number."
f" However, it was found as an odd number: {self._num_children}"
)
return self._num_children
elif self._cross_over_rate is not None:
f = len(batch) * self._cross_over_rate
result1 = math.ceil(f)
result2 = math.floor(f)
if result1 == result2:
result = result1
if (result % 2) != 0:
result += 1
else:
if (result1 % 2) == 0:
result = result1
else:
result = result2
return result
else:
assert False, "Exection should not have reached this point"
@property
def obj_index(self) -> Optional[int]:
"""The objective index according to which the selection will be done"""
return self._obj_index
@torch.no_grad()
def _do_tournament(self, batch: SolutionBatch) -> tuple:
# Compute the required number of tournaments
num_tournaments = self._compute_num_tournaments(batch)
if self._problem.is_multi_objective and self._obj_index is None:
# If the problem is multi-objective, and an objective index is not specified,
# then we do a multi-objective-specific cross-over
# At first, pareto-sort the solutions
ranks, _ = batch.compute_pareto_ranks(crowdsort=False)
n_fronts = torch.amax(ranks) + 1
# In NSGA-II-inspired pareto-sorting, smallest rank means the best front.
# Right now, we want the opposite: we want the solutions in the best front
# to have rank values which are numerically highest.
# The following line re-arranges the rank values such that the solutions
# in the best front have their ranks equal to n_fronts, and the ones
# in the worst front have their ranks equal to 1.
ranks = (n_fronts - ranks).to(torch.float)
# Because the ranks are computed front the fronts indices, we expect many
# solutions to end up with the same rank values.
# To ensure that a randomized selection will be made when comparing two
# solutions with the same rank, we add random noise to the ranks
# (between 0.0 and 0.1).
ranks += self._problem.make_uniform(len(batch), dtype=self._problem.eval_dtype, device=batch.device) * 0.1
else:
# Rank the solutions. Worst gets -0.5, best gets 0.5
ranks = batch.utility(self._obj_index, ranking_method="centered")
# Get the internal values tensor of the solution batch
indata = batch._data
# Get a tensor of random integers in the shape (num_tournaments, tournament_size)
tournament_indices = self.problem.make_randint(
(num_tournaments, self._tournament_size), n=len(batch), device=indata.device
)
tournament_ranks = ranks[tournament_indices]
# Imagine tournament size is 2, and the solutions are [ worst, bad, best, good ].
# So, what we have is (0.2s are actually 0.166666...):
#
# ranks = [ -0.5, -0.2, 0.5, 0.2 ]
#
# tournament tournament
# indices ranks
#
# 0, 1 -0.5, -0.2
# 2, 3 0.5, 0.2
# 1, 0 -0.2, -0.5
# 3, 2 0.2, 0.5
# 1, 2 -0.2, 0.5
# 0, 3 -0.5, 0.2
# 2, 0 0.5, -0.5
# 3, 1 0.2, -0.2
#
# According to tournament_indices, there are 8 tournaments.
# In tournament 0 (topmost row), parent0 and parent1 compete.
# In tournament 1 (next row), parent2 and parent3 compete; and so on.
# tournament_ranks tells us:
# In tournament 0, left-candidate has rank -0.5, and right-candidate has -0.2.
# In tournament 1, left-candidate has rank 0.5, and right-candidate has 0.2; and so on.
tournament_rows = torch.arange(0, num_tournaments, device=indata.device)
parents = tournament_indices[tournament_rows, torch.argmax(tournament_ranks, dim=-1)]
# Continuing from the [ worst, bad, best, good ] example, we end up with:
#
# T T
# tournament tournament tournament argmax parents
# rows indices ranks dim=-1
#
# 0 0, 1 -0.5, -0.2 1 1
# 1 2, 3 0.5, 0.2 0 2
# 2 1, 0 -0.2, -0.5 0 1
# 3 3, 2 0.2, 0.5 1 2
# 4 1, 2 -0.2, 0.5 1 2
# 5 0, 3 -0.5, 0.2 1 3
# 6 2, 0 0.5, -0.5 0 2
# 7 3, 1 0.2, -0.2 0 3
#
# where tournament_rows represents row indices in tournament_indices tensor (from 0 to 7).
# argmax() tells us who won the competition (0: left-candidate won, 1: right-candidate won).
#
# tournament_rows and argmax() together give us the row and column of the winner in tensor
# tournament_indices, which in turn gives us the index of the winner solution in the batch.
# We split the parents array from the middle
split_point = int(len(parents) / 2)
parents1 = indata[parents][:split_point]
parents2 = indata[parents][split_point:]
# We now have:
#
# parents1 parents2
# =============== ===============
# values of sln 1 values of sln 2 (solution1 is to generate a child with solution2)
# values of sln 2 values of sln 3 (solution2 is to generate a child with solution3)
# values of sln 1 values of sln 2 (solution1 is to generate another child with solution2)
# values of sln 2 values of sln 3 (solution2 is to generate another child with solution3)
#
# With this, the tournament selection phase is over.
return parents1, parents2
def _do_cross_over(
self,
parents1: Union[torch.Tensor, ObjectArray],
parents2: Union[torch.Tensor, ObjectArray],
) -> SolutionBatch:
"""
The actual definition of the cross-over operation.
This is a protected method, meant to be overriden by the inheriting
subclass.
The arguments passed to this function are the decision values of the
first and the second half of the selected parents, both as PyTorch
tensors or as `ObjectArray`s.
In the overriding function, for each integer i, one is expected to
recombine the values of the i-th row of `parents1` with the values of
the i-th row of `parents2` twice (twice because each pairing is
expected to generate two children).
After that, one is expected to generate a SolutionBatch and place
all the recombination results into the values of that new batch.
Args:
parents1: The decision values of the first half of the
selected parents.
parents2: The decision values of the second half of the
selected parents.
Returns:
A new SolutionBatch which contains the recombination
of the parents.
"""
raise NotImplementedError
def _make_children_batch(self, child_values: Union[torch.Tensor, ObjectArray]) -> SolutionBatch:
result = SolutionBatch(self.problem, device=child_values.device, empty=True, popsize=child_values.shape[0])
result._data = child_values
return result
def _do(self, batch: SolutionBatch) -> SolutionBatch:
parents1, parents2 = self._do_tournament(batch)
if len(parents1) != len(parents2):
raise ValueError(
f"_do_tournament() returned parents1 and parents2 with incompatible sizes. "
f"len(parents1): {len(parents1)}; len(parents2): {len(parents2)}."
)
return self._do_cross_over(parents1, parents2)
obj_index: Optional[int]
property
readonly
¶
The objective index according to which the selection will be done
__init__(self, problem, *, tournament_size, obj_index=None, num_children=None, cross_over_rate=None)
special
¶
__init__(...): Initialize the CrossOver.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object which is being worked on. |
required |
tournament_size |
int |
Size of the tournament which will be used for doing selection. |
required |
obj_index |
Optional[int] |
Index of the objective according to which the selection
will be done.
If |
None |
num_children |
Optional[int] |
How many children to generate.
Expected as an even number.
Cannot be used together with |
None |
cross_over_rate |
Optional[float] |
Rate of the cross-over operations in comparison
with the population size.
1.0 means that the number of generated children will be equal
to the original population size.
Cannot be used together with |
None |
Source code in evotorch/operators/base.py
def __init__(
self,
problem: Problem,
*,
tournament_size: int,
obj_index: Optional[int] = None,
num_children: Optional[int] = None,
cross_over_rate: Optional[float] = None,
):
"""
`__init__(...)`: Initialize the CrossOver.
Args:
problem: The problem object which is being worked on.
tournament_size: Size of the tournament which will be used for
doing selection.
obj_index: Index of the objective according to which the selection
will be done.
If `obj_index` is None and the problem is single-objective,
then the selection will be done according to that single
objective.
If `obj_index` is None and the problem is multi-objective,
then the selection will be done according to pareto-dominance
and crowding criteria, as done in NSGA-II.
If `obj_index` is an integer `i`, then the selection will be
done according to the i-th objective only, even when the
problem is multi-objective.
num_children: How many children to generate.
Expected as an even number.
Cannot be used together with `cross_over_rate`.
cross_over_rate: Rate of the cross-over operations in comparison
with the population size.
1.0 means that the number of generated children will be equal
to the original population size.
Cannot be used together with `num_children`.
"""
super().__init__(problem)
self._obj_index = None if obj_index is None else problem.normalize_obj_index(obj_index)
self._tournament_size = int(tournament_size)
if num_children is not None and cross_over_rate is not None:
raise ValueError(
"Received both `num_children` and `cross_over_rate` as values other than None."
" It was expected to receive both of them as None, or one of them as None,"
" but not both of them as values other than None."
)
self._num_children = None if num_children is None else int(num_children)
self._cross_over_rate = None if cross_over_rate is None else float(cross_over_rate)
Operator
¶
Base class for various operations on SolutionBatch objects.
Some subclasses of Operator may be operating on the batches in-place, while some others may generate new batches, leaving the original batches untouched.
One is expected to override the definition of the method _do(...)
in an inheriting subclass to define a custom Operator.
From outside, a subclass of Operator is meant to be called like a function. In more details, operators which apply in-place modifications are meant to be called like this:
my_operator_instance(my_batch)
Operators which return a new batch are meant to be called like this:
my_new_batch = my_operator_instance(my_batch)
Source code in evotorch/operators/base.py
class Operator:
"""Base class for various operations on SolutionBatch objects.
Some subclasses of Operator may be operating on the batches in-place,
while some others may generate new batches, leaving the original batches
untouched.
One is expected to override the definition of the method `_do(...)`
in an inheriting subclass to define a custom `Operator`.
From outside, a subclass of Operator is meant to be called like
a function. In more details, operators which apply in-place modifications
are meant to be called like this:
my_operator_instance(my_batch)
Operators which return a new batch are meant to be called like this:
my_new_batch = my_operator_instance(my_batch)
"""
def __init__(self, problem: Problem):
"""
`__init__(...)`: Initialize the Operator.
Args:
problem: The problem object which is being worked on.
"""
if not isinstance(problem, Problem):
raise TypeError(f"Expected a Problem object, but received {repr(problem)}")
self._problem = problem
self._lb = clone(self._problem.lower_bounds)
self._ub = clone(self._problem.upper_bounds)
@property
def problem(self) -> Problem:
"""Get the problem to which this cross-over operator is bound"""
return self._problem
@property
def dtype(self) -> DType:
"""Get the dtype of the bound problem.
If the problem does not work with Solution and
therefore it does not have a dtype, None is returned.
"""
return self.problem.dtype
@torch.no_grad()
def _respect_bounds(self, x: torch.Tensor) -> torch.Tensor:
"""
Make sure that a given PyTorch tensor respects the problem's bounds.
This is a protected method which might be used by the
inheriting subclasses to ensure that the result of their
various operations are clipped properly to respect the
boundaries set by the problem object.
Note that this function might return the tensor itself
is the problem is not bounded.
Args:
x: The PyTorch tensor to be clipped.
Returns:
The clipped tensor.
"""
if self._lb is not None:
self._lb = torch.as_tensor(self._lb, dtype=x.dtype, device=x.device)
x = torch.max(self._lb, x)
if self._ub is not None:
self._ub = torch.as_tensor(self._ub, dtype=x.dtype, device=x.device)
x = torch.min(self._ub, x)
return x
def __call__(self, batch: SolutionBatch):
"""
Apply the operator on the given batch.
"""
if not isinstance(batch, SolutionBatch):
raise TypeError(
f"The operation {self.__class__.__name__} can only work on"
f" SolutionBatch objects, but it received an object of type"
f" {repr(type(batch))}."
)
self._do(batch)
def _do(self, batch: SolutionBatch):
"""
The actual definition of the operation on the batch.
Expected to be overriden by a subclass.
"""
raise NotImplementedError
dtype: Union[str, torch.dtype, numpy.dtype, Type]
property
readonly
¶
Get the dtype of the bound problem. If the problem does not work with Solution and therefore it does not have a dtype, None is returned.
problem: Problem
property
readonly
¶
Get the problem to which this cross-over operator is bound
__call__(self, batch)
special
¶
Apply the operator on the given batch.
Source code in evotorch/operators/base.py
def __call__(self, batch: SolutionBatch):
"""
Apply the operator on the given batch.
"""
if not isinstance(batch, SolutionBatch):
raise TypeError(
f"The operation {self.__class__.__name__} can only work on"
f" SolutionBatch objects, but it received an object of type"
f" {repr(type(batch))}."
)
self._do(batch)
__init__(self, problem)
special
¶
__init__(...): Initialize the Operator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object which is being worked on. |
required |
Source code in evotorch/operators/base.py
def __init__(self, problem: Problem):
"""
`__init__(...)`: Initialize the Operator.
Args:
problem: The problem object which is being worked on.
"""
if not isinstance(problem, Problem):
raise TypeError(f"Expected a Problem object, but received {repr(problem)}")
self._problem = problem
self._lb = clone(self._problem.lower_bounds)
self._ub = clone(self._problem.upper_bounds)
SingleObjOperator (Operator)
¶
Base class for all the operators which focus on only one objective.
One is expected to override the definition of the method _do(...)
in an inheriting subclass to define a custom SingleObjOperator.
Source code in evotorch/operators/base.py
class SingleObjOperator(Operator):
"""
Base class for all the operators which focus on only one objective.
One is expected to override the definition of the method `_do(...)`
in an inheriting subclass to define a custom `SingleObjOperator`.
"""
def __init__(self, problem: Problem, obj_index: Optional[int] = None):
"""
Initialize the SingleObjOperator.
Args:
problem: The problem object which is being worked on.
obj_index: Index of the objective to focus on.
Can be given as None if the problem is single-objective.
"""
super().__init__(problem)
self._obj_index: int = problem.normalize_obj_index(obj_index)
@property
def obj_index(self) -> int:
"""Index of the objective on which this operator is to be applied"""
return self._obj_index
obj_index: int
property
readonly
¶
Index of the objective on which this operator is to be applied
__init__(self, problem, obj_index=None)
special
¶
Initialize the SingleObjOperator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object which is being worked on. |
required |
obj_index |
Optional[int] |
Index of the objective to focus on. Can be given as None if the problem is single-objective. |
None |
Source code in evotorch/operators/base.py
def __init__(self, problem: Problem, obj_index: Optional[int] = None):
"""
Initialize the SingleObjOperator.
Args:
problem: The problem object which is being worked on.
obj_index: Index of the objective to focus on.
Can be given as None if the problem is single-objective.
"""
super().__init__(problem)
self._obj_index: int = problem.normalize_obj_index(obj_index)
real
¶
This module contains operators defined to work with problems
whose dtypes are real numbers (e.g. torch.float32).
CosynePermutation (CopyingOperator)
¶
Representation of permutation operation on a SolutionBatch.
For each decision variable index, a permutation operation across all or a subset of solutions, is performed. The result is returned on a new SolutionBatch. The original SolutionBatch remains unmodified.
Reference:
F.Gomez, J.Schmidhuber, R.Miikkulainen (2008).
Accelerated Neural Evolution through Cooperatively Coevolved Synapses
Journal of Machine Learning Research 9, 937-965
Source code in evotorch/operators/real.py
class CosynePermutation(CopyingOperator):
"""
Representation of permutation operation on a SolutionBatch.
For each decision variable index, a permutation operation across
all or a subset of solutions, is performed.
The result is returned on a new SolutionBatch.
The original SolutionBatch remains unmodified.
Reference:
F.Gomez, J.Schmidhuber, R.Miikkulainen (2008).
Accelerated Neural Evolution through Cooperatively Coevolved Synapses
Journal of Machine Learning Research 9, 937-965
"""
def __init__(self, problem: Problem, obj_index: Optional[int] = None, *, permute_all: bool = False):
"""
`__init__(...)`: Initialize the CosynePermutation.
Args:
problem: The problem object to work on.
obj_index: The index of the objective according to which the
candidates for permutation will be selected.
Can be left as None if the problem is single-objective,
or if `permute_all` is given as True (in which case there
will be no candidate selection as the entire population will
be subject to permutation).
permute_all: Whether or not to apply permutation on the entire
population, instead of using a selective permutation.
"""
if permute_all:
if obj_index is not None:
raise ValueError(
"When `permute_all` is given as True (which seems to be the case)"
" `obj_index` is expected as None,"
" because the operator is independent of any objective and any fitness in this mode."
" However, `permute_all` was found to be something other than None."
)
self._obj_index = None
else:
self._obj_index = problem.normalize_obj_index(obj_index)
super().__init__(problem)
self._permute_all = bool(permute_all)
@property
def obj_index(self) -> Optional[int]:
"""Objective index according to which the operator will run.
If `permute_all` was given as True, objectives are irrelevant, in which case
`obj_index` is returned as None.
If `permute_all` was given as False, the relevant `obj_index` is provided
as an integer.
"""
return self._obj_index
@torch.no_grad()
def _do(self, batch: SolutionBatch) -> SolutionBatch:
indata = batch._data
if not self._permute_all:
n = batch.solution_length
ranks = batch.utility(self._obj_index, ranking_method="centered")
# fitnesses = batch.evals[:, self._obj_index].clone().reshape(-1)
# ranks = rank(
# fitnesses, ranking_method="centered", higher_is_better=(self.problem.senses[self.obj_index] == "max")
# )
prob_permute = (1 - (ranks + 0.5).pow(1 / float(n))).unsqueeze(1).expand(len(batch), batch.solution_length)
else:
prob_permute = torch.ones_like(indata)
perm_mask = self.problem.make_uniform_shaped_like(prob_permute) <= prob_permute
perm_mask_sorted = torch.sort(perm_mask.to(torch.long), descending=True, dim=0)[0].to(
torch.bool
) # Sort permutations
perm_rand = self.problem.make_uniform_shaped_like(prob_permute)
perm_rand[torch.logical_not(perm_mask)] = 1.0
permutations = torch.argsort(perm_rand, dim=0) # Generate permutations
perm_sort = (
torch.arange(0, perm_mask.shape[0], device=indata.device).unsqueeze(-1).repeat(1, perm_mask.shape[1])
)
perm_sort[torch.logical_not(perm_mask)] += perm_mask.shape[0] + 1
perm_sort = torch.sort(perm_sort, dim=0)[0] # Generate the origin of permutations
_, permutation_columns = torch.nonzero(perm_mask_sorted, as_tuple=True)
permutation_origin_indices = perm_sort[perm_mask_sorted]
permutation_target_indices = permutations[perm_mask_sorted]
newbatch = SolutionBatch(like=batch, empty=True)
newdata = newbatch._data
newdata[:] = indata[:]
newdata[permutation_origin_indices, permutation_columns] = newdata[
permutation_target_indices, permutation_columns
]
return newbatch
obj_index: Optional[int]
property
readonly
¶
Objective index according to which the operator will run.
If permute_all was given as True, objectives are irrelevant, in which case
obj_index is returned as None.
If permute_all was given as False, the relevant obj_index is provided
as an integer.
__init__(self, problem, obj_index=None, *, permute_all=False)
special
¶
__init__(...): Initialize the CosynePermutation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object to work on. |
required |
obj_index |
Optional[int] |
The index of the objective according to which the
candidates for permutation will be selected.
Can be left as None if the problem is single-objective,
or if |
None |
permute_all |
bool |
Whether or not to apply permutation on the entire population, instead of using a selective permutation. |
False |
Source code in evotorch/operators/real.py
def __init__(self, problem: Problem, obj_index: Optional[int] = None, *, permute_all: bool = False):
"""
`__init__(...)`: Initialize the CosynePermutation.
Args:
problem: The problem object to work on.
obj_index: The index of the objective according to which the
candidates for permutation will be selected.
Can be left as None if the problem is single-objective,
or if `permute_all` is given as True (in which case there
will be no candidate selection as the entire population will
be subject to permutation).
permute_all: Whether or not to apply permutation on the entire
population, instead of using a selective permutation.
"""
if permute_all:
if obj_index is not None:
raise ValueError(
"When `permute_all` is given as True (which seems to be the case)"
" `obj_index` is expected as None,"
" because the operator is independent of any objective and any fitness in this mode."
" However, `permute_all` was found to be something other than None."
)
self._obj_index = None
else:
self._obj_index = problem.normalize_obj_index(obj_index)
super().__init__(problem)
self._permute_all = bool(permute_all)
GaussianMutation (CopyingOperator)
¶
Gaussian mutation operator.
Follows the algorithm description in:
Sean Luke, 2013, Essentials of Metaheuristics, Lulu, second edition
available for free at http://cs.gmu.edu/~sean/book/metaheuristics/
Source code in evotorch/operators/real.py
class GaussianMutation(CopyingOperator):
"""
Gaussian mutation operator.
Follows the algorithm description in:
Sean Luke, 2013, Essentials of Metaheuristics, Lulu, second edition
available for free at http://cs.gmu.edu/~sean/book/metaheuristics/
"""
def __init__(self, problem: Problem, *, stdev: float, mutation_probability: Optional[float] = None):
"""
`__init__(...)`: Initialize the GaussianMutation.
Args:
problem: The problem object to work with.
stdev: The standard deviation of the Gaussian noise to apply on
each decision variable.
mutation_probability: The probability of mutation, for each
decision variable.
If None, the value of this argument becomes 1.0, which means
that all of the decision variables will be affected by the
mutation. Defatuls to None
"""
super().__init__(problem)
self._mutation_probability = 1.0 if mutation_probability is None else float(mutation_probability)
self._stdev = float(stdev)
@torch.no_grad()
def _do(self, batch: SolutionBatch) -> SolutionBatch:
result = deepcopy(batch)
data = result.access_values()
mutation_matrix = self.problem.make_uniform_shaped_like(data) <= self._mutation_probability
data[mutation_matrix] += self._stdev * self.problem.make_gaussian_shaped_like(data[mutation_matrix])
data[:] = self._respect_bounds(data)
return result
__init__(self, problem, *, stdev, mutation_probability=None)
special
¶
__init__(...): Initialize the GaussianMutation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object to work with. |
required |
stdev |
float |
The standard deviation of the Gaussian noise to apply on each decision variable. |
required |
mutation_probability |
Optional[float] |
The probability of mutation, for each decision variable. If None, the value of this argument becomes 1.0, which means that all of the decision variables will be affected by the mutation. Defatuls to None |
None |
Source code in evotorch/operators/real.py
def __init__(self, problem: Problem, *, stdev: float, mutation_probability: Optional[float] = None):
"""
`__init__(...)`: Initialize the GaussianMutation.
Args:
problem: The problem object to work with.
stdev: The standard deviation of the Gaussian noise to apply on
each decision variable.
mutation_probability: The probability of mutation, for each
decision variable.
If None, the value of this argument becomes 1.0, which means
that all of the decision variables will be affected by the
mutation. Defatuls to None
"""
super().__init__(problem)
self._mutation_probability = 1.0 if mutation_probability is None else float(mutation_probability)
self._stdev = float(stdev)
MultiPointCrossOver (CrossOver)
¶
Representation of a multi-point cross-over operator.
When this operator is applied on a SolutionBatch, a tournament selection technique is used for selecting parent solutions from the batch, and then those parent solutions are mated via cutting from a random position and recombining. The result of these recombination operations is a new SolutionBatch, containing the children solutions. The original SolutionBatch stays unmodified.
This operator is a generalization over the standard cross-over operators OnePointCrossOver and TwoPointCrossOver. In more details, instead of having one or two cutting points, this operator is configurable in terms of how many cutting points is desired. This generalized cross-over implementation follows the procedure described in:
Sean Luke, 2013, Essentials of Metaheuristics, Lulu, second edition
available for free at http://cs.gmu.edu/~sean/book/metaheuristics/
Source code in evotorch/operators/real.py
class MultiPointCrossOver(CrossOver):
"""
Representation of a multi-point cross-over operator.
When this operator is applied on a SolutionBatch, a tournament selection
technique is used for selecting parent solutions from the batch, and then
those parent solutions are mated via cutting from a random position and
recombining. The result of these recombination operations is a new
SolutionBatch, containing the children solutions. The original
SolutionBatch stays unmodified.
This operator is a generalization over the standard cross-over operators
[OnePointCrossOver][evotorch.operators.real.OnePointCrossOver]
and [TwoPointCrossOver][evotorch.operators.real.TwoPointCrossOver].
In more details, instead of having one or two cutting points, this operator
is configurable in terms of how many cutting points is desired.
This generalized cross-over implementation follows the procedure described
in:
Sean Luke, 2013, Essentials of Metaheuristics, Lulu, second edition
available for free at http://cs.gmu.edu/~sean/book/metaheuristics/
"""
def __init__(
self,
problem: Problem,
*,
tournament_size: int,
obj_index: Optional[int] = None,
num_points: Optional[int] = None,
num_children: Optional[int] = None,
cross_over_rate: Optional[float] = None,
):
"""
`__init__(...)`: Initialize the MultiPointCrossOver.
Args:
problem: The problem object to work on.
tournament_size: What is the size (or length) of a tournament
when selecting a parent candidate from a population
obj_index: Objective index according to which the selection
will be done.
num_points: Number of cutting points for the cross-over operator.
num_children: Optionally a number of children to produce by the
cross-over operation.
Not to be used together with `cross_over_rate`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
cross_over_rate: Optionally expected as a real number between
0.0 and 1.0. Specifies the number of cross-over operations
to perform. 1.0 means `1.0 * len(solution_batch)` amount of
cross overs will be performed, resulting in
`2.0 * len(solution_batch)` amount of children.
Not to be used together with `num_children`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
"""
super().__init__(
problem,
tournament_size=tournament_size,
obj_index=obj_index,
num_children=num_children,
cross_over_rate=cross_over_rate,
)
self._num_points = int(num_points)
if self._num_points < 1:
raise ValueError(
f"Invalid `num_points`: {self._num_points}."
f" Please provide a `num_points` which is greater than or equal to 1"
)
@torch.no_grad()
def _do_cross_over(self, parents1: torch.Tensor, parents2: torch.Tensor) -> SolutionBatch:
# What we expect here is this:
#
# parents1 parents2
# ========== ==========
# parents1[0] parents2[0]
# parents1[1] parents2[1]
# ... ...
# parents1[N] parents2[N]
#
# where parents1 and parents2 are 2D tensors, each containing values of N solutions.
# For each row i, we will apply cross-over on parents1[i] and parents2[i].
# From each cross-over, we will obtain 2 children.
# This means, there are N pairings, and 2N children.
num_pairings = parents1.shape[0]
# num_children = num_pairings * 2
device = parents1[0].device
solution_length = len(parents1[0])
num_points = self._num_points
# For each pairing, generate all gene indices (i.e. [0, 1, 2, ...] for each pairing)
gene_indices = (
torch.arange(0, solution_length, device=device).unsqueeze(0).expand(num_pairings, solution_length)
)
if num_points == 1:
# For each pairing, generate a gene index at which the parent solutions will be cut and recombined
crossover_point = self.problem.make_randint((num_pairings, 1), n=(solution_length - 1), device=device) + 1
# Make a mask for crossing over
# (False: take the value from one parent, True: take the value from the other parent).
# For gene indices less than crossover_point of that pairing, the mask takes the value 0.
# Otherwise, the mask takes the value 1.
crossover_mask = gene_indices >= crossover_point
else:
# For each pairing, generate gene indices at which the parent solutions will be cut and recombined
crossover_points = self.problem.make_randint(
(num_pairings, num_points), n=(solution_length + 1), device=device
)
# From `crossover_points`, extract each cutting point for each solution.
cutting_points = [crossover_points[:, i].reshape(-1, 1) for i in range(num_points)]
# Initialize `crossover_mask` as a tensor filled with False.
crossover_mask = torch.zeros((num_pairings, solution_length), dtype=torch.bool, device=device)
# For each cutting point p, toggle the boolean values of `crossover_mask`
# for indices bigger than the index pointed to by p
for p in cutting_points:
crossover_mask ^= gene_indices >= p
# Using the mask, generate two children.
children1 = torch.where(crossover_mask, parents1, parents2)
children2 = torch.where(crossover_mask, parents2, parents1)
# Combine the children tensors in one big tensor
children = torch.cat([children1, children2], dim=0)
# Write the children solutions into a new SolutionBatch, and return the new batch
result = self._make_children_batch(children)
return result
__init__(self, problem, *, tournament_size, obj_index=None, num_points=None, num_children=None, cross_over_rate=None)
special
¶
__init__(...): Initialize the MultiPointCrossOver.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object to work on. |
required |
tournament_size |
int |
What is the size (or length) of a tournament when selecting a parent candidate from a population |
required |
obj_index |
Optional[int] |
Objective index according to which the selection will be done. |
None |
num_points |
Optional[int] |
Number of cutting points for the cross-over operator. |
None |
num_children |
Optional[int] |
Optionally a number of children to produce by the
cross-over operation.
Not to be used together with |
None |
cross_over_rate |
Optional[float] |
Optionally expected as a real number between
0.0 and 1.0. Specifies the number of cross-over operations
to perform. 1.0 means |
None |
Source code in evotorch/operators/real.py
def __init__(
self,
problem: Problem,
*,
tournament_size: int,
obj_index: Optional[int] = None,
num_points: Optional[int] = None,
num_children: Optional[int] = None,
cross_over_rate: Optional[float] = None,
):
"""
`__init__(...)`: Initialize the MultiPointCrossOver.
Args:
problem: The problem object to work on.
tournament_size: What is the size (or length) of a tournament
when selecting a parent candidate from a population
obj_index: Objective index according to which the selection
will be done.
num_points: Number of cutting points for the cross-over operator.
num_children: Optionally a number of children to produce by the
cross-over operation.
Not to be used together with `cross_over_rate`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
cross_over_rate: Optionally expected as a real number between
0.0 and 1.0. Specifies the number of cross-over operations
to perform. 1.0 means `1.0 * len(solution_batch)` amount of
cross overs will be performed, resulting in
`2.0 * len(solution_batch)` amount of children.
Not to be used together with `num_children`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
"""
super().__init__(
problem,
tournament_size=tournament_size,
obj_index=obj_index,
num_children=num_children,
cross_over_rate=cross_over_rate,
)
self._num_points = int(num_points)
if self._num_points < 1:
raise ValueError(
f"Invalid `num_points`: {self._num_points}."
f" Please provide a `num_points` which is greater than or equal to 1"
)
OnePointCrossOver (MultiPointCrossOver)
¶
Representation of a one-point cross-over operator.
When this operator is applied on a SolutionBatch, a tournament selection technique is used for selecting parent solutions from the batch, and then those parent solutions are mated via cutting from a random position and recombining. The result of these recombination operations is a new SolutionBatch, containing the children solutions. The original SolutionBatch stays unmodified.
Let us assume that the two of the parent solutions that were selected for the cross-over operation are as follows:
For recombining parents a and b, a cutting point is first randomly
selected. In the case of this example, let us assume that the cutting
point was chosen as the point between the items with indices 2 and 3:
Considering this selected cutting point, the two children c and d
will be constructed from a and b like this:
Note that the recombination procedure explained above is be done on all of the parents chosen from the given SolutionBatch, in a vectorized manner. For each chosen pair of parents, the cutting points will be sampled differently.
Source code in evotorch/operators/real.py
class OnePointCrossOver(MultiPointCrossOver):
"""
Representation of a one-point cross-over operator.
When this operator is applied on a SolutionBatch, a tournament selection
technique is used for selecting parent solutions from the batch, and then
those parent solutions are mated via cutting from a random position and
recombining. The result of these recombination operations is a new
SolutionBatch, containing the children solutions. The original
SolutionBatch stays unmodified.
Let us assume that the two of the parent solutions that were selected for
the cross-over operation are as follows:
```
a: [ a0 , a1 , a2 , a3 , a4 , a5 ]
b: [ b0 , b1 , b2 , b3 , b4 , b5 ]
```
For recombining parents `a` and `b`, a cutting point is first randomly
selected. In the case of this example, let us assume that the cutting
point was chosen as the point between the items with indices 2 and 3:
```
a: [ a0 , a1 , a2 | a3 , a4 , a5 ]
b: [ b0 , b1 , b2 | b3 , b4 , b5 ]
|
^
Selected cutting point
```
Considering this selected cutting point, the two children `c` and `d`
will be constructed from `a` and `b` like this:
```
c: [ a0 , a1 , a2 | b3 , b4 , b5 ]
d: [ b0 , b1 , b2 | a3 , a4 , a5 ]
```
Note that the recombination procedure explained above is be done on all
of the parents chosen from the given SolutionBatch, in a vectorized manner.
For each chosen pair of parents, the cutting points will be sampled
differently.
"""
def __init__(
self,
problem: Problem,
*,
tournament_size: int,
obj_index: Optional[int] = None,
num_children: Optional[int] = None,
cross_over_rate: Optional[float] = None,
):
"""
`__init__(...)`: Initialize the OnePointCrossOver.
Args:
problem: The problem object to work on.
tournament_size: What is the size (or length) of a tournament
when selecting a parent candidate from a population
obj_index: Objective index according to which the selection
will be done.
num_children: Optionally a number of children to produce by the
cross-over operation.
Not to be used together with `cross_over_rate`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
cross_over_rate: Optionally expected as a real number between
0.0 and 1.0. Specifies the number of cross-over operations
to perform. 1.0 means `1.0 * len(solution_batch)` amount of
cross overs will be performed, resulting in
`2.0 * len(solution_batch)` amount of children.
Not to be used together with `num_children`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
"""
super().__init__(
problem,
tournament_size=tournament_size,
obj_index=obj_index,
num_points=1,
num_children=num_children,
cross_over_rate=cross_over_rate,
)
__init__(self, problem, *, tournament_size, obj_index=None, num_children=None, cross_over_rate=None)
special
¶
__init__(...): Initialize the OnePointCrossOver.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object to work on. |
required |
tournament_size |
int |
What is the size (or length) of a tournament when selecting a parent candidate from a population |
required |
obj_index |
Optional[int] |
Objective index according to which the selection will be done. |
None |
num_children |
Optional[int] |
Optionally a number of children to produce by the
cross-over operation.
Not to be used together with |
None |
cross_over_rate |
Optional[float] |
Optionally expected as a real number between
0.0 and 1.0. Specifies the number of cross-over operations
to perform. 1.0 means |
None |
Source code in evotorch/operators/real.py
def __init__(
self,
problem: Problem,
*,
tournament_size: int,
obj_index: Optional[int] = None,
num_children: Optional[int] = None,
cross_over_rate: Optional[float] = None,
):
"""
`__init__(...)`: Initialize the OnePointCrossOver.
Args:
problem: The problem object to work on.
tournament_size: What is the size (or length) of a tournament
when selecting a parent candidate from a population
obj_index: Objective index according to which the selection
will be done.
num_children: Optionally a number of children to produce by the
cross-over operation.
Not to be used together with `cross_over_rate`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
cross_over_rate: Optionally expected as a real number between
0.0 and 1.0. Specifies the number of cross-over operations
to perform. 1.0 means `1.0 * len(solution_batch)` amount of
cross overs will be performed, resulting in
`2.0 * len(solution_batch)` amount of children.
Not to be used together with `num_children`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
"""
super().__init__(
problem,
tournament_size=tournament_size,
obj_index=obj_index,
num_points=1,
num_children=num_children,
cross_over_rate=cross_over_rate,
)
PolynomialMutation (CopyingOperator)
¶
Representation of the polynomial mutation operator.
Follows the algorithm description in:
Kalyanmoy Deb, Santosh Tiwari (2008).
Omni-optimizer: A generic evolutionary algorithm for single
and multi-objective optimization
The operator ensures a non-zero probability of generating offspring in the entire search space by dividing the space into two regions and using independent probability distributions associated with each region. In contrast, the original polynomial mutation formulation may render the mutation ineffective when the decision variable approaches its boundary.
Source code in evotorch/operators/real.py
class PolynomialMutation(CopyingOperator):
"""
Representation of the polynomial mutation operator.
Follows the algorithm description in:
Kalyanmoy Deb, Santosh Tiwari (2008).
Omni-optimizer: A generic evolutionary algorithm for single
and multi-objective optimization
The operator ensures a non-zero probability of generating offspring in
the entire search space by dividing the space into two regions and using
independent probability distributions associated with each region.
In contrast, the original polynomial mutation formulation may render the
mutation ineffective when the decision variable approaches its boundary.
"""
def __init__(
self,
problem: Problem,
*,
eta: Optional[float] = None,
mutation_probability: Optional[float] = None,
):
"""
`__init__(...)`: Initialize the PolynomialMutation.
Args:
problem: The problem object to work with.
eta: The index for polynomial mutation; a large value gives a higher
probability for creating near-parent solutions, whereas a small
value allows distant solutions to be created.
If not specified, `eta` will be assumed as 20.0.
mutation_probability: The probability of mutation, for each decision
variable. If not specified, all variables will be mutated.
"""
super().__init__(problem)
if "float" not in str(problem.dtype):
raise ValueError(
f"This operator can be used only when `dtype` of the problem is float type"
f" (like, e.g. torch.float32, torch.float64, etc.)"
f" The dtype of the problem is {problem.dtype}."
)
if (self.problem.lower_bounds is None) or (self.problem.upper_bounds is None):
raise ValueError(
"The polynomial mutation operator can be used only when the problem object has"
" `lower_bounds` and `upper_bounds`."
" In the given problem object, at least one of them appears to be missing."
)
if torch.any(self.problem.lower_bounds > self.problem.upper_bounds):
raise ValueError("Some of the `lower_bounds` appear greater than their `upper_bounds`")
self._prob = None if mutation_probability is None else float(mutation_probability)
self._eta = 20.0 if eta is None else float(eta)
self._lb = self.problem.lower_bounds
self._ub = self.problem.upper_bounds
@torch.no_grad()
def _do(self, batch: SolutionBatch) -> SolutionBatch:
# Take a copy of the original batch. Modifications will be done on this copy.
result = deepcopy(batch)
# Take the decision values tensor from within the newly made copy of the batch (`result`).
# Any modification done on this tensor will affect the `result` batch.
data = result.access_values()
# Take the population size
pop_size, solution_length = data.size()
if self._prob is None:
# If a probability of mutation is not given, then we prepare our mutation mask (`to_mutate`) as a tensor
# consisting only of `True`s.
to_mutate = torch.ones(data.shape, dtype=torch.bool, device=data.device)
else:
# If a probability of mutation is given, then we produce a boolean mask that probabilistically marks which
# decision variables will be affected by this mutation operation.
to_mutate = self.problem.make_uniform_shaped_like(data) < self._prob
# Obtain a flattened (1-dimensional) tensor which addresses only the variables that are subject to mutation
# (i.e. variables that are not subject to mutation are filtered out).
selected = data[to_mutate]
# Obtain flattened (1-dimensional) lower and upper bound tensors such that `lb[i]` and `ub[i]` specify the
# bounds for `selected[i]`.
lb = self._lb.expand(pop_size, solution_length)[to_mutate]
ub = self._ub.expand(pop_size, solution_length)[to_mutate]
# Apply the mutation procedure explained by Deb & Tiwari (2008).
delta_1 = (selected - lb) / (ub - lb)
delta_2 = (ub - selected) / (ub - lb)
r = self.problem.make_uniform(selected.size())
mask = r < 0.5
mask_not = torch.logical_not(mask)
mut_str = 1.0 / (self._eta + 1.0)
delta_q = torch.zeros_like(selected)
v = 2.0 * r + (1.0 - 2.0 * r) * (1.0 - delta_1).pow(self._eta + 1.0)
d = v.pow(mut_str) - 1.0
delta_q[mask] = d[mask]
v = 2.0 * (1.0 - r) + 2.0 * (r - 0.5) * (1.0 - delta_2).pow(self._eta + 1.0)
d = 1.0 - v.pow(mut_str)
delta_q[mask_not] = d[mask_not]
mutated = selected + delta_q * (ub - lb)
# Put the mutated decision values into the decision variables tensor stored within the `result` batch.
data[to_mutate] = mutated
# Prevent violations that could happen because of numerical errors.
data[:] = self._respect_bounds(data)
# Return the `result` batch.
return result
__init__(self, problem, *, eta=None, mutation_probability=None)
special
¶
__init__(...): Initialize the PolynomialMutation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object to work with. |
required |
eta |
Optional[float] |
The index for polynomial mutation; a large value gives a higher
probability for creating near-parent solutions, whereas a small
value allows distant solutions to be created.
If not specified, |
None |
mutation_probability |
Optional[float] |
The probability of mutation, for each decision variable. If not specified, all variables will be mutated. |
None |
Source code in evotorch/operators/real.py
def __init__(
self,
problem: Problem,
*,
eta: Optional[float] = None,
mutation_probability: Optional[float] = None,
):
"""
`__init__(...)`: Initialize the PolynomialMutation.
Args:
problem: The problem object to work with.
eta: The index for polynomial mutation; a large value gives a higher
probability for creating near-parent solutions, whereas a small
value allows distant solutions to be created.
If not specified, `eta` will be assumed as 20.0.
mutation_probability: The probability of mutation, for each decision
variable. If not specified, all variables will be mutated.
"""
super().__init__(problem)
if "float" not in str(problem.dtype):
raise ValueError(
f"This operator can be used only when `dtype` of the problem is float type"
f" (like, e.g. torch.float32, torch.float64, etc.)"
f" The dtype of the problem is {problem.dtype}."
)
if (self.problem.lower_bounds is None) or (self.problem.upper_bounds is None):
raise ValueError(
"The polynomial mutation operator can be used only when the problem object has"
" `lower_bounds` and `upper_bounds`."
" In the given problem object, at least one of them appears to be missing."
)
if torch.any(self.problem.lower_bounds > self.problem.upper_bounds):
raise ValueError("Some of the `lower_bounds` appear greater than their `upper_bounds`")
self._prob = None if mutation_probability is None else float(mutation_probability)
self._eta = 20.0 if eta is None else float(eta)
self._lb = self.problem.lower_bounds
self._ub = self.problem.upper_bounds
SimulatedBinaryCrossOver (CrossOver)
¶
Representation of a simulated binary cross-over (SBX).
When this operator is applied on a SolutionBatch, a tournament selection technique is used for selecting parent solutions from the batch, and then those parent solutions are mated via SBX. The generated children solutions are given in a new SolutionBatch. The original SolutionBatch stays unmodified.
Reference:
Kalyanmoy Deb, Hans-Georg Beyer (2001).
Self-Adaptive Genetic Algorithms with Simulated Binary Crossover.
Source code in evotorch/operators/real.py
class SimulatedBinaryCrossOver(CrossOver):
"""
Representation of a simulated binary cross-over (SBX).
When this operator is applied on a SolutionBatch,
a tournament selection technique is used for selecting
parent solutions from the batch, and then those parent
solutions are mated via SBX. The generated children
solutions are given in a new SolutionBatch.
The original SolutionBatch stays unmodified.
Reference:
Kalyanmoy Deb, Hans-Georg Beyer (2001).
Self-Adaptive Genetic Algorithms with Simulated Binary Crossover.
"""
def __init__(
self,
problem: Problem,
*,
tournament_size: int,
eta: float,
obj_index: Optional[int] = None,
num_children: Optional[int] = None,
cross_over_rate: Optional[float] = None,
):
"""
`__init__(...)`: Initialize the SimulatedBinaryCrossOver.
Args:
problem: Problem object to work with.
tournament_size: What is the size (or length) of a tournament
when selecting a parent candidate from a population.
eta: The crowding index, expected as a float.
Bigger eta values result in children closer
to their parents.
obj_index: Objective index according to which the selection
will be done.
num_children: Optionally a number of children to produce by the
cross-over operation.
Not to be used together with `cross_over_rate`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
cross_over_rate: Optionally expected as a real number between
0.0 and 1.0. Specifies the number of cross-over operations
to perform. 1.0 means `1.0 * len(solution_batch)` amount of
cross overs will be performed, resulting in
`2.0 * len(solution_batch)` amount of children.
Not to be used together with `num_children`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
"""
super().__init__(
problem,
tournament_size=int(tournament_size),
obj_index=obj_index,
num_children=num_children,
cross_over_rate=cross_over_rate,
)
self._eta = float(eta)
def _do_cross_over(self, parents1: torch.Tensor, parents2: torch.Tensor) -> SolutionBatch:
# Generate u_i values which determine the spread
u = self.problem.make_uniform_shaped_like(parents1)
# Compute beta_i values from u_i values as the actual spread per dimension
betas = (2 * u).pow(1.0 / (self._eta + 1.0)) # Compute all values for u_i < 0.5 first
betas[u > 0.5] = (1.0 / (2 * (1.0 - u[u > 0.5]))).pow(
1.0 / (self._eta + 1.0)
) # Replace the values for u_i >= 0.5
children1 = 0.5 * (
(1 + betas) * parents1 + (1 - betas) * parents2
) # Create the first set of children from the beta values
children2 = 0.5 * (
(1 + betas) * parents2 + (1 - betas) * parents1
) # Create the second set of children as a mirror of the first set of children
# Combine the children tensors in one big tensor
children = torch.cat([children1, children2], dim=0)
# Respect the lower and upper bounds defined by the problem object
children = self._respect_bounds(children)
# Write the children solutions into a new SolutionBatch, and return the new batch
result = self._make_children_batch(children)
return result
__init__(self, problem, *, tournament_size, eta, obj_index=None, num_children=None, cross_over_rate=None)
special
¶
__init__(...): Initialize the SimulatedBinaryCrossOver.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
Problem object to work with. |
required |
tournament_size |
int |
What is the size (or length) of a tournament when selecting a parent candidate from a population. |
required |
eta |
float |
The crowding index, expected as a float. Bigger eta values result in children closer to their parents. |
required |
obj_index |
Optional[int] |
Objective index according to which the selection will be done. |
None |
num_children |
Optional[int] |
Optionally a number of children to produce by the
cross-over operation.
Not to be used together with |
None |
cross_over_rate |
Optional[float] |
Optionally expected as a real number between
0.0 and 1.0. Specifies the number of cross-over operations
to perform. 1.0 means |
None |
Source code in evotorch/operators/real.py
def __init__(
self,
problem: Problem,
*,
tournament_size: int,
eta: float,
obj_index: Optional[int] = None,
num_children: Optional[int] = None,
cross_over_rate: Optional[float] = None,
):
"""
`__init__(...)`: Initialize the SimulatedBinaryCrossOver.
Args:
problem: Problem object to work with.
tournament_size: What is the size (or length) of a tournament
when selecting a parent candidate from a population.
eta: The crowding index, expected as a float.
Bigger eta values result in children closer
to their parents.
obj_index: Objective index according to which the selection
will be done.
num_children: Optionally a number of children to produce by the
cross-over operation.
Not to be used together with `cross_over_rate`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
cross_over_rate: Optionally expected as a real number between
0.0 and 1.0. Specifies the number of cross-over operations
to perform. 1.0 means `1.0 * len(solution_batch)` amount of
cross overs will be performed, resulting in
`2.0 * len(solution_batch)` amount of children.
Not to be used together with `num_children`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
"""
super().__init__(
problem,
tournament_size=int(tournament_size),
obj_index=obj_index,
num_children=num_children,
cross_over_rate=cross_over_rate,
)
self._eta = float(eta)
TwoPointCrossOver (MultiPointCrossOver)
¶
Representation of a two-point cross-over operator.
When this operator is applied on a SolutionBatch, a tournament selection technique is used for selecting parent solutions from the batch, and then those parent solutions are mated via cutting from a random position and recombining. The result of these recombination operations is a new SolutionBatch, containing the children solutions. The original SolutionBatch stays unmodified.
Let us assume that the two of the parent solutions that were selected for the cross-over operation are as follows:
For recombining parents a and b, two cutting points are first randomly
selected. In the case of this example, let us assume that the cutting
point were chosen as the point between the items with indices 1 and 2,
and between 3 and 4:
a: [ a0 , a1 | a2 , a3 | a4 , a5 ]
b: [ b0 , b1 | b2 , b3 | b4 , b5 ]
| |
^ ^
First Second
cutting cutting
point point
Given these two cutting points, the two children c and d will be
constructed from a and b like this:
Note that the recombination procedure explained above is be done on all of the parents chosen from the given SolutionBatch, in a vectorized manner. For each chosen pair of parents, the cutting points will be sampled differently.
Source code in evotorch/operators/real.py
class TwoPointCrossOver(MultiPointCrossOver):
"""
Representation of a two-point cross-over operator.
When this operator is applied on a SolutionBatch, a tournament selection
technique is used for selecting parent solutions from the batch, and then
those parent solutions are mated via cutting from a random position and
recombining. The result of these recombination operations is a new
SolutionBatch, containing the children solutions. The original
SolutionBatch stays unmodified.
Let us assume that the two of the parent solutions that were selected for
the cross-over operation are as follows:
```
a: [ a0 , a1 , a2 , a3 , a4 , a5 ]
b: [ b0 , b1 , b2 , b3 , b4 , b5 ]
```
For recombining parents `a` and `b`, two cutting points are first randomly
selected. In the case of this example, let us assume that the cutting
point were chosen as the point between the items with indices 1 and 2,
and between 3 and 4:
```
a: [ a0 , a1 | a2 , a3 | a4 , a5 ]
b: [ b0 , b1 | b2 , b3 | b4 , b5 ]
| |
^ ^
First Second
cutting cutting
point point
```
Given these two cutting points, the two children `c` and `d` will be
constructed from `a` and `b` like this:
```
c: [ a0 , a1 | b2 , b3 | a4 , a5 ]
d: [ b0 , b1 | a2 , a3 | b4 , b5 ]
```
Note that the recombination procedure explained above is be done on all
of the parents chosen from the given SolutionBatch, in a vectorized manner.
For each chosen pair of parents, the cutting points will be sampled
differently.
"""
def __init__(
self,
problem: Problem,
*,
tournament_size: int,
obj_index: Optional[int] = None,
num_children: Optional[int] = None,
cross_over_rate: Optional[float] = None,
):
"""
`__init__(...)`: Initialize the TwoPointCrossOver.
Args:
problem: The problem object to work on.
tournament_size: What is the size (or length) of a tournament
when selecting a parent candidate from a population
obj_index: Objective index according to which the selection
will be done.
num_children: Optionally a number of children to produce by the
cross-over operation.
Not to be used together with `cross_over_rate`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
cross_over_rate: Optionally expected as a real number between
0.0 and 1.0. Specifies the number of cross-over operations
to perform. 1.0 means `1.0 * len(solution_batch)` amount of
cross overs will be performed, resulting in
`2.0 * len(solution_batch)` amount of children.
Not to be used together with `num_children`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
"""
super().__init__(
problem,
tournament_size=tournament_size,
obj_index=obj_index,
num_points=2,
num_children=num_children,
cross_over_rate=cross_over_rate,
)
__init__(self, problem, *, tournament_size, obj_index=None, num_children=None, cross_over_rate=None)
special
¶
__init__(...): Initialize the TwoPointCrossOver.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
problem |
Problem |
The problem object to work on. |
required |
tournament_size |
int |
What is the size (or length) of a tournament when selecting a parent candidate from a population |
required |
obj_index |
Optional[int] |
Objective index according to which the selection will be done. |
None |
num_children |
Optional[int] |
Optionally a number of children to produce by the
cross-over operation.
Not to be used together with |
None |
cross_over_rate |
Optional[float] |
Optionally expected as a real number between
0.0 and 1.0. Specifies the number of cross-over operations
to perform. 1.0 means |
None |
Source code in evotorch/operators/real.py
def __init__(
self,
problem: Problem,
*,
tournament_size: int,
obj_index: Optional[int] = None,
num_children: Optional[int] = None,
cross_over_rate: Optional[float] = None,
):
"""
`__init__(...)`: Initialize the TwoPointCrossOver.
Args:
problem: The problem object to work on.
tournament_size: What is the size (or length) of a tournament
when selecting a parent candidate from a population
obj_index: Objective index according to which the selection
will be done.
num_children: Optionally a number of children to produce by the
cross-over operation.
Not to be used together with `cross_over_rate`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
cross_over_rate: Optionally expected as a real number between
0.0 and 1.0. Specifies the number of cross-over operations
to perform. 1.0 means `1.0 * len(solution_batch)` amount of
cross overs will be performed, resulting in
`2.0 * len(solution_batch)` amount of children.
Not to be used together with `num_children`.
If `num_children` and `cross_over_rate` are both None,
then the number of children is equal to the number
of solutions received.
"""
super().__init__(
problem,
tournament_size=tournament_size,
obj_index=obj_index,
num_points=2,
num_children=num_children,
cross_over_rate=cross_over_rate,
)
sequence
¶
This module contains operators for problems whose solutions contain variable-length sequences (list-like objects).
CutAndSplice (CrossOver)
¶
Cut & Splice operator for variable-length solutions.
This class serves as a cross-over operator to be used on problems
with their dtypes set as object, and with their solutions
initialized to contain variable-length sequences (list-like objects).
Reference:
David E. Goldberg, Bradley Korb, Kalyanmoy Deb (1989).
Messy Genetic Algorithms: Motivation, Analysis, and First Results.
Complex Systems 3, 493-530.
Source code in evotorch/operators/sequence.py
class CutAndSplice(CrossOver):
"""Cut & Splice operator for variable-length solutions.
This class serves as a cross-over operator to be used on problems
with their `dtype`s set as `object`, and with their solutions
initialized to contain variable-length sequences (list-like objects).
Reference:
David E. Goldberg, Bradley Korb, Kalyanmoy Deb (1989).
Messy Genetic Algorithms: Motivation, Analysis, and First Results.
Complex Systems 3, 493-530.
"""
def _cut_and_splice(
self,
parents1: ObjectArray,
parents2: ObjectArray,
children1: SolutionBatch,
children2: SolutionBatch,
row_index: int,
):
parvals1 = parents1[row_index]
parvals2 = parents2[row_index]
length1 = len(parvals1)
length2 = len(parvals2)
cutpoint1 = int(self.problem.make_randint(tuple(), n=length1))
cutpoint2 = int(self.problem.make_randint(tuple(), n=length2))
childvals1 = parvals1[:cutpoint1]
childvals1.extend(parvals2[cutpoint2:])
childvals2 = parvals2[:cutpoint2]
childvals2.extend(parvals1[cutpoint1:])
children1.access_values(keep_evals=True)[row_index] = childvals1
children2.access_values(keep_evals=True)[row_index] = childvals2
def _do_cross_over(self, parents1: ObjectArray, parents2: ObjectArray) -> SolutionBatch:
n = len(parents1)
children1 = SolutionBatch(self.problem, popsize=n, empty=True)
children2 = SolutionBatch(self.problem, popsize=n, empty=True)
for i in range(n):
self._cut_and_splice(parents1, parents2, children1, children2, i)
return children1.concat(children2)
optimizers
¶
Optimizers (like Adam or ClipUp) to be used with distribution-based search algorithms.
Adam (TorchOptimizer)
¶
The Adam optimizer.
Reference:
Kingma, D. P. and J. Ba (2015).
Adam: A method for stochastic optimization.
In Proceedings of 3rd International Conference on Learning Representations.
Source code in evotorch/optimizers.py
class Adam(TorchOptimizer):
"""
The Adam optimizer.
Reference:
Kingma, D. P. and J. Ba (2015).
Adam: A method for stochastic optimization.
In Proceedings of 3rd International Conference on Learning Representations.
"""
def __init__(
self,
*,
solution_length: int,
dtype: DType,
device: Device = "cpu",
stepsize: Optional[float] = None,
beta1: Optional[float] = None,
beta2: Optional[float] = None,
epsilon: Optional[float] = None,
amsgrad: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the Adam optimizer.
Args:
solution_length: Length of a solution of the problem which is
being worked on.
dtype: The dtype of the problem which is being worked on.
device: The device on which the solutions are kept.
stepsize: The step size (i.e. the learning rate) employed
by the optimizer.
beta1: The beta1 hyperparameter. None means the default.
beta2: The beta2 hyperparameter. None means the default.
epsilon: The epsilon hyperparameters. None means the default.
amsgrad: Whether or not to use the amsgrad behavior.
None means the default behavior.
See `torch.optim.Adam` for details.
"""
config = {}
if stepsize is not None:
config["lr"] = float(stepsize)
if beta1 is None and beta2 is None:
pass # nothing to do
elif beta1 is not None and beta2 is not None:
config["betas"] = (float(beta1), float(beta2))
else:
raise ValueError(
"The arguments beta1 and beta2 were expected"
" as both None, or as both real numbers."
" However, one of them was encountered as None and"
" the other was encountered as something other than None."
)
if epsilon is not None:
config["eps"] = float(epsilon)
if amsgrad is not None:
config["amsgrad"] = bool(amsgrad)
super().__init__(torch.optim.Adam, solution_length=solution_length, dtype=dtype, device=device, config=config)
__init__(self, *, solution_length, dtype, device='cpu', stepsize=None, beta1=None, beta2=None, epsilon=None, amsgrad=None)
special
¶
__init__(...): Initialize the Adam optimizer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
solution_length |
int |
Length of a solution of the problem which is being worked on. |
required |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
The dtype of the problem which is being worked on. |
required |
device |
Union[str, torch.device] |
The device on which the solutions are kept. |
'cpu' |
stepsize |
Optional[float] |
The step size (i.e. the learning rate) employed by the optimizer. |
None |
beta1 |
Optional[float] |
The beta1 hyperparameter. None means the default. |
None |
beta2 |
Optional[float] |
The beta2 hyperparameter. None means the default. |
None |
epsilon |
Optional[float] |
The epsilon hyperparameters. None means the default. |
None |
amsgrad |
Optional[bool] |
Whether or not to use the amsgrad behavior.
None means the default behavior.
See |
None |
Source code in evotorch/optimizers.py
def __init__(
self,
*,
solution_length: int,
dtype: DType,
device: Device = "cpu",
stepsize: Optional[float] = None,
beta1: Optional[float] = None,
beta2: Optional[float] = None,
epsilon: Optional[float] = None,
amsgrad: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the Adam optimizer.
Args:
solution_length: Length of a solution of the problem which is
being worked on.
dtype: The dtype of the problem which is being worked on.
device: The device on which the solutions are kept.
stepsize: The step size (i.e. the learning rate) employed
by the optimizer.
beta1: The beta1 hyperparameter. None means the default.
beta2: The beta2 hyperparameter. None means the default.
epsilon: The epsilon hyperparameters. None means the default.
amsgrad: Whether or not to use the amsgrad behavior.
None means the default behavior.
See `torch.optim.Adam` for details.
"""
config = {}
if stepsize is not None:
config["lr"] = float(stepsize)
if beta1 is None and beta2 is None:
pass # nothing to do
elif beta1 is not None and beta2 is not None:
config["betas"] = (float(beta1), float(beta2))
else:
raise ValueError(
"The arguments beta1 and beta2 were expected"
" as both None, or as both real numbers."
" However, one of them was encountered as None and"
" the other was encountered as something other than None."
)
if epsilon is not None:
config["eps"] = float(epsilon)
if amsgrad is not None:
config["amsgrad"] = bool(amsgrad)
super().__init__(torch.optim.Adam, solution_length=solution_length, dtype=dtype, device=device, config=config)
ClipUp
¶
The ClipUp optimizer.
Reference:
Toklu, N. E., Liskowski, P., & Srivastava, R. K. (2020, September).
ClipUp: A Simple and Powerful Optimizer for Distribution-Based Policy Evolution.
In International Conference on Parallel Problem Solving from Nature (pp. 515-527).
Springer, Cham.
Source code in evotorch/optimizers.py
class ClipUp:
"""
The ClipUp optimizer.
Reference:
Toklu, N. E., Liskowski, P., & Srivastava, R. K. (2020, September).
ClipUp: A Simple and Powerful Optimizer for Distribution-Based Policy Evolution.
In International Conference on Parallel Problem Solving from Nature (pp. 515-527).
Springer, Cham.
"""
_param_group_items = {"lr": "_stepsize", "max_speed": "_max_speed", "momentum": "_momentum"}
_param_group_item_lb = {"lr": 0.0, "max_speed": 0.0, "momentum": 0.0}
_param_group_item_ub = {"momentum": 1.0}
def __init__(
self,
*,
solution_length: int,
dtype: DType,
stepsize: float,
momentum: float = 0.9,
max_speed: Optional[float] = None,
device: Device = "cpu",
):
"""
`__init__(...)`: Initialize the ClipUp optimizer.
Args:
solution_length: Length of a solution of the problem which is
being worked on.
dtype: The dtype of the problem which is being worked on.
stepsize: The step size (i.e. the learning rate) employed
by the optimizer.
momentum: The momentum coefficient. None means the default.
max_speed: The maximum speed. If given as None, the
`max_speed` will be taken as two times the stepsize.
device: The device on which the solutions are kept.
"""
stepsize = float(stepsize)
momentum = float(momentum)
if max_speed is None:
max_speed = stepsize * 2.0
_evolog.info(
message_from(
self,
(
f"The maximum speed for the ClipUp optimizer is set as {max_speed}"
f" which is two times the given step size."
),
)
)
else:
max_speed = float(max_speed)
solution_length = int(solution_length)
if stepsize < 0.0:
raise ValueError(f"Invalid stepsize: {stepsize}")
if momentum < 0.0 or momentum > 1.0:
raise ValueError(f"Invalid momentum: {momentum}")
if max_speed < 0.0:
raise ValueError(f"Invalid max_speed: {max_speed}")
self._stepsize = stepsize
self._momentum = momentum
self._max_speed = max_speed
self._param_groups = (ClipUpParameterGroup(self),)
self._velocity: Optional[torch.Tensor] = torch.zeros(
solution_length, dtype=to_torch_dtype(dtype), device=device
)
self._dtype = to_torch_dtype(dtype)
self._device = device
@staticmethod
def _clip(x: torch.Tensor, limit: float) -> torch.Tensor:
with torch.no_grad():
normx = torch.norm(x)
if normx > limit:
ratio = limit / normx
return x * ratio
else:
return x
@torch.no_grad()
def ascent(self, globalg: RealOrVector, *, cloned_result: bool = True) -> torch.Tensor:
"""
Compute the ascent, i.e. the step to follow.
Args:
globalg: The estimated gradient.
cloned_result: If `cloned_result` is True, then the result is a
copy, guaranteed not to be the view of any other tensor
internal to the TorchOptimizer class.
If `cloned_result` is False, then the result is not a copy.
Use `cloned_result=False` only when you are sure that your
algorithm will never do direct modification on the ascent
vector it receives.
Important: if you set `cloned_result=False`, and do in-place
modifications on the returned result of `ascent(...)`, then
the internal velocity of ClipUp will be corrupted!
Returns:
The ascent vector, representing the step to follow.
"""
globalg = ensure_tensor_length_and_dtype(
globalg,
len(self._velocity),
dtype=self._dtype,
device=self._device,
about=f"{type(self).__name__}.ascent",
)
grad = (globalg / torch.norm(globalg)) * self._stepsize
self._velocity = self._clip((self._momentum * self._velocity) + grad, self._max_speed)
result = self._velocity
if cloned_result:
result = result.clone()
return result
@property
def contained_optimizer(self) -> "ClipUp":
"""
Get this `ClipUp` instance itself.
"""
return self
@property
def param_groups(self) -> tuple:
"""
Returns a single-element tuple representing a parameter group.
The tuple contains a dictionary-like object in which the keys are the
hyperparameter names, and the values are the values of those
hyperparameters. The hyperparameter names are `lr` (the step size, or
the learning rate), `max_speed` (the maximum speed), and `momentum`
(the momentum coefficient). The values of these hyperparameters can be
read and also be written if one wishes to adjust the behavior of ClipUp
during the optimization.
"""
return self._param_groups
contained_optimizer: ClipUp
property
readonly
¶
Get this ClipUp instance itself.
param_groups: tuple
property
readonly
¶
Returns a single-element tuple representing a parameter group.
The tuple contains a dictionary-like object in which the keys are the
hyperparameter names, and the values are the values of those
hyperparameters. The hyperparameter names are lr (the step size, or
the learning rate), max_speed (the maximum speed), and momentum
(the momentum coefficient). The values of these hyperparameters can be
read and also be written if one wishes to adjust the behavior of ClipUp
during the optimization.
__init__(self, *, solution_length, dtype, stepsize, momentum=0.9, max_speed=None, device='cpu')
special
¶
__init__(...): Initialize the ClipUp optimizer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
solution_length |
int |
Length of a solution of the problem which is being worked on. |
required |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
The dtype of the problem which is being worked on. |
required |
stepsize |
float |
The step size (i.e. the learning rate) employed by the optimizer. |
required |
momentum |
float |
The momentum coefficient. None means the default. |
0.9 |
max_speed |
Optional[float] |
The maximum speed. If given as None, the
|
None |
device |
Union[str, torch.device] |
The device on which the solutions are kept. |
'cpu' |
Source code in evotorch/optimizers.py
def __init__(
self,
*,
solution_length: int,
dtype: DType,
stepsize: float,
momentum: float = 0.9,
max_speed: Optional[float] = None,
device: Device = "cpu",
):
"""
`__init__(...)`: Initialize the ClipUp optimizer.
Args:
solution_length: Length of a solution of the problem which is
being worked on.
dtype: The dtype of the problem which is being worked on.
stepsize: The step size (i.e. the learning rate) employed
by the optimizer.
momentum: The momentum coefficient. None means the default.
max_speed: The maximum speed. If given as None, the
`max_speed` will be taken as two times the stepsize.
device: The device on which the solutions are kept.
"""
stepsize = float(stepsize)
momentum = float(momentum)
if max_speed is None:
max_speed = stepsize * 2.0
_evolog.info(
message_from(
self,
(
f"The maximum speed for the ClipUp optimizer is set as {max_speed}"
f" which is two times the given step size."
),
)
)
else:
max_speed = float(max_speed)
solution_length = int(solution_length)
if stepsize < 0.0:
raise ValueError(f"Invalid stepsize: {stepsize}")
if momentum < 0.0 or momentum > 1.0:
raise ValueError(f"Invalid momentum: {momentum}")
if max_speed < 0.0:
raise ValueError(f"Invalid max_speed: {max_speed}")
self._stepsize = stepsize
self._momentum = momentum
self._max_speed = max_speed
self._param_groups = (ClipUpParameterGroup(self),)
self._velocity: Optional[torch.Tensor] = torch.zeros(
solution_length, dtype=to_torch_dtype(dtype), device=device
)
self._dtype = to_torch_dtype(dtype)
self._device = device
ascent(self, globalg, *, cloned_result=True)
¶
Compute the ascent, i.e. the step to follow.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
globalg |
Union[float, Iterable[float], torch.Tensor] |
The estimated gradient. |
required |
cloned_result |
bool |
If |
True |
Returns:
| Type | Description |
|---|---|
Tensor |
The ascent vector, representing the step to follow. |
Source code in evotorch/optimizers.py
@torch.no_grad()
def ascent(self, globalg: RealOrVector, *, cloned_result: bool = True) -> torch.Tensor:
"""
Compute the ascent, i.e. the step to follow.
Args:
globalg: The estimated gradient.
cloned_result: If `cloned_result` is True, then the result is a
copy, guaranteed not to be the view of any other tensor
internal to the TorchOptimizer class.
If `cloned_result` is False, then the result is not a copy.
Use `cloned_result=False` only when you are sure that your
algorithm will never do direct modification on the ascent
vector it receives.
Important: if you set `cloned_result=False`, and do in-place
modifications on the returned result of `ascent(...)`, then
the internal velocity of ClipUp will be corrupted!
Returns:
The ascent vector, representing the step to follow.
"""
globalg = ensure_tensor_length_and_dtype(
globalg,
len(self._velocity),
dtype=self._dtype,
device=self._device,
about=f"{type(self).__name__}.ascent",
)
grad = (globalg / torch.norm(globalg)) * self._stepsize
self._velocity = self._clip((self._momentum * self._velocity) + grad, self._max_speed)
result = self._velocity
if cloned_result:
result = result.clone()
return result
ClipUpParameterGroup (Mapping)
¶
A dictionary-like object storing the hyperparameters of a ClipUp instance.
The values of the hyperparameters within this container can be read and can also be written if one wishes to adjust the behavior of ClipUp during the optimization.
Source code in evotorch/optimizers.py
class ClipUpParameterGroup(Mapping):
"""
A dictionary-like object storing the hyperparameters of a ClipUp instance.
The values of the hyperparameters within this container can be read and
can also be written if one wishes to adjust the behavior of ClipUp during
the optimization.
"""
def __init__(self, clipup: "ClipUp"):
self.clipup = clipup
def __getitem__(self, key: str) -> float:
attrname = ClipUp._param_group_items[key]
return getattr(self.clipup, attrname)
def __setitem__(self, key: str, value: float):
attrname = ClipUp._param_group_items[key]
value = float(value)
if attrname in ClipUp._param_group_item_lb:
lb = ClipUp._param_group_item_lb[key]
if value < lb:
raise ValueError(f"Invalid value for {repr(key)}: {value}")
if attrname in ClipUp._param_group_item_ub:
ub = ClipUp._param_group_item_ub[key]
if value > ub:
raise ValueError(f"Invalid value for {repr(key)}: {value}")
setattr(self.clipup, attrname, value)
def __iter__(self):
return ClipUp._param_group_items.__iter__()
def __len__(self) -> int:
return len(ClipUp._param_group_items)
def __repr__(self) -> str:
return f"<{type(self).__name__}: {dict(self)}>"
SGD (TorchOptimizer)
¶
The SGD optimizer.
Reference regarding the momentum behavior:
Polyak, B. T. (1964).
Some methods of speeding up the convergence of iteration methods.
USSR Computational Mathematics and Mathematical Physics, 4(5):1–17.
Reference regarding the Nesterov behavior:
Yurii Nesterov (1983).
A method for unconstrained convex minimization problem with the rate ofconvergence o(1/k2).
Doklady ANSSSR (translated as Soviet.Math.Docl.), 269:543–547.
Source code in evotorch/optimizers.py
class SGD(TorchOptimizer):
"""
The SGD optimizer.
Reference regarding the momentum behavior:
Polyak, B. T. (1964).
Some methods of speeding up the convergence of iteration methods.
USSR Computational Mathematics and Mathematical Physics, 4(5):1–17.
Reference regarding the Nesterov behavior:
Yurii Nesterov (1983).
A method for unconstrained convex minimization problem with the rate ofconvergence o(1/k2).
Doklady ANSSSR (translated as Soviet.Math.Docl.), 269:543–547.
"""
def __init__(
self,
*,
solution_length: int,
dtype: DType,
stepsize: float,
device: Device = "cpu",
momentum: Optional[float] = None,
dampening: Optional[bool] = None,
nesterov: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the SGD optimizer.
Args:
solution_length: Length of a solution of the problem which is
being worked on.
dtype: The dtype of the problem which is being worked on.
stepsize: The step size (i.e. the learning rate) employed
by the optimizer.
device: The device on which the solutions are kept.
momentum: The momentum coefficient. None means the default.
dampening: Whether or not to activate the dampening behavior.
None means the default.
See `torch.optim.SGD` for details.
nesterov: Whether or not to activate the nesterov behavior.
None means the default.
See `torch.optim.SGD` for details.
"""
config = {}
config["lr"] = float(stepsize)
if momentum is not None:
config["momentum"] = float(momentum)
if dampening is not None:
config["dampening"] = float(dampening)
if nesterov is not None:
config["nesterov"] = bool(nesterov)
super().__init__(torch.optim.SGD, solution_length=solution_length, dtype=dtype, device=device, config=config)
__init__(self, *, solution_length, dtype, stepsize, device='cpu', momentum=None, dampening=None, nesterov=None)
special
¶
__init__(...): Initialize the SGD optimizer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
solution_length |
int |
Length of a solution of the problem which is being worked on. |
required |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
The dtype of the problem which is being worked on. |
required |
stepsize |
float |
The step size (i.e. the learning rate) employed by the optimizer. |
required |
device |
Union[str, torch.device] |
The device on which the solutions are kept. |
'cpu' |
momentum |
Optional[float] |
The momentum coefficient. None means the default. |
None |
dampening |
Optional[bool] |
Whether or not to activate the dampening behavior.
None means the default.
See |
None |
nesterov |
Optional[bool] |
Whether or not to activate the nesterov behavior.
None means the default.
See |
None |
Source code in evotorch/optimizers.py
def __init__(
self,
*,
solution_length: int,
dtype: DType,
stepsize: float,
device: Device = "cpu",
momentum: Optional[float] = None,
dampening: Optional[bool] = None,
nesterov: Optional[bool] = None,
):
"""
`__init__(...)`: Initialize the SGD optimizer.
Args:
solution_length: Length of a solution of the problem which is
being worked on.
dtype: The dtype of the problem which is being worked on.
stepsize: The step size (i.e. the learning rate) employed
by the optimizer.
device: The device on which the solutions are kept.
momentum: The momentum coefficient. None means the default.
dampening: Whether or not to activate the dampening behavior.
None means the default.
See `torch.optim.SGD` for details.
nesterov: Whether or not to activate the nesterov behavior.
None means the default.
See `torch.optim.SGD` for details.
"""
config = {}
config["lr"] = float(stepsize)
if momentum is not None:
config["momentum"] = float(momentum)
if dampening is not None:
config["dampening"] = float(dampening)
if nesterov is not None:
config["nesterov"] = bool(nesterov)
super().__init__(torch.optim.SGD, solution_length=solution_length, dtype=dtype, device=device, config=config)
TorchOptimizer
¶
Base class for using a PyTorch optimizer
Source code in evotorch/optimizers.py
class TorchOptimizer:
"""
Base class for using a PyTorch optimizer
"""
def __init__(
self,
torch_optimizer: Type,
*,
config: dict,
solution_length: int,
dtype: DType,
device: Device = "cpu",
):
"""
`__init__(...)`: Initialize the TorchOptimizer.
Args:
torch_optimizer: The class which represents a PyTorch optimizer.
config: The configuration dictionary to be passed to the optimizer
as keyword arguments.
solution_length: Length of a solution of the problem on which the
optimizer will work.
dtype: The dtype of the problem.
device: The device on which the solutions are kept.
"""
self._data = torch.empty(int(solution_length), dtype=to_torch_dtype(dtype), device=device)
self._optim = torch_optimizer([self._data], **config)
@torch.no_grad()
def ascent(self, globalg: RealOrVector, *, cloned_result: bool = True) -> torch.Tensor:
"""
Compute the ascent, i.e. the step to follow.
Args:
globalg: The estimated gradient.
cloned_result: If `cloned_result` is True, then the result is a
copy, guaranteed not to be the view of any other tensor
internal to the TorchOptimizer class.
If `cloned_result` is False, then the result is not a copy.
Use `cloned_result=False` only when you are sure that your
algorithm will never do direct modification on the ascent
vector it receives.
Returns:
The ascent vector, representing the step to follow.
"""
globalg = ensure_tensor_length_and_dtype(
globalg,
len(self._data),
dtype=self._data.dtype,
device=self._data.device,
about=f"{type(self).__name__}.ascent",
)
self._data.zero_()
self._data.grad = globalg
self._optim.step()
result = -1.0 * self._data
return result
@property
def contained_optimizer(self) -> torch.optim.Optimizer:
"""
Get the PyTorch optimizer contained by this wrapper
"""
return self._optim
contained_optimizer: Optimizer
property
readonly
¶
Get the PyTorch optimizer contained by this wrapper
__init__(self, torch_optimizer, *, config, solution_length, dtype, device='cpu')
special
¶
__init__(...): Initialize the TorchOptimizer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
torch_optimizer |
Type |
The class which represents a PyTorch optimizer. |
required |
config |
dict |
The configuration dictionary to be passed to the optimizer as keyword arguments. |
required |
solution_length |
int |
Length of a solution of the problem on which the optimizer will work. |
required |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
The dtype of the problem. |
required |
device |
Union[str, torch.device] |
The device on which the solutions are kept. |
'cpu' |
Source code in evotorch/optimizers.py
def __init__(
self,
torch_optimizer: Type,
*,
config: dict,
solution_length: int,
dtype: DType,
device: Device = "cpu",
):
"""
`__init__(...)`: Initialize the TorchOptimizer.
Args:
torch_optimizer: The class which represents a PyTorch optimizer.
config: The configuration dictionary to be passed to the optimizer
as keyword arguments.
solution_length: Length of a solution of the problem on which the
optimizer will work.
dtype: The dtype of the problem.
device: The device on which the solutions are kept.
"""
self._data = torch.empty(int(solution_length), dtype=to_torch_dtype(dtype), device=device)
self._optim = torch_optimizer([self._data], **config)
ascent(self, globalg, *, cloned_result=True)
¶
Compute the ascent, i.e. the step to follow.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
globalg |
Union[float, Iterable[float], torch.Tensor] |
The estimated gradient. |
required |
cloned_result |
bool |
If |
True |
Returns:
| Type | Description |
|---|---|
Tensor |
The ascent vector, representing the step to follow. |
Source code in evotorch/optimizers.py
@torch.no_grad()
def ascent(self, globalg: RealOrVector, *, cloned_result: bool = True) -> torch.Tensor:
"""
Compute the ascent, i.e. the step to follow.
Args:
globalg: The estimated gradient.
cloned_result: If `cloned_result` is True, then the result is a
copy, guaranteed not to be the view of any other tensor
internal to the TorchOptimizer class.
If `cloned_result` is False, then the result is not a copy.
Use `cloned_result=False` only when you are sure that your
algorithm will never do direct modification on the ascent
vector it receives.
Returns:
The ascent vector, representing the step to follow.
"""
globalg = ensure_tensor_length_and_dtype(
globalg,
len(self._data),
dtype=self._data.dtype,
device=self._data.device,
about=f"{type(self).__name__}.ascent",
)
self._data.zero_()
self._data.grad = globalg
self._optim.step()
result = -1.0 * self._data
return result
get_optimizer_class(s, optimizer_config=None)
¶
Get the optimizer class from the given string.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
s |
str |
A string, referring to the optimizer class. "clipsgd", "clipsga", "clipup" refers to ClipUp. "adam" refers to Adam. "sgd" or "sga" refers to SGD. |
required |
optimizer_config |
Optional[dict] |
A dictionary containing the configurations to be passed to the optimizer. If this argument is not None, then, instead of the class being referred to, a dynamically generated factory function will be returned, which will pass these configurations to the actual class upon being called. |
None |
Returns:
| Type | Description |
|---|---|
Callable |
The class, or a factory function instantiating that class. |
Source code in evotorch/optimizers.py
def get_optimizer_class(s: str, optimizer_config: Optional[dict] = None) -> Callable:
"""
Get the optimizer class from the given string.
Args:
s: A string, referring to the optimizer class.
"clipsgd", "clipsga", "clipup" refers to ClipUp.
"adam" refers to Adam.
"sgd" or "sga" refers to SGD.
optimizer_config: A dictionary containing the configurations to be
passed to the optimizer. If this argument is not None,
then, instead of the class being referred to, a dynamically
generated factory function will be returned, which will pass
these configurations to the actual class upon being called.
Returns:
The class, or a factory function instantiating that class.
"""
if s in ("clipsgd", "clipsga", "clipup"):
cls = ClipUp
elif s == "adam":
cls = Adam
elif s in ("sgd", "sga"):
cls = SGD
else:
raise ValueError(f"Unknown optimizer: {repr(s)}")
if optimizer_config is None:
return cls
else:
def f(*args, **kwargs):
nonlocal cls, optimizer_config
conf = {}
conf.update(optimizer_config)
conf.update(kwargs)
return cls(*args, **conf)
return f
testing
¶
Utility functions for evotorch-related unit testing.
TestingError (Exception)
¶
assert_allclose(actual, desired, *, rtol=None, atol=None, equal_nan=True)
¶
This function is similar to numpy.testing.assert_allclose(...) except
that atol and rtol are keyword-only arguments (which encourages
one to be more explicit when writing tests) and that the default dtype
is "float32" when the provided arguments are neither numpy arrays nor
torch tensors. Having "float32" as the default target dtype is a behavior
that is compatible with PyTorch.
This function first casts actual into the dtype of desired, then
uses numpy's assert_allclose(...) for testing the closeness of the
values.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
actual |
Iterable |
An iterable of numbers. |
required |
desired |
Iterable |
An iterable of numbers. These numbers represent the values
that we expect the |
required |
rtol |
Optional[float] |
Relative tolerance.
Can be left as None if only |
None |
atol |
Optional[float] |
Absolute tolerance.
Can be left as None if only |
None |
equal_nan |
bool |
If True, |
True |
Exceptions:
| Type | Description |
|---|---|
AssertionError |
if the numerical difference between |
TestingError |
If both |
Source code in evotorch/testing.py
def assert_allclose(
actual: Iterable,
desired: Iterable,
*,
rtol: Optional[float] = None,
atol: Optional[float] = None,
equal_nan: bool = True,
):
"""
This function is similar to `numpy.testing.assert_allclose(...)` except
that `atol` and `rtol` are keyword-only arguments (which encourages
one to be more explicit when writing tests) and that the default dtype
is "float32" when the provided arguments are neither numpy arrays nor
torch tensors. Having "float32" as the default target dtype is a behavior
that is compatible with PyTorch.
This function first casts `actual` into the dtype of `desired`, then
uses numpy's `assert_allclose(...)` for testing the closeness of the
values.
Args:
actual: An iterable of numbers.
desired: An iterable of numbers. These numbers represent the values
that we expect the `actual` to contain. If the numbers contained
by `actual` are significantly different than `desired`, the
assertion will fail.
rtol: Relative tolerance.
Can be left as None if only `atol` is to be used.
See the documentation of `numpy.testing.assert_allclose(...)`
for details about how `rtol` affects the tolerance.
atol: Absolute tolerance.
Can be left as None if only `rtol` is to be used.
See the documentation of `numpy.testing.assert_allclose(...)`
for details about how `atol` affects the tolerance.
equal_nan: If True, `nan` values will be counted as equal.
Raises:
AssertionError: if the numerical difference between `actual`
and `desired` are beyond the tolerance expressed by `atol`
and `rtol`.
TestingError: If both `rtol` and `atol` are given as None.
"""
if rtol is None and atol is None:
raise TestingError(
"Both `rtol` and `atol` were found to be None. Please either specify `rtol`, `atol`, or both."
)
elif rtol is None:
rtol = 0.0
elif atol is None:
atol = 0.0
desired = _to_numpy(desired)
actual = _to_numpy(actual, dtype=desired.dtype)
np.testing.assert_allclose(actual, desired, rtol=rtol, atol=atol, equal_nan=bool(equal_nan))
assert_almost_between(x, lb, ub, *, atol=None)
¶
Assert that the given Iterable has its values between the desired bounds.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Iterable |
An Iterable containing numeric (float) values. |
required |
lb |
Union[float, Iterable] |
Lower bound for the desired interval. Can be a scalar or an iterable of values. |
required |
ub |
Union[float, Iterable] |
Upper bound for the desired interval. Can be a scalar or an iterable of values. |
required |
atol |
Optional[float] |
Absolute tolerance. If given, then the effective interval will
be |
None |
Exceptions:
| Type | Description |
|---|---|
AssertionError |
if any element of |
Source code in evotorch/testing.py
def assert_almost_between(
x: Iterable, lb: Union[float, Iterable], ub: Union[float, Iterable], *, atol: Optional[float] = None
):
"""
Assert that the given Iterable has its values between the desired bounds.
Args:
x: An Iterable containing numeric (float) values.
lb: Lower bound for the desired interval.
Can be a scalar or an iterable of values.
ub: Upper bound for the desired interval.
Can be a scalar or an iterable of values.
atol: Absolute tolerance. If given, then the effective interval will
be `[lb-atol; ub+atol]` instead of `[lb; ub]`.
Raises:
AssertionError: if any element of `x` violates the boundaries.
"""
x = _to_numpy(x)
lb = _to_numpy(lb)
ub = _to_numpy(ub)
if lb.shape != x.shape:
lb = np.broadcast_to(lb, x.shape)
if ub.shape != x.shape:
ub = np.broadcast_to(ub, x.shape)
lb = np.asarray(lb, dtype=x.dtype)
ub = np.asarray(ub, dtype=x.dtype)
if atol is not None:
atol = float(atol)
tolerant_lb = lb - atol
tolerant_ub = ub + atol
else:
tolerant_lb = lb
tolerant_ub = ub
assert np.all((x >= tolerant_lb) & (x <= tolerant_ub)), (
f"The provided array is not within the desired boundaries."
f"Provided array: {x}. Lower bound: {lb}. Upper bound: {ub}. Absolute tolerance: {atol}."
)
assert_dtype_matches(x, dtype)
¶
Assert that the dtype of x is compatible with the given dtype.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Iterable |
An object with |
required |
dtype |
Union[str, Type, numpy.dtype, torch.dtype] |
The dtype which |
required |
Exceptions:
| Type | Description |
|---|---|
AssertionError |
if |
Source code in evotorch/testing.py
def assert_dtype_matches(x: Iterable, dtype: Union[str, Type, np.dtype, torch.dtype]):
"""
Assert that the dtype of `x` is compatible with the given `dtype`.
Args:
x: An object with `dtype` attribute (e.g. can be numpy array,
a torch tensor, an ObjectArray, a Solution, etc.)
dtype: The dtype which `x` is expected to have.
Can be given as a string, as a numpy dtype, as a torch dtype,
or as a native type (e.g. int, float, bool, object).
Raises:
AssertionError: if `x` has a different dtype.
"""
actual_dtype = x.dtype
if isinstance(actual_dtype, torch.dtype):
actual_dtype = torch.tensor([], dtype=actual_dtype).numpy().dtype
else:
actual_dtype = np.dtype(actual_dtype)
if dtype == "Any" or dtype is Any:
dtype = np.dtype(object)
elif isinstance(dtype, torch.dtype):
dtype = torch.tensor([], dtype=dtype).numpy().dtype
else:
dtype = np.dtype(dtype)
assert dtype == actual_dtype, f"dtype mismatch. Encountered dtype: {actual_dtype}, expected dtype: {dtype}"
assert_eachclose(x, value, *, rtol=None, atol=None)
¶
Assert that the given tensor or array consists of a single value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Iterable |
The tensor in which each value will be compared against |
required |
value |
Any |
A scalar |
required |
Exceptions:
| Type | Description |
|---|---|
AssertionError |
if at least one value is different enough |
Source code in evotorch/testing.py
def assert_eachclose(x: Iterable, value: Any, *, rtol: Optional[float] = None, atol: Optional[float] = None):
"""
Assert that the given tensor or array consists of a single value.
Args:
x: The tensor in which each value will be compared against `value`
value: A scalar
Raises:
AssertionError: if at least one value is different enough
"""
# If the given scalar is not a Real, then try to cast it to float
if not isinstance(value, Real):
value = float(value)
x = _to_numpy(x)
desired = np.empty_like(x)
desired[:] = value
assert_allclose(x, desired, rtol=rtol, atol=atol)
assert_shape_matches(x, shape)
¶
Assert that the dtype of x matches the given shape
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Iterable |
An object which can be converted to a PyTorch tensor. |
required |
shape |
Union[tuple, int] |
A tuple, or a torch.Size, or an integer. |
required |
Exceptions:
| Type | Description |
|---|---|
AssertionError |
if there is a shape mismatch. |
Source code in evotorch/testing.py
def assert_shape_matches(x: Iterable, shape: Union[tuple, int]):
"""
Assert that the dtype of `x` matches the given shape
Args:
x: An object which can be converted to a PyTorch tensor.
shape: A tuple, or a torch.Size, or an integer.
Raises:
AssertionError: if there is a shape mismatch.
"""
if isinstance(x, torch.Tensor):
pass # nothing to do
elif isinstance(x, np.ndarray):
x = torch.from_numpy(x)
else:
x = torch.tensor(x)
if not isinstance(shape, Iterable):
shape = (int(shape),)
assert x.shape == shape, f"Encountered a shape mismatch. Shape of the tensor: {x.shape}. Expected shape: {shape}"
tools
special
¶
This namespace contains various utility functions, classes, and type aliases.
cloning
¶
Clonable
¶
A base class allowing inheriting classes define how they should be cloned.
Any class inheriting from Clonable gains these behaviors:
(i) A new method named .clone() becomes available;
(ii) __deepcopy__ and __copy__ work as aliases for .clone();
(iii) A new method, _get_cloned_state(self, *, memo: dict) is now
defined and needs to be implemented by the inheriting class.
The method _get_cloned_state(...) expects a dictionary named memo,
which maps from the ids of already cloned objects to their clones.
If _get_cloned_state(...) is to use deep_clone(...) or deepcopy(...)
within itself, this memo dictionary can be passed to these functions.
The return value of _get_cloned_state(...) is a dictionary, which will
be used as the __dict__ of the newly made clone.
Source code in evotorch/tools/cloning.py
class Clonable:
"""
A base class allowing inheriting classes define how they should be cloned.
Any class inheriting from Clonable gains these behaviors:
(i) A new method named `.clone()` becomes available;
(ii) `__deepcopy__` and `__copy__` work as aliases for `.clone()`;
(iii) A new method, `_get_cloned_state(self, *, memo: dict)` is now
defined and needs to be implemented by the inheriting class.
The method `_get_cloned_state(...)` expects a dictionary named `memo`,
which maps from the ids of already cloned objects to their clones.
If `_get_cloned_state(...)` is to use `deep_clone(...)` or `deepcopy(...)`
within itself, this `memo` dictionary can be passed to these functions.
The return value of `_get_cloned_state(...)` is a dictionary, which will
be used as the `__dict__` of the newly made clone.
"""
def _get_cloned_state(self, *, memo: dict) -> dict:
raise NotImplementedError
def clone(self, *, memo: Optional[dict] = None) -> "Clonable":
"""
Get a clone of this object.
Args:
memo: Optionally a dictionary which maps from the ids of the
already cloned objects to their clones. In most scenarios,
when this method is called from outside, this can be left
as None.
Returns:
The clone of the object.
"""
if memo is None:
memo = {}
self_id = id(self)
if self_id in memo:
return memo[self_id]
new_object = object.__new__(type(self))
memo[id(self)] = new_object
new_object.__dict__.update(self._get_cloned_state(memo=memo))
return new_object
def __copy__(self) -> "Clonable":
return self.clone()
def __deepcopy__(self, memo: Optional[dict]):
if memo is None:
memo = {}
return self.clone(memo=memo)
clone(self, *, memo=None)
¶
Get a clone of this object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
memo |
Optional[dict] |
Optionally a dictionary which maps from the ids of the already cloned objects to their clones. In most scenarios, when this method is called from outside, this can be left as None. |
None |
Returns:
| Type | Description |
|---|---|
Clonable |
The clone of the object. |
Source code in evotorch/tools/cloning.py
def clone(self, *, memo: Optional[dict] = None) -> "Clonable":
"""
Get a clone of this object.
Args:
memo: Optionally a dictionary which maps from the ids of the
already cloned objects to their clones. In most scenarios,
when this method is called from outside, this can be left
as None.
Returns:
The clone of the object.
"""
if memo is None:
memo = {}
self_id = id(self)
if self_id in memo:
return memo[self_id]
new_object = object.__new__(type(self))
memo[id(self)] = new_object
new_object.__dict__.update(self._get_cloned_state(memo=memo))
return new_object
ReadOnlyClonable (Clonable)
¶
Clonability base class for read-only and/or immutable objects.
This is a base class specialized for the immutable containers of EvoTorch. These immutable containers have two behaviors for cloning: one where the read-only attribute is preserved and one where a mutable clone is created.
Upon being copied or deep-copied (using the standard Python functions),
the newly made clones are also read-only. However, when copied using the
clone(...) method, the newly made clone is mutable by default
(unless the clone(...) method was used with preserve_read_only=True).
This default behavior of the clone(...) method was inspired by the
copy() method of numpy arrays (the inspiration being that the .copy()
of a read-only numpy array will not be read-only anymore).
Subclasses of evotorch.immutable.ImmutableContainer inherit from
ReadOnlyClonable.
Source code in evotorch/tools/cloning.py
class ReadOnlyClonable(Clonable):
"""
Clonability base class for read-only and/or immutable objects.
This is a base class specialized for the immutable containers of EvoTorch.
These immutable containers have two behaviors for cloning:
one where the read-only attribute is preserved and one where a mutable
clone is created.
Upon being copied or deep-copied (using the standard Python functions),
the newly made clones are also read-only. However, when copied using the
`clone(...)` method, the newly made clone is mutable by default
(unless the `clone(...)` method was used with `preserve_read_only=True`).
This default behavior of the `clone(...)` method was inspired by the
`copy()` method of numpy arrays (the inspiration being that the `.copy()`
of a read-only numpy array will not be read-only anymore).
Subclasses of `evotorch.immutable.ImmutableContainer` inherit from
`ReadOnlyClonable`.
"""
def _get_mutable_clone(self, *, memo: dict) -> Any:
raise NotImplementedError
def clone(self, *, memo: Optional[dict] = None, preserve_read_only: bool = False) -> Any:
"""
Get a clone of this read-only object.
Args:
memo: Optionally a dictionary which maps from the ids of the
already cloned objects to their clones. In most scenarios,
when this method is called from outside, this can be left
as None.
preserve_read_only: Whether or not to preserve the read-only
behavior in the clone.
Returns:
The clone of the object.
"""
if memo is None:
memo = {}
if preserve_read_only:
return super().clone(memo=memo)
else:
return self._get_mutable_clone(memo=memo)
def __copy__(self) -> Any:
return self.clone(preserve_read_only=True)
def __deepcopy__(self, memo: Optional[dict]) -> Any:
if memo is None:
memo = {}
return self.clone(memo=memo, preserve_read_only=True)
clone(self, *, memo=None, preserve_read_only=False)
¶
Get a clone of this read-only object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
memo |
Optional[dict] |
Optionally a dictionary which maps from the ids of the already cloned objects to their clones. In most scenarios, when this method is called from outside, this can be left as None. |
None |
preserve_read_only |
bool |
Whether or not to preserve the read-only behavior in the clone. |
False |
Returns:
| Type | Description |
|---|---|
Any |
The clone of the object. |
Source code in evotorch/tools/cloning.py
def clone(self, *, memo: Optional[dict] = None, preserve_read_only: bool = False) -> Any:
"""
Get a clone of this read-only object.
Args:
memo: Optionally a dictionary which maps from the ids of the
already cloned objects to their clones. In most scenarios,
when this method is called from outside, this can be left
as None.
preserve_read_only: Whether or not to preserve the read-only
behavior in the clone.
Returns:
The clone of the object.
"""
if memo is None:
memo = {}
if preserve_read_only:
return super().clone(memo=memo)
else:
return self._get_mutable_clone(memo=memo)
Serializable (Clonable)
¶
Base class allowing the inheriting classes become Clonable and picklable.
Any class inheriting from Serializable becomes Clonable (since
Serializable is a subclass of Clonable) and therefore is expected to
define its own _get_cloned_state(...) (see the documentation of the
class Clonable for details).
A Serializable class gains a behavior for its __getstate__. In this
already defined and implemented __getstate__ method, the resulting
dictionary of _get_cloned_state(...) is used as the state dictionary.
Therefore, for Serializable objects, the behavior defined in their
_get_cloned_state(...) methods affect how they are pickled.
Classes inheriting from Serializable are evotorch.Problem,
evotorch.Solution, evotorch.SolutionBatch, and
evotorch.distributions.Distribution. In their _get_cloned_state(...)
implementations, these classes use deep_clone(...) on themselves to make
sure that their contained PyTorch tensors are copied using the .clone()
method, ensuring that those tensors are detached from their old storages
during the cloning operation. Thanks to being Serializable, their
contained tensors are detached from their old storages both at the moment
of copying/cloning AND at the moment of pickling.
Source code in evotorch/tools/cloning.py
class Serializable(Clonable):
"""
Base class allowing the inheriting classes become Clonable and picklable.
Any class inheriting from `Serializable` becomes `Clonable` (since
`Serializable` is a subclass of `Clonable`) and therefore is expected to
define its own `_get_cloned_state(...)` (see the documentation of the
class `Clonable` for details).
A `Serializable` class gains a behavior for its `__getstate__`. In this
already defined and implemented `__getstate__` method, the resulting
dictionary of `_get_cloned_state(...)` is used as the state dictionary.
Therefore, for `Serializable` objects, the behavior defined in their
`_get_cloned_state(...)` methods affect how they are pickled.
Classes inheriting from `Serializable` are `evotorch.Problem`,
`evotorch.Solution`, `evotorch.SolutionBatch`, and
`evotorch.distributions.Distribution`. In their `_get_cloned_state(...)`
implementations, these classes use `deep_clone(...)` on themselves to make
sure that their contained PyTorch tensors are copied using the `.clone()`
method, ensuring that those tensors are detached from their old storages
during the cloning operation. Thanks to being `Serializable`, their
contained tensors are detached from their old storages both at the moment
of copying/cloning AND at the moment of pickling.
"""
def __getstate__(self) -> dict:
memo = {id(self): self}
return self._get_cloned_state(memo=memo)
deep_clone(x, *, otherwise_deepcopy=False, otherwise_return=False, otherwise_fail=False, memo=None)
¶
A recursive cloning function similar to the standard deepcopy.
The difference between deep_clone(...) and deepcopy(...) is that
deep_clone(...), while recursively traversing, will run the .clone()
method on the PyTorch tensors it encounters, so that the cloned tensors
are forcefully detached from their storages (instead of cloning those
storages as well).
At the moment of writing this documentation, the current behavior of
PyTorch tensors upon being deep-copied is to clone themselves AND their
storages. Therefore, if a PyTorch tensor is a slice of a large tensor
(which has a large storage), then the large storage will also be
deep-copied, and the newly made clone of the tensor will point to a newly
made large storage. One might instead prefer to clone tensors in such a
way that the newly made tensor points to a newly made storage that
contains just enough data for the tensor (with the unused data being
dropped). When such a behavior is desired, one can use this
deep_clone(...) function.
Upon encountering a read-only and/or immutable data, this function will
NOT modify the read-only behavior. For example, the deep-clone of a
ReadOnlyTensor is still a ReadOnlyTensor, and the deep-clone of a
read-only numpy array is still a read-only numpy array. Note that this
behavior is different than the clone() method of a ReadOnlyTensor
and the copy() method of a numpy array. The reason for this
protective behavior is that since this is a deep-cloning operation,
the encountered tensors and/or arrays might be the components of the root
object, and changing their read-only attributes might affect the integrity
of this root object.
The deep_clone(...) function needs to know what to do when an object
of unrecognized type is encountered. Therefore, the user is expected to
set one of these arguments as True (and leave the others as False):
otherwise_deepcopy, otherwise_return, otherwise_fail.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
The object which will be deep-cloned. This object can be a standard
Python container (i.e. list, tuple, dict, set), an instance of
Problem, Solution, SolutionBatch, ObjectArray, ImmutableContainer,
Clonable, and also any other type of object if either the argument
|
required |
otherwise_deepcopy |
bool |
Setting this as True means that, when an
unrecognized object is encountered, that object will be
deep-copied. To handle shared and cyclic-referencing objects,
the |
False |
otherwise_return |
bool |
Setting this as True means that, when an unrecognized object is encountered, that object itself will be returned (i.e. will be a part of the created clone). |
False |
otherwise_fail |
bool |
Setting this as True means that, when an unrecognized object is encountered, a TypeError will be raised. |
False |
memo |
Optional[dict] |
Optionally a dictionary. In most scenarios, when this function is called from outside, this is expected to be left as None. |
None |
Returns:
| Type | Description |
|---|---|
Any |
The newly made clone of the original object. |
Source code in evotorch/tools/cloning.py
def deep_clone( # noqa: C901
x: Any,
*,
otherwise_deepcopy: bool = False,
otherwise_return: bool = False,
otherwise_fail: bool = False,
memo: Optional[dict] = None,
) -> Any:
"""
A recursive cloning function similar to the standard `deepcopy`.
The difference between `deep_clone(...)` and `deepcopy(...)` is that
`deep_clone(...)`, while recursively traversing, will run the `.clone()`
method on the PyTorch tensors it encounters, so that the cloned tensors
are forcefully detached from their storages (instead of cloning those
storages as well).
At the moment of writing this documentation, the current behavior of
PyTorch tensors upon being deep-copied is to clone themselves AND their
storages. Therefore, if a PyTorch tensor is a slice of a large tensor
(which has a large storage), then the large storage will also be
deep-copied, and the newly made clone of the tensor will point to a newly
made large storage. One might instead prefer to clone tensors in such a
way that the newly made tensor points to a newly made storage that
contains just enough data for the tensor (with the unused data being
dropped). When such a behavior is desired, one can use this
`deep_clone(...)` function.
Upon encountering a read-only and/or immutable data, this function will
NOT modify the read-only behavior. For example, the deep-clone of a
ReadOnlyTensor is still a ReadOnlyTensor, and the deep-clone of a
read-only numpy array is still a read-only numpy array. Note that this
behavior is different than the `clone()` method of a ReadOnlyTensor
and the `copy()` method of a numpy array. The reason for this
protective behavior is that since this is a deep-cloning operation,
the encountered tensors and/or arrays might be the components of the root
object, and changing their read-only attributes might affect the integrity
of this root object.
The `deep_clone(...)` function needs to know what to do when an object
of unrecognized type is encountered. Therefore, the user is expected to
set one of these arguments as True (and leave the others as False):
`otherwise_deepcopy`, `otherwise_return`, `otherwise_fail`.
Args:
x: The object which will be deep-cloned. This object can be a standard
Python container (i.e. list, tuple, dict, set), an instance of
Problem, Solution, SolutionBatch, ObjectArray, ImmutableContainer,
Clonable, and also any other type of object if either the argument
`otherwise_deepcopy` or the argument `otherwise_return` is set as
True.
otherwise_deepcopy: Setting this as True means that, when an
unrecognized object is encountered, that object will be
deep-copied. To handle shared and cyclic-referencing objects,
the `deep_clone(...)` function stores its own memo dictionary.
When the control is given to the standard `deepcopy(...)`
function, the memo dictionary of `deep_clone(...)` will be passed
to `deepcopy`.
otherwise_return: Setting this as True means that, when an
unrecognized object is encountered, that object itself will be
returned (i.e. will be a part of the created clone).
otherwise_fail: Setting this as True means that, when an unrecognized
object is encountered, a TypeError will be raised.
memo: Optionally a dictionary. In most scenarios, when this function
is called from outside, this is expected to be left as None.
Returns:
The newly made clone of the original object.
"""
from .objectarray import ObjectArray
from .readonlytensor import ReadOnlyTensor
if memo is None:
# If a memo dictionary was not given, make a new one now.
memo = {}
# Get the id of the object being cloned.
x_id = id(x)
if x_id in memo:
# If the id of the object being cloned is already in the memo dictionary, then this object was previously
# cloned. We just return that clone.
return memo[x_id]
# Count how many of the arguments `otherwise_deepcopy`, `otherwise_return`, and `otherwise_fail` was set as True.
# In this context, we call these arguments as fallback behaviors.
fallback_behaviors = (otherwise_deepcopy, otherwise_return, otherwise_fail)
enabled_behavior_count = sum(1 for behavior in fallback_behaviors if behavior)
if enabled_behavior_count == 0:
# If none of the fallback behaviors was enabled, then we raise an error.
raise ValueError(
"The action to take with objects of unrecognized types is not known because"
" none of these arguments was set as True: `otherwise_deepcopy`, `otherwise_return`, `otherwise_fail`."
" Please set one of these arguments as True."
)
elif enabled_behavior_count == 1:
# If one of the fallback behaviors was enabled, then we received our expected input. We do nothing here.
pass
else:
# If the number of enabled fallback behaviors is an unexpected value. then we raise an error.
raise ValueError(
f"The following arguments were received, which is conflicting: otherwise_deepcopy={otherwise_deepcopy},"
f" otherwise_return={otherwise_return}, otherwise_fail={otherwise_fail}."
f" Please set exactly one of these arguments as True and leave the others as False."
)
# This inner function specifies how the deep_clone function should call itself.
def call_self(obj: Any) -> Any:
return deep_clone(
obj,
otherwise_deepcopy=otherwise_deepcopy,
otherwise_return=otherwise_return,
otherwise_fail=otherwise_fail,
memo=memo,
)
# Below, we handle the cloning behaviors case by case.
if (x is None) or (x is NotImplemented) or (x is Ellipsis):
result = deepcopy(x)
elif isinstance(x, (Number, str, bytes, bytearray)):
result = deepcopy(x, memo=memo)
elif isinstance(x, np.ndarray):
result = x.copy()
result.flags["WRITEABLE"] = x.flags["WRITEABLE"]
elif isinstance(x, (ObjectArray, ReadOnlyClonable)):
result = x.clone(preserve_read_only=True, memo=memo)
elif isinstance(x, ReadOnlyTensor):
result = x.clone(preserve_read_only=True)
elif isinstance(x, torch.Tensor):
result = x.clone()
elif isinstance(x, Clonable):
result = x.clone(memo=memo)
elif isinstance(x, (dict, OrderedDict)):
result = type(x)()
memo[x_id] = result
for k, v in x.items():
result[call_self(k)] = call_self(v)
elif isinstance(x, list):
result = type(x)()
memo[x_id] = result
for item in x:
result.append(call_self(item))
elif isinstance(x, set):
result = type(x)()
memo[x_id] = result
for item in x:
result.add(call_self(item))
elif isinstance(x, tuple):
result = []
memo[x_id] = result
for item in x:
result.append(call_self(item))
if hasattr(x, "_fields"):
result = type(x)(*result)
else:
result = type(x)(result)
memo[x_id] = result
else:
# If the object is not recognized, we use the fallback behavior.
if otherwise_deepcopy:
result = deepcopy(x, memo=memo)
elif otherwise_return:
result = x
elif otherwise_fail:
raise TypeError(f"Do not know how to clone {repr(x)} (of type {type(x)}).")
else:
raise RuntimeError("The function `deep_clone` reached an unexpected state. This might be a bug.")
if (x_id not in memo) and (result is not x):
# If the newly made clone is still not in the memo dictionary AND the "clone" is not just a reference to the
# original object, we make sure that it is in the memo dictionary.
memo[x_id] = result
# Finally, the result is returned.
return result
constraints
¶
log_barrier(lhs, comparison, rhs, *, penalty_sign, sharpness=1.0, inf=None)
¶
Return a penalty based on how close the constraint is to being violated.
If the left-hand-side is equal to the right-hand-side, or if the constraint
is violated, the returned penalty will be infinite (+inf or -inf,
depending on penalty_sign). Such inf values can result in numerical
instabilities. To overcome such instabilities, you might want to set the
keyword argument inf as a large-enough finite positive quantity M, so
that very large (or infinite) penalties will be clipped down to M.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
lhs |
Union[float, torch.Tensor] |
The left-hand-side of the constraint. In the non-batched case, this is expected as a scalar. If it is given as an n-dimensional tensor where n is at least 1, this is considered as a batch of left-hand-side values. |
required |
comparison |
str |
The operator used for comparing the left-hand-side and the right-hand-side. Expected as a string. Acceptable values are: '<=', '>='. |
required |
rhs |
Union[float, torch.Tensor] |
The right-hand-side of the constraint. In the non-batched case, this is expected as a scalar. If it is given as an n-dimensional tensor where n is at least 1, this is considered as a batch of right-hand-side values. |
required |
penalty_sign |
str |
Expected as string, either as '+' or '-', which
determines the sign of the penalty (i.e. determines if the penalty
will be positive or negative). One should consider the objective
sense of the fitness function at hand for deciding |
required |
sharpness |
Union[float, torch.Tensor] |
The logarithmic penalty will be divided by this number. By default, this value is 1. A sharper log-penalization allows the constraint to get closer to its boundary, and then makes a more sudden jump towards infinity. |
1.0 |
inf |
Union[float, torch.Tensor] |
When concerned about the possible numerical instabilities caused
by infinite penalties, one can specify a finite large-enough
positive quantity |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
Log-penalty amount(s), whose sign(s) is/are determined by
|
Source code in evotorch/tools/constraints.py
def log_barrier(
lhs: Union[float, torch.Tensor],
comparison: str,
rhs: Union[float, torch.Tensor],
*,
penalty_sign: str,
sharpness: Union[float, torch.Tensor] = 1.0,
inf: Optional[Union[float, torch.Tensor]] = None,
) -> torch.Tensor:
"""
Return a penalty based on how close the constraint is to being violated.
If the left-hand-side is equal to the right-hand-side, or if the constraint
is violated, the returned penalty will be infinite (`+inf` or `-inf`,
depending on `penalty_sign`). Such `inf` values can result in numerical
instabilities. To overcome such instabilities, you might want to set the
keyword argument `inf` as a large-enough finite positive quantity `M`, so
that very large (or infinite) penalties will be clipped down to `M`.
Args:
lhs: The left-hand-side of the constraint. In the non-batched case,
this is expected as a scalar. If it is given as an n-dimensional
tensor where n is at least 1, this is considered as a batch of
left-hand-side values.
comparison: The operator used for comparing the left-hand-side and the
right-hand-side. Expected as a string. Acceptable values are:
'<=', '>='.
rhs: The right-hand-side of the constraint. In the non-batched case,
this is expected as a scalar. If it is given as an n-dimensional
tensor where n is at least 1, this is considered as a batch of
right-hand-side values.
penalty_sign: Expected as string, either as '+' or '-', which
determines the sign of the penalty (i.e. determines if the penalty
will be positive or negative). One should consider the objective
sense of the fitness function at hand for deciding `penalty_sign`.
For example, if a fitness function is written from the perspective
of maximization, the penalties should be negative, and therefore,
`penalty_sign` must be given as '-'.
sharpness: The logarithmic penalty will be divided by this number.
By default, this value is 1. A sharper log-penalization allows
the constraint to get closer to its boundary, and then makes
a more sudden jump towards infinity.
inf: When concerned about the possible numerical instabilities caused
by infinite penalties, one can specify a finite large-enough
positive quantity `M` through this argument. As a result,
infinite penalties will be clipped down to the finite `M`.
One might also think of this as temporarily replacing `inf` with
`M` while computing the log-penalties.
Returns:
Log-penalty amount(s), whose sign(s) is/are determined by
`penalty_sign`.
"""
from ..decorators import expects_ndim
if inf is None:
inf = float("inf")
return expects_ndim(_log_barrier, (0, None, 0, 0, None, 0))(lhs, comparison, rhs, sharpness, penalty_sign, inf)
penalty(lhs, comparison, rhs, *, penalty_sign, linear=None, step=None, exp=None, exp_inf=None)
¶
Return a penalty based on the amount of violation of the constraint.
Depending on the provided arguments, the penalty can be linear, or exponential, or based on step function, or a combination of these.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
lhs |
Union[float, torch.Tensor] |
The left-hand-side of the constraint. In the non-batched case, this is expected as a scalar. If it is given as an n-dimensional tensor where n is at least 1, this is considered as a batch of left-hand-side values. |
required |
comparison |
str |
The operator used for comparing the left-hand-side and the right-hand-side. Expected as a string. Acceptable values are: '<=', '==', '>='. |
required |
rhs |
Union[float, torch.Tensor] |
The right-hand-side of the constraint. In the non-batched case, this is expected as a scalar. If it is given as an n-dimensional tensor where n is at least 1, this is considered as a batch of right-hand-side values. |
required |
penalty_sign |
str |
Expected as string, either as '+' or '-', which
determines the sign of the penalty (i.e. determines if the penalty
will be positive or negative). One should consider the objective
sense of the fitness function at hand for deciding |
required |
linear |
Union[float, torch.Tensor] |
Multiplier for the linear component of the penalization. If omitted (i.e. left as None), the value of this multiplier will be 0 (meaning that there will not be any linear penalization). In the non-batched case, this argument is expected as a scalar. If this is provided as a tensor 1 or more dimensions, those dimensions will be considered as batch dimensions. |
None |
step |
Union[float, torch.Tensor] |
The constant amount that will be added onto the penalty if there is a violation. If omitted (i.e. left as None), this value is 0. In the non-batched case, this argument is expected as a scalar. If this is provided as a tensor 1 or more dimensions, those dimensions will be considered as batch dimensions. |
None |
exp |
Union[float, torch.Tensor] |
A constant |
None |
exp_inf |
Union[float, torch.Tensor] |
Upper bound for exponential penalty values. If exponential
penalty is enabled but |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The penalty amount(s), whose sign(s) is/are determined by
|
Source code in evotorch/tools/constraints.py
def penalty(
lhs: Union[float, torch.Tensor],
comparison: str,
rhs: Union[float, torch.Tensor],
*,
penalty_sign: str,
linear: Optional[Union[float, torch.Tensor]] = None,
step: Optional[Union[float, torch.Tensor]] = None,
exp: Optional[Union[float, torch.Tensor]] = None,
exp_inf: Optional[Union[float, torch.Tensor]] = None,
) -> torch.Tensor:
"""
Return a penalty based on the amount of violation of the constraint.
Depending on the provided arguments, the penalty can be linear,
or exponential, or based on step function, or a combination of these.
Args:
lhs: The left-hand-side of the constraint. In the non-batched case,
this is expected as a scalar. If it is given as an n-dimensional
tensor where n is at least 1, this is considered as a batch of
left-hand-side values.
comparison: The operator used for comparing the left-hand-side and the
right-hand-side. Expected as a string. Acceptable values are:
'<=', '==', '>='.
rhs: The right-hand-side of the constraint. In the non-batched case,
this is expected as a scalar. If it is given as an n-dimensional
tensor where n is at least 1, this is considered as a batch of
right-hand-side values.
penalty_sign: Expected as string, either as '+' or '-', which
determines the sign of the penalty (i.e. determines if the penalty
will be positive or negative). One should consider the objective
sense of the fitness function at hand for deciding `penalty_sign`.
For example, if a fitness function is written from the perspective
of maximization, the penalties should be negative, and therefore,
`penalty_sign` must be given as '-'.
linear: Multiplier for the linear component of the penalization.
If omitted (i.e. left as None), the value of this multiplier will
be 0 (meaning that there will not be any linear penalization).
In the non-batched case, this argument is expected as a scalar.
If this is provided as a tensor 1 or more dimensions, those
dimensions will be considered as batch dimensions.
step: The constant amount that will be added onto the penalty if there
is a violation. If omitted (i.e. left as None), this value is 0.
In the non-batched case, this argument is expected as a scalar.
If this is provided as a tensor 1 or more dimensions, those
dimensions will be considered as batch dimensions.
exp: A constant `p` that will enable exponential penalization in the
form `amount_of_violation ** p`. If this is left as None or is
given as `nan`, there will be no exponential penalization.
In the non-batched case, this argument is expected as a scalar.
If this is provided as a tensor 1 or more dimensions, those
dimensions will be considered as batch dimensions.
exp_inf: Upper bound for exponential penalty values. If exponential
penalty is enabled but `exp_inf` is omitted (i.e. left as None),
the exponential penalties can jump to very large values or to
infinity, potentially causing numerical instabilities. To avoid
such numerical instabilities, one might provide a large-enough
positive constant `M` via the argument `exp_inf`. When such a value
is given, exponential penalties will not be allowed to exceed `M`.
One might also think of this as temporarily replacing `inf` with
`M` while computing the exponential penalties.
Returns:
The penalty amount(s), whose sign(s) is/are determined by
`sign_penalty`.
"""
from ..decorators import expects_ndim
if linear is None:
linear = 0.0
if step is None:
step = 0.0
if exp is None:
exp = float("nan")
if exp_inf is None:
exp_inf = float("inf")
return expects_ndim(_penalty, (0, None, 0, None, 0, 0, 0, 0))(
lhs,
comparison,
rhs,
penalty_sign,
linear,
step,
exp,
exp_inf,
)
violation(lhs, comparison, rhs)
¶
Get the amount of constraint violation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
lhs |
Union[float, torch.Tensor] |
The left-hand-side of the constraint. In the non-batched case, this is expected as a scalar. If it is given as an n-dimensional tensor where n is at least 1, this is considered as a batch of left-hand-side values. |
required |
comparison |
str |
The operator used for comparing the left-hand-side and the right-hand-side. Expected as a string. Acceptable values are: '<=', '==', '>='. |
required |
rhs |
Union[float, torch.Tensor] |
The right-hand-side of the constraint. In the non-batched case, this is expected as a scalar. If it is given as an n-dimensional tensor where n is at least 1, this is considered as a batch of right-hand-side values. |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The amount of violation of the constraint. A value of 0 means that the constraint is not violated at all. The returned violation amount(s) are always non-negative. |
Source code in evotorch/tools/constraints.py
def violation(
lhs: Union[float, torch.Tensor],
comparison: str,
rhs: Union[float, torch.Tensor],
) -> torch.Tensor:
"""
Get the amount of constraint violation.
Args:
lhs: The left-hand-side of the constraint. In the non-batched case,
this is expected as a scalar. If it is given as an n-dimensional
tensor where n is at least 1, this is considered as a batch of
left-hand-side values.
comparison: The operator used for comparing the left-hand-side and the
right-hand-side. Expected as a string. Acceptable values are:
'<=', '==', '>='.
rhs: The right-hand-side of the constraint. In the non-batched case,
this is expected as a scalar. If it is given as an n-dimensional
tensor where n is at least 1, this is considered as a batch of
right-hand-side values.
Returns:
The amount of violation of the constraint. A value of 0 means that
the constraint is not violated at all. The returned violation amount(s)
are always non-negative.
"""
from ..decorators import expects_ndim
return expects_ndim(_violation, (0, None, 0))(lhs, comparison, rhs)
hook
¶
This module contains the Hook class, which is used for event handling, and for defining additional behaviors to the class instances which own the Hook.
Hook (MutableSequence)
¶
A Hook stores a list of callable objects to be called for handling certain events. A Hook itself is callable, which invokes the callables stored in its list. If the callables stored by the Hook return list-like objects or dict-like objects, their returned results are accumulated, and then those accumulated results are finally returned by the Hook.
Source code in evotorch/tools/hook.py
class Hook(MutableSequence):
"""
A Hook stores a list of callable objects to be called for handling
certain events. A Hook itself is callable, which invokes the callables
stored in its list. If the callables stored by the Hook return list-like
objects or dict-like objects, their returned results are accumulated,
and then those accumulated results are finally returned by the Hook.
"""
def __init__(
self,
callables: Optional[Iterable[Callable]] = None,
*,
args: Optional[Iterable] = None,
kwargs: Optional[Mapping] = None,
):
"""
Initialize the Hook.
Args:
callables: A sequence of callables to be stored by the Hook.
args: Positional arguments which, when the Hook is called,
are to be passed to every callable stored by the Hook.
Please note that these positional arguments will be passed
as the leftmost arguments, and, the other positional
arguments passed via the `__call__(...)` method of the
Hook will be added to the right of these arguments.
kwargs: Keyword arguments which, when the Hook is called,
are to be passed to every callable stored by the Hook.
Please note that these keyword arguments could be overriden
by the keyword arguments passed via the `__call__(...)`
method of the Hook.
"""
self._funcs: list = [] if callables is None else list(callables)
self._args: list = [] if args is None else list(args)
self._kwargs: dict = {} if kwargs is None else dict(kwargs)
def __call__(self, *args: Any, **kwargs: Any) -> Optional[Union[dict, list]]:
"""
Call every callable object stored by the Hook.
The results of the stored callable objects (which can be dict-like
or list-like objects) are accumulated and finally returned.
Args:
args: Additional positional arguments to be passed to the stored
callables.
kwargs: Additional keyword arguments to be passed to the stored
keyword arguments.
"""
all_args = []
all_args.extend(self._args)
all_args.extend(args)
all_kwargs = {}
all_kwargs.update(self._kwargs)
all_kwargs.update(kwargs)
result: Optional[Union[dict, list]] = None
for f in self._funcs:
tmp = f(*all_args, **all_kwargs)
if tmp is not None:
if isinstance(tmp, Mapping):
if result is None:
result = dict(tmp)
elif isinstance(result, list):
raise TypeError(
f"The function {f} returned a dict-like object."
f" However, previous function(s) in this hook had returned list-like object(s)."
f" Such incompatible results cannot be accumulated."
)
elif isinstance(result, dict):
result.update(tmp)
else:
raise RuntimeError
elif isinstance(tmp, Iterable):
if result is None:
result = list(tmp)
elif isinstance(result, list):
result.extend(tmp)
elif isinstance(result, dict):
raise TypeError(
f"The function {f} returned a list-like object."
f" However, previous function(s) in this hook had returned dict-like object(s)."
f" Such incompatible results cannot be accumulated."
)
else:
raise RuntimeError
else:
raise TypeError(
f"Expected the function {f} to return None, or a dict-like object, or a list-like object."
f" However, the function returned an object of type {repr(type(tmp))}."
)
return result
def accumulate_dict(self, *args: Any, **kwargs: Any) -> Optional[Union[dict, list]]:
result = self(*args, **kwargs)
if result is None:
return {}
elif isinstance(result, Mapping):
return result
else:
raise TypeError(
f"Expected the functions in this hook to accumulate"
f" dictionary-like objects. Instead, accumulated"
f" an object of type {type(result)}."
f" Hint: are the functions registered in this hook"
f" returning non-dictionary iterables?"
)
def accumulate_sequence(self, *args: Any, **kwargs: Any) -> Optional[Union[dict, list]]:
result = self(*args, **kwargs)
if result is None:
return []
elif isinstance(result, Mapping):
raise TypeError(
f"Expected the functions in this hook to accumulate"
f" sequences (that are NOT dictionaries). Instead, accumulated"
f" a dict-like object of type {type(result)}."
f" Hint: are the functions registered in this hook"
f" returning objects with Mapping interface?"
)
else:
return result
def _to_string(self) -> str:
init_args = [repr(self._funcs)]
if len(self._args) > 0:
init_args.append(f"args={self._args}")
if len(self._kwargs) > 0:
init_args.append(f"kwargs={self._kwargs}")
s_init_args = ", ".join(init_args)
return f"{type(self).__name__}({s_init_args})"
def __repr__(self) -> str:
return self._to_string()
def __str__(self) -> str:
return self._to_string()
def __getitem__(self, i: Union[int, slice]) -> Union[Callable, "Hook"]:
if isinstance(i, slice):
return Hook(self._funcs[i], args=self._args, kwargs=self._kwargs)
else:
return self._funcs[i]
def __setitem__(self, i: Union[int, slice], x: Iterable[Callable]):
self._funcs[i] = x
def __delitem__(self, i: Union[int, slice]):
del self._funcs[i]
def insert(self, i: int, x: Callable):
self._funcs.insert(i, x)
def __len__(self) -> int:
return len(self._funcs)
@property
def args(self) -> list:
"""Positional arguments that will be passed to the stored callables"""
return self._args
@property
def kwargs(self) -> dict:
"""Keyword arguments that will be passed to the stored callables"""
return self._kwargs
args: list
property
readonly
¶
Positional arguments that will be passed to the stored callables
kwargs: dict
property
readonly
¶
Keyword arguments that will be passed to the stored callables
__call__(self, *args, **kwargs)
special
¶
Call every callable object stored by the Hook. The results of the stored callable objects (which can be dict-like or list-like objects) are accumulated and finally returned.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
args |
Any |
Additional positional arguments to be passed to the stored callables. |
() |
kwargs |
Any |
Additional keyword arguments to be passed to the stored keyword arguments. |
{} |
Source code in evotorch/tools/hook.py
def __call__(self, *args: Any, **kwargs: Any) -> Optional[Union[dict, list]]:
"""
Call every callable object stored by the Hook.
The results of the stored callable objects (which can be dict-like
or list-like objects) are accumulated and finally returned.
Args:
args: Additional positional arguments to be passed to the stored
callables.
kwargs: Additional keyword arguments to be passed to the stored
keyword arguments.
"""
all_args = []
all_args.extend(self._args)
all_args.extend(args)
all_kwargs = {}
all_kwargs.update(self._kwargs)
all_kwargs.update(kwargs)
result: Optional[Union[dict, list]] = None
for f in self._funcs:
tmp = f(*all_args, **all_kwargs)
if tmp is not None:
if isinstance(tmp, Mapping):
if result is None:
result = dict(tmp)
elif isinstance(result, list):
raise TypeError(
f"The function {f} returned a dict-like object."
f" However, previous function(s) in this hook had returned list-like object(s)."
f" Such incompatible results cannot be accumulated."
)
elif isinstance(result, dict):
result.update(tmp)
else:
raise RuntimeError
elif isinstance(tmp, Iterable):
if result is None:
result = list(tmp)
elif isinstance(result, list):
result.extend(tmp)
elif isinstance(result, dict):
raise TypeError(
f"The function {f} returned a list-like object."
f" However, previous function(s) in this hook had returned dict-like object(s)."
f" Such incompatible results cannot be accumulated."
)
else:
raise RuntimeError
else:
raise TypeError(
f"Expected the function {f} to return None, or a dict-like object, or a list-like object."
f" However, the function returned an object of type {repr(type(tmp))}."
)
return result
__init__(self, callables=None, *, args=None, kwargs=None)
special
¶
Initialize the Hook.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
callables |
Optional[Iterable[Callable]] |
A sequence of callables to be stored by the Hook. |
None |
args |
Optional[Iterable] |
Positional arguments which, when the Hook is called,
are to be passed to every callable stored by the Hook.
Please note that these positional arguments will be passed
as the leftmost arguments, and, the other positional
arguments passed via the |
None |
kwargs |
Optional[collections.abc.Mapping] |
Keyword arguments which, when the Hook is called,
are to be passed to every callable stored by the Hook.
Please note that these keyword arguments could be overriden
by the keyword arguments passed via the |
None |
Source code in evotorch/tools/hook.py
def __init__(
self,
callables: Optional[Iterable[Callable]] = None,
*,
args: Optional[Iterable] = None,
kwargs: Optional[Mapping] = None,
):
"""
Initialize the Hook.
Args:
callables: A sequence of callables to be stored by the Hook.
args: Positional arguments which, when the Hook is called,
are to be passed to every callable stored by the Hook.
Please note that these positional arguments will be passed
as the leftmost arguments, and, the other positional
arguments passed via the `__call__(...)` method of the
Hook will be added to the right of these arguments.
kwargs: Keyword arguments which, when the Hook is called,
are to be passed to every callable stored by the Hook.
Please note that these keyword arguments could be overriden
by the keyword arguments passed via the `__call__(...)`
method of the Hook.
"""
self._funcs: list = [] if callables is None else list(callables)
self._args: list = [] if args is None else list(args)
self._kwargs: dict = {} if kwargs is None else dict(kwargs)
insert(self, i, x)
¶
misc
¶
Miscellaneous utility functions
DTypeAndDevice (tuple)
¶
ErroneousResult
¶
Representation of a caught error being returned as a result.
Source code in evotorch/tools/misc.py
class ErroneousResult:
"""
Representation of a caught error being returned as a result.
"""
def __init__(self, error: Exception):
self.error = error
def _to_string(self) -> str:
return f"<{type(self).__name__}, error: {self.error}>"
def __str__(self) -> str:
return self._to_string()
def __repr__(self) -> str:
return self._to_string()
def __bool__(self) -> bool:
return False
@staticmethod
def call(f, *args, **kwargs) -> Any:
"""
Call a function with the given arguments.
If the function raises an error, wrap the error in an ErroneousResult
object, and return that ErroneousResult object instead.
Returns:
The result of the function if there was no error,
or an ErroneousResult if there was an error.
"""
try:
result = f(*args, **kwargs)
except Exception as ex:
result = ErroneousResult(ex)
return result
call(f, *args, **kwargs)
staticmethod
¶
Call a function with the given arguments. If the function raises an error, wrap the error in an ErroneousResult object, and return that ErroneousResult object instead.
Returns:
| Type | Description |
|---|---|
Any |
The result of the function if there was no error, or an ErroneousResult if there was an error. |
Source code in evotorch/tools/misc.py
@staticmethod
def call(f, *args, **kwargs) -> Any:
"""
Call a function with the given arguments.
If the function raises an error, wrap the error in an ErroneousResult
object, and return that ErroneousResult object instead.
Returns:
The result of the function if there was no error,
or an ErroneousResult if there was an error.
"""
try:
result = f(*args, **kwargs)
except Exception as ex:
result = ErroneousResult(ex)
return result
as_tensor(x, *, dtype=None, device=None)
¶
Get the tensor counterpart of the given object x.
This function can be used to convert native Python objects to tensors:
my_tensor = as_tensor([1.0, 2.0, 3.0], dtype="float32")
One can also use this function to convert an existing tensor to another dtype:
my_new_tensor = as_tensor(my_tensor, dtype="float16")
This function can also be used for moving a tensor from one device to another:
my_gpu_tensor = as_tensor(my_tensor, device="cuda:0")
This function can also create ObjectArray instances when dtype is
given as object or Any or "object" or "O".
my_objects = as_tensor([1, {"a": 3}], dtype=object)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
Any object to be converted to a tensor. |
required |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32) or, for creating an |
None |
device |
Union[str, torch.device] |
The device in which the resulting tensor will be stored. |
None |
Returns:
| Type | Description |
|---|---|
Iterable |
The tensor counterpart of the given object |
Source code in evotorch/tools/misc.py
def as_tensor(x: Any, *, dtype: Optional[DType] = None, device: Optional[Device] = None) -> Iterable:
"""
Get the tensor counterpart of the given object `x`.
This function can be used to convert native Python objects to tensors:
my_tensor = as_tensor([1.0, 2.0, 3.0], dtype="float32")
One can also use this function to convert an existing tensor to another
dtype:
my_new_tensor = as_tensor(my_tensor, dtype="float16")
This function can also be used for moving a tensor from one device to
another:
my_gpu_tensor = as_tensor(my_tensor, device="cuda:0")
This function can also create ObjectArray instances when dtype is
given as `object` or `Any` or "object" or "O".
my_objects = as_tensor([1, {"a": 3}], dtype=object)
Args:
x: Any object to be converted to a tensor.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32) or, for creating an `ObjectArray`,
"object" (as string) or `object` or `Any`.
If `dtype` is not specified, the default behavior of
`torch.as_tensor(...)` will be used, that is, dtype will be
inferred from `x`.
device: The device in which the resulting tensor will be stored.
Returns:
The tensor counterpart of the given object `x`.
"""
from .objectarray import ObjectArray
if (dtype is None) and isinstance(x, (torch.Tensor, ObjectArray)):
if (device is None) or (str(device) == "cpu"):
return x
else:
raise ValueError(
f"An ObjectArray cannot be moved into a device other than 'cpu'. The received device is: {device}."
)
elif is_dtype_object(dtype):
if (device is not None) and (str(device) != "cpu"):
raise ValueError(
f"An ObjectArray cannot be created on a device other than 'cpu'. The received device is: {device}."
)
if isinstance(x, ObjectArray):
return x
else:
x = list(x)
n = len(x)
result = ObjectArray(n)
result[:] = x
return result
else:
dtype = to_torch_dtype(dtype)
return torch.as_tensor(x, dtype=dtype, device=device)
cast_tensors_in_container(container, *, dtype=None, device=None, memo=None)
¶
Cast and/or transfer all the tensors in a Python container.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
If given as a dtype and not as None, then all the PyTorch tensors in the container will be cast to this dtype. |
None |
device |
Union[str, torch.device] |
If given as a device and not as None, then all the PyTorch tensors in the container will be copied to this device. |
None |
memo |
Optional[dict] |
Optionally a memo dictionary to handle shared objects and circular references. In most scenarios, when calling this function from outside, this is expected as None. |
None |
Returns:
| Type | Description |
|---|---|
Any |
A new copy of the original container in which the tensors have the desired dtype and/or device. |
Source code in evotorch/tools/misc.py
def cast_tensors_in_container(
container: Any,
*,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
memo: Optional[dict] = None,
) -> Any:
"""
Cast and/or transfer all the tensors in a Python container.
Args:
dtype: If given as a dtype and not as None, then all the PyTorch
tensors in the container will be cast to this dtype.
device: If given as a device and not as None, then all the PyTorch
tensors in the container will be copied to this device.
memo: Optionally a memo dictionary to handle shared objects and
circular references. In most scenarios, when calling this
function from outside, this is expected as None.
Returns:
A new copy of the original container in which the tensors have the
desired dtype and/or device.
"""
if memo is None:
memo = {}
container_id = id(container)
if container_id in memo:
return memo[container_id]
cast_kwargs = {}
if dtype is not None:
cast_kwargs["dtype"] = to_torch_dtype(dtype)
if device is not None:
cast_kwargs["device"] = device
def call_self(sub_container: Any) -> Any:
return cast_tensors_in_container(sub_container, dtype=dtype, device=device, memo=memo)
if isinstance(container, torch.Tensor):
result = torch.as_tensor(container, **cast_kwargs)
memo[container_id] = result
elif (container is None) or isinstance(container, (Number, str, bytes, bool)):
result = container
elif isinstance(container, set):
result = set()
memo[container_id] = result
for x in container:
result.add(call_self(x))
elif isinstance(container, Mapping):
result = {}
memo[container_id] = result
for k, v in container.items():
result[k] = call_self(v)
elif isinstance(container, tuple):
result = []
memo[container_id] = result
for x in container:
result.append(call_self(x))
if hasattr(container, "_fields"):
result = type(container)(*result)
else:
result = type(container)(result)
memo[container_id] = result
elif isinstance(container, Iterable):
result = []
memo[container_id] = result
for x in container:
result.append(call_self(x))
else:
raise TypeError(f"Encountered an object of unrecognized type: {type(container)}")
return result
clip_tensor(x, lb=None, ub=None, ensure_copy=True)
¶
Clip the values of a tensor with respect to the given bounds.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Tensor |
The PyTorch tensor whose values will be clipped. |
required |
lb |
Union[float, Iterable] |
Lower bounds, as a PyTorch tensor. Can be None if there are no lower bounds. |
None |
ub |
Union[float, Iterable] |
Upper bounds, as a PyTorch tensor. Can be None if there are no upper bonuds. |
None |
ensure_copy |
bool |
If |
True |
Returns:
| Type | Description |
|---|---|
Tensor |
The clipped tensor. |
Source code in evotorch/tools/misc.py
@torch.no_grad()
def clip_tensor(
x: torch.Tensor,
lb: Optional[Union[float, Iterable]] = None,
ub: Optional[Union[float, Iterable]] = None,
ensure_copy: bool = True,
) -> torch.Tensor:
"""
Clip the values of a tensor with respect to the given bounds.
Args:
x: The PyTorch tensor whose values will be clipped.
lb: Lower bounds, as a PyTorch tensor.
Can be None if there are no lower bounds.
ub: Upper bounds, as a PyTorch tensor.
Can be None if there are no upper bonuds.
ensure_copy: If `ensure_copy` is True, the result will be
a clipped copy of the original tensor.
If `ensure_copy` is False, and both `lb` and `ub`
are None, then there is nothing to do, so, the result
will be the original tensor itself, not a copy of it.
Returns:
The clipped tensor.
"""
result = x
if lb is not None:
lb = torch.as_tensor(lb, dtype=x.dtype, device=x.device)
result = torch.max(result, lb)
if ub is not None:
ub = torch.as_tensor(ub, dtype=x.dtype, device=x.device)
result = torch.min(result, ub)
if ensure_copy and result is x:
result = x.clone()
return result
clone(x, *, memo=None)
¶
Get a deep copy of the given object.
The cloning is done in no_grad mode.
When this function is used on read-only containers (e.g. ReadOnlyTensor,
ImmutableContainer, etc.), the created clones preserve their read-only
behaviors. For creating a mutable clone of an immutable object,
use their clone() method instead.
Returns:
| Type | Description |
|---|---|
Any |
The deep copy of the given object. |
Source code in evotorch/tools/misc.py
@torch.no_grad()
def clone(x: Any, *, memo: Optional[dict] = None) -> Any:
"""
Get a deep copy of the given object.
The cloning is done in no_grad mode.
When this function is used on read-only containers (e.g. ReadOnlyTensor,
ImmutableContainer, etc.), the created clones preserve their read-only
behaviors. For creating a mutable clone of an immutable object,
use their `clone()` method instead.
Returns:
The deep copy of the given object.
"""
from .cloning import deep_clone
if memo is None:
memo = {}
return deep_clone(x, otherwise_deepcopy=True, memo=memo)
device_of(x)
¶
Get the device of the given object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
The object whose device is being queried.
The object can be a PyTorch tensor, or a PyTorch module
(in which case the device of the first parameter tensor
will be returned), or an ObjectArray (in which case
the returned device will be the cpu device), or any object
with the attribute |
required |
Returns:
| Type | Description |
|---|---|
Union[str, torch.device] |
The device of the given object. |
Source code in evotorch/tools/misc.py
def device_of(x: Any) -> Device:
"""
Get the device of the given object.
Args:
x: The object whose device is being queried.
The object can be a PyTorch tensor, or a PyTorch module
(in which case the device of the first parameter tensor
will be returned), or an ObjectArray (in which case
the returned device will be the cpu device), or any object
with the attribute `device`.
Returns:
The device of the given object.
"""
if isinstance(x, nn.Module):
result = None
for param in x.parameters():
result = param.device
break
if result is None:
raise ValueError(f"Cannot determine the device of the module {x}")
return result
else:
return x.device
device_of_container(container, *, visited=None, visiting=None)
¶
Get the device of the given container.
It is assumed that the given container stores PyTorch tensors from which the device information will be extracted. If the container contains only basic types like int, float, string, bool, or None, or if the container is empty, then the returned device will be None. If the container contains unrecognized objects, an error will be raised.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
container |
Any |
A sequence or a dictionary of objects from which the device information will be extracted. |
required |
visited |
Optional[dict] |
Optionally a dictionary which stores the (sub)containers which are already visited. In most cases, when this function is called from outside, this is expected as None. |
None |
visiting |
Optional[str] |
Optionally a set which stores the (sub)containers which are being visited. This set is used to prevent recursion errors while handling circular references. In most cases, when this function is called from outside, this argument is expected as None. |
None |
Returns:
| Type | Description |
|---|---|
Optional[torch.device] |
The device if available, None otherwise. |
Source code in evotorch/tools/misc.py
def device_of_container(
container: Any, *, visited: Optional[dict] = None, visiting: Optional[str] = None
) -> Optional[torch.device]:
"""
Get the device of the given container.
It is assumed that the given container stores PyTorch tensors from
which the device information will be extracted.
If the container contains only basic types like int, float, string,
bool, or None, or if the container is empty, then the returned device
will be None.
If the container contains unrecognized objects, an error will be
raised.
Args:
container: A sequence or a dictionary of objects from which the
device information will be extracted.
visited: Optionally a dictionary which stores the (sub)containers
which are already visited. In most cases, when this function
is called from outside, this is expected as None.
visiting: Optionally a set which stores the (sub)containers
which are being visited. This set is used to prevent recursion
errors while handling circular references. In most cases,
when this function is called from outside, this argument is
expected as None.
Returns:
The device if available, None otherwise.
"""
container_id = id(container)
if visited is None:
visited = {}
if container_id in visited:
return visited[container_id]
if visiting is None:
visiting = set()
if container_id in visiting:
return None
class result:
device: Optional[torch.device] = None
@classmethod
def update(cls, new_device: Optional[torch.device]):
if new_device is not None:
if cls.device is None:
cls.device = new_device
else:
if new_device != cls.device:
raise ValueError(f"Encountered tensors whose `device`s mismatch: {new_device}, {cls.device}")
def call_self(sub_container):
return device_of_container(sub_container, visited=visited, visiting=visiting)
if isinstance(container, torch.Tensor):
result.update(container.device)
elif (container is None) or isinstance(container, (Number, str, bytes, bool)):
pass
elif isinstance(container, Mapping):
visiting.add(container_id)
try:
for _, v in container.items():
result.update(call_self(v))
finally:
visiting.remove(container_id)
elif isinstance(container, Iterable):
visiting.add(container_id)
try:
for v in container:
result.update(call_self(v))
finally:
visiting.remove(container_id)
else:
raise TypeError(f"Encountered an object of unrecognized type: {type(container)}")
visited[container_id] = result.device
return result.device
dtype_of(x)
¶
Get the dtype of the given object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
The object whose dtype is being queried.
The object can be a PyTorch tensor, or a PyTorch module
(in which case the dtype of the first parameter tensor
will be returned), or an ObjectArray (in which case
the returned dtype will be |
required |
Returns:
| Type | Description |
|---|---|
Union[str, torch.dtype, numpy.dtype, Type] |
The dtype of the given object. |
Source code in evotorch/tools/misc.py
def dtype_of(x: Any) -> DType:
"""
Get the dtype of the given object.
Args:
x: The object whose dtype is being queried.
The object can be a PyTorch tensor, or a PyTorch module
(in which case the dtype of the first parameter tensor
will be returned), or an ObjectArray (in which case
the returned dtype will be `object`), or any object with
the attribute `dtype`.
Returns:
The dtype of the given object.
"""
if isinstance(x, nn.Module):
result = None
for param in x.parameters():
result = param.dtype
break
if result is None:
raise ValueError(f"Cannot determine the dtype of the module {x}")
return result
else:
return x.dtype
dtype_of_container(container, *, visited=None, visiting=None)
¶
Get the dtype of the given container.
It is assumed that the given container stores PyTorch tensors from which the dtype information will be extracted. If the container contains only basic types like int, float, string, bool, or None, or if the container is empty, then the returned dtype will be None. If the container contains unrecognized objects, an error will be raised.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
container |
Any |
A sequence or a dictionary of objects from which the dtype information will be extracted. |
required |
visited |
Optional[dict] |
Optionally a dictionary which stores the (sub)containers which are already visited. In most cases, when this function is called from outside, this is expected as None. |
None |
visiting |
Optional[str] |
Optionally a set which stores the (sub)containers which are being visited. This set is used to prevent recursion errors while handling circular references. In most cases, when this function is called from outside, this argument is expected as None. |
None |
Returns:
| Type | Description |
|---|---|
Optional[torch.dtype] |
The dtype if available, None otherwise. |
Source code in evotorch/tools/misc.py
def dtype_of_container(
container: Any, *, visited: Optional[dict] = None, visiting: Optional[str] = None
) -> Optional[torch.dtype]:
"""
Get the dtype of the given container.
It is assumed that the given container stores PyTorch tensors from
which the dtype information will be extracted.
If the container contains only basic types like int, float, string,
bool, or None, or if the container is empty, then the returned dtype
will be None.
If the container contains unrecognized objects, an error will be
raised.
Args:
container: A sequence or a dictionary of objects from which the
dtype information will be extracted.
visited: Optionally a dictionary which stores the (sub)containers
which are already visited. In most cases, when this function
is called from outside, this is expected as None.
visiting: Optionally a set which stores the (sub)containers
which are being visited. This set is used to prevent recursion
errors while handling circular references. In most cases,
when this function is called from outside, this argument is
expected as None.
Returns:
The dtype if available, None otherwise.
"""
container_id = id(container)
if visited is None:
visited = {}
if container_id in visited:
return visited[container_id]
if visiting is None:
visiting = set()
if container_id in visiting:
return None
class result:
dtype: Optional[torch.dtype] = None
@classmethod
def update(cls, new_dtype: Optional[torch.dtype]):
if new_dtype is not None:
if cls.dtype is None:
cls.dtype = new_dtype
else:
if new_dtype != cls.dtype:
raise ValueError(f"Encountered tensors whose `dtype`s mismatch: {new_dtype}, {cls.dtype}")
def call_self(sub_container):
return dtype_of_container(sub_container, visited=visited, visiting=visiting)
if isinstance(container, torch.Tensor):
result.update(container.dtype)
elif (container is None) or isinstance(container, (Number, str, bytes, bool)):
pass
elif isinstance(container, Mapping):
visiting.add(container_id)
try:
for _, v in container.items():
result.update(call_self(v))
finally:
visiting.remove(container_id)
elif isinstance(container, Iterable):
visiting.add(container_id)
try:
for v in container:
result.update(call_self(v))
finally:
visiting.remove(container_id)
else:
raise TypeError(f"Encountered an object of unrecognized type: {type(container)}")
visited[container_id] = result.dtype
return result.dtype
empty_tensor_like(source, *, shape=None, length=None, dtype=None, device=None)
¶
Make an empty tensor with attributes taken from a source tensor.
The source tensor can be a PyTorch tensor, or an ObjectArray.
Unlike torch.empty_like(...), this function allows one to redefine the
shape and/or length of the new empty tensor.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source |
Any |
The source tensor whose shape, dtype, and device will be used by default for the new empty tensor. |
required |
shape |
Union[tuple, int] |
If given as None (which is the default), then the shape of the
source tensor will be used for the new empty tensor.
If given as a tuple or a |
None |
length |
Optional[int] |
If given as None (which is the default), then the length of
the new empty tensor will be equal to the length of the source
tensor (where length here means the size of the outermost
dimension, i.e., what is returned by |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
If given as None, the dtype of the new empty tensor will be
the dtype of the source tensor.
If given as a |
None |
device |
Union[str, torch.device] |
If given as None, the device of the new empty tensor will be
the device of the source tensor.
If given as a |
None |
Returns:
| Type | Description |
|---|---|
Any |
The new empty tensor. |
Source code in evotorch/tools/misc.py
def empty_tensor_like(
source: Any,
*,
shape: Optional[Union[tuple, int]] = None,
length: Optional[int] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
) -> Any:
"""
Make an empty tensor with attributes taken from a source tensor.
The source tensor can be a PyTorch tensor, or an ObjectArray.
Unlike `torch.empty_like(...)`, this function allows one to redefine the
shape and/or length of the new empty tensor.
Args:
source: The source tensor whose shape, dtype, and device will be used
by default for the new empty tensor.
shape: If given as None (which is the default), then the shape of the
source tensor will be used for the new empty tensor.
If given as a tuple or a `torch.Size` instance, then the new empty
tensor will be in this given shape instead.
This argument cannot be used together with `length`.
length: If given as None (which is the default), then the length of
the new empty tensor will be equal to the length of the source
tensor (where length here means the size of the outermost
dimension, i.e., what is returned by `len(...)`).
If given as an integer, the length of the empty tensor will be
this given length instead.
This argument cannot be used together with `shape`.
dtype: If given as None, the dtype of the new empty tensor will be
the dtype of the source tensor.
If given as a `torch.dtype` instance, then the dtype of the
tensor will be this given dtype instead.
device: If given as None, the device of the new empty tensor will be
the device of the source tensor.
If given as a `torch.device` instance, then the device of the
tensor will be this given device instead.
Returns:
The new empty tensor.
"""
from .objectarray import ObjectArray
if isinstance(source, ObjectArray):
if length is not None and shape is None:
n = int(length)
elif shape is not None and length is None:
if isinstance(shape, Iterable):
if len(shape) != 1:
raise ValueError(
f"An ObjectArray must always be 1-dimensional."
f" Therefore, this given shape is incompatible: {shape}"
)
n = int(shape[0])
elif length is None and shape is None:
n = len(source)
else:
raise ValueError("`length` and `shape` cannot be used together")
if device is not None:
if str(device) != "cpu":
raise ValueError(
f"An ObjectArray can only be allocated on cpu. However, the specified `device` is: {device}."
)
if dtype is not None:
if not is_dtype_object(dtype):
raise ValueError(
f"The dtype of an ObjectArray can only be `object`. However, the specified `dtype` is: {dtype}."
)
return ObjectArray(n)
elif isinstance(source, torch.Tensor):
if length is not None:
if shape is not None:
raise ValueError("`length` and `shape` cannot be used together")
if source.ndim == 0:
raise ValueError("`length` can only be used when the source tensor is at least 1-dimensional")
newshape = [int(length)]
newshape.extend(source.shape[1:])
shape = tuple(newshape)
if not ((dtype is None) or isinstance(dtype, torch.dtype)):
dtype = to_torch_dtype(dtype)
return torch.empty(
source.shape if shape is None else shape,
dtype=(source.dtype if dtype is None else dtype),
device=(source.device if device is None else device),
)
else:
raise TypeError(f"The source tensor is of an unrecognized type: {type(source)}")
ensure_ray()
¶
Ensure that the ray parallelization engine is initialized. If ray is already initialized, this function does nothing.
ensure_tensor_length_and_dtype(t, length, dtype, about=None, *, allow_scalar=False, device=None)
¶
Return the given sequence as a tensor while also confirming its length, dtype, and device. If the given object is already a tensor conforming to the desired length, dtype, and device, the object will be returned as it is (there will be no copying).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
t |
Any |
The tensor, or a sequence which is convertible to a tensor. |
required |
length |
int |
The length to which the tensor is expected to conform. |
required |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
The dtype to which the tensor is expected to conform. |
required |
about |
Optional[str] |
The prefix for the error message. Can be left as None. |
None |
allow_scalar |
bool |
Whether or not to accept scalars in addition
to vector of the desired length.
If |
False |
device |
Union[str, torch.device] |
The device in which the sequence is to be stored.
If the given sequence is on a different device than the
desired device, a copy on the correct device will be made.
If device is None, the default behavior of |
None |
Returns:
| Type | Description |
|---|---|
Any |
The sequence whose correctness in terms of length, dtype, and device is ensured. |
Exceptions:
| Type | Description |
|---|---|
ValueError |
if there is a length mismatch. |
Source code in evotorch/tools/misc.py
@torch.no_grad()
def ensure_tensor_length_and_dtype(
t: Any,
length: int,
dtype: DType,
about: Optional[str] = None,
*,
allow_scalar: bool = False,
device: Optional[Device] = None,
) -> Any:
"""
Return the given sequence as a tensor while also confirming its
length, dtype, and device.
If the given object is already a tensor conforming to the desired
length, dtype, and device, the object will be returned as it is
(there will be no copying).
Args:
t: The tensor, or a sequence which is convertible to a tensor.
length: The length to which the tensor is expected to conform.
dtype: The dtype to which the tensor is expected to conform.
about: The prefix for the error message. Can be left as None.
allow_scalar: Whether or not to accept scalars in addition
to vector of the desired length.
If `allow_scalar` is False, then scalars will be converted
to sequences of the desired length. The sequence will contain
the same scalar, repeated.
If `allow_scalar` is True, then the scalar itself will be
converted to a PyTorch scalar, and then will be returned.
device: The device in which the sequence is to be stored.
If the given sequence is on a different device than the
desired device, a copy on the correct device will be made.
If device is None, the default behavior of `torch.tensor(...)`
will be used, that is: if `t` is already a tensor, the result
will be on the same device, otherwise, the result will be on
the cpu.
Returns:
The sequence whose correctness in terms of length, dtype, and
device is ensured.
Raises:
ValueError: if there is a length mismatch.
"""
device_args = {}
if device is not None:
device_args["device"] = device
t = as_tensor(t, dtype=dtype, **device_args)
if t.ndim == 0:
if allow_scalar:
return t
else:
return t.repeat(length)
else:
if t.ndim != 1 or len(t) != length:
if about is not None:
err_prefix = about + ": "
else:
err_prefix = ""
raise ValueError(
f"{err_prefix}Expected a 1-dimensional tensor of length {length}, but got a tensor with shape: {t.shape}"
)
return t
expect_none(msg_prefix, **kwargs)
¶
Expect the values associated with the given keyword arguments to be None. If not, raise error.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
msg_prefix |
str |
Prefix of the error message. |
required |
kwargs |
Keyword arguments whose values are expected to be None. |
{} |
Exceptions:
| Type | Description |
|---|---|
ValueError |
if at least one of the keyword arguments has a value other than None. |
Source code in evotorch/tools/misc.py
def expect_none(msg_prefix: str, **kwargs):
"""
Expect the values associated with the given keyword arguments
to be None. If not, raise error.
Args:
msg_prefix: Prefix of the error message.
kwargs: Keyword arguments whose values are expected to be None.
Raises:
ValueError: if at least one of the keyword arguments has a value
other than None.
"""
for k, v in kwargs.items():
if v is not None:
raise ValueError(f"{msg_prefix}: expected `{k}` as None, however, it was found to be {repr(v)}")
is_bool(x)
¶
Return True if x represents a bool.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
An object whose type is being queried. |
required |
Returns:
| Type | Description |
|---|---|
bool |
True if |
Source code in evotorch/tools/misc.py
def is_bool(x: Any) -> bool:
"""
Return True if `x` represents a bool.
Args:
x: An object whose type is being queried.
Returns:
True if `x` is a bool; False otherwise.
"""
if isinstance(x, (bool, np.bool_)):
return True
elif isinstance(x, (torch.Tensor, np.ndarray)):
if x.ndim > 0:
return False
else:
return is_dtype_bool(x.dtype)
else:
return False
is_bool_vector(x)
¶
Return True if x is a vector consisting of bools.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
An object whose elements' types are to be queried. |
required |
Returns:
| Type | Description |
|---|---|
bool |
True if the elements of |
Source code in evotorch/tools/misc.py
def is_bool_vector(x: Any) -> bool:
"""
Return True if `x` is a vector consisting of bools.
Args:
x: An object whose elements' types are to be queried.
Returns:
True if the elements of `x` are bools; False otherwise.
"""
if isinstance(x, (torch.Tensor, np.ndarray)):
if x.ndim != 1:
return False
else:
return is_dtype_bool(x.dtype)
elif isinstance(x, Iterable):
for item in x:
if not is_bool(item):
return False
return True
else:
return False
is_dtype_bool(t)
¶
Return True if the given dtype is an bool type.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
t |
Union[str, torch.dtype, numpy.dtype, Type] |
The dtype, which can be a dtype string, a numpy dtype, or a PyTorch dtype. |
required |
Returns:
| Type | Description |
|---|---|
bool |
True if t is a bool type; False otherwise. |
Source code in evotorch/tools/misc.py
is_dtype_float(t)
¶
Return True if the given dtype is an float type.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
t |
Union[str, torch.dtype, numpy.dtype, Type] |
The dtype, which can be a dtype string, a numpy dtype, or a PyTorch dtype. |
required |
Returns:
| Type | Description |
|---|---|
bool |
True if t is an float type; False otherwise. |
Source code in evotorch/tools/misc.py
is_dtype_integer(t)
¶
Return True if the given dtype is an integer type.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
t |
Union[str, torch.dtype, numpy.dtype, Type] |
The dtype, which can be a dtype string, a numpy dtype, or a PyTorch dtype. |
required |
Returns:
| Type | Description |
|---|---|
bool |
True if t is an integer type; False otherwise. |
Source code in evotorch/tools/misc.py
def is_dtype_integer(t: DType) -> bool:
"""
Return True if the given dtype is an integer type.
Args:
t: The dtype, which can be a dtype string, a numpy dtype,
or a PyTorch dtype.
Returns:
True if t is an integer type; False otherwise.
"""
t: np.dtype = to_numpy_dtype(t)
return t.kind.startswith("u") or t.kind.startswith("i")
is_dtype_object(dtype)
¶
Return True if the given dtype is object or Any.
Returns:
| Type | Description |
|---|---|
bool |
True if the given dtype is |
Source code in evotorch/tools/misc.py
def is_dtype_object(dtype: DType) -> bool:
"""
Return True if the given dtype is `object` or `Any`.
Returns:
True if the given dtype is `object` or `Any`; False otherwise.
"""
if isinstance(dtype, str):
return dtype in ("object", "Any", "O")
elif dtype is object or dtype is Any:
return True
else:
return False
is_dtype_real(t)
¶
Return True if the given dtype represents real numbers (i.e. if dtype is an integer type or is a float type).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
t |
Union[str, torch.dtype, numpy.dtype, Type] |
The dtype, which can be a dtype string, a numpy dtype, or a PyTorch dtype. |
required |
Returns:
| Type | Description |
|---|---|
bool |
True if t represents a real numbers type; False otherwise. |
Source code in evotorch/tools/misc.py
def is_dtype_real(t: DType) -> bool:
"""
Return True if the given dtype represents real numbers
(i.e. if dtype is an integer type or is a float type).
Args:
t: The dtype, which can be a dtype string, a numpy dtype,
or a PyTorch dtype.
Returns:
True if t represents a real numbers type; False otherwise.
"""
return is_dtype_float(t) or is_dtype_integer(t)
is_integer(x)
¶
Return True if x is an integer.
Note that this function does NOT consider booleans as integers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
An object whose type is being queried. |
required |
Returns:
| Type | Description |
|---|---|
bool |
True if |
Source code in evotorch/tools/misc.py
def is_integer(x: Any) -> bool:
"""
Return True if `x` is an integer.
Note that this function does NOT consider booleans as integers.
Args:
x: An object whose type is being queried.
Returns:
True if `x` is an integer; False otherwise.
"""
if is_bool(x):
return False
elif isinstance(x, Integral):
return True
elif isinstance(x, (torch.Tensor, np.ndarray)):
if x.ndim > 0:
return False
else:
return is_dtype_integer(x.dtype)
else:
return False
is_integer_vector(x)
¶
Return True if x is a vector consisting of integers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
An object whose elements' types are to be queried. |
required |
Returns:
| Type | Description |
|---|---|
bool |
True if the elements of |
Source code in evotorch/tools/misc.py
def is_integer_vector(x: Any) -> bool:
"""
Return True if `x` is a vector consisting of integers.
Args:
x: An object whose elements' types are to be queried.
Returns:
True if the elements of `x` are integers; False otherwise.
"""
if isinstance(x, (torch.Tensor, np.ndarray)):
if x.ndim != 1:
return False
else:
return is_dtype_integer(x.dtype)
elif isinstance(x, Iterable):
for item in x:
if not is_integer(item):
return False
return True
else:
return False
is_real(x)
¶
Return True if x is a real number.
Note that this function does NOT consider booleans as real numbers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
An object whose type is being queried. |
required |
Returns:
| Type | Description |
|---|---|
bool |
True if |
Source code in evotorch/tools/misc.py
def is_real(x: Any) -> bool:
"""
Return True if `x` is a real number.
Note that this function does NOT consider booleans as real numbers.
Args:
x: An object whose type is being queried.
Returns:
True if `x` is a real number; False otherwise.
"""
if is_bool(x):
return False
elif isinstance(x, Real):
return True
elif isinstance(x, (torch.Tensor, np.ndarray)):
if x.ndim > 0:
return False
else:
return is_dtype_real(x.dtype)
else:
return False
is_real_vector(x)
¶
Return True if x is a vector consisting of real numbers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
An object whose elements' types are to be queried. |
required |
Returns:
| Type | Description |
|---|---|
bool |
True if the elements of |
Source code in evotorch/tools/misc.py
def is_real_vector(x: Any) -> bool:
"""
Return True if `x` is a vector consisting of real numbers.
Args:
x: An object whose elements' types are to be queried.
Returns:
True if the elements of `x` are real numbers; False otherwise.
"""
if isinstance(x, (torch.Tensor, np.ndarray)):
if x.ndim != 1:
return False
else:
return is_dtype_real(x.dtype)
elif isinstance(x, Iterable):
for item in x:
if not is_real(item):
return False
return True
else:
return False
is_sequence(x)
¶
Return True if x is a sequence.
Note that this function considers str and bytes as scalars,
not as sequences.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
The object whose sequential nature is being queried. |
required |
Returns:
| Type | Description |
|---|---|
bool |
True if |
Source code in evotorch/tools/misc.py
def is_sequence(x: Any) -> bool:
"""
Return True if `x` is a sequence.
Note that this function considers `str` and `bytes` as scalars,
not as sequences.
Args:
x: The object whose sequential nature is being queried.
Returns:
True if `x` is a sequence; False otherwise.
"""
if isinstance(x, (str, bytes)):
return False
elif isinstance(x, (np.ndarray, torch.Tensor)):
return x.ndim > 0
elif isinstance(x, Iterable):
return True
else:
return False
is_tensor_on_cpu(tensor)
¶
make_I(size=None, *, out=None, dtype=None, device=None)
¶
Make a new identity matrix (I), or change an existing tensor into one.
The following example creates a 3x3 identity matrix:
identity_matrix = make_I(3, dtype="float32")
The following example changes an already existing square matrix such that its values will store an identity matrix:
make_I(out=existing_tensor)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Optional[int] |
A single integer or a tuple containing a single integer,
where the integer specifies the length of the target square
matrix. In this context, "length" means both rowwise length
and columnwise length, since the target is a square matrix.
Note that, if the user wishes to fill an existing tensor with
identity values, then |
None |
out |
Optional[torch.Tensor] |
Optionally, the existing tensor whose values will be changed
so that they represent an identity matrix.
If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If |
None |
device |
Union[str, torch.device] |
The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing the I matrix values |
Source code in evotorch/tools/misc.py
def make_I(
size: Optional[int] = None,
*,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
) -> torch.Tensor:
"""
Make a new identity matrix (I), or change an existing tensor into one.
The following example creates a 3x3 identity matrix:
identity_matrix = make_I(3, dtype="float32")
The following example changes an already existing square matrix such that
its values will store an identity matrix:
make_I(out=existing_tensor)
Args:
size: A single integer or a tuple containing a single integer,
where the integer specifies the length of the target square
matrix. In this context, "length" means both rowwise length
and columnwise length, since the target is a square matrix.
Note that, if the user wishes to fill an existing tensor with
identity values, then `size` is expected to be left as None.
out: Optionally, the existing tensor whose values will be changed
so that they represent an identity matrix.
If an `out` tensor is given, then `size` is expected as None.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified, the default choice of
`torch.empty(...)` is used, that is, `torch.float32`.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an `out` tensor is specified, then `device` is expected
as None.
Returns:
The created or modified tensor after placing the I matrix values
"""
if size is None:
if out is None:
raise ValueError(
"When the `size` argument is missing, the function `make_I(...)` expects an `out` tensor."
" However, the `out` argument was received as None."
)
size = tuple()
else:
if isinstance(size, tuple):
if len(size) == 1:
size = size[0]
else:
raise ValueError(
f"When the `size` argument is given as a tuple,"
f" the function `make_I(...)` expects this tuple to contain exactly one element."
f" The received tuple is {size}."
)
n = int(size)
size = (n, n)
out = _out_tensor(*size, out=out, dtype=dtype, device=device)
out.zero_()
out.fill_diagonal_(1)
return out
make_batched_false_for_vmap(device)
¶
Get False, properly batched if inside vmap(..., randomness='different').
Reasoning. Imagine we have the following function:
import torch
def sample_and_shift(target_shape: tuple, shift: torch.Tensor) -> torch.Tensor:
result = torch.empty(target_shape, device=x.device)
result.normal_()
result += shift
return result
which allocates an empty tensor, then fills it with samples from the
standard normal distribution, then shifts the samples and returns the
result. An important implementation detail regarding this example function
is that all of its operations are in-place (i.e. the method normal_()
and the operator += work on the given pre-allocated tensor).
Let us now imagine that we have a batch of shift tensors, and we would like
to generate multiple shifted sample tensors. Ideally, such a batched
operation could be done by transforming the example function with the help
of vmap:
from torch.func import vmap
batched_sample_and_shift = vmap(sample_and_shift, in_dims=0, randomness="different")
where the argument randomness="different" tells PyTorch that for each
batch item, we want to generate different samples (instead of just
duplicating the same samples across the batch dimension(s)).
Such a re-sampling approach is usually desired in applications where
preserving stochasticity is crucial, evolutionary computation being one
of such case.
Now let us call our transformed function:
batch_of_shifts = ... # a tensor like `shift`, but with an extra leftmost
# dimension for the batches
# Will fail:
batched_results = batched_sample_and_shift(shape_goes_here, batch_of_shifts)
At this point, we observe that batched_sample_and_shift fails.
The reason for this failure is that the function first allocates an empty
tensor, then tries to perform random sampling in an in-place manner.
The first allocation via empty is not properly batched (it is not aware
of the active vmap), so, when we later call .normal_() on it,
there is no room for the data that would be re-sampled for each batch item.
To remedy this, we could modify our original function slightly:
import torch
def sample_and_shift2(target_shape: tuple, shift: torch.Tensor) -> torch.Tensor:
result = torch.empty(target_shape, device=x.device)
result = result + result.make_batched_false_for_vmap(x.device)
result.normal_()
result += shift
return result
In this modified function, right after making an initial allocation, we add
onto it a batched false, and re-assign the result to the variable result.
Thanks to being the result of an interaction with a batched false, the new
result variable is now properly batched (if we are inside
vmap(..., randomness="different"). Now, let us transform our function:
from torch.func import vmap
batched_sample_and_shift2 = vmap(sample_and_shift2, in_dims=0, randomness="different")
The following code should now work:
batch_of_shifts = ... # a tensor like `shift`, but with an extra leftmost
# dimension for the batches
# Should work:
batched_results = batched_sample_and_shift2(shape_goes_here, batch_of_shifts)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
device |
Union[str, torch.device] |
The target device on which the batched |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
A scalar tensor having the value |
Source code in evotorch/tools/misc.py
def make_batched_false_for_vmap(device: Device) -> torch.Tensor:
"""
Get `False`, properly batched if inside `vmap(..., randomness='different')`.
**Reasoning.**
Imagine we have the following function:
```python
import torch
def sample_and_shift(target_shape: tuple, shift: torch.Tensor) -> torch.Tensor:
result = torch.empty(target_shape, device=x.device)
result.normal_()
result += shift
return result
```
which allocates an empty tensor, then fills it with samples from the
standard normal distribution, then shifts the samples and returns the
result. An important implementation detail regarding this example function
is that all of its operations are in-place (i.e. the method `normal_()`
and the operator `+=` work on the given pre-allocated tensor).
Let us now imagine that we have a batch of shift tensors, and we would like
to generate multiple shifted sample tensors. Ideally, such a batched
operation could be done by transforming the example function with the help
of `vmap`:
```python
from torch.func import vmap
batched_sample_and_shift = vmap(sample_and_shift, in_dims=0, randomness="different")
```
where the argument `randomness="different"` tells PyTorch that for each
batch item, we want to generate different samples (instead of just
duplicating the same samples across the batch dimension(s)).
Such a re-sampling approach is usually desired in applications where
preserving stochasticity is crucial, evolutionary computation being one
of such case.
Now let us call our transformed function:
```python
batch_of_shifts = ... # a tensor like `shift`, but with an extra leftmost
# dimension for the batches
# Will fail:
batched_results = batched_sample_and_shift(shape_goes_here, batch_of_shifts)
```
At this point, we observe that `batched_sample_and_shift` fails.
The reason for this failure is that the function first allocates an empty
tensor, then tries to perform random sampling in an in-place manner.
The first allocation via `empty` is not properly batched (it is not aware
of the active `vmap`), so, when we later call `.normal_()` on it,
there is no room for the data that would be re-sampled for each batch item.
To remedy this, we could modify our original function slightly:
```python
import torch
def sample_and_shift2(target_shape: tuple, shift: torch.Tensor) -> torch.Tensor:
result = torch.empty(target_shape, device=x.device)
result = result + result.make_batched_false_for_vmap(x.device)
result.normal_()
result += shift
return result
```
In this modified function, right after making an initial allocation, we add
onto it a batched false, and re-assign the result to the variable `result`.
Thanks to being the result of an interaction with a batched false, the new
`result` variable is now properly batched (if we are inside
`vmap(..., randomness="different")`. Now, let us transform our function:
```python
from torch.func import vmap
batched_sample_and_shift2 = vmap(sample_and_shift2, in_dims=0, randomness="different")
```
The following code should now work:
```python
batch_of_shifts = ... # a tensor like `shift`, but with an extra leftmost
# dimension for the batches
# Should work:
batched_results = batched_sample_and_shift2(shape_goes_here, batch_of_shifts)
```
Args:
device: The target device on which the batched `False` will be created
Returns:
A scalar tensor having the value `False`. This returned tensor will be
a batch of scalar tensors (i.e. a `BatchedTensor`) if we are inside
`vmap(..., randomness="different")`.
"""
return torch.randint(0, 1, tuple(), dtype=torch.bool, device=device)
make_empty(*size, *, dtype=None, device=None)
¶
Make an empty tensor.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Shape of the empty tensor to be created.
expected as multiple positional arguments of integers,
or as a single positional argument containing a tuple of
integers.
Note that when the user wishes to create an |
() |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32) or, for creating an |
None |
device |
Union[str, torch.device] |
The device in which the new empty tensor will be stored. If not specified, "cpu" will be used. |
None |
Returns:
| Type | Description |
|---|---|
Iterable |
The new empty tensor, which can be a PyTorch tensor or an
|
Source code in evotorch/tools/misc.py
def make_empty(
*size: Size,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
) -> Iterable:
"""
Make an empty tensor.
Args:
size: Shape of the empty tensor to be created.
expected as multiple positional arguments of integers,
or as a single positional argument containing a tuple of
integers.
Note that when the user wishes to create an `ObjectArray`
(i.e. when `dtype` is given as `object`), then the size
is expected as a single integer, or as a single-element
tuple containing an integer (because `ObjectArray` can only
be one-dimensional).
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32) or, for creating an `ObjectArray`,
"object" (as string) or `object` or `Any`.
If `dtype` is not specified, the default choice of
`torch.empty(...)` is used, that is, `torch.float32`.
device: The device in which the new empty tensor will be stored.
If not specified, "cpu" will be used.
Returns:
The new empty tensor, which can be a PyTorch tensor or an
`ObjectArray`.
"""
from .objectarray import ObjectArray
if (dtype is not None) and is_dtype_object(dtype):
if (device is None) or (str(device) == "cpu"):
if len(size) == 1:
size = size[0]
return ObjectArray(size)
else:
return ValueError(
f"Invalid device for ObjectArray: {repr(device)}. Note: an ObjectArray can only be stored on 'cpu'."
)
else:
kwargs = {}
if dtype is not None:
kwargs["dtype"] = to_torch_dtype(dtype)
if device is not None:
kwargs["device"] = device
return torch.empty(*size, **kwargs)
make_gaussian(*size, *, center=None, stdev=None, symmetric=False, out=None, dtype=None, device=None, generator=None)
¶
Make a new or existing tensor filled by Gaussian distributed values. This function can work only with float dtypes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Size of the new tensor to be filled with Gaussian distributed values. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor instead, then no positional argument is expected. |
() |
center |
Union[float, Iterable[float], torch.Tensor] |
Center point (i.e. mean) of the Gaussian distribution.
Can be a scalar, or a tensor.
If not specified, the center point will be taken as 0.
Note that, if one specifies |
None |
stdev |
Union[float, Iterable[float], torch.Tensor] |
Standard deviation for the Gaussian distributed values.
Can be a scalar, or a tensor.
If not specified, the standard deviation will be taken as 1.
Note that, if one specifies |
None |
symmetric |
bool |
Whether or not the values should be sampled in a symmetric (i.e. antithetic) manner. The default is False. |
False |
out |
Optional[torch.Tensor] |
Optionally, the tensor to be filled by Gaussian distributed
values. If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If |
None |
device |
Union[str, torch.device] |
The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an |
None |
generator |
Any |
Pseudo-random number generator to be used when sampling
the values. Can be a |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing the Gaussian distributed values. |
Source code in evotorch/tools/misc.py
def make_gaussian(
*size: Size,
center: Optional[RealOrVector] = None,
stdev: Optional[RealOrVector] = None,
symmetric: bool = False,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
generator: Any = None,
) -> torch.Tensor:
"""
Make a new or existing tensor filled by Gaussian distributed values.
This function can work only with float dtypes.
Args:
size: Size of the new tensor to be filled with Gaussian distributed
values. This can be given as multiple positional arguments, each
such positional argument being an integer, or as a single
positional argument of a tuple, the tuple containing multiple
integers. Note that, if the user wishes to fill an existing
tensor instead, then no positional argument is expected.
center: Center point (i.e. mean) of the Gaussian distribution.
Can be a scalar, or a tensor.
If not specified, the center point will be taken as 0.
Note that, if one specifies `center`, then `stdev` is also
expected to be explicitly specified.
stdev: Standard deviation for the Gaussian distributed values.
Can be a scalar, or a tensor.
If not specified, the standard deviation will be taken as 1.
Note that, if one specifies `stdev`, then `center` is also
expected to be explicitly specified.
symmetric: Whether or not the values should be sampled in a
symmetric (i.e. antithetic) manner.
The default is False.
out: Optionally, the tensor to be filled by Gaussian distributed
values. If an `out` tensor is given, then no `size` argument is
expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified, the default choice of
`torch.empty(...)` is used, that is, `torch.float32`.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an `out` tensor is specified, then `device` is expected
as None.
generator: Pseudo-random number generator to be used when sampling
the values. Can be a `torch.Generator`, or an object with
a `generator` attribute (such as `Problem`).
If left as None, the global generator of PyTorch will be used.
Returns:
The created or modified tensor after placing the Gaussian
distributed values.
"""
scalar_requested = _scalar_requested(*size)
if scalar_requested:
size = (1,)
out = _out_tensor_for_random_operation(*size, out=out, dtype=dtype, device=device)
gen_kwargs = _generator_kwargs(generator)
if symmetric:
leftmost_dim = out.shape[0]
if (leftmost_dim % 2) != 0:
raise ValueError(
f"Symmetric sampling cannot be done if the leftmost dimension of the target tensor is odd."
f" The shape of the target tensor is: {repr(out.shape)}."
)
out[0::2, ...].normal_(**gen_kwargs)
out[1::2, ...] = out[0::2, ...]
out[1::2, ...] *= -1
else:
out.normal_(**gen_kwargs)
if (center is None) and (stdev is None):
pass # do nothing
elif (center is not None) and (stdev is not None):
stdev = torch.as_tensor(stdev, dtype=out.dtype, device=out.device)
out *= stdev
center = torch.as_tensor(center, dtype=out.dtype, device=out.device)
out += center
else:
raise ValueError(
f"Please either specify none of `stdev` and `center`, or both of them."
f" Currently, `center` is {center}"
f" and `stdev` is {stdev}."
)
if scalar_requested:
out = out[0]
return out
make_nan(*size, *, out=None, dtype=None, device=None)
¶
Make a new tensor filled with NaN, or fill an existing tensor with NaN.
The following example creates a float32 tensor filled with NaN values, of shape (3, 5):
nan_values = make_nan(3, 5, dtype="float32")
The following example fills an existing tensor with NaNs.
make_nan(out=existing_tensor)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Size of the new tensor to be filled with NaNs. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor with NaN values, then no positional argument is expected. |
() |
out |
Optional[torch.Tensor] |
Optionally, the tensor to be filled by NaN values.
If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If |
None |
device |
Union[str, torch.device] |
The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing NaN values. |
Source code in evotorch/tools/misc.py
def make_nan(
*size: Size,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
) -> torch.Tensor:
"""
Make a new tensor filled with NaN, or fill an existing tensor with NaN.
The following example creates a float32 tensor filled with NaN values,
of shape (3, 5):
nan_values = make_nan(3, 5, dtype="float32")
The following example fills an existing tensor with NaNs.
make_nan(out=existing_tensor)
Args:
size: Size of the new tensor to be filled with NaNs.
This can be given as multiple positional arguments, each such
positional argument being an integer, or as a single positional
argument of a tuple, the tuple containing multiple integers.
Note that, if the user wishes to fill an existing tensor with
NaN values, then no positional argument is expected.
out: Optionally, the tensor to be filled by NaN values.
If an `out` tensor is given, then no `size` argument is expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified, the default choice of
`torch.empty(...)` is used, that is, `torch.float32`.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an `out` tensor is specified, then `device` is expected
as None.
Returns:
The created or modified tensor after placing NaN values.
"""
if _scalar_requested(*size):
return _scalar_tensor(float("nan"), out=out, dtype=dtype, device=device)
else:
out = _out_tensor(*size, out=out, dtype=dtype, device=device)
out[:] = float("nan")
return out
make_ones(*size, *, out=None, dtype=None, device=None)
¶
Make a new tensor filled with 1, or fill an existing tensor with 1.
The following example creates a float32 tensor filled with 1 values, of shape (3, 5):
zero_values = make_ones(3, 5, dtype="float32")
The following example fills an existing tensor with 1s:
make_ones(out=existing_tensor)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Size of the new tensor to be filled with 1. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor with 1 values, then no positional argument is expected. |
() |
out |
Optional[torch.Tensor] |
Optionally, the tensor to be filled by 1 values.
If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If |
None |
device |
Union[str, torch.device] |
The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing 1 values. |
Source code in evotorch/tools/misc.py
def make_ones(
*size: Size,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
) -> torch.Tensor:
"""
Make a new tensor filled with 1, or fill an existing tensor with 1.
The following example creates a float32 tensor filled with 1 values,
of shape (3, 5):
zero_values = make_ones(3, 5, dtype="float32")
The following example fills an existing tensor with 1s:
make_ones(out=existing_tensor)
Args:
size: Size of the new tensor to be filled with 1.
This can be given as multiple positional arguments, each such
positional argument being an integer, or as a single positional
argument of a tuple, the tuple containing multiple integers.
Note that, if the user wishes to fill an existing tensor with
1 values, then no positional argument is expected.
out: Optionally, the tensor to be filled by 1 values.
If an `out` tensor is given, then no `size` argument is expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified, the default choice of
`torch.empty(...)` is used, that is, `torch.float32`.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an `out` tensor is specified, then `device` is expected
as None.
Returns:
The created or modified tensor after placing 1 values.
"""
if _scalar_requested(*size):
return _scalar_tensor(1, out=out, dtype=dtype, device=device)
else:
out = _out_tensor(*size, out=out, dtype=dtype, device=device)
out[:] = 1
return out
make_randint(*size, *, n, out=None, dtype=None, device=None, generator=None)
¶
Make a new or existing tensor filled by random integers.
The integers are uniformly distributed within [0 ... n-1].
This function can be used with integer or float dtypes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Size of the new tensor to be filled with uniformly distributed values. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor instead, then no positional argument is expected. |
() |
n |
Union[int, float, torch.Tensor] |
Number of choice(s) for integer sampling.
The lowest possible value will be 0, and the highest possible
value will be n - 1.
|
required |
out |
Optional[torch.Tensor] |
Optionally, the tensor to be filled by the random integers.
If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "int64") or a PyTorch dtype
(e.g. torch.int64).
If |
None |
device |
Union[str, torch.device] |
The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an |
None |
generator |
Any |
Pseudo-random number generator to be used when sampling
the values. Can be a |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing the uniformly distributed values. |
Source code in evotorch/tools/misc.py
def make_randint(
*size: Size,
n: Union[int, float, torch.Tensor],
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
generator: Any = None,
) -> torch.Tensor:
"""
Make a new or existing tensor filled by random integers.
The integers are uniformly distributed within `[0 ... n-1]`.
This function can be used with integer or float dtypes.
Args:
size: Size of the new tensor to be filled with uniformly distributed
values. This can be given as multiple positional arguments, each
such positional argument being an integer, or as a single
positional argument of a tuple, the tuple containing multiple
integers. Note that, if the user wishes to fill an existing
tensor instead, then no positional argument is expected.
n: Number of choice(s) for integer sampling.
The lowest possible value will be 0, and the highest possible
value will be n - 1.
`n` can be a scalar, or a tensor.
out: Optionally, the tensor to be filled by the random integers.
If an `out` tensor is given, then no `size` argument is
expected.
dtype: Optionally a string (e.g. "int64") or a PyTorch dtype
(e.g. torch.int64).
If `dtype` is not specified, torch.int64 will be used.
device: The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an `out` tensor is specified, then `device` is expected
as None.
generator: Pseudo-random number generator to be used when sampling
the values. Can be a `torch.Generator`, or an object with
a `generator` attribute (such as `Problem`).
If left as None, the global generator of PyTorch will be used.
Returns:
The created or modified tensor after placing the uniformly
distributed values.
"""
scalar_requested = _scalar_requested(*size)
if scalar_requested:
size = (1,)
if (dtype is None) and (out is None):
dtype = torch.int64
out = _out_tensor_for_random_operation(*size, out=out, dtype=dtype, device=device)
gen_kwargs = _generator_kwargs(generator)
out.random_(**gen_kwargs)
out %= n
if scalar_requested:
out = out[0]
return out
make_tensor(data, *, dtype=None, device=None, read_only=False)
¶
Make a new tensor.
This function can be used to create PyTorch tensors, or ObjectArray instances with or without read-only behavior.
The following example creates a 2-dimensional PyTorch tensor:
my_tensor = make_tensor(
[[1, 2], [3, 4]],
dtype="float32", # alternatively, torch.float32
device="cpu",
)
The following example creates an ObjectArray from a list that contains arbitrary data:
my_obj_tensor = make_tensor(["a_string", (1, 2)], dtype=object)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data |
Any |
The data to be converted to a tensor.
If one wishes to create a PyTorch tensor, this can be anything
that can be stored by a PyTorch tensor.
If one wishes to create an |
required |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32"), or a PyTorch dtype
(e.g. torch.float32), or |
None |
device |
Union[str, torch.device] |
The device in which the tensor will be stored.
If |
None |
read_only |
bool |
Whether or not the created tensor will be read-only. By default, this is False. |
False |
Returns:
| Type | Description |
|---|---|
Iterable |
A PyTorch tensor or an ObjectArray. |
Source code in evotorch/tools/misc.py
def make_tensor(
data: Any,
*,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
read_only: bool = False,
) -> Iterable:
"""
Make a new tensor.
This function can be used to create PyTorch tensors, or ObjectArray
instances with or without read-only behavior.
The following example creates a 2-dimensional PyTorch tensor:
my_tensor = make_tensor(
[[1, 2], [3, 4]],
dtype="float32", # alternatively, torch.float32
device="cpu",
)
The following example creates an ObjectArray from a list that contains
arbitrary data:
my_obj_tensor = make_tensor(["a_string", (1, 2)], dtype=object)
Args:
data: The data to be converted to a tensor.
If one wishes to create a PyTorch tensor, this can be anything
that can be stored by a PyTorch tensor.
If one wishes to create an `ObjectArray` and therefore passes
`dtype=object`, then the provided `data` is expected as an
`Iterable`.
dtype: Optionally a string (e.g. "float32"), or a PyTorch dtype
(e.g. torch.float32), or `object` or "object" (as a string)
or `Any` if one wishes to create an `ObjectArray`.
If `dtype` is not specified, it will be assumed that the user
wishes to create a PyTorch tensor (not an `ObjectArray`) and
then `dtype` will be inferred from the provided `data`
(according to the default behavior of PyTorch).
device: The device in which the tensor will be stored.
If `device` is not specified, it will be understood from the
given `data` (according to the default behavior of PyTorch).
read_only: Whether or not the created tensor will be read-only.
By default, this is False.
Returns:
A PyTorch tensor or an ObjectArray.
"""
from .objectarray import ObjectArray
from .readonlytensor import as_read_only_tensor
if (dtype is not None) and is_dtype_object(dtype):
if not hasattr(data, "__len__"):
data = list(data)
n = len(data)
result = ObjectArray(n)
result[:] = data
else:
kwargs = {}
if dtype is not None:
kwargs["dtype"] = to_torch_dtype(dtype)
if device is not None:
kwargs["device"] = device
result = torch.tensor(data, **kwargs)
if read_only:
result = as_read_only_tensor(result)
return result
make_uniform(*size, *, lb=None, ub=None, out=None, dtype=None, device=None, generator=None)
¶
Make a new or existing tensor filled by uniformly distributed values. Both lower and upper bounds are inclusive. This function can work with both float and int dtypes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Size of the new tensor to be filled with uniformly distributed values. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor instead, then no positional argument is expected. |
() |
lb |
Union[float, Iterable[float], torch.Tensor] |
Lower bound for the uniformly distributed values.
Can be a scalar, or a tensor.
If not specified, the lower bound will be taken as 0.
Note that, if one specifies |
None |
ub |
Union[float, Iterable[float], torch.Tensor] |
Upper bound for the uniformly distributed values.
Can be a scalar, or a tensor.
If not specified, the upper bound will be taken as 1.
Note that, if one specifies |
None |
out |
Optional[torch.Tensor] |
Optionally, the tensor to be filled by uniformly distributed
values. If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If |
None |
device |
Union[str, torch.device] |
The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an |
None |
generator |
Any |
Pseudo-random number generator to be used when sampling
the values. Can be a |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing the uniformly distributed values. |
Source code in evotorch/tools/misc.py
def make_uniform(
*size: Size,
lb: Optional[RealOrVector] = None,
ub: Optional[RealOrVector] = None,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
generator: Any = None,
) -> torch.Tensor:
"""
Make a new or existing tensor filled by uniformly distributed values.
Both lower and upper bounds are inclusive.
This function can work with both float and int dtypes.
Args:
size: Size of the new tensor to be filled with uniformly distributed
values. This can be given as multiple positional arguments, each
such positional argument being an integer, or as a single
positional argument of a tuple, the tuple containing multiple
integers. Note that, if the user wishes to fill an existing
tensor instead, then no positional argument is expected.
lb: Lower bound for the uniformly distributed values.
Can be a scalar, or a tensor.
If not specified, the lower bound will be taken as 0.
Note that, if one specifies `lb`, then `ub` is also expected to
be explicitly specified.
ub: Upper bound for the uniformly distributed values.
Can be a scalar, or a tensor.
If not specified, the upper bound will be taken as 1.
Note that, if one specifies `ub`, then `lb` is also expected to
be explicitly specified.
out: Optionally, the tensor to be filled by uniformly distributed
values. If an `out` tensor is given, then no `size` argument is
expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified, the default choice of
`torch.empty(...)` is used, that is, `torch.float32`.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an `out` tensor is specified, then `device` is expected
as None.
generator: Pseudo-random number generator to be used when sampling
the values. Can be a `torch.Generator`, or an object with
a `generator` attribute (such as `Problem`).
If left as None, the global generator of PyTorch will be used.
Returns:
The created or modified tensor after placing the uniformly
distributed values.
"""
scalar_requested = _scalar_requested(*size)
if scalar_requested:
size = (1,)
def _invalid_bound_args():
raise ValueError(
f"Expected both `lb` and `ub` as None, or both `lb` and `ub` as not None."
f" It appears that one of them is None, while the other is not."
f" lb: {repr(lb)}."
f" ub: {repr(ub)}."
)
out = _out_tensor_for_random_operation(*size, out=out, dtype=dtype, device=device)
gen_kwargs = _generator_kwargs(generator)
def _cast_bounds():
nonlocal lb, ub
lb = torch.as_tensor(lb, dtype=out.dtype, device=out.device)
ub = torch.as_tensor(ub, dtype=out.dtype, device=out.device)
if out.dtype == torch.bool:
out.random_(**gen_kwargs)
if (lb is None) and (ub is None):
pass # nothing to do
elif (lb is not None) and (ub is not None):
_cast_bounds()
lb_shape_matches = lb.shape == out.shape
ub_shape_matches = ub.shape == out.shape
if (not lb_shape_matches) or (not ub_shape_matches):
all_false = torch.zeros_like(out)
if not lb_shape_matches:
lb = lb | all_false
if not ub_shape_matches:
ub = ub | all_false
mask_for_always_false = (~lb) & (~ub)
mask_for_always_true = lb & ub
out[mask_for_always_false] = False
out[mask_for_always_true] = True
else:
_invalid_bound_args()
elif out.dtype in (torch.uint8, torch.int8, torch.int16, torch.int32, torch.int64):
out.random_(**gen_kwargs)
if (lb is None) and (ub is None):
out %= 2
elif (lb is not None) and (ub is not None):
_cast_bounds()
diff = (ub - lb) + 1
out -= lb
out %= diff
out += lb
else:
_invalid_bound_args()
else:
out.uniform_(**gen_kwargs)
if (lb is None) and (ub is None):
pass # nothing to do
elif (lb is not None) and (ub is not None):
_cast_bounds()
diff = ub - lb
out *= diff
out += lb
else:
_invalid_bound_args()
if scalar_requested:
out = out[0]
return out
make_zeros(*size, *, out=None, dtype=None, device=None)
¶
Make a new tensor filled with 0, or fill an existing tensor with 0.
The following example creates a float32 tensor filled with 0 values, of shape (3, 5):
zero_values = make_zeros(3, 5, dtype="float32")
The following example fills an existing tensor with 0s:
make_zeros(out=existing_tensor)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Size of the new tensor to be filled with 0. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor with 0 values, then no positional argument is expected. |
() |
out |
Optional[torch.Tensor] |
Optionally, the tensor to be filled by 0 values.
If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If |
None |
device |
Union[str, torch.device] |
The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing 0 values. |
Source code in evotorch/tools/misc.py
def make_zeros(
*size: Size,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
) -> torch.Tensor:
"""
Make a new tensor filled with 0, or fill an existing tensor with 0.
The following example creates a float32 tensor filled with 0 values,
of shape (3, 5):
zero_values = make_zeros(3, 5, dtype="float32")
The following example fills an existing tensor with 0s:
make_zeros(out=existing_tensor)
Args:
size: Size of the new tensor to be filled with 0.
This can be given as multiple positional arguments, each such
positional argument being an integer, or as a single positional
argument of a tuple, the tuple containing multiple integers.
Note that, if the user wishes to fill an existing tensor with
0 values, then no positional argument is expected.
out: Optionally, the tensor to be filled by 0 values.
If an `out` tensor is given, then no `size` argument is expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified, the default choice of
`torch.empty(...)` is used, that is, `torch.float32`.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new tensor will be stored.
If not specified, "cpu" will be used.
If an `out` tensor is specified, then `device` is expected
as None.
Returns:
The created or modified tensor after placing 0 values.
"""
if _scalar_requested(*size):
return _scalar_tensor(0, out=out, dtype=dtype, device=device)
else:
out = _out_tensor(*size, out=out, dtype=dtype, device=device)
out.zero_()
return out
message_from(sender, message)
¶
Prepend the sender object's name and id to a string message.
Let us imagine that we have a class named Example:
from evotorch.tools import message_from
class Example:
def say_hello(self):
print(message_from(self, "Hello!"))
Let us now instantiate this class and use its say_hello method:
The output becomes something like this:
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sender |
object |
The object which produces the message |
required |
message |
Any |
The message, as something that can be converted to string |
required |
Returns:
| Type | Description |
|---|---|
str |
The new message string, with the details regarding the sender object inserted to the beginning. |
Source code in evotorch/tools/misc.py
def message_from(sender: object, message: Any) -> str:
"""
Prepend the sender object's name and id to a string message.
Let us imagine that we have a class named `Example`:
```python
from evotorch.tools import message_from
class Example:
def say_hello(self):
print(message_from(self, "Hello!"))
```
Let us now instantiate this class and use its `say_hello` method:
```python
ex = Example()
ex.say_hello()
```
The output becomes something like this:
```
Instance of `Example` (id:...) -- Hello!
```
Args:
sender: The object which produces the message
message: The message, as something that can be converted to string
Returns:
The new message string, with the details regarding the sender object
inserted to the beginning.
"""
sender_type = type(sender).__name__
sender_id = id(sender)
return f"Instance of `{sender_type}` (id:{sender_id}) -- {message}"
modify_tensor(original, target, lb=None, ub=None, max_change=None, in_place=False)
¶
Return the modified version of the original tensor, with bounds checking.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
original |
Tensor |
The original tensor. |
required |
target |
Tensor |
The target tensor which contains the values to replace the old ones in the original tensor. |
required |
lb |
Union[float, torch.Tensor] |
The lower bound(s), as a scalar or as an tensor. Values below these bounds are clipped in the resulting tensor. None means -inf. |
None |
ub |
Union[float, torch.Tensor] |
The upper bound(s), as a scalar or as an tensor. Value above these bounds are clipped in the resulting tensor. None means +inf. |
None |
max_change |
Union[float, torch.Tensor] |
The ratio of allowed change.
In more details, when given as a real number r,
modifications are allowed only within
|
None |
in_place |
bool |
Provide this as True if you wish the modification to be done within the original tensor. The default value of this argument is False, which means, the original tensor is not changed, and its modified version is returned as an independent copy. |
False |
Returns:
| Type | Description |
|---|---|
Tensor |
The modified tensor. |
Source code in evotorch/tools/misc.py
@torch.no_grad()
def modify_tensor(
original: torch.Tensor,
target: torch.Tensor,
lb: Optional[Union[float, torch.Tensor]] = None,
ub: Optional[Union[float, torch.Tensor]] = None,
max_change: Optional[Union[float, torch.Tensor]] = None,
in_place: bool = False,
) -> torch.Tensor:
"""Return the modified version of the original tensor, with bounds checking.
Args:
original: The original tensor.
target: The target tensor which contains the values to replace the
old ones in the original tensor.
lb: The lower bound(s), as a scalar or as an tensor.
Values below these bounds are clipped in the resulting tensor.
None means -inf.
ub: The upper bound(s), as a scalar or as an tensor.
Value above these bounds are clipped in the resulting tensor.
None means +inf.
max_change: The ratio of allowed change.
In more details, when given as a real number r,
modifications are allowed only within
``[original-(r*abs(original)) ... original+(r*abs(original))]``.
Modifications beyond this interval are clipped.
This argument can also be left as None if no such limitation
is needed.
in_place: Provide this as True if you wish the modification to be
done within the original tensor. The default value of this
argument is False, which means, the original tensor is not
changed, and its modified version is returned as an independent
copy.
Returns:
The modified tensor.
"""
if (lb is None) and (ub is None) and (max_change is None):
# If there is no restriction regarding how the tensor
# should be modified (no lb, no ub, no max_change),
# then we simply use the target values
# themselves for modifying the tensor.
if in_place:
original[:] = target
return original
else:
return target
else:
# If there are some restriction regarding how the tensor
# should be modified, then we turn to the following
# operations
def convert_to_tensor(x, tensorname: str):
if isinstance(x, torch.Tensor):
converted = x
else:
converted = torch.as_tensor(x, dtype=original.dtype, device=original.device)
if converted.ndim == 0 or converted.shape == original.shape:
return converted
else:
raise IndexError(
f"Argument {tensorname}: shape mismatch."
f" Shape of the original tensor: {original.shape}."
f" Shape of {tensorname}: {converted.shape}."
)
if lb is None:
# If lb is None, then it should be taken as -inf
lb = convert_to_tensor(float("-inf"), "lb")
else:
lb = convert_to_tensor(lb, "lb")
if ub is None:
# If ub is None, then it should be taken as +inf
ub = convert_to_tensor(float("inf"), "ub")
else:
ub = convert_to_tensor(ub, "ub")
if max_change is not None:
# If max_change is provided as something other than None,
# then we update the lb and ub so that they are tight
# enough to satisfy the max_change restriction.
max_change = convert_to_tensor(max_change, "max_change")
allowed_amounts = torch.abs(original) * max_change
allowed_lb = original - allowed_amounts
allowed_ub = original + allowed_amounts
lb = torch.max(lb, allowed_lb)
ub = torch.min(ub, allowed_ub)
## If in_place is given as True, the clipping (that we are about
## to perform), should be in-place.
# more_config = {}
# if in_place:
# more_config['out'] = original
#
## Return the clipped version of the target values
# return torch.clamp(target, lb, ub, **more_config)
result = torch.max(target, lb)
result = torch.min(result, ub)
if in_place:
original[:] = result
return original
else:
return result
modify_vector(original, target, *, lb=None, ub=None, max_change=None)
¶
Return the modified version(s) of the vector(s), with bounds checking.
This function is similar to modify_tensor, but it has the following
different behaviors:
- Assumes that all of its arguments are either vectors, or are batches of vectors. If some or more of its arguments have 2 or more dimensions, those arguments will be considered as batches, and the computation will be vectorized to return a batch of results.
- Designed to be
vmap-friendly. - Designed for functional programming paradigm, and therefore lacks the in-place modification option.
Source code in evotorch/tools/misc.py
def modify_vector(
original: torch.Tensor,
target: torch.Tensor,
*,
lb: Optional[Union[float, torch.Tensor]] = None,
ub: Optional[Union[float, torch.Tensor]] = None,
max_change: Optional[Union[float, torch.Tensor]] = None,
) -> torch.Tensor:
"""
Return the modified version(s) of the vector(s), with bounds checking.
This function is similar to `modify_tensor`, but it has the following
different behaviors:
- Assumes that all of its arguments are either vectors, or are batches
of vectors. If some or more of its arguments have 2 or more dimensions,
those arguments will be considered as batches, and the computation will
be vectorized to return a batch of results.
- Designed to be `vmap`-friendly.
- Designed for functional programming paradigm, and therefore lacks the
in-place modification option.
"""
from ..decorators import expects_ndim
if max_change is None:
result = target
else:
result = expects_ndim(_modify_vector_using_max_change, (1, 1, 1), allow_smaller_ndim=True)(
original, target, max_change
)
if (lb is None) and (ub is None):
pass # no strict boundaries, so, nothing more to do
elif (lb is not None) and (ub is not None):
result = expects_ndim(_modify_vector_using_bounds, (1, 1, 1), allow_smaller_ndim=True)(result, lb, ub)
else:
raise ValueError(
"`modify_vector` expects either with `lb` and `ub` given together, or with both of them omitted."
" Having only `lb` or only `ub` is not supported."
)
return result
numpy_copy(x, dtype=None)
¶
Return a numpy copy of the given iterable.
The newly created numpy array will be mutable, even if the original iterable object is read-only.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Iterable |
Any Iterable whose numpy copy will be returned. |
required |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
The desired dtype. Can be given as a numpy dtype, as a torch dtype, or a native dtype (e.g. int, float), or as a string (e.g. "float32"). If left as None, dtype will be determined according to the data contained by the original iterable object. |
None |
Returns:
| Type | Description |
|---|---|
ndarray |
The numpy copy of the original iterable object. |
Source code in evotorch/tools/misc.py
def numpy_copy(x: Iterable, dtype: Optional[DType] = None) -> np.ndarray:
"""
Return a numpy copy of the given iterable.
The newly created numpy array will be mutable, even if the
original iterable object is read-only.
Args:
x: Any Iterable whose numpy copy will be returned.
dtype: The desired dtype. Can be given as a numpy dtype,
as a torch dtype, or a native dtype (e.g. int, float),
or as a string (e.g. "float32").
If left as None, dtype will be determined according
to the data contained by the original iterable object.
Returns:
The numpy copy of the original iterable object.
"""
from .objectarray import ObjectArray
needs_casting = dtype is not None
if isinstance(x, ObjectArray):
result = x.numpy()
elif isinstance(x, torch.Tensor):
result = x.cpu().clone().numpy()
elif isinstance(x, np.ndarray):
result = x.copy()
else:
needs_casting = False
result = np.array(x, dtype=dtype)
if needs_casting:
result = result.astype(dtype)
return result
pass_info_if_needed(f, info)
¶
Pass additional arguments into a callable, the info dictionary is unpacked and passed as additional keyword arguments only if the policy is decorated with the pass_info decorator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
f |
Callable |
The callable to be called. |
required |
info |
Dict[str, Any] |
The info to be passed to the callable. |
required |
Returns:
| Type | Description |
|---|---|
Callable |
The callable with extra arguments |
Exceptions:
| Type | Description |
|---|---|
TypeError |
If the callable is decorated with the pass_info decorator, but its signature does not match the expected signature. |
Source code in evotorch/tools/misc.py
def pass_info_if_needed(f: Callable, info: Dict[str, Any]) -> Callable:
"""
Pass additional arguments into a callable, the info dictionary is unpacked
and passed as additional keyword arguments only if the policy is decorated
with the [pass_info][evotorch.decorators.pass_info] decorator.
Args:
f (Callable): The callable to be called.
info (Dict[str, Any]): The info to be passed to the callable.
Returns:
Callable: The callable with extra arguments
Raises:
TypeError: If the callable is decorated with the [pass_info][evotorch.decorators.pass_info] decorator,
but its signature does not match the expected signature.
"""
if hasattr(f, "__evotorch_pass_info__"):
try:
sig = inspect.signature(f)
sig.bind_partial(**info)
except TypeError:
raise TypeError(
"Callable {f} is decorated with @pass_info, but it doesn't expect some of the extra arguments "
f"({', '.join(info.keys())}). Hint: maybe you forgot to add **kwargs to the function signature?"
)
except Exception:
pass
return functools.partial(f, **info)
else:
return f
set_default_logger_config(logger_name='evotorch', logger_level=20, show_process=True, show_lineno=False, override=False)
¶
Configure the "EvoTorch" Python logger to print to the console with default format.
The logger will be configured to print to all messages with level INFO or lower to stdout and all messages with level WARNING or higher to stderr.
The default format is:
[2022-11-23 22:28:47] INFO <75935> evotorch: This is a log message
{asctime} {level} {process} {logger_name}: {message}
show_process=False to hide Process ID or show_lineno=True to
show the filename and line number of the log message instead of the Logger Name.
This function should be called before any other logging is performed, otherwise the default configuration will
not be applied. If the logger is already configured, this function will do nothing unless override=True is passed,
in which case the logger will be reconfigured.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
logger_name |
str |
Name of the logger to configure. |
'evotorch' |
logger_level |
int |
Level of the logger to configure. |
20 |
show_process |
bool |
Whether to show the process name in the log message. |
True |
show_lineno |
bool |
Whether to show the filename with the line number in the log message or just the name of the logger. |
False |
override |
bool |
Whether to override the logger configuration if it has already been configured. |
False |
Source code in evotorch/tools/misc.py
def set_default_logger_config(
logger_name: str = "evotorch",
logger_level: int = logging.INFO,
show_process: bool = True,
show_lineno: bool = False,
override: bool = False,
):
"""
Configure the "EvoTorch" Python logger to print to the console with default format.
The logger will be configured to print to all messages with level INFO or lower to stdout and all
messages with level WARNING or higher to stderr.
The default format is:
```
[2022-11-23 22:28:47] INFO <75935> evotorch: This is a log message
{asctime} {level} {process} {logger_name}: {message}
```
The format can be slightly customized by passing `show_process=False` to hide Process ID or `show_lineno=True` to
show the filename and line number of the log message instead of the Logger Name.
This function should be called before any other logging is performed, otherwise the default configuration will
not be applied. If the logger is already configured, this function will do nothing unless `override=True` is passed,
in which case the logger will be reconfigured.
Args:
logger_name: Name of the logger to configure.
logger_level: Level of the logger to configure.
show_process: Whether to show the process name in the log message.
show_lineno: Whether to show the filename with the line number in the log message or just the name of the logger.
override: Whether to override the logger configuration if it has already been configured.
"""
logger = logging.getLogger(logger_name)
if not override and logger.hasHandlers():
# warn user that the logger is already configured
logger.warning(
"The logger is already configured. "
"The default configuration will not be applied. "
"Call `set_default_logger_config` with `override=True` to override the current configuration."
)
return
elif override:
# remove all handlers
for handler in logger.handlers:
logger.removeHandler(handler)
logger.setLevel(logger_level)
logger.propagate = False
formatter = logging.Formatter(
"[{asctime}] "
+ "{levelname:<8s} "
+ ("<{process:5d}> " if show_process else "")
+ ("{filename}:{lineno}: " if show_lineno else "{name}: ")
+ "{message}",
datefmt="%Y-%m-%d %H:%M:%S",
style="{",
)
_stdout_handler = logging.StreamHandler(sys.stdout)
_stdout_handler.addFilter(lambda log_record: log_record.levelno < logging.WARNING)
_stdout_handler.setFormatter(formatter)
logger.addHandler(_stdout_handler)
_stderr_handler = logging.StreamHandler(sys.stderr)
_stderr_handler.addFilter(lambda log_record: log_record.levelno >= logging.WARNING)
_stderr_handler.setFormatter(formatter)
logger.addHandler(_stderr_handler)
split_workload(workload, num_actors)
¶
Split a workload among actors.
By "workload" what is meant is the total amount of a work, this amount being expressed by an integer. For example, if the "work" is the evaluation of a population, the "workload" would usually be the population size.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
workload |
int |
Total amount of work, as an integer. |
required |
num_actors |
int |
Number of actors (i.e. remote workers) among which the workload will be distributed. |
required |
Returns:
| Type | Description |
|---|---|
list |
A list of integers. The i-th item of the returned list expresses the suggested workload for the i-th actor. |
Source code in evotorch/tools/misc.py
def split_workload(workload: int, num_actors: int) -> list:
"""
Split a workload among actors.
By "workload" what is meant is the total amount of a work,
this amount being expressed by an integer.
For example, if the "work" is the evaluation of a population,
the "workload" would usually be the population size.
Args:
workload: Total amount of work, as an integer.
num_actors: Number of actors (i.e. remote workers) among
which the workload will be distributed.
Returns:
A list of integers. The i-th item of the returned list
expresses the suggested workload for the i-th actor.
"""
base_workload = workload // num_actors
extra_workload = workload % num_actors
result = [base_workload] * num_actors
for i in range(extra_workload):
result[i] += 1
return result
stdev_from_radius(radius, solution_length)
¶
Get elementwise standard deviation from a given radius.
Sometimes, for a distribution-based search algorithm, the user might
choose to configure the initial coverage area of the search distribution
not via standard deviation, but via a radius value, as was done in the
study of Toklu et al. (2020).
This function takes the desired radius value and the solution length of
the problem at hand, and returns the elementwise standard deviation value.
Let us name this returned standard deviation value as s.
When a new Gaussian distribution is constructed such that its initial
standard deviation is [s, s, s, ...] (the length of this vector being
equal to the solution length), this constructed distribution's radius
corresponds with the desired radius.
Here, the "radius" of a Gaussian distribution is defined as the norm
of the standard deviation vector. In the case of a standard normal
distribution, this radius formulation serves as a simplified approximation
to E[||Normal(0, I)||] (for which a closer approximation is used in
the study of Hansen & Ostermeier (2001)).
Reference:
Toklu, N.E., Liskowski, P., Srivastava, R.K. (2020).
ClipUp: A Simple and Powerful Optimizer
for Distribution-based Policy Evolution.
Parallel Problem Solving from Nature (PPSN 2020).
Nikolaus Hansen, Andreas Ostermeier (2001).
Completely Derandomized Self-Adaptation in Evolution Strategies.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
radius |
float |
The radius whose elementwise standard deviation counterpart will be returned. |
required |
solution_length |
int |
Length of a solution for the problem at hand. |
required |
Returns:
| Type | Description |
|---|---|
float |
An elementwise standard deviation value |
Source code in evotorch/tools/misc.py
def stdev_from_radius(radius: float, solution_length: int) -> float:
"""
Get elementwise standard deviation from a given radius.
Sometimes, for a distribution-based search algorithm, the user might
choose to configure the initial coverage area of the search distribution
not via standard deviation, but via a radius value, as was done in the
study of Toklu et al. (2020).
This function takes the desired radius value and the solution length of
the problem at hand, and returns the elementwise standard deviation value.
Let us name this returned standard deviation value as `s`.
When a new Gaussian distribution is constructed such that its initial
standard deviation is `[s, s, s, ...]` (the length of this vector being
equal to the solution length), this constructed distribution's radius
corresponds with the desired radius.
Here, the "radius" of a Gaussian distribution is defined as the norm
of the standard deviation vector. In the case of a standard normal
distribution, this radius formulation serves as a simplified approximation
to `E[||Normal(0, I)||]` (for which a closer approximation is used in
the study of Hansen & Ostermeier (2001)).
Reference:
Toklu, N.E., Liskowski, P., Srivastava, R.K. (2020).
ClipUp: A Simple and Powerful Optimizer
for Distribution-based Policy Evolution.
Parallel Problem Solving from Nature (PPSN 2020).
Nikolaus Hansen, Andreas Ostermeier (2001).
Completely Derandomized Self-Adaptation in Evolution Strategies.
Args:
radius: The radius whose elementwise standard deviation counterpart
will be returned.
solution_length: Length of a solution for the problem at hand.
Returns:
An elementwise standard deviation value `s`, such that a Gaussian
distribution constructed with the standard deviation `[s, s, s, ...]`
has the desired radius.
"""
radius = float(radius)
solution_length = int(solution_length)
return math.sqrt((radius**2) / solution_length)
storage_ptr(x)
¶
Get the pointer to the underlying storage of a tensor of an ObjectArray.
Calling storage_ptr(x) is equivalent to x.untyped_storage().data_ptr().
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Iterable |
A regular PyTorch tensor, or a ReadOnlyTensor, or an ObjectArray. |
required |
Returns:
| Type | Description |
|---|---|
int |
The address of the underlying storage. |
Source code in evotorch/tools/misc.py
def storage_ptr(x: Iterable) -> int:
"""
Get the pointer to the underlying storage of a tensor of an ObjectArray.
Calling `storage_ptr(x)` is equivalent to `x.untyped_storage().data_ptr()`.
Args:
x: A regular PyTorch tensor, or a ReadOnlyTensor, or an ObjectArray.
Returns:
The address of the underlying storage.
"""
return _storage_ptr(x)
to_numpy_dtype(dtype)
¶
Convert the given string or the given PyTorch dtype to a numpy dtype. If the argument is already a numpy dtype, then the argument is returned as it is.
Returns:
| Type | Description |
|---|---|
dtype |
The dtype, converted to a numpy dtype. |
Source code in evotorch/tools/misc.py
def to_numpy_dtype(dtype: DType) -> np.dtype:
"""
Convert the given string or the given PyTorch dtype to a numpy dtype.
If the argument is already a numpy dtype, then the argument is returned
as it is.
Returns:
The dtype, converted to a numpy dtype.
"""
if isinstance(dtype, torch.dtype):
return torch.tensor([], dtype=dtype).numpy().dtype
elif is_dtype_object(dtype):
return np.dtype(object)
elif isinstance(dtype, np.dtype):
return dtype
else:
return np.dtype(dtype)
to_stdev_init(*, solution_length, stdev_init=None, radius_init=None)
¶
Ask for both standard deviation and radius, return the standard deviation.
It is very common among the distribution-based search algorithms to ask for both standard deviation and for radius for initializing the coverage area of the search distribution. During their initialization phases, these algorithms must check which one the user provided (radius or standard deviation), and return the result as the standard deviation so that a Gaussian distribution can easily be constructed.
This function serves as a helper function for such search algorithms by performing these actions:
- If the user provided a standard deviation and not a radius, then this provided standard deviation is simply returned.
- If the user provided a radius and not a standard deviation, then this provided radius is converted to its standard deviation counterpart, and then returned.
- If both standard deviation and radius are missing, or they are both given at the same time, then an error is raised.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
solution_length |
int |
Length of a solution for the problem at hand. |
required |
stdev_init |
Union[float, Iterable[float], torch.Tensor] |
Standard deviation. If one wishes to provide a radius
instead, then |
None |
radius_init |
Union[float, Iterable[float], torch.Tensor] |
Radius. If one wishes to provide a standard deviation
instead, then |
None |
Returns:
| Type | Description |
|---|---|
Union[float, Iterable[float], torch.Tensor] |
The standard deviation for the search distribution to be constructed. |
Source code in evotorch/tools/misc.py
def to_stdev_init(
*,
solution_length: int,
stdev_init: Optional[RealOrVector] = None,
radius_init: Optional[RealOrVector] = None,
) -> RealOrVector:
"""
Ask for both standard deviation and radius, return the standard deviation.
It is very common among the distribution-based search algorithms to ask
for both standard deviation and for radius for initializing the coverage
area of the search distribution. During their initialization phases,
these algorithms must check which one the user provided (radius or
standard deviation), and return the result as the standard deviation
so that a Gaussian distribution can easily be constructed.
This function serves as a helper function for such search algorithms
by performing these actions:
- If the user provided a standard deviation and not a radius, then this
provided standard deviation is simply returned.
- If the user provided a radius and not a standard deviation, then this
provided radius is converted to its standard deviation counterpart,
and then returned.
- If both standard deviation and radius are missing, or they are both
given at the same time, then an error is raised.
Args:
solution_length: Length of a solution for the problem at hand.
stdev_init: Standard deviation. If one wishes to provide a radius
instead, then `stdev_init` is expected as None.
radius_init: Radius. If one wishes to provide a standard deviation
instead, then `radius_init` is expected as None.
Returns:
The standard deviation for the search distribution to be constructed.
"""
if (stdev_init is not None) and (radius_init is None):
return stdev_init
elif (stdev_init is None) and (radius_init is not None):
return stdev_from_radius(radius_init, solution_length)
elif (stdev_init is None) and (radius_init is None):
raise ValueError(
"Received both `stdev_init` and `radius_init` as None."
" Please provide a value either for `stdev_init` or for `radius_init`."
)
else:
raise ValueError(
"Found both `stdev_init` and `radius_init` with values other than None."
" Please provide a value either for `stdev_init` or for `radius_init`, but not for both."
)
to_torch_dtype(dtype)
¶
Convert the given string or the given numpy dtype to a PyTorch dtype. If the argument is already a PyTorch dtype, then the argument is returned as it is.
Returns:
| Type | Description |
|---|---|
dtype |
The dtype, converted to a PyTorch dtype. |
Source code in evotorch/tools/misc.py
def to_torch_dtype(dtype: DType) -> torch.dtype:
"""
Convert the given string or the given numpy dtype to a PyTorch dtype.
If the argument is already a PyTorch dtype, then the argument is returned
as it is.
Returns:
The dtype, converted to a PyTorch dtype.
"""
if isinstance(dtype, str) and hasattr(torch, dtype):
attrib_within_torch = getattr(torch, dtype)
else:
attrib_within_torch = None
if isinstance(attrib_within_torch, torch.dtype):
return attrib_within_torch
elif isinstance(dtype, torch.dtype):
return dtype
elif dtype is Any or dtype is object:
raise TypeError(f"Cannot make a numeric tensor with dtype {repr(dtype)}")
else:
return torch.from_numpy(np.array([], dtype=dtype)).dtype
objectarray
¶
This module contains the ObjectArray class, which is an array-like data structure with an interface similar to PyTorch tensors, but with an ability to store arbitrary type of data (not just numbers).
ObjectArray (Sequence, RecursivePrintable)
¶
An object container with an interface similar to PyTorch tensors.
It is strictly one-dimensional, and supports advanced indexing and slicing operations supported by PyTorch tensors.
An ObjectArray can store None values, strings, numbers, booleans,
lists, sets, dictionaries, PyTorch tensors, and numpy arrays.
When a container (such as a list, dictionary, set, is placed into an ObjectArray, an immutable clone of this container is first created, and then this newly created immutable clone gets stored within the ObjectArray. This behavior is to prevent accidental modification of the stored data.
When a numeric array (such as a PyTorch tensor or a numpy array with a
numeric dtype) is placed into an ObjectArray, the target ObjectArray
first checks if the numeric array is read-only. If the numeric array
is indeed read-only, then the array is put into the ObjectArray as it
is. If the array is not read-only, then a read-only clone of the
original numeric array is first created, and then this clone gets
stored by the ObjectArray. This behavior has the following implications:
(i) even when an ObjectArray is shared by multiple components of the
program, the risk of accidental modification of the stored data through
this shared ObjectArray is significantly reduced as the stored numeric
arrays are read-only;
(ii) although not recommended, one could still forcefully modify the
numeric arrays stored by an ObjectArray by explicitly casting them as
mutable arrays
(in the case of a numpy array, one could forcefully set the WRITEABLE
flag, and, in the case of a ReadOnlyTensor, one could forcefully cast it
as a regular PyTorch tensor);
(iii) if an already read-only array x is placed into an ObjectArray,
but x shares its memory with a mutable array y, then the contents
of the ObjectArray can be affected by modifying y.
The implication (ii) is demonstrated as follows:
objs = ObjectArray(1) # a single-element ObjectArray
# Place a numpy array into objs:
objs[0] = np.array([1, 2, 3], dtype=float)
# At this point, objs[0] is a read-only numpy array.
# objs[0] *= 2 # <- Not allowed
# Possible but NOT recommended:
objs.flags["WRITEABLE"] = True
objs[0] *= 2
The implication (iii) is demonstrated as follows:
objs = ObjectArray(1) # a single-element ObjectArray
# Make a new mutable numpy array
y = np.array([1, 2, 3], dtype=float)
# Make a read-only view to y:
x = y[:]
x.flags["WRITEABLE"] = False
# Place x into objs.
objs[0] = x
# At this point, objs[0] is a read-only numpy array.
# objs[0] *= 2 # <- Not allowed
# During the operation of setting its 0-th item, the ObjectArray
# `objs` did not clone `x` because `x` was already read-only.
# However, the contents of `x` could actually be modified because
# `x` shares its memory with the mutable array `y`.
# Possible but NOT recommended:
y *= 2 # This affects both x and objs!
When a numpy array of dtype object is placed into an ObjectArray, a read-only ObjectArray copy of the original array will first be created, and then, this newly created ObjectArray will be stored by the outer ObjectArray.
An ObjectArray itself has a read-only mode, so that, in addition to its stored data, the ObjectArray itself can be protected against undesired modifications.
An interesting feature of PyTorch: if one slices a tensor A and the result is a new tensor B, and if B is sharing storage memory with A, then A.untyped_storage().data_ptr() and B.untyped_storage().data_ptr() will return the same pointer. This means, one can compare the storage pointers of A and B and see whether or not the two are sharing memory. ObjectArray was designed to have this exact behavior, so that one can understand if two ObjectArray instances are sharing memory. Note that NumPy does NOT have such a behavior. In more details, a NumPy array C and a NumPy array D could report different pointers even when D was created via a basic slicing operation on C.
Source code in evotorch/tools/objectarray.py
class ObjectArray(Sequence, RecursivePrintable):
"""
An object container with an interface similar to PyTorch tensors.
It is strictly one-dimensional, and supports advanced indexing and
slicing operations supported by PyTorch tensors.
An ObjectArray can store `None` values, strings, numbers, booleans,
lists, sets, dictionaries, PyTorch tensors, and numpy arrays.
When a container (such as a list, dictionary, set, is placed into an
ObjectArray, an immutable clone of this container is first created, and
then this newly created immutable clone gets stored within the
ObjectArray. This behavior is to prevent accidental modification of the
stored data.
When a numeric array (such as a PyTorch tensor or a numpy array with a
numeric dtype) is placed into an ObjectArray, the target ObjectArray
first checks if the numeric array is read-only. If the numeric array
is indeed read-only, then the array is put into the ObjectArray as it
is. If the array is not read-only, then a read-only clone of the
original numeric array is first created, and then this clone gets
stored by the ObjectArray. This behavior has the following implications:
(i) even when an ObjectArray is shared by multiple components of the
program, the risk of accidental modification of the stored data through
this shared ObjectArray is significantly reduced as the stored numeric
arrays are read-only;
(ii) although not recommended, one could still forcefully modify the
numeric arrays stored by an ObjectArray by explicitly casting them as
mutable arrays
(in the case of a numpy array, one could forcefully set the WRITEABLE
flag, and, in the case of a ReadOnlyTensor, one could forcefully cast it
as a regular PyTorch tensor);
(iii) if an already read-only array `x` is placed into an ObjectArray,
but `x` shares its memory with a mutable array `y`, then the contents
of the ObjectArray can be affected by modifying `y`.
The implication (ii) is demonstrated as follows:
```python
objs = ObjectArray(1) # a single-element ObjectArray
# Place a numpy array into objs:
objs[0] = np.array([1, 2, 3], dtype=float)
# At this point, objs[0] is a read-only numpy array.
# objs[0] *= 2 # <- Not allowed
# Possible but NOT recommended:
objs.flags["WRITEABLE"] = True
objs[0] *= 2
```
The implication (iii) is demonstrated as follows:
```python
objs = ObjectArray(1) # a single-element ObjectArray
# Make a new mutable numpy array
y = np.array([1, 2, 3], dtype=float)
# Make a read-only view to y:
x = y[:]
x.flags["WRITEABLE"] = False
# Place x into objs.
objs[0] = x
# At this point, objs[0] is a read-only numpy array.
# objs[0] *= 2 # <- Not allowed
# During the operation of setting its 0-th item, the ObjectArray
# `objs` did not clone `x` because `x` was already read-only.
# However, the contents of `x` could actually be modified because
# `x` shares its memory with the mutable array `y`.
# Possible but NOT recommended:
y *= 2 # This affects both x and objs!
```
When a numpy array of dtype object is placed into an ObjectArray,
a read-only ObjectArray copy of the original array will first be
created, and then, this newly created ObjectArray will be stored
by the outer ObjectArray.
An ObjectArray itself has a read-only mode, so that, in addition to its
stored data, the ObjectArray itself can be protected against undesired
modifications.
An interesting feature of PyTorch: if one slices a tensor A and the
result is a new tensor B, and if B is sharing storage memory with A,
then A.untyped_storage().data_ptr() and B.untyped_storage().data_ptr()
will return the same pointer. This means, one can compare the storage
pointers of A and B and see whether or not the two are sharing memory.
ObjectArray was designed to have this exact behavior, so that one
can understand if two ObjectArray instances are sharing memory.
Note that NumPy does NOT have such a behavior. In more details,
a NumPy array C and a NumPy array D could report different pointers
even when D was created via a basic slicing operation on C.
"""
def __init__(
self,
size: Optional[Size] = None,
*,
slice_of: Optional[tuple] = None,
):
"""
`__init__(...)`: Instantiate a new ObjectArray.
Args:
size: Length of the ObjectArray. If this argument is present and
is an integer `n`, then the resulting ObjectArray will be
of length `n`, and will be filled with `None` values.
This argument cannot be used together with the keyword
argument `slice_of`.
slice_of: Optionally a tuple in the form
`(original_object_tensor, slice_info)`.
When this argument is present, then the resulting ObjectArray
will be a slice of the given `original_object_tensor` (which
is expected as an ObjectArray instance). `slice_info` is
either a `slice` instance, or a sequence of integers.
The resulting ObjectArray might be a view of
`original_object_tensor` (i.e. it might share its memory with
`original_object_tensor`).
This keyword argument cannot be used together with the
argument `size`.
"""
if size is not None and slice_of is not None:
raise ValueError("Expected either `size` argument or `slice_of` argument, but got both.")
elif size is None and slice_of is None:
raise ValueError("Expected either `size` argument or `slice_of` argument, but got none.")
elif size is not None:
if not is_sequence(size):
length = size
elif isinstance(size, (np.ndarray, torch.Tensor)) and (size.ndim > 1):
raise ValueError(f"Invalid size: {size}")
else:
[length] = size
length = int(length)
self._indices = torch.arange(length, dtype=torch.int64)
self._objects = [None] * length
elif slice_of is not None:
source: ObjectArray
source, slicing = slice_of
if not isinstance(source, ObjectArray):
raise TypeError(
f"`slice_of`: The first element was expected as an ObjectArray."
f" But it is of type {repr(type(source))}"
)
if isinstance(slicing, tuple) or is_integer(slicing):
raise TypeError(f"Invalid slice: {slicing}")
self._indices = source._indices[slicing]
self._objects = source._objects
if storage_ptr(self._indices) != storage_ptr(source._indices):
self._objects = clone(self._objects)
self._device = torch.device("cpu")
self._read_only = False
@property
def shape(self) -> Size:
"""Shape of the ObjectArray, as a PyTorch Size tuple."""
return self._indices.shape
def size(self) -> Size:
"""
Get the size of the ObjectArray, as a PyTorch Size tuple.
Returns:
The size (i.e. the shape) of the ObjectArray.
"""
return self._indices.size()
@property
def ndim(self) -> int:
"""
Number of dimensions handled by the ObjectArray.
This is equivalent to getting the length of the size tuple.
"""
return self._indices.ndim
def dim(self) -> int:
"""
Get the number of dimensions handled by the ObjectArray.
This is equivalent to getting the length of the size tuple.
Returns:
The number of dimensions, as an integer.
"""
return self._indices.dim()
def numel(self) -> int:
"""
Number of elements stored by the ObjectArray.
Returns:
The number of elements, as an integer.
"""
return self._indices.numel()
def repeat(self, *sizes) -> "ObjectArray":
"""
Repeat the contents of this ObjectArray.
For example, if we have an ObjectArray `objs` which stores
`["hello", "world"]`, the following line:
objs.repeat(3)
will result in an ObjectArray which stores:
`["hello", "world", "hello", "world", "hello", "world"]`
Args:
sizes: Although this argument is named `sizes` to be compatible
with PyTorch, what is expected here is a single positional
argument, as a single integer, or as a single-element
tuple.
The given integer (which can be the argument itself, or
the integer within the given single-element tuple),
specifies how many times the stored sequence will be
repeated.
Returns:
A new ObjectArray which repeats the original one's values
"""
if len(sizes) != 1:
type_name = type(self).__name__
raise ValueError(
f"The `repeat(...)` method of {type_name} expects exactly one positional argument."
f" This is because {type_name} supports only 1-dimensional storage."
f" The received positional arguments are: {sizes}."
)
if isinstance(sizes, tuple):
if len(sizes) == 1:
sizes = sizes[0]
else:
type_name = type(self).__name__
raise ValueError(
f"The `repeat(...)` method of {type_name} can accept a size tuple with only one element."
f" This is because {type_name} supports only 1-dimensional storage."
f" The received size tuple is: {sizes}."
)
num_repetitions = int(sizes[0])
self_length = len(self)
result = ObjectArray(num_repetitions * self_length)
source_index = 0
for result_index in range(len(result)):
result[result_index] = self[source_index]
source_index = (source_index + 1) % self_length
return result
@property
def device(self) -> Device:
"""
The device which stores the elements of the ObjectArray.
In the case of ObjectArray, this property always returns
the CPU device.
Returns:
The CPU device, as a torch.device object.
"""
return self._device
@property
def dtype(self) -> DType:
"""
The dtype of the elements stored by the ObjectArray.
In the case of ObjectArray, the dtype is always `object`.
"""
return object
def __getitem__(self, i: Any) -> Any:
if is_integer(i):
index = int(self._indices[i])
return self._objects[index]
else:
indices = self._indices[i]
same_ptr = storage_ptr(indices) == storage_ptr(self._indices)
result = ObjectArray(len(indices))
if same_ptr:
result._indices[:] = indices
result._objects = self._objects
else:
result._objects = []
for index in indices:
result._objects.append(self._objects[int(index)])
result._read_only = self._read_only
return result
def __setitem__(self, i: Any, x: Any):
self.set_item(i, x)
def set_item(self, i: Any, x: Any, *, memo: Optional[dict] = None):
"""
Set the i-th item of the ObjectArray as x.
Args:
i: An index or a slice.
x: The object that will be put into the ObjectArray.
memo: Optionally a dictionary which maps from the ids of the
already placed objects to their clones within ObjectArray.
In most scenarios, when this method is called from outside,
this can be left as None.
"""
from .immutable import as_immutable
if memo is None:
memo = {}
memo[id(self)] = self
if self._read_only:
raise ValueError("This ObjectArray is read-only, therefore, modification is not allowed.")
if is_integer(i):
index = int(self._indices[i])
self._objects[index] = as_immutable(x, memo=memo)
else:
indices = self._indices[i]
if not isinstance(x, Iterable):
raise TypeError(f"Expected an iterable, but got {repr(x)}")
if indices.ndim != 1:
raise ValueError(
"Received indices that would change the dimensionality of the ObjectArray."
" However, an ObjectArray can only be 1-dimensional."
)
slice_refers_to_whole_array = (len(indices) == len(self._indices)) and torch.all(indices == self._indices)
if slice_refers_to_whole_array:
memo[id(x)] = self
if not hasattr(x, "__len__"):
x = list(x)
if len(x) != len(indices):
raise TypeError(
f"The slicing operation refers to {len(indices)} elements."
f" However, the given objects sequence has {len(x)} elements."
)
for q, obj in enumerate(x):
index = int(indices[q])
self._objects[index] = as_immutable(obj, memo=memo)
def __len__(self) -> int:
return len(self._indices)
def __iter__(self):
for i in range(len(self)):
yield self[i]
def clone(self, *, preserve_read_only: bool = False, memo: Optional[dict] = None) -> Iterable:
"""
Get a deep copy of the ObjectArray.
Args:
preserve_read_only: Whether or not to preserve the read-only
attribute. Note that the default value is False, which
means that the newly made clone will NOT be read-only
even if the original ObjectArray is.
memo: Optionally a dictionary which maps from the ids of the
already cloned objects to their clones.
In most scenarios, when this method is called from outside,
this can be left as None.
Returns:
The clone of the original ObjectArray.
"""
from .cloning import deep_clone
if memo is None:
memo = {}
self_id = id(self)
if self_id in memo:
return memo[self_id]
if not preserve_read_only:
return self.numpy(memo=memo)
else:
result = ObjectArray(len(self))
memo[self_id] = result
for i, item in enumerate(self):
result[i] = deep_clone(item, otherwise_deepcopy=True, memo=memo)
return result
def __copy__(self) -> "ObjectArray":
return self.clone(preserve_read_only=True)
def __deepcopy__(self, memo: Optional[dict]) -> "ObjectArray":
if memo is None:
memo = {}
return self.clone(preserve_read_only=True, memo=memo)
def __setstate__(self, state: dict):
self.__dict__.update(state)
# After pickling and unpickling, numpy arrays become mutable.
# Since we are dealing with immutable containers here, we need to forcefully make all numpy arrays read-only.
for v in self:
if isinstance(v, np.ndarray):
v.flags["WRITEABLE"] = False
# def __getstate__(self) -> dict:
# from .cloning import deep_clone
# self_id = id(self)
# memo = {self_id: self}
# cloned_dict = deep_clone(self.__dict__, otherwise_deepcopy=True, memo=memo)
# return cloned_dict
def get_read_only_view(self) -> "ObjectArray":
"""
Get a read-only view of this ObjectArray.
"""
result = self[:]
result._read_only = True
return result
@property
def is_read_only(self) -> bool:
"""
True if this ObjectArray is read-only; False otherwise.
"""
return self._read_only
def storage(self) -> ObjectArrayStorage:
return ObjectArrayStorage(self)
def untyped_storage(self) -> ObjectArrayStorage:
return ObjectArrayStorage(self)
def numpy(self, *, memo: Optional[dict] = None) -> np.ndarray:
"""
Convert this ObjectArray to a numpy array.
The resulting numpy array will have its dtype set as `object`.
This new array itself and its contents will be mutable (those
mutable objects being the copies of their immutable sources).
Returns:
The numpy counterpart of this ObjectArray.
"""
from .immutable import mutable_copy
if memo is None:
memo = {}
n = len(self)
result = np.empty(n, dtype=object)
memo[id(self)] = result
for i, item in enumerate(self):
result[i] = mutable_copy(item, memo=memo)
return result
@staticmethod
def from_numpy(ndarray: np.ndarray) -> "ObjectArray":
"""
Convert a numpy array of dtype `object` to an `ObjectArray`.
Args:
ndarray: The numpy array that will be converted to `ObjectArray`.
Returns:
The ObjectArray counterpart of the given numpy array.
"""
if isinstance(ndarray, np.ndarray):
if ndarray.dtype == np.dtype(object):
n = len(ndarray)
result = ObjectArray(n)
for i, element in enumerate(ndarray):
result[i] = element
return result
else:
raise ValueError(
f"The dtype of the given array was expected as `object`."
f" However, the dtype was encountered as {ndarray.dtype}."
)
else:
raise TypeError(f"Expected a `numpy.ndarray` instance, but received an object of type {type(ndarray)}.")
device: Union[str, torch.device]
property
readonly
¶
The device which stores the elements of the ObjectArray. In the case of ObjectArray, this property always returns the CPU device.
Returns:
| Type | Description |
|---|---|
Union[str, torch.device] |
The CPU device, as a torch.device object. |
dtype: Union[str, torch.dtype, numpy.dtype, Type]
property
readonly
¶
The dtype of the elements stored by the ObjectArray.
In the case of ObjectArray, the dtype is always object.
is_read_only: bool
property
readonly
¶
True if this ObjectArray is read-only; False otherwise.
ndim: int
property
readonly
¶
Number of dimensions handled by the ObjectArray. This is equivalent to getting the length of the size tuple.
shape: Union[int, torch.Size]
property
readonly
¶
Shape of the ObjectArray, as a PyTorch Size tuple.
__init__(self, size=None, *, slice_of=None)
special
¶
__init__(...): Instantiate a new ObjectArray.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Length of the ObjectArray. If this argument is present and
is an integer |
None |
slice_of |
Optional[tuple] |
Optionally a tuple in the form
|
None |
Source code in evotorch/tools/objectarray.py
def __init__(
self,
size: Optional[Size] = None,
*,
slice_of: Optional[tuple] = None,
):
"""
`__init__(...)`: Instantiate a new ObjectArray.
Args:
size: Length of the ObjectArray. If this argument is present and
is an integer `n`, then the resulting ObjectArray will be
of length `n`, and will be filled with `None` values.
This argument cannot be used together with the keyword
argument `slice_of`.
slice_of: Optionally a tuple in the form
`(original_object_tensor, slice_info)`.
When this argument is present, then the resulting ObjectArray
will be a slice of the given `original_object_tensor` (which
is expected as an ObjectArray instance). `slice_info` is
either a `slice` instance, or a sequence of integers.
The resulting ObjectArray might be a view of
`original_object_tensor` (i.e. it might share its memory with
`original_object_tensor`).
This keyword argument cannot be used together with the
argument `size`.
"""
if size is not None and slice_of is not None:
raise ValueError("Expected either `size` argument or `slice_of` argument, but got both.")
elif size is None and slice_of is None:
raise ValueError("Expected either `size` argument or `slice_of` argument, but got none.")
elif size is not None:
if not is_sequence(size):
length = size
elif isinstance(size, (np.ndarray, torch.Tensor)) and (size.ndim > 1):
raise ValueError(f"Invalid size: {size}")
else:
[length] = size
length = int(length)
self._indices = torch.arange(length, dtype=torch.int64)
self._objects = [None] * length
elif slice_of is not None:
source: ObjectArray
source, slicing = slice_of
if not isinstance(source, ObjectArray):
raise TypeError(
f"`slice_of`: The first element was expected as an ObjectArray."
f" But it is of type {repr(type(source))}"
)
if isinstance(slicing, tuple) or is_integer(slicing):
raise TypeError(f"Invalid slice: {slicing}")
self._indices = source._indices[slicing]
self._objects = source._objects
if storage_ptr(self._indices) != storage_ptr(source._indices):
self._objects = clone(self._objects)
self._device = torch.device("cpu")
self._read_only = False
clone(self, *, preserve_read_only=False, memo=None)
¶
Get a deep copy of the ObjectArray.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
preserve_read_only |
bool |
Whether or not to preserve the read-only attribute. Note that the default value is False, which means that the newly made clone will NOT be read-only even if the original ObjectArray is. |
False |
memo |
Optional[dict] |
Optionally a dictionary which maps from the ids of the already cloned objects to their clones. In most scenarios, when this method is called from outside, this can be left as None. |
None |
Returns:
| Type | Description |
|---|---|
Iterable |
The clone of the original ObjectArray. |
Source code in evotorch/tools/objectarray.py
def clone(self, *, preserve_read_only: bool = False, memo: Optional[dict] = None) -> Iterable:
"""
Get a deep copy of the ObjectArray.
Args:
preserve_read_only: Whether or not to preserve the read-only
attribute. Note that the default value is False, which
means that the newly made clone will NOT be read-only
even if the original ObjectArray is.
memo: Optionally a dictionary which maps from the ids of the
already cloned objects to their clones.
In most scenarios, when this method is called from outside,
this can be left as None.
Returns:
The clone of the original ObjectArray.
"""
from .cloning import deep_clone
if memo is None:
memo = {}
self_id = id(self)
if self_id in memo:
return memo[self_id]
if not preserve_read_only:
return self.numpy(memo=memo)
else:
result = ObjectArray(len(self))
memo[self_id] = result
for i, item in enumerate(self):
result[i] = deep_clone(item, otherwise_deepcopy=True, memo=memo)
return result
dim(self)
¶
Get the number of dimensions handled by the ObjectArray. This is equivalent to getting the length of the size tuple.
Returns:
| Type | Description |
|---|---|
int |
The number of dimensions, as an integer. |
from_numpy(ndarray)
staticmethod
¶
Convert a numpy array of dtype object to an ObjectArray.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ndarray |
ndarray |
The numpy array that will be converted to |
required |
Returns:
| Type | Description |
|---|---|
ObjectArray |
The ObjectArray counterpart of the given numpy array. |
Source code in evotorch/tools/objectarray.py
@staticmethod
def from_numpy(ndarray: np.ndarray) -> "ObjectArray":
"""
Convert a numpy array of dtype `object` to an `ObjectArray`.
Args:
ndarray: The numpy array that will be converted to `ObjectArray`.
Returns:
The ObjectArray counterpart of the given numpy array.
"""
if isinstance(ndarray, np.ndarray):
if ndarray.dtype == np.dtype(object):
n = len(ndarray)
result = ObjectArray(n)
for i, element in enumerate(ndarray):
result[i] = element
return result
else:
raise ValueError(
f"The dtype of the given array was expected as `object`."
f" However, the dtype was encountered as {ndarray.dtype}."
)
else:
raise TypeError(f"Expected a `numpy.ndarray` instance, but received an object of type {type(ndarray)}.")
get_read_only_view(self)
¶
numel(self)
¶
Number of elements stored by the ObjectArray.
Returns:
| Type | Description |
|---|---|
int |
The number of elements, as an integer. |
numpy(self, *, memo=None)
¶
Convert this ObjectArray to a numpy array.
The resulting numpy array will have its dtype set as object.
This new array itself and its contents will be mutable (those
mutable objects being the copies of their immutable sources).
Returns:
| Type | Description |
|---|---|
ndarray |
The numpy counterpart of this ObjectArray. |
Source code in evotorch/tools/objectarray.py
def numpy(self, *, memo: Optional[dict] = None) -> np.ndarray:
"""
Convert this ObjectArray to a numpy array.
The resulting numpy array will have its dtype set as `object`.
This new array itself and its contents will be mutable (those
mutable objects being the copies of their immutable sources).
Returns:
The numpy counterpart of this ObjectArray.
"""
from .immutable import mutable_copy
if memo is None:
memo = {}
n = len(self)
result = np.empty(n, dtype=object)
memo[id(self)] = result
for i, item in enumerate(self):
result[i] = mutable_copy(item, memo=memo)
return result
repeat(self, *sizes)
¶
Repeat the contents of this ObjectArray.
For example, if we have an ObjectArray objs which stores
["hello", "world"], the following line:
objs.repeat(3)
will result in an ObjectArray which stores:
`["hello", "world", "hello", "world", "hello", "world"]`
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sizes |
Although this argument is named |
() |
Returns:
| Type | Description |
|---|---|
ObjectArray |
A new ObjectArray which repeats the original one's values |
Source code in evotorch/tools/objectarray.py
def repeat(self, *sizes) -> "ObjectArray":
"""
Repeat the contents of this ObjectArray.
For example, if we have an ObjectArray `objs` which stores
`["hello", "world"]`, the following line:
objs.repeat(3)
will result in an ObjectArray which stores:
`["hello", "world", "hello", "world", "hello", "world"]`
Args:
sizes: Although this argument is named `sizes` to be compatible
with PyTorch, what is expected here is a single positional
argument, as a single integer, or as a single-element
tuple.
The given integer (which can be the argument itself, or
the integer within the given single-element tuple),
specifies how many times the stored sequence will be
repeated.
Returns:
A new ObjectArray which repeats the original one's values
"""
if len(sizes) != 1:
type_name = type(self).__name__
raise ValueError(
f"The `repeat(...)` method of {type_name} expects exactly one positional argument."
f" This is because {type_name} supports only 1-dimensional storage."
f" The received positional arguments are: {sizes}."
)
if isinstance(sizes, tuple):
if len(sizes) == 1:
sizes = sizes[0]
else:
type_name = type(self).__name__
raise ValueError(
f"The `repeat(...)` method of {type_name} can accept a size tuple with only one element."
f" This is because {type_name} supports only 1-dimensional storage."
f" The received size tuple is: {sizes}."
)
num_repetitions = int(sizes[0])
self_length = len(self)
result = ObjectArray(num_repetitions * self_length)
source_index = 0
for result_index in range(len(result)):
result[result_index] = self[source_index]
source_index = (source_index + 1) % self_length
return result
set_item(self, i, x, *, memo=None)
¶
Set the i-th item of the ObjectArray as x.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
i |
Any |
An index or a slice. |
required |
x |
Any |
The object that will be put into the ObjectArray. |
required |
memo |
Optional[dict] |
Optionally a dictionary which maps from the ids of the already placed objects to their clones within ObjectArray. In most scenarios, when this method is called from outside, this can be left as None. |
None |
Source code in evotorch/tools/objectarray.py
def set_item(self, i: Any, x: Any, *, memo: Optional[dict] = None):
"""
Set the i-th item of the ObjectArray as x.
Args:
i: An index or a slice.
x: The object that will be put into the ObjectArray.
memo: Optionally a dictionary which maps from the ids of the
already placed objects to their clones within ObjectArray.
In most scenarios, when this method is called from outside,
this can be left as None.
"""
from .immutable import as_immutable
if memo is None:
memo = {}
memo[id(self)] = self
if self._read_only:
raise ValueError("This ObjectArray is read-only, therefore, modification is not allowed.")
if is_integer(i):
index = int(self._indices[i])
self._objects[index] = as_immutable(x, memo=memo)
else:
indices = self._indices[i]
if not isinstance(x, Iterable):
raise TypeError(f"Expected an iterable, but got {repr(x)}")
if indices.ndim != 1:
raise ValueError(
"Received indices that would change the dimensionality of the ObjectArray."
" However, an ObjectArray can only be 1-dimensional."
)
slice_refers_to_whole_array = (len(indices) == len(self._indices)) and torch.all(indices == self._indices)
if slice_refers_to_whole_array:
memo[id(x)] = self
if not hasattr(x, "__len__"):
x = list(x)
if len(x) != len(indices):
raise TypeError(
f"The slicing operation refers to {len(indices)} elements."
f" However, the given objects sequence has {len(x)} elements."
)
for q, obj in enumerate(x):
index = int(indices[q])
self._objects[index] = as_immutable(obj, memo=memo)
size(self)
¶
Get the size of the ObjectArray, as a PyTorch Size tuple.
Returns:
| Type | Description |
|---|---|
Union[int, torch.Size] |
The size (i.e. the shape) of the ObjectArray. |
ranking
¶
This module contains ranking functions which work with PyTorch tensors.
centered(fitnesses, *, higher_is_better=True)
¶
Apply linearly spaced 0-centered ranking on a PyTorch tensor. The lowest weight is -0.5, and the highest weight is 0.5. This is the same ranking method that was used in:
Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, Ilya Sutskever (2017).
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fitnesses |
Tensor |
A PyTorch tensor which contains real numbers which we want to rank. |
required |
higher_is_better |
bool |
Whether or not the higher values will be assigned higher ranks. Changing this to False means that lower values are interpreted as better, and therefore lower values will have higher ranks. |
True |
Returns:
| Type | Description |
|---|---|
Tensor |
The ranks, in the same device, with the same dtype with the original tensor. |
Source code in evotorch/tools/ranking.py
def centered(fitnesses: torch.Tensor, *, higher_is_better: bool = True) -> torch.Tensor:
"""
Apply linearly spaced 0-centered ranking on a PyTorch tensor.
The lowest weight is -0.5, and the highest weight is 0.5.
This is the same ranking method that was used in:
Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, Ilya Sutskever (2017).
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
Args:
fitnesses: A PyTorch tensor which contains real numbers which we want
to rank.
higher_is_better: Whether or not the higher values will be assigned
higher ranks. Changing this to False means that lower values
are interpreted as better, and therefore lower values will have
higher ranks.
Returns:
The ranks, in the same device, with the same dtype with the original
tensor.
"""
device = fitnesses.device
dtype = fitnesses.dtype
with torch.no_grad():
x = fitnesses.reshape(-1)
n = len(x)
indices = x.argsort(descending=(not higher_is_better))
weights = (torch.arange(n, dtype=dtype, device=device) / (n - 1)) - 0.5
ranks = torch.empty_like(x)
ranks[indices] = weights
return ranks.reshape(*(fitnesses.shape))
linear(fitnesses, *, higher_is_better=True)
¶
Apply linearly spaced ranking on a PyTorch tensor. The lowest weight is 0, and the highest weight is 1.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fitnesses |
Tensor |
A PyTorch tensor which contains real numbers which we want to rank. |
required |
higher_is_better |
bool |
Whether or not the higher values will be assigned higher ranks. Changing this to False means that lower values are interpreted as better, and therefore lower values will have higher ranks. |
True |
Returns:
| Type | Description |
|---|---|
Tensor |
The ranks, in the same device, with the same dtype with the original tensor. |
Source code in evotorch/tools/ranking.py
def linear(fitnesses: torch.Tensor, *, higher_is_better: bool = True) -> torch.Tensor:
"""
Apply linearly spaced ranking on a PyTorch tensor.
The lowest weight is 0, and the highest weight is 1.
Args:
fitnesses: A PyTorch tensor which contains real numbers which we want
to rank.
higher_is_better: Whether or not the higher values will be assigned
higher ranks. Changing this to False means that lower values
are interpreted as better, and therefore lower values will have
higher ranks.
Returns:
The ranks, in the same device, with the same dtype with the original
tensor.
"""
device = fitnesses.device
dtype = fitnesses.dtype
with torch.no_grad():
x = fitnesses.reshape(-1)
n = len(x)
indices = x.argsort(descending=(not higher_is_better))
weights = torch.arange(n, dtype=dtype, device=device) / (n - 1)
ranks = torch.empty_like(x)
ranks[indices] = weights
return ranks.reshape(*(fitnesses.shape))
nes(fitnesses, *, higher_is_better=True)
¶
Apply the ranking mechanism proposed in:
Wierstra, D., Schaul, T., Glasmachers, T., Sun, Y., Peters, J., & Schmidhuber, J. (2014).
Natural evolution strategies. The Journal of Machine Learning Research, 15(1), 949-980.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fitnesses |
Tensor |
A PyTorch tensor which contains real numbers which we want to rank. |
required |
higher_is_better |
bool |
Whether or not the higher values will be assigned higher ranks. Changing this to False means that lower values are interpreted as better, and therefore lower values will have higher ranks. |
True |
Returns:
| Type | Description |
|---|---|
Tensor |
The ranks, in the same device, with the same dtype with the original tensor. |
Source code in evotorch/tools/ranking.py
def nes(fitnesses: torch.Tensor, *, higher_is_better: bool = True) -> torch.Tensor:
"""
Apply the ranking mechanism proposed in:
Wierstra, D., Schaul, T., Glasmachers, T., Sun, Y., Peters, J., & Schmidhuber, J. (2014).
Natural evolution strategies. The Journal of Machine Learning Research, 15(1), 949-980.
Args:
fitnesses: A PyTorch tensor which contains real numbers which we want
to rank.
higher_is_better: Whether or not the higher values will be assigned
higher ranks. Changing this to False means that lower values
are interpreted as better, and therefore lower values will have
higher ranks.
Returns:
The ranks, in the same device, with the same dtype with the original
tensor.
"""
device = fitnesses.device
dtype = fitnesses.dtype
with torch.no_grad():
x = fitnesses.reshape(-1)
n = len(x)
incr_indices = torch.arange(n, dtype=dtype, device=device)
N = torch.tensor(n, dtype=dtype, device=device)
weights = torch.max(
torch.tensor(0, dtype=dtype, device=device), torch.log((N / 2.0) + 1.0) - torch.log(N - incr_indices)
)
indices = torch.argsort(x, descending=(not higher_is_better))
ranks = torch.empty(n, dtype=indices.dtype, device=device)
ranks[indices] = torch.arange(n, dtype=indices.dtype, device=device)
utils = weights[ranks]
utils /= torch.sum(utils)
utils -= 1 / N
return utils.reshape(*(fitnesses.shape))
normalized(fitnesses, *, higher_is_better=True)
¶
Normalize the fitnesses and return the result as ranks.
The normalization is done in such a way that the mean becomes 0.0 and the standard deviation becomes 1.0.
According to the value of higher_is_better, it will be ensured that
better solutions will have numerically higher rank.
In more details, if higher_is_better is set as False, then the
fitnesses will be multiplied by -1.0 in addition to being subject
to normalization.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fitnesses |
Tensor |
A PyTorch tensor which contains real numbers which we want to rank. |
required |
higher_is_better |
bool |
Whether or not the higher values will be assigned higher ranks. Changing this to False means that lower values are interpreted as better, and therefore lower values will have higher ranks. |
True |
Returns:
| Type | Description |
|---|---|
Tensor |
The ranks, in the same device, with the same dtype with the original tensor. |
Source code in evotorch/tools/ranking.py
def normalized(fitnesses: torch.Tensor, *, higher_is_better: bool = True) -> torch.Tensor:
"""
Normalize the fitnesses and return the result as ranks.
The normalization is done in such a way that the mean becomes 0.0 and
the standard deviation becomes 1.0.
According to the value of `higher_is_better`, it will be ensured that
better solutions will have numerically higher rank.
In more details, if `higher_is_better` is set as False, then the
fitnesses will be multiplied by -1.0 in addition to being subject
to normalization.
Args:
fitnesses: A PyTorch tensor which contains real numbers which we want
to rank.
higher_is_better: Whether or not the higher values will be assigned
higher ranks. Changing this to False means that lower values
are interpreted as better, and therefore lower values will have
higher ranks.
Returns:
The ranks, in the same device, with the same dtype with the original
tensor.
"""
if not higher_is_better:
fitnesses = -fitnesses
fitness_mean = torch.mean(fitnesses)
fitness_stdev = torch.std(fitnesses)
fitnesses = fitnesses - fitness_mean
fitnesses = fitnesses / fitness_stdev
return fitnesses
rank(fitnesses, ranking_method, *, higher_is_better)
¶
Get the ranks of the given sequence of numbers.
Better solutions will have numerically higher ranks.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fitnesses |
Iterable[float] |
A sequence of numbers to be ranked. |
required |
ranking_method |
str |
The ranking method to be used.
Can be "centered", which means 0-centered linear ranking
from -0.5 to 0.5.
Can be "linear", which means a linear ranking from 0 to 1.
Can be "nes", which means the ranking method used by
Natural Evolution Strategies.
Can be "normalized", which means that the ranks will be
the normalized counterparts of the fitnesses.
Can be "raw", which means that the fitnesses themselves
(or, if |
required |
higher_is_better |
bool |
Whether or not the higher values will be assigned higher ranks. Changing this to False means that lower values are interpreted as better, and therefore lower values will have higher ranks. |
required |
Source code in evotorch/tools/ranking.py
def rank(fitnesses: Iterable[float], ranking_method: str, *, higher_is_better: bool):
"""
Get the ranks of the given sequence of numbers.
Better solutions will have numerically higher ranks.
Args:
fitnesses: A sequence of numbers to be ranked.
ranking_method: The ranking method to be used.
Can be "centered", which means 0-centered linear ranking
from -0.5 to 0.5.
Can be "linear", which means a linear ranking from 0 to 1.
Can be "nes", which means the ranking method used by
Natural Evolution Strategies.
Can be "normalized", which means that the ranks will be
the normalized counterparts of the fitnesses.
Can be "raw", which means that the fitnesses themselves
(or, if `higher_is_better` is False, their inverted
counterparts, inversion meaning the operation of
multiplying by -1 in this context) will be the ranks.
higher_is_better: Whether or not the higher values will be assigned
higher ranks. Changing this to False means that lower values
are interpreted as better, and therefore lower values will have
higher ranks.
"""
fitnesses = torch.as_tensor(fitnesses)
rank_func = rankers[ranking_method]
return rank_func(fitnesses, higher_is_better=higher_is_better)
raw(fitnesses, *, higher_is_better=True)
¶
Return the fitnesses themselves as ranks.
If higher_is_better is given as False, then the fitnesses will first
be multiplied by -1 and then the result will be returned as ranks.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fitnesses |
Tensor |
A PyTorch tensor which contains real numbers which we want to rank. |
required |
higher_is_better |
bool |
Whether or not the higher values will be assigned higher ranks. Changing this to False means that lower values are interpreted as better, and therefore lower values will have higher ranks. |
True |
Returns:
| Type | Description |
|---|---|
Tensor |
The ranks, in the same device, with the same dtype with the original tensor. |
Source code in evotorch/tools/ranking.py
def raw(fitnesses: torch.Tensor, *, higher_is_better: bool = True) -> torch.Tensor:
"""
Return the fitnesses themselves as ranks.
If `higher_is_better` is given as False, then the fitnesses will first
be multiplied by -1 and then the result will be returned as ranks.
Args:
fitnesses: A PyTorch tensor which contains real numbers which we want
to rank.
higher_is_better: Whether or not the higher values will be assigned
higher ranks. Changing this to False means that lower values
are interpreted as better, and therefore lower values will have
higher ranks.
Returns:
The ranks, in the same device, with the same dtype with the original
tensor.
"""
if not higher_is_better:
fitnesses = -fitnesses
return fitnesses
readonlytensor
¶
ReadOnlyTensor (Tensor)
¶
A special type of tensor which is read-only.
This is a subclass of torch.Tensor which explicitly disallows
operations that would cause in-place modifications.
Since ReadOnlyTensor if a subclass of torch.Tensor, most
non-destructive PyTorch operations are on this tensor are supported.
Cloning a ReadOnlyTensor using the clone() method or Python's
deepcopy(...) function results in a regular PyTorch tensor.
Reshaping or slicing operations might return a ReadOnlyTensor if the
result ends up being a view of the original ReadOnlyTensor; otherwise,
the returned tensor is a regular torch.Tensor.
Source code in evotorch/tools/readonlytensor.py
class ReadOnlyTensor(torch.Tensor):
"""
A special type of tensor which is read-only.
This is a subclass of `torch.Tensor` which explicitly disallows
operations that would cause in-place modifications.
Since ReadOnlyTensor if a subclass of `torch.Tensor`, most
non-destructive PyTorch operations are on this tensor are supported.
Cloning a ReadOnlyTensor using the `clone()` method or Python's
`deepcopy(...)` function results in a regular PyTorch tensor.
Reshaping or slicing operations might return a ReadOnlyTensor if the
result ends up being a view of the original ReadOnlyTensor; otherwise,
the returned tensor is a regular `torch.Tensor`.
"""
def __getattribute__(self, attribute_name: str) -> Any:
if (
isinstance(attribute_name, str)
and attribute_name.endswith("_")
and (not ((attribute_name.startswith("__")) and (attribute_name.endswith("__"))))
):
raise AttributeError(
f"A ReadOnlyTensor explicitly disables all members whose names end with '_'."
f" Cannot access member {repr(attribute_name)}."
)
else:
return super().__getattribute__(attribute_name)
def __cannot_modify(self, *ignore, **ignore_too):
raise TypeError("The contents of a ReadOnlyTensor cannot be modified")
__setitem__ = __cannot_modify
__iadd__ = __cannot_modify
__iand__ = __cannot_modify
__idiv__ = __cannot_modify
__ifloordiv__ = __cannot_modify
__ilshift__ = __cannot_modify
__imatmul__ = __cannot_modify
__imod__ = __cannot_modify
__imul__ = __cannot_modify
__ior__ = __cannot_modify
__ipow__ = __cannot_modify
__irshift__ = __cannot_modify
__isub__ = __cannot_modify
__itruediv__ = __cannot_modify
__ixor__ = __cannot_modify
if _torch_older_than_1_12:
# Define __str__ and __repr__ for when using PyTorch 1.11 or older.
# With PyTorch 1.12, overriding __str__ and __repr__ are not necessary.
def __to_string(self) -> str:
s = super().__repr__()
if "\n" not in s:
return f"ReadOnlyTensor({super().__repr__()})"
else:
indenter = " " * 4
s = (indenter + s.replace("\n", "\n" + indenter)).rstrip()
return f"ReadOnlyTensor(\n{s}\n)"
__str__ = __to_string
__repr__ = __to_string
def clone(self, *, preserve_read_only: bool = False) -> torch.Tensor:
result = super().clone()
if not preserve_read_only:
result = result.as_subclass(torch.Tensor)
return result
def __mutable_if_independent(self, other: torch.Tensor) -> torch.Tensor:
from .misc import storage_ptr
self_ptr = storage_ptr(self)
other_ptr = storage_ptr(other)
if self_ptr != other_ptr:
other = other.as_subclass(torch.Tensor)
return other
def __getitem__(self, index_or_slice) -> torch.Tensor:
result = super().__getitem__(index_or_slice)
return self.__mutable_if_independent(result)
def reshape(self, *args, **kwargs) -> torch.Tensor:
result = super().reshape(*args, **kwargs)
return self.__mutable_if_independent(result)
def numpy(self) -> np.ndarray:
arr: np.ndarray = torch.Tensor.numpy(self)
arr.flags["WRITEABLE"] = False
return arr
def __array__(self, *args, **kwargs) -> np.ndarray:
arr: np.ndarray = super().__array__(*args, **kwargs)
arr.flags["WRITEABLE"] = False
return arr
def __copy__(self):
return self.clone(preserve_read_only=True)
def __deepcopy__(self, memo):
return self.clone(preserve_read_only=True)
@classmethod
def __torch_function__(cls, func: Callable, types: Iterable, args: tuple = (), kwargs: Optional[Mapping] = None):
if (kwargs is not None) and ("out" in kwargs):
if isinstance(kwargs["out"], ReadOnlyTensor):
raise TypeError(
f"The `out` keyword argument passed to {func} is a ReadOnlyTensor."
f" A ReadOnlyTensor explicitly fails when referenced via the `out` keyword argument of any torch"
f" function."
f" This restriction is for making sure that the torch operations which could normally do in-place"
f" modifications do not operate on ReadOnlyTensor instances."
)
return super().__torch_function__(func, types, args, kwargs)
__torch_function__(func, types, args=(), kwargs=None)
classmethod
special
¶
This torch_function implementation wraps subclasses such that
methods called on subclasses return a subclass instance instead of
a torch.Tensor instance.
One corollary to this is that you need coverage for torch.Tensor methods if implementing torch_function for subclasses.
We recommend always calling super().__torch_function__ as the base
case when doing the above.
While not mandatory, we recommend making __torch_function__ a classmethod.
Source code in evotorch/tools/readonlytensor.py
@classmethod
def __torch_function__(cls, func: Callable, types: Iterable, args: tuple = (), kwargs: Optional[Mapping] = None):
if (kwargs is not None) and ("out" in kwargs):
if isinstance(kwargs["out"], ReadOnlyTensor):
raise TypeError(
f"The `out` keyword argument passed to {func} is a ReadOnlyTensor."
f" A ReadOnlyTensor explicitly fails when referenced via the `out` keyword argument of any torch"
f" function."
f" This restriction is for making sure that the torch operations which could normally do in-place"
f" modifications do not operate on ReadOnlyTensor instances."
)
return super().__torch_function__(func, types, args, kwargs)
clone(self, *, preserve_read_only=False)
¶
clone(*, memory_format=torch.preserve_format) -> Tensor
See :func:torch.clone
numpy(self)
¶
numpy(*, force=False) -> numpy.ndarray
Returns the tensor as a NumPy :class:ndarray.
If :attr:force is False (the default), the conversion
is performed only if the tensor is on the CPU, does not require grad,
does not have its conjugate bit set, and is a dtype and layout that
NumPy supports. The returned ndarray and the tensor will share their
storage, so changes to the tensor will be reflected in the ndarray
and vice versa.
If :attr:force is True this is equivalent to
calling t.detach().cpu().resolve_conj().resolve_neg().numpy().
If the tensor isn't on the CPU or the conjugate or negative bit is set,
the tensor won't share its storage with the returned ndarray.
Setting :attr:force to True can be a useful shorthand.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
force |
bool |
if |
required |
reshape(self, *args, **kwargs)
¶
reshape(*shape) -> Tensor
Returns a tensor with the same data and number of elements as :attr:self
but with the specified shape. This method returns a view if :attr:shape is
compatible with the current shape. See :meth:torch.Tensor.view on when it is
possible to return a view.
See :func:torch.reshape
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
shape |
tuple of ints or int... |
the desired shape |
required |
as_read_only_tensor(x, *, dtype=None, device=None)
¶
Convert the given object to a ReadOnlyTensor.
The provided object can be a scalar, or an Iterable of numeric data, or an ObjectArray.
This function can be thought as the read-only counterpart of PyTorch's
torch.as_tensor(...) function.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
The object to be converted to a ReadOnlyTensor. |
required |
dtype |
Optional[torch.dtype] |
The dtype of the new ReadOnlyTensor (e.g. torch.float32).
If this argument is not specified, dtype will be inferred from |
None |
device |
Union[str, torch.device] |
The device in which the ReadOnlyTensor will be stored
(e.g. "cpu").
If this argument is not specified, the device which is storing
the original |
None |
Returns:
| Type | Description |
|---|---|
Iterable |
The read-only counterpart of the provided object. |
Source code in evotorch/tools/readonlytensor.py
def as_read_only_tensor(
x: Any, *, dtype: Optional[torch.dtype] = None, device: Optional[Union[str, torch.device]] = None
) -> Iterable:
"""
Convert the given object to a ReadOnlyTensor.
The provided object can be a scalar, or an Iterable of numeric data,
or an ObjectArray.
This function can be thought as the read-only counterpart of PyTorch's
`torch.as_tensor(...)` function.
Args:
x: The object to be converted to a ReadOnlyTensor.
dtype: The dtype of the new ReadOnlyTensor (e.g. torch.float32).
If this argument is not specified, dtype will be inferred from `x`.
For example, if `x` is a PyTorch tensor or a numpy array, its
existing dtype will be kept.
device: The device in which the ReadOnlyTensor will be stored
(e.g. "cpu").
If this argument is not specified, the device which is storing
the original `x` will be re-used.
Returns:
The read-only counterpart of the provided object.
"""
from .objectarray import ObjectArray
kwargs = _device_and_dtype_kwargs(dtype=dtype, device=device)
if isinstance(x, ObjectArray):
if len(kwargs) != 0:
raise ValueError(
f"read_only_tensor(...): when making a read-only tensor from an ObjectArray,"
f" the arguments `dtype` and `device` were not expected."
f" However, the received keyword arguments are: {kwargs}."
)
return x.get_read_only_view()
else:
return torch.as_tensor(x, **kwargs).as_subclass(ReadOnlyTensor)
read_only_tensor(x, *, dtype=None, device=None)
¶
Make a ReadOnlyTensor from the given object.
The provided object can be a scalar, or an Iterable of numeric data, or an ObjectArray.
This function can be thought as the read-only counterpart of PyTorch's
torch.tensor(...) function.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
The object from which the new ReadOnlyTensor will be made. |
required |
dtype |
Optional[torch.dtype] |
The dtype of the new ReadOnlyTensor (e.g. torch.float32). |
None |
device |
Union[str, torch.device] |
The device in which the ReadOnlyTensor will be stored (e.g. "cpu"). |
None |
Returns:
| Type | Description |
|---|---|
Iterable |
The new read-only tensor. |
Source code in evotorch/tools/readonlytensor.py
def read_only_tensor(
x: Any, *, dtype: Optional[torch.dtype] = None, device: Optional[Union[str, torch.device]] = None
) -> Iterable:
"""
Make a ReadOnlyTensor from the given object.
The provided object can be a scalar, or an Iterable of numeric data,
or an ObjectArray.
This function can be thought as the read-only counterpart of PyTorch's
`torch.tensor(...)` function.
Args:
x: The object from which the new ReadOnlyTensor will be made.
dtype: The dtype of the new ReadOnlyTensor (e.g. torch.float32).
device: The device in which the ReadOnlyTensor will be stored
(e.g. "cpu").
Returns:
The new read-only tensor.
"""
from .objectarray import ObjectArray
kwargs = _device_and_dtype_kwargs(dtype=dtype, device=device)
if isinstance(x, ObjectArray):
if len(kwargs) != 0:
raise ValueError(
f"read_only_tensor(...): when making a read-only tensor from an ObjectArray,"
f" the arguments `dtype` and `device` were not expected."
f" However, the received keyword arguments are: {kwargs}."
)
return x.get_read_only_view()
else:
return torch.as_tensor(x, **kwargs).as_subclass(ReadOnlyTensor)
recursiveprintable
¶
RecursivePrintable
¶
A base class for making a class printable.
This base class considers custom container types which can recursively
contain themselves (even in a cyclic manner). Classes inheriting from
RecursivePrintable will gain a new ready-to-use method named
to_string(...). This to_string(...) method, upon being called,
checks if the current class is an Iterable or a Mapping, and prints
the representation accordingly, with a recursion limit to avoid
RecursionError. The methods __str__(...) and __repr__(...)
are also defined as aliases of this to_string method.
Source code in evotorch/tools/recursiveprintable.py
class RecursivePrintable:
"""
A base class for making a class printable.
This base class considers custom container types which can recursively
contain themselves (even in a cyclic manner). Classes inheriting from
`RecursivePrintable` will gain a new ready-to-use method named
`to_string(...)`. This `to_string(...)` method, upon being called,
checks if the current class is an Iterable or a Mapping, and prints
the representation accordingly, with a recursion limit to avoid
`RecursionError`. The methods `__str__(...)` and `__repr__(...)`
are also defined as aliases of this `to_string` method.
"""
def to_string(self, *, max_depth: int = 10) -> str:
if max_depth <= 0:
return "<...>"
def item_repr(x: Any) -> str:
if isinstance(x, RecursivePrintable):
return x.to_string(max_depth=(max_depth - 1))
else:
return repr(x)
result = []
def puts(*x: Any):
for item_of_x in x:
result.append(str(item_of_x))
clsname = type(self).__name__
first_one = True
if isinstance(self, Mapping):
puts(clsname, "({")
for k, v in self.items():
if first_one:
first_one = False
else:
puts(", ")
puts(item_repr(k), ": ", item_repr(v))
puts("})")
elif isinstance(self, Iterable):
puts(clsname, "([")
for v in self:
if first_one:
first_one = False
else:
puts(", ")
puts(item_repr(v))
puts("])")
else:
raise NotImplementedError
return "".join(result)
def __str__(self) -> str:
return self.to_string()
def __repr__(self) -> str:
return self.to_string()
structures
¶
This namespace contains data structures whose underlying storages are contiguous and therefore vectorization-friendly.
CBag (Structure)
¶
An integer bag from which one can do sampling without replacement.
Let us imagine that we wish to create a bag whose maximum length (i.e. whose maximum number of contained elements) is 5. For this, we can do:
which gives us an empty bag (i.e. a bag in which all pre-allocated slots are empty):
_________________________________________________
| | | | | |
| <empty> | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
Given that the maximum length for this bag is 5, the default set of acceptable values for this bag is 0, 1, 2, 3, 4. Let us put three values into our bag:
After these push operations, our bag can be visualized like this:
_________________________________________________
| | | | | |
| 1 | 3 | 4 | <empty> | <empty> |
|_________|_________|_________|_________|_________|
Let us now sample an element from this bag:
Because this is the first time we are sampling from this bag, the elements will be first shuffled. Let us assume that the shuffling resulted in:
_________________________________________________
| | | | | |
| 3 | 1 | 4 | <empty> | <empty> |
|_________|_________|_________|_________|_________|
Given this shuffed state, our call to pop_(...) will pop the leftmost
element (3 in this case). Therefore, the value of sampled1 will be 3
(as a scalar PyTorch tensor), and the state of the bag after the pop
operation will be:
_________________________________________________
| | | | | |
| 1 | 4 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
Let us keep sampling until the bag is empty:
The value of sampled2 becomes 1, and the value of sampled3 becomes 4.
This class can also represent a contiguous batch of bags. As an example, let us create 4 bags, each of length 5:
After this instantiation, bag_batch can be visualized like this:
__[ batch item 0 ]_______________________________
| | | | | |
| <empty> | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 1 ]_______________________________
| | | | | |
| <empty> | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 2 ]_______________________________
| | | | | |
| <empty> | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 3 ]_______________________________
| | | | | |
| <empty> | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
We can add values to our batch like this:
which would result in:
__[ batch item 0 ]_______________________________
| | | | | |
| 3 | 3 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 1 ]_______________________________
| | | | | |
| 2 | 1 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 2 ]_______________________________
| | | | | |
| 3 | 1 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 3 ]_______________________________
| | | | | |
| 1 | 4 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
We can also add values only to some of the bags within the batch:
which would result in:
__[ batch item 0 ]_______________________________
| | | | | |
| 3 | 3 | 0 | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 1 ]_______________________________
| | | | | |
| 2 | 1 | 2 | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 2 ]_______________________________
| | | | | |
| 3 | 1 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 3 ]_______________________________
| | | | | |
| 1 | 4 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
Notice that the batch items 2 and 3 were not affected, because their
corresponding values in the where tensor were given as False.
Let us now assume that we wish to obtain a sample from each bag. We can do:
Since this is the first sampling operation on this bag batch, each bag will first be shuffled. Let us assume that the shuffling resulted in:
__[ batch item 0 ]_______________________________
| | | | | |
| 0 | 3 | 3 | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 1 ]_______________________________
| | | | | |
| 1 | 2 | 2 | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 2 ]_______________________________
| | | | | |
| 3 | 1 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 3 ]_______________________________
| | | | | |
| 4 | 1 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
Given this shuffled state, the pop operation takes the leftmost element
from each bag. Therefore, the value of sample_batch1 becomes a
1-dimensional tensor containing [0, 1, 3, 4]. Once the pop operation
is completed, the state of the batch of bags becomes:
__[ batch item 0 ]_______________________________
| | | | | |
| 3 | 3 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 1 ]_______________________________
| | | | | |
| 2 | 2 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 2 ]_______________________________
| | | | | |
| 1 | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 3 ]_______________________________
| | | | | |
| 1 | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
Now, if we wish to pop only from some of the bags, we can do:
which makes the value of sample_batch2 a 1-dimensional tensor containing
[3, 2, 1, 1] (the leftmost element for each bag). The state of our batch
of bags will become:
__[ batch item 0 ]_______________________________
| | | | | |
| 3 | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 1 ]_______________________________
| | | | | |
| 2 | 2 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 2 ]_______________________________
| | | | | |
| <empty> | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 3 ]_______________________________
| | | | | |
| 1 | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
Notice that the batch items 1 and 3 were not modified, because their
corresponding values in the where argument were given as False.
Source code in evotorch/tools/structures.py
class CBag(Structure):
"""
An integer bag from which one can do sampling without replacement.
Let us imagine that we wish to create a bag whose maximum length (i.e.
whose maximum number of contained elements) is 5. For this, we can do:
```python
bag = CBag(max_length=5)
```
which gives us an empty bag (i.e. a bag in which all pre-allocated slots
are empty):
```
_________________________________________________
| | | | | |
| <empty> | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
```
Given that the maximum length for this bag is 5, the default set of
acceptable values for this bag is 0, 1, 2, 3, 4. Let us put three values
into our bag:
```
bag.push_(torch.tensor(1))
bag.push_(torch.tensor(3))
bag.push_(torch.tensor(4))
```
After these push operations, our bag can be visualized like this:
```
_________________________________________________
| | | | | |
| 1 | 3 | 4 | <empty> | <empty> |
|_________|_________|_________|_________|_________|
```
Let us now sample an element from this bag:
```python
sampled1 = bag.pop_()
```
Because this is the first time we are sampling from this bag, the elements
will be first shuffled. Let us assume that the shuffling resulted in:
```
_________________________________________________
| | | | | |
| 3 | 1 | 4 | <empty> | <empty> |
|_________|_________|_________|_________|_________|
```
Given this shuffed state, our call to `pop_(...)` will pop the leftmost
element (3 in this case). Therefore, the value of `sampled1` will be 3
(as a scalar PyTorch tensor), and the state of the bag after the pop
operation will be:
```
_________________________________________________
| | | | | |
| 1 | 4 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
```
Let us keep sampling until the bag is empty:
```python
sampled2 = bag.pop_()
sampled3 = bag.pop_()
```
The value of `sampled2` becomes 1, and the value of `sampled3` becomes 4.
This class can also represent a contiguous batch of bags. As an example,
let us create 4 bags, each of length 5:
```python
bag_batch = CBag(batch_size=4, max_length=5)
```
After this instantiation, `bag_batch` can be visualized like this:
```
__[ batch item 0 ]_______________________________
| | | | | |
| <empty> | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 1 ]_______________________________
| | | | | |
| <empty> | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 2 ]_______________________________
| | | | | |
| <empty> | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 3 ]_______________________________
| | | | | |
| <empty> | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
```
We can add values to our batch like this:
```python
bag_batch.push_(torch.tensor([3, 2, 3, 1]))
bag_batch.push_(torch.tensor([3, 1, 1, 4]))
```
which would result in:
```
__[ batch item 0 ]_______________________________
| | | | | |
| 3 | 3 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 1 ]_______________________________
| | | | | |
| 2 | 1 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 2 ]_______________________________
| | | | | |
| 3 | 1 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 3 ]_______________________________
| | | | | |
| 1 | 4 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
```
We can also add values only to some of the bags within the batch:
```
bag_batch.push_(
torch.tensor([0, 2, 1, 0]),
where=torch.tensor([True, True, False, False])),
)
```
which would result in:
```
__[ batch item 0 ]_______________________________
| | | | | |
| 3 | 3 | 0 | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 1 ]_______________________________
| | | | | |
| 2 | 1 | 2 | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 2 ]_______________________________
| | | | | |
| 3 | 1 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 3 ]_______________________________
| | | | | |
| 1 | 4 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
```
Notice that the batch items 2 and 3 were not affected, because their
corresponding values in the `where` tensor were given as False.
Let us now assume that we wish to obtain a sample from each bag. We can do:
```python
sample_batch1 = bag_batch.pop_()
```
Since this is the first sampling operation on this bag batch, each bag
will first be shuffled. Let us assume that the shuffling resulted in:
```
__[ batch item 0 ]_______________________________
| | | | | |
| 0 | 3 | 3 | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 1 ]_______________________________
| | | | | |
| 1 | 2 | 2 | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 2 ]_______________________________
| | | | | |
| 3 | 1 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 3 ]_______________________________
| | | | | |
| 4 | 1 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
```
Given this shuffled state, the pop operation takes the leftmost element
from each bag. Therefore, the value of `sample_batch1` becomes a
1-dimensional tensor containing `[0, 1, 3, 4]`. Once the pop operation
is completed, the state of the batch of bags becomes:
```
__[ batch item 0 ]_______________________________
| | | | | |
| 3 | 3 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 1 ]_______________________________
| | | | | |
| 2 | 2 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 2 ]_______________________________
| | | | | |
| 1 | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 3 ]_______________________________
| | | | | |
| 1 | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
```
Now, if we wish to pop only from some of the bags, we can do:
```python
sample_batch2 = bag_batch.pop_(
where=torch.tensor([True, False, True, False]),
)
```
which makes the value of `sample_batch2` a 1-dimensional tensor containing
`[3, 2, 1, 1]` (the leftmost element for each bag). The state of our batch
of bags will become:
```
__[ batch item 0 ]_______________________________
| | | | | |
| 3 | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 1 ]_______________________________
| | | | | |
| 2 | 2 | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 2 ]_______________________________
| | | | | |
| <empty> | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
__[ batch item 3 ]_______________________________
| | | | | |
| 1 | <empty> | <empty> | <empty> | <empty> |
|_________|_________|_________|_________|_________|
```
Notice that the batch items 1 and 3 were not modified, because their
corresponding values in the `where` argument were given as False.
"""
def __init__(
self,
*,
max_length: int,
value_range: Optional[tuple] = None,
batch_size: Optional[Union[int, tuple, list]] = None,
batch_shape: Optional[Union[int, tuple, list]] = None,
generator: Any = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
verify: bool = True,
):
"""
Initialize the CBag.
Args:
max_length: Maximum length (i.e. maximum capacity for storing
elements).
value_range: Optionally expected as a tuple of integers in the
form `(a, b)` where `a` is the lower bound and `b` is the
exclusive upper bound for the range of acceptable integer
values. If this argument is omitted, the range will be
`(0, n)` where `n` is `max_length`.
batch_size: Optionally an integer or a size tuple, for when
one wishes to create not just a single bag, but a batch
of bags.
batch_shape: Alias for the argument `batch_size`.
generator: Optionally an instance of `torch.Generator` or any
object with an attribute (or a property) named `generator`
(in which case it will be expected that this attribute will
provide the actual `torch.Generator` instance). If this
argument is provided, then the shuffling operation will use
this generator. Otherwise, the global generator of PyTorch
will be used.
dtype: dtype for the values contained by the bag(s).
By default, the dtype is `torch.int64`.
device: The device on which the bag(s) will be stored.
By default, the device is `torch.device("cpu")`.
verify: Whether or not to do explicit checks for the correctness
of the operations (against popping from an empty bag or
pushing into a full bag). By default, this is True.
If you are sure that such errors will not occur, you might
turn this to False for getting a performance gain.
"""
if dtype is None:
dtype = torch.int64
else:
dtype = to_torch_dtype(dtype)
if dtype not in (torch.int16, torch.int32, torch.int64):
raise RuntimeError(
f"CBag currently supports only torch.int16, torch.int32, and torch.int64."
f" This dtype is not supported: {repr(dtype)}."
)
self._gen_kwargs = {}
if generator is not None:
if isinstance(generator, torch.Generator):
self._gen_kwargs["generator"] = generator
else:
generator = generator.generator
if generator is not None:
self._gen_kwargs["generator"] = generator
max_length = int(max_length)
self._data = CList(
max_length=max_length,
batch_size=batch_size,
batch_shape=batch_shape,
dtype=dtype,
device=device,
verify=verify,
)
if value_range is None:
a = 0
b = max_length
else:
a, b = value_range
self._low_item = int(a)
self._high_item = int(b) # upper bound is exclusive
self._choice_count = self._high_item - self._low_item
self._bignum = self._choice_count + 1
if self._low_item < 1:
self._shift = 1 - self._low_item
else:
self._shift = 0
self._empty = self._low_item - 1
self._data.data[:] = self._empty
self._sampling_phase: bool = False
def push_(self, value: Numbers, where: Optional[Numbers] = None):
"""
Push new value(s) into the bag(s).
Args:
value: The value(s) to be pushed into the bag(s).
where: Optionally a boolean tensor. If this is given, then only
the bags with their corresponding boolean flags set as True
will be affected.
"""
if self._sampling_phase:
raise RuntimeError("Cannot put a new element into the CBag after calling `sample_(...)`")
self._data.push_(value, where)
def _shuffle(self):
dtype = self._data.dtype
device = self._data.device
nrows, ncols = self._data.data.shape
try:
gaussian_noise = torch.randn(nrows, ncols, dtype=torch.float32, device=device, **(self._gen_kwargs))
noise = gaussian_noise.argsort().to(dtype=dtype) * self._bignum
self._data.data[:] += torch.where(
self._data.data != self._empty, self._shift + noise, torch.tensor(0, dtype=dtype, device=device)
)
self._data.data[:] = self._data.data.sort(dim=-1, descending=True, stable=False).values
finally:
self._data.data[:] %= self._bignum
self._data.data[:] -= self._shift
def pop_(self, where: Optional[Numbers] = None) -> torch.Tensor:
"""
Sample value(s) from the bag(s).
Upon being called for the first time, this method will cause the
contained elements to be shuffled.
Args:
where: Optionally a boolean tensor. If this is given, then only
the bags with their corresponding boolean flags set as True
will be affected.
"""
if not self._sampling_phase:
self._shuffle()
self._sampling_phase = True
return self._data.pop_(where)
def clear(self):
"""
Clear the bag(s).
"""
self._data.data[:] = self._empty
self._data.clear()
self._sampling_phase = False
@property
def length(self) -> torch.Tensor:
"""
The length(s) of the bag(s)
"""
return self._data.length
@property
def data(self) -> torch.Tensor:
"""
The underlying data tensor
"""
return self._data.data
data: Tensor
property
readonly
¶
The underlying data tensor
length: Tensor
property
readonly
¶
The length(s) of the bag(s)
__init__(self, *, max_length, value_range=None, batch_size=None, batch_shape=None, generator=None, dtype=None, device=None, verify=True)
special
¶
Initialize the CBag.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
max_length |
int |
Maximum length (i.e. maximum capacity for storing elements). |
required |
value_range |
Optional[tuple] |
Optionally expected as a tuple of integers in the
form |
None |
batch_size |
Union[int, tuple, list] |
Optionally an integer or a size tuple, for when one wishes to create not just a single bag, but a batch of bags. |
None |
batch_shape |
Union[int, tuple, list] |
Alias for the argument |
None |
generator |
Any |
Optionally an instance of |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
dtype for the values contained by the bag(s).
By default, the dtype is |
None |
device |
Union[str, torch.device] |
The device on which the bag(s) will be stored.
By default, the device is |
None |
verify |
bool |
Whether or not to do explicit checks for the correctness of the operations (against popping from an empty bag or pushing into a full bag). By default, this is True. If you are sure that such errors will not occur, you might turn this to False for getting a performance gain. |
True |
Source code in evotorch/tools/structures.py
def __init__(
self,
*,
max_length: int,
value_range: Optional[tuple] = None,
batch_size: Optional[Union[int, tuple, list]] = None,
batch_shape: Optional[Union[int, tuple, list]] = None,
generator: Any = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
verify: bool = True,
):
"""
Initialize the CBag.
Args:
max_length: Maximum length (i.e. maximum capacity for storing
elements).
value_range: Optionally expected as a tuple of integers in the
form `(a, b)` where `a` is the lower bound and `b` is the
exclusive upper bound for the range of acceptable integer
values. If this argument is omitted, the range will be
`(0, n)` where `n` is `max_length`.
batch_size: Optionally an integer or a size tuple, for when
one wishes to create not just a single bag, but a batch
of bags.
batch_shape: Alias for the argument `batch_size`.
generator: Optionally an instance of `torch.Generator` or any
object with an attribute (or a property) named `generator`
(in which case it will be expected that this attribute will
provide the actual `torch.Generator` instance). If this
argument is provided, then the shuffling operation will use
this generator. Otherwise, the global generator of PyTorch
will be used.
dtype: dtype for the values contained by the bag(s).
By default, the dtype is `torch.int64`.
device: The device on which the bag(s) will be stored.
By default, the device is `torch.device("cpu")`.
verify: Whether or not to do explicit checks for the correctness
of the operations (against popping from an empty bag or
pushing into a full bag). By default, this is True.
If you are sure that such errors will not occur, you might
turn this to False for getting a performance gain.
"""
if dtype is None:
dtype = torch.int64
else:
dtype = to_torch_dtype(dtype)
if dtype not in (torch.int16, torch.int32, torch.int64):
raise RuntimeError(
f"CBag currently supports only torch.int16, torch.int32, and torch.int64."
f" This dtype is not supported: {repr(dtype)}."
)
self._gen_kwargs = {}
if generator is not None:
if isinstance(generator, torch.Generator):
self._gen_kwargs["generator"] = generator
else:
generator = generator.generator
if generator is not None:
self._gen_kwargs["generator"] = generator
max_length = int(max_length)
self._data = CList(
max_length=max_length,
batch_size=batch_size,
batch_shape=batch_shape,
dtype=dtype,
device=device,
verify=verify,
)
if value_range is None:
a = 0
b = max_length
else:
a, b = value_range
self._low_item = int(a)
self._high_item = int(b) # upper bound is exclusive
self._choice_count = self._high_item - self._low_item
self._bignum = self._choice_count + 1
if self._low_item < 1:
self._shift = 1 - self._low_item
else:
self._shift = 0
self._empty = self._low_item - 1
self._data.data[:] = self._empty
self._sampling_phase: bool = False
clear(self)
¶
pop_(self, where=None)
¶
Sample value(s) from the bag(s).
Upon being called for the first time, this method will cause the contained elements to be shuffled.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean tensor. If this is given, then only the bags with their corresponding boolean flags set as True will be affected. |
None |
Source code in evotorch/tools/structures.py
def pop_(self, where: Optional[Numbers] = None) -> torch.Tensor:
"""
Sample value(s) from the bag(s).
Upon being called for the first time, this method will cause the
contained elements to be shuffled.
Args:
where: Optionally a boolean tensor. If this is given, then only
the bags with their corresponding boolean flags set as True
will be affected.
"""
if not self._sampling_phase:
self._shuffle()
self._sampling_phase = True
return self._data.pop_(where)
push_(self, value, where=None)
¶
Push new value(s) into the bag(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) to be pushed into the bag(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean tensor. If this is given, then only the bags with their corresponding boolean flags set as True will be affected. |
None |
Source code in evotorch/tools/structures.py
def push_(self, value: Numbers, where: Optional[Numbers] = None):
"""
Push new value(s) into the bag(s).
Args:
value: The value(s) to be pushed into the bag(s).
where: Optionally a boolean tensor. If this is given, then only
the bags with their corresponding boolean flags set as True
will be affected.
"""
if self._sampling_phase:
raise RuntimeError("Cannot put a new element into the CBag after calling `sample_(...)`")
self._data.push_(value, where)
CDict (Structure)
¶
Representation of a batchable dictionary.
This structure is very similar to a CMemory, but with the additional
behavior of separately keeping track of which keys exist and which keys
do not exist.
Let us consider an example where we have 5 keys, and each key is associated with a tensor of length 7. Such a dictionary could be allocated like this:
Our allocated dictionary can be visualized as follows:
_______________________________________
| key 0 -> ( missing ) |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
Let us now sample a Gaussian noise and put it into the 0-th slot:
which results in:
_________________________________________
| key 0 -> [ Gaussian noise of length 7 ] |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_________________________________________|
Let us now consider another example where we deal with not a single dictionary but with a dictionary batch. For the sake of this example, let us say that our desired batch size is 3. The allocation of such a batch would be as follows:
Our dictionary batch can be visualized like this:
__[ batch item 0 ]_____________________
| key 0 -> ( missing ) |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> ( missing ) |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> ( missing ) |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
If we wish to set the 0-th element of each batch item, we could do:
dict_batch[0] = torch.tensor(
[
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0],
[2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0],
],
)
and the result would be:
__[ batch item 0 ]_____________________
| key 0 -> [ 0. 0. 0. 0. 0. 0. 0. ] |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ 1. 1. 1. 1. 1. 1. 1. ] |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ 2. 2. 2. 2. 2. 2. 2. ] |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
Continuing from the same example, if we wish to set the slot with key 1 in the 0th batch item, slot with key 2 in the 1st batch item, and slot with key 3 in the 2nd batch item, all in one go, we could do:
# Longer version: dict_batch[torch.tensor([1, 2, 3])] = ...
dict_batch[[1, 2, 3]] = torch.tensor(
[
[5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0],
[6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0],
[7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0],
],
)
Our updated dictionary batch would then look like this:
__[ batch item 0 ]_____________________
| key 0 -> [ 0. 0. 0. 0. 0. 0. 0. ] |
| key 1 -> [ 5. 5. 5. 5. 5. 5. 5. ] |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ 1. 1. 1. 1. 1. 1. 1. ] |
| key 1 -> ( missing ) |
| key 2 -> [ 6. 6. 6. 6. 6. 6. 6. ] |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ 2. 2. 2. 2. 2. 2. 2. ] |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> [ 7. 7. 7. 7. 7. 7. 7. ] |
| key 4 -> ( missing ) |
|_______________________________________|
Conditional modifications via boolean masks is also supported.
For example, the following update on our dict_batch:
dict_batch.set_(
[4, 3, 1],
torch.tensor(
[
[8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0],
[9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0],
[10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0],
]
),
where=[True, True, False], # or: where=torch.tensor([True,True,False]),
)
would result in:
__[ batch item 0 ]_____________________
| key 0 -> [ 0. 0. 0. 0. 0. 0. 0. ] |
| key 1 -> [ 5. 5. 5. 5. 5. 5. 5. ] |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> [ 8. 8. 8. 8. 8. 8. 8. ] |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ 1. 1. 1. 1. 1. 1. 1. ] |
| key 1 -> ( missing ) |
| key 2 -> [ 6. 6. 6. 6. 6. 6. 6. ] |
| key 3 -> [ 9. 9. 9. 9. 9. 9. 9. ] |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ 2. 2. 2. 2. 2. 2. 2. ] |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> [ 7. 7. 7. 7. 7. 7. 7. ] |
| key 4 -> ( missing ) |
|_______________________________________|
Please notice above that the slot with key 1 of the batch item 2 was not modified because its corresponding mask value was given as False.
After all these modifications, querying whether or not an element with key 0 would give us the following output:
which means that, for each dictionary within the batch, an element with key 0 exists. The same query for the key 3 would give us:
which means that the 0-th dictionary within the batch does not have an element with key 3, but the dictionaries 1 and 2 do have their elements with that key.
Source code in evotorch/tools/structures.py
class CDict(Structure):
"""
Representation of a batchable dictionary.
This structure is very similar to a `CMemory`, but with the additional
behavior of separately keeping track of which keys exist and which keys
do not exist.
Let us consider an example where we have 5 keys, and each key is associated
with a tensor of length 7. Such a dictionary could be allocated like this:
```python
dictnry = CDict(7, num_keys=5)
```
Our allocated dictionary can be visualized as follows:
```text
_______________________________________
| key 0 -> ( missing ) |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
```
Let us now sample a Gaussian noise and put it into the 0-th slot:
```python
dictnry[0] = torch.randn(7) # or: dictnry[torch.tensor(0)] = torch.randn(7)
```
which results in:
```text
_________________________________________
| key 0 -> [ Gaussian noise of length 7 ] |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_________________________________________|
```
Let us now consider another example where we deal with not a single
dictionary but with a dictionary batch. For the sake of this example, let
us say that our desired batch size is 3. The allocation of such a batch
would be as follows:
```python
dict_batch = CDict(7, num_keys=5, batch_size=3)
```
Our dictionary batch can be visualized like this:
```text
__[ batch item 0 ]_____________________
| key 0 -> ( missing ) |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> ( missing ) |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> ( missing ) |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
```
If we wish to set the 0-th element of each batch item, we could do:
```python
dict_batch[0] = torch.tensor(
[
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0],
[2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0],
],
)
```
and the result would be:
```text
__[ batch item 0 ]_____________________
| key 0 -> [ 0. 0. 0. 0. 0. 0. 0. ] |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ 1. 1. 1. 1. 1. 1. 1. ] |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ 2. 2. 2. 2. 2. 2. 2. ] |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
```
Continuing from the same example, if we wish to set the slot with key 1
in the 0th batch item, slot with key 2 in the 1st batch item, and
slot with key 3 in the 2nd batch item, all in one go, we could do:
```python
# Longer version: dict_batch[torch.tensor([1, 2, 3])] = ...
dict_batch[[1, 2, 3]] = torch.tensor(
[
[5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0],
[6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0],
[7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0],
],
)
```
Our updated dictionary batch would then look like this:
```text
__[ batch item 0 ]_____________________
| key 0 -> [ 0. 0. 0. 0. 0. 0. 0. ] |
| key 1 -> [ 5. 5. 5. 5. 5. 5. 5. ] |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ 1. 1. 1. 1. 1. 1. 1. ] |
| key 1 -> ( missing ) |
| key 2 -> [ 6. 6. 6. 6. 6. 6. 6. ] |
| key 3 -> ( missing ) |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ 2. 2. 2. 2. 2. 2. 2. ] |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> [ 7. 7. 7. 7. 7. 7. 7. ] |
| key 4 -> ( missing ) |
|_______________________________________|
```
Conditional modifications via boolean masks is also supported.
For example, the following update on our `dict_batch`:
```python
dict_batch.set_(
[4, 3, 1],
torch.tensor(
[
[8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0],
[9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0],
[10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0],
]
),
where=[True, True, False], # or: where=torch.tensor([True,True,False]),
)
```
would result in:
```text
__[ batch item 0 ]_____________________
| key 0 -> [ 0. 0. 0. 0. 0. 0. 0. ] |
| key 1 -> [ 5. 5. 5. 5. 5. 5. 5. ] |
| key 2 -> ( missing ) |
| key 3 -> ( missing ) |
| key 4 -> [ 8. 8. 8. 8. 8. 8. 8. ] |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ 1. 1. 1. 1. 1. 1. 1. ] |
| key 1 -> ( missing ) |
| key 2 -> [ 6. 6. 6. 6. 6. 6. 6. ] |
| key 3 -> [ 9. 9. 9. 9. 9. 9. 9. ] |
| key 4 -> ( missing ) |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ 2. 2. 2. 2. 2. 2. 2. ] |
| key 1 -> ( missing ) |
| key 2 -> ( missing ) |
| key 3 -> [ 7. 7. 7. 7. 7. 7. 7. ] |
| key 4 -> ( missing ) |
|_______________________________________|
```
Please notice above that the slot with key 1 of the batch item 2 was not
modified because its corresponding mask value was given as False.
After all these modifications, querying whether or not an element with
key 0 would give us the following output:
```text
>>> dict_batch.contains(0)
torch.tensor([True, True, True], dtype=torch.bool)
```
which means that, for each dictionary within the batch, an element with
key 0 exists. The same query for the key 3 would give us:
```text
>>> dict_batch.contains(3)
torch.tensor([False, True, True], dtype=torch.bool)
```
which means that the 0-th dictionary within the batch does not have an
element with key 3, but the dictionaries 1 and 2 do have their elements
with that key.
"""
def __init__(
self,
*size: Union[int, tuple, list],
num_keys: Union[int, tuple, list],
key_offset: Optional[Union[int, tuple, list]] = None,
batch_size: Optional[Union[int, tuple, list]] = None,
batch_shape: Optional[Union[int, tuple, list]] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
verify: bool = True,
):
"""
`__init__(...)`: Initialize the CDict.
Args:
size: Size of a tensor associated with a key, expected as an
integer, or as multiple positional arguments (each positional
argument being an integer), or as a tuple of integers.
num_keys: How many keys (and therefore how many slots) can the
dictionary have. If given as an integer `n`, then there will be
`n` slots available in the dictionary, and to access a slot one
will need to use an integer key `k` (where, by default, the
minimum acceptable `k` is 0 and the maximum acceptable `k` is
`n-1`). If given as a tuple of integers, then the number of slots
available in the dictionary will be computed as the product of
all the integers in the tuple, and a key will be expected as a
tuple. For example, when `num_keys` is `(3, 5)`, there will be
15 slots available in the dictionary (where, by default, the
minimum acceptable key will be `(0, 0)` and the maximum
acceptable key will be `(2, 4)`.
key_offset: Optionally can be used to shift the integer values of
the keys. For example, if `num_keys` is 10, then, by default,
the minimum key is 0 and the maximum key is 9. But together
with `num_keys=10`, if `key_offset` is given as 1, then the
minimum key will be 1 and the maximum key will be 10.
This argument can also be used together with a tuple-valued
`num_keys`. For example, with `num_keys` set as `(3, 5)`,
if `key_offset` is given as 1, then the minimum key value
will be `(1, 1)` (instead of `(0, 0)`) and the maximum key
value will be `(3, 5)` (instead of `(2, 4)`).
Also, with a tuple-valued `num_keys`, `key_offset` can be
given as a tuple, to shift the key values differently for each
item in the tuple.
batch_size: If given as None, then this dictionary will not be
batched. If given as an integer `n`, then this object will
represent a contiguous batch containing `n` dictionary blocks.
If given as a tuple `(size0, size1, ...)`, then this object
will represent a contiguous batch of dictionary, shape of this
batch being determined by the given tuple.
batch_shape: Alias for the argument `batch_size`.
fill_with: Optionally a numeric value using which the values will
be initialized. If no initialization is needed, then this
argument can be left as None.
dtype: The `dtype` of the values stored by this CDict.
device: The device on which the dictionary will be allocated.
verify: If True, then explicit checks will be done to verify
that there are no indexing errors. Can be set as False for
performance.
"""
self._data = CMemory(
*size,
num_keys=num_keys,
key_offset=key_offset,
batch_size=batch_size,
batch_shape=batch_shape,
dtype=dtype,
device=device,
verify=verify,
)
self._exist = CMemory(
num_keys=num_keys,
key_offset=key_offset,
batch_size=batch_size,
batch_shape=batch_shape,
dtype=torch.bool,
device=device,
verify=verify,
)
def get(self, key: Numbers, default: Optional[Numbers] = None) -> torch.Tensor:
"""
Get the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
default: Optionally can be specified as the fallback value for when
the element(s) with the given key(s) do not exist.
Returns:
The value(s) associated with the given key(s).
"""
if default is None:
return self._data[key]
else:
exist = self._exist[key]
default = self._get_value(default)
return do_where(exist, self._data[key], default)
def set_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Set the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The new value(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self._data.set_(key, value, where)
self._exist.set_(key, True, where)
def add_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Add value(s) onto the existing values of slots with the given key(s).
Note that this operation does not change the existence flags of the
keys. In other words, if element(s) with `key` do not exist, then
they will still be flagged as non-existent after this operation.
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be added onto the existing value(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self._data.add_(key, value, where)
def subtract_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Subtract value(s) from existing values of slots with the given key(s).
Note that this operation does not change the existence flags of the
keys. In other words, if element(s) with `key` do not exist, then
they will still be flagged as non-existent after this operation.
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be subtracted from existing value(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self._data.subtract_(key, value, where)
def divide_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Divide the existing values of slots with the given key(s).
Note that this operation does not change the existence flags of the
keys. In other words, if element(s) with `key` do not exist, then
they will still be flagged as non-existent after this operation.
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be used as divisor(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self._data.divide_(key, value, where)
def multiply_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Multiply the existing values of slots with the given key(s).
Note that this operation does not change the existence flags of the
keys. In other words, if element(s) with `key` do not exist, then
they will still be flagged as non-existent after this operation.
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be used as the multiplier(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self._data.multiply_(key, value, where)
def contains(self, key: Numbers) -> torch.Tensor:
"""
Query whether or not the element(s) with the given key(s) exist.
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
Returns:
A boolean tensor indicating whether or not the element(s) with the
specified key(s) exist.
"""
return self._exist[key]
def __getitem__(self, key: Numbers) -> torch.Tensor:
"""
Get the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
Returns:
The value(s) associated with the given key(s).
"""
return self.get(key)
def __setitem__(self, key: Numbers, value: Numbers):
"""
Set the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The new value(s).
"""
self.set_(key, value)
def clear(self, where: Optional[torch.Tensor] = None):
"""
Clear the dictionaries.
In the context of this data structure, to "clear" means to set the
status for each key to non-existent.
Args:
where: Optionally a boolean tensor, specifying which dictionaries
within the batch should be cleared. If this argument is omitted
(i.e. left as None), then all dictionaries will be cleared.
"""
if where is None:
self._exist.data[:] = False
else:
where = self._get_where(where)
all_false = torch.tensor(False, dtype=torch.bool, device=self._exist.device).expand(self._exist.shape)
self._exist.data[:] = do_where(where, all_false, self._exist.data[:])
@property
def data(self) -> torch.Tensor:
"""
The entire value tensor
"""
return self._data.data
data: Tensor
property
readonly
¶
The entire value tensor
__getitem__(self, key)
special
¶
Get the value(s) associated with the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The value(s) associated with the given key(s). |
Source code in evotorch/tools/structures.py
def __getitem__(self, key: Numbers) -> torch.Tensor:
"""
Get the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
Returns:
The value(s) associated with the given key(s).
"""
return self.get(key)
__init__(self, *size, *, num_keys, key_offset=None, batch_size=None, batch_shape=None, dtype=None, device=None, verify=True)
special
¶
__init__(...): Initialize the CDict.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, tuple, list] |
Size of a tensor associated with a key, expected as an integer, or as multiple positional arguments (each positional argument being an integer), or as a tuple of integers. |
() |
num_keys |
Union[int, tuple, list] |
How many keys (and therefore how many slots) can the
dictionary have. If given as an integer |
required |
key_offset |
Union[int, tuple, list] |
Optionally can be used to shift the integer values of
the keys. For example, if |
None |
batch_size |
Union[int, tuple, list] |
If given as None, then this dictionary will not be
batched. If given as an integer |
None |
batch_shape |
Union[int, tuple, list] |
Alias for the argument |
None |
fill_with |
Optionally a numeric value using which the values will be initialized. If no initialization is needed, then this argument can be left as None. |
required | |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
The |
None |
device |
Union[str, torch.device] |
The device on which the dictionary will be allocated. |
None |
verify |
bool |
If True, then explicit checks will be done to verify that there are no indexing errors. Can be set as False for performance. |
True |
Source code in evotorch/tools/structures.py
def __init__(
self,
*size: Union[int, tuple, list],
num_keys: Union[int, tuple, list],
key_offset: Optional[Union[int, tuple, list]] = None,
batch_size: Optional[Union[int, tuple, list]] = None,
batch_shape: Optional[Union[int, tuple, list]] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
verify: bool = True,
):
"""
`__init__(...)`: Initialize the CDict.
Args:
size: Size of a tensor associated with a key, expected as an
integer, or as multiple positional arguments (each positional
argument being an integer), or as a tuple of integers.
num_keys: How many keys (and therefore how many slots) can the
dictionary have. If given as an integer `n`, then there will be
`n` slots available in the dictionary, and to access a slot one
will need to use an integer key `k` (where, by default, the
minimum acceptable `k` is 0 and the maximum acceptable `k` is
`n-1`). If given as a tuple of integers, then the number of slots
available in the dictionary will be computed as the product of
all the integers in the tuple, and a key will be expected as a
tuple. For example, when `num_keys` is `(3, 5)`, there will be
15 slots available in the dictionary (where, by default, the
minimum acceptable key will be `(0, 0)` and the maximum
acceptable key will be `(2, 4)`.
key_offset: Optionally can be used to shift the integer values of
the keys. For example, if `num_keys` is 10, then, by default,
the minimum key is 0 and the maximum key is 9. But together
with `num_keys=10`, if `key_offset` is given as 1, then the
minimum key will be 1 and the maximum key will be 10.
This argument can also be used together with a tuple-valued
`num_keys`. For example, with `num_keys` set as `(3, 5)`,
if `key_offset` is given as 1, then the minimum key value
will be `(1, 1)` (instead of `(0, 0)`) and the maximum key
value will be `(3, 5)` (instead of `(2, 4)`).
Also, with a tuple-valued `num_keys`, `key_offset` can be
given as a tuple, to shift the key values differently for each
item in the tuple.
batch_size: If given as None, then this dictionary will not be
batched. If given as an integer `n`, then this object will
represent a contiguous batch containing `n` dictionary blocks.
If given as a tuple `(size0, size1, ...)`, then this object
will represent a contiguous batch of dictionary, shape of this
batch being determined by the given tuple.
batch_shape: Alias for the argument `batch_size`.
fill_with: Optionally a numeric value using which the values will
be initialized. If no initialization is needed, then this
argument can be left as None.
dtype: The `dtype` of the values stored by this CDict.
device: The device on which the dictionary will be allocated.
verify: If True, then explicit checks will be done to verify
that there are no indexing errors. Can be set as False for
performance.
"""
self._data = CMemory(
*size,
num_keys=num_keys,
key_offset=key_offset,
batch_size=batch_size,
batch_shape=batch_shape,
dtype=dtype,
device=device,
verify=verify,
)
self._exist = CMemory(
num_keys=num_keys,
key_offset=key_offset,
batch_size=batch_size,
batch_shape=batch_shape,
dtype=torch.bool,
device=device,
verify=verify,
)
__setitem__(self, key, value)
special
¶
Set the value(s) associated with the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The new value(s). |
required |
Source code in evotorch/tools/structures.py
add_(self, key, value, where=None)
¶
Add value(s) onto the existing values of slots with the given key(s).
Note that this operation does not change the existence flags of the
keys. In other words, if element(s) with key do not exist, then
they will still be flagged as non-existent after this operation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) that will be added onto the existing value(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Source code in evotorch/tools/structures.py
def add_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Add value(s) onto the existing values of slots with the given key(s).
Note that this operation does not change the existence flags of the
keys. In other words, if element(s) with `key` do not exist, then
they will still be flagged as non-existent after this operation.
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be added onto the existing value(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self._data.add_(key, value, where)
clear(self, where=None)
¶
Clear the dictionaries.
In the context of this data structure, to "clear" means to set the status for each key to non-existent.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
where |
Optional[torch.Tensor] |
Optionally a boolean tensor, specifying which dictionaries within the batch should be cleared. If this argument is omitted (i.e. left as None), then all dictionaries will be cleared. |
None |
Source code in evotorch/tools/structures.py
def clear(self, where: Optional[torch.Tensor] = None):
"""
Clear the dictionaries.
In the context of this data structure, to "clear" means to set the
status for each key to non-existent.
Args:
where: Optionally a boolean tensor, specifying which dictionaries
within the batch should be cleared. If this argument is omitted
(i.e. left as None), then all dictionaries will be cleared.
"""
if where is None:
self._exist.data[:] = False
else:
where = self._get_where(where)
all_false = torch.tensor(False, dtype=torch.bool, device=self._exist.device).expand(self._exist.shape)
self._exist.data[:] = do_where(where, all_false, self._exist.data[:])
contains(self, key)
¶
Query whether or not the element(s) with the given key(s) exist.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
A boolean tensor indicating whether or not the element(s) with the specified key(s) exist. |
Source code in evotorch/tools/structures.py
def contains(self, key: Numbers) -> torch.Tensor:
"""
Query whether or not the element(s) with the given key(s) exist.
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
Returns:
A boolean tensor indicating whether or not the element(s) with the
specified key(s) exist.
"""
return self._exist[key]
divide_(self, key, value, where=None)
¶
Divide the existing values of slots with the given key(s).
Note that this operation does not change the existence flags of the
keys. In other words, if element(s) with key do not exist, then
they will still be flagged as non-existent after this operation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) that will be used as divisor(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Source code in evotorch/tools/structures.py
def divide_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Divide the existing values of slots with the given key(s).
Note that this operation does not change the existence flags of the
keys. In other words, if element(s) with `key` do not exist, then
they will still be flagged as non-existent after this operation.
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be used as divisor(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self._data.divide_(key, value, where)
get(self, key, default=None)
¶
Get the value(s) associated with the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
default |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally can be specified as the fallback value for when the element(s) with the given key(s) do not exist. |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The value(s) associated with the given key(s). |
Source code in evotorch/tools/structures.py
def get(self, key: Numbers, default: Optional[Numbers] = None) -> torch.Tensor:
"""
Get the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
default: Optionally can be specified as the fallback value for when
the element(s) with the given key(s) do not exist.
Returns:
The value(s) associated with the given key(s).
"""
if default is None:
return self._data[key]
else:
exist = self._exist[key]
default = self._get_value(default)
return do_where(exist, self._data[key], default)
multiply_(self, key, value, where=None)
¶
Multiply the existing values of slots with the given key(s).
Note that this operation does not change the existence flags of the
keys. In other words, if element(s) with key do not exist, then
they will still be flagged as non-existent after this operation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) that will be used as the multiplier(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Source code in evotorch/tools/structures.py
def multiply_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Multiply the existing values of slots with the given key(s).
Note that this operation does not change the existence flags of the
keys. In other words, if element(s) with `key` do not exist, then
they will still be flagged as non-existent after this operation.
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be used as the multiplier(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self._data.multiply_(key, value, where)
set_(self, key, value, where=None)
¶
Set the value(s) associated with the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The new value(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Source code in evotorch/tools/structures.py
def set_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Set the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The new value(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self._data.set_(key, value, where)
self._exist.set_(key, True, where)
subtract_(self, key, value, where=None)
¶
Subtract value(s) from existing values of slots with the given key(s).
Note that this operation does not change the existence flags of the
keys. In other words, if element(s) with key do not exist, then
they will still be flagged as non-existent after this operation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) that will be subtracted from existing value(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Source code in evotorch/tools/structures.py
def subtract_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Subtract value(s) from existing values of slots with the given key(s).
Note that this operation does not change the existence flags of the
keys. In other words, if element(s) with `key` do not exist, then
they will still be flagged as non-existent after this operation.
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be subtracted from existing value(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self._data.subtract_(key, value, where)
CList (Structure)
¶
Representation of a batchable, contiguous, variable-length list structure.
This CList structure works with a pre-allocated contiguous block of memory with a separately stored length. In the batched case, each batch item has its own length.
This structure supports negative indexing (meaning that -1 refers to the last item, -2 refers to the second last item, etc.).
Let us imagine that we need a list where each element has a shape (3,),
and our maximum length is 5.
Such a list could be instantiated via:
In its initial state, the list is empty, which can be visualized like:
_______________________________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | <unused> | <unused> | <unused> | <unused> | <unused> |
|________|__________|__________|__________|__________|__________|
We can add elements into our list like this:
After these two push operations, our list looks like this:
__________________________________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [1. 2. 3.] | [4. 5. 6] | <unused> | <unused> | <unused> |
|________|____________|___________|__________|__________|__________|
Here, lst[0] returns [1. 2. 3.] and lst[1] returns [4. 5. 6.].
A CList also supports negative indices, allowing lst[-1] to return
[4. 5. 6.] (the last element) and lst[-2] to return [1. 2. 3.]
(the second last element).
One can also create a batch of lists. Let us imagine that we wish to create a batch of lists such that the batch size is 4, length of an element is 3, and the maximum length is 5. Such a batch can be created as follows:
Our batch can be visualized like this:
__[ batch item 0 ]_____________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | <unused> | <unused> | <unused> | <unused> | <unused> |
|________|__________|__________|__________|__________|__________|
__[ batch item 1 ]_____________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | <unused> | <unused> | <unused> | <unused> | <unused> |
|________|__________|__________|__________|__________|__________|
__[ batch item 2 ]_____________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | <unused> | <unused> | <unused> | <unused> | <unused> |
|________|__________|__________|__________|__________|__________|
__[ batch item 3 ]_____________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | <unused> | <unused> | <unused> | <unused> | <unused> |
|________|__________|__________|__________|__________|__________|
Let us now add [1. 1. 1.] to the batch item 0, [2. 2. 2.] to the batch
item 1, and so on:
list_batch.append_(
torch.tensor(
[
[1.0, 1.0, 1.0],
[2.0, 2.0, 2.0],
[3.0, 3.0, 3.0],
[4.0, 4.0, 4.0],
]
)
)
After these operations, list_batch looks like this:
__[ batch item 0 ]_______________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [1. 1. 1.] | <unused> | <unused> | <unused> | <unused> |
|________|____________|__________|__________|__________|__________|
__[ batch item 1 ]_______________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [2. 2. 2.] | <unused> | <unused> | <unused> | <unused> |
|________|____________|__________|__________|__________|__________|
__[ batch item 2 ]_______________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [3. 3. 3.] | <unused> | <unused> | <unused> | <unused> |
|________|____________|__________|__________|__________|__________|
__[ batch item 3 ]_______________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [4. 4. 4.] | <unused> | <unused> | <unused> | <unused> |
|________|____________|__________|__________|__________|__________|
We can also use a boolean mask to add to only some of the lists within the batch:
list_batch.append_(
torch.tensor(
[
[5.0, 5.0, 5.0],
[6.0, 6.0, 6.0],
[7.0, 7.0, 7.0],
[8.0, 8.0, 8.0],
]
),
where=torch.tensor([True, False, False, True]),
)
which would update our batch of lists like this:
__[ batch item 0 ]_________________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [1. 1. 1.] | [5. 5. 5.] | <unused> | <unused> | <unused> |
|________|____________|____________|__________|__________|__________|
__[ batch item 1 ]_________________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [2. 2. 2.] | <unused> | <unused> | <unused> | <unused> |
|________|____________|____________|__________|__________|__________|
__[ batch item 2 ]_________________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [3. 3. 3.] | <unused> | <unused> | <unused> | <unused> |
|________|____________|____________|__________|__________|__________|
__[ batch item 3 ]_________________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [4. 4. 4.] | [8. 8. 8.] | <unused> | <unused> | <unused> |
|________|____________|____________|__________|__________|__________|
Please notice above how the batch items 1 and 2 were not modified because
their corresponding boolean values in the where tensor were given as
False.
After all these modifications we would get the following results:
>>> list_batch[0]
torch.tensor(
[[1. 1. 1.],
[2. 2. 2.],
[3. 3. 3.],
[4. 4. 4.]]
)
>>> list_batch[[1, 0, 0, 1]]
torch.tensor(
[[5. 5. 5.],
[2. 2. 2.],
[3. 3. 3.],
[8. 8. 8.]]
)
>>> list_batch[[-1, -1, -1, -1]]
torch.tensor(
[[5. 5. 5.],
[2. 2. 2.],
[3. 3. 3.],
[8. 8. 8.]]
)
Note that this CList structure also supports the ability to insert to the beginning, or to remove from the beginning. These operations internally shift the addresses for the beginning of the data within the underlying memory, and therefore, they are not any more costly than adding to or removing from the end of the list.
Source code in evotorch/tools/structures.py
class CList(Structure):
"""
Representation of a batchable, contiguous, variable-length list structure.
This CList structure works with a pre-allocated contiguous block of memory
with a separately stored length. In the batched case, each batch item
has its own length.
This structure supports negative indexing (meaning that -1 refers to the
last item, -2 refers to the second last item, etc.).
Let us imagine that we need a list where each element has a shape `(3,)`,
and our maximum length is 5.
Such a list could be instantiated via:
```python
lst = CList(3, max_length=5)
```
In its initial state, the list is empty, which can be visualized like:
```text
_______________________________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | <unused> | <unused> | <unused> | <unused> | <unused> |
|________|__________|__________|__________|__________|__________|
```
We can add elements into our list like this:
```python
lst.append_(torch.tensor([1.0, 2.0, 3.0]))
lst.append_(torch.tensor([4.0, 5.0, 6.0]))
```
After these two push operations, our list looks like this:
```text
__________________________________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [1. 2. 3.] | [4. 5. 6] | <unused> | <unused> | <unused> |
|________|____________|___________|__________|__________|__________|
```
Here, `lst[0]` returns `[1. 2. 3.]` and `lst[1]` returns `[4. 5. 6.]`.
A `CList` also supports negative indices, allowing `lst[-1]` to return
`[4. 5. 6.]` (the last element) and `lst[-2]` to return `[1. 2. 3.]`
(the second last element).
One can also create a batch of lists. Let us imagine that we wish to
create a batch of lists such that the batch size is 4, length of an
element is 3, and the maximum length is 5. Such a batch can be created
as follows:
```python
list_batch = CList(3, max_length=5, batch_size=4)
```
Our batch can be visualized like this:
```text
__[ batch item 0 ]_____________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | <unused> | <unused> | <unused> | <unused> | <unused> |
|________|__________|__________|__________|__________|__________|
__[ batch item 1 ]_____________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | <unused> | <unused> | <unused> | <unused> | <unused> |
|________|__________|__________|__________|__________|__________|
__[ batch item 2 ]_____________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | <unused> | <unused> | <unused> | <unused> | <unused> |
|________|__________|__________|__________|__________|__________|
__[ batch item 3 ]_____________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | <unused> | <unused> | <unused> | <unused> | <unused> |
|________|__________|__________|__________|__________|__________|
```
Let us now add `[1. 1. 1.]` to the batch item 0, `[2. 2. 2.]` to the batch
item 1, and so on:
```python
list_batch.append_(
torch.tensor(
[
[1.0, 1.0, 1.0],
[2.0, 2.0, 2.0],
[3.0, 3.0, 3.0],
[4.0, 4.0, 4.0],
]
)
)
```
After these operations, `list_batch` looks like this:
```text
__[ batch item 0 ]_______________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [1. 1. 1.] | <unused> | <unused> | <unused> | <unused> |
|________|____________|__________|__________|__________|__________|
__[ batch item 1 ]_______________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [2. 2. 2.] | <unused> | <unused> | <unused> | <unused> |
|________|____________|__________|__________|__________|__________|
__[ batch item 2 ]_______________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [3. 3. 3.] | <unused> | <unused> | <unused> | <unused> |
|________|____________|__________|__________|__________|__________|
__[ batch item 3 ]_______________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [4. 4. 4.] | <unused> | <unused> | <unused> | <unused> |
|________|____________|__________|__________|__________|__________|
```
We can also use a boolean mask to add to only some of the lists within
the batch:
```python
list_batch.append_(
torch.tensor(
[
[5.0, 5.0, 5.0],
[6.0, 6.0, 6.0],
[7.0, 7.0, 7.0],
[8.0, 8.0, 8.0],
]
),
where=torch.tensor([True, False, False, True]),
)
```
which would update our batch of lists like this:
```text
__[ batch item 0 ]_________________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [1. 1. 1.] | [5. 5. 5.] | <unused> | <unused> | <unused> |
|________|____________|____________|__________|__________|__________|
__[ batch item 1 ]_________________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [2. 2. 2.] | <unused> | <unused> | <unused> | <unused> |
|________|____________|____________|__________|__________|__________|
__[ batch item 2 ]_________________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [3. 3. 3.] | <unused> | <unused> | <unused> | <unused> |
|________|____________|____________|__________|__________|__________|
__[ batch item 3 ]_________________________________________________
| index | 0 | 1 | 2 | 3 | 4 |
| values | [4. 4. 4.] | [8. 8. 8.] | <unused> | <unused> | <unused> |
|________|____________|____________|__________|__________|__________|
```
Please notice above how the batch items 1 and 2 were not modified because
their corresponding boolean values in the `where` tensor were given as
`False`.
After all these modifications we would get the following results:
```text
>>> list_batch[0]
torch.tensor(
[[1. 1. 1.],
[2. 2. 2.],
[3. 3. 3.],
[4. 4. 4.]]
)
>>> list_batch[[1, 0, 0, 1]]
torch.tensor(
[[5. 5. 5.],
[2. 2. 2.],
[3. 3. 3.],
[8. 8. 8.]]
)
>>> list_batch[[-1, -1, -1, -1]]
torch.tensor(
[[5. 5. 5.],
[2. 2. 2.],
[3. 3. 3.],
[8. 8. 8.]]
)
```
Note that this CList structure also supports the ability to insert to the
beginning, or to remove from the beginning. These operations internally
shift the addresses for the beginning of the data within the underlying
memory, and therefore, they are not any more costly than adding to or
removing from the end of the list.
"""
def __init__(
self,
*size: Union[int, list, tuple],
max_length: int,
batch_size: Optional[Union[int, tuple, list]] = None,
batch_shape: Optional[Union[int, tuple, list]] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
verify: bool = True,
):
self._verify = bool(verify)
self._max_length = int(max_length)
self._data = CMemory(
*size,
num_keys=self._max_length,
batch_size=batch_size,
batch_shape=batch_shape,
dtype=dtype,
device=device,
verify=False,
)
self._begin, self._end = [
CMemory(
num_keys=1,
batch_size=batch_size,
batch_shape=batch_shape,
dtype=torch.int64,
device=device,
verify=False,
fill_with=-1,
)
for _ in range(2)
]
if "float" in str(self._data.dtype):
self._pop_fallback = float("nan")
else:
self._pop_fallback = 0
if self._begin.batch_ndim == 0:
self._all_zeros = torch.tensor(0, dtype=torch.int64, device=self._begin.device)
else:
self._all_zeros = torch.zeros(1, dtype=torch.int64, device=self._begin.device).expand(
self._begin.batch_shape
)
def _is_empty(self) -> torch.Tensor:
# return (self._begin[self._all_zeros] == -1) & (self._end[self._all_zeros] == -1)
return self._begin[self._all_zeros] == -1
def _has_one_element(self) -> torch.Tensor:
begin = self._begin[self._all_zeros]
end = self._end[self._all_zeros]
return (begin == end) & (begin >= 0)
def _is_full(self) -> torch.Tensor:
begin = self._begin[self._all_zeros]
end = self._end[self._all_zeros]
return ((end - begin) % self._max_length) == (self._max_length - 1)
@staticmethod
def _considering_where(other_mask: torch.Tensor, where: Optional[torch.Tensor]) -> torch.Tensor:
return other_mask if where is None else other_mask & where
def _get_info_for_adding_element(self, where: Optional[torch.Tensor]) -> _InfoForAddingElement:
is_empty = self._is_empty()
is_full = self._is_full()
to_be_declared_non_empty = self._considering_where(is_empty, where)
if self._verify:
invalid_move = self._considering_where(is_full, where)
if torch.any(invalid_move):
raise IndexError("Some of the queues are full, and therefore elements cannot be added to them")
valid_move = self._considering_where((~is_empty) & (~is_full), where)
return _InfoForAddingElement(valid_move=valid_move, to_be_declared_non_empty=to_be_declared_non_empty)
def _get_info_for_removing_element(self, where: Optional[torch.Tensor]) -> _InfoForRemovingElement:
is_empty = self._is_empty()
has_one_element = self._has_one_element()
if self._verify:
invalid_move = self._considering_where(is_empty, where)
if torch.any(invalid_move):
raise IndexError(
"Some of the queues are already empty, and therefore elements cannot be removed from them"
)
to_be_declared_empty = self._considering_where(has_one_element, where)
valid_move = self._considering_where((~is_empty) & (~has_one_element), where)
return _InfoForRemovingElement(valid_move=valid_move, to_be_declared_empty=to_be_declared_empty)
def _move_begin_forward(self, where: Optional[torch.Tensor]):
valid_move, to_be_declared_empty = self._get_info_for_removing_element(where)
self._begin.set_(self._all_zeros, -1, where=to_be_declared_empty)
self._end.set_(self._all_zeros, -1, where=to_be_declared_empty)
self._begin.add_circular_(self._all_zeros, 1, self._max_length, where=valid_move)
def _move_end_forward(self, where: Optional[torch.Tensor]):
valid_move, to_be_declared_non_empty = self._get_info_for_adding_element(where)
self._begin.set_(self._all_zeros, 0, where=to_be_declared_non_empty)
self._end.set_(self._all_zeros, 0, where=to_be_declared_non_empty)
self._end.add_circular_(self._all_zeros, 1, self._max_length, where=valid_move)
def _move_begin_backward(self, where: Optional[torch.Tensor]):
valid_move, to_be_declared_non_empty = self._get_info_for_adding_element(where)
self._begin.set_(self._all_zeros, 0, where=to_be_declared_non_empty)
self._end.set_(self._all_zeros, 0, where=to_be_declared_non_empty)
self._begin.add_circular_(self._all_zeros, -1, self._max_length, where=valid_move)
def _move_end_backward(self, where: Optional[torch.Tensor]):
valid_move, to_be_declared_empty = self._get_info_for_removing_element(where)
self._begin.set_(self._all_zeros, -1, where=to_be_declared_empty)
self._end.set_(self._all_zeros, -1, where=to_be_declared_empty)
self._end.add_circular_(self._all_zeros, -1, self._max_length, where=valid_move)
def _get_key(self, key: Numbers) -> torch.Tensor:
key = torch.as_tensor(key, dtype=torch.int64, device=self._data.device)
batch_shape = self._data.batch_shape
if key.shape != batch_shape:
if key.ndim == 0:
key = key.expand(self._data.batch_shape)
else:
raise ValueError(
f"Expected the keys of shape {batch_shape}, but received them in this shape: {key.shape}"
)
return key
def _is_underlying_key_valid(self, underlying_key: torch.Tensor) -> torch.Tensor:
within_valid_range = (underlying_key >= 0) & (underlying_key < self._max_length)
begin = self._begin[self._all_zeros]
end = self._end[self._all_zeros]
empty = self._is_empty()
non_empty = ~empty
larger_end = non_empty & (end > begin)
smaller_end = non_empty & (end < begin)
same_begin_end = (begin == end) & (~empty)
valid = within_valid_range & (
(same_begin_end & (underlying_key == begin))
| (larger_end & (underlying_key >= begin) & (underlying_key <= end))
| (smaller_end & ((underlying_key <= end) | (underlying_key >= begin)))
)
return valid
def _mod_underlying_key(self, underlying_key: torch.Tensor, *, verify: Optional[bool] = None) -> torch.Tensor:
verify = self._verify if verify is None else verify
if self._verify:
where_negative = underlying_key < 0
where_too_large = underlying_key >= self._max_length
underlying_key = underlying_key.clone()
underlying_key[where_negative] += self._max_length
underlying_key[where_too_large] -= self._max_length
else:
underlying_key = underlying_key % self._max_length
return underlying_key
def _get_underlying_key(
self,
key: Numbers,
*,
verify: Optional[bool] = None,
return_validity: bool = False,
where: Optional[torch.Tensor] = None,
) -> Union[torch.Tensor, tuple]:
if where is not None:
where = self._get_where(where)
verify = self._verify if verify is None else verify
key = self._get_key(key)
underlying_key_for_pos_index = self._begin[self._all_zeros] + key
underlying_key_for_neg_index = self._end[self._all_zeros] + key + 1
underlying_key = torch.where(key >= 0, underlying_key_for_pos_index, underlying_key_for_neg_index)
underlying_key = self._mod_underlying_key(underlying_key, verify=verify)
if verify or return_validity:
valid = self._is_underlying_key_valid(underlying_key)
else:
valid = None
if verify:
okay = valid if where is None else valid | (~where)
if not torch.all(okay):
raise IndexError("Encountered invalid index/indices")
if return_validity:
return underlying_key, valid
else:
return underlying_key
def get(self, key: Numbers, default: Optional[Numbers] = None) -> torch.Tensor:
"""
Get the value(s) from the specified element(s).
Args:
key: The index/indices pointing to the element(s) whose value(s)
is/are queried.
default: Default value(s) to be returned for when the specified
index/indices are invalid and/or out of range.
Returns:
The value(s) stored by the element(s).
"""
if default is None:
underlying_key = self._get_underlying_key(key)
return self._data[underlying_key]
else:
default = self._data._get_value(default)
underlying_key, valid_key = self._get_underlying_key(key, verify=False, return_validity=True)
return do_where(valid_key, self._data[underlying_key % self._max_length], default)
def __getitem__(self, key: Numbers) -> torch.Tensor:
"""
Get the value(s) from the specified element(s).
Args:
key: The index/indices pointing to the element(s) whose value(s)
is/are queried.
Returns:
The value(s) stored by the element(s).
"""
return self.get(key)
def _apply_modification_method(
self, method_name: str, key: Numbers, value: Numbers, where: Optional[Numbers] = None
):
underlying_key = self._get_underlying_key(key, where=where)
getattr(self._data, method_name)(underlying_key, value, where)
def set_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Set the element(s) addressed to by the given key(s).
Args:
key: The index/indices tensor.
value: The new value(s).
where: Optionally a boolean mask. When provided, only the elements
whose corresponding mask value(s) is/are True will be subject
to modification.
"""
self._apply_modification_method("set_", key, value, where)
def __setitem__(self, key: Numbers, value: Numbers):
"""
Set the element(s) addressed to by the given key(s).
Args:
key: The index/indices tensor.
value: The new value(s).
"""
self._apply_modification_method("set_", key, value)
def add_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Add to the element(s) addressed to by the given key(s).
Please note that the word "add" is used in the arithmetic sense
(i.e. in the sense of performing addition). For putting a new
element into this list, please see the method `append_(...)`.
Args:
key: The index/indices tensor.
value: The value(s) that will be added onto the existing
element(s).
where: Optionally a boolean mask. When provided, only the elements
whose corresponding mask value(s) is/are True will be subject
to modification.
"""
self._apply_modification_method("add_", key, value, where)
def subtract_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Subtract from the element(s) addressed to by the given key(s).
Args:
key: The index/indices tensor.
value: The value(s) that will be subtracted from the existing
element(s).
where: Optionally a boolean mask. When provided, only the elements
whose corresponding mask value(s) is/are True will be subject
to modification.
"""
self._apply_modification_method("subtract_", key, value, where)
def multiply_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Multiply the element(s) addressed to by the given key(s).
Args:
key: The index/indices tensor.
value: The value(s) that will be used as the multiplier(s) on the
existing element(s).
where: Optionally a boolean mask. When provided, only the elements
whose corresponding mask value(s) is/are True will be subject
to modification.
"""
self._apply_modification_method("multiply_", key, value, where)
def divide_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Divide the element(s) addressed to by the given key(s).
Args:
key: The index/indices tensor.
value: The value(s) that will be used as the divisor(s) on the
existing element(s).
where: Optionally a boolean mask. When provided, only the elements
whose corresponding mask value(s) is/are True will be subject
to modification.
"""
self._apply_modification_method("divide_", key, value, where)
def append_(self, value: Numbers, where: Optional[Numbers] = None):
"""
Add new item(s) to the end(s) of the list(s).
The length(s) of the updated list(s) will increase by 1.
Args:
value: The element that will be added to the list.
In the non-batched case, this element is expected as a tensor
whose shape matches `value_shape`.
In the batched case, this value is expected as a batch of
elements with extra leftmost dimensions (those extra leftmost
dimensions being expressed by `batch_shape`).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then additions will happen only
on the lists whose corresponding mask values are True.
"""
where = None if where is None else self._get_where(where)
self._move_end_forward(where)
self.set_(-1, value, where=where)
def push_(self, value: Numbers, where: Optional[Numbers] = None):
"""
Alias for the method `append_(...)`.
We provide this alternative name so that users who wish to use this
CList structure like a stack will be able to use familiar terminology.
"""
return self.append_(value, where=where)
def appendleft_(self, value: Numbers, where: Optional[Numbers] = None):
"""
Add new item(s) to the beginning point(s) of the list(s).
The length(s) of the updated list(s) will increase by 1.
Args:
value: The element that will be added to the list.
In the non-batched case, this element is expected as a tensor
whose shape matches `value_shape`.
In the batched case, this value is expected as a batch of
elements with extra leftmost dimensions (those extra leftmost
dimensions being expressed by `batch_shape`).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then additions will happen only
on the lists whose corresponding mask values are True.
"""
where = None if where is None else self._get_where(where)
self._move_begin_backward(where)
self.set_(0, value, where=where)
def pop_(self, where: Optional[Numbers] = None):
"""
Pop the last item(s) from the ending point(s) list(s).
The length(s) of the updated list(s) will decrease by 1.
Args:
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then the pop operations will happen
only on the lists whose corresponding mask values are True.
Returns:
The popped item(s).
"""
where = None if where is None else self._get_where(where)
result = self.get(-1, default=self._pop_fallback)
self._move_end_backward(where)
return result
def popleft_(self, where: Optional[Numbers] = None):
"""
Pop the last item(s) from the beginning point(s) list(s).
The length(s) of the updated list(s) will decrease by 1.
Args:
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then the pop operations will happen
only on the lists whose corresponding mask values are True.
Returns:
The popped item(s).
"""
where = None if where is None else self._get_where(where)
result = self.get(0, default=self._pop_fallback)
self._move_begin_forward(where)
return result
def clear(self, where: Optional[torch.Tensor] = None):
"""
Clear the list(s).
In the context of this data structure, to "clear" means to reduce their
lengths to 0.
Args:
where: Optionally a boolean tensor, specifying which lists within
the batch will be cleared. If this argument is omitted (i.e.
left as None), then all of the lists will be cleared.
"""
if where is None:
self._begin.data[:] = -1
self._end.data[:] = -1
else:
where = self._get_where(where)
all_minus_ones = torch.tensor(-1, dtype=torch.int64, device=self._begin.device).expand(self._begin.shape)
self._begin.data[:] = do_where(where, all_minus_ones, self._begin.data)
self._end.data[:] = do_where(where, all_minus_ones, self._end.data)
@property
def data(self) -> torch.Tensor:
"""
The underlying tensor which stores all the data
"""
return self._data.data
@property
def length(self) -> torch.Tensor:
"""
The length(s) of the list(s)
"""
is_empty = self._is_empty()
is_full = self._is_full()
result = ((self._end[self._all_zeros] - self._begin[self._all_zeros]) % self._max_length) + 1
result[is_empty] = 0
result[is_full] = self._max_length
return result
@property
def max_length(self) -> int:
"""
Maximum length for the list(s)
"""
return self._max_length
data: Tensor
property
readonly
¶
The underlying tensor which stores all the data
length: Tensor
property
readonly
¶
The length(s) of the list(s)
max_length: int
property
readonly
¶
Maximum length for the list(s)
__getitem__(self, key)
special
¶
Get the value(s) from the specified element(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
The index/indices pointing to the element(s) whose value(s) is/are queried. |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The value(s) stored by the element(s). |
Source code in evotorch/tools/structures.py
__setitem__(self, key, value)
special
¶
Set the element(s) addressed to by the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
The index/indices tensor. |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The new value(s). |
required |
add_(self, key, value, where=None)
¶
Add to the element(s) addressed to by the given key(s).
Please note that the word "add" is used in the arithmetic sense
(i.e. in the sense of performing addition). For putting a new
element into this list, please see the method append_(...).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
The index/indices tensor. |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) that will be added onto the existing element(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask. When provided, only the elements whose corresponding mask value(s) is/are True will be subject to modification. |
None |
Source code in evotorch/tools/structures.py
def add_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Add to the element(s) addressed to by the given key(s).
Please note that the word "add" is used in the arithmetic sense
(i.e. in the sense of performing addition). For putting a new
element into this list, please see the method `append_(...)`.
Args:
key: The index/indices tensor.
value: The value(s) that will be added onto the existing
element(s).
where: Optionally a boolean mask. When provided, only the elements
whose corresponding mask value(s) is/are True will be subject
to modification.
"""
self._apply_modification_method("add_", key, value, where)
append_(self, value, where=None)
¶
Add new item(s) to the end(s) of the list(s).
The length(s) of the updated list(s) will increase by 1.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The element that will be added to the list.
In the non-batched case, this element is expected as a tensor
whose shape matches |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Source code in evotorch/tools/structures.py
def append_(self, value: Numbers, where: Optional[Numbers] = None):
"""
Add new item(s) to the end(s) of the list(s).
The length(s) of the updated list(s) will increase by 1.
Args:
value: The element that will be added to the list.
In the non-batched case, this element is expected as a tensor
whose shape matches `value_shape`.
In the batched case, this value is expected as a batch of
elements with extra leftmost dimensions (those extra leftmost
dimensions being expressed by `batch_shape`).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then additions will happen only
on the lists whose corresponding mask values are True.
"""
where = None if where is None else self._get_where(where)
self._move_end_forward(where)
self.set_(-1, value, where=where)
appendleft_(self, value, where=None)
¶
Add new item(s) to the beginning point(s) of the list(s).
The length(s) of the updated list(s) will increase by 1.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The element that will be added to the list.
In the non-batched case, this element is expected as a tensor
whose shape matches |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Source code in evotorch/tools/structures.py
def appendleft_(self, value: Numbers, where: Optional[Numbers] = None):
"""
Add new item(s) to the beginning point(s) of the list(s).
The length(s) of the updated list(s) will increase by 1.
Args:
value: The element that will be added to the list.
In the non-batched case, this element is expected as a tensor
whose shape matches `value_shape`.
In the batched case, this value is expected as a batch of
elements with extra leftmost dimensions (those extra leftmost
dimensions being expressed by `batch_shape`).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then additions will happen only
on the lists whose corresponding mask values are True.
"""
where = None if where is None else self._get_where(where)
self._move_begin_backward(where)
self.set_(0, value, where=where)
clear(self, where=None)
¶
Clear the list(s).
In the context of this data structure, to "clear" means to reduce their lengths to 0.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
where |
Optional[torch.Tensor] |
Optionally a boolean tensor, specifying which lists within the batch will be cleared. If this argument is omitted (i.e. left as None), then all of the lists will be cleared. |
None |
Source code in evotorch/tools/structures.py
def clear(self, where: Optional[torch.Tensor] = None):
"""
Clear the list(s).
In the context of this data structure, to "clear" means to reduce their
lengths to 0.
Args:
where: Optionally a boolean tensor, specifying which lists within
the batch will be cleared. If this argument is omitted (i.e.
left as None), then all of the lists will be cleared.
"""
if where is None:
self._begin.data[:] = -1
self._end.data[:] = -1
else:
where = self._get_where(where)
all_minus_ones = torch.tensor(-1, dtype=torch.int64, device=self._begin.device).expand(self._begin.shape)
self._begin.data[:] = do_where(where, all_minus_ones, self._begin.data)
self._end.data[:] = do_where(where, all_minus_ones, self._end.data)
divide_(self, key, value, where=None)
¶
Divide the element(s) addressed to by the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
The index/indices tensor. |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) that will be used as the divisor(s) on the existing element(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask. When provided, only the elements whose corresponding mask value(s) is/are True will be subject to modification. |
None |
Source code in evotorch/tools/structures.py
def divide_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Divide the element(s) addressed to by the given key(s).
Args:
key: The index/indices tensor.
value: The value(s) that will be used as the divisor(s) on the
existing element(s).
where: Optionally a boolean mask. When provided, only the elements
whose corresponding mask value(s) is/are True will be subject
to modification.
"""
self._apply_modification_method("divide_", key, value, where)
get(self, key, default=None)
¶
Get the value(s) from the specified element(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
The index/indices pointing to the element(s) whose value(s) is/are queried. |
required |
default |
Union[numbers.Number, Iterable[numbers.Number]] |
Default value(s) to be returned for when the specified index/indices are invalid and/or out of range. |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The value(s) stored by the element(s). |
Source code in evotorch/tools/structures.py
def get(self, key: Numbers, default: Optional[Numbers] = None) -> torch.Tensor:
"""
Get the value(s) from the specified element(s).
Args:
key: The index/indices pointing to the element(s) whose value(s)
is/are queried.
default: Default value(s) to be returned for when the specified
index/indices are invalid and/or out of range.
Returns:
The value(s) stored by the element(s).
"""
if default is None:
underlying_key = self._get_underlying_key(key)
return self._data[underlying_key]
else:
default = self._data._get_value(default)
underlying_key, valid_key = self._get_underlying_key(key, verify=False, return_validity=True)
return do_where(valid_key, self._data[underlying_key % self._max_length], default)
multiply_(self, key, value, where=None)
¶
Multiply the element(s) addressed to by the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
The index/indices tensor. |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) that will be used as the multiplier(s) on the existing element(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask. When provided, only the elements whose corresponding mask value(s) is/are True will be subject to modification. |
None |
Source code in evotorch/tools/structures.py
def multiply_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Multiply the element(s) addressed to by the given key(s).
Args:
key: The index/indices tensor.
value: The value(s) that will be used as the multiplier(s) on the
existing element(s).
where: Optionally a boolean mask. When provided, only the elements
whose corresponding mask value(s) is/are True will be subject
to modification.
"""
self._apply_modification_method("multiply_", key, value, where)
pop_(self, where=None)
¶
Pop the last item(s) from the ending point(s) list(s).
The length(s) of the updated list(s) will decrease by 1.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Returns:
| Type | Description |
|---|---|
The popped item(s). |
Source code in evotorch/tools/structures.py
def pop_(self, where: Optional[Numbers] = None):
"""
Pop the last item(s) from the ending point(s) list(s).
The length(s) of the updated list(s) will decrease by 1.
Args:
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then the pop operations will happen
only on the lists whose corresponding mask values are True.
Returns:
The popped item(s).
"""
where = None if where is None else self._get_where(where)
result = self.get(-1, default=self._pop_fallback)
self._move_end_backward(where)
return result
popleft_(self, where=None)
¶
Pop the last item(s) from the beginning point(s) list(s).
The length(s) of the updated list(s) will decrease by 1.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Returns:
| Type | Description |
|---|---|
The popped item(s). |
Source code in evotorch/tools/structures.py
def popleft_(self, where: Optional[Numbers] = None):
"""
Pop the last item(s) from the beginning point(s) list(s).
The length(s) of the updated list(s) will decrease by 1.
Args:
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then the pop operations will happen
only on the lists whose corresponding mask values are True.
Returns:
The popped item(s).
"""
where = None if where is None else self._get_where(where)
result = self.get(0, default=self._pop_fallback)
self._move_begin_forward(where)
return result
push_(self, value, where=None)
¶
Alias for the method append_(...).
We provide this alternative name so that users who wish to use this
CList structure like a stack will be able to use familiar terminology.
Source code in evotorch/tools/structures.py
set_(self, key, value, where=None)
¶
Set the element(s) addressed to by the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
The index/indices tensor. |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The new value(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask. When provided, only the elements whose corresponding mask value(s) is/are True will be subject to modification. |
None |
Source code in evotorch/tools/structures.py
def set_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Set the element(s) addressed to by the given key(s).
Args:
key: The index/indices tensor.
value: The new value(s).
where: Optionally a boolean mask. When provided, only the elements
whose corresponding mask value(s) is/are True will be subject
to modification.
"""
self._apply_modification_method("set_", key, value, where)
subtract_(self, key, value, where=None)
¶
Subtract from the element(s) addressed to by the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
The index/indices tensor. |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) that will be subtracted from the existing element(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask. When provided, only the elements whose corresponding mask value(s) is/are True will be subject to modification. |
None |
Source code in evotorch/tools/structures.py
def subtract_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Subtract from the element(s) addressed to by the given key(s).
Args:
key: The index/indices tensor.
value: The value(s) that will be subtracted from the existing
element(s).
where: Optionally a boolean mask. When provided, only the elements
whose corresponding mask value(s) is/are True will be subject
to modification.
"""
self._apply_modification_method("subtract_", key, value, where)
CMemory
¶
Representation of a batchable contiguous memory.
This container can be seen as a batchable primitive dictionary where the keys are allowed either as integers or as tuples of integers. Please also note that, a memory block for each key is already allocated, meaning that unlike a dictionary of Python, each key already exists and is associated with a tensor.
Let us consider an example where we have 5 keys, and each key is associated with a tensor of length 7. Such a memory could be allocated like this:
Our allocated memory can be visualized as follows:
_______________________________________
| key 0 -> [ empty tensor of length 7 ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
Let us now sample a Gaussian noise and put it into the 0-th slot:
which results in:
_________________________________________
| key 0 -> [ Gaussian noise of length 7 ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_________________________________________|
Let us now consider another example where we deal with not a single CMemory, but with a CMemory batch. For the sake of this example, let us say that our desired batch size is 3. The allocation of such a batch would be as follows:
Our memory batch can be visualized like this:
__[ batch item 0 ]_____________________
| key 0 -> [ empty tensor of length 7 ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ empty tensor of length 7 ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ empty tensor of length 7 ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
If we wish to set the 0-th element of each batch item, we could do:
memory_batch[0] = torch.tensor(
[
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0],
[2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0],
],
)
and the result would be:
__[ batch item 0 ]_____________________
| key 0 -> [ 0. 0. 0. 0. 0. 0. 0. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ 1. 1. 1. 1. 1. 1. 1. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ 2. 2. 2. 2. 2. 2. 2. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
Continuing from the same example, if we wish to set the slot with key 1 in the 0th batch item, slot with key 2 in the 1st batch item, and slot with key 3 in the 2nd batch item, all in one go, we could do:
# Longer version: memory_batch[torch.tensor([1, 2, 3])] = ...
memory_batch[[1, 2, 3]] = torch.tensor(
[
[5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0],
[6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0],
[7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0],
],
)
Our updated memory batch would then look like this:
__[ batch item 0 ]_____________________
| key 0 -> [ 0. 0. 0. 0. 0. 0. 0. ] |
| key 1 -> [ 5. 5. 5. 5. 5. 5. 5. ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ 1. 1. 1. 1. 1. 1. 1. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ 6. 6. 6. 6. 6. 6. 6. ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ 2. 2. 2. 2. 2. 2. 2. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ 7. 7. 7. 7. 7. 7. 7. ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
Conditional modifications via boolean masks is also supported.
For example, the following update on our memory_batch:
memory_batch.set_(
[4, 3, 1],
torch.tensor(
[
[8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0],
[9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0],
[10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0],
]
),
where=[True, True, False], # or: where=torch.tensor([True,True,False]),
)
would result in:
__[ batch item 0 ]_____________________
| key 0 -> [ 0. 0. 0. 0. 0. 0. 0. ] |
| key 1 -> [ 5. 5. 5. 5. 5. 5. 5. ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ 8. 8. 8. 8. 8. 8. 8. ] |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ 1. 1. 1. 1. 1. 1. 1. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ 6. 6. 6. 6. 6. 6. 6. ] |
| key 3 -> [ 9. 9. 9. 9. 9. 9. 9. ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ 2. 2. 2. 2. 2. 2. 2. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ 7. 7. 7. 7. 7. 7. 7. ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
Please notice above that the slot with key 1 of the batch item 2 was not modified because its corresponding mask value was given as False.
Source code in evotorch/tools/structures.py
class CMemory:
"""
Representation of a batchable contiguous memory.
This container can be seen as a batchable primitive dictionary where the
keys are allowed either as integers or as tuples of integers. Please also
note that, a memory block for each key is already allocated, meaning that
unlike a dictionary of Python, each key already exists and is associated
with a tensor.
Let us consider an example where we have 5 keys, and each key is associated
with a tensor of length 7. Such a memory could be allocated like this:
```python
memory = CMemory(7, num_keys=5)
```
Our allocated memory can be visualized as follows:
```text
_______________________________________
| key 0 -> [ empty tensor of length 7 ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
```
Let us now sample a Gaussian noise and put it into the 0-th slot:
```python
memory[0] = torch.randn(7) # or: memory[torch.tensor(0)] = torch.randn(7)
```
which results in:
```text
_________________________________________
| key 0 -> [ Gaussian noise of length 7 ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_________________________________________|
```
Let us now consider another example where we deal with not a single CMemory,
but with a CMemory batch. For the sake of this example, let us say that our
desired batch size is 3. The allocation of such a batch would be as
follows:
```python
memory_batch = CMemory(7, num_keys=5, batch_size=3)
```
Our memory batch can be visualized like this:
```text
__[ batch item 0 ]_____________________
| key 0 -> [ empty tensor of length 7 ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ empty tensor of length 7 ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ empty tensor of length 7 ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
```
If we wish to set the 0-th element of each batch item, we could do:
```python
memory_batch[0] = torch.tensor(
[
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0],
[2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0],
],
)
```
and the result would be:
```text
__[ batch item 0 ]_____________________
| key 0 -> [ 0. 0. 0. 0. 0. 0. 0. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ 1. 1. 1. 1. 1. 1. 1. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ 2. 2. 2. 2. 2. 2. 2. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
```
Continuing from the same example, if we wish to set the slot with key 1
in the 0th batch item, slot with key 2 in the 1st batch item, and
slot with key 3 in the 2nd batch item, all in one go, we could do:
```python
# Longer version: memory_batch[torch.tensor([1, 2, 3])] = ...
memory_batch[[1, 2, 3]] = torch.tensor(
[
[5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0],
[6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 6.0],
[7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0],
],
)
```
Our updated memory batch would then look like this:
```text
__[ batch item 0 ]_____________________
| key 0 -> [ 0. 0. 0. 0. 0. 0. 0. ] |
| key 1 -> [ 5. 5. 5. 5. 5. 5. 5. ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ 1. 1. 1. 1. 1. 1. 1. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ 6. 6. 6. 6. 6. 6. 6. ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ 2. 2. 2. 2. 2. 2. 2. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ 7. 7. 7. 7. 7. 7. 7. ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
```
Conditional modifications via boolean masks is also supported.
For example, the following update on our `memory_batch`:
```python
memory_batch.set_(
[4, 3, 1],
torch.tensor(
[
[8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0],
[9.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0],
[10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0],
]
),
where=[True, True, False], # or: where=torch.tensor([True,True,False]),
)
```
would result in:
```text
__[ batch item 0 ]_____________________
| key 0 -> [ 0. 0. 0. 0. 0. 0. 0. ] |
| key 1 -> [ 5. 5. 5. 5. 5. 5. 5. ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ empty tensor of length 7 ] |
| key 4 -> [ 8. 8. 8. 8. 8. 8. 8. ] |
|_______________________________________|
__[ batch item 1 ]_____________________
| key 0 -> [ 1. 1. 1. 1. 1. 1. 1. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ 6. 6. 6. 6. 6. 6. 6. ] |
| key 3 -> [ 9. 9. 9. 9. 9. 9. 9. ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
__[ batch item 2 ]_____________________
| key 0 -> [ 2. 2. 2. 2. 2. 2. 2. ] |
| key 1 -> [ empty tensor of length 7 ] |
| key 2 -> [ empty tensor of length 7 ] |
| key 3 -> [ 7. 7. 7. 7. 7. 7. 7. ] |
| key 4 -> [ empty tensor of length 7 ] |
|_______________________________________|
```
Please notice above that the slot with key 1 of the batch item 2 was not
modified because its corresponding mask value was given as False.
"""
def __init__(
self,
*size: Union[int, tuple, list],
num_keys: Union[int, tuple, list],
key_offset: Optional[Union[int, tuple, list]] = None,
batch_size: Optional[Union[int, tuple, list]] = None,
batch_shape: Optional[Union[int, tuple, list]] = None,
fill_with: Optional[Numbers] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
verify: bool = True,
):
"""
`__init__(...)`: Initialize the CMemory.
Args:
size: Size of a tensor associated with a key, expected as an
integer, or as multiple positional arguments (each positional
argument being an integer), or as a tuple of integers.
num_keys: How many keys (and therefore how many slots) will the
memory have. If given as an integer `n`, then there will be `n`
slots in the memory, and to access a slot one will need to use
an integer key `k` (where, by default, the minimum acceptable
`k` is 0 and the maximum acceptable `k` is `n-1`).
If given as a tuple of integers, then the number of slots in
the memory will be computed as the product of all the integers
in the tuple, and a key will be expected as a tuple.
For example, when `num_keys` is `(3, 5)`, there will be 15
slots in the memory (where, by default, the minimum acceptable
key will be `(0, 0)` and the maximum acceptable key will be
`(2, 4)`.
key_offset: Optionally can be used to shift the integer values of
the keys. For example, if `num_keys` is 10, then, by default,
the minimum key is 0 and the maximum key is 9. But together
with `num_keys=10`, if `key_offset` is given as 1, then the
minimum key will be 1 and the maximum key will be 10.
This argument can also be used together with a tuple-valued
`num_keys`. For example, with `num_keys` set as `(3, 5)`,
if `key_offset` is given as 1, then the minimum key value
will be `(1, 1)` (instead of `(0, 0)`) and the maximum key
value will be `(3, 5)` (instead of `(2, 4)`).
Also, with a tuple-valued `num_keys`, `key_offset` can be
given as a tuple, to shift the key values differently for each
item in the tuple.
batch_size: If given as None, then this memory will not be batched.
If given as an integer `n`, then this object will represent
a contiguous batch containing `n` memory blocks.
If given as a tuple `(size0, size1, ...)`, then this object
will represent a contiguous batch of memory, shape of this
batch being determined by the given tuple.
batch_shape: Alias for the argument `batch_size`.
fill_with: Optionally a numeric value using which the values will
be initialized. If no initialization is needed, then this
argument can be left as None.
dtype: The `dtype` of the memory tensor.
device: The device on which the memory will be allocated.
verify: If True, then explicit checks will be done to verify
that there are no indexing errors. Can be set as False for
performance.
"""
self._dtype = torch.float32 if dtype is None else to_torch_dtype(dtype)
self._device = torch.device("cpu") if device is None else torch.device(device)
self._verify = bool(verify)
if isinstance(num_keys, (list, tuple)):
if len(num_keys) < 2:
raise RuntimeError(
f"When expressed via a list or a tuple, the length of `num_keys` must be at least 2."
f" However, the encountered `num_keys` is {repr(num_keys)}, whose length is {len(num_keys)}."
)
self._multi_key = True
self._num_keys = tuple((int(n) for n in num_keys))
self._internal_key_shape = torch.Size(self._num_keys)
else:
self._multi_key = False
self._num_keys = int(num_keys)
self._internal_key_shape = torch.Size([self._num_keys])
self._internal_key_ndim = len(self._internal_key_shape)
if key_offset is None:
self._key_offset = None
else:
if self._multi_key:
if isinstance(key_offset, (list, tuple)):
key_offset = [int(n) for n in key_offset]
if len(key_offset) != len(self._num_keys):
raise RuntimeError("The length of `key_offset` does not match the length of `num_keys`")
else:
key_offset = [int(key_offset) for _ in range(len(self._num_keys))]
self._key_offset = torch.as_tensor(key_offset, dtype=torch.int64, device=self._device)
else:
if isinstance(key_offset, (list, tuple)):
raise RuntimeError("`key_offset` cannot be a sequence of integers when `num_keys` is a scalar")
else:
self._key_offset = torch.as_tensor(int(key_offset), dtype=torch.int64, device=self._device)
if self._verify:
if self._multi_key:
self._min_key = torch.zeros(len(self._num_keys), dtype=torch.int64, device=self._device)
self._max_key = torch.tensor(list(self._num_keys), dtype=torch.int64, device=self._device) - 1
else:
self._min_key = torch.tensor(0, dtype=torch.int64, device=self._device)
self._max_key = torch.tensor(self._num_keys - 1, dtype=torch.int64, device=self._device)
if self._key_offset is not None:
self._min_key += self._key_offset
self._max_key += self._key_offset
else:
self._min_key = None
self._max_key = None
nsize = len(size)
if nsize == 0:
self._value_shape = torch.Size([])
elif nsize == 1:
if isinstance(size[0], (tuple, list)):
self._value_shape = torch.Size((int(n) for n in size[0]))
else:
self._value_shape = torch.Size([int(size[0])])
else:
self._value_shape = torch.Size((int(n) for n in size))
self._value_ndim = len(self._value_shape)
if (batch_size is None) and (batch_shape is None):
batch_size = None
elif (batch_size is not None) and (batch_shape is None):
pass
elif (batch_size is None) and (batch_shape is not None):
batch_size = batch_shape
else:
raise RuntimeError(
"Encountered both `batch_shape` and `batch_size` at the same time."
" None of them or one of them can be accepted, but not both of them at the same time."
)
if batch_size is None:
self._batch_shape = torch.Size([])
elif isinstance(batch_size, (tuple, list)):
self._batch_shape = torch.Size((int(n) for n in batch_size))
else:
self._batch_shape = torch.Size([int(batch_size)])
self._batch_ndim = len(self._batch_shape)
self._for_all_batches = tuple(
(
torch.arange(self._batch_shape[i], dtype=torch.int64, device=self._device)
for i in range(self._batch_ndim)
)
)
self._data = torch.empty(
self._batch_shape + self._internal_key_shape + self._value_shape,
dtype=(self._dtype),
device=(self._device),
)
if fill_with is not None:
self._data[:] = fill_with
@property
def _is_dtype_bool(self) -> bool:
return self._data.dtype is torch.bool
def _key_must_be_valid(self, key: torch.Tensor) -> torch.Tensor:
lb_satisfied = key >= self._min_key
ub_satisfied = key <= self._max_key
all_satisfied = lb_satisfied & ub_satisfied
if not torch.all(all_satisfied):
raise KeyError("Encountered invalid key(s)")
def _get_key(self, key: Numbers, where: Optional[torch.Tensor] = None) -> torch.Tensor:
key = torch.as_tensor(key, dtype=torch.int64, device=self._data.device)
expected_shape = self.batch_shape + self.key_shape
if key.shape == expected_shape:
result = key
elif key.shape == self.key_shape:
result = key.expand(expected_shape)
else:
raise RuntimeError(f"The key tensor has an incompatible shape: {key.shape}")
if where is not None:
min_key = (
torch.tensor(0, dtype=torch.int64, device=self._data.device) if self._min_key is None else self._min_key
)
key = do_where(where, key, min_key.expand(expected_shape))
if self._verify:
self._key_must_be_valid(key)
return result
def _get_value(self, value: Numbers) -> torch.Tensor:
value = torch.as_tensor(value, dtype=self._data.dtype, device=self._data.device)
expected_shape = self.batch_shape + self.value_shape
if value.shape == expected_shape:
return value
elif (value.ndim == 0) or (value.shape == self.value_shape):
return value.expand(expected_shape)
else:
raise RuntimeError(f"The value tensor has an incompatible shape: {value.shape}")
return value
def _get_where(self, where: Numbers) -> torch.Tensor:
where = torch.as_tensor(where, dtype=torch.bool, device=self._data.device)
if where.shape != self.batch_shape:
raise RuntimeError(
f"The boolean mask `where` has an incompatible shape: {where.shape}."
f" Acceptable shape is: {self.batch_shape}"
)
return where
def prepare_key_tensor(self, key: Numbers) -> torch.Tensor:
"""
Return the tensor-counterpart of a key.
Args:
key: A key which can be a sequence of integers or a PyTorch tensor
with an integer dtype.
The shape of the given key must conform with the `key_shape`
of this memory object.
To address to a different key in each batch item, the shape of
the given key can also have extra leftmost dimensions expressed
by `batch_shape`.
Returns:
A copy of the key that is converted to PyTorch tensor.
"""
return self._get_key(key)
def prepare_value_tensor(self, value: Numbers) -> torch.Tensor:
"""
Return the tensor-counterpart of a value.
Args:
value: A value that can be a numeric sequence or a PyTorch tensor.
The shape of the given value must conform with the
`value_shape` of this memory object.
To express a different value for each batch item, the shape of
the given value can also have extra leftmost dimensions
expressed by `value_shape`.
Returns:
A copy of the given value(s), converted to PyTorch tensor.
"""
return self._get_value(value)
def prepare_where_tensor(self, where: Numbers) -> torch.Tensor:
"""
Return the tensor-counterpart of a boolean mask.
Args:
where: A boolean mask expressed as a sequence of bools or as a
boolean PyTorch tensor.
The shape of the given mask must conform with the batch shape
that is expressed by the property `batch_shape`.
Returns:
A copy of the boolean mask, converted to PyTorch tensor.
"""
return self._get_where(where)
def _get_address(self, key: Numbers, where: Optional[torch.Tensor] = None) -> tuple:
key = self._get_key(key, where=where)
if self._key_offset is not None:
key = key - self._key_offset
if self._multi_key:
keys = tuple((key[..., j] for j in range(self._internal_key_ndim)))
return self._for_all_batches + keys
else:
return self._for_all_batches + (key,)
def get(self, key: Numbers) -> torch.Tensor:
"""
Get the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
Returns:
The value(s) associated with the given key(s).
"""
address = self._get_address(key)
return self._data[address]
def set_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Set the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The new value(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
where = None if where is None else self._get_where(where)
address = self._get_address(key, where=where)
value = self._get_value(value)
if where is None:
self._data[address] = value
else:
old_value = self._data[address]
new_value = value
self._data[address] = do_where(where, new_value, old_value)
def add_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Add value(s) onto the existing values of slots with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be added onto the existing value(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
where = None if where is None else self._get_where(where)
address = self._get_address(key, where=where)
value = self._get_value(value)
if where is None:
if self._is_dtype_bool:
self._data[address] |= value
else:
self._data[address] += value
else:
if self._is_dtype_bool:
mask_shape = self._batch_shape + tuple((1 for _ in range(self._value_ndim)))
self._data[address] |= value & where.reshape(mask_shape)
else:
self._data[address] += do_where(where, value, torch.tensor(0, dtype=value.dtype, device=value.device))
def add_circular_(self, key: Numbers, value: Numbers, mod: Numbers, where: Optional[Numbers] = None):
"""
Increase the values of the specified slots in a circular manner.
This operation combines the add and modulo operations.
Circularly adding `value` onto `x` with a modulo `mod` means:
`x = (x + value) % mod`.
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be added onto the existing value(s).
mod: After the raw adding operation, the modulos according to this
`mod` argument will be computed and placed.
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
where = None if where is None else self._get_where(where)
address = self._get_address(key, where=where)
value = self._get_value(value)
mod = self._get_value(mod)
if self._is_dtype_bool:
raise ValueError("Circular addition is not supported for dtype `torch.bool`")
if where is None:
self._data[address] = (self._data[address] + value) % mod
else:
old_value = self._data[address]
new_value = (old_value + value) % mod
self._data[address] = do_where(where, new_value, old_value)
def multiply_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Multiply the existing values of slots with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be used as the multiplier(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
where = None if where is None else self._get_where(where)
address = self._get_address(key, where=where)
value = self._get_value(value)
if where is None:
if self._is_dtype_bool:
self._data[address] &= value
else:
self._data[address] += value
else:
if self._is_dtype_bool:
self._data[address] &= do_where(
where, value, torch.tensor(True, dtype=value.dtype, device=value.device)
)
else:
self._data[address] *= do_where(where, value, torch.tensor(1, dtype=value.dtype, device=value.device))
def subtract_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Subtract value(s) from existing values of slots with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be subtracted from existing value(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self.add_(key, -value, where)
def divide_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Divide the existing values of slots with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be used as divisor(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self.multiply_(key, 1 / value, where)
def __getitem__(self, key: Numbers) -> torch.Tensor:
"""
Get the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
Returns:
The value(s) associated with the given key(s).
"""
return self.get(key)
def __setitem__(self, key: Numbers, value: Numbers):
"""
Set the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The new value(s).
"""
self.set_(key, value)
@property
def data(self) -> torch.Tensor:
"""
The entire value tensor
"""
return self._data
@property
def key_shape(self) -> torch.Size:
"""
Shape of a key
"""
return torch.Size([self._internal_key_ndim]) if self._multi_key else torch.Size([])
@property
def key_ndim(self) -> int:
"""
Number of dimensions of a key
"""
return 1 if self._multi_key else 0
@property
def batch_shape(self) -> torch.Size:
"""
Batch size of this memory object
"""
return self._batch_shape
@property
def batch_ndim(self) -> int:
"""
Number of dimensions expressed by `batch_shape`
"""
return self._batch_ndim
@property
def is_batched(self) -> bool:
"""
True if this CMemory object is batched; False otherwise.
"""
return self._batch_ndim > 0
@property
def value_shape(self) -> torch.Size:
"""
Tensor shape of a single value
"""
return self._value_shape
@property
def value_ndim(self) -> int:
"""
Number of dimensions expressed by `value_shape`
"""
return self._value_ndim
@property
def dtype(self) -> torch.dtype:
"""
`dtype` of the value tensor
"""
return self._data.dtype
@property
def device(self) -> torch.device:
"""
The device on which this memory object lives
"""
return self._data.device
batch_ndim: int
property
readonly
¶
Number of dimensions expressed by batch_shape
batch_shape: Size
property
readonly
¶
Batch size of this memory object
data: Tensor
property
readonly
¶
The entire value tensor
device: device
property
readonly
¶
The device on which this memory object lives
dtype: dtype
property
readonly
¶
dtype of the value tensor
is_batched: bool
property
readonly
¶
True if this CMemory object is batched; False otherwise.
key_ndim: int
property
readonly
¶
Number of dimensions of a key
key_shape: Size
property
readonly
¶
Shape of a key
value_ndim: int
property
readonly
¶
Number of dimensions expressed by value_shape
value_shape: Size
property
readonly
¶
Tensor shape of a single value
__getitem__(self, key)
special
¶
Get the value(s) associated with the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The value(s) associated with the given key(s). |
Source code in evotorch/tools/structures.py
def __getitem__(self, key: Numbers) -> torch.Tensor:
"""
Get the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
Returns:
The value(s) associated with the given key(s).
"""
return self.get(key)
__init__(self, *size, *, num_keys, key_offset=None, batch_size=None, batch_shape=None, fill_with=None, dtype=None, device=None, verify=True)
special
¶
__init__(...): Initialize the CMemory.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, tuple, list] |
Size of a tensor associated with a key, expected as an integer, or as multiple positional arguments (each positional argument being an integer), or as a tuple of integers. |
() |
num_keys |
Union[int, tuple, list] |
How many keys (and therefore how many slots) will the
memory have. If given as an integer |
required |
key_offset |
Union[int, tuple, list] |
Optionally can be used to shift the integer values of
the keys. For example, if |
None |
batch_size |
Union[int, tuple, list] |
If given as None, then this memory will not be batched.
If given as an integer |
None |
batch_shape |
Union[int, tuple, list] |
Alias for the argument |
None |
fill_with |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a numeric value using which the values will be initialized. If no initialization is needed, then this argument can be left as None. |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
The |
None |
device |
Union[str, torch.device] |
The device on which the memory will be allocated. |
None |
verify |
bool |
If True, then explicit checks will be done to verify that there are no indexing errors. Can be set as False for performance. |
True |
Source code in evotorch/tools/structures.py
def __init__(
self,
*size: Union[int, tuple, list],
num_keys: Union[int, tuple, list],
key_offset: Optional[Union[int, tuple, list]] = None,
batch_size: Optional[Union[int, tuple, list]] = None,
batch_shape: Optional[Union[int, tuple, list]] = None,
fill_with: Optional[Numbers] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
verify: bool = True,
):
"""
`__init__(...)`: Initialize the CMemory.
Args:
size: Size of a tensor associated with a key, expected as an
integer, or as multiple positional arguments (each positional
argument being an integer), or as a tuple of integers.
num_keys: How many keys (and therefore how many slots) will the
memory have. If given as an integer `n`, then there will be `n`
slots in the memory, and to access a slot one will need to use
an integer key `k` (where, by default, the minimum acceptable
`k` is 0 and the maximum acceptable `k` is `n-1`).
If given as a tuple of integers, then the number of slots in
the memory will be computed as the product of all the integers
in the tuple, and a key will be expected as a tuple.
For example, when `num_keys` is `(3, 5)`, there will be 15
slots in the memory (where, by default, the minimum acceptable
key will be `(0, 0)` and the maximum acceptable key will be
`(2, 4)`.
key_offset: Optionally can be used to shift the integer values of
the keys. For example, if `num_keys` is 10, then, by default,
the minimum key is 0 and the maximum key is 9. But together
with `num_keys=10`, if `key_offset` is given as 1, then the
minimum key will be 1 and the maximum key will be 10.
This argument can also be used together with a tuple-valued
`num_keys`. For example, with `num_keys` set as `(3, 5)`,
if `key_offset` is given as 1, then the minimum key value
will be `(1, 1)` (instead of `(0, 0)`) and the maximum key
value will be `(3, 5)` (instead of `(2, 4)`).
Also, with a tuple-valued `num_keys`, `key_offset` can be
given as a tuple, to shift the key values differently for each
item in the tuple.
batch_size: If given as None, then this memory will not be batched.
If given as an integer `n`, then this object will represent
a contiguous batch containing `n` memory blocks.
If given as a tuple `(size0, size1, ...)`, then this object
will represent a contiguous batch of memory, shape of this
batch being determined by the given tuple.
batch_shape: Alias for the argument `batch_size`.
fill_with: Optionally a numeric value using which the values will
be initialized. If no initialization is needed, then this
argument can be left as None.
dtype: The `dtype` of the memory tensor.
device: The device on which the memory will be allocated.
verify: If True, then explicit checks will be done to verify
that there are no indexing errors. Can be set as False for
performance.
"""
self._dtype = torch.float32 if dtype is None else to_torch_dtype(dtype)
self._device = torch.device("cpu") if device is None else torch.device(device)
self._verify = bool(verify)
if isinstance(num_keys, (list, tuple)):
if len(num_keys) < 2:
raise RuntimeError(
f"When expressed via a list or a tuple, the length of `num_keys` must be at least 2."
f" However, the encountered `num_keys` is {repr(num_keys)}, whose length is {len(num_keys)}."
)
self._multi_key = True
self._num_keys = tuple((int(n) for n in num_keys))
self._internal_key_shape = torch.Size(self._num_keys)
else:
self._multi_key = False
self._num_keys = int(num_keys)
self._internal_key_shape = torch.Size([self._num_keys])
self._internal_key_ndim = len(self._internal_key_shape)
if key_offset is None:
self._key_offset = None
else:
if self._multi_key:
if isinstance(key_offset, (list, tuple)):
key_offset = [int(n) for n in key_offset]
if len(key_offset) != len(self._num_keys):
raise RuntimeError("The length of `key_offset` does not match the length of `num_keys`")
else:
key_offset = [int(key_offset) for _ in range(len(self._num_keys))]
self._key_offset = torch.as_tensor(key_offset, dtype=torch.int64, device=self._device)
else:
if isinstance(key_offset, (list, tuple)):
raise RuntimeError("`key_offset` cannot be a sequence of integers when `num_keys` is a scalar")
else:
self._key_offset = torch.as_tensor(int(key_offset), dtype=torch.int64, device=self._device)
if self._verify:
if self._multi_key:
self._min_key = torch.zeros(len(self._num_keys), dtype=torch.int64, device=self._device)
self._max_key = torch.tensor(list(self._num_keys), dtype=torch.int64, device=self._device) - 1
else:
self._min_key = torch.tensor(0, dtype=torch.int64, device=self._device)
self._max_key = torch.tensor(self._num_keys - 1, dtype=torch.int64, device=self._device)
if self._key_offset is not None:
self._min_key += self._key_offset
self._max_key += self._key_offset
else:
self._min_key = None
self._max_key = None
nsize = len(size)
if nsize == 0:
self._value_shape = torch.Size([])
elif nsize == 1:
if isinstance(size[0], (tuple, list)):
self._value_shape = torch.Size((int(n) for n in size[0]))
else:
self._value_shape = torch.Size([int(size[0])])
else:
self._value_shape = torch.Size((int(n) for n in size))
self._value_ndim = len(self._value_shape)
if (batch_size is None) and (batch_shape is None):
batch_size = None
elif (batch_size is not None) and (batch_shape is None):
pass
elif (batch_size is None) and (batch_shape is not None):
batch_size = batch_shape
else:
raise RuntimeError(
"Encountered both `batch_shape` and `batch_size` at the same time."
" None of them or one of them can be accepted, but not both of them at the same time."
)
if batch_size is None:
self._batch_shape = torch.Size([])
elif isinstance(batch_size, (tuple, list)):
self._batch_shape = torch.Size((int(n) for n in batch_size))
else:
self._batch_shape = torch.Size([int(batch_size)])
self._batch_ndim = len(self._batch_shape)
self._for_all_batches = tuple(
(
torch.arange(self._batch_shape[i], dtype=torch.int64, device=self._device)
for i in range(self._batch_ndim)
)
)
self._data = torch.empty(
self._batch_shape + self._internal_key_shape + self._value_shape,
dtype=(self._dtype),
device=(self._device),
)
if fill_with is not None:
self._data[:] = fill_with
__setitem__(self, key, value)
special
¶
Set the value(s) associated with the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The new value(s). |
required |
Source code in evotorch/tools/structures.py
add_(self, key, value, where=None)
¶
Add value(s) onto the existing values of slots with the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) that will be added onto the existing value(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Source code in evotorch/tools/structures.py
def add_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Add value(s) onto the existing values of slots with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be added onto the existing value(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
where = None if where is None else self._get_where(where)
address = self._get_address(key, where=where)
value = self._get_value(value)
if where is None:
if self._is_dtype_bool:
self._data[address] |= value
else:
self._data[address] += value
else:
if self._is_dtype_bool:
mask_shape = self._batch_shape + tuple((1 for _ in range(self._value_ndim)))
self._data[address] |= value & where.reshape(mask_shape)
else:
self._data[address] += do_where(where, value, torch.tensor(0, dtype=value.dtype, device=value.device))
add_circular_(self, key, value, mod, where=None)
¶
Increase the values of the specified slots in a circular manner.
This operation combines the add and modulo operations.
Circularly adding value onto x with a modulo mod means:
x = (x + value) % mod.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) that will be added onto the existing value(s). |
required |
mod |
Union[numbers.Number, Iterable[numbers.Number]] |
After the raw adding operation, the modulos according to this
|
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Source code in evotorch/tools/structures.py
def add_circular_(self, key: Numbers, value: Numbers, mod: Numbers, where: Optional[Numbers] = None):
"""
Increase the values of the specified slots in a circular manner.
This operation combines the add and modulo operations.
Circularly adding `value` onto `x` with a modulo `mod` means:
`x = (x + value) % mod`.
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be added onto the existing value(s).
mod: After the raw adding operation, the modulos according to this
`mod` argument will be computed and placed.
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
where = None if where is None else self._get_where(where)
address = self._get_address(key, where=where)
value = self._get_value(value)
mod = self._get_value(mod)
if self._is_dtype_bool:
raise ValueError("Circular addition is not supported for dtype `torch.bool`")
if where is None:
self._data[address] = (self._data[address] + value) % mod
else:
old_value = self._data[address]
new_value = (old_value + value) % mod
self._data[address] = do_where(where, new_value, old_value)
divide_(self, key, value, where=None)
¶
Divide the existing values of slots with the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) that will be used as divisor(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Source code in evotorch/tools/structures.py
def divide_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Divide the existing values of slots with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be used as divisor(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self.multiply_(key, 1 / value, where)
get(self, key)
¶
Get the value(s) associated with the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
The value(s) associated with the given key(s). |
Source code in evotorch/tools/structures.py
def get(self, key: Numbers) -> torch.Tensor:
"""
Get the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
Returns:
The value(s) associated with the given key(s).
"""
address = self._get_address(key)
return self._data[address]
multiply_(self, key, value, where=None)
¶
Multiply the existing values of slots with the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) that will be used as the multiplier(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Source code in evotorch/tools/structures.py
def multiply_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Multiply the existing values of slots with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be used as the multiplier(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
where = None if where is None else self._get_where(where)
address = self._get_address(key, where=where)
value = self._get_value(value)
if where is None:
if self._is_dtype_bool:
self._data[address] &= value
else:
self._data[address] += value
else:
if self._is_dtype_bool:
self._data[address] &= do_where(
where, value, torch.tensor(True, dtype=value.dtype, device=value.device)
)
else:
self._data[address] *= do_where(where, value, torch.tensor(1, dtype=value.dtype, device=value.device))
prepare_key_tensor(self, key)
¶
Return the tensor-counterpart of a key.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A key which can be a sequence of integers or a PyTorch tensor
with an integer dtype.
The shape of the given key must conform with the |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
A copy of the key that is converted to PyTorch tensor. |
Source code in evotorch/tools/structures.py
def prepare_key_tensor(self, key: Numbers) -> torch.Tensor:
"""
Return the tensor-counterpart of a key.
Args:
key: A key which can be a sequence of integers or a PyTorch tensor
with an integer dtype.
The shape of the given key must conform with the `key_shape`
of this memory object.
To address to a different key in each batch item, the shape of
the given key can also have extra leftmost dimensions expressed
by `batch_shape`.
Returns:
A copy of the key that is converted to PyTorch tensor.
"""
return self._get_key(key)
prepare_value_tensor(self, value)
¶
Return the tensor-counterpart of a value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value |
Union[numbers.Number, Iterable[numbers.Number]] |
A value that can be a numeric sequence or a PyTorch tensor.
The shape of the given value must conform with the
|
required |
Returns:
| Type | Description |
|---|---|
Tensor |
A copy of the given value(s), converted to PyTorch tensor. |
Source code in evotorch/tools/structures.py
def prepare_value_tensor(self, value: Numbers) -> torch.Tensor:
"""
Return the tensor-counterpart of a value.
Args:
value: A value that can be a numeric sequence or a PyTorch tensor.
The shape of the given value must conform with the
`value_shape` of this memory object.
To express a different value for each batch item, the shape of
the given value can also have extra leftmost dimensions
expressed by `value_shape`.
Returns:
A copy of the given value(s), converted to PyTorch tensor.
"""
return self._get_value(value)
prepare_where_tensor(self, where)
¶
Return the tensor-counterpart of a boolean mask.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
where |
Union[numbers.Number, Iterable[numbers.Number]] |
A boolean mask expressed as a sequence of bools or as a
boolean PyTorch tensor.
The shape of the given mask must conform with the batch shape
that is expressed by the property |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
A copy of the boolean mask, converted to PyTorch tensor. |
Source code in evotorch/tools/structures.py
def prepare_where_tensor(self, where: Numbers) -> torch.Tensor:
"""
Return the tensor-counterpart of a boolean mask.
Args:
where: A boolean mask expressed as a sequence of bools or as a
boolean PyTorch tensor.
The shape of the given mask must conform with the batch shape
that is expressed by the property `batch_shape`.
Returns:
A copy of the boolean mask, converted to PyTorch tensor.
"""
return self._get_where(where)
set_(self, key, value, where=None)
¶
Set the value(s) associated with the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The new value(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Source code in evotorch/tools/structures.py
def set_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Set the value(s) associated with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The new value(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
where = None if where is None else self._get_where(where)
address = self._get_address(key, where=where)
value = self._get_value(value)
if where is None:
self._data[address] = value
else:
old_value = self._data[address]
new_value = value
self._data[address] = do_where(where, new_value, old_value)
subtract_(self, key, value, where=None)
¶
Subtract value(s) from existing values of slots with the given key(s).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key |
Union[numbers.Number, Iterable[numbers.Number]] |
A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the |
required |
value |
Union[numbers.Number, Iterable[numbers.Number]] |
The value(s) that will be subtracted from existing value(s). |
required |
where |
Union[numbers.Number, Iterable[numbers.Number]] |
Optionally a boolean mask whose shape matches |
None |
Source code in evotorch/tools/structures.py
def subtract_(self, key: Numbers, value: Numbers, where: Optional[Numbers] = None):
"""
Subtract value(s) from existing values of slots with the given key(s).
Args:
key: A single key, or multiple keys (where the leftmost dimension
of the given keys conform with the `batch_shape`).
value: The value(s) that will be subtracted from existing value(s).
where: Optionally a boolean mask whose shape matches `batch_shape`.
If a `where` mask is given, then modifications will happen only
on the memory slots whose corresponding mask values are True.
"""
self.add_(key, -value, where)
Structure
¶
A mixin class for vectorized structures.
This mixin class assumes that the inheriting structure has a protected
attribute _data which is either a CMemory object or another
Structure. With this assumption, this mixin class provides certain
methods and properties to bring a unified interface for all vectorized
structures provided in this namespace.
Source code in evotorch/tools/structures.py
class Structure:
"""
A mixin class for vectorized structures.
This mixin class assumes that the inheriting structure has a protected
attribute `_data` which is either a `CMemory` object or another
`Structure`. With this assumption, this mixin class provides certain
methods and properties to bring a unified interface for all vectorized
structures provided in this namespace.
"""
_data: Union[CMemory, "Structure"]
@property
def value_shape(self) -> torch.Size:
"""
Shape of a single value
"""
return self._data.value_shape
@property
def value_ndim(self) -> int:
"""
Number of dimensions expressed by `value_shape`
"""
return self._data.value_ndim
@property
def batch_shape(self) -> torch.Size:
"""
Batch size of this structure
"""
return self._data.batch_shape
@property
def batch_ndim(self) -> int:
"""
Number of dimensions expressed by `batch_shape`
"""
return self._data.batch_ndim
@property
def is_batched(self) -> bool:
"""
True if this structure is batched; False otherwise.
"""
return self._batch_ndim > 0
@property
def dtype(self) -> torch.dtype:
"""
`dtype` of the values
"""
return self._data.dtype
@property
def device(self) -> torch.device:
"""
The device on which this structure lives
"""
return self._data.device
def prepare_value_tensor(self, value: Numbers) -> torch.Tensor:
"""
Return the tensor-counterpart of a value.
Args:
value: A value that can be a numeric sequence or a PyTorch tensor.
The shape of the given value must conform with the
`value_shape` of this memory object.
To express a different value for each batch item, the shape of
the given value can also have extra leftmost dimensions
expressed by `value_shape`.
Returns:
A copy of the given value(s), converted to PyTorch tensor.
"""
return self._data.prepare_value_tensor(value)
def prepare_where_tensor(self, where: Numbers) -> torch.Tensor:
"""
Return the tensor-counterpart of a boolean mask.
Args:
where: A boolean mask expressed as a sequence of bools or as a
boolean PyTorch tensor.
The shape of the given mask must conform with the batch shape
that is expressed by the property `batch_shape`.
Returns:
A copy of the boolean mask, converted to PyTorch tensor.
"""
return self._data.prepare_where_tensor(where)
def _get_value(self, value: Numbers) -> torch.Tensor:
return self._data.prepare_value_tensor(value)
def _get_where(self, where: Numbers) -> torch.Tensor:
return self._data.prepare_where_tensor(where)
def __contains__(self, x: Any) -> torch.Tensor:
raise TypeError("This structure does not support the `in` operator")
batch_ndim: int
property
readonly
¶
Number of dimensions expressed by batch_shape
batch_shape: Size
property
readonly
¶
Batch size of this structure
device: device
property
readonly
¶
The device on which this structure lives
dtype: dtype
property
readonly
¶
dtype of the values
is_batched: bool
property
readonly
¶
True if this structure is batched; False otherwise.
value_ndim: int
property
readonly
¶
Number of dimensions expressed by value_shape
value_shape: Size
property
readonly
¶
Shape of a single value
prepare_value_tensor(self, value)
¶
Return the tensor-counterpart of a value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value |
Union[numbers.Number, Iterable[numbers.Number]] |
A value that can be a numeric sequence or a PyTorch tensor.
The shape of the given value must conform with the
|
required |
Returns:
| Type | Description |
|---|---|
Tensor |
A copy of the given value(s), converted to PyTorch tensor. |
Source code in evotorch/tools/structures.py
def prepare_value_tensor(self, value: Numbers) -> torch.Tensor:
"""
Return the tensor-counterpart of a value.
Args:
value: A value that can be a numeric sequence or a PyTorch tensor.
The shape of the given value must conform with the
`value_shape` of this memory object.
To express a different value for each batch item, the shape of
the given value can also have extra leftmost dimensions
expressed by `value_shape`.
Returns:
A copy of the given value(s), converted to PyTorch tensor.
"""
return self._data.prepare_value_tensor(value)
prepare_where_tensor(self, where)
¶
Return the tensor-counterpart of a boolean mask.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
where |
Union[numbers.Number, Iterable[numbers.Number]] |
A boolean mask expressed as a sequence of bools or as a
boolean PyTorch tensor.
The shape of the given mask must conform with the batch shape
that is expressed by the property |
required |
Returns:
| Type | Description |
|---|---|
Tensor |
A copy of the boolean mask, converted to PyTorch tensor. |
Source code in evotorch/tools/structures.py
def prepare_where_tensor(self, where: Numbers) -> torch.Tensor:
"""
Return the tensor-counterpart of a boolean mask.
Args:
where: A boolean mask expressed as a sequence of bools or as a
boolean PyTorch tensor.
The shape of the given mask must conform with the batch shape
that is expressed by the property `batch_shape`.
Returns:
A copy of the boolean mask, converted to PyTorch tensor.
"""
return self._data.prepare_where_tensor(where)
tensormaker
¶
Base classes with various utilities for creating tensors.
TensorMakerMixin
¶
Source code in evotorch/tools/tensormaker.py
class TensorMakerMixin:
def __get_dtype_and_device_kwargs(
self,
*,
dtype: Optional[DType],
device: Optional[Device],
use_eval_dtype: bool,
out: Optional[Iterable],
) -> dict:
result = {}
if out is None:
if dtype is None:
if use_eval_dtype:
if hasattr(self, "eval_dtype"):
result["dtype"] = self.eval_dtype
else:
raise AttributeError(
f"Received `use_eval_dtype` as {repr(use_eval_dtype)}, which represents boolean truth."
f" However, evaluation dtype cannot be determined, because this object does not have"
f" an attribute named `eval_dtype`."
)
else:
result["dtype"] = self.dtype
else:
if use_eval_dtype:
raise ValueError(
f"Received both a `dtype` argument ({repr(dtype)}) and `use_eval_dtype` as True."
f" These arguments are conflicting."
f" Please either provide a `dtype`, or leave `dtype` as None and pass `use_eval_dtype=True`."
)
else:
result["dtype"] = dtype
if device is None:
result["device"] = self.device
else:
result["device"] = device
return result
def __get_size_args(self, *size: Size, num_solutions: Optional[int], out: Optional[Iterable]) -> tuple:
if out is None:
nsize = len(size)
if (nsize == 0) and (num_solutions is None):
return tuple()
elif (nsize >= 1) and (num_solutions is None):
return size
elif (nsize == 0) and (num_solutions is not None):
if hasattr(self, "solution_length"):
num_solutions = int(num_solutions)
if self.solution_length is None:
return (num_solutions,)
else:
return (num_solutions, self.solution_length)
else:
raise AttributeError(
f"Received `num_solutions` as {repr(num_solutions)}."
f" However, to determine the target tensor's size via `num_solutions`, this object"
f" needs to have an attribute named `solution_length`, which seems to be missing."
)
else:
raise ValueError(
f"Encountered both `size` arguments ({repr(size)})"
f" and `num_solutions` keyword argument (num_solutions={repr(num_solutions)})."
f" Specifying both `size` and `num_solutions` is not valid."
)
else:
return tuple()
def __get_generator_kwargs(self, *, generator: Any) -> dict:
result = {}
if generator is None:
if hasattr(self, "generator"):
result["generator"] = self.generator
else:
result["generator"] = generator
return result
def __get_all_args_for_maker(
self,
*size: Size,
num_solutions: Optional[int],
out: Optional[Iterable],
dtype: Optional[DType],
device: Optional[Device],
use_eval_dtype: bool,
) -> tuple:
args = self.__get_size_args(*size, num_solutions=num_solutions, out=out)
kwargs = self.__get_dtype_and_device_kwargs(dtype=dtype, device=device, use_eval_dtype=use_eval_dtype, out=out)
if out is not None:
kwargs["out"] = out
return args, kwargs
def __get_all_args_for_random_maker(
self,
*size: Size,
num_solutions: Optional[int],
out: Optional[Iterable],
dtype: Optional[DType],
device: Optional[Device],
use_eval_dtype: bool,
generator: Any,
):
args = self.__get_size_args(*size, num_solutions=num_solutions, out=out)
kwargs = {}
kwargs.update(
self.__get_dtype_and_device_kwargs(dtype=dtype, device=device, use_eval_dtype=use_eval_dtype, out=out)
)
kwargs.update(self.__get_generator_kwargs(generator=generator))
if out is not None:
kwargs["out"] = out
return args, kwargs
def make_tensor(
self,
data: Any,
*,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
read_only: bool = False,
) -> Iterable:
"""
Make a new tensor.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
data: The data to be converted to a tensor.
If one wishes to create a PyTorch tensor, this can be anything
that can be stored by a PyTorch tensor.
If one wishes to create an `ObjectArray` and therefore passes
`dtype=object`, then the provided `data` is expected as an
`Iterable`.
dtype: Optionally a string (e.g. "float32"), or a PyTorch dtype
(e.g. torch.float32), or `object` or "object" (as a string)
or `Any` if one wishes to create an `ObjectArray`.
If `dtype` is not specified it will be assumed that the user
wishes to create a tensor using the dtype of this method's
parent object.
device: The device in which the tensor will be stored.
If `device` is not specified, it will be assumed that the user
wishes to create a tensor on the device of this method's
parent object.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
read_only: Whether or not the created tensor will be read-only.
By default, this is False.
Returns:
A PyTorch tensor or an ObjectArray.
"""
kwargs = self.__get_dtype_and_device_kwargs(dtype=dtype, device=device, use_eval_dtype=use_eval_dtype, out=None)
return misc.make_tensor(data, read_only=read_only, **kwargs)
def make_empty(
self,
*size: Size,
num_solutions: Optional[int] = None,
out: Optional[Iterable] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> Iterable:
"""
Make an empty tensor.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
size: Shape of the empty tensor to be created.
expected as multiple positional arguments of integers,
or as a single positional argument containing a tuple of
integers.
Note that when the user wishes to create an `ObjectArray`
(i.e. when `dtype` is given as `object`), then the size
is expected as a single integer, or as a single-element
tuple containing an integer (because `ObjectArray` can only
be one-dimensional).
num_solutions: This can be used instead of the `size` arguments
for specifying the shape of the target tensor.
Expected as an integer, when `num_solutions` is specified
as `n`, the shape of the resulting tensor will be
`(n, m)` where `m` is the solution length reported by this
method's parent object's `solution_length` attribute.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32) or, for creating an `ObjectArray`,
"object" (as string) or `object` or `Any`.
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
Returns:
The new empty tensor, which can be a PyTorch tensor or an
`ObjectArray`.
"""
args, kwargs = self.__get_all_args_for_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
)
return misc.make_empty(*args, **kwargs)
def make_zeros(
self,
*size: Size,
num_solutions: Optional[int] = None,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> torch.Tensor:
"""
Make a new tensor filled with 0, or fill an existing tensor with 0.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
size: Size of the new tensor to be filled with 0.
This can be given as multiple positional arguments, each such
positional argument being an integer, or as a single positional
argument of a tuple, the tuple containing multiple integers.
Note that, if the user wishes to fill an existing tensor with
0 values, then no positional argument is expected.
num_solutions: This can be used instead of the `size` arguments
for specifying the shape of the target tensor.
Expected as an integer, when `num_solutions` is specified
as `n`, the shape of the resulting tensor will be
`(n, m)` where `m` is the solution length reported by this
method's parent object's `solution_length` attribute.
out: Optionally, the tensor to be filled by 0 values.
If an `out` tensor is given, then no `size` argument is expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
Returns:
The created or modified tensor after placing 0 values.
"""
args, kwargs = self.__get_all_args_for_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
)
return misc.make_zeros(*args, **kwargs)
def make_ones(
self,
*size: Size,
num_solutions: Optional[int] = None,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> torch.Tensor:
"""
Make a new tensor filled with 1, or fill an existing tensor with 1.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
size: Size of the new tensor to be filled with 1.
This can be given as multiple positional arguments, each such
positional argument being an integer, or as a single positional
argument of a tuple, the tuple containing multiple integers.
Note that, if the user wishes to fill an existing tensor with
1 values, then no positional argument is expected.
num_solutions: This can be used instead of the `size` arguments
for specifying the shape of the target tensor.
Expected as an integer, when `num_solutions` is specified
as `n`, the shape of the resulting tensor will be
`(n, m)` where `m` is the solution length reported by this
method's parent object's `solution_length` attribute.
out: Optionally, the tensor to be filled by 1 values.
If an `out` tensor is given, then no `size` argument is expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
Returns:
The created or modified tensor after placing 1 values.
"""
args, kwargs = self.__get_all_args_for_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
)
return misc.make_ones(*args, **kwargs)
def make_nan(
self,
*size: Size,
num_solutions: Optional[int] = None,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> torch.Tensor:
"""
Make a new tensor filled with NaN values, or fill an existing tensor
with NaN values.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
size: Size of the new tensor to be filled with NaN.
This can be given as multiple positional arguments, each such
positional argument being an integer, or as a single positional
argument of a tuple, the tuple containing multiple integers.
Note that, if the user wishes to fill an existing tensor with
NaN values, then no positional argument is expected.
num_solutions: This can be used instead of the `size` arguments
for specifying the shape of the target tensor.
Expected as an integer, when `num_solutions` is specified
as `n`, the shape of the resulting tensor will be
`(n, m)` where `m` is the solution length reported by this
method's parent object's `solution_length` attribute.
out: Optionally, the tensor to be filled by NaN values.
If an `out` tensor is given, then no `size` argument is expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
Returns:
The created or modified tensor after placing NaN values.
"""
args, kwargs = self.__get_all_args_for_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
)
return misc.make_nan(*args, **kwargs)
def make_I(
self,
size: Optional[Union[int, tuple]] = None,
*,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> torch.Tensor:
"""
Make a new identity matrix (I), or change an existing tensor so that
it expresses the identity matrix.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
size: A single integer or a tuple containing a single integer,
where the integer specifies the length of the target square
matrix. In this context, "length" means both rowwise length
and columnwise length, since the target is a square matrix.
Note that, if the user wishes to fill an existing tensor with
identity values, then `size` is expected to be left as None.
out: Optionally, the existing tensor whose values will be changed
so that they represent an identity matrix.
If an `out` tensor is given, then `size` is expected as None.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
Returns:
The created or modified tensor after placing the I matrix values
"""
if size is None:
if out is None:
if hasattr(self, "solution_length"):
size_args = (self.solution_length,)
else:
raise AttributeError(
"The method `.make_I(...)` was used without any `size`"
" arguments."
" When the `size` argument is missing, the default"
" behavior of this method is to create an identity matrix"
" of size (n, n), n being the length of a solution."
" However, the parent object of this method does not have"
" an attribute name `solution_length`."
)
else:
size_args = tuple()
elif isinstance(size, tuple):
if len(size) != 1:
raise ValueError(
f"When the size argument is given as a tuple, the method `make_I(...)` expects the tuple to have"
f" only one element. The given tuple is {size}."
)
size_args = size
else:
size_args = (int(size),)
args, kwargs = self.__get_all_args_for_maker(
*size_args,
num_solutions=None,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
)
return misc.make_I(*args, **kwargs)
def make_uniform(
self,
*size: Size,
num_solutions: Optional[int] = None,
lb: Optional[RealOrVector] = None,
ub: Optional[RealOrVector] = None,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
generator: Any = None,
) -> torch.Tensor:
"""
Make a new or existing tensor filled by uniformly distributed values.
Both lower and upper bounds are inclusive.
This function can work with both float and int dtypes.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
size: Size of the new tensor to be filled with uniformly distributed
values. This can be given as multiple positional arguments, each
such positional argument being an integer, or as a single
positional argument of a tuple, the tuple containing multiple
integers. Note that, if the user wishes to fill an existing
tensor instead, then no positional argument is expected.
num_solutions: This can be used instead of the `size` arguments
for specifying the shape of the target tensor.
Expected as an integer, when `num_solutions` is specified
as `n`, the shape of the resulting tensor will be
`(n, m)` where `m` is the solution length reported by this
method's parent object's `solution_length` attribute.
lb: Lower bound for the uniformly distributed values.
Can be a scalar, or a tensor.
If not specified, the lower bound will be taken as 0.
Note that, if one specifies `lb`, then `ub` is also expected to
be explicitly specified.
ub: Upper bound for the uniformly distributed values.
Can be a scalar, or a tensor.
If not specified, the upper bound will be taken as 1.
Note that, if one specifies `ub`, then `lb` is also expected to
be explicitly specified.
out: Optionally, the tensor to be filled by uniformly distributed
values. If an `out` tensor is given, then no `size` argument is
expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
generator: Pseudo-random generator to be used when sampling
the values. Can be a `torch.Generator` or any object with
a `generator` attribute (e.g. a Problem object).
If not given, then this method's parent object will be
analyzed whether or not it has its own generator.
If it does, that generator will be used.
If not, the global generator of PyTorch will be used.
Returns:
The created or modified tensor after placing the uniformly
distributed values.
"""
args, kwargs = self.__get_all_args_for_random_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
generator=generator,
)
return misc.make_uniform(*args, lb=lb, ub=ub, **kwargs)
def make_gaussian(
self,
*size: Size,
num_solutions: Optional[int] = None,
center: Optional[RealOrVector] = None,
stdev: Optional[RealOrVector] = None,
symmetric: bool = False,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
generator: Any = None,
) -> torch.Tensor:
"""
Make a new or existing tensor filled by Gaussian distributed values.
This function can work only with float dtypes.
Args:
size: Size of the new tensor to be filled with Gaussian distributed
values. This can be given as multiple positional arguments, each
such positional argument being an integer, or as a single
positional argument of a tuple, the tuple containing multiple
integers. Note that, if the user wishes to fill an existing
tensor instead, then no positional argument is expected.
num_solutions: This can be used instead of the `size` arguments
for specifying the shape of the target tensor.
Expected as an integer, when `num_solutions` is specified
as `n`, the shape of the resulting tensor will be
`(n, m)` where `m` is the solution length reported by this
method's parent object's `solution_length` attribute.
center: Center point (i.e. mean) of the Gaussian distribution.
Can be a scalar, or a tensor.
If not specified, the center point will be taken as 0.
Note that, if one specifies `center`, then `stdev` is also
expected to be explicitly specified.
stdev: Standard deviation for the Gaussian distributed values.
Can be a scalar, or a tensor.
If not specified, the standard deviation will be taken as 1.
Note that, if one specifies `stdev`, then `center` is also
expected to be explicitly specified.
symmetric: Whether or not the values should be sampled in a
symmetric (i.e. antithetic) manner.
The default is False.
out: Optionally, the tensor to be filled by Gaussian distributed
values. If an `out` tensor is given, then no `size` argument is
expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
generator: Pseudo-random generator to be used when sampling
the values. Can be a `torch.Generator` or any object with
a `generator` attribute (e.g. a Problem object).
If not given, then this method's parent object will be
analyzed whether or not it has its own generator.
If it does, that generator will be used.
If not, the global generator of PyTorch will be used.
Returns:
The created or modified tensor after placing the Gaussian
distributed values.
"""
args, kwargs = self.__get_all_args_for_random_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
generator=generator,
)
return misc.make_gaussian(*args, center=center, stdev=stdev, symmetric=symmetric, **kwargs)
def make_randint(
self,
*size: Size,
n: Union[int, float, torch.Tensor],
num_solutions: Optional[int] = None,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
generator: Any = None,
) -> torch.Tensor:
"""
Make a new or existing tensor filled by random integers.
The integers are uniformly distributed within `[0 ... n-1]`.
This function can be used with integer or float dtypes.
Args:
size: Size of the new tensor to be filled with uniformly distributed
values. This can be given as multiple positional arguments, each
such positional argument being an integer, or as a single
positional argument of a tuple, the tuple containing multiple
integers. Note that, if the user wishes to fill an existing
tensor instead, then no positional argument is expected.
n: Number of choice(s) for integer sampling.
The lowest possible value will be 0, and the highest possible
value will be n - 1.
`n` can be a scalar, or a tensor.
out: Optionally, the tensor to be filled by the random integers.
If an `out` tensor is given, then no `size` argument is
expected.
dtype: Optionally a string (e.g. "int64") or a PyTorch dtype
(e.g. torch.int64).
If `dtype` is not specified (and also `out` is None),
`torch.int64` will be used.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
generator: Pseudo-random generator to be used when sampling
the values. Can be a `torch.Generator` or any object with
a `generator` attribute (e.g. a Problem object).
If not given, then this method's parent object will be
analyzed whether or not it has its own generator.
If it does, that generator will be used.
If not, the global generator of PyTorch will be used.
Returns:
The created or modified tensor after placing the uniformly
distributed values.
"""
if (dtype is None) and (out is None):
dtype = torch.int64
args, kwargs = self.__get_all_args_for_random_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
generator=generator,
)
return misc.make_randint(*args, n=n, **kwargs)
def as_tensor(
self,
x: Any,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> torch.Tensor:
"""
Get the tensor counterpart of the given object `x`.
Args:
x: Any object to be converted to a tensor.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32) or, for creating an `ObjectArray`,
"object" (as string) or `object` or `Any`.
If `dtype` is not specified, the dtype of this method's
parent object will be used.
device: The device in which the resulting tensor will be stored.
If `device` is not specified, the device of this method's
parent object will be used.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
Returns:
The tensor counterpart of the given object `x`.
"""
kwargs = self.__get_dtype_and_device_kwargs(dtype=dtype, device=device, use_eval_dtype=use_eval_dtype, out=None)
return misc.as_tensor(x, **kwargs)
def ensure_tensor_length_and_dtype(
self,
t: Any,
length: Optional[int] = None,
dtype: Optional[DType] = None,
about: Optional[str] = None,
*,
allow_scalar: bool = False,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> Iterable:
"""
Return the given sequence as a tensor while also confirming its
length, dtype, and device.
Default length, dtype, device are taken from this method's
parent object.
In more details, these attributes belonging to this method's parent
object will be used for determining the the defaults:
`solution_length`, `dtype`, and `device`.
Args:
t: The tensor, or a sequence which is convertible to a tensor.
length: The length to which the tensor is expected to conform.
If missing, the `solution_length` attribute of this method's
parent object will be used as the default value.
dtype: The dtype to which the tensor is expected to conform.
If `dtype` argument is missing and `use_eval_dtype` is False,
then the default dtype will be determined by the `dtype`
attribute of this method's parent object.
If `dtype` argument is missing and `use_eval_dtype` is True,
then the default dtype will be determined by the `eval_dtype`
attribute of this method's parent object.
about: The prefix for the error message. Can be left as None.
allow_scalar: Whether or not to accept scalars in addition
to vector of the desired length.
If `allow_scalar` is False, then scalars will be converted
to sequences of the desired length. The sequence will contain
the same scalar, repeated.
If `allow_scalar` is True, then the scalar itself will be
converted to a PyTorch scalar, and then will be returned.
device: The device in which the sequence is to be stored.
If the given sequence is on a different device than the
desired device, a copy on the correct device will be made.
If device is None, the default behavior of `torch.tensor(...)`
will be used, that is: if `t` is already a tensor, the result
will be on the same device, otherwise, the result will be on
the cpu.
use_eval_dtype: Whether or not to use the evaluation dtype
(instead of the dtype of decision values).
If this is given as True, the `dtype` argument is expected
as None.
If `dtype` argument is missing and `use_eval_dtype` is False,
then the default dtype will be determined by the `dtype`
attribute of this method's parent object.
If `dtype` argument is missing and `use_eval_dtype` is True,
then the default dtype will be determined by the `eval_dtype`
attribute of this method's parent object.
Returns:
The sequence whose correctness in terms of length, dtype, and
device is ensured.
Raises:
ValueError: if there is a length mismatch.
"""
if length is None:
if hasattr(self, "solution_length"):
length = self.solution_length
else:
raise AttributeError(
f"{about}: The argument `length` was found to be None."
f" When the `length` argument is None, the default behavior is to use the `solution_length`"
f" attribute of this method's parent object."
f" However, this method's parent object does NOT have a `solution_length` attribute."
)
dtype_and_device = self.__get_dtype_and_device_kwargs(
dtype=dtype, device=device, use_eval_dtype=use_eval_dtype, out=None
)
return misc.ensure_tensor_length_and_dtype(
t, length=length, about=about, allow_scalar=allow_scalar, **dtype_and_device
)
def make_uniform_shaped_like(
self,
t: torch.Tensor,
*,
lb: Optional[RealOrVector] = None,
ub: Optional[RealOrVector] = None,
) -> torch.Tensor:
"""
Make a new uniformly-filled tensor, shaped like the given tensor.
The `dtype` and `device` will be determined by the parent of this
method (not by the given tensor).
If the parent of this method has its own random generator, then that
generator will be used.
Args:
t: The tensor according to which the result will be shaped.
lb: The inclusive lower bounds for the uniform distribution.
Can be a scalar or a tensor.
If left as None, 0.0 will be used as the upper bound.
ub: The inclusive upper bounds for the uniform distribution.
Can be a scalar or a tensor.
If left as None, 1.0 will be used as the upper bound.
Returns:
A new tensor whose shape is the same with the given tensor.
"""
return self.make_uniform(t.shape, lb=lb, ub=ub)
def make_gaussian_shaped_like(
self,
t: torch.Tensor,
*,
center: Optional[RealOrVector] = None,
stdev: Optional[RealOrVector] = None,
) -> torch.Tensor:
"""
Make a new tensor, shaped like the given tensor, with its values
filled by the Gaussian distribution.
The `dtype` and `device` will be determined by the parent of this
method (not by the given tensor).
If the parent of this method has its own random generator, then that
generator will be used.
Args:
t: The tensor according to which the result will be shaped.
center: Center point for the Gaussian distribution.
Can be a scalar or a tensor.
If left as None, 0.0 will be used as the center point.
stdev: The standard deviation for the Gaussian distribution.
Can be a scalar or a tensor.
If left as None, 1.0 will be used as the standard deviation.
Returns:
A new tensor whose shape is the same with the given tensor.
"""
return self.make_gaussian(t.shape, center=center, stdev=stdev)
as_tensor(self, x, dtype=None, device=None, use_eval_dtype=False)
¶
Get the tensor counterpart of the given object x.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x |
Any |
Any object to be converted to a tensor. |
required |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32) or, for creating an |
None |
device |
Union[str, torch.device] |
The device in which the resulting tensor will be stored.
If |
None |
use_eval_dtype |
bool |
If this is given as True and a |
False |
Returns:
| Type | Description |
|---|---|
Tensor |
The tensor counterpart of the given object |
Source code in evotorch/tools/tensormaker.py
def as_tensor(
self,
x: Any,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> torch.Tensor:
"""
Get the tensor counterpart of the given object `x`.
Args:
x: Any object to be converted to a tensor.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32) or, for creating an `ObjectArray`,
"object" (as string) or `object` or `Any`.
If `dtype` is not specified, the dtype of this method's
parent object will be used.
device: The device in which the resulting tensor will be stored.
If `device` is not specified, the device of this method's
parent object will be used.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
Returns:
The tensor counterpart of the given object `x`.
"""
kwargs = self.__get_dtype_and_device_kwargs(dtype=dtype, device=device, use_eval_dtype=use_eval_dtype, out=None)
return misc.as_tensor(x, **kwargs)
ensure_tensor_length_and_dtype(self, t, length=None, dtype=None, about=None, *, allow_scalar=False, device=None, use_eval_dtype=False)
¶
Return the given sequence as a tensor while also confirming its length, dtype, and device.
Default length, dtype, device are taken from this method's
parent object.
In more details, these attributes belonging to this method's parent
object will be used for determining the the defaults:
solution_length, dtype, and device.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
t |
Any |
The tensor, or a sequence which is convertible to a tensor. |
required |
length |
Optional[int] |
The length to which the tensor is expected to conform.
If missing, the |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
The dtype to which the tensor is expected to conform.
If |
None |
about |
Optional[str] |
The prefix for the error message. Can be left as None. |
None |
allow_scalar |
bool |
Whether or not to accept scalars in addition
to vector of the desired length.
If |
False |
device |
Union[str, torch.device] |
The device in which the sequence is to be stored.
If the given sequence is on a different device than the
desired device, a copy on the correct device will be made.
If device is None, the default behavior of |
None |
use_eval_dtype |
bool |
Whether or not to use the evaluation dtype
(instead of the dtype of decision values).
If this is given as True, the |
False |
Returns:
| Type | Description |
|---|---|
Iterable |
The sequence whose correctness in terms of length, dtype, and device is ensured. |
Exceptions:
| Type | Description |
|---|---|
ValueError |
if there is a length mismatch. |
Source code in evotorch/tools/tensormaker.py
def ensure_tensor_length_and_dtype(
self,
t: Any,
length: Optional[int] = None,
dtype: Optional[DType] = None,
about: Optional[str] = None,
*,
allow_scalar: bool = False,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> Iterable:
"""
Return the given sequence as a tensor while also confirming its
length, dtype, and device.
Default length, dtype, device are taken from this method's
parent object.
In more details, these attributes belonging to this method's parent
object will be used for determining the the defaults:
`solution_length`, `dtype`, and `device`.
Args:
t: The tensor, or a sequence which is convertible to a tensor.
length: The length to which the tensor is expected to conform.
If missing, the `solution_length` attribute of this method's
parent object will be used as the default value.
dtype: The dtype to which the tensor is expected to conform.
If `dtype` argument is missing and `use_eval_dtype` is False,
then the default dtype will be determined by the `dtype`
attribute of this method's parent object.
If `dtype` argument is missing and `use_eval_dtype` is True,
then the default dtype will be determined by the `eval_dtype`
attribute of this method's parent object.
about: The prefix for the error message. Can be left as None.
allow_scalar: Whether or not to accept scalars in addition
to vector of the desired length.
If `allow_scalar` is False, then scalars will be converted
to sequences of the desired length. The sequence will contain
the same scalar, repeated.
If `allow_scalar` is True, then the scalar itself will be
converted to a PyTorch scalar, and then will be returned.
device: The device in which the sequence is to be stored.
If the given sequence is on a different device than the
desired device, a copy on the correct device will be made.
If device is None, the default behavior of `torch.tensor(...)`
will be used, that is: if `t` is already a tensor, the result
will be on the same device, otherwise, the result will be on
the cpu.
use_eval_dtype: Whether or not to use the evaluation dtype
(instead of the dtype of decision values).
If this is given as True, the `dtype` argument is expected
as None.
If `dtype` argument is missing and `use_eval_dtype` is False,
then the default dtype will be determined by the `dtype`
attribute of this method's parent object.
If `dtype` argument is missing and `use_eval_dtype` is True,
then the default dtype will be determined by the `eval_dtype`
attribute of this method's parent object.
Returns:
The sequence whose correctness in terms of length, dtype, and
device is ensured.
Raises:
ValueError: if there is a length mismatch.
"""
if length is None:
if hasattr(self, "solution_length"):
length = self.solution_length
else:
raise AttributeError(
f"{about}: The argument `length` was found to be None."
f" When the `length` argument is None, the default behavior is to use the `solution_length`"
f" attribute of this method's parent object."
f" However, this method's parent object does NOT have a `solution_length` attribute."
)
dtype_and_device = self.__get_dtype_and_device_kwargs(
dtype=dtype, device=device, use_eval_dtype=use_eval_dtype, out=None
)
return misc.ensure_tensor_length_and_dtype(
t, length=length, about=about, allow_scalar=allow_scalar, **dtype_and_device
)
make_I(self, size=None, *, out=None, dtype=None, device=None, use_eval_dtype=False)
¶
Make a new identity matrix (I), or change an existing tensor so that it expresses the identity matrix.
When not explicitly specified via arguments, the dtype and the device of the resulting tensor is determined by this method's parent object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[tuple, int] |
A single integer or a tuple containing a single integer,
where the integer specifies the length of the target square
matrix. In this context, "length" means both rowwise length
and columnwise length, since the target is a square matrix.
Note that, if the user wishes to fill an existing tensor with
identity values, then |
None |
out |
Optional[torch.Tensor] |
Optionally, the existing tensor whose values will be changed
so that they represent an identity matrix.
If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If |
None |
device |
Union[str, torch.device] |
The device in which the new empty tensor will be stored.
If not specified (and also |
None |
use_eval_dtype |
bool |
If this is given as True and a |
False |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing the I matrix values |
Source code in evotorch/tools/tensormaker.py
def make_I(
self,
size: Optional[Union[int, tuple]] = None,
*,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> torch.Tensor:
"""
Make a new identity matrix (I), or change an existing tensor so that
it expresses the identity matrix.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
size: A single integer or a tuple containing a single integer,
where the integer specifies the length of the target square
matrix. In this context, "length" means both rowwise length
and columnwise length, since the target is a square matrix.
Note that, if the user wishes to fill an existing tensor with
identity values, then `size` is expected to be left as None.
out: Optionally, the existing tensor whose values will be changed
so that they represent an identity matrix.
If an `out` tensor is given, then `size` is expected as None.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
Returns:
The created or modified tensor after placing the I matrix values
"""
if size is None:
if out is None:
if hasattr(self, "solution_length"):
size_args = (self.solution_length,)
else:
raise AttributeError(
"The method `.make_I(...)` was used without any `size`"
" arguments."
" When the `size` argument is missing, the default"
" behavior of this method is to create an identity matrix"
" of size (n, n), n being the length of a solution."
" However, the parent object of this method does not have"
" an attribute name `solution_length`."
)
else:
size_args = tuple()
elif isinstance(size, tuple):
if len(size) != 1:
raise ValueError(
f"When the size argument is given as a tuple, the method `make_I(...)` expects the tuple to have"
f" only one element. The given tuple is {size}."
)
size_args = size
else:
size_args = (int(size),)
args, kwargs = self.__get_all_args_for_maker(
*size_args,
num_solutions=None,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
)
return misc.make_I(*args, **kwargs)
make_empty(self, *size, *, num_solutions=None, out=None, dtype=None, device=None, use_eval_dtype=False)
¶
Make an empty tensor.
When not explicitly specified via arguments, the dtype and the device of the resulting tensor is determined by this method's parent object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Shape of the empty tensor to be created.
expected as multiple positional arguments of integers,
or as a single positional argument containing a tuple of
integers.
Note that when the user wishes to create an |
() |
num_solutions |
Optional[int] |
This can be used instead of the |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32) or, for creating an |
None |
device |
Union[str, torch.device] |
The device in which the new empty tensor will be stored.
If not specified (and also |
None |
use_eval_dtype |
bool |
If this is given as True and a |
False |
Returns:
| Type | Description |
|---|---|
Iterable |
The new empty tensor, which can be a PyTorch tensor or an
|
Source code in evotorch/tools/tensormaker.py
def make_empty(
self,
*size: Size,
num_solutions: Optional[int] = None,
out: Optional[Iterable] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> Iterable:
"""
Make an empty tensor.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
size: Shape of the empty tensor to be created.
expected as multiple positional arguments of integers,
or as a single positional argument containing a tuple of
integers.
Note that when the user wishes to create an `ObjectArray`
(i.e. when `dtype` is given as `object`), then the size
is expected as a single integer, or as a single-element
tuple containing an integer (because `ObjectArray` can only
be one-dimensional).
num_solutions: This can be used instead of the `size` arguments
for specifying the shape of the target tensor.
Expected as an integer, when `num_solutions` is specified
as `n`, the shape of the resulting tensor will be
`(n, m)` where `m` is the solution length reported by this
method's parent object's `solution_length` attribute.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32) or, for creating an `ObjectArray`,
"object" (as string) or `object` or `Any`.
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
Returns:
The new empty tensor, which can be a PyTorch tensor or an
`ObjectArray`.
"""
args, kwargs = self.__get_all_args_for_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
)
return misc.make_empty(*args, **kwargs)
make_gaussian(self, *size, *, num_solutions=None, center=None, stdev=None, symmetric=False, out=None, dtype=None, device=None, use_eval_dtype=False, generator=None)
¶
Make a new or existing tensor filled by Gaussian distributed values. This function can work only with float dtypes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Size of the new tensor to be filled with Gaussian distributed values. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor instead, then no positional argument is expected. |
() |
num_solutions |
Optional[int] |
This can be used instead of the |
None |
center |
Union[float, Iterable[float], torch.Tensor] |
Center point (i.e. mean) of the Gaussian distribution.
Can be a scalar, or a tensor.
If not specified, the center point will be taken as 0.
Note that, if one specifies |
None |
stdev |
Union[float, Iterable[float], torch.Tensor] |
Standard deviation for the Gaussian distributed values.
Can be a scalar, or a tensor.
If not specified, the standard deviation will be taken as 1.
Note that, if one specifies |
None |
symmetric |
bool |
Whether or not the values should be sampled in a symmetric (i.e. antithetic) manner. The default is False. |
False |
out |
Optional[torch.Tensor] |
Optionally, the tensor to be filled by Gaussian distributed
values. If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If |
None |
device |
Union[str, torch.device] |
The device in which the new empty tensor will be stored.
If not specified (and also |
None |
use_eval_dtype |
bool |
If this is given as True and a |
False |
generator |
Any |
Pseudo-random generator to be used when sampling
the values. Can be a |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing the Gaussian distributed values. |
Source code in evotorch/tools/tensormaker.py
def make_gaussian(
self,
*size: Size,
num_solutions: Optional[int] = None,
center: Optional[RealOrVector] = None,
stdev: Optional[RealOrVector] = None,
symmetric: bool = False,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
generator: Any = None,
) -> torch.Tensor:
"""
Make a new or existing tensor filled by Gaussian distributed values.
This function can work only with float dtypes.
Args:
size: Size of the new tensor to be filled with Gaussian distributed
values. This can be given as multiple positional arguments, each
such positional argument being an integer, or as a single
positional argument of a tuple, the tuple containing multiple
integers. Note that, if the user wishes to fill an existing
tensor instead, then no positional argument is expected.
num_solutions: This can be used instead of the `size` arguments
for specifying the shape of the target tensor.
Expected as an integer, when `num_solutions` is specified
as `n`, the shape of the resulting tensor will be
`(n, m)` where `m` is the solution length reported by this
method's parent object's `solution_length` attribute.
center: Center point (i.e. mean) of the Gaussian distribution.
Can be a scalar, or a tensor.
If not specified, the center point will be taken as 0.
Note that, if one specifies `center`, then `stdev` is also
expected to be explicitly specified.
stdev: Standard deviation for the Gaussian distributed values.
Can be a scalar, or a tensor.
If not specified, the standard deviation will be taken as 1.
Note that, if one specifies `stdev`, then `center` is also
expected to be explicitly specified.
symmetric: Whether or not the values should be sampled in a
symmetric (i.e. antithetic) manner.
The default is False.
out: Optionally, the tensor to be filled by Gaussian distributed
values. If an `out` tensor is given, then no `size` argument is
expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
generator: Pseudo-random generator to be used when sampling
the values. Can be a `torch.Generator` or any object with
a `generator` attribute (e.g. a Problem object).
If not given, then this method's parent object will be
analyzed whether or not it has its own generator.
If it does, that generator will be used.
If not, the global generator of PyTorch will be used.
Returns:
The created or modified tensor after placing the Gaussian
distributed values.
"""
args, kwargs = self.__get_all_args_for_random_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
generator=generator,
)
return misc.make_gaussian(*args, center=center, stdev=stdev, symmetric=symmetric, **kwargs)
make_gaussian_shaped_like(self, t, *, center=None, stdev=None)
¶
Make a new tensor, shaped like the given tensor, with its values filled by the Gaussian distribution.
The dtype and device will be determined by the parent of this
method (not by the given tensor).
If the parent of this method has its own random generator, then that
generator will be used.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
t |
Tensor |
The tensor according to which the result will be shaped. |
required |
center |
Union[float, Iterable[float], torch.Tensor] |
Center point for the Gaussian distribution. Can be a scalar or a tensor. If left as None, 0.0 will be used as the center point. |
None |
stdev |
Union[float, Iterable[float], torch.Tensor] |
The standard deviation for the Gaussian distribution. Can be a scalar or a tensor. If left as None, 1.0 will be used as the standard deviation. |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
A new tensor whose shape is the same with the given tensor. |
Source code in evotorch/tools/tensormaker.py
def make_gaussian_shaped_like(
self,
t: torch.Tensor,
*,
center: Optional[RealOrVector] = None,
stdev: Optional[RealOrVector] = None,
) -> torch.Tensor:
"""
Make a new tensor, shaped like the given tensor, with its values
filled by the Gaussian distribution.
The `dtype` and `device` will be determined by the parent of this
method (not by the given tensor).
If the parent of this method has its own random generator, then that
generator will be used.
Args:
t: The tensor according to which the result will be shaped.
center: Center point for the Gaussian distribution.
Can be a scalar or a tensor.
If left as None, 0.0 will be used as the center point.
stdev: The standard deviation for the Gaussian distribution.
Can be a scalar or a tensor.
If left as None, 1.0 will be used as the standard deviation.
Returns:
A new tensor whose shape is the same with the given tensor.
"""
return self.make_gaussian(t.shape, center=center, stdev=stdev)
make_nan(self, *size, *, num_solutions=None, out=None, dtype=None, device=None, use_eval_dtype=False)
¶
Make a new tensor filled with NaN values, or fill an existing tensor with NaN values.
When not explicitly specified via arguments, the dtype and the device of the resulting tensor is determined by this method's parent object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Size of the new tensor to be filled with NaN. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor with NaN values, then no positional argument is expected. |
() |
num_solutions |
Optional[int] |
This can be used instead of the |
None |
out |
Optional[torch.Tensor] |
Optionally, the tensor to be filled by NaN values.
If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If |
None |
device |
Union[str, torch.device] |
The device in which the new empty tensor will be stored.
If not specified (and also |
None |
use_eval_dtype |
bool |
If this is given as True and a |
False |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing NaN values. |
Source code in evotorch/tools/tensormaker.py
def make_nan(
self,
*size: Size,
num_solutions: Optional[int] = None,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> torch.Tensor:
"""
Make a new tensor filled with NaN values, or fill an existing tensor
with NaN values.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
size: Size of the new tensor to be filled with NaN.
This can be given as multiple positional arguments, each such
positional argument being an integer, or as a single positional
argument of a tuple, the tuple containing multiple integers.
Note that, if the user wishes to fill an existing tensor with
NaN values, then no positional argument is expected.
num_solutions: This can be used instead of the `size` arguments
for specifying the shape of the target tensor.
Expected as an integer, when `num_solutions` is specified
as `n`, the shape of the resulting tensor will be
`(n, m)` where `m` is the solution length reported by this
method's parent object's `solution_length` attribute.
out: Optionally, the tensor to be filled by NaN values.
If an `out` tensor is given, then no `size` argument is expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
Returns:
The created or modified tensor after placing NaN values.
"""
args, kwargs = self.__get_all_args_for_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
)
return misc.make_nan(*args, **kwargs)
make_ones(self, *size, *, num_solutions=None, out=None, dtype=None, device=None, use_eval_dtype=False)
¶
Make a new tensor filled with 1, or fill an existing tensor with 1.
When not explicitly specified via arguments, the dtype and the device of the resulting tensor is determined by this method's parent object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Size of the new tensor to be filled with 1. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor with 1 values, then no positional argument is expected. |
() |
num_solutions |
Optional[int] |
This can be used instead of the |
None |
out |
Optional[torch.Tensor] |
Optionally, the tensor to be filled by 1 values.
If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If |
None |
device |
Union[str, torch.device] |
The device in which the new empty tensor will be stored.
If not specified (and also |
None |
use_eval_dtype |
bool |
If this is given as True and a |
False |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing 1 values. |
Source code in evotorch/tools/tensormaker.py
def make_ones(
self,
*size: Size,
num_solutions: Optional[int] = None,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> torch.Tensor:
"""
Make a new tensor filled with 1, or fill an existing tensor with 1.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
size: Size of the new tensor to be filled with 1.
This can be given as multiple positional arguments, each such
positional argument being an integer, or as a single positional
argument of a tuple, the tuple containing multiple integers.
Note that, if the user wishes to fill an existing tensor with
1 values, then no positional argument is expected.
num_solutions: This can be used instead of the `size` arguments
for specifying the shape of the target tensor.
Expected as an integer, when `num_solutions` is specified
as `n`, the shape of the resulting tensor will be
`(n, m)` where `m` is the solution length reported by this
method's parent object's `solution_length` attribute.
out: Optionally, the tensor to be filled by 1 values.
If an `out` tensor is given, then no `size` argument is expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
Returns:
The created or modified tensor after placing 1 values.
"""
args, kwargs = self.__get_all_args_for_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
)
return misc.make_ones(*args, **kwargs)
make_randint(self, *size, *, n, num_solutions=None, out=None, dtype=None, device=None, use_eval_dtype=False, generator=None)
¶
Make a new or existing tensor filled by random integers.
The integers are uniformly distributed within [0 ... n-1].
This function can be used with integer or float dtypes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Size of the new tensor to be filled with uniformly distributed values. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor instead, then no positional argument is expected. |
() |
n |
Union[int, float, torch.Tensor] |
Number of choice(s) for integer sampling.
The lowest possible value will be 0, and the highest possible
value will be n - 1.
|
required |
out |
Optional[torch.Tensor] |
Optionally, the tensor to be filled by the random integers.
If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "int64") or a PyTorch dtype
(e.g. torch.int64).
If |
None |
device |
Union[str, torch.device] |
The device in which the new empty tensor will be stored.
If not specified (and also |
None |
use_eval_dtype |
bool |
If this is given as True and a |
False |
generator |
Any |
Pseudo-random generator to be used when sampling
the values. Can be a |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing the uniformly distributed values. |
Source code in evotorch/tools/tensormaker.py
def make_randint(
self,
*size: Size,
n: Union[int, float, torch.Tensor],
num_solutions: Optional[int] = None,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
generator: Any = None,
) -> torch.Tensor:
"""
Make a new or existing tensor filled by random integers.
The integers are uniformly distributed within `[0 ... n-1]`.
This function can be used with integer or float dtypes.
Args:
size: Size of the new tensor to be filled with uniformly distributed
values. This can be given as multiple positional arguments, each
such positional argument being an integer, or as a single
positional argument of a tuple, the tuple containing multiple
integers. Note that, if the user wishes to fill an existing
tensor instead, then no positional argument is expected.
n: Number of choice(s) for integer sampling.
The lowest possible value will be 0, and the highest possible
value will be n - 1.
`n` can be a scalar, or a tensor.
out: Optionally, the tensor to be filled by the random integers.
If an `out` tensor is given, then no `size` argument is
expected.
dtype: Optionally a string (e.g. "int64") or a PyTorch dtype
(e.g. torch.int64).
If `dtype` is not specified (and also `out` is None),
`torch.int64` will be used.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
generator: Pseudo-random generator to be used when sampling
the values. Can be a `torch.Generator` or any object with
a `generator` attribute (e.g. a Problem object).
If not given, then this method's parent object will be
analyzed whether or not it has its own generator.
If it does, that generator will be used.
If not, the global generator of PyTorch will be used.
Returns:
The created or modified tensor after placing the uniformly
distributed values.
"""
if (dtype is None) and (out is None):
dtype = torch.int64
args, kwargs = self.__get_all_args_for_random_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
generator=generator,
)
return misc.make_randint(*args, n=n, **kwargs)
make_tensor(self, data, *, dtype=None, device=None, use_eval_dtype=False, read_only=False)
¶
Make a new tensor.
When not explicitly specified via arguments, the dtype and the device of the resulting tensor is determined by this method's parent object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data |
Any |
The data to be converted to a tensor.
If one wishes to create a PyTorch tensor, this can be anything
that can be stored by a PyTorch tensor.
If one wishes to create an |
required |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32"), or a PyTorch dtype
(e.g. torch.float32), or |
None |
device |
Union[str, torch.device] |
The device in which the tensor will be stored.
If |
None |
use_eval_dtype |
bool |
If this is given as True and a |
False |
read_only |
bool |
Whether or not the created tensor will be read-only. By default, this is False. |
False |
Returns:
| Type | Description |
|---|---|
Iterable |
A PyTorch tensor or an ObjectArray. |
Source code in evotorch/tools/tensormaker.py
def make_tensor(
self,
data: Any,
*,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
read_only: bool = False,
) -> Iterable:
"""
Make a new tensor.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
data: The data to be converted to a tensor.
If one wishes to create a PyTorch tensor, this can be anything
that can be stored by a PyTorch tensor.
If one wishes to create an `ObjectArray` and therefore passes
`dtype=object`, then the provided `data` is expected as an
`Iterable`.
dtype: Optionally a string (e.g. "float32"), or a PyTorch dtype
(e.g. torch.float32), or `object` or "object" (as a string)
or `Any` if one wishes to create an `ObjectArray`.
If `dtype` is not specified it will be assumed that the user
wishes to create a tensor using the dtype of this method's
parent object.
device: The device in which the tensor will be stored.
If `device` is not specified, it will be assumed that the user
wishes to create a tensor on the device of this method's
parent object.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
read_only: Whether or not the created tensor will be read-only.
By default, this is False.
Returns:
A PyTorch tensor or an ObjectArray.
"""
kwargs = self.__get_dtype_and_device_kwargs(dtype=dtype, device=device, use_eval_dtype=use_eval_dtype, out=None)
return misc.make_tensor(data, read_only=read_only, **kwargs)
make_uniform(self, *size, *, num_solutions=None, lb=None, ub=None, out=None, dtype=None, device=None, use_eval_dtype=False, generator=None)
¶
Make a new or existing tensor filled by uniformly distributed values. Both lower and upper bounds are inclusive. This function can work with both float and int dtypes.
When not explicitly specified via arguments, the dtype and the device of the resulting tensor is determined by this method's parent object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Size of the new tensor to be filled with uniformly distributed values. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor instead, then no positional argument is expected. |
() |
num_solutions |
Optional[int] |
This can be used instead of the |
None |
lb |
Union[float, Iterable[float], torch.Tensor] |
Lower bound for the uniformly distributed values.
Can be a scalar, or a tensor.
If not specified, the lower bound will be taken as 0.
Note that, if one specifies |
None |
ub |
Union[float, Iterable[float], torch.Tensor] |
Upper bound for the uniformly distributed values.
Can be a scalar, or a tensor.
If not specified, the upper bound will be taken as 1.
Note that, if one specifies |
None |
out |
Optional[torch.Tensor] |
Optionally, the tensor to be filled by uniformly distributed
values. If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If |
None |
device |
Union[str, torch.device] |
The device in which the new empty tensor will be stored.
If not specified (and also |
None |
use_eval_dtype |
bool |
If this is given as True and a |
False |
generator |
Any |
Pseudo-random generator to be used when sampling
the values. Can be a |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing the uniformly distributed values. |
Source code in evotorch/tools/tensormaker.py
def make_uniform(
self,
*size: Size,
num_solutions: Optional[int] = None,
lb: Optional[RealOrVector] = None,
ub: Optional[RealOrVector] = None,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
generator: Any = None,
) -> torch.Tensor:
"""
Make a new or existing tensor filled by uniformly distributed values.
Both lower and upper bounds are inclusive.
This function can work with both float and int dtypes.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
size: Size of the new tensor to be filled with uniformly distributed
values. This can be given as multiple positional arguments, each
such positional argument being an integer, or as a single
positional argument of a tuple, the tuple containing multiple
integers. Note that, if the user wishes to fill an existing
tensor instead, then no positional argument is expected.
num_solutions: This can be used instead of the `size` arguments
for specifying the shape of the target tensor.
Expected as an integer, when `num_solutions` is specified
as `n`, the shape of the resulting tensor will be
`(n, m)` where `m` is the solution length reported by this
method's parent object's `solution_length` attribute.
lb: Lower bound for the uniformly distributed values.
Can be a scalar, or a tensor.
If not specified, the lower bound will be taken as 0.
Note that, if one specifies `lb`, then `ub` is also expected to
be explicitly specified.
ub: Upper bound for the uniformly distributed values.
Can be a scalar, or a tensor.
If not specified, the upper bound will be taken as 1.
Note that, if one specifies `ub`, then `lb` is also expected to
be explicitly specified.
out: Optionally, the tensor to be filled by uniformly distributed
values. If an `out` tensor is given, then no `size` argument is
expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
generator: Pseudo-random generator to be used when sampling
the values. Can be a `torch.Generator` or any object with
a `generator` attribute (e.g. a Problem object).
If not given, then this method's parent object will be
analyzed whether or not it has its own generator.
If it does, that generator will be used.
If not, the global generator of PyTorch will be used.
Returns:
The created or modified tensor after placing the uniformly
distributed values.
"""
args, kwargs = self.__get_all_args_for_random_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
generator=generator,
)
return misc.make_uniform(*args, lb=lb, ub=ub, **kwargs)
make_uniform_shaped_like(self, t, *, lb=None, ub=None)
¶
Make a new uniformly-filled tensor, shaped like the given tensor.
The dtype and device will be determined by the parent of this
method (not by the given tensor).
If the parent of this method has its own random generator, then that
generator will be used.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
t |
Tensor |
The tensor according to which the result will be shaped. |
required |
lb |
Union[float, Iterable[float], torch.Tensor] |
The inclusive lower bounds for the uniform distribution. Can be a scalar or a tensor. If left as None, 0.0 will be used as the upper bound. |
None |
ub |
Union[float, Iterable[float], torch.Tensor] |
The inclusive upper bounds for the uniform distribution. Can be a scalar or a tensor. If left as None, 1.0 will be used as the upper bound. |
None |
Returns:
| Type | Description |
|---|---|
Tensor |
A new tensor whose shape is the same with the given tensor. |
Source code in evotorch/tools/tensormaker.py
def make_uniform_shaped_like(
self,
t: torch.Tensor,
*,
lb: Optional[RealOrVector] = None,
ub: Optional[RealOrVector] = None,
) -> torch.Tensor:
"""
Make a new uniformly-filled tensor, shaped like the given tensor.
The `dtype` and `device` will be determined by the parent of this
method (not by the given tensor).
If the parent of this method has its own random generator, then that
generator will be used.
Args:
t: The tensor according to which the result will be shaped.
lb: The inclusive lower bounds for the uniform distribution.
Can be a scalar or a tensor.
If left as None, 0.0 will be used as the upper bound.
ub: The inclusive upper bounds for the uniform distribution.
Can be a scalar or a tensor.
If left as None, 1.0 will be used as the upper bound.
Returns:
A new tensor whose shape is the same with the given tensor.
"""
return self.make_uniform(t.shape, lb=lb, ub=ub)
make_zeros(self, *size, *, num_solutions=None, out=None, dtype=None, device=None, use_eval_dtype=False)
¶
Make a new tensor filled with 0, or fill an existing tensor with 0.
When not explicitly specified via arguments, the dtype and the device of the resulting tensor is determined by this method's parent object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
size |
Union[int, torch.Size] |
Size of the new tensor to be filled with 0. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor with 0 values, then no positional argument is expected. |
() |
num_solutions |
Optional[int] |
This can be used instead of the |
None |
out |
Optional[torch.Tensor] |
Optionally, the tensor to be filled by 0 values.
If an |
None |
dtype |
Union[str, torch.dtype, numpy.dtype, Type] |
Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If |
None |
device |
Union[str, torch.device] |
The device in which the new empty tensor will be stored.
If not specified (and also |
None |
use_eval_dtype |
bool |
If this is given as True and a |
False |
Returns:
| Type | Description |
|---|---|
Tensor |
The created or modified tensor after placing 0 values. |
Source code in evotorch/tools/tensormaker.py
def make_zeros(
self,
*size: Size,
num_solutions: Optional[int] = None,
out: Optional[torch.Tensor] = None,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
use_eval_dtype: bool = False,
) -> torch.Tensor:
"""
Make a new tensor filled with 0, or fill an existing tensor with 0.
When not explicitly specified via arguments, the dtype and the device
of the resulting tensor is determined by this method's parent object.
Args:
size: Size of the new tensor to be filled with 0.
This can be given as multiple positional arguments, each such
positional argument being an integer, or as a single positional
argument of a tuple, the tuple containing multiple integers.
Note that, if the user wishes to fill an existing tensor with
0 values, then no positional argument is expected.
num_solutions: This can be used instead of the `size` arguments
for specifying the shape of the target tensor.
Expected as an integer, when `num_solutions` is specified
as `n`, the shape of the resulting tensor will be
`(n, m)` where `m` is the solution length reported by this
method's parent object's `solution_length` attribute.
out: Optionally, the tensor to be filled by 0 values.
If an `out` tensor is given, then no `size` argument is expected.
dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
(e.g. torch.float32).
If `dtype` is not specified (and also `out` is None),
it will be assumed that the user wishes to create a tensor
using the dtype of this method's parent object.
If an `out` tensor is specified, then `dtype` is expected
as None.
device: The device in which the new empty tensor will be stored.
If not specified (and also `out` is None), it will be
assumed that the user wishes to create a tensor on the
same device with this method's parent object.
If an `out` tensor is specified, then `device` is expected
as None.
use_eval_dtype: If this is given as True and a `dtype` is not
specified, then the `dtype` of the result will be taken
from the `eval_dtype` attribute of this method's parent
object.
Returns:
The created or modified tensor after placing 0 values.
"""
args, kwargs = self.__get_all_args_for_maker(
*size,
num_solutions=num_solutions,
out=out,
dtype=dtype,
device=device,
use_eval_dtype=use_eval_dtype,
)
return misc.make_zeros(*args, **kwargs)