Index

This namespace contains various utility functions, classes, and type aliases.

ErroneousResult

Representation of a caught error being returned as a result.

Source code in evotorch/tools/misc.py

class ErroneousResult:
    """
    Representation of a caught error being returned as a result.
    """

    def __init__(self, error: Exception):
        self.error = error

    def _to_string(self) -> str:
        return f"<{type(self).__name__}, error: {self.error}>"

    def __str__(self) -> str:
        return self._to_string()

    def __repr__(self) -> str:
        return self._to_string()

    def __bool__(self) -> bool:
        return False

    @staticmethod
    def call(f, *args, **kwargs) -> Any:
        """
        Call a function with the given arguments.
        If the function raises an error, wrap the error in an ErroneousResult
        object, and return that ErroneousResult object instead.

        Returns:
            The result of the function if there was no error,
            or an ErroneousResult if there was an error.
        """
        try:
            result = f(*args, **kwargs)
        except Exception as ex:
            result = ErroneousResult(ex)
        return result

call(f, *args, **kwargs) staticmethod

Call a function with the given arguments. If the function raises an error, wrap the error in an ErroneousResult object, and return that ErroneousResult object instead.

Returns:

Type	Description
Any	The result of the function if there was no error,
Any	or an ErroneousResult if there was an error.

Source code in evotorch/tools/misc.py

@staticmethod
def call(f, *args, **kwargs) -> Any:
    """
    Call a function with the given arguments.
    If the function raises an error, wrap the error in an ErroneousResult
    object, and return that ErroneousResult object instead.

    Returns:
        The result of the function if there was no error,
        or an ErroneousResult if there was an error.
    """
    try:
        result = f(*args, **kwargs)
    except Exception as ex:
        result = ErroneousResult(ex)
    return result

Hook

Bases: MutableSequence

A Hook stores a list of callable objects to be called for handling certain events. A Hook itself is callable, which invokes the callables stored in its list. If the callables stored by the Hook return list-like objects or dict-like objects, their returned results are accumulated, and then those accumulated results are finally returned by the Hook.

Source code in evotorch/tools/hook.py

class Hook(MutableSequence):
    """
    A Hook stores a list of callable objects to be called for handling
    certain events. A Hook itself is callable, which invokes the callables
    stored in its list. If the callables stored by the Hook return list-like
    objects or dict-like objects, their returned results are accumulated,
    and then those accumulated results are finally returned by the Hook.
    """

    def __init__(
        self,
        callables: Optional[Iterable[Callable]] = None,
        *,
        args: Optional[Iterable] = None,
        kwargs: Optional[Mapping] = None,
    ):
        """
        Initialize the Hook.

        Args:
            callables: A sequence of callables to be stored by the Hook.
            args: Positional arguments which, when the Hook is called,
                are to be passed to every callable stored by the Hook.
                Please note that these positional arguments will be passed
                as the leftmost arguments, and, the other positional
                arguments passed via the `__call__(...)` method of the
                Hook will be added to the right of these arguments.
            kwargs: Keyword arguments which, when the Hook is called,
                are to be passed to every callable stored by the Hook.
                Please note that these keyword arguments could be overriden
                by the keyword arguments passed via the `__call__(...)`
                method of the Hook.
        """
        self._funcs: list = [] if callables is None else list(callables)
        self._args: list = [] if args is None else list(args)
        self._kwargs: dict = {} if kwargs is None else dict(kwargs)

    def __call__(self, *args: Any, **kwargs: Any) -> Optional[Union[dict, list]]:
        """
        Call every callable object stored by the Hook.
        The results of the stored callable objects (which can be dict-like
        or list-like objects) are accumulated and finally returned.

        Args:
            args: Additional positional arguments to be passed to the stored
                callables.
            kwargs: Additional keyword arguments to be passed to the stored
                keyword arguments.
        """

        all_args = []
        all_args.extend(self._args)
        all_args.extend(args)

        all_kwargs = {}
        all_kwargs.update(self._kwargs)
        all_kwargs.update(kwargs)

        result: Optional[Union[dict, list]] = None

        for f in self._funcs:
            tmp = f(*all_args, **all_kwargs)
            if tmp is not None:
                if isinstance(tmp, Mapping):
                    if result is None:
                        result = dict(tmp)
                    elif isinstance(result, list):
                        raise TypeError(
                            f"The function {f} returned a dict-like object."
                            f" However, previous function(s) in this hook had returned list-like object(s)."
                            f" Such incompatible results cannot be accumulated."
                        )
                    elif isinstance(result, dict):
                        result.update(tmp)
                    else:
                        raise RuntimeError
                elif isinstance(tmp, Iterable):
                    if result is None:
                        result = list(tmp)
                    elif isinstance(result, list):
                        result.extend(tmp)
                    elif isinstance(result, dict):
                        raise TypeError(
                            f"The function {f} returned a list-like object."
                            f" However, previous function(s) in this hook had returned dict-like object(s)."
                            f" Such incompatible results cannot be accumulated."
                        )
                    else:
                        raise RuntimeError
                else:
                    raise TypeError(
                        f"Expected the function {f} to return None, or a dict-like object, or a list-like object."
                        f" However, the function returned an object of type {repr(type(tmp))}."
                    )

        return result

    def accumulate_dict(self, *args: Any, **kwargs: Any) -> Optional[Union[dict, list]]:
        result = self(*args, **kwargs)
        if result is None:
            return {}
        elif isinstance(result, Mapping):
            return result
        else:
            raise TypeError(
                f"Expected the functions in this hook to accumulate"
                f" dictionary-like objects. Instead, accumulated"
                f" an object of type {type(result)}."
                f" Hint: are the functions registered in this hook"
                f" returning non-dictionary iterables?"
            )

    def accumulate_sequence(self, *args: Any, **kwargs: Any) -> Optional[Union[dict, list]]:
        result = self(*args, **kwargs)
        if result is None:
            return []
        elif isinstance(result, Mapping):
            raise TypeError(
                f"Expected the functions in this hook to accumulate"
                f" sequences (that are NOT dictionaries). Instead, accumulated"
                f" a dict-like object of type {type(result)}."
                f" Hint: are the functions registered in this hook"
                f" returning objects with Mapping interface?"
            )
        else:
            return result

    def _to_string(self) -> str:
        init_args = [repr(self._funcs)]

        if len(self._args) > 0:
            init_args.append(f"args={self._args}")

        if len(self._kwargs) > 0:
            init_args.append(f"kwargs={self._kwargs}")

        s_init_args = ", ".join(init_args)

        return f"{type(self).__name__}({s_init_args})"

    def __repr__(self) -> str:
        return self._to_string()

    def __str__(self) -> str:
        return self._to_string()

    def __getitem__(self, i: Union[int, slice]) -> Union[Callable, "Hook"]:
        if isinstance(i, slice):
            return Hook(self._funcs[i], args=self._args, kwargs=self._kwargs)
        else:
            return self._funcs[i]

    def __setitem__(self, i: Union[int, slice], x: Iterable[Callable]):
        self._funcs[i] = x

    def __delitem__(self, i: Union[int, slice]):
        del self._funcs[i]

    def insert(self, i: int, x: Callable):
        self._funcs.insert(i, x)

    def __len__(self) -> int:
        return len(self._funcs)

    @property
    def args(self) -> list:
        """Positional arguments that will be passed to the stored callables"""
        return self._args

    @property
    def kwargs(self) -> dict:
        """Keyword arguments that will be passed to the stored callables"""
        return self._kwargs

args property

Positional arguments that will be passed to the stored callables

kwargs property

Keyword arguments that will be passed to the stored callables

__call__(*args, **kwargs)

Call every callable object stored by the Hook. The results of the stored callable objects (which can be dict-like or list-like objects) are accumulated and finally returned.

Parameters:

Name	Type	Description	Default
args	Any	Additional positional arguments to be passed to the stored callables.	()
kwargs	Any	Additional keyword arguments to be passed to the stored keyword arguments.	{}

Source code in evotorch/tools/hook.py

def __call__(self, *args: Any, **kwargs: Any) -> Optional[Union[dict, list]]:
    """
    Call every callable object stored by the Hook.
    The results of the stored callable objects (which can be dict-like
    or list-like objects) are accumulated and finally returned.

    Args:
        args: Additional positional arguments to be passed to the stored
            callables.
        kwargs: Additional keyword arguments to be passed to the stored
            keyword arguments.
    """

    all_args = []
    all_args.extend(self._args)
    all_args.extend(args)

    all_kwargs = {}
    all_kwargs.update(self._kwargs)
    all_kwargs.update(kwargs)

    result: Optional[Union[dict, list]] = None

    for f in self._funcs:
        tmp = f(*all_args, **all_kwargs)
        if tmp is not None:
            if isinstance(tmp, Mapping):
                if result is None:
                    result = dict(tmp)
                elif isinstance(result, list):
                    raise TypeError(
                        f"The function {f} returned a dict-like object."
                        f" However, previous function(s) in this hook had returned list-like object(s)."
                        f" Such incompatible results cannot be accumulated."
                    )
                elif isinstance(result, dict):
                    result.update(tmp)
                else:
                    raise RuntimeError
            elif isinstance(tmp, Iterable):
                if result is None:
                    result = list(tmp)
                elif isinstance(result, list):
                    result.extend(tmp)
                elif isinstance(result, dict):
                    raise TypeError(
                        f"The function {f} returned a list-like object."
                        f" However, previous function(s) in this hook had returned dict-like object(s)."
                        f" Such incompatible results cannot be accumulated."
                    )
                else:
                    raise RuntimeError
            else:
                raise TypeError(
                    f"Expected the function {f} to return None, or a dict-like object, or a list-like object."
                    f" However, the function returned an object of type {repr(type(tmp))}."
                )

    return result

__init__(callables=None, *, args=None, kwargs=None)

Initialize the Hook.

Parameters:

Name	Type	Description	Default
callables	Optional[Iterable[Callable]]	A sequence of callables to be stored by the Hook.	None
args	Optional[Iterable]	Positional arguments which, when the Hook is called, are to be passed to every callable stored by the Hook. Please note that these positional arguments will be passed as the leftmost arguments, and, the other positional arguments passed via the __call__(...) method of the Hook will be added to the right of these arguments.	None
kwargs	Optional[Mapping]	Keyword arguments which, when the Hook is called, are to be passed to every callable stored by the Hook. Please note that these keyword arguments could be overriden by the keyword arguments passed via the __call__(...) method of the Hook.	None

Source code in evotorch/tools/hook.py

def __init__(
    self,
    callables: Optional[Iterable[Callable]] = None,
    *,
    args: Optional[Iterable] = None,
    kwargs: Optional[Mapping] = None,
):
    """
    Initialize the Hook.

    Args:
        callables: A sequence of callables to be stored by the Hook.
        args: Positional arguments which, when the Hook is called,
            are to be passed to every callable stored by the Hook.
            Please note that these positional arguments will be passed
            as the leftmost arguments, and, the other positional
            arguments passed via the `__call__(...)` method of the
            Hook will be added to the right of these arguments.
        kwargs: Keyword arguments which, when the Hook is called,
            are to be passed to every callable stored by the Hook.
            Please note that these keyword arguments could be overriden
            by the keyword arguments passed via the `__call__(...)`
            method of the Hook.
    """
    self._funcs: list = [] if callables is None else list(callables)
    self._args: list = [] if args is None else list(args)
    self._kwargs: dict = {} if kwargs is None else dict(kwargs)

ObjectArray

Bases: Sequence, RecursivePrintable

An object container with an interface similar to PyTorch tensors.

It is strictly one-dimensional, and supports advanced indexing and slicing operations supported by PyTorch tensors.

An ObjectArray can store None values, strings, numbers, booleans, lists, sets, dictionaries, PyTorch tensors, and numpy arrays.

When a container (such as a list, dictionary, set, is placed into an ObjectArray, an immutable clone of this container is first created, and then this newly created immutable clone gets stored within the ObjectArray. This behavior is to prevent accidental modification of the stored data.

When a numeric array (such as a PyTorch tensor or a numpy array with a numeric dtype) is placed into an ObjectArray, the target ObjectArray first checks if the numeric array is read-only. If the numeric array is indeed read-only, then the array is put into the ObjectArray as it is. If the array is not read-only, then a read-only clone of the original numeric array is first created, and then this clone gets stored by the ObjectArray. This behavior has the following implications: (i) even when an ObjectArray is shared by multiple components of the program, the risk of accidental modification of the stored data through this shared ObjectArray is significantly reduced as the stored numeric arrays are read-only; (ii) although not recommended, one could still forcefully modify the numeric arrays stored by an ObjectArray by explicitly casting them as mutable arrays (in the case of a numpy array, one could forcefully set the WRITEABLE flag, and, in the case of a ReadOnlyTensor, one could forcefully cast it as a regular PyTorch tensor); (iii) if an already read-only array x is placed into an ObjectArray, but x shares its memory with a mutable array y, then the contents of the ObjectArray can be affected by modifying y. The implication (ii) is demonstrated as follows:

objs = ObjectArray(1)  # a single-element ObjectArray

# Place a numpy array into objs:
objs[0] = np.array([1, 2, 3], dtype=float)

# At this point, objs[0] is a read-only numpy array.
# objs[0] *= 2   # <- Not allowed

# Possible but NOT recommended:
objs.flags["WRITEABLE"] = True
objs[0] *= 2

The implication (iii) is demonstrated as follows:

objs = ObjectArray(1)  # a single-element ObjectArray

# Make a new mutable numpy array
y = np.array([1, 2, 3], dtype=float)

# Make a read-only view to y:
x = y[:]
x.flags["WRITEABLE"] = False

# Place x into objs.
objs[0] = x

# At this point, objs[0] is a read-only numpy array.
# objs[0] *= 2   # <- Not allowed

# During the operation of setting its 0-th item, the ObjectArray
# `objs` did not clone `x` because `x` was already read-only.
# However, the contents of `x` could actually be modified because
# `x` shares its memory with the mutable array `y`.

# Possible but NOT recommended:
y *= 2  # This affects both x and objs!

When a numpy array of dtype object is placed into an ObjectArray, a read-only ObjectArray copy of the original array will first be created, and then, this newly created ObjectArray will be stored by the outer ObjectArray.

An ObjectArray itself has a read-only mode, so that, in addition to its stored data, the ObjectArray itself can be protected against undesired modifications.

An interesting feature of PyTorch: if one slices a tensor A and the result is a new tensor B, and if B is sharing storage memory with A, then A.untyped_storage().data_ptr() and B.untyped_storage().data_ptr() will return the same pointer. This means, one can compare the storage pointers of A and B and see whether or not the two are sharing memory. ObjectArray was designed to have this exact behavior, so that one can understand if two ObjectArray instances are sharing memory. Note that NumPy does NOT have such a behavior. In more details, a NumPy array C and a NumPy array D could report different pointers even when D was created via a basic slicing operation on C.

Source code in evotorch/tools/objectarray.py

class ObjectArray(Sequence, RecursivePrintable):
    """
    An object container with an interface similar to PyTorch tensors.

    It is strictly one-dimensional, and supports advanced indexing and
    slicing operations supported by PyTorch tensors.

    An ObjectArray can store `None` values, strings, numbers, booleans,
    lists, sets, dictionaries, PyTorch tensors, and numpy arrays.

    When a container (such as a list, dictionary, set, is placed into an
    ObjectArray, an immutable clone of this container is first created, and
    then this newly created immutable clone gets stored within the
    ObjectArray. This behavior is to prevent accidental modification of the
    stored data.

    When a numeric array (such as a PyTorch tensor or a numpy array with a
    numeric dtype) is placed into an ObjectArray, the target ObjectArray
    first checks if the numeric array is read-only. If the numeric array
    is indeed read-only, then the array is put into the ObjectArray as it
    is. If the array is not read-only, then a read-only clone of the
    original numeric array is first created, and then this clone gets
    stored by the ObjectArray. This behavior has the following implications:
    (i) even when an ObjectArray is shared by multiple components of the
    program, the risk of accidental modification of the stored data through
    this shared ObjectArray is significantly reduced as the stored numeric
    arrays are read-only;
    (ii) although not recommended, one could still forcefully modify the
    numeric arrays stored by an ObjectArray by explicitly casting them as
    mutable arrays
    (in the case of a numpy array, one could forcefully set the WRITEABLE
    flag, and, in the case of a ReadOnlyTensor, one could forcefully cast it
    as a regular PyTorch tensor);
    (iii) if an already read-only array `x` is placed into an ObjectArray,
    but `x` shares its memory with a mutable array `y`, then the contents
    of the ObjectArray can be affected by modifying `y`.
    The implication (ii) is demonstrated as follows:

    ```python
    objs = ObjectArray(1)  # a single-element ObjectArray

    # Place a numpy array into objs:
    objs[0] = np.array([1, 2, 3], dtype=float)

    # At this point, objs[0] is a read-only numpy array.
    # objs[0] *= 2   # <- Not allowed

    # Possible but NOT recommended:
    objs.flags["WRITEABLE"] = True
    objs[0] *= 2
    ```

    The implication (iii) is demonstrated as follows:

    ```python
    objs = ObjectArray(1)  # a single-element ObjectArray

    # Make a new mutable numpy array
    y = np.array([1, 2, 3], dtype=float)

    # Make a read-only view to y:
    x = y[:]
    x.flags["WRITEABLE"] = False

    # Place x into objs.
    objs[0] = x

    # At this point, objs[0] is a read-only numpy array.
    # objs[0] *= 2   # <- Not allowed

    # During the operation of setting its 0-th item, the ObjectArray
    # `objs` did not clone `x` because `x` was already read-only.
    # However, the contents of `x` could actually be modified because
    # `x` shares its memory with the mutable array `y`.

    # Possible but NOT recommended:
    y *= 2  # This affects both x and objs!
    ```

    When a numpy array of dtype object is placed into an ObjectArray,
    a read-only ObjectArray copy of the original array will first be
    created, and then, this newly created ObjectArray will be stored
    by the outer ObjectArray.

    An ObjectArray itself has a read-only mode, so that, in addition to its
    stored data, the ObjectArray itself can be protected against undesired
    modifications.

    An interesting feature of PyTorch: if one slices a tensor A and the
    result is a new tensor B, and if B is sharing storage memory with A,
    then A.untyped_storage().data_ptr() and B.untyped_storage().data_ptr()
    will return the same pointer. This means, one can compare the storage
    pointers of A and B and see whether or not the two are sharing memory.
    ObjectArray was designed to have this exact behavior, so that one
    can understand if two ObjectArray instances are sharing memory.
    Note that NumPy does NOT have such a behavior. In more details,
    a NumPy array C and a NumPy array D could report different pointers
    even when D was created via a basic slicing operation on C.
    """

    def __init__(
        self,
        size: Optional[Size] = None,
        *,
        slice_of: Optional[tuple] = None,
    ):
        """
        `__init__(...)`: Instantiate a new ObjectArray.

        Args:
            size: Length of the ObjectArray. If this argument is present and
                is an integer `n`, then the resulting ObjectArray will be
                of length `n`, and will be filled with `None` values.
                This argument cannot be used together with the keyword
                argument `slice_of`.
            slice_of: Optionally a tuple in the form
                `(original_object_tensor, slice_info)`.
                When this argument is present, then the resulting ObjectArray
                will be a slice of the given `original_object_tensor` (which
                is expected as an ObjectArray instance). `slice_info` is
                either a `slice` instance, or a sequence of integers.
                The resulting ObjectArray might be a view of
                `original_object_tensor` (i.e. it might share its memory with
                `original_object_tensor`).
                This keyword argument cannot be used together with the
                argument `size`.
        """
        if size is not None and slice_of is not None:
            raise ValueError("Expected either `size` argument or `slice_of` argument, but got both.")
        elif size is None and slice_of is None:
            raise ValueError("Expected either `size` argument or `slice_of` argument, but got none.")
        elif size is not None:
            if not is_sequence(size):
                length = size
            elif isinstance(size, (np.ndarray, torch.Tensor)) and (size.ndim > 1):
                raise ValueError(f"Invalid size: {size}")
            else:
                [length] = size
            length = int(length)
            self._indices = torch.arange(length, dtype=torch.int64)
            self._objects = [None] * length
        elif slice_of is not None:
            source: ObjectArray

            source, slicing = slice_of

            if not isinstance(source, ObjectArray):
                raise TypeError(
                    f"`slice_of`: The first element was expected as an ObjectArray."
                    f" But it is of type {repr(type(source))}"
                )

            if isinstance(slicing, tuple) or is_integer(slicing):
                raise TypeError(f"Invalid slice: {slicing}")

            self._indices = source._indices[slicing]
            self._objects = source._objects

            if storage_ptr(self._indices) != storage_ptr(source._indices):
                self._objects = clone(self._objects)

        self._device = torch.device("cpu")
        self._read_only = False

    @property
    def shape(self) -> Size:
        """Shape of the ObjectArray, as a PyTorch Size tuple."""
        return self._indices.shape

    def size(self) -> Size:
        """
        Get the size of the ObjectArray, as a PyTorch Size tuple.

        Returns:
            The size (i.e. the shape) of the ObjectArray.
        """
        return self._indices.size()

    @property
    def ndim(self) -> int:
        """
        Number of dimensions handled by the ObjectArray.
        This is equivalent to getting the length of the size tuple.
        """
        return self._indices.ndim

    def dim(self) -> int:
        """
        Get the number of dimensions handled by the ObjectArray.
        This is equivalent to getting the length of the size tuple.

        Returns:
            The number of dimensions, as an integer.
        """
        return self._indices.dim()

    def numel(self) -> int:
        """
        Number of elements stored by the ObjectArray.

        Returns:
            The number of elements, as an integer.
        """
        return self._indices.numel()

    def repeat(self, *sizes) -> "ObjectArray":
        """
        Repeat the contents of this ObjectArray.

        For example, if we have an ObjectArray `objs` which stores
        `["hello", "world"]`, the following line:

            objs.repeat(3)

        will result in an ObjectArray which stores:

            `["hello", "world", "hello", "world", "hello", "world"]`

        Args:
            sizes: Although this argument is named `sizes` to be compatible
                with PyTorch, what is expected here is a single positional
                argument, as a single integer, or as a single-element
                tuple.
                The given integer (which can be the argument itself, or
                the integer within the given single-element tuple),
                specifies how many times the stored sequence will be
                repeated.
        Returns:
            A new ObjectArray which repeats the original one's values
        """

        if len(sizes) != 1:
            type_name = type(self).__name__
            raise ValueError(
                f"The `repeat(...)` method of {type_name} expects exactly one positional argument."
                f" This is because {type_name} supports only 1-dimensional storage."
                f" The received positional arguments are: {sizes}."
            )
        if isinstance(sizes, tuple):
            if len(sizes) == 1:
                sizes = sizes[0]
            else:
                type_name = type(self).__name__
                raise ValueError(
                    f"The `repeat(...)` method of {type_name} can accept a size tuple with only one element."
                    f" This is because {type_name} supports only 1-dimensional storage."
                    f" The received size tuple is: {sizes}."
                )
        num_repetitions = int(sizes[0])
        self_length = len(self)
        result = ObjectArray(num_repetitions * self_length)

        source_index = 0
        for result_index in range(len(result)):
            result[result_index] = self[source_index]
            source_index = (source_index + 1) % self_length

        return result

    @property
    def device(self) -> Device:
        """
        The device which stores the elements of the ObjectArray.
        In the case of ObjectArray, this property always returns
        the CPU device.

        Returns:
            The CPU device, as a torch.device object.
        """
        return self._device

    @property
    def dtype(self) -> DType:
        """
        The dtype of the elements stored by the ObjectArray.
        In the case of ObjectArray, the dtype is always `object`.
        """
        return object

    def __getitem__(self, i: Any) -> Any:
        if is_integer(i):
            index = int(self._indices[i])
            return self._objects[index]
        else:
            indices = self._indices[i]

            same_ptr = storage_ptr(indices) == storage_ptr(self._indices)

            result = ObjectArray(len(indices))

            if same_ptr:
                result._indices[:] = indices
                result._objects = self._objects
            else:
                result._objects = []
                for index in indices:
                    result._objects.append(self._objects[int(index)])

            result._read_only = self._read_only

            return result

    def __setitem__(self, i: Any, x: Any):
        self.set_item(i, x)

    def set_item(self, i: Any, x: Any, *, memo: Optional[dict] = None):
        """
        Set the i-th item of the ObjectArray as x.

        Args:
            i: An index or a slice.
            x: The object that will be put into the ObjectArray.
            memo: Optionally a dictionary which maps from the ids of the
                already placed objects to their clones within ObjectArray.
                In most scenarios, when this method is called from outside,
                this can be left as None.
        """
        from .immutable import as_immutable

        if memo is None:
            memo = {}

        memo[id(self)] = self

        if self._read_only:
            raise ValueError("This ObjectArray is read-only, therefore, modification is not allowed.")

        if is_integer(i):
            index = int(self._indices[i])
            self._objects[index] = as_immutable(x, memo=memo)
        else:
            indices = self._indices[i]
            if not isinstance(x, Iterable):
                raise TypeError(f"Expected an iterable, but got {repr(x)}")

            if indices.ndim != 1:
                raise ValueError(
                    "Received indices that would change the dimensionality of the ObjectArray."
                    " However, an ObjectArray can only be 1-dimensional."
                )

            slice_refers_to_whole_array = (len(indices) == len(self._indices)) and torch.all(indices == self._indices)
            if slice_refers_to_whole_array:
                memo[id(x)] = self

            if not hasattr(x, "__len__"):
                x = list(x)

            if len(x) != len(indices):
                raise TypeError(
                    f"The slicing operation refers to {len(indices)} elements."
                    f" However, the given objects sequence has {len(x)} elements."
                )

            for q, obj in enumerate(x):
                index = int(indices[q])
                self._objects[index] = as_immutable(obj, memo=memo)

    def __len__(self) -> int:
        return len(self._indices)

    def __iter__(self):
        for i in range(len(self)):
            yield self[i]

    def clone(self, *, preserve_read_only: bool = False, memo: Optional[dict] = None) -> Iterable:
        """
        Get a deep copy of the ObjectArray.

        Args:
            preserve_read_only: Whether or not to preserve the read-only
                attribute. Note that the default value is False, which
                means that the newly made clone will NOT be read-only
                even if the original ObjectArray is.
            memo: Optionally a dictionary which maps from the ids of the
                already cloned objects to their clones.
                In most scenarios, when this method is called from outside,
                this can be left as None.
        Returns:
            The clone of the original ObjectArray.
        """
        from .cloning import deep_clone

        if memo is None:
            memo = {}

        self_id = id(self)
        if self_id in memo:
            return memo[self_id]

        if not preserve_read_only:
            return self.numpy(memo=memo)
        else:
            result = ObjectArray(len(self))
            memo[self_id] = result

            for i, item in enumerate(self):
                result[i] = deep_clone(item, otherwise_deepcopy=True, memo=memo)

            return result

    def __copy__(self) -> "ObjectArray":
        return self.clone(preserve_read_only=True)

    def __deepcopy__(self, memo: Optional[dict]) -> "ObjectArray":
        if memo is None:
            memo = {}
        return self.clone(preserve_read_only=True, memo=memo)

    def __setstate__(self, state: dict):
        self.__dict__.update(state)

        # After pickling and unpickling, numpy arrays become mutable.
        # Since we are dealing with immutable containers here, we need to forcefully make all numpy arrays read-only.
        for v in self:
            if isinstance(v, np.ndarray):
                v.flags["WRITEABLE"] = False

    # def __getstate__(self) -> dict:
    #     from .cloning import deep_clone
    #     self_id = id(self)
    #     memo = {self_id: self}
    #     cloned_dict = deep_clone(self.__dict__, otherwise_deepcopy=True, memo=memo)
    #     return cloned_dict

    def get_read_only_view(self) -> "ObjectArray":
        """
        Get a read-only view of this ObjectArray.
        """
        result = self[:]
        result._read_only = True
        return result

    @property
    def is_read_only(self) -> bool:
        """
        True if this ObjectArray is read-only; False otherwise.
        """
        return self._read_only

    def storage(self) -> ObjectArrayStorage:
        return ObjectArrayStorage(self)

    def untyped_storage(self) -> ObjectArrayStorage:
        return ObjectArrayStorage(self)

    def numpy(self, *, memo: Optional[dict] = None) -> np.ndarray:
        """
        Convert this ObjectArray to a numpy array.

        The resulting numpy array will have its dtype set as `object`.
        This new array itself and its contents will be mutable (those
        mutable objects being the copies of their immutable sources).

        Returns:
            The numpy counterpart of this ObjectArray.
        """
        from .immutable import mutable_copy

        if memo is None:
            memo = {}

        n = len(self)
        result = np.empty(n, dtype=object)

        memo[id(self)] = result

        for i, item in enumerate(self):
            result[i] = mutable_copy(item, memo=memo)

        return result

    @staticmethod
    def from_numpy(ndarray: np.ndarray) -> "ObjectArray":
        """
        Convert a numpy array of dtype `object` to an `ObjectArray`.

        Args:
            ndarray: The numpy array that will be converted to `ObjectArray`.
        Returns:
            The ObjectArray counterpart of the given numpy array.
        """
        if isinstance(ndarray, np.ndarray):
            if ndarray.dtype == np.dtype(object):
                n = len(ndarray)
                result = ObjectArray(n)
                for i, element in enumerate(ndarray):
                    result[i] = element
                return result
            else:
                raise ValueError(
                    f"The dtype of the given array was expected as `object`."
                    f" However, the dtype was encountered as {ndarray.dtype}."
                )
        else:
            raise TypeError(f"Expected a `numpy.ndarray` instance, but received an object of type {type(ndarray)}.")

device property

The device which stores the elements of the ObjectArray. In the case of ObjectArray, this property always returns the CPU device.

Returns:

Type	Description
Device	The CPU device, as a torch.device object.

dtype property

The dtype of the elements stored by the ObjectArray. In the case of ObjectArray, the dtype is always object.

is_read_only property

True if this ObjectArray is read-only; False otherwise.

ndim property

Number of dimensions handled by the ObjectArray. This is equivalent to getting the length of the size tuple.

shape property

Shape of the ObjectArray, as a PyTorch Size tuple.

__init__(size=None, *, slice_of=None)

__init__(...): Instantiate a new ObjectArray.

Parameters:

Name	Type	Description	Default
size	Optional[Size]	Length of the ObjectArray. If this argument is present and is an integer n, then the resulting ObjectArray will be of length n, and will be filled with None values. This argument cannot be used together with the keyword argument slice_of.	None
slice_of	Optional[tuple]	Optionally a tuple in the form (original_object_tensor, slice_info). When this argument is present, then the resulting ObjectArray will be a slice of the given original_object_tensor (which is expected as an ObjectArray instance). slice_info is either a slice instance, or a sequence of integers. The resulting ObjectArray might be a view of original_object_tensor (i.e. it might share its memory with original_object_tensor). This keyword argument cannot be used together with the argument size.	None

Source code in evotorch/tools/objectarray.py

def __init__(
    self,
    size: Optional[Size] = None,
    *,
    slice_of: Optional[tuple] = None,
):
    """
    `__init__(...)`: Instantiate a new ObjectArray.

    Args:
        size: Length of the ObjectArray. If this argument is present and
            is an integer `n`, then the resulting ObjectArray will be
            of length `n`, and will be filled with `None` values.
            This argument cannot be used together with the keyword
            argument `slice_of`.
        slice_of: Optionally a tuple in the form
            `(original_object_tensor, slice_info)`.
            When this argument is present, then the resulting ObjectArray
            will be a slice of the given `original_object_tensor` (which
            is expected as an ObjectArray instance). `slice_info` is
            either a `slice` instance, or a sequence of integers.
            The resulting ObjectArray might be a view of
            `original_object_tensor` (i.e. it might share its memory with
            `original_object_tensor`).
            This keyword argument cannot be used together with the
            argument `size`.
    """
    if size is not None and slice_of is not None:
        raise ValueError("Expected either `size` argument or `slice_of` argument, but got both.")
    elif size is None and slice_of is None:
        raise ValueError("Expected either `size` argument or `slice_of` argument, but got none.")
    elif size is not None:
        if not is_sequence(size):
            length = size
        elif isinstance(size, (np.ndarray, torch.Tensor)) and (size.ndim > 1):
            raise ValueError(f"Invalid size: {size}")
        else:
            [length] = size
        length = int(length)
        self._indices = torch.arange(length, dtype=torch.int64)
        self._objects = [None] * length
    elif slice_of is not None:
        source: ObjectArray

        source, slicing = slice_of

        if not isinstance(source, ObjectArray):
            raise TypeError(
                f"`slice_of`: The first element was expected as an ObjectArray."
                f" But it is of type {repr(type(source))}"
            )

        if isinstance(slicing, tuple) or is_integer(slicing):
            raise TypeError(f"Invalid slice: {slicing}")

        self._indices = source._indices[slicing]
        self._objects = source._objects

        if storage_ptr(self._indices) != storage_ptr(source._indices):
            self._objects = clone(self._objects)

    self._device = torch.device("cpu")
    self._read_only = False

clone(*, preserve_read_only=False, memo=None)

Get a deep copy of the ObjectArray.

Parameters:

Name	Type	Description	Default
preserve_read_only	bool	Whether or not to preserve the read-only attribute. Note that the default value is False, which means that the newly made clone will NOT be read-only even if the original ObjectArray is.	False
memo	Optional[dict]	Optionally a dictionary which maps from the ids of the already cloned objects to their clones. In most scenarios, when this method is called from outside, this can be left as None.	None

Source code in evotorch/tools/objectarray.py

def clone(self, *, preserve_read_only: bool = False, memo: Optional[dict] = None) -> Iterable:
    """
    Get a deep copy of the ObjectArray.

    Args:
        preserve_read_only: Whether or not to preserve the read-only
            attribute. Note that the default value is False, which
            means that the newly made clone will NOT be read-only
            even if the original ObjectArray is.
        memo: Optionally a dictionary which maps from the ids of the
            already cloned objects to their clones.
            In most scenarios, when this method is called from outside,
            this can be left as None.
    Returns:
        The clone of the original ObjectArray.
    """
    from .cloning import deep_clone

    if memo is None:
        memo = {}

    self_id = id(self)
    if self_id in memo:
        return memo[self_id]

    if not preserve_read_only:
        return self.numpy(memo=memo)
    else:
        result = ObjectArray(len(self))
        memo[self_id] = result

        for i, item in enumerate(self):
            result[i] = deep_clone(item, otherwise_deepcopy=True, memo=memo)

        return result

dim()

Get the number of dimensions handled by the ObjectArray. This is equivalent to getting the length of the size tuple.

Returns:

Type	Description
int	The number of dimensions, as an integer.

Source code in evotorch/tools/objectarray.py

def dim(self) -> int:
    """
    Get the number of dimensions handled by the ObjectArray.
    This is equivalent to getting the length of the size tuple.

    Returns:
        The number of dimensions, as an integer.
    """
    return self._indices.dim()

from_numpy(ndarray) staticmethod

Convert a numpy array of dtype object to an ObjectArray.

Parameters:

Name	Type	Description	Default
ndarray	ndarray	The numpy array that will be converted to ObjectArray.	required

Source code in evotorch/tools/objectarray.py

@staticmethod
def from_numpy(ndarray: np.ndarray) -> "ObjectArray":
    """
    Convert a numpy array of dtype `object` to an `ObjectArray`.

    Args:
        ndarray: The numpy array that will be converted to `ObjectArray`.
    Returns:
        The ObjectArray counterpart of the given numpy array.
    """
    if isinstance(ndarray, np.ndarray):
        if ndarray.dtype == np.dtype(object):
            n = len(ndarray)
            result = ObjectArray(n)
            for i, element in enumerate(ndarray):
                result[i] = element
            return result
        else:
            raise ValueError(
                f"The dtype of the given array was expected as `object`."
                f" However, the dtype was encountered as {ndarray.dtype}."
            )
    else:
        raise TypeError(f"Expected a `numpy.ndarray` instance, but received an object of type {type(ndarray)}.")

get_read_only_view()

Get a read-only view of this ObjectArray.

Source code in evotorch/tools/objectarray.py

def get_read_only_view(self) -> "ObjectArray":
    """
    Get a read-only view of this ObjectArray.
    """
    result = self[:]
    result._read_only = True
    return result

numel()

Number of elements stored by the ObjectArray.

Returns:

Type	Description
int	The number of elements, as an integer.

Source code in evotorch/tools/objectarray.py

def numel(self) -> int:
    """
    Number of elements stored by the ObjectArray.

    Returns:
        The number of elements, as an integer.
    """
    return self._indices.numel()

numpy(*, memo=None)

Convert this ObjectArray to a numpy array.

The resulting numpy array will have its dtype set as object. This new array itself and its contents will be mutable (those mutable objects being the copies of their immutable sources).

Returns:

Type	Description
ndarray	The numpy counterpart of this ObjectArray.

Source code in evotorch/tools/objectarray.py

def numpy(self, *, memo: Optional[dict] = None) -> np.ndarray:
    """
    Convert this ObjectArray to a numpy array.

    The resulting numpy array will have its dtype set as `object`.
    This new array itself and its contents will be mutable (those
    mutable objects being the copies of their immutable sources).

    Returns:
        The numpy counterpart of this ObjectArray.
    """
    from .immutable import mutable_copy

    if memo is None:
        memo = {}

    n = len(self)
    result = np.empty(n, dtype=object)

    memo[id(self)] = result

    for i, item in enumerate(self):
        result[i] = mutable_copy(item, memo=memo)

    return result

repeat(*sizes)

Repeat the contents of this ObjectArray.

For example, if we have an ObjectArray objs which stores ["hello", "world"], the following line:

objs.repeat(3)

will result in an ObjectArray which stores:

`["hello", "world", "hello", "world", "hello", "world"]`

Parameters:

Name	Type	Description	Default
sizes		Although this argument is named sizes to be compatible with PyTorch, what is expected here is a single positional argument, as a single integer, or as a single-element tuple. The given integer (which can be the argument itself, or the integer within the given single-element tuple), specifies how many times the stored sequence will be repeated.	()

Source code in evotorch/tools/objectarray.py

def repeat(self, *sizes) -> "ObjectArray":
    """
    Repeat the contents of this ObjectArray.

    For example, if we have an ObjectArray `objs` which stores
    `["hello", "world"]`, the following line:

        objs.repeat(3)

    will result in an ObjectArray which stores:

        `["hello", "world", "hello", "world", "hello", "world"]`

    Args:
        sizes: Although this argument is named `sizes` to be compatible
            with PyTorch, what is expected here is a single positional
            argument, as a single integer, or as a single-element
            tuple.
            The given integer (which can be the argument itself, or
            the integer within the given single-element tuple),
            specifies how many times the stored sequence will be
            repeated.
    Returns:
        A new ObjectArray which repeats the original one's values
    """

    if len(sizes) != 1:
        type_name = type(self).__name__
        raise ValueError(
            f"The `repeat(...)` method of {type_name} expects exactly one positional argument."
            f" This is because {type_name} supports only 1-dimensional storage."
            f" The received positional arguments are: {sizes}."
        )
    if isinstance(sizes, tuple):
        if len(sizes) == 1:
            sizes = sizes[0]
        else:
            type_name = type(self).__name__
            raise ValueError(
                f"The `repeat(...)` method of {type_name} can accept a size tuple with only one element."
                f" This is because {type_name} supports only 1-dimensional storage."
                f" The received size tuple is: {sizes}."
            )
    num_repetitions = int(sizes[0])
    self_length = len(self)
    result = ObjectArray(num_repetitions * self_length)

    source_index = 0
    for result_index in range(len(result)):
        result[result_index] = self[source_index]
        source_index = (source_index + 1) % self_length

    return result

set_item(i, x, *, memo=None)

Set the i-th item of the ObjectArray as x.

Parameters:

Name	Type	Description	Default
i	Any	An index or a slice.	required
x	Any	The object that will be put into the ObjectArray.	required
memo	Optional[dict]	Optionally a dictionary which maps from the ids of the already placed objects to their clones within ObjectArray. In most scenarios, when this method is called from outside, this can be left as None.	None

Source code in evotorch/tools/objectarray.py

def set_item(self, i: Any, x: Any, *, memo: Optional[dict] = None):
    """
    Set the i-th item of the ObjectArray as x.

    Args:
        i: An index or a slice.
        x: The object that will be put into the ObjectArray.
        memo: Optionally a dictionary which maps from the ids of the
            already placed objects to their clones within ObjectArray.
            In most scenarios, when this method is called from outside,
            this can be left as None.
    """
    from .immutable import as_immutable

    if memo is None:
        memo = {}

    memo[id(self)] = self

    if self._read_only:
        raise ValueError("This ObjectArray is read-only, therefore, modification is not allowed.")

    if is_integer(i):
        index = int(self._indices[i])
        self._objects[index] = as_immutable(x, memo=memo)
    else:
        indices = self._indices[i]
        if not isinstance(x, Iterable):
            raise TypeError(f"Expected an iterable, but got {repr(x)}")

        if indices.ndim != 1:
            raise ValueError(
                "Received indices that would change the dimensionality of the ObjectArray."
                " However, an ObjectArray can only be 1-dimensional."
            )

        slice_refers_to_whole_array = (len(indices) == len(self._indices)) and torch.all(indices == self._indices)
        if slice_refers_to_whole_array:
            memo[id(x)] = self

        if not hasattr(x, "__len__"):
            x = list(x)

        if len(x) != len(indices):
            raise TypeError(
                f"The slicing operation refers to {len(indices)} elements."
                f" However, the given objects sequence has {len(x)} elements."
            )

        for q, obj in enumerate(x):
            index = int(indices[q])
            self._objects[index] = as_immutable(obj, memo=memo)

size()

Get the size of the ObjectArray, as a PyTorch Size tuple.

Returns:

Type	Description
Size	The size (i.e. the shape) of the ObjectArray.

Source code in evotorch/tools/objectarray.py

def size(self) -> Size:
    """
    Get the size of the ObjectArray, as a PyTorch Size tuple.

    Returns:
        The size (i.e. the shape) of the ObjectArray.
    """
    return self._indices.size()

ReadOnlyTensor

Bases: Tensor

A special type of tensor which is read-only.

This is a subclass of torch.Tensor which explicitly disallows operations that would cause in-place modifications.

Since ReadOnlyTensor if a subclass of torch.Tensor, most non-destructive PyTorch operations are on this tensor are supported.

Cloning a ReadOnlyTensor using the clone() method or Python's deepcopy(...) function results in a regular PyTorch tensor.

Reshaping or slicing operations might return a ReadOnlyTensor if the result ends up being a view of the original ReadOnlyTensor; otherwise, the returned tensor is a regular torch.Tensor.

Source code in evotorch/tools/readonlytensor.py

class ReadOnlyTensor(torch.Tensor):
    """
    A special type of tensor which is read-only.

    This is a subclass of `torch.Tensor` which explicitly disallows
    operations that would cause in-place modifications.

    Since ReadOnlyTensor if a subclass of `torch.Tensor`, most
    non-destructive PyTorch operations are on this tensor are supported.

    Cloning a ReadOnlyTensor using the `clone()` method or Python's
    `deepcopy(...)` function results in a regular PyTorch tensor.

    Reshaping or slicing operations might return a ReadOnlyTensor if the
    result ends up being a view of the original ReadOnlyTensor; otherwise,
    the returned tensor is a regular `torch.Tensor`.
    """

    def __getattribute__(self, attribute_name: str) -> Any:
        if (
            isinstance(attribute_name, str)
            and attribute_name.endswith("_")
            and (not ((attribute_name.startswith("__")) and (attribute_name.endswith("__"))))
        ):
            raise AttributeError(
                f"A ReadOnlyTensor explicitly disables all members whose names end with '_'."
                f" Cannot access member {repr(attribute_name)}."
            )
        else:
            return super().__getattribute__(attribute_name)

    def __cannot_modify(self, *ignore, **ignore_too):
        raise TypeError("The contents of a ReadOnlyTensor cannot be modified")

    __setitem__ = __cannot_modify
    __iadd__ = __cannot_modify
    __iand__ = __cannot_modify
    __idiv__ = __cannot_modify
    __ifloordiv__ = __cannot_modify
    __ilshift__ = __cannot_modify
    __imatmul__ = __cannot_modify
    __imod__ = __cannot_modify
    __imul__ = __cannot_modify
    __ior__ = __cannot_modify
    __ipow__ = __cannot_modify
    __irshift__ = __cannot_modify
    __isub__ = __cannot_modify
    __itruediv__ = __cannot_modify
    __ixor__ = __cannot_modify

    if _torch_older_than_1_12:
        # Define __str__ and __repr__ for when using PyTorch 1.11 or older.
        # With PyTorch 1.12, overriding __str__ and __repr__ are not necessary.
        def __to_string(self) -> str:
            s = super().__repr__()
            if "\n" not in s:
                return f"ReadOnlyTensor({super().__repr__()})"
            else:
                indenter = " " * 4
                s = (indenter + s.replace("\n", "\n" + indenter)).rstrip()
                return f"ReadOnlyTensor(\n{s}\n)"

        __str__ = __to_string
        __repr__ = __to_string

    def clone(self, *, preserve_read_only: bool = False) -> torch.Tensor:
        result = super().clone()
        if not preserve_read_only:
            result = result.as_subclass(torch.Tensor)
        return result

    def __mutable_if_independent(self, other: torch.Tensor) -> torch.Tensor:
        from .misc import storage_ptr

        self_ptr = storage_ptr(self)
        other_ptr = storage_ptr(other)
        if self_ptr != other_ptr:
            other = other.as_subclass(torch.Tensor)
        return other

    def __getitem__(self, index_or_slice) -> torch.Tensor:
        result = super().__getitem__(index_or_slice)
        return self.__mutable_if_independent(result)

    def reshape(self, *args, **kwargs) -> torch.Tensor:
        result = super().reshape(*args, **kwargs)
        return self.__mutable_if_independent(result)

    def numpy(self) -> np.ndarray:
        arr: np.ndarray = torch.Tensor.numpy(self)
        arr.flags["WRITEABLE"] = False
        return arr

    def __array__(self, *args, **kwargs) -> np.ndarray:
        arr: np.ndarray = super().__array__(*args, **kwargs)
        arr.flags["WRITEABLE"] = False
        return arr

    def __copy__(self):
        return self.clone(preserve_read_only=True)

    def __deepcopy__(self, memo):
        return self.clone(preserve_read_only=True)

    @classmethod
    def __torch_function__(cls, func: Callable, types: Iterable, args: tuple = (), kwargs: Optional[Mapping] = None):
        if (kwargs is not None) and ("out" in kwargs):
            if isinstance(kwargs["out"], ReadOnlyTensor):
                raise TypeError(
                    f"The `out` keyword argument passed to {func} is a ReadOnlyTensor."
                    f" A ReadOnlyTensor explicitly fails when referenced via the `out` keyword argument of any torch"
                    f" function."
                    f" This restriction is for making sure that the torch operations which could normally do in-place"
                    f" modifications do not operate on ReadOnlyTensor instances."
                )
        return super().__torch_function__(func, types, args, kwargs)

TensorFrame

Bases: RecursivePrintable

A structure which allows one to manipulate tensors in a tabular manner. The interface of this structure is inspired by the DataFrame class of the pandas library.

Motivation. It is a common scenario to have to work with arrays/tensors that are associated with each other (e.g. when working on a knapsack problem, we could have arrays A and B, where A[i] represents the value of the i-th item and B[i] represents the weight of the i-th item). A practical approach for such cases is to organize those arrays and operate on them in a tabular manner. pandas is a popular library for doing such tabular operations.

In the context of evolutionary computation, efficiency via vectorization and/or parallelization becomes an important concern. For example, if we have a fitness function in which a solution vector is evaluated with the help of tabular operations, we would like to be able to obtain a batched version of that function, so that not just a solution, but an entire population can be evaluated efficiently. Another example is when developing custom evolutionary algorithms, where solutions, fitnesses, and algorithm-specific (or problem-specific) metadata can be organized in a tabular manner, and operating on such tabular data has to be vectorized and/or parallelized for increased efficiency.

TensorFrame is introduced to address these concerns. In more details, with evolutionary computation in mind, it has these features/behaviors:

(i) The columns of a TensorFrame are expressed via PyTorch tensors (or via evotorch.tools.ReadOnlyTensor instances), allowing one to place the tabular data on devices such as cuda.

(ii) A TensorFrame can be placed into a function that is transformed via torch.vmap or evotorch.decorators.expects_ndim or evotorch.decorators.rowwise, therefore it can work on batched data.

(iii) Upon being pickled or cloned, a TensorFrame applies the clone method on all its columns and ensures that the cloned tensors have minimally sized storages (even when their originals might have shared their storages with larger tensors). Therefore, one can send a TensorFrame to a remote worker (e.g. using ray library), without having to worry about oversized shared tensor storages.

(iv) A TensorFrame can be placed as an item into an evotorch.tools.ObjectArray container. Therefore, it can serve as a value in a solution of a problem whose dtype is object.

Basic usage. A tensorframe can be instantiated like this:

from evotorch.tools import TensorFrame
import torch

my_tensorframe = TensorFrame(
    {
        "COLUMN1": torch.FloatTensor([1, 2, 3, 4]),
        "COLUMN2": torch.FloatTensor([10, 20, 30, 40]),
        "COLUMN3": torch.FloatTensor([-10, -20, -30, -40]),
    }
)

which represents the following tabular data:

  float32    float32     <- dtype of the column
    cpu        cpu       <- device of the column

  COLUMN1    COLUMN2    COLUMN3
 =========  =========  =========
    1.0        10.0      -10.0
    2.0        20.0      -20.0
    3.0        30.0      -30.0
    4.0        40.0      -40.0

Rows can be picked and re-organized like this:

my_tensorframe = my_tensorframe.pick[[0, 3, 2]]

which causes my_tensorframe to now store:

  float32    float32    float32
    cpu        cpu        cpu

  COLUMN1    COLUMN2    COLUMN3
 =========  =========  =========
    1.0        10.0      -10.0
    4.0        40.0      -40.0
    3.0        30.0      -30.0

A tensor of a column can be received like this:

print(my_tensorframe["COLUMN1"])
# Note: alternatively: print(my_tensorframe.COLUMN1)

# Prints: torch.tensor([1.0, 4.0, 3.0], dtype=torch.float32)

Multiple columns can be received like this:

print(my_tensorframe[["COLUMN1", "COLUMN2"]])

# Prints:
#
#  float32    float32
#    cpu        cpu
#
#  COLUMN1    COLUMN2
# =========  =========
#    1.0        10.0
#    4.0        40.0
#    3.0        30.0

The values of a column can be changed like this:

my_tensorframe.pick[1:, "COLUMN1"] = torch.FloatTensor([7.0, 9.0])

which causes my_tensorframe to become:

  float32    float32    float32
    cpu        cpu        cpu

  COLUMN1    COLUMN2    COLUMN3
 =========  =========  =========
    1.0        10.0      -10.0
    7.0        40.0      -40.0
    9.0        30.0      -30.0

Multiple columns can be changed like this:

my_tensorframe.pick[1:, ["COLUMN1", "COLUMN2"]] = TensorFrame(
    {
        "COLUMN1": torch.FloatTensor([11.0, 12.0]),
        "COLUMN2": torch.FloatTensor([44.0, 55.0]),
    }
)

# Note: alternatively, the right-hand side can be given as a dictionary:
# my_tensorframe.pick[1:, ["COLUMN1", "COLUMN2"]] = {
#     "COLUMN1": torch.FloatTensor([11.0, 12.0]),
#     "COLUMN2": torch.FloatTensor([44.0, 55.0]),
# }

which causes my_tensorframe to become:

  float32    float32    float32
    cpu        cpu        cpu

  COLUMN1    COLUMN2    COLUMN3
 =========  =========  =========
    1.0        10.0      -10.0
   11.0        44.0      -40.0
   12.0        55.0      -30.0

Further notes.

• A tensor under a TensorFrame column can have more than one dimension. Across different columns, the size of the leftmost dimensions must match.
• Unlike a pandas.DataFrame, a TensorFrame does not have a special index column.

Source code in evotorch/tools/tensorframe.py

class TensorFrame(RecursivePrintable):
    """
    A structure which allows one to manipulate tensors in a tabular manner.
    The interface of this structure is inspired by the `DataFrame` class
    of the `pandas` library.

    **Motivation.**
    It is a common scenario to have to work with arrays/tensors that are
    associated with each other (e.g. when working on a knapsack problem, we
    could have arrays `A` and `B`, where `A[i]` represents the value of the
    i-th item and `B[i]` represents the weight of the i-th item).
    A practical approach for such cases is to organize those arrays and
    operate on them in a tabular manner. `pandas` is a popular library
    for doing such tabular operations.

    In the context of evolutionary computation, efficiency via vectorization
    and/or parallelization becomes an important concern. For example, if we
    have a fitness function in which a solution vector is evaluated with the
    help of tabular operations, we would like to be able to obtain a batched
    version of that function, so that not just a solution, but an entire
    population can be evaluated efficiently. Another example is when
    developing custom evolutionary algorithms, where solutions, fitnesses,
    and algorithm-specific (or problem-specific) metadata can be organized
    in a tabular manner, and operating on such tabular data has to be
    vectorized and/or parallelized for increased efficiency.

    `TensorFrame` is introduced to address these concerns. In more details,
    with evolutionary computation in mind, it has these features/behaviors:

    (i) The columns of a `TensorFrame` are expressed via PyTorch tensors
    (or via `evotorch.tools.ReadOnlyTensor` instances), allowing one
    to place the tabular data on devices such as cuda.

    (ii) A `TensorFrame` can be placed into a function that is transformed
    via `torch.vmap` or `evotorch.decorators.expects_ndim` or
    `evotorch.decorators.rowwise`, therefore it can work on batched data.

    (iii) Upon being pickled or cloned, a `TensorFrame` applies the `clone`
    method on all its columns and ensures that the cloned tensors have
    minimally sized storages (even when their originals might have shared
    their storages with larger tensors). Therefore, one can send a
    `TensorFrame` to a remote worker (e.g. using `ray` library), without
    having to worry about oversized shared tensor storages.

    (iv) A `TensorFrame` can be placed as an item into an
    `evotorch.tools.ObjectArray` container. Therefore, it can serve as a
    value in a solution of a problem whose `dtype` is `object`.

    **Basic usage.**
    A tensorframe can be instantiated like this:

    ```python
    from evotorch.tools import TensorFrame
    import torch

    my_tensorframe = TensorFrame(
        {
            "COLUMN1": torch.FloatTensor([1, 2, 3, 4]),
            "COLUMN2": torch.FloatTensor([10, 20, 30, 40]),
            "COLUMN3": torch.FloatTensor([-10, -20, -30, -40]),
        }
    )
    ```

    which represents the following tabular data:

    ```text
      float32    float32     <- dtype of the column
        cpu        cpu       <- device of the column

      COLUMN1    COLUMN2    COLUMN3
     =========  =========  =========
        1.0        10.0      -10.0
        2.0        20.0      -20.0
        3.0        30.0      -30.0
        4.0        40.0      -40.0
    ```

    Rows can be picked and re-organized like this:

    ```python
    my_tensorframe = my_tensorframe.pick[[0, 3, 2]]
    ```

    which causes `my_tensorframe` to now store:

    ```text
      float32    float32    float32
        cpu        cpu        cpu

      COLUMN1    COLUMN2    COLUMN3
     =========  =========  =========
        1.0        10.0      -10.0
        4.0        40.0      -40.0
        3.0        30.0      -30.0
    ```

    A tensor of a column can be received like this:

    ```python
    print(my_tensorframe["COLUMN1"])
    # Note: alternatively: print(my_tensorframe.COLUMN1)

    # Prints: torch.tensor([1.0, 4.0, 3.0], dtype=torch.float32)
    ```

    Multiple columns can be received like this:

    ```python
    print(my_tensorframe[["COLUMN1", "COLUMN2"]])

    # Prints:
    #
    #  float32    float32
    #    cpu        cpu
    #
    #  COLUMN1    COLUMN2
    # =========  =========
    #    1.0        10.0
    #    4.0        40.0
    #    3.0        30.0
    ```

    The values of a column can be changed like this:

    ```python
    my_tensorframe.pick[1:, "COLUMN1"] = torch.FloatTensor([7.0, 9.0])
    ```

    which causes `my_tensorframe` to become:

    ```text
      float32    float32    float32
        cpu        cpu        cpu

      COLUMN1    COLUMN2    COLUMN3
     =========  =========  =========
        1.0        10.0      -10.0
        7.0        40.0      -40.0
        9.0        30.0      -30.0
    ```

    Multiple columns can be changed like this:

    ```python
    my_tensorframe.pick[1:, ["COLUMN1", "COLUMN2"]] = TensorFrame(
        {
            "COLUMN1": torch.FloatTensor([11.0, 12.0]),
            "COLUMN2": torch.FloatTensor([44.0, 55.0]),
        }
    )

    # Note: alternatively, the right-hand side can be given as a dictionary:
    # my_tensorframe.pick[1:, ["COLUMN1", "COLUMN2"]] = {
    #     "COLUMN1": torch.FloatTensor([11.0, 12.0]),
    #     "COLUMN2": torch.FloatTensor([44.0, 55.0]),
    # }
    ```

    which causes `my_tensorframe` to become:

    ```text
      float32    float32    float32
        cpu        cpu        cpu

      COLUMN1    COLUMN2    COLUMN3
     =========  =========  =========
        1.0        10.0      -10.0
       11.0        44.0      -40.0
       12.0        55.0      -30.0
    ```

    **Further notes.**

    - A tensor under a TensorFrame column can have more than one dimension.
        Across different columns, the size of the leftmost dimensions must
        match.
    - Unlike a `pandas.DataFrame`, a TensorFrame does not have a special index
        column.
    """

    def __init__(
        self,
        data: Optional[Union[Mapping, "TensorFrame", _PandasDataFrame]] = None,
        *,
        read_only: bool = False,
        device: Optional[Union[torch.device, str]] = None,
    ):
        """
        `__init__(...)`: Initialize the TensorFrame

        Args:
            data: Optionally a TensorFrame, or a dictionary (where the keys
                are column names and the values are column values), or a
                `pandas.DataFrame` object. If provided, the tabular data
                given here are used for initializing the new TensorFrame.
            read_only: Whether or not the newly made TensorFrame will be
                read-only. A read-only TensorFrame's columns will be
                ReadOnlyTensors, and its columns and values will not change.
            device: If left as None, each column can be on a different device.
                If given as a string or a `torch.device`, each column will
                be forcefully moved to this given device.
        """
        from .readonlytensor import as_read_only_tensor

        self.__is_read_only = False
        self.__device = None if device is None else torch.device(device)

        if read_only:

            def prepare_value(x):
                return as_read_only_tensor(x)

        else:

            def prepare_value(x):
                return x

        self.__data = OrderedDict()
        if data is None:
            pass  # no data is given, nothing to do
        elif _is_pandas_dataframe(data):
            for k in data.columns:
                v = data[k]
                self.__setitem__(k, prepare_value(v))
        elif isinstance(data, Mapping):
            for k, v in data.items():
                self.__setitem__(k, prepare_value(v))
        elif isinstance(data, TensorFrame):
            for k, v in data.__data.items():
                self.__setitem__(k, prepare_value(v))
        else:
            raise TypeError(
                "When constructing a new TensorFrame, the argument `data` was expected as a dictionary-like object"
                " (an instance of Mapping), or as a TensorFrame."
                f" However, the encountered object is of type {type(data)}."
            )

        self.__is_read_only = read_only
        self._initialized = True

    def __get_first_tensor(self) -> Optional[torch.Tensor]:
        return None if len(self.__data) == 0 else next(iter(self.__data.values()))

    def __get_default_device_kwargs(self) -> dict:
        return {} if self.__device is None else {"device": self.__device}

    def __get_first_tensor_device_kwargs(self) -> dict:
        first_tensor = self.__get_first_tensor()
        return {} if first_tensor is None else {"device": first_tensor.device}

    def as_tensor(
        self,
        x: Any,
        *,
        to_work_with: Optional[Union[str, np.str_, torch.Tensor]] = None,
        broadcast_if_scalar: bool = False,
    ) -> torch.Tensor:
        """
        Convert the given object `x` to a PyTorch tensor.

        Args:
            x: The object to be converted to a PyTorch tensor.
            to_work_with: Optionally a string, referring to an existing column
                within this TensorFrame, or a PyTorch tensor. The object `x`
                will be converted to a PyTorch tensor on the same device with
                this given column/tensor. If `to_work_with` is left as None,
                `x` will be converted to a PyTorch tensor on the same device
                with this TensorFrame's first column.
            broadcast_if_scalar: If this argument is given as True and if `x`
                is a scalar, its tensor-counterpart will be broadcast to a
                vector of length `n`, where `n` is the number of rows of this
                TensorFrame.
        Returns:
            The tensor counterpart of `x`.
        """
        from .readonlytensor import as_read_only_tensor

        if to_work_with is not None:
            if isinstance(to_work_with, torch.Tensor):
                pass  # nothing to do
            elif isinstance(to_work_with, (str, np.str_)):
                to_work_with = self.__data[str(to_work_with)]
            else:
                raise TypeError(
                    "The argument `to_work_with` was expected as a PyTorch tensor or as a string"
                    " (that refers to one of the columns of this TensorFrame)."
                    f" However, it was received as an instance of {type(to_work_with)}."
                )
            result = torch.as_tensor(x, device=to_work_with.device)
        else:
            if self.__is_read_only:
                convert = as_read_only_tensor
            else:
                convert = torch.as_tensor

            if isinstance(x, torch.Tensor):
                result = convert(x, **(self.__get_default_device_kwargs()))
            else:
                result = convert(x, **(self.__get_first_tensor_device_kwargs()))

        if broadcast_if_scalar and (result.ndim == 0):
            first_tensor = self.__get_first_tensor()
            if first_tensor is None:
                raise ValueError("The first column cannot be given as a scalar.")
            result = result * torch.ones(first_tensor.shape[0], dtype=result.dtype, device=result.device)

        return result

    def __setitem__(self, column_name: Union[str, np.str_], values: Any):
        if self.__is_read_only:
            raise TypeError("Cannot modify a read-only TensorFrame")

        column_name = str(column_name)
        values = self.as_tensor(values, broadcast_if_scalar=True)
        self.__data[column_name] = values

    def __getitem__(
        self,
        column_name_or_mask: Union[str, np.str_, Sequence[Union[str, np.str_]]],
    ) -> torch.Tensor:
        if (isinstance(column_name_or_mask, np.ndarray) and (column_name_or_mask.dtype == np.bool_)) or (
            isinstance(column_name_or_mask, torch.Tensor) and (column_name_or_mask.dtype == torch.bool)
        ):
            return self.pick[column_name_or_mask]
        elif isinstance(column_name_or_mask, (str, np.str_)):
            return self.__data[str(column_name_or_mask)]
        elif isinstance(column_name_or_mask, Sequence):
            result = TensorFrame()
            for col in column_name_or_mask:
                if not isinstance(col, (str, np.str_)):
                    raise TypeError(f"The sequence of column names has an item of this unrecognized type: {type(col)}")
                result[col] = self[col]
            if self.is_read_only:
                result = result.get_read_only_view()
            return result
        else:
            raise TypeError(
                "Expected a column name (as a string) or a sequence of column names, but encountered an instance of"
                f" {type(column_name_or_mask)}."
            )

    def __setattr__(self, attr_name: str, value: Any) -> torch.Tensor:
        if attr_name == "__dict__":
            object.__setattr__(self, attr_name, value)
        elif ("_initialized" in self.__dict__) and (self.__dict__["_initialized"]):
            if attr_name in self.__dict__:
                self.__dict__[attr_name] = value
            elif attr_name in self.__data:
                raise ValueError(
                    f"Please do not use the dot notation to change the column {repr(attr_name)}."
                    f" Hint: you could use: tensorframe[{repr(attr_name)}] = ..."
                )
            else:
                raise ValueError(
                    f"Unknown attribute: {repr(attr_name)}."
                    f" Hint: to add a new column {attr_name}, you could use: tensorframe[{repr(attr_name)}] = ..."
                )
        else:
            self.__dict__[attr_name] = value

    def __getattr__(self, column_name: str) -> torch.Tensor:
        if column_name in self.__data:
            return self.__data[column_name]
        else:
            raise AttributeError(column_name)

    def without_enforced_device(self) -> "TensorFrame":
        """
        Make a shallow copy of this TensorFrame without any enforced device.

        In the newly made shallow copy, columns will be able to exist on
        different devices.

        Returns:
            A shallow copy of this TensorFrame without any enforced device.
        """
        return TensorFrame(self.__data, read_only=self.__is_read_only, device=None)

    def with_enforced_device(self, device: Union[str, torch.device]) -> "TensorFrame":
        """
        Make a shallow copy of this TensorFrame with an enforced device.

        In the newly made shallow copy, columns will be forcefully moved onto
        the specified device.

        Args:
            device: The device to which the new TensorFrame's columns will move.
        Returns:
            A shallow copy of this TensorFrame with an enforced device.
        """
        if device is None:
            raise TypeError("When using the method `with_enforced_device`, the argument `device` cannot be None")

        return TensorFrame(self.__data, read_only=self.__is_read_only, device=device)

    def to(self, device: Union[str, torch.device]) -> "TensorFrame":
        if self.__device is None:
            enforce_device_kwargs = {}
        else:
            enforce_device_kwargs = {"device": device}

        return TensorFrame(
            {k: v.to(device) for k, v in self.__data.items()},
            read_only=self.__is_read_only,
            **enforce_device_kwargs,
        )

    def cpu(self) -> "TensorFrame":
        """
        Get a shallow copy of this TensorFrame with all columns moved to cpu.
        """
        return self.to("cpu")

    def cuda(self) -> "TensorFrame":
        """
        Get a shallow copy of this TensorFrame with all columns moved to cuda.
        """
        return self.to("cuda")

    @property
    def device(self) -> Optional[Union[torch.device, set]]:
        """
        Get the device(s) of this TensorFrame.

        If different columns exist on different devices, a set of devices
        will be returned. If all the columns exist on the same device, then
        that device will be returned.
        """
        devices = set(v.device for v in self.__data.values())
        n = len(devices)
        if n == 0:
            return None
        elif n == 1:
            [only_device] = devices
            return only_device
        else:
            return devices

    @property
    def columns(self) -> list:
        """
        Columns as a list of strings
        """
        return list(self.__data.keys())

    @property
    def pick(self) -> "Picker":
        """
        Get or set values of this TensorFrame.

        **Usage.**

        ```python
        # Get values
        tensorframe.pick[slice_or_indexlist, column_name]

        # Set values
        tensorframe.pick[slice_or_indexlist, column_name] = (
            ...  # A tensor or a list/tensor of indices or a TensorFrame
        )
        ```
        """
        return Picker(self)

    def __non_tabular_repr(self, *, max_depth: int = DEFAULT_MAX_DEPTH_FOR_PRINTING) -> str:
        f = StringIO()
        if max_depth >= DEFAULT_MAX_DEPTH_FOR_PRINTING:
            indent = " " * 4
            dbl_indent = " " * 8
            colon_for_dict = ":"
            comma_for_dict = ","

            def prn(*items):
                print(*items, sep="", file=f)

        else:
            indent = ""
            dbl_indent = ""
            colon_for_dict = ": "
            comma_for_dict = ", "

            def prn(*items):
                print(*items, sep="", end="", file=f)

        try:
            prn(type(self).__name__, "(")
            prn(indent, "{")
            last_i = len(self.__data) - 1
            for i, (k, v) in enumerate(self.__data.items()):
                if i > 0:
                    prn()
                prn(dbl_indent, repr(k), colon_for_dict)
                comma = "" if i == last_i else comma_for_dict
                if isinstance(v, RecursivePrintable):
                    v_as_str = v.to_string(max_depth=(max_depth - 1))
                elif isinstance(v, torch.Tensor):
                    v_as_str = str(v).replace("\n", "\n" + dbl_indent).strip()
                else:
                    v_as_str = str(v)
                prn(dbl_indent, v_as_str, comma)
            prn(indent, "}")

            if self.__is_read_only and (self.__device is not None):
                prn(indent, comma_for_dict, "read_only=True", comma_for_dict)
                prn(indent, "device=", repr(self.__device))
            elif self.__is_read_only:
                prn(indent, comma_for_dict, "read_only=True")
            elif self.__device is not None:
                prn(indent, comma_for_dict, "device=", repr(self.__device))

            prn(")")
            f.seek(0)
            result = f.read()
        finally:
            f.close()

        return result

    def __tabular_repr(self) -> str:
        from itertools import chain

        half_rows_to_show = 15
        num_rows_to_show = half_rows_to_show * 2

        n = len(self)
        if n <= num_rows_to_show:
            row_indices_to_show = range(n)
            three_dots_after_row = None
        else:
            row_indices_to_show = chain(range(half_rows_to_show), range(n - half_rows_to_show, n))
            three_dots_after_row = half_rows_to_show - 1

        col_lengths = {}

        def update_col_length(col_index: int, new_col_length: int):
            if col_index in col_lengths:
                current_col_length = col_lengths[col_index]
                if new_col_length > current_col_length:
                    col_lengths[col_index] = new_col_length
            else:
                col_lengths[col_index] = new_col_length

        lines = []

        def fill_line(*items):
            current_line = []
            for col_index, item in enumerate(items):
                if isinstance(item, torch.Tensor) and (item.ndim == 0):
                    # item_as_str = " " + str(item.cpu().item()) + " "
                    item_as_str = " " + str(np.asarray(item.cpu()).reshape(1)[0]) + " "
                else:
                    item_as_str = " " + str(item) + " "
                update_col_length(col_index, len(item_as_str))
                current_line.append(item_as_str)
            lines.append(current_line)

        fill_line("", *(_without_torch_dot(self[column].dtype) for column in self.columns))
        fill_line("", *((self[column].device) for column in self.columns))
        fill_line()
        fill_line("", *(column for column in self.columns))
        fill_line("", *(("=" * col_lengths[j + 1]) for j in range(len(self.columns))))
        for row_index in row_indices_to_show:
            fill_line(row_index, *(self[column][row_index] for column in self.columns))
            if (three_dots_after_row is not None) and (row_index == three_dots_after_row):
                fill_line("...", *("..." for column in self.columns))

        f = StringIO()
        try:
            for line in lines:
                for col_index, col in enumerate(line):
                    print(col.center(col_lengths[col_index]), end="", file=f)
                print(file=f)
            needs_another_line = False
            if self.__device is not None:
                print(" device=", repr(self.__device), sep="", end="", file=f)
                needs_another_line = True
            if self.__is_read_only:
                print(" read_only=True", end="", file=f)
                needs_another_line = True
            if needs_another_line:
                print(file=f)
            f.seek(0)
            result = f.read()
        finally:
            f.close()
        return result

    def __all_columns_are_one_dimensional(self) -> bool:
        if len(self.columns) == 0:
            return False

        for column in self.columns:
            if self[column].ndim != 1:
                return False

        return True

    def to_string(self, *, max_depth: int = DEFAULT_MAX_DEPTH_FOR_PRINTING) -> str:
        """
        Return the string representation of this TensorFrame
        """
        if len(self.columns) == 0:
            return type(self).__name__ + "()"

        if (max_depth >= DEFAULT_MAX_DEPTH_FOR_PRINTING) and self.__all_columns_are_one_dimensional():
            return self.__tabular_repr()
        else:
            return self.__non_tabular_repr(max_depth=max_depth)

    def __tabular_repr_html(self) -> str:
        from itertools import chain

        half_rows_to_show = 15
        num_rows_to_show = half_rows_to_show * 2

        n = len(self)
        if n <= num_rows_to_show:
            row_indices_to_show = range(n)
            three_dots_after_row = None
        else:
            row_indices_to_show = chain(range(half_rows_to_show), range(n - half_rows_to_show, n))
            three_dots_after_row = half_rows_to_show - 1

        f = StringIO()

        def safe_html(s: Any) -> str:
            s = str(s)

            replacements = (
                ("&", "&amp;"),
                ("<", "&lt;"),
                (">", "&gt;"),
                ("'", "&apos;"),
                ('"', "&quot;"),
            )

            for a, b in replacements:
                s = s.replace(a, b)

            return s

        def prn(*msg):
            print(*msg, sep="", end="", file=f)

        try:
            prn("<table>")

            prn("<tr>")
            prn("<th></th>")
            for column in self.columns:
                t = self[column]
                prn("<th>")
                prn(safe_html(_without_torch_dot(t.dtype)), "<br/>")
                prn(safe_html(t.device), "<br/><br/>")
                prn(safe_html(column))
                prn("</th>")
            prn("</tr>")

            for i in row_indices_to_show:
                prn("<tr>")
                prn("<td><strong>", i, "</strong></td>")
                for column in self.columns:
                    prn("<td>", safe_html(np.asarray(self[column][i].cpu()).reshape(1)[0]), "</td>")
                prn("</tr>")
                if (three_dots_after_row is not None) and (i == three_dots_after_row):
                    prn("<tr>")
                    prn("<td>...</td>")
                    for column in self.columns:
                        prn("<td>...</td>")
                    prn("</tr>")

            prn("<tr>")
            prn("<th></th>")
            for column in self.columns:
                t = self[column]
                prn("<th>")
                prn(safe_html(column))
                prn("</th>")
            prn("</tr>")

            prn("</table>")
            f.seek(0)
            result = f.read()
        finally:
            f.close()
        return result

    def _repr_html_(self) -> str:
        if len(self.columns) == 0:
            return type(self).__name__ + "()"

        if self.__all_columns_are_one_dimensional():
            return self.__tabular_repr_html()
        else:
            return "<pre>" + self.__non_tabular_repr().replace("<", "[").replace(">", "]") + "</pre>"

    @property
    def is_read_only(self) -> bool:
        """
        True if this TensorFrame is read-only; False otherwise.
        """
        return self.__is_read_only

    def get_read_only_view(self) -> "TensorFrame":
        """
        Get a read-only counterpart of this TensorFrame.
        """
        return TensorFrame(self.__data, read_only=True, device=self.__device)

    def clone(self, *, preserve_read_only: bool = False, memo: Optional[dict] = None) -> "TensorFrame":
        """
        Get a clone of this TensorFrame.

        Args:
            preserve_read_only: If True, the newly made clone will be read-only
                only if this TensorFrame is also read-only.
        Returns:
            The clone of this TensorFrame.
        """
        if memo is None:
            memo = {}

        self_id = id(self)
        if self_id in memo:
            return memo[self_id]

        new_read_only = self.__is_read_only if preserve_read_only else False

        newdata = OrderedDict()
        for k in self.columns:
            newdata[k] = self.__data[k].clone()

        return TensorFrame(newdata, read_only=new_read_only, device=self.__device)

    def __copy__(self) -> "TensorFrame":
        return self.clone(preserve_read_only=True)

    def __deepcopy__(self, memo: Optional[dict]) -> "TensorFrame":
        return self.clone(preserve_read_only=True, memo=memo)

    def __getstate__(self) -> dict:
        cloned_frame = self.clone(preserve_read_only=True)
        return cloned_frame.__dict__

    def __setstate__(self, d: dict):
        object.__setattr__(self, "__dict__", {})
        self.__dict__.update(d)

    def __len__(self) -> int:
        for v in self.__data.values():
            return v.shape[0]
        return 0

    def argsort(
        self,
        by: Union[str, np.str_],
        *,
        indices: Optional[Union[str, np.str_]] = None,
        ranks: Optional[Union[str, np.str_]] = None,
        descending: bool = False,
        join: bool = False,
    ) -> Union[torch.Tensor, "TensorFrame"]:
        """
        Return row indices (also optionally ranks) for sorting the TensorFrame.

        For example, let us assume that we have a TensorFrame named `table`.
        We can sort this `table` like this:

        ```python
        indices_for_sorting = table.argsort(by="A")  # sort by the column A
        sorted_table = table.pick[indices_for_sorting]
        ```

        Args:
            by: The name of the column according to which the TensorFrame
                will be sorted.
            indices: If given as a string `s`, the result will be a new
                TensorFrame, and the sorting indices will be listed under
                a column named `s`.
            ranks: If given as a string `z`, the result will be a new
                TensorFrame, and the ranks will be listed under a column
                named `z`.
            descending: If True, the sorting will be in descending order.
            join: Can be used only if column names are given via `indices`
                and/or `ranks`. If `join` is True, then the resulting
                TensorFrame will have the sorted TensorFrame's columns
                as well.
        Returns:
            A tensor of indices, or a TensorFrame.
        """
        by = str(by)
        target_column = self[by]
        indices_for_sorting = torch.argsort(target_column, descending=descending)
        if (indices is None) and (ranks is None):
            if join:
                raise ValueError(
                    "When the argument `join` is given as True,"
                    " the arguments `indices` and/or `ranks` are also required."
                    " However, both `indices` and `ranks` are encountered as None."
                )
            return indices_for_sorting
        result = TensorFrame()
        if indices is not None:
            result[indices] = indices_for_sorting
        if ranks is not None:
            rank_integers = torch.empty_like(indices_for_sorting)
            [n] = rank_integers.shape
            increasing_indices = torch.arange(n, device=rank_integers.device)
            rank_integers[indices_for_sorting] = increasing_indices
            result[ranks] = rank_integers
        if join:
            return self.hstack(result)
        return result

    def sort(self, by: Union[str, np.str_], *, descending: bool = False) -> "TensorFrame":
        """
        Return a sorted copy of this TensorFrame.

        Args:
            by: Name of the column according to which the sorting will be done.
            descending: If True, the sorting will be in descending order.
        Returns:
            The sorted copy of this TensorFrame.
        """
        indices_for_sorting = self.argsort(by, descending=descending)
        return self.pick[indices_for_sorting]

    def hstack(
        self,
        other: "TensorFrame",
        *,
        override: bool = False,
    ) -> "TensorFrame":
        """
        Horizontally join this TensorFrame with another.

        Args:
            other: The other TensorFrame.
            override: If this is given as True and if the other TensorFrame
                has overlapping columns, the other TensorFrame's values will
                override (i.e. will take priority) in the joined result.
        Returns:
            A new TensorFrame formed from joining this TensorFrame with the
            other TensorFrame.
        """
        if not override:
            left_columns = set(self.columns)
            right_columns = set(other.columns)
            common_columns = left_columns.intersection(right_columns)
            if len(common_columns) > 0:
                raise ValueError(
                    "Cannot horizontally stack these two TensorFrame objects,"
                    f" because they have the following shared column(s): {common_columns}."
                )

        if len(other) != len(self):
            raise ValueError(
                "Cannot horizontally stack these two TensorFrame objects,"
                f" because the number of rows of the first TensorFrame is {len(self)}"
                f" while the length of the second TensorFrame is {len(other)}."
            )

        result = TensorFrame(self, device=self.__device)
        for right_column in other.columns:
            result[right_column] = other[right_column]

        return result

    def vstack(self, other: "TensorFrame") -> "TensorFrame":
        """
        Vertically join this TensorFrame with the other TensorFrame.

        Args:
            other: The other TensorFrame which will be at the bottom.
        Returns:
            The joined TensorFrame.
        """
        if set(self.columns) != set(other.columns):
            raise ValueError(
                "Cannot vertically stack these two TensorFrame objects, because their columns do not perfectly match."
            )

        def combine_tensors(a: torch.Tensor, b: torch.Tensor) -> torch.Tensor:
            b = torch.as_tensor(b, device=a.device)
            if a.ndim != b.ndim:
                raise ValueError("Cannot combine two columns that have different numbers of dimensions")
            if a.ndim == 1:
                return torch.cat([a, b])
            elif a.ndim > 1:
                return torch.vstack([a, b])
            else:
                raise RuntimeError("Execution should not have reached this point. This is probably a bug.")

        newdata = OrderedDict()
        for col in self.columns:
            newdata[col] = combine_tensors(self[col], other[col])

        return TensorFrame(newdata, device=self.__device)

    def each(
        self,
        fn: Callable,
        *,
        chunk_size: Optional[int] = None,
        randomness: str = "error",
        join: bool = False,
        override: bool = False,
    ) -> "TensorFrame":
        """
        For each row of this TensorFrame, perform the operations of `fn`.

        `fn` is executed on the rows in a vectorized manner, with the help
        of `torch.vmap`.

        The function `fn` is expected to have this interface:

        ```python
        def fn(row: dict) -> dict:
            # `row` is a dictionary where the keys are column names.
            # This function is expected to return another dictionary.
            ...
        ```

        For example, if we have a TensorFrame with columns A and B, and
        if we want to create a new column C where, for each row, the
        value under C is the sum of A's value and B's value, then the
        function would look like this:

        ```python
        def do_summation_for_each_row(row: dict) -> dict:
            a = row["A"]
            b = row["B"]
            return {"C": a + b}
        ```

        Now, if our current TensorFrame looks like this:

        ```
         A    B
        ===  ===
         1    4
         2    5
         3    6
        ```

        Running `tensorframe.each(do_summation_for_each_row)` will result in
        the following new TensorFrame:

        ```
         C
        ===
         5
         7
         9
        ```

        Args:
            fn: A function which receives a dictionary as its argument, and
                returns another dictionary.
            chunk_size: For performing `fn` on each row, this `each` method
                uses `torch.vmap`. This `chunk_size` argument configures the
                size of the chunks on which the transformed `fn` will
                operate. If `chunk_size` is ignored, `fn` will operate on the
                whole batch.
            randomness: If given as "error" (which is the default), any random
                number generation operation within `fn` will raise an error.
                If given as "different", random numbers generated by `fn`
                will differ from row to row.
                If given as "same", random numbers generated by `fn` will be
                the same across the rows.
            join: If given as True, the resulting TensorFrame will also contain
                this TensorFrame's columns.
            override: If given as True (and if `join` is also True), and if
                the resulting TensorFrame has overlapping columns, the new
                values under the overlapping columns will take precedence.
        Returns:
            A new TensorFrame which stores the results of `fn`
        """
        if (not join) and override:
            raise ValueError("The argument `override` can be set as True only if `join` is also True.")
        input_dict = {k: self.__data[k] for k in self.columns}
        output_dict = vmap(fn, chunk_size=chunk_size, randomness=randomness)(input_dict)
        result = TensorFrame(output_dict, read_only=self.is_read_only, **(self.__get_default_device_kwargs()))
        if join:
            result = self.hstack(result, override=override)
        return result

    def sort_values(
        self,
        by: Union[str, np.str_, Sequence],
        *,
        ascending: Union[bool, Sequence] = True,
    ) -> "TensorFrame":
        """
        Like the `sort` method, but with a more pandas-like interface.

        Args:
            by: Column according to which this TensorFrame will be sorted.
            ascending: If True, the sorting will be in ascending order.
        Returns:
            The sorted counterpart of this TensorFrame.
        """
        by = _get_only_one_column_name(by)
        ascending = _get_only_one_boolean(ascending)
        return self.sort(by, descending=(not ascending))

    def nlargest(self, n: int, columns: Union[str, np.str_, Sequence]) -> "TensorFrame":
        """
        Sort this TensorFrame and take the largest `n` rows.

        Args:
            n: Number of rows of the resulting TensorFrame.
            columns: The name of the column according to which the rows will
                be sorted. Although the name of this argument is plural
                ("columns") for compatibility with pandas' interface, only
                one column name is supported.
        Returns:
            A new TensorFrame that contains the largest n rows of the original
            TensorFrame.
        """
        return self.sort_values(columns, ascending=False).pick[:n]

    def nsmallest(self, n: int, columns: Union[str, np.str_, Sequence]) -> "TensorFrame":
        """
        Sort this TensorFrame and take the smallest `n` rows.

        Args:
            n: Number of rows of the resulting TensorFrame.
            columns: The name of the column according to which the rows will
                be sorted. Although the name of this argument is plural
                ("columns") for compatibility with pandas' interface, only
                one column name is supported.
        Returns:
            A new TensorFrame that contains the smallest n rows of the original
            TensorFrame.
        """
        return self.sort_values(columns, ascending=True).pick[:n]

    def join(self, t: Union["TensorFrame", Sequence]) -> "TensorFrame":
        """
        Like the `hstack` method, but with a more pandas-like interface.

        Joins this TensorFrame with the other TensorFrame.

        Args:
            t: The TensorFrame that will be horizontally stacked to the right.
        Returns:
            A new TensorFrame which is the result of horizontally joining this
            TensorFrame with the other (`t`).
        """
        t = _get_only_one_tensorframe(t)
        return self.hstack(t)

    def drop(self, *, columns: Union[str, np.str_, Sequence]) -> "TensorFrame":
        """
        Get a new TensorFrame where the given columns are dropped.

        Args:
            columns: A single column name or a sequence of column names
                to be dropped.
        Returns:
            A modified copy of this TensorFrame where the specified `columns`
            are dropped.
        """
        if isinstance(columns, (str, np.str_)):
            columns = [columns]
        elif isinstance(columns, Sequence):
            pass  # nothing to do
        else:
            raise TypeError(
                "The argument `columns` was expected as a string or as a sequence of strings."
                f" However, it was received as an instance of this unrecognized type: {type(columns)}."
            )
        all_columns = set(self.__data.keys())
        columns_to_drop = set(str(s) for s in columns)
        if not columns_to_drop.issubset(all_columns):
            raise ValueError(
                "Some of the `columns` cannot be found within the original TensorFrame,"
                " and therefore, they cannot be dropped."
            )
        result = TensorFrame(device=self.__device)
        for col in self.__data.keys():
            if col not in columns_to_drop:
                result[col] = self.__data[col]
        if self.is_read_only:
            result = result.get_read_only_view()
        return result

    def with_columns(self, **kwargs) -> "TensorFrame":
        """
        Get a modified copy of this TensorFrame with some columns added/updated.

        The columns to be updated or added are expected as keyword arguments.
        For example, if a keyword argument is given as `A=new_a_values`, then,
        if `A` already exists in the original TensorFrame, those values will be
        dropped and the resulting TensorFrame will have the new values
        (`new_a_values`). On the other hand, if `A` does not exist in the
        original TensorFrame, the resulting TensorFrame will have a new column
        `A` with the given `new_a_values`.
        """
        columns_to_update = set(kwargs.keys())
        columns_already_updated = set()
        result = TensorFrame(device=self.__device)
        for col in self.__data.keys():
            if col in columns_to_update:
                result[col] = kwargs[col]
                columns_already_updated.add(col)
            else:
                result[col] = self.__data[col]
        for col in columns_to_update.difference(columns_already_updated):
            result[col] = kwargs[col]
        if self.is_read_only:
            result = result.get_read_only_view()
        return result

columns property

Columns as a list of strings

device property

Get the device(s) of this TensorFrame.

If different columns exist on different devices, a set of devices will be returned. If all the columns exist on the same device, then that device will be returned.

is_read_only property

True if this TensorFrame is read-only; False otherwise.

pick property

Get or set values of this TensorFrame.

Usage.

# Get values
tensorframe.pick[slice_or_indexlist, column_name]

# Set values
tensorframe.pick[slice_or_indexlist, column_name] = (
    ...  # A tensor or a list/tensor of indices or a TensorFrame
)

__init__(data=None, *, read_only=False, device=None)

__init__(...): Initialize the TensorFrame

Parameters:

Name	Type	Description	Default
data	Optional[Union[Mapping, TensorFrame, _PandasDataFrame]]	Optionally a TensorFrame, or a dictionary (where the keys are column names and the values are column values), or a pandas.DataFrame object. If provided, the tabular data given here are used for initializing the new TensorFrame.	None
read_only	bool	Whether or not the newly made TensorFrame will be read-only. A read-only TensorFrame's columns will be ReadOnlyTensors, and its columns and values will not change.	False
device	Optional[Union[device, str]]	If left as None, each column can be on a different device. If given as a string or a torch.device, each column will be forcefully moved to this given device.	None

Source code in evotorch/tools/tensorframe.py

def __init__(
    self,
    data: Optional[Union[Mapping, "TensorFrame", _PandasDataFrame]] = None,
    *,
    read_only: bool = False,
    device: Optional[Union[torch.device, str]] = None,
):
    """
    `__init__(...)`: Initialize the TensorFrame

    Args:
        data: Optionally a TensorFrame, or a dictionary (where the keys
            are column names and the values are column values), or a
            `pandas.DataFrame` object. If provided, the tabular data
            given here are used for initializing the new TensorFrame.
        read_only: Whether or not the newly made TensorFrame will be
            read-only. A read-only TensorFrame's columns will be
            ReadOnlyTensors, and its columns and values will not change.
        device: If left as None, each column can be on a different device.
            If given as a string or a `torch.device`, each column will
            be forcefully moved to this given device.
    """
    from .readonlytensor import as_read_only_tensor

    self.__is_read_only = False
    self.__device = None if device is None else torch.device(device)

    if read_only:

        def prepare_value(x):
            return as_read_only_tensor(x)

    else:

        def prepare_value(x):
            return x

    self.__data = OrderedDict()
    if data is None:
        pass  # no data is given, nothing to do
    elif _is_pandas_dataframe(data):
        for k in data.columns:
            v = data[k]
            self.__setitem__(k, prepare_value(v))
    elif isinstance(data, Mapping):
        for k, v in data.items():
            self.__setitem__(k, prepare_value(v))
    elif isinstance(data, TensorFrame):
        for k, v in data.__data.items():
            self.__setitem__(k, prepare_value(v))
    else:
        raise TypeError(
            "When constructing a new TensorFrame, the argument `data` was expected as a dictionary-like object"
            " (an instance of Mapping), or as a TensorFrame."
            f" However, the encountered object is of type {type(data)}."
        )

    self.__is_read_only = read_only
    self._initialized = True

argsort(by, *, indices=None, ranks=None, descending=False, join=False)

Return row indices (also optionally ranks) for sorting the TensorFrame.

For example, let us assume that we have a TensorFrame named table. We can sort this table like this:

indices_for_sorting = table.argsort(by="A")  # sort by the column A
sorted_table = table.pick[indices_for_sorting]

Parameters:

Name	Type	Description	Default
by	Union[str, str_]	The name of the column according to which the TensorFrame will be sorted.	required
indices	Optional[Union[str, str_]]	If given as a string s, the result will be a new TensorFrame, and the sorting indices will be listed under a column named s.	None
ranks	Optional[Union[str, str_]]	If given as a string z, the result will be a new TensorFrame, and the ranks will be listed under a column named z.	None
descending	bool	If True, the sorting will be in descending order.	False
join	bool	Can be used only if column names are given via indices and/or ranks. If join is True, then the resulting TensorFrame will have the sorted TensorFrame's columns as well.	False

Source code in evotorch/tools/tensorframe.py

def argsort(
    self,
    by: Union[str, np.str_],
    *,
    indices: Optional[Union[str, np.str_]] = None,
    ranks: Optional[Union[str, np.str_]] = None,
    descending: bool = False,
    join: bool = False,
) -> Union[torch.Tensor, "TensorFrame"]:
    """
    Return row indices (also optionally ranks) for sorting the TensorFrame.

    For example, let us assume that we have a TensorFrame named `table`.
    We can sort this `table` like this:

    ```python
    indices_for_sorting = table.argsort(by="A")  # sort by the column A
    sorted_table = table.pick[indices_for_sorting]
    ```

    Args:
        by: The name of the column according to which the TensorFrame
            will be sorted.
        indices: If given as a string `s`, the result will be a new
            TensorFrame, and the sorting indices will be listed under
            a column named `s`.
        ranks: If given as a string `z`, the result will be a new
            TensorFrame, and the ranks will be listed under a column
            named `z`.
        descending: If True, the sorting will be in descending order.
        join: Can be used only if column names are given via `indices`
            and/or `ranks`. If `join` is True, then the resulting
            TensorFrame will have the sorted TensorFrame's columns
            as well.
    Returns:
        A tensor of indices, or a TensorFrame.
    """
    by = str(by)
    target_column = self[by]
    indices_for_sorting = torch.argsort(target_column, descending=descending)
    if (indices is None) and (ranks is None):
        if join:
            raise ValueError(
                "When the argument `join` is given as True,"
                " the arguments `indices` and/or `ranks` are also required."
                " However, both `indices` and `ranks` are encountered as None."
            )
        return indices_for_sorting
    result = TensorFrame()
    if indices is not None:
        result[indices] = indices_for_sorting
    if ranks is not None:
        rank_integers = torch.empty_like(indices_for_sorting)
        [n] = rank_integers.shape
        increasing_indices = torch.arange(n, device=rank_integers.device)
        rank_integers[indices_for_sorting] = increasing_indices
        result[ranks] = rank_integers
    if join:
        return self.hstack(result)
    return result

as_tensor(x, *, to_work_with=None, broadcast_if_scalar=False)

Convert the given object x to a PyTorch tensor.

Parameters:

Name	Type	Description	Default
x	Any	The object to be converted to a PyTorch tensor.	required
to_work_with	Optional[Union[str, str_, Tensor]]	Optionally a string, referring to an existing column within this TensorFrame, or a PyTorch tensor. The object x will be converted to a PyTorch tensor on the same device with this given column/tensor. If to_work_with is left as None, x will be converted to a PyTorch tensor on the same device with this TensorFrame's first column.	None
broadcast_if_scalar	bool	If this argument is given as True and if x is a scalar, its tensor-counterpart will be broadcast to a vector of length n, where n is the number of rows of this TensorFrame.	False

Source code in evotorch/tools/tensorframe.py

def as_tensor(
    self,
    x: Any,
    *,
    to_work_with: Optional[Union[str, np.str_, torch.Tensor]] = None,
    broadcast_if_scalar: bool = False,
) -> torch.Tensor:
    """
    Convert the given object `x` to a PyTorch tensor.

    Args:
        x: The object to be converted to a PyTorch tensor.
        to_work_with: Optionally a string, referring to an existing column
            within this TensorFrame, or a PyTorch tensor. The object `x`
            will be converted to a PyTorch tensor on the same device with
            this given column/tensor. If `to_work_with` is left as None,
            `x` will be converted to a PyTorch tensor on the same device
            with this TensorFrame's first column.
        broadcast_if_scalar: If this argument is given as True and if `x`
            is a scalar, its tensor-counterpart will be broadcast to a
            vector of length `n`, where `n` is the number of rows of this
            TensorFrame.
    Returns:
        The tensor counterpart of `x`.
    """
    from .readonlytensor import as_read_only_tensor

    if to_work_with is not None:
        if isinstance(to_work_with, torch.Tensor):
            pass  # nothing to do
        elif isinstance(to_work_with, (str, np.str_)):
            to_work_with = self.__data[str(to_work_with)]
        else:
            raise TypeError(
                "The argument `to_work_with` was expected as a PyTorch tensor or as a string"
                " (that refers to one of the columns of this TensorFrame)."
                f" However, it was received as an instance of {type(to_work_with)}."
            )
        result = torch.as_tensor(x, device=to_work_with.device)
    else:
        if self.__is_read_only:
            convert = as_read_only_tensor
        else:
            convert = torch.as_tensor

        if isinstance(x, torch.Tensor):
            result = convert(x, **(self.__get_default_device_kwargs()))
        else:
            result = convert(x, **(self.__get_first_tensor_device_kwargs()))

    if broadcast_if_scalar and (result.ndim == 0):
        first_tensor = self.__get_first_tensor()
        if first_tensor is None:
            raise ValueError("The first column cannot be given as a scalar.")
        result = result * torch.ones(first_tensor.shape[0], dtype=result.dtype, device=result.device)

    return result

clone(*, preserve_read_only=False, memo=None)

Get a clone of this TensorFrame.

Parameters:

Name	Type	Description	Default
preserve_read_only	bool	If True, the newly made clone will be read-only only if this TensorFrame is also read-only.	False

Source code in evotorch/tools/tensorframe.py

def clone(self, *, preserve_read_only: bool = False, memo: Optional[dict] = None) -> "TensorFrame":
    """
    Get a clone of this TensorFrame.

    Args:
        preserve_read_only: If True, the newly made clone will be read-only
            only if this TensorFrame is also read-only.
    Returns:
        The clone of this TensorFrame.
    """
    if memo is None:
        memo = {}

    self_id = id(self)
    if self_id in memo:
        return memo[self_id]

    new_read_only = self.__is_read_only if preserve_read_only else False

    newdata = OrderedDict()
    for k in self.columns:
        newdata[k] = self.__data[k].clone()

    return TensorFrame(newdata, read_only=new_read_only, device=self.__device)

cpu()

Get a shallow copy of this TensorFrame with all columns moved to cpu.

Source code in evotorch/tools/tensorframe.py

def cpu(self) -> "TensorFrame":
    """
    Get a shallow copy of this TensorFrame with all columns moved to cpu.
    """
    return self.to("cpu")

cuda()

Get a shallow copy of this TensorFrame with all columns moved to cuda.

Source code in evotorch/tools/tensorframe.py

def cuda(self) -> "TensorFrame":
    """
    Get a shallow copy of this TensorFrame with all columns moved to cuda.
    """
    return self.to("cuda")

drop(*, columns)

Get a new TensorFrame where the given columns are dropped.

Parameters:

Name	Type	Description	Default
columns	Union[str, str_, Sequence]	A single column name or a sequence of column names to be dropped.	required

Source code in evotorch/tools/tensorframe.py

def drop(self, *, columns: Union[str, np.str_, Sequence]) -> "TensorFrame":
    """
    Get a new TensorFrame where the given columns are dropped.

    Args:
        columns: A single column name or a sequence of column names
            to be dropped.
    Returns:
        A modified copy of this TensorFrame where the specified `columns`
        are dropped.
    """
    if isinstance(columns, (str, np.str_)):
        columns = [columns]
    elif isinstance(columns, Sequence):
        pass  # nothing to do
    else:
        raise TypeError(
            "The argument `columns` was expected as a string or as a sequence of strings."
            f" However, it was received as an instance of this unrecognized type: {type(columns)}."
        )
    all_columns = set(self.__data.keys())
    columns_to_drop = set(str(s) for s in columns)
    if not columns_to_drop.issubset(all_columns):
        raise ValueError(
            "Some of the `columns` cannot be found within the original TensorFrame,"
            " and therefore, they cannot be dropped."
        )
    result = TensorFrame(device=self.__device)
    for col in self.__data.keys():
        if col not in columns_to_drop:
            result[col] = self.__data[col]
    if self.is_read_only:
        result = result.get_read_only_view()
    return result

each(fn, *, chunk_size=None, randomness='error', join=False, override=False)

For each row of this TensorFrame, perform the operations of fn.

fn is executed on the rows in a vectorized manner, with the help of torch.vmap.

The function fn is expected to have this interface:

def fn(row: dict) -> dict:
    # `row` is a dictionary where the keys are column names.
    # This function is expected to return another dictionary.
    ...

For example, if we have a TensorFrame with columns A and B, and if we want to create a new column C where, for each row, the value under C is the sum of A's value and B's value, then the function would look like this:

def do_summation_for_each_row(row: dict) -> dict:
    a = row["A"]
    b = row["B"]
    return {"C": a + b}

Now, if our current TensorFrame looks like this:

 A    B
===  ===
 1    4
 2    5
 3    6

Running tensorframe.each(do_summation_for_each_row) will result in the following new TensorFrame:

 C
===
 5
 7
 9

Parameters:

Name	Type	Description	Default
fn	Callable	A function which receives a dictionary as its argument, and returns another dictionary.	required
chunk_size	Optional[int]	For performing fn on each row, this each method uses torch.vmap. This chunk_size argument configures the size of the chunks on which the transformed fn will operate. If chunk_size is ignored, fn will operate on the whole batch.	None
randomness	str	If given as "error" (which is the default), any random number generation operation within fn will raise an error. If given as "different", random numbers generated by fn will differ from row to row. If given as "same", random numbers generated by fn will be the same across the rows.	'error'
join	bool	If given as True, the resulting TensorFrame will also contain this TensorFrame's columns.	False
override	bool	If given as True (and if join is also True), and if the resulting TensorFrame has overlapping columns, the new values under the overlapping columns will take precedence.	False

Source code in evotorch/tools/tensorframe.py

def each(
    self,
    fn: Callable,
    *,
    chunk_size: Optional[int] = None,
    randomness: str = "error",
    join: bool = False,
    override: bool = False,
) -> "TensorFrame":
    """
    For each row of this TensorFrame, perform the operations of `fn`.

    `fn` is executed on the rows in a vectorized manner, with the help
    of `torch.vmap`.

    The function `fn` is expected to have this interface:

    ```python
    def fn(row: dict) -> dict:
        # `row` is a dictionary where the keys are column names.
        # This function is expected to return another dictionary.
        ...
    ```

    For example, if we have a TensorFrame with columns A and B, and
    if we want to create a new column C where, for each row, the
    value under C is the sum of A's value and B's value, then the
    function would look like this:

    ```python
    def do_summation_for_each_row(row: dict) -> dict:
        a = row["A"]
        b = row["B"]
        return {"C": a + b}
    ```

    Now, if our current TensorFrame looks like this:

    ```
     A    B
    ===  ===
     1    4
     2    5
     3    6
    ```

    Running `tensorframe.each(do_summation_for_each_row)` will result in
    the following new TensorFrame:

    ```
     C
    ===
     5
     7
     9
    ```

    Args:
        fn: A function which receives a dictionary as its argument, and
            returns another dictionary.
        chunk_size: For performing `fn` on each row, this `each` method
            uses `torch.vmap`. This `chunk_size` argument configures the
            size of the chunks on which the transformed `fn` will
            operate. If `chunk_size` is ignored, `fn` will operate on the
            whole batch.
        randomness: If given as "error" (which is the default), any random
            number generation operation within `fn` will raise an error.
            If given as "different", random numbers generated by `fn`
            will differ from row to row.
            If given as "same", random numbers generated by `fn` will be
            the same across the rows.
        join: If given as True, the resulting TensorFrame will also contain
            this TensorFrame's columns.
        override: If given as True (and if `join` is also True), and if
            the resulting TensorFrame has overlapping columns, the new
            values under the overlapping columns will take precedence.
    Returns:
        A new TensorFrame which stores the results of `fn`
    """
    if (not join) and override:
        raise ValueError("The argument `override` can be set as True only if `join` is also True.")
    input_dict = {k: self.__data[k] for k in self.columns}
    output_dict = vmap(fn, chunk_size=chunk_size, randomness=randomness)(input_dict)
    result = TensorFrame(output_dict, read_only=self.is_read_only, **(self.__get_default_device_kwargs()))
    if join:
        result = self.hstack(result, override=override)
    return result

get_read_only_view()

Get a read-only counterpart of this TensorFrame.

Source code in evotorch/tools/tensorframe.py

def get_read_only_view(self) -> "TensorFrame":
    """
    Get a read-only counterpart of this TensorFrame.
    """
    return TensorFrame(self.__data, read_only=True, device=self.__device)

hstack(other, *, override=False)

Horizontally join this TensorFrame with another.

Parameters:

Name	Type	Description	Default
other	TensorFrame	The other TensorFrame.	required
override	bool	If this is given as True and if the other TensorFrame has overlapping columns, the other TensorFrame's values will override (i.e. will take priority) in the joined result.	False

Source code in evotorch/tools/tensorframe.py

def hstack(
    self,
    other: "TensorFrame",
    *,
    override: bool = False,
) -> "TensorFrame":
    """
    Horizontally join this TensorFrame with another.

    Args:
        other: The other TensorFrame.
        override: If this is given as True and if the other TensorFrame
            has overlapping columns, the other TensorFrame's values will
            override (i.e. will take priority) in the joined result.
    Returns:
        A new TensorFrame formed from joining this TensorFrame with the
        other TensorFrame.
    """
    if not override:
        left_columns = set(self.columns)
        right_columns = set(other.columns)
        common_columns = left_columns.intersection(right_columns)
        if len(common_columns) > 0:
            raise ValueError(
                "Cannot horizontally stack these two TensorFrame objects,"
                f" because they have the following shared column(s): {common_columns}."
            )

    if len(other) != len(self):
        raise ValueError(
            "Cannot horizontally stack these two TensorFrame objects,"
            f" because the number of rows of the first TensorFrame is {len(self)}"
            f" while the length of the second TensorFrame is {len(other)}."
        )

    result = TensorFrame(self, device=self.__device)
    for right_column in other.columns:
        result[right_column] = other[right_column]

    return result

join(t)

Like the hstack method, but with a more pandas-like interface.

Joins this TensorFrame with the other TensorFrame.

Parameters:

Name	Type	Description	Default
t	Union[TensorFrame, Sequence]	The TensorFrame that will be horizontally stacked to the right.	required

Source code in evotorch/tools/tensorframe.py

def join(self, t: Union["TensorFrame", Sequence]) -> "TensorFrame":
    """
    Like the `hstack` method, but with a more pandas-like interface.

    Joins this TensorFrame with the other TensorFrame.

    Args:
        t: The TensorFrame that will be horizontally stacked to the right.
    Returns:
        A new TensorFrame which is the result of horizontally joining this
        TensorFrame with the other (`t`).
    """
    t = _get_only_one_tensorframe(t)
    return self.hstack(t)

nlargest(n, columns)

Sort this TensorFrame and take the largest n rows.

Parameters:

Name	Type	Description	Default
n	int	Number of rows of the resulting TensorFrame.	required
columns	Union[str, str_, Sequence]	The name of the column according to which the rows will be sorted. Although the name of this argument is plural ("columns") for compatibility with pandas' interface, only one column name is supported.	required

Source code in evotorch/tools/tensorframe.py

def nlargest(self, n: int, columns: Union[str, np.str_, Sequence]) -> "TensorFrame":
    """
    Sort this TensorFrame and take the largest `n` rows.

    Args:
        n: Number of rows of the resulting TensorFrame.
        columns: The name of the column according to which the rows will
            be sorted. Although the name of this argument is plural
            ("columns") for compatibility with pandas' interface, only
            one column name is supported.
    Returns:
        A new TensorFrame that contains the largest n rows of the original
        TensorFrame.
    """
    return self.sort_values(columns, ascending=False).pick[:n]

nsmallest(n, columns)

Sort this TensorFrame and take the smallest n rows.

Parameters:

Name	Type	Description	Default
n	int	Number of rows of the resulting TensorFrame.	required
columns	Union[str, str_, Sequence]	The name of the column according to which the rows will be sorted. Although the name of this argument is plural ("columns") for compatibility with pandas' interface, only one column name is supported.	required

Source code in evotorch/tools/tensorframe.py

def nsmallest(self, n: int, columns: Union[str, np.str_, Sequence]) -> "TensorFrame":
    """
    Sort this TensorFrame and take the smallest `n` rows.

    Args:
        n: Number of rows of the resulting TensorFrame.
        columns: The name of the column according to which the rows will
            be sorted. Although the name of this argument is plural
            ("columns") for compatibility with pandas' interface, only
            one column name is supported.
    Returns:
        A new TensorFrame that contains the smallest n rows of the original
        TensorFrame.
    """
    return self.sort_values(columns, ascending=True).pick[:n]

sort(by, *, descending=False)

Return a sorted copy of this TensorFrame.

Parameters:

Name	Type	Description	Default
by	Union[str, str_]	Name of the column according to which the sorting will be done.	required
descending	bool	If True, the sorting will be in descending order.	False

Source code in evotorch/tools/tensorframe.py

def sort(self, by: Union[str, np.str_], *, descending: bool = False) -> "TensorFrame":
    """
    Return a sorted copy of this TensorFrame.

    Args:
        by: Name of the column according to which the sorting will be done.
        descending: If True, the sorting will be in descending order.
    Returns:
        The sorted copy of this TensorFrame.
    """
    indices_for_sorting = self.argsort(by, descending=descending)
    return self.pick[indices_for_sorting]

sort_values(by, *, ascending=True)

Like the sort method, but with a more pandas-like interface.

Parameters:

Name	Type	Description	Default
by	Union[str, str_, Sequence]	Column according to which this TensorFrame will be sorted.	required
ascending	Union[bool, Sequence]	If True, the sorting will be in ascending order.	True

Source code in evotorch/tools/tensorframe.py

def sort_values(
    self,
    by: Union[str, np.str_, Sequence],
    *,
    ascending: Union[bool, Sequence] = True,
) -> "TensorFrame":
    """
    Like the `sort` method, but with a more pandas-like interface.

    Args:
        by: Column according to which this TensorFrame will be sorted.
        ascending: If True, the sorting will be in ascending order.
    Returns:
        The sorted counterpart of this TensorFrame.
    """
    by = _get_only_one_column_name(by)
    ascending = _get_only_one_boolean(ascending)
    return self.sort(by, descending=(not ascending))

to_string(*, max_depth=DEFAULT_MAX_DEPTH_FOR_PRINTING)

Return the string representation of this TensorFrame

Source code in evotorch/tools/tensorframe.py

def to_string(self, *, max_depth: int = DEFAULT_MAX_DEPTH_FOR_PRINTING) -> str:
    """
    Return the string representation of this TensorFrame
    """
    if len(self.columns) == 0:
        return type(self).__name__ + "()"

    if (max_depth >= DEFAULT_MAX_DEPTH_FOR_PRINTING) and self.__all_columns_are_one_dimensional():
        return self.__tabular_repr()
    else:
        return self.__non_tabular_repr(max_depth=max_depth)

vstack(other)

Vertically join this TensorFrame with the other TensorFrame.

Parameters:

Name	Type	Description	Default
other	TensorFrame	The other TensorFrame which will be at the bottom.	required

Source code in evotorch/tools/tensorframe.py

def vstack(self, other: "TensorFrame") -> "TensorFrame":
    """
    Vertically join this TensorFrame with the other TensorFrame.

    Args:
        other: The other TensorFrame which will be at the bottom.
    Returns:
        The joined TensorFrame.
    """
    if set(self.columns) != set(other.columns):
        raise ValueError(
            "Cannot vertically stack these two TensorFrame objects, because their columns do not perfectly match."
        )

    def combine_tensors(a: torch.Tensor, b: torch.Tensor) -> torch.Tensor:
        b = torch.as_tensor(b, device=a.device)
        if a.ndim != b.ndim:
            raise ValueError("Cannot combine two columns that have different numbers of dimensions")
        if a.ndim == 1:
            return torch.cat([a, b])
        elif a.ndim > 1:
            return torch.vstack([a, b])
        else:
            raise RuntimeError("Execution should not have reached this point. This is probably a bug.")

    newdata = OrderedDict()
    for col in self.columns:
        newdata[col] = combine_tensors(self[col], other[col])

    return TensorFrame(newdata, device=self.__device)

with_columns(**kwargs)

Get a modified copy of this TensorFrame with some columns added/updated.

The columns to be updated or added are expected as keyword arguments. For example, if a keyword argument is given as A=new_a_values, then, if A already exists in the original TensorFrame, those values will be dropped and the resulting TensorFrame will have the new values (new_a_values). On the other hand, if A does not exist in the original TensorFrame, the resulting TensorFrame will have a new column A with the given new_a_values.

Source code in evotorch/tools/tensorframe.py

def with_columns(self, **kwargs) -> "TensorFrame":
    """
    Get a modified copy of this TensorFrame with some columns added/updated.

    The columns to be updated or added are expected as keyword arguments.
    For example, if a keyword argument is given as `A=new_a_values`, then,
    if `A` already exists in the original TensorFrame, those values will be
    dropped and the resulting TensorFrame will have the new values
    (`new_a_values`). On the other hand, if `A` does not exist in the
    original TensorFrame, the resulting TensorFrame will have a new column
    `A` with the given `new_a_values`.
    """
    columns_to_update = set(kwargs.keys())
    columns_already_updated = set()
    result = TensorFrame(device=self.__device)
    for col in self.__data.keys():
        if col in columns_to_update:
            result[col] = kwargs[col]
            columns_already_updated.add(col)
        else:
            result[col] = self.__data[col]
    for col in columns_to_update.difference(columns_already_updated):
        result[col] = kwargs[col]
    if self.is_read_only:
        result = result.get_read_only_view()
    return result

with_enforced_device(device)

Make a shallow copy of this TensorFrame with an enforced device.

In the newly made shallow copy, columns will be forcefully moved onto the specified device.

Parameters:

Name	Type	Description	Default
device	Union[str, device]	The device to which the new TensorFrame's columns will move.	required

Source code in evotorch/tools/tensorframe.py

def with_enforced_device(self, device: Union[str, torch.device]) -> "TensorFrame":
    """
    Make a shallow copy of this TensorFrame with an enforced device.

    In the newly made shallow copy, columns will be forcefully moved onto
    the specified device.

    Args:
        device: The device to which the new TensorFrame's columns will move.
    Returns:
        A shallow copy of this TensorFrame with an enforced device.
    """
    if device is None:
        raise TypeError("When using the method `with_enforced_device`, the argument `device` cannot be None")

    return TensorFrame(self.__data, read_only=self.__is_read_only, device=device)

without_enforced_device()

Make a shallow copy of this TensorFrame without any enforced device.

In the newly made shallow copy, columns will be able to exist on different devices.

Returns:

Type	Description
TensorFrame	A shallow copy of this TensorFrame without any enforced device.

Source code in evotorch/tools/tensorframe.py

def without_enforced_device(self) -> "TensorFrame":
    """
    Make a shallow copy of this TensorFrame without any enforced device.

    In the newly made shallow copy, columns will be able to exist on
    different devices.

    Returns:
        A shallow copy of this TensorFrame without any enforced device.
    """
    return TensorFrame(self.__data, read_only=self.__is_read_only, device=None)

as_read_only_tensor(x, *, dtype=None, device=None)

Convert the given object to a ReadOnlyTensor.

The provided object can be a scalar, or an Iterable of numeric data, or an ObjectArray.

This function can be thought as the read-only counterpart of PyTorch's torch.as_tensor(...) function.

Parameters:

Name	Type	Description	Default
x	Any	The object to be converted to a ReadOnlyTensor.	required
dtype	Optional[dtype]	The dtype of the new ReadOnlyTensor (e.g. torch.float32). If this argument is not specified, dtype will be inferred from x. For example, if x is a PyTorch tensor or a numpy array, its existing dtype will be kept.	None
device	Optional[Union[str, device]]	The device in which the ReadOnlyTensor will be stored (e.g. "cpu"). If this argument is not specified, the device which is storing the original x will be re-used.	None

Source code in evotorch/tools/readonlytensor.py

def as_read_only_tensor(
    x: Any, *, dtype: Optional[torch.dtype] = None, device: Optional[Union[str, torch.device]] = None
) -> Iterable:
    """
    Convert the given object to a ReadOnlyTensor.

    The provided object can be a scalar, or an Iterable of numeric data,
    or an ObjectArray.

    This function can be thought as the read-only counterpart of PyTorch's
    `torch.as_tensor(...)` function.

    Args:
        x: The object to be converted to a ReadOnlyTensor.
        dtype: The dtype of the new ReadOnlyTensor (e.g. torch.float32).
            If this argument is not specified, dtype will be inferred from `x`.
            For example, if `x` is a PyTorch tensor or a numpy array, its
            existing dtype will be kept.
        device: The device in which the ReadOnlyTensor will be stored
            (e.g. "cpu").
            If this argument is not specified, the device which is storing
            the original `x` will be re-used.
    Returns:
        The read-only counterpart of the provided object.
    """
    from .objectarray import ObjectArray

    kwargs = _device_and_dtype_kwargs(dtype=dtype, device=device)
    if isinstance(x, ObjectArray):
        if len(kwargs) != 0:
            raise ValueError(
                f"read_only_tensor(...): when making a read-only tensor from an ObjectArray,"
                f" the arguments `dtype` and `device` were not expected."
                f" However, the received keyword arguments are: {kwargs}."
            )
        return x.get_read_only_view()
    else:
        return torch.as_tensor(x, **kwargs).as_subclass(ReadOnlyTensor)

as_tensor(x, *, dtype=None, device=None)

Get the tensor counterpart of the given object x.

This function can be used to convert native Python objects to tensors:

my_tensor = as_tensor([1.0, 2.0, 3.0], dtype="float32")

One can also use this function to convert an existing tensor to another dtype:

my_new_tensor = as_tensor(my_tensor, dtype="float16")

This function can also be used for moving a tensor from one device to another:

my_gpu_tensor = as_tensor(my_tensor, device="cuda:0")

This function can also create ObjectArray instances when dtype is given as object or Any or "object" or "O".

my_objects = as_tensor([1, {"a": 3}], dtype=object)

Parameters:

Name	Type	Description	Default
x	Any	Any object to be converted to a tensor.	required
dtype	Optional[DType]	Optionally a string (e.g. "float32") or a PyTorch dtype (e.g. torch.float32) or, for creating an ObjectArray, "object" (as string) or object or Any. If dtype is not specified, the default behavior of torch.as_tensor(...) will be used, that is, dtype will be inferred from x.	None
device	Optional[Device]	The device in which the resulting tensor will be stored.	None

Source code in evotorch/tools/misc.py

def as_tensor(x: Any, *, dtype: Optional[DType] = None, device: Optional[Device] = None) -> Iterable:
    """
    Get the tensor counterpart of the given object `x`.

    This function can be used to convert native Python objects to tensors:

        my_tensor = as_tensor([1.0, 2.0, 3.0], dtype="float32")

    One can also use this function to convert an existing tensor to another
    dtype:

        my_new_tensor = as_tensor(my_tensor, dtype="float16")

    This function can also be used for moving a tensor from one device to
    another:

        my_gpu_tensor = as_tensor(my_tensor, device="cuda:0")

    This function can also create ObjectArray instances when dtype is
    given as `object` or `Any` or "object" or "O".

        my_objects = as_tensor([1, {"a": 3}], dtype=object)

    Args:
        x: Any object to be converted to a tensor.
        dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
            (e.g. torch.float32) or, for creating an `ObjectArray`,
            "object" (as string) or `object` or `Any`.
            If `dtype` is not specified, the default behavior of
            `torch.as_tensor(...)` will be used, that is, dtype will be
            inferred from `x`.
        device: The device in which the resulting tensor will be stored.
    Returns:
        The tensor counterpart of the given object `x`.
    """
    from .objectarray import ObjectArray

    if (dtype is None) and isinstance(x, (torch.Tensor, ObjectArray)):
        if (device is None) or (str(device) == "cpu"):
            return x
        else:
            raise ValueError(
                f"An ObjectArray cannot be moved into a device other than 'cpu'. The received device is: {device}."
            )
    elif is_dtype_object(dtype):
        if (device is not None) and (str(device) != "cpu"):
            raise ValueError(
                f"An ObjectArray cannot be created on a device other than 'cpu'. The received device is: {device}."
            )
        if isinstance(x, ObjectArray):
            return x
        else:
            x = list(x)
            n = len(x)
            result = ObjectArray(n)
            result[:] = x
            return result
    else:
        dtype = to_torch_dtype(dtype)
        return torch.as_tensor(x, dtype=dtype, device=device)

cast_tensors_in_container(container, *, dtype=None, device=None, memo=None)

Cast and/or transfer all the tensors in a Python container.

Parameters:

Name	Type	Description	Default
dtype	Optional[DType]	If given as a dtype and not as None, then all the PyTorch tensors in the container will be cast to this dtype.	None
device	Optional[Device]	If given as a device and not as None, then all the PyTorch tensors in the container will be copied to this device.	None
memo	Optional[dict]	Optionally a memo dictionary to handle shared objects and circular references. In most scenarios, when calling this function from outside, this is expected as None.	None

Source code in evotorch/tools/misc.py

def cast_tensors_in_container(
    container: Any,
    *,
    dtype: Optional[DType] = None,
    device: Optional[Device] = None,
    memo: Optional[dict] = None,
) -> Any:
    """
    Cast and/or transfer all the tensors in a Python container.

    Args:
        dtype: If given as a dtype and not as None, then all the PyTorch
            tensors in the container will be cast to this dtype.
        device: If given as a device and not as None, then all the PyTorch
            tensors in the container will be copied to this device.
        memo: Optionally a memo dictionary to handle shared objects and
            circular references. In most scenarios, when calling this
            function from outside, this is expected as None.
    Returns:
        A new copy of the original container in which the tensors have the
        desired dtype and/or device.
    """

    if memo is None:
        memo = {}

    container_id = id(container)
    if container_id in memo:
        return memo[container_id]

    cast_kwargs = {}
    if dtype is not None:
        cast_kwargs["dtype"] = to_torch_dtype(dtype)
    if device is not None:
        cast_kwargs["device"] = device

    def call_self(sub_container: Any) -> Any:
        return cast_tensors_in_container(sub_container, dtype=dtype, device=device, memo=memo)

    if isinstance(container, torch.Tensor):
        result = torch.as_tensor(container, **cast_kwargs)
        memo[container_id] = result
    elif (container is None) or isinstance(container, (Number, str, bytes, bool)):
        result = container
    elif isinstance(container, set):
        result = set()
        memo[container_id] = result
        for x in container:
            result.add(call_self(x))
    elif isinstance(container, Mapping):
        result = {}
        memo[container_id] = result
        for k, v in container.items():
            result[k] = call_self(v)
    elif isinstance(container, tuple):
        result = []
        memo[container_id] = result
        for x in container:
            result.append(call_self(x))
        if hasattr(container, "_fields"):
            result = type(container)(*result)
        else:
            result = type(container)(result)
        memo[container_id] = result
    elif isinstance(container, Iterable):
        result = []
        memo[container_id] = result
        for x in container:
            result.append(call_self(x))
    else:
        raise TypeError(f"Encountered an object of unrecognized type: {type(container)}")

    return result

clip_tensor(x, lb=None, ub=None, ensure_copy=True)

Clip the values of a tensor with respect to the given bounds.

Parameters:

Name	Type	Description	Default
x	Tensor	The PyTorch tensor whose values will be clipped.	required
lb	Optional[Union[float, Iterable]]	Lower bounds, as a PyTorch tensor. Can be None if there are no lower bounds.	None
ub	Optional[Union[float, Iterable]]	Upper bounds, as a PyTorch tensor. Can be None if there are no upper bonuds.	None
ensure_copy	bool	If ensure_copy is True, the result will be a clipped copy of the original tensor. If ensure_copy is False, and both lb and ub are None, then there is nothing to do, so, the result will be the original tensor itself, not a copy of it.	True

Source code in evotorch/tools/misc.py

@torch.no_grad()
def clip_tensor(
    x: torch.Tensor,
    lb: Optional[Union[float, Iterable]] = None,
    ub: Optional[Union[float, Iterable]] = None,
    ensure_copy: bool = True,
) -> torch.Tensor:
    """
    Clip the values of a tensor with respect to the given bounds.

    Args:
        x: The PyTorch tensor whose values will be clipped.
        lb: Lower bounds, as a PyTorch tensor.
            Can be None if there are no lower bounds.
        ub: Upper bounds, as a PyTorch tensor.
            Can be None if there are no upper bonuds.
        ensure_copy: If `ensure_copy` is True, the result will be
            a clipped copy of the original tensor.
            If `ensure_copy` is False, and both `lb` and `ub`
            are None, then there is nothing to do, so, the result
            will be the original tensor itself, not a copy of it.
    Returns:
        The clipped tensor.
    """
    result = x
    if lb is not None:
        lb = torch.as_tensor(lb, dtype=x.dtype, device=x.device)
        result = torch.max(result, lb)
    if ub is not None:
        ub = torch.as_tensor(ub, dtype=x.dtype, device=x.device)
        result = torch.min(result, ub)
    if ensure_copy and result is x:
        result = x.clone()
    return result

clone(x, *, memo=None)

Get a deep copy of the given object.

The cloning is done in no_grad mode.

When this function is used on read-only containers (e.g. ReadOnlyTensor, ImmutableContainer, etc.), the created clones preserve their read-only behaviors. For creating a mutable clone of an immutable object, use their clone() method instead.

Returns:

Type	Description
Any	The deep copy of the given object.

Source code in evotorch/tools/misc.py

@torch.no_grad()
def clone(x: Any, *, memo: Optional[dict] = None) -> Any:
    """
    Get a deep copy of the given object.

    The cloning is done in no_grad mode.

    When this function is used on read-only containers (e.g. ReadOnlyTensor,
    ImmutableContainer, etc.), the created clones preserve their read-only
    behaviors. For creating a mutable clone of an immutable object,
    use their `clone()` method instead.

    Returns:
        The deep copy of the given object.
    """
    from .cloning import deep_clone

    if memo is None:
        memo = {}
    return deep_clone(x, otherwise_deepcopy=True, memo=memo)

device_of(x)

Get the device of the given object.

Parameters:

Name	Type	Description	Default
x	Any	The object whose device is being queried. The object can be a PyTorch tensor, or a PyTorch module (in which case the device of the first parameter tensor will be returned), or an ObjectArray (in which case the returned device will be the cpu device), or any object with the attribute device.	required

Source code in evotorch/tools/misc.py

def device_of(x: Any) -> Device:
    """
    Get the device of the given object.

    Args:
        x: The object whose device is being queried.
            The object can be a PyTorch tensor, or a PyTorch module
            (in which case the device of the first parameter tensor
            will be returned), or an ObjectArray (in which case
            the returned device will be the cpu device), or any object
            with the attribute `device`.
    Returns:
        The device of the given object.
    """
    if isinstance(x, nn.Module):
        result = None
        for param in x.parameters():
            result = param.device
            break
        if result is None:
            raise ValueError(f"Cannot determine the device of the module {x}")
        return result
    else:
        return x.device

device_of_container(container, *, visited=None, visiting=None)

Get the device of the given container.

It is assumed that the given container stores PyTorch tensors from which the device information will be extracted. If the container contains only basic types like int, float, string, bool, or None, or if the container is empty, then the returned device will be None. If the container contains unrecognized objects, an error will be raised.

Parameters:

Name	Type	Description	Default
container	Any	A sequence or a dictionary of objects from which the device information will be extracted.	required
visited	Optional[dict]	Optionally a dictionary which stores the (sub)containers which are already visited. In most cases, when this function is called from outside, this is expected as None.	None
visiting	Optional[str]	Optionally a set which stores the (sub)containers which are being visited. This set is used to prevent recursion errors while handling circular references. In most cases, when this function is called from outside, this argument is expected as None.	None

Source code in evotorch/tools/misc.py

def device_of_container(
    container: Any, *, visited: Optional[dict] = None, visiting: Optional[str] = None
) -> Optional[torch.device]:
    """
    Get the device of the given container.

    It is assumed that the given container stores PyTorch tensors from
    which the device information will be extracted.
    If the container contains only basic types like int, float, string,
    bool, or None, or if the container is empty, then the returned device
    will be None.
    If the container contains unrecognized objects, an error will be
    raised.

    Args:
        container: A sequence or a dictionary of objects from which the
            device information will be extracted.
        visited: Optionally a dictionary which stores the (sub)containers
            which are already visited. In most cases, when this function
            is called from outside, this is expected as None.
        visiting: Optionally a set which stores the (sub)containers
            which are being visited. This set is used to prevent recursion
            errors while handling circular references. In most cases,
            when this function is called from outside, this argument is
            expected as None.
    Returns:
        The device if available, None otherwise.
    """
    container_id = id(container)

    if visited is None:
        visited = {}

    if container_id in visited:
        return visited[container_id]

    if visiting is None:
        visiting = set()

    if container_id in visiting:
        return None

    class result:
        device: Optional[torch.device] = None

        @classmethod
        def update(cls, new_device: Optional[torch.device]):
            if new_device is not None:
                if cls.device is None:
                    cls.device = new_device
                else:
                    if new_device != cls.device:
                        raise ValueError(f"Encountered tensors whose `device`s mismatch: {new_device}, {cls.device}")

    def call_self(sub_container):
        return device_of_container(sub_container, visited=visited, visiting=visiting)

    if isinstance(container, torch.Tensor):
        result.update(container.device)
    elif (container is None) or isinstance(container, (Number, str, bytes, bool)):
        pass
    elif isinstance(container, Mapping):
        visiting.add(container_id)
        try:
            for _, v in container.items():
                result.update(call_self(v))
        finally:
            visiting.remove(container_id)
    elif isinstance(container, Iterable):
        visiting.add(container_id)
        try:
            for v in container:
                result.update(call_self(v))
        finally:
            visiting.remove(container_id)
    else:
        raise TypeError(f"Encountered an object of unrecognized type: {type(container)}")

    visited[container_id] = result.device
    return result.device

dtype_of(x)

Get the dtype of the given object.

Parameters:

Name	Type	Description	Default
x	Any	The object whose dtype is being queried. The object can be a PyTorch tensor, or a PyTorch module (in which case the dtype of the first parameter tensor will be returned), or an ObjectArray (in which case the returned dtype will be object), or any object with the attribute dtype.	required

Source code in evotorch/tools/misc.py

def dtype_of(x: Any) -> DType:
    """
    Get the dtype of the given object.

    Args:
        x: The object whose dtype is being queried.
            The object can be a PyTorch tensor, or a PyTorch module
            (in which case the dtype of the first parameter tensor
            will be returned), or an ObjectArray (in which case
            the returned dtype will be `object`), or any object with
            the attribute `dtype`.
    Returns:
        The dtype of the given object.
    """
    if isinstance(x, nn.Module):
        result = None
        for param in x.parameters():
            result = param.dtype
            break
        if result is None:
            raise ValueError(f"Cannot determine the dtype of the module {x}")
        return result
    else:
        return x.dtype

dtype_of_container(container, *, visited=None, visiting=None)

Get the dtype of the given container.

It is assumed that the given container stores PyTorch tensors from which the dtype information will be extracted. If the container contains only basic types like int, float, string, bool, or None, or if the container is empty, then the returned dtype will be None. If the container contains unrecognized objects, an error will be raised.

Parameters:

Name	Type	Description	Default
container	Any	A sequence or a dictionary of objects from which the dtype information will be extracted.	required
visited	Optional[dict]	Optionally a dictionary which stores the (sub)containers which are already visited. In most cases, when this function is called from outside, this is expected as None.	None
visiting	Optional[str]	Optionally a set which stores the (sub)containers which are being visited. This set is used to prevent recursion errors while handling circular references. In most cases, when this function is called from outside, this argument is expected as None.	None

Source code in evotorch/tools/misc.py

def dtype_of_container(
    container: Any, *, visited: Optional[dict] = None, visiting: Optional[str] = None
) -> Optional[torch.dtype]:
    """
    Get the dtype of the given container.

    It is assumed that the given container stores PyTorch tensors from
    which the dtype information will be extracted.
    If the container contains only basic types like int, float, string,
    bool, or None, or if the container is empty, then the returned dtype
    will be None.
    If the container contains unrecognized objects, an error will be
    raised.

    Args:
        container: A sequence or a dictionary of objects from which the
            dtype information will be extracted.
        visited: Optionally a dictionary which stores the (sub)containers
            which are already visited. In most cases, when this function
            is called from outside, this is expected as None.
        visiting: Optionally a set which stores the (sub)containers
            which are being visited. This set is used to prevent recursion
            errors while handling circular references. In most cases,
            when this function is called from outside, this argument is
            expected as None.
    Returns:
        The dtype if available, None otherwise.
    """

    container_id = id(container)

    if visited is None:
        visited = {}

    if container_id in visited:
        return visited[container_id]

    if visiting is None:
        visiting = set()

    if container_id in visiting:
        return None

    class result:
        dtype: Optional[torch.dtype] = None

        @classmethod
        def update(cls, new_dtype: Optional[torch.dtype]):
            if new_dtype is not None:
                if cls.dtype is None:
                    cls.dtype = new_dtype
                else:
                    if new_dtype != cls.dtype:
                        raise ValueError(f"Encountered tensors whose `dtype`s mismatch: {new_dtype}, {cls.dtype}")

    def call_self(sub_container):
        return dtype_of_container(sub_container, visited=visited, visiting=visiting)

    if isinstance(container, torch.Tensor):
        result.update(container.dtype)
    elif (container is None) or isinstance(container, (Number, str, bytes, bool)):
        pass
    elif isinstance(container, Mapping):
        visiting.add(container_id)
        try:
            for _, v in container.items():
                result.update(call_self(v))
        finally:
            visiting.remove(container_id)
    elif isinstance(container, Iterable):
        visiting.add(container_id)
        try:
            for v in container:
                result.update(call_self(v))
        finally:
            visiting.remove(container_id)
    else:
        raise TypeError(f"Encountered an object of unrecognized type: {type(container)}")

    visited[container_id] = result.dtype
    return result.dtype

empty_tensor_like(source, *, shape=None, length=None, dtype=None, device=None)

Make an empty tensor with attributes taken from a source tensor.

The source tensor can be a PyTorch tensor, or an ObjectArray.

Unlike torch.empty_like(...), this function allows one to redefine the shape and/or length of the new empty tensor.

Parameters:

Name	Type	Description	Default
source	Any	The source tensor whose shape, dtype, and device will be used by default for the new empty tensor.	required
shape	Optional[Union[tuple, int]]	If given as None (which is the default), then the shape of the source tensor will be used for the new empty tensor. If given as a tuple or a torch.Size instance, then the new empty tensor will be in this given shape instead. This argument cannot be used together with length.	None
length	Optional[int]	If given as None (which is the default), then the length of the new empty tensor will be equal to the length of the source tensor (where length here means the size of the outermost dimension, i.e., what is returned by len(...)). If given as an integer, the length of the empty tensor will be this given length instead. This argument cannot be used together with shape.	None
dtype	Optional[DType]	If given as None, the dtype of the new empty tensor will be the dtype of the source tensor. If given as a torch.dtype instance, then the dtype of the tensor will be this given dtype instead.	None
device	Optional[Device]	If given as None, the device of the new empty tensor will be the device of the source tensor. If given as a torch.device instance, then the device of the tensor will be this given device instead.	None

Source code in evotorch/tools/misc.py

def empty_tensor_like(
    source: Any,
    *,
    shape: Optional[Union[tuple, int]] = None,
    length: Optional[int] = None,
    dtype: Optional[DType] = None,
    device: Optional[Device] = None,
) -> Any:
    """
    Make an empty tensor with attributes taken from a source tensor.

    The source tensor can be a PyTorch tensor, or an ObjectArray.

    Unlike `torch.empty_like(...)`, this function allows one to redefine the
    shape and/or length of the new empty tensor.

    Args:
        source: The source tensor whose shape, dtype, and device will be used
            by default for the new empty tensor.
        shape: If given as None (which is the default), then the shape of the
            source tensor will be used for the new empty tensor.
            If given as a tuple or a `torch.Size` instance, then the new empty
            tensor will be in this given shape instead.
            This argument cannot be used together with `length`.
        length: If given as None (which is the default), then the length of
            the new empty tensor will be equal to the length of the source
            tensor (where length here means the size of the outermost
            dimension, i.e., what is returned by `len(...)`).
            If given as an integer, the length of the empty tensor will be
            this given length instead.
            This argument cannot be used together with `shape`.
        dtype: If given as None, the dtype of the new empty tensor will be
            the dtype of the source tensor.
            If given as a `torch.dtype` instance, then the dtype of the
            tensor will be this given dtype instead.
        device: If given as None, the device of the new empty tensor will be
            the device of the source tensor.
            If given as a `torch.device` instance, then the device of the
            tensor will be this given device instead.
    Returns:
        The new empty tensor.
    """
    from .objectarray import ObjectArray

    if isinstance(source, ObjectArray):
        if length is not None and shape is None:
            n = int(length)
        elif shape is not None and length is None:
            if isinstance(shape, Iterable):
                if len(shape) != 1:
                    raise ValueError(
                        f"An ObjectArray must always be 1-dimensional."
                        f" Therefore, this given shape is incompatible: {shape}"
                    )
                n = int(shape[0])
        elif length is None and shape is None:
            n = len(source)
        else:
            raise ValueError("`length` and `shape` cannot be used together")

        if device is not None:
            if str(device) != "cpu":
                raise ValueError(
                    f"An ObjectArray can only be allocated on cpu. However, the specified `device` is: {device}."
                )

        if dtype is not None:
            if not is_dtype_object(dtype):
                raise ValueError(
                    f"The dtype of an ObjectArray can only be `object`. However, the specified `dtype` is: {dtype}."
                )

        return ObjectArray(n)
    elif isinstance(source, torch.Tensor):
        if length is not None:
            if shape is not None:
                raise ValueError("`length` and `shape` cannot be used together")
            if source.ndim == 0:
                raise ValueError("`length` can only be used when the source tensor is at least 1-dimensional")
            newshape = [int(length)]
            newshape.extend(source.shape[1:])
            shape = tuple(newshape)

        if not ((dtype is None) or isinstance(dtype, torch.dtype)):
            dtype = to_torch_dtype(dtype)

        return torch.empty(
            source.shape if shape is None else shape,
            dtype=(source.dtype if dtype is None else dtype),
            device=(source.device if device is None else device),
        )
    else:
        raise TypeError(f"The source tensor is of an unrecognized type: {type(source)}")

ensure_ray()

Ensure that the ray parallelization engine is initialized. If ray is already initialized, this function does nothing.

Source code in evotorch/tools/misc.py

def ensure_ray():
    """
    Ensure that the ray parallelization engine is initialized.
    If ray is already initialized, this function does nothing.
    """
    import ray

    if not ray.is_initialized():
        ray.init()

ensure_tensor_length_and_dtype(t, length, dtype, about=None, *, allow_scalar=False, device=None)

Return the given sequence as a tensor while also confirming its length, dtype, and device. If the given object is already a tensor conforming to the desired length, dtype, and device, the object will be returned as it is (there will be no copying).

Parameters:

Name	Type	Description	Default
t	Any	The tensor, or a sequence which is convertible to a tensor.	required
length	int	The length to which the tensor is expected to conform.	required
dtype	DType	The dtype to which the tensor is expected to conform.	required
about	Optional[str]	The prefix for the error message. Can be left as None.	None
allow_scalar	bool	Whether or not to accept scalars in addition to vector of the desired length. If allow_scalar is False, then scalars will be converted to sequences of the desired length. The sequence will contain the same scalar, repeated. If allow_scalar is True, then the scalar itself will be converted to a PyTorch scalar, and then will be returned.	False
device	Optional[Device]	The device in which the sequence is to be stored. If the given sequence is on a different device than the desired device, a copy on the correct device will be made. If device is None, the default behavior of torch.tensor(...) will be used, that is: if t is already a tensor, the result will be on the same device, otherwise, the result will be on the cpu.	None

Source code in evotorch/tools/misc.py

@torch.no_grad()
def ensure_tensor_length_and_dtype(
    t: Any,
    length: int,
    dtype: DType,
    about: Optional[str] = None,
    *,
    allow_scalar: bool = False,
    device: Optional[Device] = None,
) -> Any:
    """
    Return the given sequence as a tensor while also confirming its
    length, dtype, and device.
    If the given object is already a tensor conforming to the desired
    length, dtype, and device, the object will be returned as it is
    (there will be no copying).

    Args:
        t: The tensor, or a sequence which is convertible to a tensor.
        length: The length to which the tensor is expected to conform.
        dtype: The dtype to which the tensor is expected to conform.
        about: The prefix for the error message. Can be left as None.
        allow_scalar: Whether or not to accept scalars in addition
            to vector of the desired length.
            If `allow_scalar` is False, then scalars will be converted
            to sequences of the desired length. The sequence will contain
            the same scalar, repeated.
            If `allow_scalar` is True, then the scalar itself will be
            converted to a PyTorch scalar, and then will be returned.
        device: The device in which the sequence is to be stored.
            If the given sequence is on a different device than the
            desired device, a copy on the correct device will be made.
            If device is None, the default behavior of `torch.tensor(...)`
            will be used, that is: if `t` is already a tensor, the result
            will be on the same device, otherwise, the result will be on
            the cpu.
    Returns:
        The sequence whose correctness in terms of length, dtype, and
        device is ensured.
    Raises:
        ValueError: if there is a length mismatch.
    """
    device_args = {}
    if device is not None:
        device_args["device"] = device

    t = as_tensor(t, dtype=dtype, **device_args)

    if t.ndim == 0:
        if allow_scalar:
            return t
        else:
            return t.repeat(length)
    else:
        if t.ndim != 1 or len(t) != length:
            if about is not None:
                err_prefix = about + ": "
            else:
                err_prefix = ""
            raise ValueError(
                f"{err_prefix}Expected a 1-dimensional tensor of length {length}, but got a tensor with shape: {t.shape}"
            )
        return t

expect_none(msg_prefix, **kwargs)

Expect the values associated with the given keyword arguments to be None. If not, raise error.

Parameters:

Name	Type	Description	Default
msg_prefix	str	Prefix of the error message.	required
kwargs		Keyword arguments whose values are expected to be None.	{}

Source code in evotorch/tools/misc.py

def expect_none(msg_prefix: str, **kwargs):
    """
    Expect the values associated with the given keyword arguments
    to be None. If not, raise error.

    Args:
        msg_prefix: Prefix of the error message.
        kwargs: Keyword arguments whose values are expected to be None.
    Raises:
        ValueError: if at least one of the keyword arguments has a value
            other than None.
    """
    for k, v in kwargs.items():
        if v is not None:
            raise ValueError(f"{msg_prefix}: expected `{k}` as None, however, it was found to be {repr(v)}")

is_bool(x)

Return True if x represents a bool.

Parameters:

Name	Type	Description	Default
x	Any	An object whose type is being queried.	required

Source code in evotorch/tools/misc.py

def is_bool(x: Any) -> bool:
    """
    Return True if `x` represents a bool.

    Args:
        x: An object whose type is being queried.
    Returns:
        True if `x` is a bool; False otherwise.
    """
    if isinstance(x, (bool, np.bool_)):
        return True
    elif isinstance(x, (torch.Tensor, np.ndarray)):
        if x.ndim > 0:
            return False
        else:
            return is_dtype_bool(x.dtype)
    else:
        return False

is_bool_vector(x)

Return True if x is a vector consisting of bools.

Parameters:

Name	Type	Description	Default
x	Any	An object whose elements' types are to be queried.	required

Source code in evotorch/tools/misc.py

def is_bool_vector(x: Any) -> bool:
    """
    Return True if `x` is a vector consisting of bools.

    Args:
        x: An object whose elements' types are to be queried.
    Returns:
        True if the elements of `x` are bools; False otherwise.
    """
    if isinstance(x, (torch.Tensor, np.ndarray)):
        if x.ndim != 1:
            return False
        else:
            return is_dtype_bool(x.dtype)
    elif isinstance(x, Iterable):
        for item in x:
            if not is_bool(item):
                return False
        return True
    else:
        return False

is_dtype_bool(t)

Return True if the given dtype is an bool type.

Parameters:

Name	Type	Description	Default
t	DType	The dtype, which can be a dtype string, a numpy dtype, or a PyTorch dtype.	required

Source code in evotorch/tools/misc.py

def is_dtype_bool(t: DType) -> bool:
    """
    Return True if the given dtype is an bool type.

    Args:
        t: The dtype, which can be a dtype string, a numpy dtype,
            or a PyTorch dtype.
    Returns:
        True if t is a bool type; False otherwise.
    """
    t: np.dtype = to_numpy_dtype(t)
    return t.kind.startswith("b")

is_dtype_float(t)

Return True if the given dtype is an float type.

Parameters:

Name	Type	Description	Default
t	DType	The dtype, which can be a dtype string, a numpy dtype, or a PyTorch dtype.	required

Source code in evotorch/tools/misc.py

def is_dtype_float(t: DType) -> bool:
    """
    Return True if the given dtype is an float type.

    Args:
        t: The dtype, which can be a dtype string, a numpy dtype,
            or a PyTorch dtype.
    Returns:
        True if t is an float type; False otherwise.
    """
    t: np.dtype = to_numpy_dtype(t)
    return t.kind.startswith("f")

is_dtype_integer(t)

Return True if the given dtype is an integer type.

Parameters:

Name	Type	Description	Default
t	DType	The dtype, which can be a dtype string, a numpy dtype, or a PyTorch dtype.	required

Source code in evotorch/tools/misc.py

def is_dtype_integer(t: DType) -> bool:
    """
    Return True if the given dtype is an integer type.

    Args:
        t: The dtype, which can be a dtype string, a numpy dtype,
            or a PyTorch dtype.
    Returns:
        True if t is an integer type; False otherwise.
    """
    t: np.dtype = to_numpy_dtype(t)
    return t.kind.startswith("u") or t.kind.startswith("i")

is_dtype_object(dtype)

Return True if the given dtype is object or Any.

Returns:

Type	Description
bool	True if the given dtype is object or Any; False otherwise.

Source code in evotorch/tools/misc.py

def is_dtype_object(dtype: DType) -> bool:
    """
    Return True if the given dtype is `object` or `Any`.

    Returns:
        True if the given dtype is `object` or `Any`; False otherwise.
    """
    if isinstance(dtype, str):
        return dtype in ("object", "Any", "O")
    elif dtype is object or dtype is Any:
        return True
    else:
        return False

is_dtype_real(t)

Return True if the given dtype represents real numbers (i.e. if dtype is an integer type or is a float type).

Parameters:

Name	Type	Description	Default
t	DType	The dtype, which can be a dtype string, a numpy dtype, or a PyTorch dtype.	required

Source code in evotorch/tools/misc.py

def is_dtype_real(t: DType) -> bool:
    """
    Return True if the given dtype represents real numbers
    (i.e. if dtype is an integer type or is a float type).

    Args:
        t: The dtype, which can be a dtype string, a numpy dtype,
            or a PyTorch dtype.
    Returns:
        True if t represents a real numbers type; False otherwise.
    """
    return is_dtype_float(t) or is_dtype_integer(t)

is_integer(x)

Return True if x is an integer.

Note that this function does NOT consider booleans as integers.

Parameters:

Name	Type	Description	Default
x	Any	An object whose type is being queried.	required

Source code in evotorch/tools/misc.py

def is_integer(x: Any) -> bool:
    """
    Return True if `x` is an integer.

    Note that this function does NOT consider booleans as integers.

    Args:
        x: An object whose type is being queried.
    Returns:
        True if `x` is an integer; False otherwise.
    """
    if is_bool(x):
        return False
    elif isinstance(x, Integral):
        return True
    elif isinstance(x, (torch.Tensor, np.ndarray)):
        if x.ndim > 0:
            return False
        else:
            return is_dtype_integer(x.dtype)
    else:
        return False

is_integer_vector(x)

Return True if x is a vector consisting of integers.

Parameters:

Name	Type	Description	Default
x	Any	An object whose elements' types are to be queried.	required

Source code in evotorch/tools/misc.py

def is_integer_vector(x: Any) -> bool:
    """
    Return True if `x` is a vector consisting of integers.

    Args:
        x: An object whose elements' types are to be queried.
    Returns:
        True if the elements of `x` are integers; False otherwise.
    """
    if isinstance(x, (torch.Tensor, np.ndarray)):
        if x.ndim != 1:
            return False
        else:
            return is_dtype_integer(x.dtype)
    elif isinstance(x, Iterable):
        for item in x:
            if not is_integer(item):
                return False
        return True
    else:
        return False

is_real(x)

Return True if x is a real number.

Note that this function does NOT consider booleans as real numbers.

Parameters:

Name	Type	Description	Default
x	Any	An object whose type is being queried.	required

Source code in evotorch/tools/misc.py

def is_real(x: Any) -> bool:
    """
    Return True if `x` is a real number.

    Note that this function does NOT consider booleans as real numbers.

    Args:
        x: An object whose type is being queried.
    Returns:
        True if `x` is a real number; False otherwise.
    """
    if is_bool(x):
        return False
    elif isinstance(x, Real):
        return True
    elif isinstance(x, (torch.Tensor, np.ndarray)):
        if x.ndim > 0:
            return False
        else:
            return is_dtype_real(x.dtype)
    else:
        return False

is_real_vector(x)

Return True if x is a vector consisting of real numbers.

Parameters:

Name	Type	Description	Default
x	Any	An object whose elements' types are to be queried.	required

Source code in evotorch/tools/misc.py

def is_real_vector(x: Any) -> bool:
    """
    Return True if `x` is a vector consisting of real numbers.

    Args:
        x: An object whose elements' types are to be queried.
    Returns:
        True if the elements of `x` are real numbers; False otherwise.
    """
    if isinstance(x, (torch.Tensor, np.ndarray)):
        if x.ndim != 1:
            return False
        else:
            return is_dtype_real(x.dtype)
    elif isinstance(x, Iterable):
        for item in x:
            if not is_real(item):
                return False
        return True
    else:
        return False

is_sequence(x)

Return True if x is a sequence. Note that this function considers str and bytes as scalars, not as sequences.

Parameters:

Name	Type	Description	Default
x	Any	The object whose sequential nature is being queried.	required

Source code in evotorch/tools/misc.py

def is_sequence(x: Any) -> bool:
    """
    Return True if `x` is a sequence.
    Note that this function considers `str` and `bytes` as scalars,
    not as sequences.

    Args:
        x: The object whose sequential nature is being queried.
    Returns:
        True if `x` is a sequence; False otherwise.
    """
    if isinstance(x, (str, bytes)):
        return False
    elif isinstance(x, (np.ndarray, torch.Tensor)):
        return x.ndim > 0
    elif isinstance(x, Iterable):
        return True
    else:
        return False

is_tensor_on_cpu(tensor)

Return True of the given PyTorch tensor or ObjectArray is on cpu.

Source code in evotorch/tools/misc.py

def is_tensor_on_cpu(tensor) -> bool:
    """
    Return True of the given PyTorch tensor or ObjectArray is on cpu.
    """
    return str(tensor.device) == "cpu"

log_barrier(lhs, comparison, rhs, *, penalty_sign, sharpness=1.0, inf=None)

Return a penalty based on how close the constraint is to being violated.

If the left-hand-side is equal to the right-hand-side, or if the constraint is violated, the returned penalty will be infinite (+inf or -inf, depending on penalty_sign). Such inf values can result in numerical instabilities. To overcome such instabilities, you might want to set the keyword argument inf as a large-enough finite positive quantity M, so that very large (or infinite) penalties will be clipped down to M.

Parameters:

Name	Type	Description	Default
lhs	Union[float, Tensor]	The left-hand-side of the constraint. In the non-batched case, this is expected as a scalar. If it is given as an n-dimensional tensor where n is at least 1, this is considered as a batch of left-hand-side values.	required
comparison	str	The operator used for comparing the left-hand-side and the right-hand-side. Expected as a string. Acceptable values are: '<=', '>='.	required
rhs	Union[float, Tensor]	The right-hand-side of the constraint. In the non-batched case, this is expected as a scalar. If it is given as an n-dimensional tensor where n is at least 1, this is considered as a batch of right-hand-side values.	required
penalty_sign	str	Expected as string, either as '+' or '-', which determines the sign of the penalty (i.e. determines if the penalty will be positive or negative). One should consider the objective sense of the fitness function at hand for deciding penalty_sign. For example, if a fitness function is written from the perspective of maximization, the penalties should be negative, and therefore, penalty_sign must be given as '-'.	required
sharpness	Union[float, Tensor]	The logarithmic penalty will be divided by this number. By default, this value is 1. A sharper log-penalization allows the constraint to get closer to its boundary, and then makes a more sudden jump towards infinity.	1.0
inf	Optional[Union[float, Tensor]]	When concerned about the possible numerical instabilities caused by infinite penalties, one can specify a finite large-enough positive quantity M through this argument. As a result, infinite penalties will be clipped down to the finite M. One might also think of this as temporarily replacing inf with M while computing the log-penalties.	None

Source code in evotorch/tools/constraints.py

def log_barrier(
    lhs: Union[float, torch.Tensor],
    comparison: str,
    rhs: Union[float, torch.Tensor],
    *,
    penalty_sign: str,
    sharpness: Union[float, torch.Tensor] = 1.0,
    inf: Optional[Union[float, torch.Tensor]] = None,
) -> torch.Tensor:
    """
    Return a penalty based on how close the constraint is to being violated.

    If the left-hand-side is equal to the right-hand-side, or if the constraint
    is violated, the returned penalty will be infinite (`+inf` or `-inf`,
    depending on `penalty_sign`). Such `inf` values can result in numerical
    instabilities. To overcome such instabilities, you might want to set the
    keyword argument `inf` as a large-enough finite positive quantity `M`, so
    that very large (or infinite) penalties will be clipped down to `M`.

    Args:
        lhs: The left-hand-side of the constraint. In the non-batched case,
            this is expected as a scalar. If it is given as an n-dimensional
            tensor where n is at least 1, this is considered as a batch of
            left-hand-side values.
        comparison: The operator used for comparing the left-hand-side and the
            right-hand-side. Expected as a string. Acceptable values are:
            '<=', '>='.
        rhs: The right-hand-side of the constraint. In the non-batched case,
            this is expected as a scalar. If it is given as an n-dimensional
            tensor where n is at least 1, this is considered as a batch of
            right-hand-side values.
        penalty_sign: Expected as string, either as '+' or '-', which
            determines the sign of the penalty (i.e. determines if the penalty
            will be positive or negative). One should consider the objective
            sense of the fitness function at hand for deciding `penalty_sign`.
            For example, if a fitness function is written from the perspective
            of maximization, the penalties should be negative, and therefore,
            `penalty_sign` must be given as '-'.
        sharpness: The logarithmic penalty will be divided by this number.
            By default, this value is 1. A sharper log-penalization allows
            the constraint to get closer to its boundary, and then makes
            a more sudden jump towards infinity.
        inf: When concerned about the possible numerical instabilities caused
            by infinite penalties, one can specify a finite large-enough
            positive quantity `M` through this argument. As a result,
            infinite penalties will be clipped down to the finite `M`.
            One might also think of this as temporarily replacing `inf` with
            `M` while computing the log-penalties.
    Returns:
        Log-penalty amount(s), whose sign(s) is/are determined by
        `penalty_sign`.
    """
    from ..decorators import expects_ndim

    if inf is None:
        inf = float("inf")

    return expects_ndim(_log_barrier, (0, None, 0, 0, None, 0))(lhs, comparison, rhs, sharpness, penalty_sign, inf)

make_I(size=None, *, out=None, dtype=None, device=None)

Make a new identity matrix (I), or change an existing tensor into one.

The following example creates a 3x3 identity matrix:

identity_matrix = make_I(3, dtype="float32")

The following example changes an already existing square matrix such that its values will store an identity matrix:

make_I(out=existing_tensor)

Parameters:

Name	Type	Description	Default
size	Optional[int]	A single integer or a tuple containing a single integer, where the integer specifies the length of the target square matrix. In this context, "length" means both rowwise length and columnwise length, since the target is a square matrix. Note that, if the user wishes to fill an existing tensor with identity values, then size is expected to be left as None.	None
out	Optional[Tensor]	Optionally, the existing tensor whose values will be changed so that they represent an identity matrix. If an out tensor is given, then size is expected as None.	None
dtype	Optional[DType]	Optionally a string (e.g. "float32") or a PyTorch dtype (e.g. torch.float32). If dtype is not specified, the default choice of torch.empty(...) is used, that is, torch.float32. If an out tensor is specified, then dtype is expected as None.	None
device	Optional[Device]	The device in which the new tensor will be stored. If not specified, "cpu" will be used. If an out tensor is specified, then device is expected as None.	None

Source code in evotorch/tools/misc.py

def make_I(
    size: Optional[int] = None,
    *,
    out: Optional[torch.Tensor] = None,
    dtype: Optional[DType] = None,
    device: Optional[Device] = None,
) -> torch.Tensor:
    """
    Make a new identity matrix (I), or change an existing tensor into one.

    The following example creates a 3x3 identity matrix:

        identity_matrix = make_I(3, dtype="float32")

    The following example changes an already existing square matrix such that
    its values will store an identity matrix:

        make_I(out=existing_tensor)

    Args:
        size: A single integer or a tuple containing a single integer,
            where the integer specifies the length of the target square
            matrix. In this context, "length" means both rowwise length
            and columnwise length, since the target is a square matrix.
            Note that, if the user wishes to fill an existing tensor with
            identity values, then `size` is expected to be left as None.
        out: Optionally, the existing tensor whose values will be changed
            so that they represent an identity matrix.
            If an `out` tensor is given, then `size` is expected as None.
        dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
            (e.g. torch.float32).
            If `dtype` is not specified, the default choice of
            `torch.empty(...)` is used, that is, `torch.float32`.
            If an `out` tensor is specified, then `dtype` is expected
            as None.
        device: The device in which the new tensor will be stored.
            If not specified, "cpu" will be used.
            If an `out` tensor is specified, then `device` is expected
            as None.
    Returns:
        The created or modified tensor after placing the I matrix values
    """
    if size is None:
        if out is None:
            raise ValueError(
                "When the `size` argument is missing, the function `make_I(...)` expects an `out` tensor."
                " However, the `out` argument was received as None."
            )
        size = tuple()
    else:
        if isinstance(size, tuple):
            if len(size) == 1:
                size = size[0]
            else:
                raise ValueError(
                    f"When the `size` argument is given as a tuple,"
                    f" the function `make_I(...)` expects this tuple to contain exactly one element."
                    f" The received tuple is {size}."
                )
        n = int(size)
        size = (n, n)
    out = _out_tensor(*size, out=out, dtype=dtype, device=device)
    out.zero_()
    out.fill_diagonal_(1)
    return out

make_batched_false_for_vmap(device)

Get False, properly batched if inside vmap(..., randomness='different').

Reasoning. Imagine we have the following function:

import torch


def sample_and_shift(target_shape: tuple, shift: torch.Tensor) -> torch.Tensor:
    result = torch.empty(target_shape, device=x.device)
    result.normal_()
    result += shift
    return result

which allocates an empty tensor, then fills it with samples from the standard normal distribution, then shifts the samples and returns the result. An important implementation detail regarding this example function is that all of its operations are in-place (i.e. the method normal_() and the operator += work on the given pre-allocated tensor).

Let us now imagine that we have a batch of shift tensors, and we would like to generate multiple shifted sample tensors. Ideally, such a batched operation could be done by transforming the example function with the help of vmap:

from torch.func import vmap

batched_sample_and_shift = vmap(sample_and_shift, in_dims=0, randomness="different")

where the argument randomness="different" tells PyTorch that for each batch item, we want to generate different samples (instead of just duplicating the same samples across the batch dimension(s)). Such a re-sampling approach is usually desired in applications where preserving stochasticity is crucial, evolutionary computation being one of such case.

Now let us call our transformed function:

batch_of_shifts = ...  # a tensor like `shift`, but with an extra leftmost
# dimension for the batches

# Will fail:
batched_results = batched_sample_and_shift(shape_goes_here, batch_of_shifts)

At this point, we observe that batched_sample_and_shift fails. The reason for this failure is that the function first allocates an empty tensor, then tries to perform random sampling in an in-place manner. The first allocation via empty is not properly batched (it is not aware of the active vmap), so, when we later call .normal_() on it, there is no room for the data that would be re-sampled for each batch item. To remedy this, we could modify our original function slightly:

import torch


def sample_and_shift2(target_shape: tuple, shift: torch.Tensor) -> torch.Tensor:
    result = torch.empty(target_shape, device=x.device)
    result = result + result.make_batched_false_for_vmap(x.device)
    result.normal_()
    result += shift
    return result

In this modified function, right after making an initial allocation, we add onto it a batched false, and re-assign the result to the variable result. Thanks to being the result of an interaction with a batched false, the new result variable is now properly batched (if we are inside vmap(..., randomness="different"). Now, let us transform our function:

from torch.func import vmap

batched_sample_and_shift2 = vmap(sample_and_shift2, in_dims=0, randomness="different")

The following code should now work:

batch_of_shifts = ...  # a tensor like `shift`, but with an extra leftmost
# dimension for the batches

# Should work:
batched_results = batched_sample_and_shift2(shape_goes_here, batch_of_shifts)

Parameters:

Name	Type	Description	Default
device	Device	The target device on which the batched False will be created	required

Source code in evotorch/tools/misc.py

def make_batched_false_for_vmap(device: Device) -> torch.Tensor:
    """
    Get `False`, properly batched if inside `vmap(..., randomness='different')`.

    **Reasoning.**
    Imagine we have the following function:

    ```python
    import torch


    def sample_and_shift(target_shape: tuple, shift: torch.Tensor) -> torch.Tensor:
        result = torch.empty(target_shape, device=x.device)
        result.normal_()
        result += shift
        return result
    ```

    which allocates an empty tensor, then fills it with samples from the
    standard normal distribution, then shifts the samples and returns the
    result. An important implementation detail regarding this example function
    is that all of its operations are in-place (i.e. the method `normal_()`
    and the operator `+=` work on the given pre-allocated tensor).

    Let us now imagine that we have a batch of shift tensors, and we would like
    to generate multiple shifted sample tensors. Ideally, such a batched
    operation could be done by transforming the example function with the help
    of `vmap`:

    ```python
    from torch.func import vmap

    batched_sample_and_shift = vmap(sample_and_shift, in_dims=0, randomness="different")
    ```

    where the argument `randomness="different"` tells PyTorch that for each
    batch item, we want to generate different samples (instead of just
    duplicating the same samples across the batch dimension(s)).
    Such a re-sampling approach is usually desired in applications where
    preserving stochasticity is crucial, evolutionary computation being one
    of such case.

    Now let us call our transformed function:

    ```python
    batch_of_shifts = ...  # a tensor like `shift`, but with an extra leftmost
    # dimension for the batches

    # Will fail:
    batched_results = batched_sample_and_shift(shape_goes_here, batch_of_shifts)
    ```

    At this point, we observe that `batched_sample_and_shift` fails.
    The reason for this failure is that the function first allocates an empty
    tensor, then tries to perform random sampling in an in-place manner.
    The first allocation via `empty` is not properly batched (it is not aware
    of the active `vmap`), so, when we later call `.normal_()` on it,
    there is no room for the data that would be re-sampled for each batch item.
    To remedy this, we could modify our original function slightly:

    ```python
    import torch


    def sample_and_shift2(target_shape: tuple, shift: torch.Tensor) -> torch.Tensor:
        result = torch.empty(target_shape, device=x.device)
        result = result + result.make_batched_false_for_vmap(x.device)
        result.normal_()
        result += shift
        return result
    ```

    In this modified function, right after making an initial allocation, we add
    onto it a batched false, and re-assign the result to the variable `result`.
    Thanks to being the result of an interaction with a batched false, the new
    `result` variable is now properly batched (if we are inside
    `vmap(..., randomness="different")`. Now, let us transform our function:

    ```python
    from torch.func import vmap

    batched_sample_and_shift2 = vmap(sample_and_shift2, in_dims=0, randomness="different")
    ```

    The following code should now work:

    ```python
    batch_of_shifts = ...  # a tensor like `shift`, but with an extra leftmost
    # dimension for the batches

    # Should work:
    batched_results = batched_sample_and_shift2(shape_goes_here, batch_of_shifts)
    ```

    Args:
        device: The target device on which the batched `False` will be created
    Returns:
        A scalar tensor having the value `False`. This returned tensor will be
        a batch of scalar tensors (i.e. a `BatchedTensor`) if we are inside
        `vmap(..., randomness="different")`.
    """
    return torch.randint(0, 1, tuple(), dtype=torch.bool, device=device)

make_empty(*size, dtype=None, device=None)

Make an empty tensor.

Parameters:

Name	Type	Description	Default
size	Size	Shape of the empty tensor to be created. expected as multiple positional arguments of integers, or as a single positional argument containing a tuple of integers. Note that when the user wishes to create an ObjectArray (i.e. when dtype is given as object), then the size is expected as a single integer, or as a single-element tuple containing an integer (because ObjectArray can only be one-dimensional).	()
dtype	Optional[DType]	Optionally a string (e.g. "float32") or a PyTorch dtype (e.g. torch.float32) or, for creating an ObjectArray, "object" (as string) or object or Any. If dtype is not specified, the default choice of torch.empty(...) is used, that is, torch.float32.	None
device	Optional[Device]	The device in which the new empty tensor will be stored. If not specified, "cpu" will be used.	None

Source code in evotorch/tools/misc.py

def make_empty(
    *size: Size,
    dtype: Optional[DType] = None,
    device: Optional[Device] = None,
) -> Iterable:
    """
    Make an empty tensor.

    Args:
        size: Shape of the empty tensor to be created.
            expected as multiple positional arguments of integers,
            or as a single positional argument containing a tuple of
            integers.
            Note that when the user wishes to create an `ObjectArray`
            (i.e. when `dtype` is given as `object`), then the size
            is expected as a single integer, or as a single-element
            tuple containing an integer (because `ObjectArray` can only
            be one-dimensional).
        dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
            (e.g. torch.float32) or, for creating an `ObjectArray`,
            "object" (as string) or `object` or `Any`.
            If `dtype` is not specified, the default choice of
            `torch.empty(...)` is used, that is, `torch.float32`.
        device: The device in which the new empty tensor will be stored.
            If not specified, "cpu" will be used.
    Returns:
        The new empty tensor, which can be a PyTorch tensor or an
        `ObjectArray`.
    """
    from .objectarray import ObjectArray

    if (dtype is not None) and is_dtype_object(dtype):
        if (device is None) or (str(device) == "cpu"):
            if len(size) == 1:
                size = size[0]
            return ObjectArray(size)
        else:
            return ValueError(
                f"Invalid device for ObjectArray: {repr(device)}. Note: an ObjectArray can only be stored on 'cpu'."
            )
    else:
        kwargs = {}
        if dtype is not None:
            kwargs["dtype"] = to_torch_dtype(dtype)
        if device is not None:
            kwargs["device"] = device
        return torch.empty(*size, **kwargs)

make_gaussian(*size, center=None, stdev=None, symmetric=False, out=None, dtype=None, device=None, generator=None)

Make a new or existing tensor filled by Gaussian distributed values. This function can work only with float dtypes.

Parameters:

Name	Type	Description	Default
size	Size	Size of the new tensor to be filled with Gaussian distributed values. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor instead, then no positional argument is expected.	()
center	Optional[RealOrVector]	Center point (i.e. mean) of the Gaussian distribution. Can be a scalar, or a tensor. If not specified, the center point will be taken as 0. Note that, if one specifies center, then stdev is also expected to be explicitly specified.	None
stdev	Optional[RealOrVector]	Standard deviation for the Gaussian distributed values. Can be a scalar, or a tensor. If not specified, the standard deviation will be taken as 1. Note that, if one specifies stdev, then center is also expected to be explicitly specified.	None
symmetric	bool	Whether or not the values should be sampled in a symmetric (i.e. antithetic) manner. The default is False.	False
out	Optional[Tensor]	Optionally, the tensor to be filled by Gaussian distributed values. If an out tensor is given, then no size argument is expected.	None
dtype	Optional[DType]	Optionally a string (e.g. "float32") or a PyTorch dtype (e.g. torch.float32). If dtype is not specified, the default choice of torch.empty(...) is used, that is, torch.float32. If an out tensor is specified, then dtype is expected as None.	None
device	Optional[Device]	The device in which the new tensor will be stored. If not specified, "cpu" will be used. If an out tensor is specified, then device is expected as None.	None
generator	Any	Pseudo-random number generator to be used when sampling the values. Can be a torch.Generator, or an object with a generator attribute (such as Problem). If left as None, the global generator of PyTorch will be used.	None

Source code in evotorch/tools/misc.py

def make_gaussian(
    *size: Size,
    center: Optional[RealOrVector] = None,
    stdev: Optional[RealOrVector] = None,
    symmetric: bool = False,
    out: Optional[torch.Tensor] = None,
    dtype: Optional[DType] = None,
    device: Optional[Device] = None,
    generator: Any = None,
) -> torch.Tensor:
    """
    Make a new or existing tensor filled by Gaussian distributed values.
    This function can work only with float dtypes.

    Args:
        size: Size of the new tensor to be filled with Gaussian distributed
            values. This can be given as multiple positional arguments, each
            such positional argument being an integer, or as a single
            positional argument of a tuple, the tuple containing multiple
            integers. Note that, if the user wishes to fill an existing
            tensor instead, then no positional argument is expected.
        center: Center point (i.e. mean) of the Gaussian distribution.
            Can be a scalar, or a tensor.
            If not specified, the center point will be taken as 0.
            Note that, if one specifies `center`, then `stdev` is also
            expected to be explicitly specified.
        stdev: Standard deviation for the Gaussian distributed values.
            Can be a scalar, or a tensor.
            If not specified, the standard deviation will be taken as 1.
            Note that, if one specifies `stdev`, then `center` is also
            expected to be explicitly specified.
        symmetric: Whether or not the values should be sampled in a
            symmetric (i.e. antithetic) manner.
            The default is False.
        out: Optionally, the tensor to be filled by Gaussian distributed
            values. If an `out` tensor is given, then no `size` argument is
            expected.
        dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
            (e.g. torch.float32).
            If `dtype` is not specified, the default choice of
            `torch.empty(...)` is used, that is, `torch.float32`.
            If an `out` tensor is specified, then `dtype` is expected
            as None.
        device: The device in which the new tensor will be stored.
            If not specified, "cpu" will be used.
            If an `out` tensor is specified, then `device` is expected
            as None.
        generator: Pseudo-random number generator to be used when sampling
            the values. Can be a `torch.Generator`, or an object with
            a `generator` attribute (such as `Problem`).
            If left as None, the global generator of PyTorch will be used.
    Returns:
        The created or modified tensor after placing the Gaussian
        distributed values.
    """
    scalar_requested = _scalar_requested(*size)
    if scalar_requested:
        size = (1,)

    out = _out_tensor_for_random_operation(*size, out=out, dtype=dtype, device=device)
    gen_kwargs = _generator_kwargs(generator)

    if symmetric:
        leftmost_dim = out.shape[0]
        if (leftmost_dim % 2) != 0:
            raise ValueError(
                f"Symmetric sampling cannot be done if the leftmost dimension of the target tensor is odd."
                f" The shape of the target tensor is: {repr(out.shape)}."
            )
        out[0::2, ...].normal_(**gen_kwargs)
        out[1::2, ...] = out[0::2, ...]
        out[1::2, ...] *= -1
    else:
        out.normal_(**gen_kwargs)

    if (center is None) and (stdev is None):
        pass  # do nothing
    elif (center is not None) and (stdev is not None):
        stdev = torch.as_tensor(stdev, dtype=out.dtype, device=out.device)
        out *= stdev
        center = torch.as_tensor(center, dtype=out.dtype, device=out.device)
        out += center
    else:
        raise ValueError(
            f"Please either specify none of `stdev` and `center`, or both of them."
            f" Currently, `center` is {center}"
            f" and `stdev` is {stdev}."
        )

    if scalar_requested:
        out = out[0]

    return out

make_nan(*size, out=None, dtype=None, device=None)

Make a new tensor filled with NaN, or fill an existing tensor with NaN.

The following example creates a float32 tensor filled with NaN values, of shape (3, 5):

nan_values = make_nan(3, 5, dtype="float32")

The following example fills an existing tensor with NaNs.

make_nan(out=existing_tensor)

Parameters:

Name	Type	Description	Default
size	Size	Size of the new tensor to be filled with NaNs. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor with NaN values, then no positional argument is expected.	()
out	Optional[Tensor]	Optionally, the tensor to be filled by NaN values. If an out tensor is given, then no size argument is expected.	None
dtype	Optional[DType]	Optionally a string (e.g. "float32") or a PyTorch dtype (e.g. torch.float32). If dtype is not specified, the default choice of torch.empty(...) is used, that is, torch.float32. If an out tensor is specified, then dtype is expected as None.	None
device	Optional[Device]	The device in which the new tensor will be stored. If not specified, "cpu" will be used. If an out tensor is specified, then device is expected as None.	None

Source code in evotorch/tools/misc.py

def make_nan(
    *size: Size,
    out: Optional[torch.Tensor] = None,
    dtype: Optional[DType] = None,
    device: Optional[Device] = None,
) -> torch.Tensor:
    """
    Make a new tensor filled with NaN, or fill an existing tensor with NaN.

    The following example creates a float32 tensor filled with NaN values,
    of shape (3, 5):

        nan_values = make_nan(3, 5, dtype="float32")

    The following example fills an existing tensor with NaNs.

        make_nan(out=existing_tensor)

    Args:
        size: Size of the new tensor to be filled with NaNs.
            This can be given as multiple positional arguments, each such
            positional argument being an integer, or as a single positional
            argument of a tuple, the tuple containing multiple integers.
            Note that, if the user wishes to fill an existing tensor with
            NaN values, then no positional argument is expected.
        out: Optionally, the tensor to be filled by NaN values.
            If an `out` tensor is given, then no `size` argument is expected.
        dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
            (e.g. torch.float32).
            If `dtype` is not specified, the default choice of
            `torch.empty(...)` is used, that is, `torch.float32`.
            If an `out` tensor is specified, then `dtype` is expected
            as None.
        device: The device in which the new tensor will be stored.
            If not specified, "cpu" will be used.
            If an `out` tensor is specified, then `device` is expected
            as None.
    Returns:
        The created or modified tensor after placing NaN values.
    """
    if _scalar_requested(*size):
        return _scalar_tensor(float("nan"), out=out, dtype=dtype, device=device)
    else:
        out = _out_tensor(*size, out=out, dtype=dtype, device=device)
        out[:] = float("nan")
        return out

make_ones(*size, out=None, dtype=None, device=None)

Make a new tensor filled with 1, or fill an existing tensor with 1.

The following example creates a float32 tensor filled with 1 values, of shape (3, 5):

zero_values = make_ones(3, 5, dtype="float32")

The following example fills an existing tensor with 1s:

make_ones(out=existing_tensor)

Parameters:

Name	Type	Description	Default
size	Size	Size of the new tensor to be filled with 1. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor with 1 values, then no positional argument is expected.	()
out	Optional[Tensor]	Optionally, the tensor to be filled by 1 values. If an out tensor is given, then no size argument is expected.	None
dtype	Optional[DType]	Optionally a string (e.g. "float32") or a PyTorch dtype (e.g. torch.float32). If dtype is not specified, the default choice of torch.empty(...) is used, that is, torch.float32. If an out tensor is specified, then dtype is expected as None.	None
device	Optional[Device]	The device in which the new tensor will be stored. If not specified, "cpu" will be used. If an out tensor is specified, then device is expected as None.	None

Source code in evotorch/tools/misc.py

def make_ones(
    *size: Size,
    out: Optional[torch.Tensor] = None,
    dtype: Optional[DType] = None,
    device: Optional[Device] = None,
) -> torch.Tensor:
    """
    Make a new tensor filled with 1, or fill an existing tensor with 1.

    The following example creates a float32 tensor filled with 1 values,
    of shape (3, 5):

        zero_values = make_ones(3, 5, dtype="float32")

    The following example fills an existing tensor with 1s:

        make_ones(out=existing_tensor)

    Args:
        size: Size of the new tensor to be filled with 1.
            This can be given as multiple positional arguments, each such
            positional argument being an integer, or as a single positional
            argument of a tuple, the tuple containing multiple integers.
            Note that, if the user wishes to fill an existing tensor with
            1 values, then no positional argument is expected.
        out: Optionally, the tensor to be filled by 1 values.
            If an `out` tensor is given, then no `size` argument is expected.
        dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
            (e.g. torch.float32).
            If `dtype` is not specified, the default choice of
            `torch.empty(...)` is used, that is, `torch.float32`.
            If an `out` tensor is specified, then `dtype` is expected
            as None.
        device: The device in which the new tensor will be stored.
            If not specified, "cpu" will be used.
            If an `out` tensor is specified, then `device` is expected
            as None.
    Returns:
        The created or modified tensor after placing 1 values.
    """
    if _scalar_requested(*size):
        return _scalar_tensor(1, out=out, dtype=dtype, device=device)
    else:
        out = _out_tensor(*size, out=out, dtype=dtype, device=device)
        out[:] = 1
        return out

make_randint(*size, n, out=None, dtype=None, device=None, generator=None)

Make a new or existing tensor filled by random integers. The integers are uniformly distributed within [0 ... n-1]. This function can be used with integer or float dtypes.

Parameters:

Name	Type	Description	Default
size	Size	Size of the new tensor to be filled with uniformly distributed values. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor instead, then no positional argument is expected.	()
n	Union[int, float, Tensor]	Number of choice(s) for integer sampling. The lowest possible value will be 0, and the highest possible value will be n - 1. n can be a scalar, or a tensor.	required
out	Optional[Tensor]	Optionally, the tensor to be filled by the random integers. If an out tensor is given, then no size argument is expected.	None
dtype	Optional[DType]	Optionally a string (e.g. "int64") or a PyTorch dtype (e.g. torch.int64). If dtype is not specified, torch.int64 will be used.	None
device	Optional[Device]	The device in which the new tensor will be stored. If not specified, "cpu" will be used. If an out tensor is specified, then device is expected as None.	None
generator	Any	Pseudo-random number generator to be used when sampling the values. Can be a torch.Generator, or an object with a generator attribute (such as Problem). If left as None, the global generator of PyTorch will be used.	None

Source code in evotorch/tools/misc.py

def make_randint(
    *size: Size,
    n: Union[int, float, torch.Tensor],
    out: Optional[torch.Tensor] = None,
    dtype: Optional[DType] = None,
    device: Optional[Device] = None,
    generator: Any = None,
) -> torch.Tensor:
    """
    Make a new or existing tensor filled by random integers.
    The integers are uniformly distributed within `[0 ... n-1]`.
    This function can be used with integer or float dtypes.

    Args:
        size: Size of the new tensor to be filled with uniformly distributed
            values. This can be given as multiple positional arguments, each
            such positional argument being an integer, or as a single
            positional argument of a tuple, the tuple containing multiple
            integers. Note that, if the user wishes to fill an existing
            tensor instead, then no positional argument is expected.
        n: Number of choice(s) for integer sampling.
            The lowest possible value will be 0, and the highest possible
            value will be n - 1.
            `n` can be a scalar, or a tensor.
        out: Optionally, the tensor to be filled by the random integers.
            If an `out` tensor is given, then no `size` argument is
            expected.
        dtype: Optionally a string (e.g. "int64") or a PyTorch dtype
            (e.g. torch.int64).
            If `dtype` is not specified, torch.int64 will be used.
        device: The device in which the new tensor will be stored.
            If not specified, "cpu" will be used.
            If an `out` tensor is specified, then `device` is expected
            as None.
        generator: Pseudo-random number generator to be used when sampling
            the values. Can be a `torch.Generator`, or an object with
            a `generator` attribute (such as `Problem`).
            If left as None, the global generator of PyTorch will be used.
    Returns:
            The created or modified tensor after placing the uniformly
            distributed values.
    """
    scalar_requested = _scalar_requested(*size)
    if scalar_requested:
        size = (1,)

    if (dtype is None) and (out is None):
        dtype = torch.int64
    out = _out_tensor_for_random_operation(*size, out=out, dtype=dtype, device=device)
    gen_kwargs = _generator_kwargs(generator)
    out.random_(**gen_kwargs)
    out %= n

    if scalar_requested:
        out = out[0]

    return out

make_tensor(data, *, dtype=None, device=None, read_only=False)

Make a new tensor.

This function can be used to create PyTorch tensors, or ObjectArray instances with or without read-only behavior.

The following example creates a 2-dimensional PyTorch tensor:

my_tensor = make_tensor(
    [[1, 2], [3, 4]],
    dtype="float32",    # alternatively, torch.float32
    device="cpu",
)

The following example creates an ObjectArray from a list that contains arbitrary data:

my_obj_tensor = make_tensor(["a_string", (1, 2)], dtype=object)

Parameters:

Name	Type	Description	Default
data	Any	The data to be converted to a tensor. If one wishes to create a PyTorch tensor, this can be anything that can be stored by a PyTorch tensor. If one wishes to create an ObjectArray and therefore passes dtype=object, then the provided data is expected as an Iterable.	required
dtype	Optional[DType]	Optionally a string (e.g. "float32"), or a PyTorch dtype (e.g. torch.float32), or object or "object" (as a string) or Any if one wishes to create an ObjectArray. If dtype is not specified, it will be assumed that the user wishes to create a PyTorch tensor (not an ObjectArray) and then dtype will be inferred from the provided data (according to the default behavior of PyTorch).	None
device	Optional[Device]	The device in which the tensor will be stored. If device is not specified, it will be understood from the given data (according to the default behavior of PyTorch).	None
read_only	bool	Whether or not the created tensor will be read-only. By default, this is False.	False

Source code in evotorch/tools/misc.py

def make_tensor(
    data: Any,
    *,
    dtype: Optional[DType] = None,
    device: Optional[Device] = None,
    read_only: bool = False,
) -> Iterable:
    """
    Make a new tensor.

    This function can be used to create PyTorch tensors, or ObjectArray
    instances with or without read-only behavior.

    The following example creates a 2-dimensional PyTorch tensor:

        my_tensor = make_tensor(
            [[1, 2], [3, 4]],
            dtype="float32",    # alternatively, torch.float32
            device="cpu",
        )

    The following example creates an ObjectArray from a list that contains
    arbitrary data:

        my_obj_tensor = make_tensor(["a_string", (1, 2)], dtype=object)

    Args:
        data: The data to be converted to a tensor.
            If one wishes to create a PyTorch tensor, this can be anything
            that can be stored by a PyTorch tensor.
            If one wishes to create an `ObjectArray` and therefore passes
            `dtype=object`, then the provided `data` is expected as an
            `Iterable`.
        dtype: Optionally a string (e.g. "float32"), or a PyTorch dtype
            (e.g. torch.float32), or `object` or "object" (as a string)
            or `Any` if one wishes to create an `ObjectArray`.
            If `dtype` is not specified, it will be assumed that the user
            wishes to create a PyTorch tensor (not an `ObjectArray`) and
            then `dtype` will be inferred from the provided `data`
            (according to the default behavior of PyTorch).
        device: The device in which the tensor will be stored.
            If `device` is not specified, it will be understood from the
            given `data` (according to the default behavior of PyTorch).
        read_only: Whether or not the created tensor will be read-only.
            By default, this is False.
    Returns:
        A PyTorch tensor or an ObjectArray.
    """
    from .objectarray import ObjectArray
    from .readonlytensor import as_read_only_tensor

    if (dtype is not None) and is_dtype_object(dtype):
        if not hasattr(data, "__len__"):
            data = list(data)
        n = len(data)
        result = ObjectArray(n)
        result[:] = data
    else:
        kwargs = {}
        if dtype is not None:
            kwargs["dtype"] = to_torch_dtype(dtype)
        if device is not None:
            kwargs["device"] = device
        result = torch.tensor(data, **kwargs)

    if read_only:
        result = as_read_only_tensor(result)

    return result

make_uniform(*size, lb=None, ub=None, out=None, dtype=None, device=None, generator=None)

Make a new or existing tensor filled by uniformly distributed values. Both lower and upper bounds are inclusive. This function can work with both float and int dtypes.

Parameters:

Name	Type	Description	Default
size	Size	Size of the new tensor to be filled with uniformly distributed values. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor instead, then no positional argument is expected.	()
lb	Optional[RealOrVector]	Lower bound for the uniformly distributed values. Can be a scalar, or a tensor. If not specified, the lower bound will be taken as 0. Note that, if one specifies lb, then ub is also expected to be explicitly specified.	None
ub	Optional[RealOrVector]	Upper bound for the uniformly distributed values. Can be a scalar, or a tensor. If not specified, the upper bound will be taken as 1. Note that, if one specifies ub, then lb is also expected to be explicitly specified.	None
out	Optional[Tensor]	Optionally, the tensor to be filled by uniformly distributed values. If an out tensor is given, then no size argument is expected.	None
dtype	Optional[DType]	Optionally a string (e.g. "float32") or a PyTorch dtype (e.g. torch.float32). If dtype is not specified, the default choice of torch.empty(...) is used, that is, torch.float32. If an out tensor is specified, then dtype is expected as None.	None
device	Optional[Device]	The device in which the new tensor will be stored. If not specified, "cpu" will be used. If an out tensor is specified, then device is expected as None.	None
generator	Any	Pseudo-random number generator to be used when sampling the values. Can be a torch.Generator, or an object with a generator attribute (such as Problem). If left as None, the global generator of PyTorch will be used.	None

Source code in evotorch/tools/misc.py

def make_uniform(
    *size: Size,
    lb: Optional[RealOrVector] = None,
    ub: Optional[RealOrVector] = None,
    out: Optional[torch.Tensor] = None,
    dtype: Optional[DType] = None,
    device: Optional[Device] = None,
    generator: Any = None,
) -> torch.Tensor:
    """
    Make a new or existing tensor filled by uniformly distributed values.
    Both lower and upper bounds are inclusive.
    This function can work with both float and int dtypes.

    Args:
        size: Size of the new tensor to be filled with uniformly distributed
            values. This can be given as multiple positional arguments, each
            such positional argument being an integer, or as a single
            positional argument of a tuple, the tuple containing multiple
            integers. Note that, if the user wishes to fill an existing
            tensor instead, then no positional argument is expected.
        lb: Lower bound for the uniformly distributed values.
            Can be a scalar, or a tensor.
            If not specified, the lower bound will be taken as 0.
            Note that, if one specifies `lb`, then `ub` is also expected to
            be explicitly specified.
        ub: Upper bound for the uniformly distributed values.
            Can be a scalar, or a tensor.
            If not specified, the upper bound will be taken as 1.
            Note that, if one specifies `ub`, then `lb` is also expected to
            be explicitly specified.
        out: Optionally, the tensor to be filled by uniformly distributed
            values. If an `out` tensor is given, then no `size` argument is
            expected.
        dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
            (e.g. torch.float32).
            If `dtype` is not specified, the default choice of
            `torch.empty(...)` is used, that is, `torch.float32`.
            If an `out` tensor is specified, then `dtype` is expected
            as None.
        device: The device in which the new tensor will be stored.
            If not specified, "cpu" will be used.
            If an `out` tensor is specified, then `device` is expected
            as None.
        generator: Pseudo-random number generator to be used when sampling
            the values. Can be a `torch.Generator`, or an object with
            a `generator` attribute (such as `Problem`).
            If left as None, the global generator of PyTorch will be used.
    Returns:
        The created or modified tensor after placing the uniformly
        distributed values.
    """

    scalar_requested = _scalar_requested(*size)
    if scalar_requested:
        size = (1,)

    def _invalid_bound_args():
        raise ValueError(
            f"Expected both `lb` and `ub` as None, or both `lb` and `ub` as not None."
            f" It appears that one of them is None, while the other is not."
            f" lb: {repr(lb)}."
            f" ub: {repr(ub)}."
        )

    out = _out_tensor_for_random_operation(*size, out=out, dtype=dtype, device=device)
    gen_kwargs = _generator_kwargs(generator)

    def _cast_bounds():
        nonlocal lb, ub
        lb = torch.as_tensor(lb, dtype=out.dtype, device=out.device)
        ub = torch.as_tensor(ub, dtype=out.dtype, device=out.device)

    if out.dtype == torch.bool:
        out.random_(**gen_kwargs)
        if (lb is None) and (ub is None):
            pass  # nothing to do
        elif (lb is not None) and (ub is not None):
            _cast_bounds()
            lb_shape_matches = lb.shape == out.shape
            ub_shape_matches = ub.shape == out.shape
            if (not lb_shape_matches) or (not ub_shape_matches):
                all_false = torch.zeros_like(out)
                if not lb_shape_matches:
                    lb = lb | all_false
                if not ub_shape_matches:
                    ub = ub | all_false
            mask_for_always_false = (~lb) & (~ub)
            mask_for_always_true = lb & ub
            out[mask_for_always_false] = False
            out[mask_for_always_true] = True
        else:
            _invalid_bound_args()
    elif out.dtype in (torch.uint8, torch.int8, torch.int16, torch.int32, torch.int64):
        out.random_(**gen_kwargs)
        if (lb is None) and (ub is None):
            out %= 2
        elif (lb is not None) and (ub is not None):
            _cast_bounds()
            diff = (ub - lb) + 1
            out -= lb
            out %= diff
            out += lb
        else:
            _invalid_bound_args()
    else:
        out.uniform_(**gen_kwargs)
        if (lb is None) and (ub is None):
            pass  # nothing to do
        elif (lb is not None) and (ub is not None):
            _cast_bounds()
            diff = ub - lb
            out *= diff
            out += lb
        else:
            _invalid_bound_args()

    if scalar_requested:
        out = out[0]

    return out

make_zeros(*size, out=None, dtype=None, device=None)

Make a new tensor filled with 0, or fill an existing tensor with 0.

The following example creates a float32 tensor filled with 0 values, of shape (3, 5):

zero_values = make_zeros(3, 5, dtype="float32")

The following example fills an existing tensor with 0s:

make_zeros(out=existing_tensor)

Parameters:

Name	Type	Description	Default
size	Size	Size of the new tensor to be filled with 0. This can be given as multiple positional arguments, each such positional argument being an integer, or as a single positional argument of a tuple, the tuple containing multiple integers. Note that, if the user wishes to fill an existing tensor with 0 values, then no positional argument is expected.	()
out	Optional[Tensor]	Optionally, the tensor to be filled by 0 values. If an out tensor is given, then no size argument is expected.	None
dtype	Optional[DType]	Optionally a string (e.g. "float32") or a PyTorch dtype (e.g. torch.float32). If dtype is not specified, the default choice of torch.empty(...) is used, that is, torch.float32. If an out tensor is specified, then dtype is expected as None.	None
device	Optional[Device]	The device in which the new tensor will be stored. If not specified, "cpu" will be used. If an out tensor is specified, then device is expected as None.	None

Source code in evotorch/tools/misc.py

def make_zeros(
    *size: Size,
    out: Optional[torch.Tensor] = None,
    dtype: Optional[DType] = None,
    device: Optional[Device] = None,
) -> torch.Tensor:
    """
    Make a new tensor filled with 0, or fill an existing tensor with 0.

    The following example creates a float32 tensor filled with 0 values,
    of shape (3, 5):

        zero_values = make_zeros(3, 5, dtype="float32")

    The following example fills an existing tensor with 0s:

        make_zeros(out=existing_tensor)

    Args:
        size: Size of the new tensor to be filled with 0.
            This can be given as multiple positional arguments, each such
            positional argument being an integer, or as a single positional
            argument of a tuple, the tuple containing multiple integers.
            Note that, if the user wishes to fill an existing tensor with
            0 values, then no positional argument is expected.
        out: Optionally, the tensor to be filled by 0 values.
            If an `out` tensor is given, then no `size` argument is expected.
        dtype: Optionally a string (e.g. "float32") or a PyTorch dtype
            (e.g. torch.float32).
            If `dtype` is not specified, the default choice of
            `torch.empty(...)` is used, that is, `torch.float32`.
            If an `out` tensor is specified, then `dtype` is expected
            as None.
        device: The device in which the new tensor will be stored.
            If not specified, "cpu" will be used.
            If an `out` tensor is specified, then `device` is expected
            as None.
    Returns:
        The created or modified tensor after placing 0 values.
    """
    if _scalar_requested(*size):
        return _scalar_tensor(0, out=out, dtype=dtype, device=device)
    else:
        out = _out_tensor(*size, out=out, dtype=dtype, device=device)
        out.zero_()
        return out

modify_tensor(original, target, lb=None, ub=None, max_change=None, in_place=False)

Return the modified version of the original tensor, with bounds checking.

Parameters:

Name	Type	Description	Default
original	Tensor	The original tensor.	required
target	Tensor	The target tensor which contains the values to replace the old ones in the original tensor.	required
lb	Optional[Union[float, Tensor]]	The lower bound(s), as a scalar or as an tensor. Values below these bounds are clipped in the resulting tensor. None means -inf.	None
ub	Optional[Union[float, Tensor]]	The upper bound(s), as a scalar or as an tensor. Value above these bounds are clipped in the resulting tensor. None means +inf.	None
max_change	Optional[Union[float, Tensor]]	The ratio of allowed change. In more details, when given as a real number r, modifications are allowed only within [original-(r*abs(original)) ... original+(r*abs(original))]. Modifications beyond this interval are clipped. This argument can also be left as None if no such limitation is needed.	None
in_place	bool	Provide this as True if you wish the modification to be done within the original tensor. The default value of this argument is False, which means, the original tensor is not changed, and its modified version is returned as an independent copy.	False

Source code in evotorch/tools/misc.py

@torch.no_grad()
def modify_tensor(
    original: torch.Tensor,
    target: torch.Tensor,
    lb: Optional[Union[float, torch.Tensor]] = None,
    ub: Optional[Union[float, torch.Tensor]] = None,
    max_change: Optional[Union[float, torch.Tensor]] = None,
    in_place: bool = False,
) -> torch.Tensor:
    """Return the modified version of the original tensor, with bounds checking.

    Args:
        original: The original tensor.
        target: The target tensor which contains the values to replace the
            old ones in the original tensor.
        lb: The lower bound(s), as a scalar or as an tensor.
            Values below these bounds are clipped in the resulting tensor.
            None means -inf.
        ub: The upper bound(s), as a scalar or as an tensor.
            Value above these bounds are clipped in the resulting tensor.
            None means +inf.
        max_change: The ratio of allowed change.
            In more details, when given as a real number r,
            modifications are allowed only within
            ``[original-(r*abs(original)) ... original+(r*abs(original))]``.
            Modifications beyond this interval are clipped.
            This argument can also be left as None if no such limitation
            is needed.
        in_place: Provide this as True if you wish the modification to be
            done within the original tensor. The default value of this
            argument is False, which means, the original tensor is not
            changed, and its modified version is returned as an independent
            copy.
    Returns:
        The modified tensor.
    """
    if (lb is None) and (ub is None) and (max_change is None):
        # If there is no restriction regarding how the tensor
        # should be modified (no lb, no ub, no max_change),
        # then we simply use the target values
        # themselves for modifying the tensor.

        if in_place:
            original[:] = target
            return original
        else:
            return target
    else:
        # If there are some restriction regarding how the tensor
        # should be modified, then we turn to the following
        # operations

        def convert_to_tensor(x, tensorname: str):
            if isinstance(x, torch.Tensor):
                converted = x
            else:
                converted = torch.as_tensor(x, dtype=original.dtype, device=original.device)
            if converted.ndim == 0 or converted.shape == original.shape:
                return converted
            else:
                raise IndexError(
                    f"Argument {tensorname}: shape mismatch."
                    f" Shape of the original tensor: {original.shape}."
                    f" Shape of {tensorname}: {converted.shape}."
                )

        if lb is None:
            # If lb is None, then it should be taken as -inf
            lb = convert_to_tensor(float("-inf"), "lb")
        else:
            lb = convert_to_tensor(lb, "lb")

        if ub is None:
            # If ub is None, then it should be taken as +inf
            ub = convert_to_tensor(float("inf"), "ub")
        else:
            ub = convert_to_tensor(ub, "ub")

        if max_change is not None:
            # If max_change is provided as something other than None,
            # then we update the lb and ub so that they are tight
            # enough to satisfy the max_change restriction.

            max_change = convert_to_tensor(max_change, "max_change")
            allowed_amounts = torch.abs(original) * max_change
            allowed_lb = original - allowed_amounts
            allowed_ub = original + allowed_amounts
            lb = torch.max(lb, allowed_lb)
            ub = torch.min(ub, allowed_ub)

        ## If in_place is given as True, the clipping (that we are about
        ## to perform), should be in-place.
        # more_config = {}
        # if in_place:
        #    more_config['out'] = original
        #
        ## Return the clipped version of the target values
        # return torch.clamp(target, lb, ub, **more_config)

        result = torch.max(target, lb)
        result = torch.min(result, ub)

        if in_place:
            original[:] = result
            return original
        else:
            return result

modify_vector(original, target, *, lb=None, ub=None, max_change=None)

Return the modified version(s) of the vector(s), with bounds checking.

This function is similar to modify_tensor, but it has the following different behaviors:

• Assumes that all of its arguments are either vectors, or are batches of vectors. If some or more of its arguments have 2 or more dimensions, those arguments will be considered as batches, and the computation will be vectorized to return a batch of results.
• Designed to be vmap-friendly.
• Designed for functional programming paradigm, and therefore lacks the in-place modification option.

Source code in evotorch/tools/misc.py

def modify_vector(
    original: torch.Tensor,
    target: torch.Tensor,
    *,
    lb: Optional[Union[float, torch.Tensor]] = None,
    ub: Optional[Union[float, torch.Tensor]] = None,
    max_change: Optional[Union[float, torch.Tensor]] = None,
) -> torch.Tensor:
    """
    Return the modified version(s) of the vector(s), with bounds checking.

    This function is similar to `modify_tensor`, but it has the following
    different behaviors:

    - Assumes that all of its arguments are either vectors, or are batches
      of vectors. If some or more of its arguments have 2 or more dimensions,
      those arguments will be considered as batches, and the computation will
      be vectorized to return a batch of results.
    - Designed to be `vmap`-friendly.
    - Designed for functional programming paradigm, and therefore lacks the
      in-place modification option.
    """
    from ..decorators import expects_ndim

    if max_change is None:
        result = target
    else:
        result = expects_ndim(_modify_vector_using_max_change, (1, 1, 1), allow_smaller_ndim=True)(
            original, target, max_change
        )

    if (lb is None) and (ub is None):
        pass  # no strict boundaries, so, nothing more to do
    elif (lb is not None) and (ub is not None):
        result = expects_ndim(_modify_vector_using_bounds, (1, 1, 1), allow_smaller_ndim=True)(result, lb, ub)
    else:
        raise ValueError(
            "`modify_vector` expects either with `lb` and `ub` given together, or with both of them omitted."
            " Having only `lb` or only `ub` is not supported."
        )
    return result

numpy_copy(x, dtype=None)

Return a numpy copy of the given iterable.

The newly created numpy array will be mutable, even if the original iterable object is read-only.

Parameters:

Name	Type	Description	Default
x	Iterable	Any Iterable whose numpy copy will be returned.	required
dtype	Optional[DType]	The desired dtype. Can be given as a numpy dtype, as a torch dtype, or a native dtype (e.g. int, float), or as a string (e.g. "float32"). If left as None, dtype will be determined according to the data contained by the original iterable object.	None

Source code in evotorch/tools/misc.py

def numpy_copy(x: Iterable, dtype: Optional[DType] = None) -> np.ndarray:
    """
    Return a numpy copy of the given iterable.

    The newly created numpy array will be mutable, even if the
    original iterable object is read-only.

    Args:
        x: Any Iterable whose numpy copy will be returned.
        dtype: The desired dtype. Can be given as a numpy dtype,
            as a torch dtype, or a native dtype (e.g. int, float),
            or as a string (e.g. "float32").
            If left as None, dtype will be determined according
            to the data contained by the original iterable object.
    Returns:
        The numpy copy of the original iterable object.
    """
    from .objectarray import ObjectArray

    needs_casting = dtype is not None

    if isinstance(x, ObjectArray):
        result = x.numpy()
    elif isinstance(x, torch.Tensor):
        result = x.cpu().clone().numpy()
    elif isinstance(x, np.ndarray):
        result = x.copy()
    else:
        needs_casting = False
        result = np.array(x, dtype=dtype)

    if needs_casting:
        result = result.astype(dtype)

    return result

pass_info_if_needed(f, info)

Pass additional arguments into a callable, the info dictionary is unpacked and passed as additional keyword arguments only if the policy is decorated with the pass_info decorator.

Parameters:

Name	Type	Description	Default
f	Callable	The callable to be called.	required
info	Dict[str, Any]	The info to be passed to the callable.	required

Source code in evotorch/tools/misc.py

def pass_info_if_needed(f: Callable, info: Dict[str, Any]) -> Callable:
    """
    Pass additional arguments into a callable, the info dictionary is unpacked
    and passed as additional keyword arguments only if the policy is decorated
    with the [pass_info][evotorch.decorators.pass_info] decorator.

    Args:
        f (Callable): The callable to be called.
        info (Dict[str, Any]): The info to be passed to the callable.
    Returns:
        Callable: The callable with extra arguments
    Raises:
        TypeError: If the callable is decorated with the [pass_info][evotorch.decorators.pass_info] decorator,
            but its signature does not match the expected signature.
    """
    if hasattr(f, "__evotorch_pass_info__"):
        try:
            sig = inspect.signature(f)
            sig.bind_partial(**info)
        except TypeError:
            raise TypeError(
                "Callable {f} is decorated with @pass_info, but it doesn't expect some of the extra arguments "
                f"({', '.join(info.keys())}). Hint: maybe you forgot to add **kwargs to the function signature?"
            )
        except Exception:
            pass

        return functools.partial(f, **info)
    else:
        return f

penalty(lhs, comparison, rhs, *, penalty_sign, linear=None, step=None, exp=None, exp_inf=None)

Return a penalty based on the amount of violation of the constraint.

Depending on the provided arguments, the penalty can be linear, or exponential, or based on step function, or a combination of these.

Parameters:

Name	Type	Description	Default
lhs	Union[float, Tensor]	The left-hand-side of the constraint. In the non-batched case, this is expected as a scalar. If it is given as an n-dimensional tensor where n is at least 1, this is considered as a batch of left-hand-side values.	required
comparison	str	The operator used for comparing the left-hand-side and the right-hand-side. Expected as a string. Acceptable values are: '<=', '==', '>='.	required
rhs	Union[float, Tensor]	The right-hand-side of the constraint. In the non-batched case, this is expected as a scalar. If it is given as an n-dimensional tensor where n is at least 1, this is considered as a batch of right-hand-side values.	required
penalty_sign	str	Expected as string, either as '+' or '-', which determines the sign of the penalty (i.e. determines if the penalty will be positive or negative). One should consider the objective sense of the fitness function at hand for deciding penalty_sign. For example, if a fitness function is written from the perspective of maximization, the penalties should be negative, and therefore, penalty_sign must be given as '-'.	required
linear	Optional[Union[float, Tensor]]	Multiplier for the linear component of the penalization. If omitted (i.e. left as None), the value of this multiplier will be 0 (meaning that there will not be any linear penalization). In the non-batched case, this argument is expected as a scalar. If this is provided as a tensor 1 or more dimensions, those dimensions will be considered as batch dimensions.	None
step	Optional[Union[float, Tensor]]	The constant amount that will be added onto the penalty if there is a violation. If omitted (i.e. left as None), this value is 0. In the non-batched case, this argument is expected as a scalar. If this is provided as a tensor 1 or more dimensions, those dimensions will be considered as batch dimensions.	None
exp	Optional[Union[float, Tensor]]	A constant p that will enable exponential penalization in the form amount_of_violation ** p. If this is left as None or is given as nan, there will be no exponential penalization. In the non-batched case, this argument is expected as a scalar. If this is provided as a tensor 1 or more dimensions, those dimensions will be considered as batch dimensions.	None
exp_inf	Optional[Union[float, Tensor]]	Upper bound for exponential penalty values. If exponential penalty is enabled but exp_inf is omitted (i.e. left as None), the exponential penalties can jump to very large values or to infinity, potentially causing numerical instabilities. To avoid such numerical instabilities, one might provide a large-enough positive constant M via the argument exp_inf. When such a value is given, exponential penalties will not be allowed to exceed M. One might also think of this as temporarily replacing inf with M while computing the exponential penalties.	None

Source code in evotorch/tools/constraints.py

def penalty(
    lhs: Union[float, torch.Tensor],
    comparison: str,
    rhs: Union[float, torch.Tensor],
    *,
    penalty_sign: str,
    linear: Optional[Union[float, torch.Tensor]] = None,
    step: Optional[Union[float, torch.Tensor]] = None,
    exp: Optional[Union[float, torch.Tensor]] = None,
    exp_inf: Optional[Union[float, torch.Tensor]] = None,
) -> torch.Tensor:
    """
    Return a penalty based on the amount of violation of the constraint.

    Depending on the provided arguments, the penalty can be linear,
    or exponential, or based on step function, or a combination of these.

    Args:
        lhs: The left-hand-side of the constraint. In the non-batched case,
            this is expected as a scalar. If it is given as an n-dimensional
            tensor where n is at least 1, this is considered as a batch of
            left-hand-side values.
        comparison: The operator used for comparing the left-hand-side and the
            right-hand-side. Expected as a string. Acceptable values are:
            '<=', '==', '>='.
        rhs: The right-hand-side of the constraint. In the non-batched case,
            this is expected as a scalar. If it is given as an n-dimensional
            tensor where n is at least 1, this is considered as a batch of
            right-hand-side values.
        penalty_sign: Expected as string, either as '+' or '-', which
            determines the sign of the penalty (i.e. determines if the penalty
            will be positive or negative). One should consider the objective
            sense of the fitness function at hand for deciding `penalty_sign`.
            For example, if a fitness function is written from the perspective
            of maximization, the penalties should be negative, and therefore,
            `penalty_sign` must be given as '-'.
        linear: Multiplier for the linear component of the penalization.
            If omitted (i.e. left as None), the value of this multiplier will
            be 0 (meaning that there will not be any linear penalization).
            In the non-batched case, this argument is expected as a scalar.
            If this is provided as a tensor 1 or more dimensions, those
            dimensions will be considered as batch dimensions.
        step: The constant amount that will be added onto the penalty if there
            is a violation. If omitted (i.e. left as None), this value is 0.
            In the non-batched case, this argument is expected as a scalar.
            If this is provided as a tensor 1 or more dimensions, those
            dimensions will be considered as batch dimensions.
        exp: A constant `p` that will enable exponential penalization in the
            form `amount_of_violation ** p`. If this is left as None or is
            given as `nan`, there will be no exponential penalization.
            In the non-batched case, this argument is expected as a scalar.
            If this is provided as a tensor 1 or more dimensions, those
            dimensions will be considered as batch dimensions.
        exp_inf: Upper bound for exponential penalty values. If exponential
            penalty is enabled but `exp_inf` is omitted (i.e. left as None),
            the exponential penalties can jump to very large values or to
            infinity, potentially causing numerical instabilities. To avoid
            such numerical instabilities, one might provide a large-enough
            positive constant `M` via the argument `exp_inf`. When such a value
            is given, exponential penalties will not be allowed to exceed `M`.
            One might also think of this as temporarily replacing `inf` with
            `M` while computing the exponential penalties.
    Returns:
        The penalty amount(s), whose sign(s) is/are determined by
        `sign_penalty`.
    """
    from ..decorators import expects_ndim

    if linear is None:
        linear = 0.0
    if step is None:
        step = 0.0
    if exp is None:
        exp = float("nan")
    if exp_inf is None:
        exp_inf = float("inf")

    return expects_ndim(_penalty, (0, None, 0, None, 0, 0, 0, 0))(
        lhs,
        comparison,
        rhs,
        penalty_sign,
        linear,
        step,
        exp,
        exp_inf,
    )

rank(fitnesses, ranking_method, *, higher_is_better)

Get the ranks of the given sequence of numbers.

Better solutions will have numerically higher ranks.

Parameters:

Name	Type	Description	Default
fitnesses	Iterable[float]	A sequence of numbers to be ranked.	required
ranking_method	str	The ranking method to be used. Can be "centered", which means 0-centered linear ranking from -0.5 to 0.5. Can be "linear", which means a linear ranking from 0 to 1. Can be "nes", which means the ranking method used by Natural Evolution Strategies. Can be "normalized", which means that the ranks will be the normalized counterparts of the fitnesses. Can be "raw", which means that the fitnesses themselves (or, if higher_is_better is False, their inverted counterparts, inversion meaning the operation of multiplying by -1 in this context) will be the ranks.	required
higher_is_better	bool	Whether or not the higher values will be assigned higher ranks. Changing this to False means that lower values are interpreted as better, and therefore lower values will have higher ranks.	required

Source code in evotorch/tools/ranking.py

def rank(fitnesses: Iterable[float], ranking_method: str, *, higher_is_better: bool):
    """
    Get the ranks of the given sequence of numbers.

    Better solutions will have numerically higher ranks.

    Args:
        fitnesses: A sequence of numbers to be ranked.
        ranking_method: The ranking method to be used.
            Can be "centered", which means 0-centered linear ranking
                from -0.5 to 0.5.
            Can be "linear", which means a linear ranking from 0 to 1.
            Can be "nes", which means the ranking method used by
                Natural Evolution Strategies.
            Can be "normalized", which means that the ranks will be
                the normalized counterparts of the fitnesses.
            Can be "raw", which means that the fitnesses themselves
                (or, if `higher_is_better` is False, their inverted
                counterparts, inversion meaning the operation of
                multiplying by -1 in this context) will be the ranks.
        higher_is_better: Whether or not the higher values will be assigned
             higher ranks. Changing this to False means that lower values
             are interpreted as better, and therefore lower values will have
             higher ranks.
    """
    fitnesses = torch.as_tensor(fitnesses)
    rank_func = rankers[ranking_method]
    return rank_func(fitnesses, higher_is_better=higher_is_better)

read_only_tensor(x, *, dtype=None, device=None)

Make a ReadOnlyTensor from the given object.

The provided object can be a scalar, or an Iterable of numeric data, or an ObjectArray.

This function can be thought as the read-only counterpart of PyTorch's torch.tensor(...) function.

Parameters:

Name	Type	Description	Default
x	Any	The object from which the new ReadOnlyTensor will be made.	required
dtype	Optional[dtype]	The dtype of the new ReadOnlyTensor (e.g. torch.float32).	None
device	Optional[Union[str, device]]	The device in which the ReadOnlyTensor will be stored (e.g. "cpu").	None

Source code in evotorch/tools/readonlytensor.py

def read_only_tensor(
    x: Any, *, dtype: Optional[torch.dtype] = None, device: Optional[Union[str, torch.device]] = None
) -> Iterable:
    """
    Make a ReadOnlyTensor from the given object.

    The provided object can be a scalar, or an Iterable of numeric data,
    or an ObjectArray.

    This function can be thought as the read-only counterpart of PyTorch's
    `torch.tensor(...)` function.

    Args:
        x: The object from which the new ReadOnlyTensor will be made.
        dtype: The dtype of the new ReadOnlyTensor (e.g. torch.float32).
        device: The device in which the ReadOnlyTensor will be stored
            (e.g. "cpu").
    Returns:
        The new read-only tensor.
    """
    from .objectarray import ObjectArray

    kwargs = _device_and_dtype_kwargs(dtype=dtype, device=device)
    if isinstance(x, ObjectArray):
        if len(kwargs) != 0:
            raise ValueError(
                f"read_only_tensor(...): when making a read-only tensor from an ObjectArray,"
                f" the arguments `dtype` and `device` were not expected."
                f" However, the received keyword arguments are: {kwargs}."
            )
        return x.get_read_only_view()
    else:
        return torch.as_tensor(x, **kwargs).as_subclass(ReadOnlyTensor)

set_default_logger_config(logger_name='evotorch', logger_level=logging.INFO, show_process=True, show_lineno=False, override=False)

Configure the "EvoTorch" Python logger to print to the console with default format.

The logger will be configured to print to all messages with level INFO or lower to stdout and all messages with level WARNING or higher to stderr.

The default format is:

[2022-11-23 22:28:47] INFO     <75935>   evotorch:      This is a log message
{asctime}             {level}  {process} {logger_name}: {message}

The format can be slightly customized by passing show_process=False to hide Process ID or show_lineno=True to show the filename and line number of the log message instead of the Logger Name.

This function should be called before any other logging is performed, otherwise the default configuration will not be applied. If the logger is already configured, this function will do nothing unless override=True is passed, in which case the logger will be reconfigured.

Parameters:

Name	Type	Description	Default
logger_name	str	Name of the logger to configure.	'evotorch'
logger_level	int	Level of the logger to configure.	INFO
show_process	bool	Whether to show the process name in the log message.	True
show_lineno	bool	Whether to show the filename with the line number in the log message or just the name of the logger.	False
override	bool	Whether to override the logger configuration if it has already been configured.	False

Source code in evotorch/tools/misc.py

def set_default_logger_config(
    logger_name: str = "evotorch",
    logger_level: int = logging.INFO,
    show_process: bool = True,
    show_lineno: bool = False,
    override: bool = False,
):
    """
    Configure the "EvoTorch" Python logger to print to the console with default format.

    The logger will be configured to print to all messages with level INFO or lower to stdout and all
    messages with level WARNING or higher to stderr.

    The default format is:
    ```
    [2022-11-23 22:28:47] INFO     <75935>   evotorch:      This is a log message
    {asctime}             {level}  {process} {logger_name}: {message}
    ```
    The format can be slightly customized by passing `show_process=False` to hide Process ID or `show_lineno=True` to
    show the filename and line number of the log message instead of the Logger Name.

    This function should be called before any other logging is performed, otherwise the default configuration will
    not be applied. If the logger is already configured, this function will do nothing unless `override=True` is passed,
    in which case the logger will be reconfigured.

    Args:
        logger_name: Name of the logger to configure.
        logger_level: Level of the logger to configure.
        show_process: Whether to show the process name in the log message.
        show_lineno: Whether to show the filename with the line number in the log message or just the name of the logger.
        override: Whether to override the logger configuration if it has already been configured.
    """
    logger = logging.getLogger(logger_name)

    if not override and logger.hasHandlers():
        # warn user that the logger is already configured
        logger.warning(
            "The logger is already configured. "
            "The default configuration will not be applied. "
            "Call `set_default_logger_config` with `override=True` to override the current configuration."
        )
        return
    elif override:
        # remove all handlers
        for handler in logger.handlers:
            logger.removeHandler(handler)

    logger.setLevel(logger_level)
    logger.propagate = False

    formatter = logging.Formatter(
        "[{asctime}] "
        + "{levelname:<8s} "
        + ("<{process:5d}> " if show_process else "")
        + ("{filename}:{lineno}: " if show_lineno else "{name}: ")
        + "{message}",
        datefmt="%Y-%m-%d %H:%M:%S",
        style="{",
    )

    _stdout_handler = logging.StreamHandler(sys.stdout)
    _stdout_handler.addFilter(lambda log_record: log_record.levelno < logging.WARNING)
    _stdout_handler.setFormatter(formatter)
    logger.addHandler(_stdout_handler)

    _stderr_handler = logging.StreamHandler(sys.stderr)
    _stderr_handler.addFilter(lambda log_record: log_record.levelno >= logging.WARNING)
    _stderr_handler.setFormatter(formatter)
    logger.addHandler(_stderr_handler)

split_workload(workload, num_actors)

Split a workload among actors.

By "workload" what is meant is the total amount of a work, this amount being expressed by an integer. For example, if the "work" is the evaluation of a population, the "workload" would usually be the population size.

Parameters:

Name	Type	Description	Default
workload	int	Total amount of work, as an integer.	required
num_actors	int	Number of actors (i.e. remote workers) among which the workload will be distributed.	required

Source code in evotorch/tools/misc.py

def split_workload(workload: int, num_actors: int) -> list:
    """
    Split a workload among actors.

    By "workload" what is meant is the total amount of a work,
    this amount being expressed by an integer.
    For example, if the "work" is the evaluation of a population,
    the "workload" would usually be the population size.

    Args:
        workload: Total amount of work, as an integer.
        num_actors: Number of actors (i.e. remote workers) among
            which the workload will be distributed.
    Returns:
        A list of integers. The i-th item of the returned list
        expresses the suggested workload for the i-th actor.
    """
    base_workload = workload // num_actors
    extra_workload = workload % num_actors
    result = [base_workload] * num_actors
    for i in range(extra_workload):
        result[i] += 1
    return result

stdev_from_radius(radius, solution_length)

Get elementwise standard deviation from a given radius.

Sometimes, for a distribution-based search algorithm, the user might choose to configure the initial coverage area of the search distribution not via standard deviation, but via a radius value, as was done in the study of Toklu et al. (2020). This function takes the desired radius value and the solution length of the problem at hand, and returns the elementwise standard deviation value. Let us name this returned standard deviation value as s. When a new Gaussian distribution is constructed such that its initial standard deviation is [s, s, s, ...] (the length of this vector being equal to the solution length), this constructed distribution's radius corresponds with the desired radius.

Here, the "radius" of a Gaussian distribution is defined as the norm of the standard deviation vector. In the case of a standard normal distribution, this radius formulation serves as a simplified approximation to E[||Normal(0, I)||] (for which a closer approximation is used in the study of Hansen & Ostermeier (2001)).

Reference:

Toklu, N.E., Liskowski, P., Srivastava, R.K. (2020).
ClipUp: A Simple and Powerful Optimizer
for Distribution-based Policy Evolution.
Parallel Problem Solving from Nature (PPSN 2020).

Nikolaus Hansen, Andreas Ostermeier (2001).
Completely Derandomized Self-Adaptation in Evolution Strategies.

Parameters:

Name	Type	Description	Default
radius	float	The radius whose elementwise standard deviation counterpart will be returned.	required
solution_length	int	Length of a solution for the problem at hand.	required

Source code in evotorch/tools/misc.py

def stdev_from_radius(radius: float, solution_length: int) -> float:
    """
    Get elementwise standard deviation from a given radius.

    Sometimes, for a distribution-based search algorithm, the user might
    choose to configure the initial coverage area of the search distribution
    not via standard deviation, but via a radius value, as was done in the
    study of Toklu et al. (2020).
    This function takes the desired radius value and the solution length of
    the problem at hand, and returns the elementwise standard deviation value.
    Let us name this returned standard deviation value as `s`.
    When a new Gaussian distribution is constructed such that its initial
    standard deviation is `[s, s, s, ...]` (the length of this vector being
    equal to the solution length), this constructed distribution's radius
    corresponds with the desired radius.

    Here, the "radius" of a Gaussian distribution is defined as the norm
    of the standard deviation vector. In the case of a standard normal
    distribution, this radius formulation serves as a simplified approximation
    to `E[||Normal(0, I)||]` (for which a closer approximation is used in
    the study of Hansen & Ostermeier (2001)).

    Reference:

        Toklu, N.E., Liskowski, P., Srivastava, R.K. (2020).
        ClipUp: A Simple and Powerful Optimizer
        for Distribution-based Policy Evolution.
        Parallel Problem Solving from Nature (PPSN 2020).

        Nikolaus Hansen, Andreas Ostermeier (2001).
        Completely Derandomized Self-Adaptation in Evolution Strategies.

    Args:
        radius: The radius whose elementwise standard deviation counterpart
            will be returned.
        solution_length: Length of a solution for the problem at hand.
    Returns:
        An elementwise standard deviation value `s`, such that a Gaussian
        distribution constructed with the standard deviation `[s, s, s, ...]`
        has the desired radius.
    """
    radius = float(radius)
    solution_length = int(solution_length)
    return math.sqrt((radius**2) / solution_length)

storage_ptr(x)

Get the pointer to the underlying storage of a tensor of an ObjectArray.

Calling storage_ptr(x) is equivalent to x.untyped_storage().data_ptr().

Parameters:

Name	Type	Description	Default
x	Iterable	A regular PyTorch tensor, or a ReadOnlyTensor, or an ObjectArray.	required

Source code in evotorch/tools/misc.py

def storage_ptr(x: Iterable) -> int:
    """
    Get the pointer to the underlying storage of a tensor of an ObjectArray.

    Calling `storage_ptr(x)` is equivalent to `x.untyped_storage().data_ptr()`.

    Args:
        x: A regular PyTorch tensor, or a ReadOnlyTensor, or an ObjectArray.
    Returns:
        The address of the underlying storage.
    """
    return _storage_ptr(x)

to_numpy_dtype(dtype)

Convert the given string or the given PyTorch dtype to a numpy dtype. If the argument is already a numpy dtype, then the argument is returned as it is.

Returns:

Type	Description
dtype	The dtype, converted to a numpy dtype.

Source code in evotorch/tools/misc.py

def to_numpy_dtype(dtype: DType) -> np.dtype:
    """
    Convert the given string or the given PyTorch dtype to a numpy dtype.
    If the argument is already a numpy dtype, then the argument is returned
    as it is.

    Returns:
        The dtype, converted to a numpy dtype.
    """
    if isinstance(dtype, torch.dtype):
        return torch.tensor([], dtype=dtype).numpy().dtype
    elif is_dtype_object(dtype):
        return np.dtype(object)
    elif isinstance(dtype, np.dtype):
        return dtype
    else:
        return np.dtype(dtype)

to_stdev_init(*, solution_length, stdev_init=None, radius_init=None)

Ask for both standard deviation and radius, return the standard deviation.

It is very common among the distribution-based search algorithms to ask for both standard deviation and for radius for initializing the coverage area of the search distribution. During their initialization phases, these algorithms must check which one the user provided (radius or standard deviation), and return the result as the standard deviation so that a Gaussian distribution can easily be constructed.

This function serves as a helper function for such search algorithms by performing these actions:

• If the user provided a standard deviation and not a radius, then this provided standard deviation is simply returned.
• If the user provided a radius and not a standard deviation, then this provided radius is converted to its standard deviation counterpart, and then returned.
• If both standard deviation and radius are missing, or they are both given at the same time, then an error is raised.

Parameters:

Name	Type	Description	Default
solution_length	int	Length of a solution for the problem at hand.	required
stdev_init	Optional[RealOrVector]	Standard deviation. If one wishes to provide a radius instead, then stdev_init is expected as None.	None
radius_init	Optional[RealOrVector]	Radius. If one wishes to provide a standard deviation instead, then radius_init is expected as None.	None

Source code in evotorch/tools/misc.py

def to_stdev_init(
    *,
    solution_length: int,
    stdev_init: Optional[RealOrVector] = None,
    radius_init: Optional[RealOrVector] = None,
) -> RealOrVector:
    """
    Ask for both standard deviation and radius, return the standard deviation.

    It is very common among the distribution-based search algorithms to ask
    for both standard deviation and for radius for initializing the coverage
    area of the search distribution. During their initialization phases,
    these algorithms must check which one the user provided (radius or
    standard deviation), and return the result as the standard deviation
    so that a Gaussian distribution can easily be constructed.

    This function serves as a helper function for such search algorithms
    by performing these actions:

    - If the user provided a standard deviation and not a radius, then this
      provided standard deviation is simply returned.
    - If the user provided a radius and not a standard deviation, then this
      provided radius is converted to its standard deviation counterpart,
      and then returned.
    - If both standard deviation and radius are missing, or they are both
      given at the same time, then an error is raised.

    Args:
        solution_length: Length of a solution for the problem at hand.
        stdev_init: Standard deviation. If one wishes to provide a radius
            instead, then `stdev_init` is expected as None.
        radius_init: Radius. If one wishes to provide a standard deviation
            instead, then `radius_init` is expected as None.
    Returns:
        The standard deviation for the search distribution to be constructed.
    """
    if (stdev_init is not None) and (radius_init is None):
        return stdev_init
    elif (stdev_init is None) and (radius_init is not None):
        return stdev_from_radius(radius_init, solution_length)
    elif (stdev_init is None) and (radius_init is None):
        raise ValueError(
            "Received both `stdev_init` and `radius_init` as None."
            " Please provide a value either for `stdev_init` or for `radius_init`."
        )
    else:
        raise ValueError(
            "Found both `stdev_init` and `radius_init` with values other than None."
            " Please provide a value either for `stdev_init` or for `radius_init`, but not for both."
        )

to_torch_dtype(dtype)

Convert the given string or the given numpy dtype to a PyTorch dtype. If the argument is already a PyTorch dtype, then the argument is returned as it is.

Returns:

Type	Description
dtype	The dtype, converted to a PyTorch dtype.

Source code in evotorch/tools/misc.py

def to_torch_dtype(dtype: DType) -> torch.dtype:
    """
    Convert the given string or the given numpy dtype to a PyTorch dtype.
    If the argument is already a PyTorch dtype, then the argument is returned
    as it is.

    Returns:
        The dtype, converted to a PyTorch dtype.
    """
    if isinstance(dtype, str) and hasattr(torch, dtype):
        attrib_within_torch = getattr(torch, dtype)
    else:
        attrib_within_torch = None

    if isinstance(attrib_within_torch, torch.dtype):
        return attrib_within_torch
    elif isinstance(dtype, torch.dtype):
        return dtype
    elif dtype is Any or dtype is object:
        raise TypeError(f"Cannot make a numeric tensor with dtype {repr(dtype)}")
    else:
        return torch.from_numpy(np.array([], dtype=dtype)).dtype

violation(lhs, comparison, rhs)

Get the amount of constraint violation.

Parameters:

Name	Type	Description	Default
lhs	Union[float, Tensor]	The left-hand-side of the constraint. In the non-batched case, this is expected as a scalar. If it is given as an n-dimensional tensor where n is at least 1, this is considered as a batch of left-hand-side values.	required
comparison	str	The operator used for comparing the left-hand-side and the right-hand-side. Expected as a string. Acceptable values are: '<=', '==', '>='.	required
rhs	Union[float, Tensor]	The right-hand-side of the constraint. In the non-batched case, this is expected as a scalar. If it is given as an n-dimensional tensor where n is at least 1, this is considered as a batch of right-hand-side values.	required

Source code in evotorch/tools/constraints.py

def violation(
    lhs: Union[float, torch.Tensor],
    comparison: str,
    rhs: Union[float, torch.Tensor],
) -> torch.Tensor:
    """
    Get the amount of constraint violation.

    Args:
        lhs: The left-hand-side of the constraint. In the non-batched case,
            this is expected as a scalar. If it is given as an n-dimensional
            tensor where n is at least 1, this is considered as a batch of
            left-hand-side values.
        comparison: The operator used for comparing the left-hand-side and the
            right-hand-side. Expected as a string. Acceptable values are:
            '<=', '==', '>='.
        rhs: The right-hand-side of the constraint. In the non-batched case,
            this is expected as a scalar. If it is given as an n-dimensional
            tensor where n is at least 1, this is considered as a batch of
            right-hand-side values.
    Returns:
        The amount of violation of the constraint. A value of 0 means that
        the constraint is not violated at all. The returned violation amount(s)
        are always non-negative.
    """
    from ..decorators import expects_ndim

    return expects_ndim(_violation, (0, None, 0))(lhs, comparison, rhs)