diff --git a/docs/source/gene_values.md b/docs/source/gene_values.md index 0f9dc94..3712d71 100644 --- a/docs/source/gene_values.md +++ b/docs/source/gene_values.md @@ -262,6 +262,50 @@ python examples/example_gene_space.py +## Categorical Genes + +PyGAD 3.9.0 adds the `gene_categorical` parameter to the constructor of the `pygad.GA` class. It marks the genes whose values are categories with no order. + +For example, an activation function can be stored as `0` for `relu`, `1` for `tanh`, and `2` for `sigmoid`. The numbers are only labels. `tanh` is not between `relu` and `sigmoid`. + +SBX crossover and polynomial mutation treat a gene value as a point on a number line. They move it to nearby values. This is right for a learning rate or a number of neurons, but it has no meaning for a category. For a categorical gene, these two operators work differently: + +1. Polynomial mutation replaces the value with one of the other allowed values. Each of them is equally likely. `polynomial_mutation_eta` is not used for this gene. +2. SBX crossover gives the child the value of one of the two parents. Each parent is equally likely. + +The other crossover and mutation types already copy or pick allowed values, so `gene_categorical` does not change them. + +The `gene_categorical` parameter accepts: + +1. `None`: The default. Every gene is ordered, which is the behavior before PyGAD 3.9.0. +2. A single `bool`: Applies to all genes. +3. A list or tuple with one `bool` per gene. + +A categorical gene must have a finite gene space: a list of numbers, a `range`, or a dictionary with a `step`. A list that has `None` is not accepted because `None` stands for any value in the initialization range. Otherwise, PyGAD raises a `ValueError`. + +This example has a continuous learning rate, an integer number of neurons, and a categorical activation function. + +```python +import pygad + +def fitness_func(ga_instance, solution, solution_idx): + learning_rate, num_neurons, activation = solution + ... + +ga_instance = pygad.GA(num_generations=20, + num_parents_mating=4, + sol_per_pop=10, + num_genes=3, + fitness_func=fitness_func, + gene_space=[{'low': 0.0001, 'high': 0.1}, + {'low': 16, 'high': 129, 'step': 1}, + [0, 1, 2]], + gene_type=[float, int, int], + gene_categorical=[False, False, True], + crossover_type="sbx", + mutation_type="polynomial") +``` + @@ -633,6 +677,7 @@ Let us look at some examples. PyGAD applies the same conversion rules to generated and supplied initial populations, mutation candidates, and custom operator outputs. `on_parents`, `on_crossover`, and `on_mutation` receive converted values; any replacements returned or made in place by these callbacks are converted again before use. These rules apply whether `allow_duplicate_genes` is `True` or `False`. - Integer conversion truncates the fractional part towards zero. For example, `1.9` becomes `1` and `-1.9` becomes `-1`. Additive mutation adds the random value before converting the result. +- SBX crossover and polynomial mutation are the exception. Their results are rounded to the nearest integer, so `1.9` becomes `2` and `-1.9` becomes `-2`. For a gene with a finite gene space, the result moves to the nearest allowed value. Truncation would make the highest value of the gene almost impossible to reach and would pull values toward zero. - Random and adaptive mutation clip generated values to the limits of a NumPy integer gene type before casting. This applies to both additive and replacement mutation, including random candidates from `None` gene-space entries, constraint checks, and duplicate repair. For example, a `numpy.uint8` gene at `0` with an offset of `-1` stays at `0`, and a `numpy.int8` gene at `127` with an offset of `1` stays at `127`. Values within the type's limits keep the same conversion behavior. Python `int`, floating-point, and `object` genes keep their existing behavior. - Floating-point precision rounds before casting to the requested type. This avoids rounding in a narrow type such as `numpy.float16` before the final conversion. - Precision must be an integer or `None`. `None` keeps the value unrounded, `0` rounds to whole numbers, and a negative precision rounds to positions before the decimal point. For example, `[float, -1]` rounds to tens. Integer types may only be paired with `None`. diff --git a/docs/source/releases.md b/docs/source/releases.md index 72ed469..3884fbe 100644 --- a/docs/source/releases.md +++ b/docs/source/releases.md @@ -8,6 +8,15 @@ Release notes are listed from newest to oldest. Unreleased contains changes plan No changes yet. +## PyGAD 3.9.0 + +Release Date: October 16, 2026. + +1. [Roulette wheel selection](utils.md#roulette_wheel_selection) and [stochastic universal selection](utils.md#stochastic_universal_selection) sum fitness values in a floating-point type wide enough for the input. Large `int64`, `uint64`, `float16`, and `float32` fitness values no longer overflow and raise `ZeroDivisionError`. Object arrays of Python integers keep exact values. See [PR #381](https://github.com/ahmedfgad/GeneticAlgorithmPython/pull/381). +2. [SBX crossover](utils.md#sbx_crossover) and [polynomial mutation](utils.md#polynomial_mutationoffspring) round integer results to the nearest integer instead of truncating them. For a gene with a finite gene space, the result moves to the nearest allowed value. Before this change, the highest value of an integer gene was almost never produced, values were pulled toward zero, and SBX could not keep a parent's highest value. Results for integer genes change for the same random seed. Float genes are not affected. See [Conversion and Rounding Rules](gene_values.md#conversion-and-rounding-rules). +3. A new `gene_categorical` parameter marks genes whose values are categories with no order. For these genes, polynomial mutation picks one of the other allowed values at random and SBX crossover copies the value of one parent. The default `None` keeps the previous behavior. See [Categorical Genes](gene_values.md#categorical-genes). +4. Regression tests cover reaching both ends of integer and finite gene spaces, balanced steps up and down, negative ranges, stepped and uneven spaces, and every `gene_categorical` validation rule and operator behavior. The `pygad.helper` and `pygad.utils` submodule versions are `1.4.7` and `1.5.10`. + ## PyGAD 3.8.1 Release Date: October 11, 2026. diff --git a/docs/source/utils.md b/docs/source/utils.md index 635dedf..42c95df 100644 --- a/docs/source/utils.md +++ b/docs/source/utils.md @@ -266,6 +266,8 @@ Applies simulated binary crossover for numeric genes. The `sbx_crossover_eta` pa For each crossed gene, SBX produces two possible values on opposite sides of the parents' midpoint. PyGAD selects either value with probability `0.5`, avoiding the downward bias from always selecting the lower child. Equal parent values are copied unchanged. +For an integer gene, the child value is rounded to the nearest integer. For a gene with a finite gene space, it moves to the nearest allowed value. For a gene marked in [`gene_categorical`](gene_values.md#categorical-genes), the child copies the value of one parent, each with probability `0.5`. + ## `pygad.utils.mutation` Submodule The `pygad.utils.mutation` module has a class named `Mutation` with the supported mutation operations: @@ -351,6 +353,8 @@ The count-based and probability-based adaptive mutation methods use the same com Applies polynomial mutation to the passed two-dimensional offspring array in place and returns it. Each gene is selected with `mutation_probability`, or with probability `1 / num_genes` when that parameter is `None`. `polynomial_mutation_eta` controls the size of the change; higher values favor smaller changes. Bounds come from `init_range_low` and `init_range_high` for each gene, and mutated values are clipped to those bounds. Genes whose range has effectively zero width are skipped. When `allow_duplicate_genes=False`, the existing random duplicate-resolution helper is applied after changing a gene. +For an integer gene, the new value is rounded to the nearest integer. For a gene with a finite gene space, it moves to the nearest allowed value. For a gene marked in [`gene_categorical`](gene_values.md#categorical-genes), the new value is one of the other allowed values, each equally likely, and `polynomial_mutation_eta` is not used. + ### Mutation Helper Methods The `pygad.utils.mutation` module has some helper methods to assist applying the mutation operation: diff --git a/pygad/_version.py b/pygad/_version.py index e4e78c0..fcd7ddb 100644 --- a/pygad/_version.py +++ b/pygad/_version.py @@ -1 +1 @@ -__version__ = "3.8.1" +__version__ = "3.9.0" diff --git a/pygad/helper/__init__.py b/pygad/helper/__init__.py index 90f0727..99e094f 100644 --- a/pygad/helper/__init__.py +++ b/pygad/helper/__init__.py @@ -1,4 +1,4 @@ from pygad.helper import unique from pygad.helper import misc -__version__ = "1.4.6" +__version__ = "1.4.7" diff --git a/pygad/helper/misc.py b/pygad/helper/misc.py index c32c313..280c850 100644 --- a/pygad/helper/misc.py +++ b/pygad/helper/misc.py @@ -665,6 +665,12 @@ def convert_bounded_operator_gene_value(self, gene_index, value): lower = lower.item() if isinstance(lower, numpy.generic) else lower upper = upper.item() if isinstance(upper, numpy.generic) else upper dtype = self.get_gene_dtype(gene_index)[0] + # When the gene has a finite list of allowed values, move the + # generated decimal straight to the nearest allowed value. Rounding + # it to an integer first would round twice and favor lower values. + nearest_allowed_value = self._nearest_finite_gene_space_value(gene_index, value) + if nearest_allowed_value is not None: + value = nearest_allowed_value if numpy.issubdtype(numpy.dtype(dtype), numpy.integer): value = value.item() if isinstance(value, numpy.generic) else value type_limits = numpy.iinfo(self._get_gene_storage_dtype(dtype)) @@ -672,10 +678,13 @@ def convert_bounded_operator_gene_value(self, gene_index, value): last = min(math.floor(upper), int(type_limits.max)) if first > last: raise ValueError(f'The operator bounds have no value representable by gene_type for the gene at index {gene_index}.') + # Round to the nearest integer. Dropping the fraction instead + # would never reach the highest value and would pull every gene + # toward zero. # Real-coded arithmetic may round an integer type's maximum # up to the next power of two. Clip as Python integers before # casting so a valid bounded proposal does not overflow. - value = min(max(math.trunc(value), first), last) + value = min(max(math.floor(value + 0.5), first), last) converted = self.change_gene_dtype_and_round(gene_index, value) if not lower <= float(converted) <= upper: if numpy.issubdtype(numpy.dtype(self.get_gene_dtype(gene_index)[0]), numpy.integer): @@ -708,6 +717,124 @@ def convert_bounded_operator_gene_value(self, gene_index, value): converted = min(candidates, key=lambda candidate: abs(float(candidate) - float(converted))) return converted if dtype is object else dtype(converted) + def _nearest_finite_gene_space_value(self, gene_index, value): + """ + Return the allowed value of a finite gene space that is nearest + to a value generated by SBX crossover or polynomial mutation. + + A gene space is finite when it is a range, a dictionary with a + ``step``, or a list of numbers. Lists that contain ``None`` are + not finite because ``None`` stands for any value in the + initialization range. + + Parameters + ---------- + gene_index : int + Index of the gene whose space is used. + value : float + The decimal value generated by the operator. + + Returns + ------- + number or None + The nearest allowed value, or None when the gene has no + finite gene space. + """ + if self.gene_space is None: + return None + space = self.gene_space[gene_index] if self.gene_space_nested else self.gene_space + value = float(value) + if isinstance(space, range) or (isinstance(space, dict) and 'step' in space): + # The values are evenly spaced, so the nearest one is found + # from its position without listing all the values. + count = self._finite_gene_space_length(space) + low = space.start if isinstance(space, range) else space['low'] + step = space.step if isinstance(space, range) else space['step'] + position = math.floor((value - low) / step + 0.5) + position = min(max(position, 0), count - 1) + return self._finite_gene_space_value(space, position) + if isinstance(space, (list, tuple, numpy.ndarray)): + if any(item is None for item in space): + return None + return min(space, key=lambda allowed_value: abs(float(allowed_value) - value)) + return None + + def is_categorical_gene(self, gene_index): + """ + Check whether a gene is marked as categorical in ``gene_categorical``. + + Parameters + ---------- + gene_index : int + Index of the gene. + + Returns + ------- + bool + True when the gene is categorical. A GA instance saved by an + older PyGAD version has no ``gene_categorical`` attribute, so + its genes are treated as ordered. + """ + gene_categorical = getattr(self, 'gene_categorical', None) + if gene_categorical is None: + return False + return bool(gene_categorical[gene_index]) + + def pick_other_categorical_value(self, gene_index, current_value): + """ + Pick one allowed value of a categorical gene, other than its + current value, with every other value equally likely. + + Parameters + ---------- + gene_index : int + Index of the categorical gene. Its gene space is finite, which + ``gene_categorical`` validation guarantees. + current_value : number + The value the gene has now. + + Returns + ------- + number + The new value. When the gene has only one allowed value, that + value is returned. When the current value is not one of the + allowed values, any allowed value can be returned. + """ + space = self.gene_space[gene_index] if self.gene_space_nested else self.gene_space + if isinstance(space, range) or isinstance(space, dict): + # Evenly spaced values: work with positions so a long range is + # never listed in full. + count = self._finite_gene_space_length(space) + low = space.start if isinstance(space, range) else space['low'] + step = space.step if isinstance(space, range) else space['step'] + current_position = int(round((float(current_value) - low) / step)) + if not (0 <= current_position < count + and math.isclose(float(self._finite_gene_space_value(space, current_position)), float(current_value))): + current_position = None + allowed_value_at = lambda position: self._finite_gene_space_value(space, position) + else: + count = len(space) + current_position = None + for position, allowed_value in enumerate(space): + if float(allowed_value) == float(current_value): + current_position = position + break + allowed_value_at = lambda position: space[position] + + if count == 1: + new_position = 0 + elif current_position is None: + new_position = int(self.numpy_random_generator.randint(0, count)) + else: + # Draw from the other count - 1 positions, then skip over the + # current position so it can never be chosen. + new_position = int(self.numpy_random_generator.randint(0, count - 1)) + if new_position >= current_position: + new_position += 1 + # The chosen value is already allowed. The conversion only applies + # the gene type, for example turning 2.0 into 2 for an int gene. + return self.convert_bounded_operator_gene_value(gene_index, allowed_value_at(new_position)) + def is_bounded_operator_gene_value_in_space(self, gene_index, value): """Check a bounded operator against the original space, including None entries.""" if self.gene_space is None: diff --git a/pygad/pygad.py b/pygad/pygad.py index 497d876..6978d5f 100644 --- a/pygad/pygad.py +++ b/pygad/pygad.py @@ -69,7 +69,8 @@ def __init__(self, stop_criteria=None, parallel_processing=None, random_seed=None, - logger=None): + logger=None, + gene_categorical=None): """ The constructor of the GA class accepts all parameters required to create an instance of the GA class. It validates such parameters. @@ -114,6 +115,7 @@ def __init__(self, gene_space: It accepts a list of all possible values of the gene. This list is used in the mutation step. It should be used only if the gene space is a set of discrete values. No need for the 2 parameters (random_mutation_min_val and random_mutation_max_val) if the parameter gene_space exists. Added in PyGAD 2.5.0. In PyGAD 2.11.0, the gene_space can be assigned a dict. gene_constraint: It accepts a list of constraints for the genes. Each constraint is a Python function. Added in PyGAD 3.5.0. + gene_categorical: Marks the genes whose values are categories with no order, such as 0 for relu, 1 for tanh, and 2 for sigmoid. It accepts None (the default, every gene is ordered), a single bool applied to all genes, or a list with one bool per gene. A categorical gene must have a finite gene_space (a list of numbers, a range, or a dict with a step). For a categorical gene, polynomial mutation picks one of the other allowed values at random and SBX crossover copies the value of one parent. The other operators are not affected. Added in PyGAD 3.9.0. sample_size: To select a gene value that respects a constraint, this variable defines the size of the sample from which a value is selected randomly. Useful if either allow_duplicate_genes or gene_constraint is used. Added in PyGAD 3.5.0. on_start: Accepts a function/method to be called only once before the genetic algorithm starts its evolution. If function, then it must accept a single parameter representing the instance of the genetic algorithm. If method, then it must accept 2 parameters where the second one refers to the method's object. Added in PyGAD 2.6.0. @@ -183,7 +185,8 @@ def __init__(self, stop_criteria=stop_criteria, parallel_processing=parallel_processing, random_seed=random_seed, - logger=logger) + logger=logger, + gene_categorical=gene_categorical) except Exception as e: if hasattr(self, 'logger'): self.logger.exception(e) diff --git a/pygad/utils/__init__.py b/pygad/utils/__init__.py index d919162..31eb4ad 100644 --- a/pygad/utils/__init__.py +++ b/pygad/utils/__init__.py @@ -9,4 +9,4 @@ from pygad.utils import validation from pygad.utils import engine -__version__ = "1.5.8" +__version__ = "1.5.10" diff --git a/pygad/utils/crossover.py b/pygad/utils/crossover.py index 6ac0365..bfb6fb1 100644 --- a/pygad/utils/crossover.py +++ b/pygad/utils/crossover.py @@ -258,6 +258,14 @@ def sbx_crossover(self, parents, offspring_size): parent2_idx = (k + 1) % parents.shape[0] for gene_idx in range(offspring_size[1]): + if self.is_categorical_gene(gene_idx): + # Blending two categories has no meaning, so the child + # takes the value of one parent, each with equal chance. + if self.numpy_random_generator.random() < 0.5: + offspring[k, gene_idx] = parents[parent1_idx, gene_idx] + else: + offspring[k, gene_idx] = parents[parent2_idx, gene_idx] + continue range_min, range_max = self.get_bounded_operator_gene_range(gene_idx) lower, upper = float(range_min), float(range_max) p1 = float(numpy.clip(parents[parent1_idx, gene_idx], lower, upper)) diff --git a/pygad/utils/mutation.py b/pygad/utils/mutation.py index 8333e55..8b59cb2 100644 --- a/pygad/utils/mutation.py +++ b/pygad/utils/mutation.py @@ -416,6 +416,11 @@ def polynomial_mutation(self, offspring): selected_any_gene = False for gene_index in gene_indices: selected_any_gene = True + if self.is_categorical_gene(gene_index): + # A categorical value has no neighbors on a number line, + # so pick one of the other allowed values at random. + solution[gene_index] = self.pick_other_categorical_value(gene_index, solution[gene_index]) + continue lower, upper = map(float, self.get_bounded_operator_gene_range(gene_index)) if upper - lower < 1e-14: solution[gene_index] = self.convert_bounded_operator_gene_value(gene_index, lower) diff --git a/pygad/utils/parent_selection.py b/pygad/utils/parent_selection.py index 6237fdf..94b1daf 100644 --- a/pygad/utils/parent_selection.py +++ b/pygad/utils/parent_selection.py @@ -204,13 +204,13 @@ def roulette_wheel_selection(self, fitness, num_parents): if len(fitness.shape) > 1: # Multi-objective optimization problem. # Sum the fitness values of each solution to reduce the fitness from M-D array to just 1D array. - fitness = numpy.sum(fitness, axis=1) + fitness = numpy.sum(fitness, axis=1, dtype=numpy.result_type(fitness.dtype, float)) else: # Single-objective optimization problem. pass # Reaching this step confirms that fitness is a 1D array. - fitness_sum = numpy.sum(fitness) + fitness_sum = numpy.sum(fitness, dtype=numpy.result_type(fitness.dtype, float)) if fitness_sum == 0: self.logger.error("Cannot proceed because the sum of fitness values is zero. Cannot divide by zero.") raise ZeroDivisionError("Cannot proceed because the sum of fitness values is zero. Cannot divide by zero.") @@ -313,13 +313,13 @@ def stochastic_universal_selection(self, fitness, num_parents): if len(fitness.shape) > 1: # Multi-objective optimization problem. # Sum the fitness values of each solution to reduce the fitness from M-D array to just 1D array. - fitness = numpy.sum(fitness, axis=1) + fitness = numpy.sum(fitness, axis=1, dtype=numpy.result_type(fitness.dtype, float)) else: # Single-objective optimization problem. pass # Reaching this step confirms that fitness is a 1D array. - fitness_sum = numpy.sum(fitness) + fitness_sum = numpy.sum(fitness, dtype=numpy.result_type(fitness.dtype, float)) if fitness_sum == 0: self.logger.error("Cannot proceed because the sum of fitness values is zero. Cannot divide by zero.") raise ZeroDivisionError("Cannot proceed because the sum of fitness values is zero. Cannot divide by zero.") diff --git a/pygad/utils/report.py b/pygad/utils/report.py index 5064ea0..c1f2ba2 100644 --- a/pygad/utils/report.py +++ b/pygad/utils/report.py @@ -109,7 +109,7 @@ ("Population", ["num_generations", "num_parents_mating", "sol_per_pop", "num_genes", "init_range_low", "init_range_high", "gene_type", "gene_space", "allow_duplicate_genes", - "gene_constraint", "sample_size"]), + "gene_constraint", "gene_categorical", "sample_size"]), ("Parent selection", ["parent_selection_type", "K_tournament", "nsga3_num_divisions", "keep_parents", "keep_elitism"]), diff --git a/pygad/utils/validation.py b/pygad/utils/validation.py index 6abef7b..de926fb 100644 --- a/pygad/utils/validation.py +++ b/pygad/utils/validation.py @@ -349,6 +349,64 @@ def _validate_gene_constraint(self, gene_constraint): self._validate_callable_parameter(constraint, f'gene_constraint at index {index}', 2) for index, constraint in enumerate(gene_constraint)] + def _validate_gene_categorical(self, gene_categorical): + """ + Validate the ``gene_categorical`` parameter and store one bool per + gene in ``self.gene_categorical``. + + A categorical gene holds a choice whose values have no order. It + needs a finite gene space so the operators can pick another + allowed value. + + Parameters + ---------- + gene_categorical : None, bool, list, or tuple + None marks every gene as ordered. A single bool applies to all + genes. A list or tuple gives one bool per gene. + + Raises + ------ + TypeError + If the value, or one of its entries, is not a bool. + ValueError + If the list length differs from the number of genes, or a + categorical gene has no finite gene space. + """ + boolean_types = (bool, numpy.bool_) + if gene_categorical is None: + self.gene_categorical = [False] * self.num_genes + return + if type(gene_categorical) in boolean_types: + gene_categorical = [bool(gene_categorical)] * self.num_genes + elif isinstance(gene_categorical, (list, tuple)): + if len(gene_categorical) != self.num_genes: + self.valid_parameters = False + raise ValueError(f"The length of gene_categorical ({len(gene_categorical)}) must equal the number of genes ({self.num_genes}).") + for index, entry in enumerate(gene_categorical): + if type(entry) not in boolean_types: + self.valid_parameters = False + raise TypeError(f"Each entry in gene_categorical must be a bool, but the entry at index {index} is {entry!r} of type {type(entry)}.") + gene_categorical = [bool(entry) for entry in gene_categorical] + else: + self.valid_parameters = False + raise TypeError(f"gene_categorical must be None, a bool, or a list or tuple of bools, but {type(gene_categorical)} found.") + + # Every categorical gene needs a fixed list of allowed values. + for gene_index, is_categorical in enumerate(gene_categorical): + if not is_categorical: + continue + space = None + if self.gene_space is not None: + space = self.gene_space[gene_index] if self.gene_space_nested else self.gene_space + is_finite = (isinstance(space, range) + or (isinstance(space, dict) and 'step' in space) + or (isinstance(space, (list, tuple, numpy.ndarray)) + and not any(item is None for item in space))) + if not is_finite: + self.valid_parameters = False + raise ValueError(f"The gene at index {gene_index} is marked as categorical in gene_categorical, so its gene_space must be a list of numbers, a range, or a dict with a 'step'. Found: {space!r}.") + self.gene_categorical = gene_categorical + def _validate_crossover(self, crossover_type, crossover_probability, sbx_crossover_eta=30): """Resolve crossover and validate its probability and finite distribution index.""" operators = {'single_point': 'single_point_crossover', 'two_points': 'two_points_crossover', @@ -717,7 +775,8 @@ def validate_parameters(self, stop_criteria, parallel_processing, random_seed, - logger): + logger, + gene_categorical=None): """ Validate every parameter passed to ``pygad.GA.__init__`` and store the parsed values on the GA instance. This method is @@ -759,6 +818,8 @@ def validate_parameters(self, if self.gene_space_nested and len(gene_space) != self.num_genes: self.valid_parameters = False raise ValueError(f"When gene_space is nested, its length ({len(gene_space)}) must equal the number of genes ({self.num_genes}).") + # Needs the validated gene_space, so it comes after the length check. + self._validate_gene_categorical(gene_categorical) self._validate_mutation_range(random_mutation_min_val, random_mutation_max_val) self.num_parents_mating = self._validate_integer_parameter( diff --git a/tests/test_bounded_operator_rounding.py b/tests/test_bounded_operator_rounding.py new file mode 100644 index 0000000..a9f8e80 --- /dev/null +++ b/tests/test_bounded_operator_rounding.py @@ -0,0 +1,152 @@ +""" +Tests for how SBX crossover and polynomial mutation convert their real +valued results back into integer and finite gene values. + +Both operators compute a decimal number between the gene bounds. The +decimal must then become a value the gene accepts. These tests check +that this conversion picks the nearest accepted value. Chopping the +fraction instead makes the highest value unreachable and pulls every +gene toward zero. +""" + +import numpy + +import pygad + + +# Number of offspring rows used to measure each distribution. It is large +# enough to make the shares below stable for a fixed seed. +NUM_TRIALS = 3000 + + +def _zero_fitness(ga, solution, solution_idx): + return 0.0 + + +def _make_one_gene_ga(gene_space, gene_type=int, **operator_settings): + """Build a GA with a single gene so each operator can be called directly.""" + return pygad.GA(num_generations=1, + num_parents_mating=2, + fitness_func=_zero_fitness, + sol_per_pop=4, + num_genes=1, + gene_space=[gene_space], + gene_type=gene_type, + mutation_num_genes=1, + random_seed=1, + suppress_warnings=True, + **operator_settings) + + +def _mutate_from(ga, start_value): + """Apply polynomial mutation to many copies of one value and return the results.""" + offspring = numpy.full((NUM_TRIALS, 1), start_value) + return ga.polynomial_mutation(offspring)[:, 0].astype(float) + + +def _share(values, condition): + return float(numpy.mean(condition(values))) + + +def test_polynomial_mutation_reaches_the_highest_integer_value(): + # A gene with the values 0 to 4 starting at 3 must sometimes move to 4. + ga = _make_one_gene_ga({"low": 0, "high": 5, "step": 1}, + mutation_type="polynomial", + polynomial_mutation_eta=5) + values = _mutate_from(ga, 3) + assert _share(values, lambda v: v == 4) > 0.05 + + +def test_polynomial_mutation_reaches_the_last_value_of_a_list_space(): + # The same rule applies to a gene space given as a list. + ga = _make_one_gene_ga([0, 1, 2, 3, 4], + mutation_type="polynomial", + polynomial_mutation_eta=5) + values = _mutate_from(ga, 3) + assert _share(values, lambda v: v == 4) > 0.05 + + +def test_polynomial_mutation_reaches_the_highest_value_of_a_continuous_integer_space(): + # A continuous gene space used with an integer gene type. The 'high' + # value is excluded, so the allowed integers are 0 to 3. + ga = _make_one_gene_ga({"low": 0, "high": 4}, + mutation_type="polynomial", + polynomial_mutation_eta=5) + values = _mutate_from(ga, 2) + assert _share(values, lambda v: v == 3) > 0.05 + + +def test_polynomial_mutation_moves_up_and_down_equally_often(): + # From the middle value, moving down and moving up must be about + # equally likely. Chopping the fraction makes moving down far more + # likely than moving up. + ga = _make_one_gene_ga({"low": 0, "high": 5, "step": 1}, + mutation_type="polynomial", + polynomial_mutation_eta=5) + values = _mutate_from(ga, 2) + moved_down = _share(values, lambda v: v < 2) + moved_up = _share(values, lambda v: v > 2) + assert abs(moved_down - moved_up) < 0.05 + + +def test_polynomial_mutation_is_balanced_on_a_negative_range(): + # Chopping the fraction pulls values toward zero, which is upward for + # negative numbers. Starting at -2 in the range -4 to 0 checks that. + ga = _make_one_gene_ga({"low": -4, "high": 1, "step": 1}, + mutation_type="polynomial", + polynomial_mutation_eta=5) + values = _mutate_from(ga, -2) + moved_down = _share(values, lambda v: v < -2) + moved_up = _share(values, lambda v: v > -2) + assert abs(moved_down - moved_up) < 0.05 + + +def test_polynomial_mutation_is_balanced_on_a_stepped_space(): + # With the values 0, 2, 4, 6, 8 the decimal result must go to the + # nearest allowed value. Rounding it to an integer first would send + # results such as 5.2 to 4 instead of 6. + ga = _make_one_gene_ga({"low": 0, "high": 9, "step": 2}, + mutation_type="polynomial", + polynomial_mutation_eta=5) + values = _mutate_from(ga, 4) + moved_down = _share(values, lambda v: v < 4) + moved_up = _share(values, lambda v: v > 4) + assert abs(moved_down - moved_up) < 0.05 + + +def test_polynomial_mutation_keeps_values_inside_the_gene_space(): + # Rounding must never produce a value outside the allowed values. + spaces = [{"low": 0, "high": 5, "step": 1}, + {"low": 0, "high": 9, "step": 2}, + {"low": -4, "high": 1, "step": 1}, + [16, 32, 64]] + for space in spaces: + ga = _make_one_gene_ga(space, + mutation_type="polynomial", + polynomial_mutation_eta=5) + low, high = ga.get_bounded_operator_gene_range(0) + for start_value in (low, high): + for value in _mutate_from(ga, start_value): + assert ga.is_gene_value_in_space(0, value, value), (space, value) + + +def test_sbx_crossover_can_keep_the_highest_parent_value(): + # With parents 1 and 4 in the range 0 to 4, a child must sometimes + # keep the value 4. Chopping the fraction turns it into 3 every time. + ga = _make_one_gene_ga({"low": 0, "high": 5, "step": 1}, + crossover_type="sbx", + sbx_crossover_eta=5) + parents = numpy.array([[1], [4]]) + children = ga.sbx_crossover(parents, (NUM_TRIALS, 1))[:, 0].astype(float) + assert _share(children, lambda v: v == 4) > 0.3 + assert _share(children, lambda v: v == 1) > 0.3 + + +def test_float_genes_are_not_rounded(): + # Float genes keep their decimal values. + ga = _make_one_gene_ga({"low": 0.0, "high": 4.0}, + gene_type=float, + mutation_type="polynomial", + polynomial_mutation_eta=5) + values = _mutate_from(ga, 2.0) + assert _share(values, lambda v: v != numpy.round(v)) > 0.9 diff --git a/tests/test_categorical_genes.py b/tests/test_categorical_genes.py new file mode 100644 index 0000000..fa9d0dd --- /dev/null +++ b/tests/test_categorical_genes.py @@ -0,0 +1,215 @@ +""" +Tests for the gene_categorical parameter. + +A categorical gene holds a choice from a list where the order has no +meaning, such as an activation function stored as 0 for relu, 1 for tanh +and 2 for sigmoid. SBX crossover and polynomial mutation normally treat +the gene value as a point on a number line. For a categorical gene, they +must treat it as a choice instead: + +1. Polynomial mutation picks one of the other allowed values at random. +2. SBX crossover gives the child the value of one of the two parents. +""" + +import numpy +import pytest + +import pygad + + +# Number of offspring rows used to measure each distribution. +NUM_TRIALS = 3000 + + +def _zero_fitness(ga, solution, solution_idx): + return 0.0 + + +def _make_ga(gene_space, gene_categorical, gene_type=int, num_genes=1, **other_settings): + """Build a small GA whose operators can be called directly.""" + settings = dict(num_generations=1, + num_parents_mating=2, + fitness_func=_zero_fitness, + sol_per_pop=4, + num_genes=num_genes, + gene_space=gene_space, + gene_type=gene_type, + gene_categorical=gene_categorical, + mutation_type="polynomial", + polynomial_mutation_eta=5, + mutation_num_genes=1, + random_seed=1, + suppress_warnings=True) + settings.update(other_settings) + return pygad.GA(**settings) + + +def _shares(values, allowed_values): + """Return the share of each allowed value among the given values.""" + values = numpy.asarray(values, dtype=float) + return {allowed: float(numpy.mean(values == allowed)) for allowed in allowed_values} + + +# Validation of the parameter. + +def test_default_marks_every_gene_as_ordered(): + ga = _make_ga(gene_space=[[0, 1, 2], [0, 1, 2]], gene_categorical=None, num_genes=2) + assert ga.gene_categorical == [False, False] + + +def test_single_boolean_applies_to_every_gene(): + ga = _make_ga(gene_space=[[0, 1, 2], [5, 6]], gene_categorical=True, num_genes=2) + assert ga.gene_categorical == [True, True] + + +def test_list_marks_each_gene_separately(): + ga = _make_ga(gene_space=[{"low": 0.0, "high": 1.0}, [0, 1, 2]], + gene_categorical=[False, True], + gene_type=[float, int], + num_genes=2) + assert ga.gene_categorical == [False, True] + + +def test_rejects_a_value_that_is_not_a_boolean_or_a_list(): + with pytest.raises(TypeError, match="gene_categorical"): + _make_ga(gene_space=[[0, 1, 2]], gene_categorical="yes") + + +def test_rejects_a_list_with_a_non_boolean_entry(): + with pytest.raises(TypeError, match="gene_categorical"): + _make_ga(gene_space=[[0, 1, 2]], gene_categorical=[1]) + + +def test_rejects_a_list_with_the_wrong_length(): + with pytest.raises(ValueError, match="gene_categorical"): + _make_ga(gene_space=[[0, 1, 2], [0, 1, 2]], gene_categorical=[True], num_genes=2) + + +def test_rejects_a_categorical_gene_without_a_gene_space(): + with pytest.raises(ValueError, match="gene_categorical"): + _make_ga(gene_space=None, gene_categorical=True) + + +def test_rejects_a_categorical_gene_with_a_continuous_range(): + with pytest.raises(ValueError, match="gene_categorical"): + _make_ga(gene_space=[{"low": 0, "high": 5}], gene_categorical=True) + + +def test_rejects_a_categorical_gene_whose_list_contains_none(): + # None in a gene space stands for any value in the initialization + # range, so the allowed values are not a fixed list. + with pytest.raises(ValueError, match="gene_categorical"): + _make_ga(gene_space=[[0, 1, None]], gene_categorical=True) + + +def test_accepts_a_categorical_gene_with_a_stepped_range(): + ga = _make_ga(gene_space=[{"low": 0, "high": 5, "step": 1}], gene_categorical=True) + assert ga.gene_categorical == [True] + + +# Polynomial mutation. + +def test_polynomial_mutation_picks_every_other_category_equally_often(): + ga = _make_ga(gene_space=[[0, 1, 2, 3, 4]], gene_categorical=True) + offspring = numpy.full((NUM_TRIALS, 1), 0) + values = ga.polynomial_mutation(offspring)[:, 0] + shares = _shares(values, [0, 1, 2, 3, 4]) + # The mutated gene must change, and each other category is equally likely. + assert shares[0] == 0.0 + for category in (1, 2, 3, 4): + assert 0.2 < shares[category] < 0.3, shares + + +def test_polynomial_mutation_uses_only_allowed_values_of_an_uneven_list(): + ga = _make_ga(gene_space=[[16, 32, 64]], gene_categorical=True) + offspring = numpy.full((NUM_TRIALS, 1), 32) + values = ga.polynomial_mutation(offspring)[:, 0] + shares = _shares(values, [16, 32, 64]) + assert shares[32] == 0.0 + assert 0.45 < shares[16] < 0.55 + assert 0.45 < shares[64] < 0.55 + + +def test_polynomial_mutation_keeps_the_only_allowed_value(): + ga = _make_ga(gene_space=[[7]], gene_categorical=True) + offspring = numpy.full((10, 1), 7) + values = ga.polynomial_mutation(offspring)[:, 0] + assert numpy.all(values == 7) + + +def test_polynomial_mutation_still_moves_ordered_genes_on_the_number_line(): + # The float gene is ordered and must still get small decimal steps, + # while the categorical gene only takes values from its list. + ga = _make_ga(gene_space=[{"low": 0.0, "high": 10.0}, [0, 1, 2]], + gene_categorical=[False, True], + gene_type=[float, int], + num_genes=2, + mutation_num_genes=2) + offspring = numpy.array([[5.0, 0]] * NUM_TRIALS, dtype=object) + result = ga.polynomial_mutation(offspring) + float_gene = result[:, 0].astype(float) + categorical_gene = result[:, 1].astype(float) + assert numpy.mean(float_gene != numpy.round(float_gene)) > 0.9 + assert set(categorical_gene) == {1.0, 2.0} + + +# SBX crossover. + +def test_sbx_crossover_copies_one_parent_value_for_a_categorical_gene(): + ga = _make_ga(gene_space=[[0, 1, 2, 3, 4]], + gene_categorical=True, + crossover_type="sbx", + sbx_crossover_eta=5) + parents = numpy.array([[1], [4]]) + children = ga.sbx_crossover(parents, (NUM_TRIALS, 1))[:, 0] + shares = _shares(children, [0, 1, 2, 3, 4]) + assert shares[0] == shares[2] == shares[3] == 0.0 + assert 0.45 < shares[1] < 0.55 + assert 0.45 < shares[4] < 0.55 + + +# Interaction with other gene rules and with a full run. + +def test_categorical_mutation_respects_a_gene_constraint(): + # The constraint forbids the value 2. + ga = _make_ga(gene_space=[[0, 1, 2, 3]], + gene_categorical=True, + gene_constraint=[lambda solution, values: [value for value in values if value != 2]]) + offspring = numpy.full((NUM_TRIALS, 1), 0) + values = ga.polynomial_mutation(offspring)[:, 0] + assert 2 not in set(values.astype(int)) + + +def test_categorical_mutation_respects_allow_duplicate_genes(): + ga = _make_ga(gene_space=[0, 1, 2, 3, 4], + gene_categorical=True, + num_genes=2, + allow_duplicate_genes=False) + offspring = numpy.array([[0, 1]] * NUM_TRIALS) + result = ga.polynomial_mutation(offspring) + assert numpy.all(result[:, 0] != result[:, 1]) + + +def test_full_run_with_mixed_genes_keeps_every_value_allowed(): + def fitness(ga, solution, solution_idx): + return float(solution[0]) - float(solution[1]) + + ga = pygad.GA(num_generations=15, + num_parents_mating=4, + fitness_func=fitness, + sol_per_pop=10, + num_genes=3, + gene_space=[{"low": 0.0, "high": 1.0}, + {"low": 16, "high": 129, "step": 1}, + [0, 1, 2]], + gene_type=[float, int, int], + gene_categorical=[False, False, True], + crossover_type="sbx", + mutation_type="polynomial", + random_seed=3, + suppress_warnings=True) + ga.run() + population = numpy.asarray(ga.population, dtype=float) + assert numpy.all((population[:, 0] >= 0.0) & (population[:, 0] <= 1.0)) + assert numpy.all((population[:, 1] >= 16) & (population[:, 1] <= 128)) + assert set(population[:, 2]) <= {0.0, 1.0, 2.0} diff --git a/tests/test_parent_selection_regressions.py b/tests/test_parent_selection_regressions.py index 3bffd65..65ebe98 100644 --- a/tests/test_parent_selection_regressions.py +++ b/tests/test_parent_selection_regressions.py @@ -150,3 +150,65 @@ def test_sus_selected_parents_do_not_alias_the_population(): parents[:] = -1 numpy.testing.assert_array_equal(ga_instance.population, original) + + +@pytest.mark.parametrize("selection", ["rws", "sus"]) +@pytest.mark.parametrize("dtype,value", [ + (numpy.int64, 2**62), + (numpy.uint64, 2**63), + (numpy.float16, 40000), + (numpy.float32, 2e38), +]) +@pytest.mark.parametrize("multi_objective", [False, True]) +def test_fitness_sum_does_not_overflow_during_selection( + selection, dtype, value, multi_objective): + def large_fitness(ga_instance, solution, solution_idx): + fitness = dtype(value) + return [fitness, fitness] if multi_objective else fitness + + ga_instance = pygad.GA(num_generations=1, + num_parents_mating=2, + fitness_func=large_fitness, + initial_population=[[1, 2], [3, 4], [5, 6], [7, 8]], + parent_selection_type=selection, + mutation_type=None, + random_seed=17, + suppress_warnings=True) + ga_instance.run() + + assert ga_instance.generations_completed == 1 + assert ga_instance.last_generation_parents_indices.shape == (2,) + numpy.testing.assert_array_equal( + ga_instance.last_generation_parents, + ga_instance.population[ga_instance.last_generation_parents_indices]) + assert numpy.all(ga_instance.last_generation_fitness > 0) + + +@pytest.mark.parametrize("selection", ["rws", "sus"]) +def test_probability_selection_still_rejects_zero_sum_fitness(selection): + ga_instance = pygad.GA(num_generations=1, + num_parents_mating=2, + fitness_func=fitness_func, + initial_population=[[1, 2], [3, 4], [5, 6], [7, 8]], + parent_selection_type=selection, + mutation_type=None, + suppress_warnings=True) + with pytest.raises(ZeroDivisionError, match="sum of fitness values is zero"): + ga_instance.select_parents(numpy.zeros(4), 2) + + +@pytest.mark.parametrize("selection", ["rws", "sus"]) +def test_probability_selection_preserves_python_integer_fitness(selection): + ga_instance = pygad.GA(num_generations=1, + num_parents_mating=2, + fitness_func=fitness_func, + initial_population=[[1, 2], [3, 4], [5, 6], [7, 8]], + parent_selection_type=selection, + mutation_type=None, + random_seed=17, + suppress_warnings=True) + fitness = numpy.array([2**1500] * 4, dtype=object) + parents, indices = ga_instance.select_parents(fitness, 2) + + assert indices.shape == (2,) + numpy.testing.assert_array_equal(parents, ga_instance.population[indices])