feat(problems): gradients for batch 10b's functions, and gradients and constraint values through Shifted and Rotated

AGENTS.md

@@ -136,10 +136,10 @@ Stops: `Stop::target(score)` (at least as good), `generations(n)`, `evaluations(
 - Maximize is the default; use `.minimize()`, don't negate.
 - `None`, `Fitness::invalid()` and NaN are invalid: worse than everything.
 - **Constraints:** return `(score, violation)`, 0 when feasible, adding up `constraint::at_most(value, limit)`, `at_least`, `equal(value, target, tolerance)`. Deb's rules: feasible beats infeasible, then score or violation decides. Select with `Tournament` or `Rank`: roulette and SUS give infeasible solutions no weight. `Penalty::new(weight)?.fitness(objective, score, violation)` is a static penalty instead.
-- **Test problems:** `problems::{Sphere, AxisParallelEllipsoid, Schwefel1_2, Rastrigin, Rosenbrock, Ackley, Griewank, Schwefel2_26, Levy, Zakharov, StyblinskiTang, Michalewicz, Schwefel2_21, Schwefel2_22, DixonPrice, Trid, Powell, SumOfDifferentPowers, Step, Quartic, Penalized1, Penalized2, HighConditionedElliptic, BentCigar, Discus, DifferentPowers, BucheRastrigin, NonContinuousRastrigin, Weierstrass, Katsuura, HappyCat, HgBat, SchafferF7, RotatedHyperEllipsoid}::new(n)` (`Powell` takes a multiple of 4; `Quartic::noisy(n)` adds noise drawn from the genome) and `problems::{Himmelblau, Branin, GoldsteinPrice, SixHumpCamel, Hartmann3, Hartmann6, Shekel5, Shekel7, Shekel10, Easom, Eggholder, SchafferF6, Beale, Booth, Matyas, Bohachevsky1, Bohachevsky2, Bohachevsky3, ThreeHumpCamel, Langermann, ShekelFoxholes, Kowalik}` are fitness functions for `Engine::new(algorithm, problem)`, all minimized. The `problems::Problem` trait gives `representation()` (the bounds), `optimum()` (`value()`, `solutions()`), `reference()`; `problems::all()` lists them as `Box<dyn DynProblem>`. `problems::Shifted::new(problem, seed)` and `problems::Rotated::new(problem, seed)` make CEC/BBOB-style instances of any of them (e.g. CEC 2005's F10: `Rotated::new(Shifted::new(Rastrigin::new(n), seed), seed)`), keeping the optimum's value. Constrained, with fitness `(score, violation)` and `constraints(&x)` (`g <= 0`, then `h = 0`): `problems::cec2006::{G01, …, G24}` (equalities met within `EQUALITY_TOLERANCE` = 1e-4; `with_tolerance(δ)` for the problems with equalities, e.g. `G03::with_tolerance(δ)`), and `problems::engineering::{WeldedBeam, WeldedBeamRagsdell, PressureVessel, TensionCompressionSpring, SpeedReducer, ThreeBarTruss, CantileverBeam, CarSideImpact}`. `PressureVessel` and `SpeedReducer` round their discrete genes when evaluated; `design(&x)` gives the rounded design. `engineering::GearTrain` has an `Integer` genome and isn't in `all()`. `Optimum::is_proven()` is false for a best known value.
+- **Test problems:** `problems::{Sphere, AxisParallelEllipsoid, Schwefel1_2, Rastrigin, Rosenbrock, Ackley, Griewank, Schwefel2_26, Levy, Zakharov, StyblinskiTang, Michalewicz, Schwefel2_21, Schwefel2_22, DixonPrice, Trid, Powell, SumOfDifferentPowers, Step, Quartic, Penalized1, Penalized2, HighConditionedElliptic, BentCigar, Discus, DifferentPowers, BucheRastrigin, NonContinuousRastrigin, Weierstrass, Katsuura, HappyCat, HgBat, SchafferF7, RotatedHyperEllipsoid}::new(n)` (`Powell` takes a multiple of 4; `Quartic::noisy(n)` adds noise drawn from the genome) and `problems::{Himmelblau, Branin, GoldsteinPrice, SixHumpCamel, Hartmann3, Hartmann6, Shekel5, Shekel7, Shekel10, Easom, Eggholder, SchafferF6, Beale, Booth, Matyas, Bohachevsky1, Bohachevsky2, Bohachevsky3, ThreeHumpCamel, Langermann, ShekelFoxholes, Kowalik}` are fitness functions for `Engine::new(algorithm, problem)`, all minimized. The `problems::Problem` trait gives `representation()` (the bounds), `optimum()` (`value()`, `solutions()`), `reference()`; `problems::all()` lists them as `Box<dyn DynProblem>`. `problems::Shifted::new(problem, seed)` and `problems::Rotated::new(problem, seed)` make CEC/BBOB-style instances of any of them (e.g. CEC 2005's F10: `Rotated::new(Shifted::new(Rastrigin::new(n), seed), seed)`), keeping the optimum's value and what the wrapped problem provides: its gradient (by the chain rule through the rotation), constraint values and their Jacobian. Constrained, with fitness `(score, violation)` and `constraints(&x)` (`g <= 0`, then `h = 0`): `problems::cec2006::{G01, …, G24}` (equalities met within `EQUALITY_TOLERANCE` = 1e-4; `with_tolerance(δ)` for the problems with equalities, e.g. `G03::with_tolerance(δ)`), and `problems::engineering::{WeldedBeam, WeldedBeamRagsdell, PressureVessel, TensionCompressionSpring, SpeedReducer, ThreeBarTruss, CantileverBeam, CarSideImpact}`. `PressureVessel` and `SpeedReducer` round their discrete genes when evaluated; `design(&x)` gives the rounded design. `engineering::GearTrain` has an `Integer` genome and isn't in `all()`. `Optimum::is_proven()` is false for a best known value.
 - **Extras:** return `Evaluated::new(value, info)` (`value` any of the above, `info` any `Send + Sync + 'static` type, e.g. a struct with a penalty's terms) to keep what the fitness function computed. Read it by type: `outcome.best_info::<T>()`, `snapshot.info::<T>(genome)` and `snapshot.best_info::<T>()` in `.on_generation`, `hall_of_fame.info::<T>(genome)`, and in `MultiEngine` `snapshot.info` and `outcome.info(genome)` for the front; `None` for another type. Never used by the search. Kept by genome for the population, the discarded and the best (copies share it); not in checkpoints.
 - **Gradients:** `Differentiable(|x: &Reals, gradient: &mut [f64]| value)`, for gradient-based methods; see [Gradients](#gradients-supplying-them).
-- **Constraint values:** `Constrained::new(m, |x: &Reals, g: &mut [f64]| score)` writes the values of m constraints `gᵢ(x) <= 0`; `Constrained::differentiable(m, |x, gradient, g, jacobian| score)` also the gradient and the Jacobian (`jacobian[i * n + j]` = ∂gᵢ/∂xⱼ). Either is `(score, Σ max(0, gᵢ))` for any algorithm, and gives the values one by one to those that use them (`Mma`). The CEC 2006 problems with inequalities only and the engineering problems give their values (`problem.provides().inequalities`), not their gradients.
+- **Constraint values:** `Constrained::new(m, |x: &Reals, g: &mut [f64]| score)` writes the values of m constraints `gᵢ(x) <= 0`; `Constrained::differentiable(m, |x, gradient, g, jacobian| score)` also the gradient and the Jacobian (`jacobian[i * n + j]` = ∂gᵢ/∂xⱼ). Either is `(score, Σ max(0, gᵢ))` for any algorithm, and gives the values one by one to those that use them (`Mma`). The CEC 2006 problems with inequalities only and the engineering problems give their values (`problem.provides().inequalities`), shifted or rotated too, not their gradients.
 - **Batch:** `Batch(|genomes: &[&G]| -> Vec<T>)` scores a generation in one call, in order (SIMD, GPU, remote), in `Engine` or `MultiEngine`; the slice can be empty. A wrong count is `Error::FitnessCount`. See `examples/gpu` (wgpu).
 
 ## Templates
@@ -909,7 +909,7 @@ How fitness functions give gradients to gradient-based methods ([L-BFGS-B](#l-bf
 
 - `Differentiable(|x: &Reals, gradient: &mut [f64]| value)` writes the gradient (zeroed, one value per gene) and returns the value; any algorithm takes it as a plain fitness function. `Batch(Differentiable(|xs: &[&Reals], gradients: &mut [f64]| values))`: flat, row-major, a row per genome.
 - A `FitnessFunction` declares it with `fn provides(&self) -> Provided { Provided::GRADIENT }` and writes it in `fn evaluate_with(&self, x, extras: &mut Extras<'_>)` when `extras.gradient()` is `Some` (`genoxide::engine::{Extras, Provided}`); the value must be `evaluate`'s, to the bit.
-- The smooth test problems supply theirs: every classic function in `problems` but `Eggholder`, `Schwefel2_21` and `Schwefel2_22`, and not yet the CEC- and BBOB-style ones (`SumOfDifferentPowers` to `RotatedHyperEllipsoid`) or the `Shifted` and `Rotated` wrappers; `problem.provides().gradient`.
+- The test problems supply theirs: every classic function in `problems` but `Eggholder`, `Schwefel2_21`, `Schwefel2_22`, `Step`, `NonContinuousRastrigin`, `Katsuura` and `Quartic::noisy`, whose derivative is undefined or 0 on sets of positive measure (at an isolated cone or cusp, such as `Ackley`'s or `HappyCat`'s, the kinked term's gradient is 0); `Shifted` and `Rotated` pass the wrapped problem's on (`Mᵀ ∇f` for a rotation); `problem.provides().gradient`.
 - `gradient::check(&function, &x)?` compares a supplied gradient with central differences: `.largest()` about 1e-10 when right, `.worst_gene()`.
 - An algorithm's `gradient::Gradients` setting: `Auto` (default: supplied if provided, else forward differences, n evaluations per gradient, up to `gradient::AUTO_LIMIT` = 10⁴ genes), `Supplied` (an error at the start of a run without one), `Forward { step: None }`, `Central { step: None }` (2n per gradient, more accurate). Finite differences count towards `Stop::evaluations`.
 - A NaN in a gradient follows the `NanPolicy`: invalid fitness, or `Error::NanFitness`.

docs/features.md

@@ -73,7 +73,7 @@ What genoxide has on main; [docs.rs](https://docs.rs/genoxide) documents the lat
   - **Constraints:** the values of `Constrained` fitness functions and of the constrained test problems, a Gaussian process per constraint, and the acquisition weighed by the probability of feasibility (Gardner et al. 2014); before a feasible point, that probability alone.
   - **Integer genomes:** the genes rounded inside the kernel (Garrido-Merchán and Hernández-Lobato 2020), the acquisition maximized on the lattice by a hill climb (every point of a small lattice at once), the run finished once the lattice is evaluated.
 - **Gaussian processes** (`model::gp`, unstable for one release): regression with a constant mean, an ARD Matérn 5/2 or squared exponential kernel, no noise by default (the model interpolates a deterministic function) or learned noise, inputs scaled to the unit cube and outputs standardized; hyperparameters by maximum marginal likelihood (Rasmussen and Williams, eq. 2.30 and 5.9) with genoxide's L-BFGS-B from fixed and seeded random starts; the posterior mean and variance with their gradients; Cholesky factorizations with a growing jitter. In Python as `gx.model.gp`.
-- **Test problems** (`problems`): Sphere, the axis-parallel ellipsoid, Schwefel 1.2 and 2.26, Rastrigin, Rosenbrock, Ackley, Griewank, Levy, Zakharov, Styblinski-Tang, Michalewicz, Himmelblau, Branin, Goldstein-Price, the six-hump camel, Hartmann's functions in 3 and 6 dimensions, Shekel's with 5, 7 and 10 wells, Easom, the eggholder, Schaffer's F6, Schwefel 2.21 and 2.22, Dixon-Price, Trid, Powell's singular function, Beale, Booth, Matyas, Bohachevsky's three functions, the three-hump camel, Langermann, Shekel's foxholes, Kowalik, the sum of different powers, the step function, the quartic (with or without noise), Yao, Liu and Lin's two penalized functions, the high-conditioned elliptic, the bent cigar, the discus, BBOB's different powers, Büche-Rastrigin, the non-continuous Rastrigin, Weierstrass, Katsuura, HappyCat, HGBat, Schaffer's F7 and the rotated hyper-ellipsoid, each with its bounds, known optimum (or best known, for those found numerically) and reference, in Rust and Python, and the analytic gradient of the smooth ones up to Kowalik and Powell (all but the eggholder and Schwefel 2.21 and 2.22, which aren't differentiable; not yet the CEC- and BBOB-style ones); and shift and rotation wrappers, generated from a seed, for CEC- and BBOB-style instances of any of them.
+- **Test problems** (`problems`): Sphere, the axis-parallel ellipsoid, Schwefel 1.2 and 2.26, Rastrigin, Rosenbrock, Ackley, Griewank, Levy, Zakharov, Styblinski-Tang, Michalewicz, Himmelblau, Branin, Goldstein-Price, the six-hump camel, Hartmann's functions in 3 and 6 dimensions, Shekel's with 5, 7 and 10 wells, Easom, the eggholder, Schaffer's F6, Schwefel 2.21 and 2.22, Dixon-Price, Trid, Powell's singular function, Beale, Booth, Matyas, Bohachevsky's three functions, the three-hump camel, Langermann, Shekel's foxholes, Kowalik, the sum of different powers, the step function, the quartic (with or without noise), Yao, Liu and Lin's two penalized functions, the high-conditioned elliptic, the bent cigar, the discus, BBOB's different powers, Büche-Rastrigin, the non-continuous Rastrigin, Weierstrass, Katsuura, HappyCat, HGBat, Schaffer's F7 and the rotated hyper-ellipsoid, each with its bounds, known optimum (or best known, for those found numerically) and reference, in Rust and Python, and the analytic gradient of all but seven (the eggholder, Schwefel 2.21 and 2.22, the step function, the non-continuous Rastrigin, Katsuura and the noisy quartic, whose derivative is undefined or 0 on sets of positive measure); and shift and rotation wrappers, generated from a seed, for CEC- and BBOB-style instances of any of them, which pass on the wrapped problem's gradient (by the chain rule), constraint values and Jacobian.
 - **Constrained test problems:** CEC 2006's g01-g24 (`problems::cec2006`), and the engineering design problems (`problems::engineering`): the welded beam in two forms, the pressure vessel, the tension/compression spring, the speed reducer, the gear train (integer), the three-bar truss, the cantilever beam and the car side impact, each with its optimum or best known solution and references, in Rust and Python. Those with inequalities only give their constraints' values one by one to the algorithms that use them.
 
 ## Multi-objective

docs/problems-plan.md

@@ -207,7 +207,15 @@ as (1, …, 1) (it's (−1, …, −1)) and its table I drops the square of f13'
 report prints Schaffer F7 with sin for BBOB's sin²; Katsuura's minima include the bounds ±5, where a
 PSO that stops particles at the bounds lands on them; at G06's minimum, shifted, rounding in x − o
 leaves an active constraint violated by 6e-14. The minima are proven from the formulas (every term
-at least 0), but the noisy quartic's, which isn't known.
+at least 0), but the noisy quartic's, which isn't known. Gradients (#416): all but the step
+function, the non-continuous Rastrigin (flat steps), Katsuura (kinks 2⁻³³ apart) and the noisy
+quartic (noise that jumps between any two genomes) supply their analytic gradient, 0 for the
+kinked term at the cones and cusps of measure 0 (different powers, HappyCat, HGBat, Schaffer F7);
+Büche-Rastrigin is differentiable at 0 despite T_osz, its terms being O(x²) there. Weierstrass's
+is checked on its partial sums: the computed phase 2π 3ᵏ (x + 0.5) of its highest terms rounds by
+2·10⁻⁶ rad, which no differences of the function resolve. `Shifted` passes the wrapped problem's
+extras on at `x − o`; `Rotated` its gradient as `Mᵀ ∇f`, its constraint values, and its
+Jacobian as `J M`.
 
 **Hand-computed test values (VC):** Rosenbrock (−1.2, 1) = 24.2; Powell (3, −1, 0, 1) = 215;
 Goldstein-Price (0, −1) = 3 and its three local minima's values from the paper; Beale (3, 0.5) =

python/genoxide/problems/__init__.py

@@ -57,6 +57,13 @@
     cmaes = gx.Cmaes(problem.genome, restarts="ipop", objective="minimize", seed=1)
     result = cmaes.run(problem, target=problem.optimum.value + 1e-8, evaluations=200_000)
 
+The gradient methods (:class:`genoxide.Lbfgsb`, :class:`genoxide.FirstOrder`, :class:`genoxide.Mma`)
+take a problem's analytic gradient, computed in Rust. Every classic function has one but the
+eggholder, Schwefel 2.21 and 2.22, the step function, the non-continuous Rastrigin, Katsuura and
+the noisy quartic, whose derivative is undefined or 0 on sets of positive measure. The wrappers
+pass the problem's gradient on (turned by the rotation), and a constrained problem's constraint
+values, which :class:`genoxide.Bo` models.
+
 All problems here are minimized, on :class:`genoxide.Real` genomes except the gear train's
 :class:`genoxide.Integer` and :class:`Zdt5`'s :class:`genoxide.Binary`. Each class's docstring
 gives
@@ -1507,7 +1514,8 @@ class Shifted(Problem[Real]):
     value, its solutions shifted (those that leave the box are dropped). That holds when the
     problem's minimum is its minimum over all of ℝⁿ, as for the functions that CEC and BBOB shift,
     not for one whose minimum is only the lowest in its box, such as :class:`Schwefel2_26`. The
-    same seed gives the same shift as Rust's ``problems::Shifted`` on every platform.
+    problem's analytic gradient and constraint values pass through, those at ``x − o``. The same
+    seed gives the same shift as Rust's ``problems::Shifted`` on every platform.
     """
 
     problem: Problem[Real]
@@ -1535,8 +1543,9 @@ class Rotated(Problem[Real]):
     turns about the optimum, as CEC 2005 and BBOB do, so the optimum stays in place with its
     value. Rotating a :class:`Shifted` problem gives CEC 2005's shifted rotated functions, such as
     its F10, ``Rotated(Shifted(Rastrigin(n), seed), seed)``. The bounds, the name and the
-    reference are the problem's. The same seed gives the same matrix as Rust's
-    ``problems::Rotated`` on every platform.
+    reference are the problem's. The problem's analytic gradient passes through by the chain rule,
+    ``Mᵀ ∇f(c + M (x − c))``, and so do the constraint values at ``c + M (x − c)``. The same seed
+    gives the same matrix as Rust's ``problems::Rotated`` on every platform.
     """
 
     problem: Problem[Real]

python/src/problems.rs

@@ -3,7 +3,7 @@
 //! evaluated in Rust.
 
 use crate::run::{WithObjectives, with_objectives};
-use genoxide::engine::{FitnessFunction, IntoFitness};
+use genoxide::engine::{Extras, FitnessFunction, IntoFitness, Provided};
 use genoxide::genome::{Binary, Bits, Integer, Integers, Real, Reals, Representation};
 use genoxide::multi::problems::{self as multi, DynMultiProblem, MultiProblem, try_boxed};
 use genoxide::multi::{IntoScores, MultiFitnessFunction, Scores};
@@ -1813,6 +1813,15 @@ impl FitnessFunction<Reals> for Wrapped {
     fn evaluate(&self, genome: &Reals) -> Fitness {
         self.0.evaluate(genome)
     }
+
+    // the wrapped problem's gradient and constraint values, which the wrappers pass on
+    fn provides(&self) -> Provided {
+        self.0.provides()
+    }
+
+    fn evaluate_with(&self, genome: &Reals, extras: &mut Extras<'_>) -> Fitness {
+        self.0.evaluate_with(genome, extras)
+    }
 }
 
 impl problems::Problem for Wrapped {

python/tests/test_bo.py

@@ -355,6 +355,11 @@ def test_wrong_constraints_are_errors():
         bo.run(toy, constraints=2, batch=True, evaluations=10)
     with pytest.raises(ValueError, match="own constraints"):
         bo.run(gx.problems.cec2006.G24(), constraints=2, evaluations=10)
+    # and so does a shifted or rotated one: the wrappers pass the values on
+    g24 = gx.problems.cec2006.G24()
+    for wrapped in [gx.problems.Shifted(g24, seed=1), gx.problems.Rotated(g24, seed=1)]:
+        with pytest.raises(ValueError, match="own constraints"):
+            bo.run(wrapped, constraints=2, evaluations=10)
     ucb = gx.Bo(gx.Real((0.0, 1.0), length=2), acquisition=gx.UpperConfidenceBound(2.0))
     with pytest.raises(ValueError, match="upper confidence bound"):
         ucb.run(toy, constraints=2, evaluations=10)