diff --git a/LMLTutorial/Pages/DefiningAlgorithm.lean b/LMLTutorial/Pages/DefiningAlgorithm.lean
index 2286647b..5b406cec 100644
--- a/LMLTutorial/Pages/DefiningAlgorithm.lean
+++ b/LMLTutorial/Pages/DefiningAlgorithm.lean
@@ -33,22 +33,30 @@ We then illustrate them with the UCB bandit algorithm.
 # Algorithm and environment
 
 In LML, we prove theorems about the interaction of an algorithm with an environment.
-An algorithm takes actions, to which the environment responds with feedback (e.g., rewards for the bandit case, the gradient of a function in optimization problems).
-In general, both action and feedback can depend on the entire history up to the current time and can be randomized.
+Each round of the interaction consists of three stages: the environment draws an observation (e.g., the context in a contextual bandit), the algorithm takes an action based on that observation, and the environment responds with feedback (e.g., rewards for the bandit case, the gradient of a function in optimization problems).
+In general, observation, action and feedback can depend on the entire history up to the current time and can be randomized.
+One round is recorded by the `Round` abbreviation and a history of `n` rounds by the `Hist` abbreviation.
+
+{docstring Round}
+
+{docstring Hist}
+
 The `Algorithm` structure is defined as follows:
 
 {docstring Algorithm}
 
-This structure refers to two types, the type of actions `𝓐` and the type of feedback `𝓨`.
-Both are measurable spaces, since we consider stochastic algorithms and environments.
-Before time `n`, there is a history of actions and feedbacks `Fin n → 𝓐 × 𝓨` (the `n` pairs of action and feedback at times `0, ..., n - 1`; the processes are 0-indexed).
-The `policy` field contains for each time `n` a kernel from that history to the action space.
-That is, it maps every possible history to a random action at time `n` (and that map is measurable).
-The `h_policy` field records that the measure describing the action is a probability measure (and it is in square brackets to tell Lean to infer it automatically whenever possible).
-At time `0` the history is empty: `Fin 0 → 𝓐 × 𝓨` has a unique element, and the distribution of the first action is `policy 0` applied to that element.
-That distribution is called `Algorithm.p0`.
+This structure refers to three types, the type of observations `𝓞`, the type of actions `𝓐` and the type of feedback `𝓨`.
+All three are measurable spaces, since we consider stochastic algorithms and environments.
+Before time `n`, there is a history of `n` complete rounds `Hist 𝓞 𝓐 𝓨 n = Fin n → 𝓞 × 𝓐 × 𝓨` (the observation-action-feedback triples at times `0, ..., n - 1`; the processes are 0-indexed).
+The `policy` field contains for each time `n` a kernel from that history together with the observation at time `n` to the action space.
+That is, it maps every possible history and current observation to a random action at time `n` (and that map is measurable).
+The `isMarkovKernel_policy` field records that the measure describing the action is a probability measure (and it is in square brackets to tell Lean to infer it automatically whenever possible).
+At time `0` the history is empty: `Hist 𝓞 𝓐 𝓨 0` has a unique element, and the distribution of the first action given the first observation is `policy 0` applied to that element.
+That kernel is called `Algorithm.p0`.
+
+Many settings have no observations at all: the algorithm sees only the past rounds. Those are described by taking `𝓞 = Unit`, and we write `noObs Ω` for the corresponding (constant) observation process.
 
-If the algorithms actions are not random, we can use the `detAlgorithm` definition to build an algorithm from the data of a measurable function for the action at each time, as a function of the history before that time.
+If the algorithms actions are not random, we can use the `detAlgorithm` definition to build an algorithm from the data of a measurable function for the action at each time, as a function of the history before that time and of the current observation.
 The first action is the value of that function at time `0` on the empty history.
 
 {docstring detAlgorithm}
@@ -56,19 +64,21 @@ The first action is the value of that function at time `0` on the empty history.
 We can see here that we did not need to prove that the kernels are `IsMarkovKernel`.
 Lean knows that deterministic kernels are Markov.
 
-The `Environment` structure is the mirror of the `Algorithm` structure, with a kernel for the feedback instead of the actions.
+The `Environment` structure is the mirror of the `Algorithm` structure, with a kernel for the observation and a kernel for the feedback instead of the actions.
 
 {docstring Environment}
 
-`feedback n` gives the distribution of the feedback at time `n` given the history before `n` and the action at time `n`.
-The distribution of the first feedback given the first action is `feedback 0` applied to the empty history; it is called `Environment.ν0`.
+`obs n` gives the distribution of the observation at time `n` given the history before `n`.
+`feedback n` gives the distribution of the feedback at time `n` given the history before `n`, the observation and the action at time `n`.
+The distribution of the first observation is `obs 0` applied to the empty history; it is called `Environment.obs0`.
+The distribution of the first feedback given the first observation and action is `feedback 0` applied to the empty history; it is called `Environment.ν0`.
 
-In many applications the feedback depends only on the last action and not on the prior history.
+In many applications there is no observation and the feedback depends only on the last action, not on the prior history.
 We provide an `obliviousEnv` definition that builds an environment for those cases.
 
 {docstring obliviousEnv}
 
-`(ν n).prodMkLeft _` is the kernel `ν n` seen as a `Kernel ((Fin n → 𝓐 × 𝓨) × 𝓐) 𝓨` by ignoring the history.
+`(ν n).prodMkLeft _` is the kernel `ν n` seen as a `Kernel ((Hist Unit 𝓐 𝓨 n × Unit) × 𝓐) 𝓨` by ignoring the history and the observation.
 
 If furthermore the feedback kernel does not change with time, we can use the `stationaryEnv` definition to build the environment.
 
@@ -82,7 +92,7 @@ This is done by the `IsAlgEnvSeq` structure.
 
 {docstring IsAlgEnvSeq}
 
-This structure takes as input two sequences of random variables (two stochastic processes), `A` and `Y`, which represent the actions and feedback generated by the interaction of the algorithm with the environment.
+This structure takes as input three sequences of random variables (three stochastic processes), `O`, `A` and `Y`, which represent the observations, actions and feedback generated by the interaction of the algorithm with the environment.
 It states that those sequences are measurable and that they have the correct conditional distributions given by the algorithm and environment.
 The measurable space `Ω` and the measure `P` are not imposed: they can be chosen as we want, as long as the conditions of `IsAlgEnvSeq` are satisfied.
 This definition requires `𝓐` and `𝓨` to be nonempty standard Borel spaces, because Mathlib's theory about conditional distributions requires those assumptions.
@@ -149,7 +159,7 @@ The theorem gives an upper bound on the expected regret of UCB at time `n`.
 
 # Building vs analyzing algorithms
 
-When building an algorithm, we describe it with functions from the history `(Fin n → 𝓐 × R)` to the action space `𝓐`.
+When building an algorithm, we describe it with functions from the history and the current observation `(Hist 𝓞 𝓐 R n × 𝓞)` to the action space `𝓐`.
 Thus, to construct UCB, we used the following empirical mean function.
 
 {docstring empMean'}
diff --git a/LeanMachineLearning.lean b/LeanMachineLearning.lean
index d8eaac63..6516f85c 100644
--- a/LeanMachineLearning.lean
+++ b/LeanMachineLearning.lean
@@ -50,6 +50,7 @@ public import LeanMachineLearning.SequentialLearning.Algorithms.RandomSampling.T
 public import LeanMachineLearning.SequentialLearning.Algorithms.RoundRobin
 public import LeanMachineLearning.SequentialLearning.Algorithms.Uniform
 public import LeanMachineLearning.SequentialLearning.BayesStationaryEnv
+public import LeanMachineLearning.SequentialLearning.Comap
 public import LeanMachineLearning.SequentialLearning.Deterministic
 public import LeanMachineLearning.SequentialLearning.DivergenceDecomposition
 public import LeanMachineLearning.SequentialLearning.EvaluationEnv
diff --git a/LeanMachineLearning/ForMathlib/MeasureTheory/MeasurableSpace/Embedding.lean b/LeanMachineLearning/ForMathlib/MeasureTheory/MeasurableSpace/Embedding.lean
index 47e61a4b..859212b0 100644
--- a/LeanMachineLearning/ForMathlib/MeasureTheory/MeasurableSpace/Embedding.lean
+++ b/LeanMachineLearning/ForMathlib/MeasureTheory/MeasurableSpace/Embedding.lean
@@ -26,6 +26,18 @@ learning algorithms (elements of `Fin n → 𝓐 × 𝓨` or `Iic n → 𝓐 ×
 
 open Finset Preorder
 
+/-- `Prod.mk x` is a measurable embedding as soon as `{x}` is measurable. This generalises
+`measurableEmbedding_prodMk_left`, which assumes `MeasurableSingletonClass`. -/
+lemma measurableEmbedding_prodMk_left_of_measurableSet {α β : Type*} [MeasurableSpace α]
+    [MeasurableSpace β] {x : α} (hx : MeasurableSet {x}) :
+    MeasurableEmbedding (Prod.mk x : β → α × β) where
+  injective _ _ h := (Prod.ext_iff.mp h).2
+  measurable := by fun_prop
+  measurableSet_image' s hs := by
+    convert! hx.prod hs
+    ext p
+    simp [Prod.ext_iff, eq_comm, and_left_comm]
+
 lemma coe_default_Iic_zero : ((default : Iic 0) : ℕ) = 0 := rfl
 
 namespace MeasurableEquiv
diff --git a/LeanMachineLearning/ForMathlib/Probability/HasCondDistrib.lean b/LeanMachineLearning/ForMathlib/Probability/HasCondDistrib.lean
index b6e91c89..03af128e 100644
--- a/LeanMachineLearning/ForMathlib/Probability/HasCondDistrib.lean
+++ b/LeanMachineLearning/ForMathlib/Probability/HasCondDistrib.lean
@@ -145,6 +145,17 @@ lemma HasLaw.prod_of_hasCondDistrib {P : Measure β}
     HasLaw (fun ω ↦ (X ω, Y ω)) (P ⊗ₘ κ) μ :=
   ⟨by fun_prop, by rw [h2.map_eq, h1.map_eq]⟩
 
+/-- `HasCondDistrib` only depends on the almost everywhere equivalence classes of the two random
+variables. -/
+lemma HasCondDistrib.congr {X' : α → β} {Y' : α → Ω} (h : HasCondDistrib Y X κ μ)
+    (hX : X' =ᵐ[μ] X) (hY : Y' =ᵐ[μ] Y) :
+    HasCondDistrib Y' X' κ μ := by
+  have h_pair : (fun a ↦ (X' a, Y' a)) =ᵐ[μ] fun a ↦ (X a, Y a) := by
+    filter_upwards [hX, hY] with a h1 h2
+    rw [h1, h2]
+  exact ⟨h.aemeasurable.congr h_pair.symm, by rw [Measure.map_congr h_pair,
+    Measure.map_congr hX, h.map_eq]⟩
+
 lemma HasCondDistrib.hasLaw_comp [SFinite μ] [IsSFiniteKernel κ] (h : HasCondDistrib Y X κ μ) :
     HasLaw Y (κ ∘ₘ (μ.map X)) μ := by
   refine ⟨by fun_prop, ?_⟩
@@ -160,6 +171,13 @@ lemma HasCondDistrib.prod {Z : α → Ω'} {η : Kernel (β × Ω) Ω'}
     AEMeasurable.map_map_of_aemeasurable (by fun_prop) (by fun_prop)]
   rfl
 
+lemma hasCondDistrib_comp_self [SFinite μ] {f : β → Ω} (hf : Measurable f)
+    (hX : AEMeasurable X μ) :
+    HasCondDistrib (f ∘ X) X (Kernel.deterministic f hf) μ := by
+  refine ⟨hX.prodMk (hf.comp_aemeasurable hX), ?_⟩
+  rw [Measure.compProd_deterministic, AEMeasurable.map_map_of_aemeasurable (by fun_prop) hX]
+  rfl
+
 lemma ae_eq_of_hasCondDistrib_deterministic [MeasurableEq Ω] [SFinite μ] {f : β → Ω}
     (hf : Measurable f) (hX : AEMeasurable X μ)
     (hY : AEMeasurable Y μ) (h : HasCondDistrib Y X (Kernel.deterministic f hf) μ) :
@@ -169,6 +187,14 @@ lemma ae_eq_of_hasCondDistrib_deterministic [MeasurableEq Ω] [SFinite μ] {f :
     AEMeasurable.map_map_of_aemeasurable (by fun_prop) (by fun_prop)]
   rfl
 
+lemma hasCondDistrib_deterministic_iff [MeasurableEq Ω] [SFinite μ] {f : β → Ω}
+    (hf : Measurable f) (hX : AEMeasurable X μ) (hY : AEMeasurable Y μ) :
+    HasCondDistrib Y X (Kernel.deterministic f hf) μ ↔ Y =ᵐ[μ] f ∘ X := by
+  refine ⟨ae_eq_of_hasCondDistrib_deterministic hf hX hY, fun h ↦ ?_⟩
+  refine HasCondDistrib.congr ?_ ?_ h (X := X)
+  · exact hasCondDistrib_comp_self hf hX
+  · rfl
+
 section Const
 
 section CompRight
@@ -350,5 +376,4 @@ lemma HasCondDistrib.hasCondDistrib_sectR [IsFiniteMeasure μ] [StandardBorelSpa
   rw [Kernel.map_apply _ hf] at ha
   filter_upwards [hc, ha] with b hcb hab using hcb.trans hab
 
-
 end ProbabilityTheory
diff --git a/LeanMachineLearning/ForMathlib/Probability/WithDensity.lean b/LeanMachineLearning/ForMathlib/Probability/WithDensity.lean
index 8471c6ff..eef5d6b7 100644
--- a/LeanMachineLearning/ForMathlib/Probability/WithDensity.lean
+++ b/LeanMachineLearning/ForMathlib/Probability/WithDensity.lean
@@ -120,6 +120,30 @@ lemma compProd_withDensity_left {κ : Kernel α β} {η : Kernel (α × β) γ}
     _ = ((κ ⊗ₖ η).withDensity (fun a bc ↦ f a bc.1)) a := by
         rw [← compProd_apply_eq_compProd_sectR, Kernel.withDensity_apply _ (by fun_prop)]
 
+lemma sectR_withDensity {η : Kernel (α × β) γ} {g : α × β → γ → ℝ≥0∞} [IsSFiniteKernel η]
+    (hg : Measurable (Function.uncurry g)) (a : α) :
+    (η.withDensity g).sectR a = (η.sectR a).withDensity (fun b ↦ g (a, b)) := by
+  ext b s hs
+  rw [Kernel.sectR_apply, Kernel.withDensity_apply' _ hg,
+    Kernel.withDensity_apply' _ (by fun_prop), Kernel.sectR_apply]
+
+lemma compProd_withDensity_right {κ : Kernel α β} {η : Kernel (α × β) γ} {g : α × β → γ → ℝ≥0∞}
+    [IsSFiniteKernel κ] [IsSFiniteKernel η] [IsSFiniteKernel (η.withDensity g)]
+    (hg : Measurable (Function.uncurry g)) :
+    κ ⊗ₖ (η.withDensity g) = (κ ⊗ₖ η).withDensity (fun a bc ↦ g (a, bc.1) bc.2) := by
+  ext a : 1
+  have h_sf : IsSFiniteKernel ((η.sectR a).withDensity (fun b ↦ g (a, b))) := by
+    rw [← sectR_withDensity hg]
+    infer_instance
+  calc (κ ⊗ₖ (η.withDensity g)) a
+      = (κ a) ⊗ₘ ((η.withDensity g).sectR a) := compProd_apply_eq_compProd_sectR ..
+    _ = (κ a) ⊗ₘ ((η.sectR a).withDensity (fun b ↦ g (a, b))) := by rw [sectR_withDensity hg]
+    _ = ((κ a) ⊗ₘ (η.sectR a)).withDensity (fun p ↦ g (a, p.1) p.2) := by
+        refine Measure.compProd_withDensity ?_
+        fun_prop
+    _ = ((κ ⊗ₖ η).withDensity (fun a bc ↦ g (a, bc.1) bc.2)) a := by
+        rw [← compProd_apply_eq_compProd_sectR, Kernel.withDensity_apply _ (by fun_prop)]
+
 lemma withDensity_rnDeriv_eq' {κ η : Kernel α β} [MeasurableSpace.CountableOrCountablyGenerated α β]
     [IsFiniteKernel κ] [IsFiniteKernel η] (h : ∀ a, κ a ≪ η a) :
     η.withDensity (κ.rnDeriv η) = κ :=
diff --git a/LeanMachineLearning/Online/Bandit/Algorithms/ETC.lean b/LeanMachineLearning/Online/Bandit/Algorithms/ETC.lean
index 7b87e5f8..f6a53d31 100644
--- a/LeanMachineLearning/Online/Bandit/Algorithms/ETC.lean
+++ b/LeanMachineLearning/Online/Bandit/Algorithms/ETC.lean
@@ -29,12 +29,12 @@ For `n < K * m`, this is arm `n % K`.
 For `n = K * m`, this is the arm with the highest empirical mean after the exploration phase.
 For `n > K * m`, this is the same arm as at time `n - 1`. -/
 noncomputable
-def ETC.nextArm (hK : 0 < K) (m n : ℕ) (h : Fin n → Fin K × ℝ) : Fin K :=
+def ETC.nextArm (hK : 0 < K) (m n : ℕ) (h : Hist Unit (Fin K) ℝ n) : Fin K :=
   have : Nonempty (Fin K) := Fin.pos_iff_nonempty.mp hK
   if hn : n < K * m then RoundRobin.nextAction hK n
   else
     if hn_eq : n = K * m then argmax (empMean' n h)
-    else (h ⟨n - 1, by omega⟩).1
+    else (h ⟨n - 1, by omega⟩).action
 
 /-- The next arm pulled by ETC is chosen in a measurable way. -/
 @[fun_prop]
@@ -54,8 +54,8 @@ lemma ETC.measurable_nextArm (hK : 0 < K) (m n : ℕ) : Measurable (nextArm hK m
 /-- The Explore-Then-Commit algorithm: deterministic algorithm that chooses the next arm according
 to `ETC.nextArm`. -/
 noncomputable
-def etcAlgorithm (hK : 0 < K) (m : ℕ) : Algorithm (Fin K) ℝ :=
-  detAlgorithm (ETC.nextArm hK m) (by fun_prop)
+def etcAlgorithm (hK : 0 < K) (m : ℕ) : Algorithm Unit (Fin K) ℝ :=
+  detAlgorithm (fun n p ↦ ETC.nextArm hK m n p.1) (by fun_prop)
 
 end AlgorithmDefinition
 
@@ -64,15 +64,17 @@ namespace ETC
 variable {hK : 0 < K} {m : ℕ} {ν : Kernel (Fin K) ℝ} [IsMarkovKernel ν]
   {Ω : Type*} {mΩ : MeasurableSpace Ω}
   {P : Measure Ω} [IsProbabilityMeasure P]
-  {A : ℕ → Ω → Fin K} {R : ℕ → Ω → ℝ}
+  {O : ℕ → Ω → Unit} {A : ℕ → Ω → Fin K} {R : ℕ → Ω → ℝ}
   {σ2 : ℝ≥0}
 
 /-- Before round `K * m`, the ETC algorithm behaves like the Round-Robin algorithm. -/
 lemma isAlgEnvSeqUntil_roundRobinAlgorithm
-    (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P) :
-    IsAlgEnvSeqUntil A R (roundRobinAlgorithm hK) (stationaryEnv ν) P (K * m) where
+    (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P) :
+    IsAlgEnvSeqUntil O A R (roundRobinAlgorithm hK) (stationaryEnv ν) P (K * m) where
+  measurable_obs := h.measurable_obs
   measurable_action := h.measurable_action
   measurable_feedback := h.measurable_feedback
+  hasCondDistrib_obs n _ := h.hasCondDistrib_obs n
   hasCondDistrib_action n hn := by
     convert h.hasCondDistrib_action n using 1
     simp only [roundRobinAlgorithm, detAlgorithm_policy, etcAlgorithm]
@@ -82,13 +84,13 @@ lemma isAlgEnvSeqUntil_roundRobinAlgorithm
 
 section AlgorithmBehavior
 
-lemma arm_ae_eq_etcNextArm (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P) (n : ℕ) :
-    A n =ᵐ[P] fun ω ↦ nextArm hK m n (history A R n ω) := by
+lemma arm_ae_eq_etcNextArm (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P) (n : ℕ) :
+    A n =ᵐ[P] fun ω ↦ nextArm hK m n (history O A R n ω) := by
   have : Nonempty (Fin K) := Fin.pos_iff_nonempty.mp hK
   exact h.action_detAlgorithm_ae_eq n
 
 /-- For `n < K * m`, the arm pulled at time `n` is the arm `n % K`. -/
-lemma arm_of_lt (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P)
+lemma arm_of_lt (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P)
     {n : ℕ} (hn : n < K * m) :
     A n =ᵐ[P] fun _ ↦ ⟨n % K, Nat.mod_lt _ hK⟩ :=
   RoundRobin.action_ae_eq n ((isAlgEnvSeqUntil_roundRobinAlgorithm h).mono hn)
@@ -96,13 +98,13 @@ lemma arm_of_lt (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P)
 /-- The arm pulled at time `K * m` is the arm with the highest empirical mean after the exploration
 phase. -/
 lemma arm_mul [Nonempty (Fin K)]
-    (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P) :
-    A (K * m) =ᵐ[P] fun ω ↦ argmax (empMean' (K * m) (history A R (K * m) ω)) := by
+    (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P) :
+    A (K * m) =ᵐ[P] fun ω ↦ argmax (empMean' (K * m) (history O A R (K * m) ω)) := by
   filter_upwards [arm_ae_eq_etcNextArm h (K * m)] with ω hn_eq
   rw [hn_eq, nextArm, dite_eq_right (by simp), dite_eq_left rfl]
 
 /-- For `n ≥ K * m`, the arm pulled at time `n + 1` is the same as the arm pulled at time `n`. -/
-lemma arm_add_one_of_ge (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P)
+lemma arm_add_one_of_ge (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P)
     {n : ℕ} (hn : K * m ≤ n) :
     A (n + 1) =ᵐ[P] fun ω ↦ A n ω := by
   filter_upwards [arm_ae_eq_etcNextArm h (n + 1)] with ω hn_eq
@@ -110,7 +112,7 @@ lemma arm_add_one_of_ge (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv
   rfl
 
 /-- For `n ≥ K * m`, the arm pulled at time `n` is the same as the arm pulled at time `K * m`. -/
-lemma arm_of_ge (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P)
+lemma arm_of_ge (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P)
     {n : ℕ} (hn : K * m ≤ n) :
     A n =ᵐ[P] A (K * m) := by
   have h_ae n : K * m ≤ n → A (n + 1) =ᵐ[P] fun ω ↦ A n ω := arm_add_one_of_ge h
@@ -121,11 +123,11 @@ lemma arm_of_ge (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P)
   | succ n hmn h_ind => rw [h_ae n hmn, h_ind]
 
 /-- At time `K * m`, the number of pulls of each arm is equal to `m`. -/
-lemma pullCount_mul (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P) (a : Fin K) :
+lemma pullCount_mul (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P) (a : Fin K) :
     pullCount A a (K * m) =ᵐ[P] fun _ ↦ m :=
   RoundRobin.pullCount_mul m (isAlgEnvSeqUntil_roundRobinAlgorithm h) a
 
-lemma pullCount_add_one_of_ge (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P)
+lemma pullCount_add_one_of_ge (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P)
     (a : Fin K) {n : ℕ} (hn : K * m ≤ n) :
     pullCount A a (n + 1)
       =ᵐ[P] fun ω ↦ pullCount A a n ω + {ω' | A (K * m) ω' = a}.indicator (fun _ ↦ 1) ω := by
@@ -135,7 +137,7 @@ lemma pullCount_add_one_of_ge (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationa
 
 /-- For `n ≥ K * m`, the number of pulls of each arm `a` at time `n` is equal to `m` plus
 `n - K * m` if arm `a` is the best arm after the exploration phase. -/
-lemma pullCount_of_ge (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P)
+lemma pullCount_of_ge (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P)
     (a : Fin K) {n : ℕ} (hn : K * m ≤ n) :
     pullCount A a n
       =ᵐ[P] fun ω ↦ m + (n - K * m) * {ω' | A (K * m) ω' = a}.indicator (fun _ ↦ 1) ω := by
@@ -155,16 +157,16 @@ lemma pullCount_of_ge (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν
 /-- If at time `K * m` the algorithm chooses arm `a`, then the total reward obtained by pulling
 arm `a` is at least the total reward obtained by pulling the best arm. -/
 lemma sumRewards_bestArm_le_of_arm_mul_eq [Nonempty (Fin K)]
-    (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P) (a : Fin K) (hm : m ≠ 0) :
+    (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P) (a : Fin K) (hm : m ≠ 0) :
     ∀ᵐ h ∂P, A (K * m) h = a → sumRewards A R (bestArm ν) (K * m) h ≤
       sumRewards A R a (K * m) h := by
   filter_upwards [arm_mul h, pullCount_mul h a, pullCount_mul h (bestArm ν)]
     with h h_arm ha h_best h_eq
-  have h_max := isMaxOn_argmax (empMean' (K * m) (history A R (K * m) h)) (bestArm ν)
+  have h_max := isMaxOn_argmax (empMean' (K * m) (history O A R (K * m) h)) (bestArm ν)
   rw [← h_arm, h_eq] at h_max
   rw [sumRewards_eq_pullCount_mul_empMean, sumRewards_eq_pullCount_mul_empMean, ha, h_best]
   · gcongr
-    rwa [empMean_eq_empMean', empMean_eq_empMean']
+    rwa [empMean_eq_empMean' (O := O), empMean_eq_empMean' (O := O)]
   · simp [ha, hm]
   · simp [h_best, hm]
 
@@ -173,7 +175,7 @@ end AlgorithmBehavior
 section Regret
 
 lemma probReal_sumRewards_le_sumRewards_le [Nonempty (Fin K)]
-    (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P)
     (hν : ∀ a, HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a)) (a : Fin K) :
     P.real {ω | sumRewards A R (bestArm ν) (K * m) ω ≤ sumRewards A R a (K * m) ω} ≤
       Real.exp (-↑m * gap ν a ^ 2 / (4 * σ2)) := by
@@ -190,7 +192,7 @@ lemma probReal_sumRewards_le_sumRewards_le [Nonempty (Fin K)]
 /-- The probability that at time `K * m` the ETC algorithm chooses arm `a` is at most
 `exp(- m * Δ_a^2 / 4)`. -/
 lemma prob_arm_mul_eq_le [Nonempty (Fin K)]
-    (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P)
     (hν : ∀ a, HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a)) (a : Fin K)
     (hm : m ≠ 0) :
     P.real {ω | A (K * m) ω = a} ≤ Real.exp (- (m : ℝ) * gap ν a ^ 2 / (4 * σ2)) := by
@@ -206,7 +208,7 @@ lemma prob_arm_mul_eq_le [Nonempty (Fin K)]
 
 /-- Bound on the expectation of the number of pulls of each arm by the ETC algorithm. -/
 lemma expectation_pullCount_le [Nonempty (Fin K)]
-    (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P)
     (hν : ∀ a, HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a))
     (a : Fin K) (hm : m ≠ 0) {n : ℕ} (hn : K * m ≤ n) :
     P[fun ω ↦ (pullCount A a n ω : ℝ)]
@@ -235,7 +237,7 @@ lemma expectation_pullCount_le [Nonempty (Fin K)]
 
 /-- Regret bound for the ETC algorithm. -/
 theorem regret_le [Nonempty (Fin K)]
-    (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R (etcAlgorithm hK m) (stationaryEnv ν) P)
     (hν : ∀ a, HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a)) (hm : m ≠ 0)
     (n : ℕ) (hn : K * m ≤ n) :
     P[regret ν A n] ≤
diff --git a/LeanMachineLearning/Online/Bandit/Algorithms/Regret/BayesRegretTS.lean b/LeanMachineLearning/Online/Bandit/Algorithms/Regret/BayesRegretTS.lean
index 8508a313..a9e39543 100644
--- a/LeanMachineLearning/Online/Bandit/Algorithms/Regret/BayesRegretTS.lean
+++ b/LeanMachineLearning/Online/Bandit/Algorithms/Regret/BayesRegretTS.lean
@@ -93,19 +93,19 @@ lemma integrable_uncurry_ucb_comp [MeasurableSpace Ω] (hA : ∀ t, Measurable (
 
 /-- Clipped upper confidence bound (history-based version). -/
 noncomputable
-def ucb' (n : ℕ) (h : Fin n → Fin K × ℝ) (l u σ2 δ : ℝ) (a : Fin K) : ℝ :=
+def ucb' (n : ℕ) (h : Hist Unit (Fin K) ℝ n) (l u σ2 δ : ℝ) (a : Fin K) : ℝ :=
   if pullCount' n h a = 0 then u
   else max l (min u (empMean' n h a + √(2 * σ2 * Real.log (1 / δ) / (pullCount' n h a))))
 
 @[fun_prop]
 lemma measurable_uncurry_ucb' {n : ℕ} :
-    Measurable (fun p : (Fin n → Fin K × ℝ) × Fin K ↦ ucb' n p.1 l u σ2 δ p.2) :=
+    Measurable (fun p : Hist Unit (Fin K) ℝ n × Fin K ↦ ucb' n p.1 l u σ2 δ p.2) :=
   Measurable.ite (by measurability) (by fun_prop) (by fun_prop)
 
-lemma ucb_eq_ucb' {a : Fin K} {n : ℕ} {ω : Ω} :
-    ucb A R l u σ2 δ a n ω = ucb' n (history A R n ω) l u σ2 δ a := by
-  have hp : pullCount A a n ω = pullCount' n (history A R n ω) a := pullCount_eq_pullCount'
-  have he : empMean A R a n ω = empMean' n (history A R n ω) a := empMean_eq_empMean'
+lemma ucb_eq_ucb' {O : ℕ → Ω → Unit} {a : Fin K} {n : ℕ} {ω : Ω} :
+    ucb A R l u σ2 δ a n ω = ucb' n (history O A R n ω) l u σ2 δ a := by
+  have hp : pullCount A a n ω = pullCount' n (history O A R n ω) a := pullCount_eq_pullCount'
+  have he : empMean A R a n ω = empMean' n (history O A R n ω) a := empMean_eq_empMean'
   rw [ucb, ucb', hp, he]
 
 /-- Helper for `sum_ucb_sub_mean_le`. -/
@@ -200,11 +200,11 @@ variable [Nonempty (Fin K)]
 variable [MeasurableSpace Ω]
 variable {𝓔 : Type*} [MeasurableSpace 𝓔]
 variable {E : Ω → 𝓔}
-variable {Q : Measure 𝓔} {κ : Kernel (𝓔 × Fin K) ℝ} [IsMarkovKernel κ]
+variable {Q : Measure 𝓔} [IsProbabilityMeasure Q] {κ : Kernel (𝓔 × Fin K) ℝ} [IsMarkovKernel κ]
 variable {P : Measure Ω} [IsProbabilityMeasure P]
 
-lemma integral_sum_range_actionMean_bestAction_sub_ucb_bestAction_le {alg : Algorithm (Fin K) ℝ}
-    (h : IsBayesAlgEnvSeq Q κ alg E A R P)
+lemma integral_sum_range_actionMean_bestAction_sub_ucb_bestAction_le
+    {alg : Algorithm Unit (Fin K) ℝ} (h : IsBayesAlgEnvSeq Q κ alg E A R P)
     (hlu : l ≤ u) (hm : ∀ e a, (κ (e, a))[id] ∈ (Set.Icc l u)) (hσ2 : 0 < σ2)
     (hs : ∀ e a, HasSubgaussianMGF (fun x ↦ x - (κ (e, a))[id]) ⟨σ2, hσ2.le⟩ (κ (e, a)))
     (hδ : 0 < δ) (n : ℕ) :
@@ -251,7 +251,7 @@ lemma integral_sum_range_actionMean_bestAction_sub_ucb_bestAction_le {alg : Algo
     _ = _ := by
       ring
 
-lemma integral_sum_range_ucb_action_sub_actionMean_action_le {alg : Algorithm (Fin K) ℝ}
+lemma integral_sum_range_ucb_action_sub_actionMean_action_le {alg : Algorithm Unit (Fin K) ℝ}
     (h : IsBayesAlgEnvSeq Q κ alg E A R P)
     (hlu : l ≤ u) (hm : ∀ e a, (κ (e, a))[id] ∈ (Set.Icc l u)) (hσ2 : 0 < σ2)
     (hs : ∀ e a, HasSubgaussianMGF (fun x ↦ x - (κ (e, a))[id]) ⟨σ2, hσ2.le⟩ (κ (e, a)))
@@ -321,18 +321,18 @@ lemma integral_ucb_action_eq_integral_ucb_bestAction (hK : 0 < K)
   have := h.measurable_action
   have := h.measurable_param
   have := h.measurable_feedback
-  let uc (ha : (Fin n → Fin K × ℝ) × Fin K) := ucb' n ha.1 l u σ2 δ ha.2
+  let uc (ha : Hist Unit (Fin K) ℝ n × Fin K) := ucb' n ha.1 l u σ2 δ ha.2
   calc
-    _  = P[fun ω ↦ uc (history A R n ω, A n ω)] := by
-        simp_rw [uc, ucb_eq_ucb']
-    _ = ∫ ha, uc ha ∂P.map (fun ω ↦ (history A R n ω, A n ω)) := by
+    _  = P[fun ω ↦ uc (history (noObs Ω) A R n ω, A n ω)] := by
+        simp_rw [uc, ucb_eq_ucb' (O := noObs Ω)]
+    _ = ∫ ha, uc ha ∂P.map (fun ω ↦ (history (noObs Ω) A R n ω, A n ω)) := by
         rw [← integral_map (by fun_prop) (by fun_prop)]
-    _ = ∫ ha, uc ha ∂P.map (fun ω ↦ (history A R n ω, bestAction κ E ω)) := by
+    _ = ∫ ha, uc ha ∂P.map (fun ω ↦ (history (noObs Ω) A R n ω, bestAction κ E ω)) := by
         rw [← compProd_map_condDistrib (by fun_prop), ← compProd_map_condDistrib (by fun_prop),
             Measure.compProd_congr (hasCondDistrib_action hK h n).condDistrib_eq]
     _ = P[fun ω ↦ ucb A R l u σ2 δ (bestAction κ E ω) n ω] := by
         rw [integral_map (by fun_prop) (by fun_prop)]
-        simp_rw [uc, ucb_eq_ucb']
+        simp_rw [uc, ucb_eq_ucb' (O := noObs Ω)]
 
 lemma integral_regret_eq_add (hK : 0 < K) (h : IsBayesAlgEnvSeq Q κ (tsAlgorithm hK Q κ) E A R P)
     (hm : ∀ e a, (κ (e, a))[id] ∈ (Set.Icc l u)) (n : ℕ) :
diff --git a/LeanMachineLearning/Online/Bandit/Algorithms/TS.lean b/LeanMachineLearning/Online/Bandit/Algorithms/TS.lean
index d457ddbe..e4982c72 100644
--- a/LeanMachineLearning/Online/Bandit/Algorithms/TS.lean
+++ b/LeanMachineLearning/Online/Bandit/Algorithms/TS.lean
@@ -47,13 +47,18 @@ under the posterior over environments given the history so far.
 The posterior under a uniform algorithm is used to avoid a circular definition. -/
 noncomputable
 def TS.policy (hK : 0 < K) (Q : Measure 𝓔) [IsProbabilityMeasure Q] (κ : Kernel (𝓔 × Fin K) ℝ)
-    [IsMarkovKernel κ] (n : ℕ) : Kernel (Fin n → (Fin K) × ℝ) (Fin K) :=
+    [IsMarkovKernel κ] (n : ℕ) : Kernel (Hist Unit (Fin K) ℝ n × Unit) (Fin K) :=
   have : Nonempty (Fin K) := Fin.pos_iff_nonempty.mp hK
-  (IT.bayesTrajMeasurePosterior Q κ uniformAlgorithm n).map (bestAction κ id)
+  (((IT.bayesTrajMeasurePosterior Q κ uniformAlgorithm n).map
+    (bestAction κ id)).prodMkRight Unit)
 
 instance {hK : 0 < K} {Q : Measure 𝓔} [IsProbabilityMeasure Q] {κ : Kernel (𝓔 × Fin K) ℝ}
-    [IsMarkovKernel κ] {n : ℕ} : IsMarkovKernel (TS.policy hK Q κ n) :=
-  Kernel.IsMarkovKernel.map _ (by fun_prop)
+    [IsMarkovKernel κ] {n : ℕ} : IsMarkovKernel (TS.policy hK Q κ n) := by
+  have : Nonempty (Fin K) := Fin.pos_iff_nonempty.mp hK
+  have : IsMarkovKernel ((IT.bayesTrajMeasurePosterior Q κ uniformAlgorithm n).map
+      (bestAction κ id)) := Kernel.IsMarkovKernel.map _ (by fun_prop)
+  unfold TS.policy
+  infer_instance
 
 /-- The Thompson sampling algorithm with actions in `Fin K`, where `Q : Measure 𝓔` is a prior
   distribution over parameters, and `κ : Kernel (𝓔 × Fin K) ℝ` is a Markov kernel that defines the
@@ -64,7 +69,7 @@ instance {hK : 0 < K} {Q : Measure 𝓔} [IsProbabilityMeasure Q] {κ : Kernel (
   `n` is sampled according to these probabilities. -/
 noncomputable
 def tsAlgorithm (hK : 0 < K) (Q : Measure 𝓔) [IsProbabilityMeasure Q] (κ : Kernel (𝓔 × Fin K) ℝ)
-    [IsMarkovKernel κ] : Algorithm (Fin K) ℝ where
+    [IsMarkovKernel κ] : Algorithm Unit (Fin K) ℝ where
   policy := TS.policy hK Q κ
 
 end Algorithm
@@ -79,36 +84,39 @@ variable {P : Measure Ω} [IsProbabilityMeasure P]
 /-- The first action of Thompson sampling is sampled according to its probability of being optimal
 under the prior over environments. -/
 lemma TS.p0_tsAlgorithm (hK : 0 < K) :
-    (tsAlgorithm hK Q κ).p0 = Q.map (bestAction κ id) := by
-  rw [Algorithm.p0_def]
-  change TS.policy hK Q κ 0 default = _
-  rw [TS.policy, Kernel.map_apply _ (by fun_prop), IT.bayesTrajMeasurePosterior_zero,
-    Kernel.const_apply]
+    (tsAlgorithm hK Q κ).p0 () = Q.map (bestAction κ id) := by
+  rw [Algorithm.p0_apply]
+  change TS.policy hK Q κ 0 (default, ()) = _
+  rw [TS.policy, Kernel.prodMkRight_apply, Kernel.map_apply _ (by fun_prop),
+    IT.bayesTrajMeasurePosterior_zero, Kernel.const_apply]
 
 /-- If Thompson sampling has the correct prior over environments, then the conditional distribution
 of the next action given the history so far is equal to the conditional distribution of the best
 action given the history so far. -/
 lemma TS.hasCondDistrib_action (hK : 0 < K) (h : IsBayesAlgEnvSeq Q κ (tsAlgorithm hK Q κ) E A R P)
     (n : ℕ) :
-    HasCondDistrib (A n) (history A R n)
-      (condDistrib (bestAction κ E) (history A R n) P) P where
-  aemeasurable := ((measurable_history h.measurable_action h.measurable_feedback n).prodMk
-      (h.measurable_action n)).aemeasurable
+    HasCondDistrib (A n) (history (noObs Ω) A R n)
+      (condDistrib (bestAction κ E) (history (noObs Ω) A R n) P) P where
+  aemeasurable := ((measurable_history (fun _ ↦ measurable_const) h.measurable_action
+      h.measurable_feedback n).prodMk (h.measurable_action n)).aemeasurable
   map_eq := by
     have hm : Measurable (bestAction κ id) := by fun_prop
-    rw [(h.hasCondDistrib_action' n).map_eq]
+    have h' : HasCondDistrib (A n) (history (noObs Ω) A R n)
+        (((tsAlgorithm hK Q κ).policy n).sectL ()) P :=
+      hasCondDistrib_prodMk_right_unique_iff.mp (h.hasCondDistrib_action' n)
+    rw [h'.map_eq]
     refine Measure.compProd_congr ?_
     calc
-      _ =ᵐ[P.map (history A R n)]
+      _ =ᵐ[P.map (history (noObs Ω) A R n)]
           (IT.bayesTrajMeasurePosterior Q κ uniformAlgorithm n).map (bestAction κ id) := by rfl
-      _ =ᵐ[P.map (history A R n)]
-          (condDistrib E (history A R n) P).map (bestAction κ id) := by
+      _ =ᵐ[P.map (history (noObs Ω) A R n)]
+          (condDistrib E (history (noObs Ω) A R n) P).map (bestAction κ id) := by
           filter_upwards [(h.hasCondDistrib_env_history
             (IT.isBayesAlgEnvSeq_bayesTrajMeasure Q κ uniformAlgorithm)
             absolutelyContinuous_uniformAlgorithm n).condDistrib_eq] with _ hc
           simp_rw [Kernel.map_apply _ hm, IT.bayesTrajMeasurePosterior, hc]
-      _ =ᵐ[P.map (history A R n)]
-          condDistrib (bestAction κ E) (history A R n) P :=
-          (condDistrib_comp (history A R n) h.measurable_param.aemeasurable hm).symm
+      _ =ᵐ[P.map (history (noObs Ω) A R n)]
+          condDistrib (bestAction κ E) (history (noObs Ω) A R n) P :=
+          (condDistrib_comp (history (noObs Ω) A R n) h.measurable_param.aemeasurable hm).symm
 
 end Bandits
diff --git a/LeanMachineLearning/Online/Bandit/Algorithms/UCB.lean b/LeanMachineLearning/Online/Bandit/Algorithms/UCB.lean
index 94ada521..2fd9cde1 100644
--- a/LeanMachineLearning/Online/Bandit/Algorithms/UCB.lean
+++ b/LeanMachineLearning/Online/Bandit/Algorithms/UCB.lean
@@ -28,13 +28,13 @@ section Algorithm
 
 /-- The exploration bonus of the UCB algorithm, which corresponds to the width of
 a confidence interval. -/
-noncomputable def ucbWidth' (c : ℝ) (n : ℕ) (h : Fin n → Fin K × ℝ) (a : Fin K) : ℝ :=
+noncomputable def ucbWidth' (c : ℝ) (n : ℕ) (h : Hist Unit (Fin K) ℝ n) (a : Fin K) : ℝ :=
   √(2 * c * log (n + 1) / pullCount' n h a)
 
 open Classical in
 /-- Arm pulled by the UCB algorithm at time `n`, as a function of the history before `n`. -/
 noncomputable
-def UCB.nextArm (hK : 0 < K) (c : ℝ) (n : ℕ) (h : Fin n → Fin K × ℝ) : Fin K :=
+def UCB.nextArm (hK : 0 < K) (c : ℝ) (n : ℕ) (h : Hist Unit (Fin K) ℝ n) : Fin K :=
   have : Nonempty (Fin K) := Fin.pos_iff_nonempty.mp hK
   if n < K then RoundRobin.nextAction hK n else
   argmax (fun a ↦ empMean' n h a + ucbWidth' c n h a)
@@ -48,8 +48,8 @@ lemma UCB.measurable_nextArm (hK : 0 < K) (c : ℝ) (n : ℕ) : Measurable (next
 
 /-- The UCB algorithm. -/
 noncomputable
-def ucbAlgorithm (hK : 0 < K) (c : ℝ) : Algorithm (Fin K) ℝ :=
-  detAlgorithm (UCB.nextArm hK c) (by fun_prop)
+def ucbAlgorithm (hK : 0 < K) (c : ℝ) : Algorithm Unit (Fin K) ℝ :=
+  detAlgorithm (fun n p ↦ UCB.nextArm hK c n p.1) (by fun_prop)
 end Algorithm
 
 namespace UCB
@@ -57,15 +57,17 @@ namespace UCB
 variable {hK : 0 < K} {c : ℝ} {ν : Kernel (Fin K) ℝ} [IsMarkovKernel ν]
   {Ω : Type*} {mΩ : MeasurableSpace Ω}
   {P : Measure Ω} [IsProbabilityMeasure P]
-  {A : ℕ → Ω → Fin K} {R : ℕ → Ω → ℝ}
+  {O : ℕ → Ω → Unit} {A : ℕ → Ω → Fin K} {R : ℕ → Ω → ℝ}
   {σ2 : ℝ≥0} {n : ℕ} {ω : Ω}
 
 /-- Before round `K`, the UCB algorithm behaves like the Round-Robin algorithm. -/
 lemma isAlgEnvSeqUntil_roundRobinAlgorithm
-    (h : IsAlgEnvSeq A R (ucbAlgorithm hK c) (stationaryEnv ν) P) :
-    IsAlgEnvSeqUntil A R (roundRobinAlgorithm hK) (stationaryEnv ν) P K where
+    (h : IsAlgEnvSeq O A R (ucbAlgorithm hK c) (stationaryEnv ν) P) :
+    IsAlgEnvSeqUntil O A R (roundRobinAlgorithm hK) (stationaryEnv ν) P K where
+  measurable_obs := h.measurable_obs
   measurable_action := h.measurable_action
   measurable_feedback := h.measurable_feedback
+  hasCondDistrib_obs n _ := h.hasCondDistrib_obs n
   hasCondDistrib_action n hn := by
     convert h.hasCondDistrib_action n using 1
     simp only [roundRobinAlgorithm, detAlgorithm_policy, ucbAlgorithm]
@@ -87,36 +89,35 @@ lemma measurable_ucbWidth (hA : ∀ n, Measurable (A n)) (c : ℝ) (a : Fin K) :
   fun_prop
 
 lemma ucbWidth_eq_ucbWidth' (c : ℝ) (a : Fin K) (n : ℕ) (ω : Ω) :
-    ucbWidth A c a n ω = ucbWidth' c n (history A R n ω) a := by
-  rw [ucbWidth, ucbWidth', pullCount_eq_pullCount' (A := A) (R' := R)]
-  rfl
+    ucbWidth A c a n ω = ucbWidth' c n (history O A R n ω) a := by
+  rw [ucbWidth, ucbWidth', pullCount_eq_pullCount' (O := O) (A := A) (R' := R)]
 
-lemma arm_zero (h : IsAlgEnvSeq A R (ucbAlgorithm hK c) (stationaryEnv ν) P) :
+lemma arm_zero (h : IsAlgEnvSeq O A R (ucbAlgorithm hK c) (stationaryEnv ν) P) :
     A 0 =ᵐ[P] fun _ ↦ ⟨0, hK⟩ :=
   RoundRobin.action_zero ((isAlgEnvSeqUntil_roundRobinAlgorithm h).mono hK)
 
-lemma arm_ae_eq_ucbNextArm (h : IsAlgEnvSeq A R (ucbAlgorithm hK c) (stationaryEnv ν) P) (n : ℕ) :
-    A n =ᵐ[P] fun ω ↦ nextArm hK c n (history A R n ω) := by
+lemma arm_ae_eq_ucbNextArm (h : IsAlgEnvSeq O A R (ucbAlgorithm hK c) (stationaryEnv ν) P) (n : ℕ) :
+    A n =ᵐ[P] fun ω ↦ nextArm hK c n (history O A R n ω) := by
   have : Nonempty (Fin K) := Fin.pos_iff_nonempty.mp hK
   exact h.action_detAlgorithm_ae_eq n
 
-lemma arm_ae_all_eq (h : IsAlgEnvSeq A R (ucbAlgorithm hK c) (stationaryEnv ν) P) :
-    ∀ᵐ h ∂P, ∀ n, A n h = nextArm hK c n (history A R n h) :=
+lemma arm_ae_all_eq (h : IsAlgEnvSeq O A R (ucbAlgorithm hK c) (stationaryEnv ν) P) :
+    ∀ᵐ h ∂P, ∀ n, A n h = nextArm hK c n (history O A R n h) :=
   ae_all_iff.mpr (arm_ae_eq_ucbNextArm h)
 
 lemma ucbIndex_le_ucbIndex_arm
-    (h : IsAlgEnvSeq A R (ucbAlgorithm hK c) (stationaryEnv ν) P) (a : Fin K) (hn : K ≤ n) :
+    (h : IsAlgEnvSeq O A R (ucbAlgorithm hK c) (stationaryEnv ν) P) (a : Fin K) (hn : K ≤ n) :
     ∀ᵐ h ∂P, empMean A R a n h + ucbWidth A c a n h ≤
       empMean A R (A n h) n h + ucbWidth A c (A n h) n h := by
   filter_upwards [arm_ae_eq_ucbNextArm h n] with h h_arm
   have h_not_lt : ¬ n < K := by grind
   simp only [nextArm, h_not_lt, ↓reduceIte] at h_arm
   have : Nonempty (Fin K) := Fin.pos_iff_nonempty.mp hK
-  simp_rw [h_arm, empMean_eq_empMean', ucbWidth_eq_ucbWidth' (A := A) (R := R)]
-  exact isMaxOn_argmax (fun a ↦ empMean' n (history A R n h) a
-    + ucbWidth' c n (history A R n h) a) _
+  simp_rw [h_arm, empMean_eq_empMean' (O := O), ucbWidth_eq_ucbWidth' (O := O) (A := A) (R := R)]
+  exact isMaxOn_argmax (fun a ↦ empMean' n (history O A R n h) a
+    + ucbWidth' c n (history O A R n h) a) _
 
-lemma forall_arm_eq_mod_of_lt (h : IsAlgEnvSeq A R (ucbAlgorithm hK c) (stationaryEnv ν) P) :
+lemma forall_arm_eq_mod_of_lt (h : IsAlgEnvSeq O A R (ucbAlgorithm hK c) (stationaryEnv ν) P) :
     ∀ᵐ h ∂P, ∀ n < K, A n h = ⟨n % K, Nat.mod_lt _ hK⟩ := by
   simp_rw [ae_all_iff]
   intro n hn
@@ -125,7 +126,7 @@ lemma forall_arm_eq_mod_of_lt (h : IsAlgEnvSeq A R (ucbAlgorithm hK c) (stationa
   simp only [nextArm, hn, ↓reduceIte, RoundRobin.nextAction]
 
 lemma forall_ucbIndex_le_ucbIndex_arm
-    (h : IsAlgEnvSeq A R (ucbAlgorithm hK c) (stationaryEnv ν) P) (a : Fin K) :
+    (h : IsAlgEnvSeq O A R (ucbAlgorithm hK c) (stationaryEnv ν) P) (a : Fin K) :
     ∀ᵐ h ∂P, ∀ n, K ≤ n →
       empMean A R a n h + ucbWidth A c a n h ≤
         empMean A R (A n h) n h + ucbWidth A c (A n h) n h := by
@@ -133,7 +134,7 @@ lemma forall_ucbIndex_le_ucbIndex_arm
   exact fun _ ↦ ucbIndex_le_ucbIndex_arm h a
 
 lemma forall_arm_prop
-    (h : IsAlgEnvSeq A R (ucbAlgorithm hK c) (stationaryEnv ν) P) :
+    (h : IsAlgEnvSeq O A R (ucbAlgorithm hK c) (stationaryEnv ν) P) :
     ∀ᵐ h ∂P,
       (∀ n < K, A n h = ⟨n % K, Nat.mod_lt _ hK⟩) ∧
       (∀ n, K ≤ n → ∀ a, empMean A R a n h + ucbWidth A c a n h ≤
@@ -148,12 +149,12 @@ lemma forall_arm_prop
     exact h_ae n hn
 
 lemma time_gt_of_pullCount_gt_one
-    (h : IsAlgEnvSeq A R (ucbAlgorithm hK c) (stationaryEnv ν) P) (a : Fin K) :
+    (h : IsAlgEnvSeq O A R (ucbAlgorithm hK c) (stationaryEnv ν) P) (a : Fin K) :
     ∀ᵐ ω ∂P, ∀ n, 1 < pullCount A a n ω → K < n :=
   RoundRobin.time_gt_of_pullCount_gt_one (isAlgEnvSeqUntil_roundRobinAlgorithm h) a
 
 lemma pullCount_pos_of_pullCount_gt_one
-    (h : IsAlgEnvSeq A R (ucbAlgorithm hK c) (stationaryEnv ν) P) (a : Fin K) :
+    (h : IsAlgEnvSeq O A R (ucbAlgorithm hK c) (stationaryEnv ν) P) (a : Fin K) :
     ∀ᵐ ω ∂P, ∀ n, 1 < pullCount A a n ω → ∀ b : Fin K, 0 < pullCount A b n ω :=
   RoundRobin.pullCount_pos_of_pullCount_gt_one (isAlgEnvSeqUntil_roundRobinAlgorithm h) a
 
@@ -196,8 +197,8 @@ lemma pullCount_arm_le [Nonempty (Fin K)] (hc : 0 ≤ c)
     positivity
 
 -- todo: this is not about UCB but about any algorithm with subgaussian rewards. Move it?
-lemma prob_ucbIndex_le [Nonempty (Fin K)] {alg : Algorithm (Fin K) ℝ}
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+lemma prob_ucbIndex_le [Nonempty (Fin K)] {alg : Algorithm Unit (Fin K) ℝ}
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     (hν : ∀ a, HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a))
     (hσ2 : σ2 ≠ 0) (hc : 0 ≤ c) (a : Fin K) (n : ℕ) :
     P {h | 0 < pullCount A a n h ∧ empMean A R a n h + ucbWidth A (c * σ2) a n h ≤ (ν a)[id]} ≤
@@ -239,8 +240,8 @@ lemma prob_ucbIndex_le [Nonempty (Fin K)] {alg : Algorithm (Fin K) ℝ}
       ENNReal.div_eq_inv_mul, ENNReal.mul_inv (by simp) (by simp), inv_inv]
 
 -- todo: this is not about UCB but about any algorithm with subgaussian rewards. Move it?
-lemma prob_ucbIndex_ge [Nonempty (Fin K)] {alg : Algorithm (Fin K) ℝ}
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+lemma prob_ucbIndex_ge [Nonempty (Fin K)] {alg : Algorithm Unit (Fin K) ℝ}
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     (hν : ∀ a, HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a))
     (hσ2 : σ2 ≠ 0) (hc : 0 ≤ c) (a : Fin K) (n : ℕ) :
     P {h | 0 < pullCount A a n h ∧
@@ -281,8 +282,8 @@ lemma prob_ucbIndex_ge [Nonempty (Fin K)] {alg : Algorithm (Fin K) ℝ}
     rw [ENNReal.rpow_sub _ _ (by simp) (by finiteness), ENNReal.rpow_one, div_eq_mul_inv,
       ENNReal.div_eq_inv_mul, ENNReal.mul_inv (by simp) (by simp), inv_inv]
 
-lemma probReal_ucbIndex_le [Nonempty (Fin K)] {alg : Algorithm (Fin K) ℝ}
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+lemma probReal_ucbIndex_le [Nonempty (Fin K)] {alg : Algorithm Unit (Fin K) ℝ}
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     (hν : ∀ a, HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a))
     (hσ2 : σ2 ≠ 0) (hc : 0 ≤ c) (a : Fin K) (n : ℕ) :
     P.real {h | 0 < pullCount A a n h ∧ empMean A R a n h + ucbWidth A (c * σ2) a n h ≤ (ν a)[id]} ≤
@@ -294,8 +295,8 @@ lemma probReal_ucbIndex_le [Nonempty (Fin K)] {alg : Algorithm (Fin K) ℝ}
   rw [← ENNReal.toReal_rpow]
   norm_cast
 
-lemma probReal_ucbIndex_ge [Nonempty (Fin K)] {alg : Algorithm (Fin K) ℝ}
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+lemma probReal_ucbIndex_ge [Nonempty (Fin K)] {alg : Algorithm Unit (Fin K) ℝ}
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     (hν : ∀ a, HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a))
     (hσ2 : σ2 ≠ 0) (hc : 0 ≤ c) (a : Fin K) (n : ℕ) :
     P.real {h | 0 < pullCount A a n h ∧
@@ -351,7 +352,7 @@ lemma pullCount_le_add_three [Nonempty (Fin K)] (a : Fin K) (n C : ℕ) (ω : Ω
       rw [Finset.sum_add_distrib, Finset.sum_add_distrib]
 
 lemma pullCount_le_add_three_ae [Nonempty (Fin K)]
-    (h : IsAlgEnvSeq A R (ucbAlgorithm hK c) (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R (ucbAlgorithm hK c) (stationaryEnv ν) P)
     (a : Fin K) (n C : ℕ) (hC : C ≠ 0) :
     ∀ᵐ ω ∂P,
     pullCount A a n ω ≤ C + 1 +
@@ -373,7 +374,7 @@ lemma pullCount_le_add_three_ae [Nonempty (Fin K)]
   · exact fun h_gt ↦ hω _ (lt_of_le_of_lt (by grind) h_gt) _
 
 lemma some_sum_eq_zero [Nonempty (Fin K)]
-    (h : IsAlgEnvSeq A R (ucbAlgorithm hK (c * σ2)) (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R (ucbAlgorithm hK (c * σ2)) (stationaryEnv ν) P)
     (hc : 0 ≤ c) (a : Fin K) (h_gap : 0 < gap ν a) (n C : ℕ)
     (hC : C ≠ 0) (hC' : 8 * c * σ2 * log (n + 1) / gap ν a ^ 2 ≤ C) :
     ∀ᵐ ω ∂P,
@@ -406,7 +407,7 @@ lemma some_sum_eq_zero [Nonempty (Fin K)]
     gcongr
 
 lemma pullCount_ae_le_add_two [Nonempty (Fin K)]
-    (h : IsAlgEnvSeq A R (ucbAlgorithm hK (c * σ2)) (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R (ucbAlgorithm hK (c * σ2)) (stationaryEnv ν) P)
     (hc : 0 ≤ c) (a : Fin K) (h_gap : 0 < gap ν a)
     (n C : ℕ) (hC : C ≠ 0) (hC' : 8 * c * σ2 * log (n + 1) / gap ν a ^ 2 ≤ C) :
     ∀ᵐ ω ∂P,
@@ -435,7 +436,7 @@ lemma constSum_lt_top (c : ℝ) (n : ℕ) : constSum c n < ∞ := by
 
 /-- Bound on the expectation of the number of pulls of each arm by the UCB algorithm. -/
 lemma expectation_pullCount_le'
-    (h : IsAlgEnvSeq A R (ucbAlgorithm hK (c * σ2)) (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R (ucbAlgorithm hK (c * σ2)) (stationaryEnv ν) P)
     (hν : ∀ a, HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a))
     (hσ2 : σ2 ≠ 0) (hc : 0 < c) (a : Fin K) (h_gap : 0 < gap ν a) (n : ℕ) :
     ∫⁻ ω, pullCount A a n ω ∂P ≤
@@ -524,7 +525,7 @@ lemma expectation_pullCount_le'
 
 /-- Bound on the expectation of the number of pulls of each arm by the UCB algorithm. -/
 lemma expectation_pullCount_le
-    (h : IsAlgEnvSeq A R (ucbAlgorithm hK (c * σ2)) (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R (ucbAlgorithm hK (c * σ2)) (stationaryEnv ν) P)
     (hν : ∀ a, HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a))
     (hσ2 : σ2 ≠ 0) (hc : 0 < c) (a : Fin K) (h_gap : 0 < gap ν a) (n : ℕ) :
     P[fun ω ↦ (pullCount A a n ω : ℝ)] ≤
@@ -554,7 +555,7 @@ lemma expectation_pullCount_le
   ring
 
 /-- Regret bound for the UCB algorithm. -/
-theorem regret_le (h : IsAlgEnvSeq A R (ucbAlgorithm hK (c * σ2)) (stationaryEnv ν) P)
+theorem regret_le (h : IsAlgEnvSeq O A R (ucbAlgorithm hK (c * σ2)) (stationaryEnv ν) P)
     (hν : ∀ a, HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a))
     (hσ2 : σ2 ≠ 0) (hc : 0 < c) (n : ℕ) :
     P[regret ν A n] ≤
diff --git a/LeanMachineLearning/Online/Bandit/ArrayProbSpace.lean b/LeanMachineLearning/Online/Bandit/ArrayProbSpace.lean
index 19459e87..d650f43a 100644
--- a/LeanMachineLearning/Online/Bandit/ArrayProbSpace.lean
+++ b/LeanMachineLearning/Online/Bandit/ArrayProbSpace.lean
@@ -215,23 +215,23 @@ variable [Nonempty 𝓐] [StandardBorelSpace 𝓐]
 
 /-- The next action is the image of the history and a uniform random variable by this function. -/
 noncomputable
-def algFunction (alg : Algorithm 𝓐 𝓡) (n : ℕ) :
-    (Fin n → 𝓐 × 𝓡) → I → 𝓐 :=
-  (Kernel.exists_measurable_map_eq_unitInterval (alg.policy n)).choose
+def algFunction (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) :
+    Hist Unit 𝓐 𝓡 n → I → 𝓐 :=
+  (Kernel.exists_measurable_map_eq_unitInterval ((alg.policy n).sectL ())).choose
 
-lemma algFunction_map (alg : Algorithm 𝓐 𝓡) (n : ℕ) (h : Fin n → 𝓐 × 𝓡) :
-    volume.map (algFunction alg n h) = alg.policy n h :=
-  (Kernel.exists_measurable_map_eq_unitInterval (alg.policy n)).choose_spec.2 h
+lemma algFunction_map (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) (h : Hist Unit 𝓐 𝓡 n) :
+    volume.map (algFunction alg n h) = alg.policy n (h, ()) :=
+  (Kernel.exists_measurable_map_eq_unitInterval ((alg.policy n).sectL ())).choose_spec.2 h
 
 /-- The initial action is the image of a uniform random variable by `algFunction alg 0 default`. -/
-lemma algFunction_zero_map (alg : Algorithm 𝓐 𝓡) :
-    volume.map (algFunction alg 0 default) = alg.p0 :=
+lemma algFunction_zero_map (alg : Algorithm Unit 𝓐 𝓡) :
+    volume.map (algFunction alg 0 default) = alg.p0 () :=
   algFunction_map alg 0 default
 
 @[fun_prop]
-lemma measurable_algFunction (alg : Algorithm 𝓐 𝓡) (n : ℕ) :
+lemma measurable_algFunction (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) :
     Measurable (Function.uncurry (algFunction alg n)) :=
-  (Kernel.exists_measurable_map_eq_unitInterval (alg.policy n)).choose_spec.1
+  (Kernel.exists_measurable_map_eq_unitInterval ((alg.policy n).sectL ())).choose_spec.1
 
 end ProbabilitySpace
 
@@ -241,28 +241,29 @@ section HistoryActionReward
 
 /-- History of actions and rewards before time `n` in the array model. -/
 noncomputable
-def hist [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) : (n : ℕ) → Fin n → 𝓐 × 𝓡
+def hist [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) :
+    (n : ℕ) → Hist Unit 𝓐 𝓡 n
 | 0 => default
 | n + 1 =>
-  let hn : Fin n → 𝓐 × 𝓡 := hist alg ω n
+  let hn : Hist Unit 𝓐 𝓡 n := hist alg ω n
   let a : 𝓐 := algFunction alg n hn (ω.1 n)
-  Fin.snoc hn (a, ω.2 (pullCount' n hn a) a)
+  Fin.snoc hn ((), a, ω.2 (pullCount' n hn a) a)
 
 @[simp]
-lemma hist_zero [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) :
+lemma hist_zero [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) :
     hist alg ω 0 = default := rfl
 
-lemma hist_add_one [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) (n : ℕ) :
+lemma hist_add_one [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) (n : ℕ) :
     hist alg ω (n + 1) =
-      Fin.snoc (hist alg ω n) (algFunction alg n (hist alg ω n) (ω.1 n),
+      Fin.snoc (hist alg ω n) ((), algFunction alg n (hist alg ω n) (ω.1 n),
         ω.2 (pullCount' n (hist alg ω n) (algFunction alg n (hist alg ω n) (ω.1 n)))
           (algFunction alg n (hist alg ω n) (ω.1 n))) := rfl
 
-lemma hist_add_one_eq_finSuccProd' [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (ω : probSpace 𝓐 𝓡)
-    (n : ℕ) :
+lemma hist_add_one_eq_finSuccProd' [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡)
+    (ω : probSpace 𝓐 𝓡) (n : ℕ) :
     hist alg ω (n + 1) =
-      (MeasurableEquiv.finSuccProd (𝓐 × 𝓡) n).symm
-        (hist alg ω n, (algFunction alg n (hist alg ω n) (ω.1 n),
+      (MeasurableEquiv.finSuccProd (Round Unit 𝓐 𝓡) n).symm
+        (hist alg ω n, ((), algFunction alg n (hist alg ω n) (ω.1 n),
           ω.2 (pullCount' n (hist alg ω n) (algFunction alg n (hist alg ω n) (ω.1 n)))
             (algFunction alg n (hist alg ω n) (ω.1 n)))) := by
   rw [MeasurableEquiv.finSuccProd_symm_apply]
@@ -270,44 +271,44 @@ lemma hist_add_one_eq_finSuccProd' [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡
 
 /-- Action taken at time `n` in the array model. -/
 noncomputable
-def action [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (n : ℕ) (ω : probSpace 𝓐 𝓡) : 𝓐 :=
+def action [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) (ω : probSpace 𝓐 𝓡) : 𝓐 :=
   algFunction alg n (hist alg ω n) (ω.1 n)
 
-lemma action_eq [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (n : ℕ) :
+lemma action_eq [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) :
     action alg n = fun ω ↦ algFunction alg n (hist alg ω n) (ω.1 n) := rfl
 
-lemma action_zero [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) :
+lemma action_zero [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) :
     action alg 0 = fun ω ↦ algFunction alg 0 default (ω.1 0) := rfl
 
 /-- Reward received at time `n` in the array model. -/
 noncomputable
-def reward [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (n : ℕ) (ω : probSpace 𝓐 𝓡) : 𝓡 :=
-  (hist alg ω (n + 1) (Fin.last n)).2
+def reward [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) (ω : probSpace 𝓐 𝓡) : 𝓡 :=
+  (hist alg ω (n + 1) (Fin.last n)).feedback
 
-lemma reward_eq' [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (n : ℕ) (ω : probSpace 𝓐 𝓡) :
+lemma reward_eq' [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) (ω : probSpace 𝓐 𝓡) :
     reward alg n ω = ω.2 (pullCount' n (hist alg ω n) (action alg n ω)) (action alg n ω) := by
-  change (hist alg ω (n + 1) (Fin.last n)).2 = _
+  change (hist alg ω (n + 1) (Fin.last n)).feedback = _
   rw [hist_add_one, Fin.snoc_last]
   rfl
 
-lemma hist_succ [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) (n : ℕ) :
-    hist alg ω (n + 1) = Fin.snoc (hist alg ω n) (action alg n ω, reward alg n ω) := by
+lemma hist_succ [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) (n : ℕ) :
+    hist alg ω (n + 1) = Fin.snoc (hist alg ω n) ((), action alg n ω, reward alg n ω) := by
   rw [hist_add_one, reward_eq']
   rfl
 
-lemma hist_succ_eq_finSuccProd [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (ω : probSpace 𝓐 𝓡)
+lemma hist_succ_eq_finSuccProd [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (ω : probSpace 𝓐 𝓡)
     (n : ℕ) :
     hist alg ω (n + 1) =
-      (MeasurableEquiv.finSuccProd (𝓐 × 𝓡) n).symm
-        (hist alg ω n, (action alg n ω, reward alg n ω)) := by
+      (MeasurableEquiv.finSuccProd (Round Unit 𝓐 𝓡) n).symm
+        (hist alg ω n, ((), action alg n ω, reward alg n ω)) := by
   rw [MeasurableEquiv.finSuccProd_symm_apply, hist_succ]
 
-lemma hist_apply_last [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) (n : ℕ) :
-    hist alg ω (n + 1) (Fin.last n) = (action alg n ω, reward alg n ω) := by
+lemma hist_apply_last [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) (n : ℕ) :
+    hist alg ω (n + 1) (Fin.last n) = ((), action alg n ω, reward alg n ω) := by
   rw [hist_succ, Fin.snoc_last]
 
-lemma hist_eq [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) (n : ℕ) :
-    hist alg ω n = fun i : Fin n ↦ (action alg i ω, reward alg i ω) := by
+lemma hist_eq [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) (n : ℕ) :
+    hist alg ω n = fun i : Fin n ↦ ((), action alg i ω, reward alg i ω) := by
   induction n with
   | zero => exact Subsingleton.elim _ _
   | succ n hn =>
@@ -319,25 +320,27 @@ lemma hist_eq [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (ω : probSpace 
       simp
 
 /-- The history in the array model is the history of the action and reward processes. -/
-lemma hist_eq_history [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) (n : ℕ) :
-    hist alg ω n = history (action alg) (reward alg) n ω := hist_eq alg ω n
+lemma hist_eq_history [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (ω : probSpace 𝓐 𝓡) (n : ℕ) :
+    hist alg ω n = history (noObs _) (action alg) (reward alg) n ω := hist_eq alg ω n
 
-lemma pullCount_action_eq [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (a : 𝓐) (n : ℕ)
+lemma pullCount_action_eq [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (a : 𝓐) (n : ℕ)
     (ω : probSpace 𝓐 𝓡) :
     pullCount (action alg) a n ω = pullCount' n (hist alg ω n) a := by
-  rw [pullCount_eq_pullCount' (R' := reward alg), hist_eq]
+  rw [pullCount_eq_pullCount' (O := noObs _) (R' := reward alg), hist_eq]
+  rfl
 
-lemma pullCount_action_eq_comp [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (a : 𝓐) (n : ℕ) :
+lemma pullCount_action_eq_comp [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (a : 𝓐) (n : ℕ) :
     pullCount (action alg) a n = (fun h ↦ pullCount' n h a) ∘ (hist alg · n) := by
   ext ω
   exact pullCount_action_eq alg a n ω
 
-lemma reward_eq [DecidableEq 𝓐] (alg : Algorithm 𝓐 𝓡) (n : ℕ) :
+lemma reward_eq [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) :
     reward alg n = fun ω ↦ ω.2 (pullCount (action alg) (action alg n ω) n ω) (action alg n ω) := by
   ext ω
   rw [reward_eq', pullCount_action_eq]
 
-lemma sumRewards_eq [DecidableEq 𝓐] (alg : Algorithm 𝓐 ℝ) (a : 𝓐) (n : ℕ) (ω : probSpace 𝓐 ℝ) :
+lemma sumRewards_eq [DecidableEq 𝓐] (alg : Algorithm Unit 𝓐 ℝ) (a : 𝓐) (n : ℕ)
+    (ω : probSpace 𝓐 ℝ) :
     sumRewards (action alg) (reward alg) a n ω =
       ∑ i ∈ range (pullCount (action alg) a n ω), ω.2 i a := by
   induction n with
@@ -349,11 +352,11 @@ lemma sumRewards_eq [DecidableEq 𝓐] (alg : Algorithm 𝓐 ℝ) (a : 𝓐) (n
 
 section Measurability
 
-lemma measurable_action' [DecidableEq 𝓐] {alg : Algorithm 𝓐 𝓡}
+lemma measurable_action' [DecidableEq 𝓐] {alg : Algorithm Unit 𝓐 𝓡}
     (n : ℕ) (h : Measurable (hist alg · n)) :
     Measurable (fun x ↦ algFunction alg n (hist alg x n) (x.1 n)) := by fun_prop
 
-lemma measurable_pullCount'_action [DecidableEq 𝓐] {alg : Algorithm 𝓐 𝓡}
+lemma measurable_pullCount'_action [DecidableEq 𝓐] {alg : Algorithm Unit 𝓐 𝓡}
     (n : ℕ) (h_hist : Measurable (hist alg · n)) :
     Measurable (fun x ↦
       pullCount' n (hist alg x n) (algFunction alg n (hist alg x n) (x.1 n))) := by
@@ -362,7 +365,7 @@ lemma measurable_pullCount'_action [DecidableEq 𝓐] {alg : Algorithm 𝓐 𝓡
   exact (measurable_uncurry_pullCount' (𝓐 := 𝓐) n).comp (h_hist.prodMk h_alg_meas)
 
 @[fun_prop]
-lemma measurable_hist [DecidableEq 𝓐] [Countable 𝓐] (alg : Algorithm 𝓐 𝓡) (n : ℕ) :
+lemma measurable_hist [DecidableEq 𝓐] [Countable 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) :
     Measurable (fun ω ↦ hist alg ω n) := by
   induction n with
   | zero =>
@@ -370,15 +373,15 @@ lemma measurable_hist [DecidableEq 𝓐] [Countable 𝓐] (alg : Algorithm 𝓐
     exact measurable_const
   | succ n hn =>
     have h_eq : (fun ω ↦ hist alg ω (n + 1)) =
-        (MeasurableEquiv.finSuccProd (𝓐 × 𝓡) n).symm ∘
-          (fun ω ↦ (hist alg ω n, (algFunction alg n (hist alg ω n) (ω.1 n),
+        (MeasurableEquiv.finSuccProd (Round Unit 𝓐 𝓡) n).symm ∘
+          (fun ω ↦ (hist alg ω n, ((), algFunction alg n (hist alg ω n) (ω.1 n),
             ω.2 (pullCount' n (hist alg ω n) (algFunction alg n (hist alg ω n) (ω.1 n)))
               (algFunction alg n (hist alg ω n) (ω.1 n))))) := by
       ext ω : 1
       exact hist_add_one_eq_finSuccProd' alg ω n
     rw [h_eq]
     refine (MeasurableEquiv.measurable _).comp (hn.prodMk ?_)
-    refine Measurable.prodMk (measurable_action' n hn) ?_
+    refine Measurable.prodMk measurable_const (Measurable.prodMk (measurable_action' n hn) ?_)
     change Measurable ((fun (x : (ℕ → 𝓐 → 𝓡) × ℕ × 𝓐) ↦ x.1 x.2.1 x.2.2) ∘
       (fun x ↦ (x.2, pullCount' n (hist alg x n) (algFunction alg n (hist alg x n) (x.1 n)),
         (algFunction alg n (hist alg x n) (x.1 n)))))
@@ -388,15 +391,15 @@ lemma measurable_hist [DecidableEq 𝓐] [Countable 𝓐] (alg : Algorithm 𝓐
     exact (measurable_pullCount'_action n hn).prodMk (measurable_action' n hn)
 
 @[fun_prop]
-lemma measurable_action [DecidableEq 𝓐] [Countable 𝓐] (alg : Algorithm 𝓐 𝓡) (n : ℕ) :
+lemma measurable_action [DecidableEq 𝓐] [Countable 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) :
     Measurable (action alg n) := by unfold action; fun_prop
 
 @[fun_prop]
-lemma measurable_reward [DecidableEq 𝓐] [Countable 𝓐] (alg : Algorithm 𝓐 𝓡) (n : ℕ) :
+lemma measurable_reward [DecidableEq 𝓐] [Countable 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) :
     Measurable (reward alg n) := by unfold reward; fun_prop
 
 @[fun_prop]
-lemma measurable_pullCount_action [DecidableEq 𝓐] [Countable 𝓐] (alg : Algorithm 𝓐 𝓡)
+lemma measurable_pullCount_action [DecidableEq 𝓐] [Countable 𝓐] (alg : Algorithm Unit 𝓐 𝓡)
     (n : ℕ) :
     Measurable (fun ω ↦ pullCount (action alg) (action alg n ω) n ω) := by
   change Measurable ((fun p : (probSpace 𝓐 𝓡) × 𝓐 ↦ pullCount (action alg) p.2 n p.1) ∘
@@ -412,7 +415,7 @@ variable [DecidableEq 𝓐]
 section Congruence
 
 -- very useful to prove measurability
-lemma hist_congr (alg : Algorithm 𝓐 𝓡) (n : ℕ) {ω ω' : probSpace 𝓐 𝓡}
+lemma hist_congr (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) {ω ω' : probSpace 𝓐 𝓡}
     (hω1 : ∀ i < n, ω.1 i = ω'.1 i)
     (hω2 : ∀ i a, i < pullCount (action alg) a n ω → ω.2 i a = ω'.2 i a) :
     hist alg ω n = hist alg ω' n := by
@@ -431,7 +434,7 @@ lemma hist_congr (alg : Algorithm 𝓐 𝓡) (n : ℕ) {ω ω' : probSpace 𝓐
       exact Nat.lt_succ_self _
     rw [hist_succ, hist_succ, h_hist, h_action, h_reward]
 
-lemma action_eq_and_pullCount_eq_congr_aux (alg : Algorithm 𝓐 𝓡)
+lemma action_eq_and_pullCount_eq_congr_aux (alg : Algorithm Unit 𝓐 𝓡)
     (a : 𝓐) (m n : ℕ) {ω ω' : probSpace 𝓐 𝓡}
     (hω1 : ∀ i, ω.1 i = ω'.1 i) (hω2_ne : ∀ i b, b ≠ a → ω.2 i b = ω'.2 i b)
     (hω2_eq : ∀ i, i + 1 ≤ m → ω.2 i a = ω'.2 i a)
@@ -449,7 +452,7 @@ lemma action_eq_and_pullCount_eq_congr_aux (alg : Algorithm 𝓐 𝓡)
     simp only [action, h_hist, hω1]
   · rw [← h_pc, pullCount_action_eq, pullCount_action_eq, h_hist]
 
-lemma action_eq_and_pullCount_eq_congr (alg : Algorithm 𝓐 𝓡) (a : 𝓐) (m n : ℕ)
+lemma action_eq_and_pullCount_eq_congr (alg : Algorithm Unit 𝓐 𝓡) (a : 𝓐) (m n : ℕ)
     {ω ω' : probSpace 𝓐 𝓡}
     (hω1 : ∀ i, ω.1 i = ω'.1 i) (hω2_ne : ∀ i b, b ≠ a → ω.2 i b = ω'.2 i b)
     (hω2_eq : ∀ i, i + 1 ≤ m → ω.2 i a = ω'.2 i a) :
@@ -458,7 +461,7 @@ lemma action_eq_and_pullCount_eq_congr (alg : Algorithm 𝓐 𝓡) (a : 𝓐) (m
   ⟨action_eq_and_pullCount_eq_congr_aux alg a m n hω1 hω2_ne hω2_eq,
     action_eq_and_pullCount_eq_congr_aux alg a m n (by grind) (by grind) (by grind)⟩
 
-lemma indicator_action_eq_and_pullCount_eq_congr (alg : Algorithm 𝓐 𝓡) (a : 𝓐) (m n : ℕ)
+lemma indicator_action_eq_and_pullCount_eq_congr (alg : Algorithm Unit 𝓐 𝓡) (a : 𝓐) (m n : ℕ)
     {ω ω' : probSpace 𝓐 𝓡}
     (hω1 : ∀ i, ω.1 i = ω'.1 i) (hω2_ne : ∀ i b, b ≠ a → ω.2 i b = ω'.2 i b)
     (hω2_eq : ∀ i, i + 1 ≤ m → ω.2 i a = ω'.2 i a) :
@@ -471,7 +474,7 @@ end Congruence
 
 section MeasurabilityAdvanced
 
-lemma measurable_hist_comap [Countable 𝓐] (alg : Algorithm 𝓐 𝓡) (n : ℕ) :
+lemma measurable_hist_comap [Countable 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) :
     Measurable[MeasurableSpace.comap (fun ω ↦ (fun (i : Fin n) ↦ ω.1 i, ω.2)) inferInstance]
       (hist alg · n) := by
   have h_eq : (hist alg · n) =
@@ -495,11 +498,12 @@ not been observed before time `n` are replaced. The history before time `n` is m
 respect to this function (see `measurable_hist_truncRowPullCount`), and on the event
 `pullCount (action alg) a n = m` it coincides with `truncRow a m`. -/
 noncomputable
-def truncRowPullCount (alg : Algorithm 𝓐 𝓡) (a : 𝓐) (n : ℕ) (ω : probSpace 𝓐 𝓡) :
+def truncRowPullCount (alg : Algorithm Unit 𝓐 𝓡) (a : 𝓐) (n : ℕ) (ω : probSpace 𝓐 𝓡) :
     probSpace 𝓐 𝓡 :=
   truncRow a (pullCount (action alg) a n ω) ω
 
-lemma measurable_hist_truncRowPullCount [Countable 𝓐] (alg : Algorithm 𝓐 𝓡) (a : 𝓐) (n : ℕ) :
+lemma measurable_hist_truncRowPullCount [Countable 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (a : 𝓐)
+    (n : ℕ) :
     Measurable[MeasurableSpace.comap (truncRowPullCount alg a n) inferInstance] (hist alg · n) := by
   have h_eq : (hist alg · n) = (hist alg · n) ∘ (truncRowPullCount alg a n) := by
     ext ω : 1
@@ -511,13 +515,13 @@ lemma measurable_hist_truncRowPullCount [Countable 𝓐] (alg : Algorithm 𝓐 
   rw [h_eq]
   exact (measurable_hist alg n).comp (Measurable.of_comap_le le_rfl)
 
-lemma measurableSet_action_eq_and_pullCount_eq [Countable 𝓐] (alg : Algorithm 𝓐 𝓡) (a : 𝓐)
+lemma measurableSet_action_eq_and_pullCount_eq [Countable 𝓐] (alg : Algorithm Unit 𝓐 𝓡) (a : 𝓐)
     (n m : ℕ) :
     MeasurableSet {ω | action alg n ω = a ∧ pullCount (action alg) a n ω = m} :=
   MeasurableSet.inter ((measurableSet_singleton _).preimage (by fun_prop))
     ((measurableSet_singleton _).preimage (by fun_prop))
 
-lemma preimage_action_pullCount_eq (alg : Algorithm 𝓐 𝓡) (a : 𝓐) (n m : ℕ) :
+lemma preimage_action_pullCount_eq (alg : Algorithm Unit 𝓐 𝓡) (a : 𝓐) (n m : ℕ) :
     (fun ω ↦ (action alg n ω, pullCount (action alg) (action alg n ω) n ω)) ⁻¹' {(a, m)} =
       {ω | action alg n ω = a ∧ pullCount (action alg) a n ω = m}.indicator (fun _ ↦ 1)
         ⁻¹' {1} := by
@@ -528,7 +532,7 @@ lemma preimage_action_pullCount_eq (alg : Algorithm 𝓐 𝓡) (a : 𝓐) (n m :
   intro ha
   simp [ha]
 
-lemma measurable_indicator_action_eq_and_pullCount_eq [Countable 𝓐] (alg : Algorithm 𝓐 𝓡)
+lemma measurable_indicator_action_eq_and_pullCount_eq [Countable 𝓐] (alg : Algorithm Unit 𝓐 𝓡)
     (a : 𝓐) (m n : ℕ) :
     Measurable[MeasurableSpace.comap (truncRow a m) inferInstance]
       (({ω | action alg n ω = a ∧ pullCount (action alg) a n ω = m}).indicator (fun _ ↦ 1)) := by
@@ -542,12 +546,12 @@ lemma measurable_indicator_action_eq_and_pullCount_eq [Countable 𝓐] (alg : Al
   exact (Measurable.indicator (by fun_prop)
     (measurableSet_action_eq_and_pullCount_eq alg a n m)).comp (Measurable.of_comap_le le_rfl)
 
-lemma measurable_pullCount_action_hist (alg : Algorithm 𝓐 𝓡) (n : ℕ) :
+lemma measurable_pullCount_action_hist (alg : Algorithm Unit 𝓐 𝓡) (n : ℕ) :
     Measurable[MeasurableSpace.comap (fun ω ↦ (hist alg ω n, action alg n ω)) inferInstance]
       (fun ω ↦ pullCount (action alg) (action alg n ω) n ω) := by
   simp_rw [pullCount_action_eq]
   change Measurable[MeasurableSpace.comap (fun ω ↦ (hist alg ω n, action alg n ω)) inferInstance]
-    ((fun p : (Fin n → 𝓐 × 𝓡) × 𝓐 ↦ pullCount' n p.1 p.2) ∘
+    ((fun p : Hist Unit 𝓐 𝓡 n × 𝓐 ↦ pullCount' n p.1 p.2) ∘
       (fun ω ↦ (hist alg ω n, action alg n ω)))
   exact measurable_comp_comap _ (measurable_uncurry_pullCount' n)
 
@@ -579,7 +583,7 @@ lemma indepFun_fst_aux (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] (n : ℕ) :
 
 variable [StandardBorelSpace 𝓡] [Nonempty 𝓡]
 
-lemma indepFun_fst_hist [Countable 𝓐] (alg : Algorithm 𝓐 𝓡)
+lemma indepFun_fst_hist [Countable 𝓐] (alg : Algorithm Unit 𝓐 𝓡)
     (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] (n : ℕ) :
     IndepFun (fun ω ↦ ω.1 n) (hist alg · n) (arrayMeasure ν) :=
   (indepFun_fst_aux ν n).of_measurable_right (measurable_hist_comap alg n)
@@ -610,14 +614,14 @@ lemma indepFun_snd_apply_truncRow (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] (a
   exact h.snd_prod (μ := Measure.infinitePi fun _ ↦ volume) (by fun_prop) hTm
 
 omit [StandardBorelSpace 𝓡] [Nonempty 𝓡] in
-lemma indepFun_snd_apply_pullCount_action [Countable 𝓐] (alg : Algorithm 𝓐 𝓡)
+lemma indepFun_snd_apply_pullCount_action [Countable 𝓐] (alg : Algorithm Unit 𝓐 𝓡)
     (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] (a : 𝓐) (m n : ℕ) :
     (fun ω ↦ ω.2 m a) ⟂ᵢ[arrayMeasure ν]
       ({ω | action alg n ω = a ∧ pullCount (action alg) a n ω = m}).indicator (fun _ ↦ 1) :=
   (indepFun_snd_apply_truncRow ν a m).of_measurable_right
     (measurable_indicator_action_eq_and_pullCount_eq alg a m n)
 
-lemma indepFun_snd_hist_cond [Countable 𝓐] (alg : Algorithm 𝓐 𝓡)
+lemma indepFun_snd_hist_cond [Countable 𝓐] (alg : Algorithm Unit 𝓐 𝓡)
     (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] (a : 𝓐) (n m : ℕ) :
     (fun ω ↦ ω.2 m a) ⟂ᵢ[(arrayMeasure ν)[|(fun ω ↦ (action alg n ω,
       pullCount (action alg) (action alg n ω) n ω)) ⁻¹' {(a, m)}]]
@@ -646,18 +650,19 @@ section Laws
 
 variable [Countable 𝓐] [StandardBorelSpace 𝓡] [Nonempty 𝓡]
 
-lemma hasCondDistrib_action' (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] (n : ℕ) :
-    HasCondDistrib (action alg n) (hist alg · n) (alg.policy n) (arrayMeasure ν) := by
+lemma hasCondDistrib_action' (alg : Algorithm Unit 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν]
+    (n : ℕ) :
+    HasCondDistrib (action alg n) (hist alg · n) ((alg.policy n).sectL ()) (arrayMeasure ν) := by
   have hU : HasCondDistrib (fun ω ↦ ω.1 n) (fun ω ↦ (hist alg ω n, hist alg ω n))
       (Kernel.const _ volume) (arrayMeasure ν) :=
     ((indepFun_fst_hist alg ν n).symm.comp (measurable_id.prodMk measurable_id)
       measurable_id).hasCondDistrib_const (by fun_prop) (hasLaw_fst_apply_arrayMeasure ν n)
   have h := ((hasCondDistrib_self (X := (hist alg · n)) (by fun_prop)).prod hU).comp_left
     (measurable_algFunction alg n)
-  have h_ker : alg.policy n = (Kernel.id ⊗ₖ Kernel.const _ volume).map
+  have h_ker : (alg.policy n).sectL () = (Kernel.id ⊗ₖ Kernel.const _ volume).map
       (Function.uncurry (algFunction alg n)) := by
     ext h s hs
-    rw [Kernel.map_apply' _ (measurable_algFunction alg n) _ hs,
+    rw [Kernel.sectL_apply, Kernel.map_apply' _ (measurable_algFunction alg n) _ hs,
       Kernel.compProd_apply (measurable_algFunction alg n hs), Kernel.id_apply]
     simp only [Kernel.const_apply]
     rw [lintegral_dirac' _ (measurable_measure_prodMk_left (measurable_algFunction alg n hs)),
@@ -667,7 +672,7 @@ lemma hasCondDistrib_action' (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡)
   exact h
 
 omit [StandardBorelSpace 𝓡] [Nonempty 𝓡] in
-lemma reward_ae_eq_cond (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) (a : 𝓐) (n m : ℕ) :
+lemma reward_ae_eq_cond (alg : Algorithm Unit 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) (a : 𝓐) (n m : ℕ) :
     reward alg n =ᵐ[(arrayMeasure ν)[|(fun ω ↦ (action alg n ω,
         pullCount (action alg) (action alg n ω) n ω)) ⁻¹' {(a, m)}]]
       (fun ω ↦ ω.2 m a) := by
@@ -681,14 +686,14 @@ lemma reward_ae_eq_cond (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) (a :
 the action at time `n`, and the number of times that action has been pulled before time `n`,
 is equal to the kernel `ν`. -/
 lemma hasCondDistrib_reward_hist_action_pullCount
-    (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] (n : ℕ) :
+    (alg : Algorithm Unit 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] (n : ℕ) :
     HasCondDistrib (reward alg n)
       (fun ω ↦ (hist alg ω n, action alg n ω, pullCount (action alg) (action alg n ω) n ω))
       ((ν.prodMkRight _).prodMkLeft _) (arrayMeasure ν) := by
   refine hasCondDistrib_of_condDistrib_eq (by fun_prop) (by fun_prop) ?_
   refine condDistrib_prod_of_forall_condDistrib_cond (by fun_prop) (by fun_prop) (by fun_prop) _ ?_
   intro (a, m) ham
-  have h_eq : ((ν.prodMkRight _).prodMkLeft _).comap (fun ω : (Fin n → 𝓐 × 𝓡) ↦ (ω, a, m))
+  have h_eq : ((ν.prodMkRight _).prodMkLeft _).comap (fun ω : Hist Unit 𝓐 𝓡 n ↦ (ω, a, m))
         (by fun_prop) =
       Kernel.const _ (ν a) := by ext; simp
   rw [h_eq, condDistrib_congr_left (reward_ae_eq_cond alg ν a n m)]
@@ -705,7 +710,8 @@ lemma hasCondDistrib_reward_hist_action_pullCount
 /-- The reward at time `n` is conditionally independent of the history before time `n`,
 given the action at time `n` and the number of times that action has been pulled before
 time `n`. -/
-lemma condIndepFun_reward_hist (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] (n : ℕ) :
+lemma condIndepFun_reward_hist (alg : Algorithm Unit 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν]
+    (n : ℕ) :
     (reward alg n) ⟂ᵢ[(fun ω ↦ (action alg n ω, pullCount (action alg) (action alg n ω) n ω)),
         Measurable.prodMk (by fun_prop) (measurable_pullCount_action alg n);
         arrayMeasure ν]
@@ -717,7 +723,8 @@ lemma condIndepFun_reward_hist (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 
 
 /-- The conditional distribution of the reward at time `n`, given the history before time `n`
 and the action at time `n`, is equal to the kernel `ν`. -/
-lemma hasCondDistrib_reward' (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] (n : ℕ) :
+lemma hasCondDistrib_reward' (alg : Algorithm Unit 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν]
+    (n : ℕ) :
     HasCondDistrib (reward alg n) (fun ω ↦ (hist alg ω n, action alg n ω))
       (ν.prodMkLeft _) (arrayMeasure ν) := by
   have h := (hasCondDistrib_reward_hist_action_pullCount alg ν n).measurableEquiv_comp_right
@@ -728,7 +735,7 @@ lemma hasCondDistrib_reward' (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡)
       fun ω ↦ ((hist alg ω n, action alg n ω), f (hist alg ω n, action alg n ω)) := by
     funext ω
     exact Prod.ext rfl (congrFun hf_eq ω)
-  have h_ker : ((ν.prodMkRight ℕ).prodMkLeft (Fin n → 𝓐 × 𝓡)).comap
+  have h_ker : ((ν.prodMkRight ℕ).prodMkLeft (Hist Unit 𝓐 𝓡 n)).comap
       MeasurableEquiv.prodAssoc.symm.symm MeasurableEquiv.prodAssoc.symm.symm.measurable =
       (ν.prodMkLeft _).prodMkRight ℕ := by
     ext p : 1
@@ -736,31 +743,51 @@ lemma hasCondDistrib_reward' (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡)
   rw [h_eq, h_ker] at h
   exact (hasCondDistrib_prod_right_iff _ _ hf).1 h
 
-lemma hasCondDistrib_action (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] (n : ℕ) :
-    HasCondDistrib (action alg n) (history (action alg) (reward alg) n) (alg.policy n)
+lemma hasCondDistrib_action (alg : Algorithm Unit 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν]
+    (n : ℕ) :
+    HasCondDistrib (action alg n)
+      (fun ω ↦ (history (noObs _) (action alg) (reward alg) n ω, noObs _ n ω)) (alg.policy n)
       (arrayMeasure ν) := by
+  refine hasCondDistrib_prodMk_right_unique_iff.mpr ?_
   simpa only [hist_eq_history] using hasCondDistrib_action' alg ν n
 
-lemma hasCondDistrib_reward (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν]
+lemma hasCondDistrib_reward (alg : Algorithm Unit 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν]
     (n : ℕ) :
     HasCondDistrib (reward alg n)
-      (fun ω ↦ (history (action alg) (reward alg) n ω, action alg n ω))
+      (fun ω ↦ ((history (noObs _) (action alg) (reward alg) n ω, noObs _ n ω), action alg n ω))
       ((stationaryEnv ν).feedback n) (arrayMeasure ν) := by
-  simpa only [hist_eq_history, feedback_stationaryEnv] using hasCondDistrib_reward' alg ν n
+  let e : (Hist Unit 𝓐 𝓡 n × 𝓐) ≃ᵐ ((Hist Unit 𝓐 𝓡 n × Unit) × 𝓐) :=
+    { toFun := fun p ↦ ((p.1, ()), p.2)
+      invFun := fun p ↦ (p.1.1, p.2)
+      left_inv := fun _ ↦ rfl
+      right_inv := fun _ ↦ rfl
+      measurable_toFun := by simp only [Equiv.coe_fn_mk]; fun_prop
+      measurable_invFun := by simp only [Equiv.symm_mk, Equiv.coe_fn_mk]; fun_prop }
+  rw [feedback_stationaryEnv]
+  have h := (hasCondDistrib_reward' alg ν n).measurableEquiv_comp_right e
+  simp only [hist_eq_history] at h
+  exact h
 
-lemma isAlgEnvSeq_arrayMeasure (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] :
-    IsAlgEnvSeq (action alg) (reward alg) alg (stationaryEnv ν) (arrayMeasure ν) where
+lemma isAlgEnvSeq_arrayMeasure (alg : Algorithm Unit 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] :
+    IsAlgEnvSeq (noObs _) (action alg) (reward alg) alg (stationaryEnv ν) (arrayMeasure ν) where
+  hasCondDistrib_obs n :=
+    hasCondDistrib_unit (measurable_history (fun _ ↦ measurable_const)
+      (measurable_action alg) (measurable_reward alg) n).aemeasurable _ _
   hasCondDistrib_action := hasCondDistrib_action alg ν
   hasCondDistrib_feedback := hasCondDistrib_reward alg ν
 
-lemma hasLaw_action_zero (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] :
-    HasLaw (action alg 0) alg.p0 (arrayMeasure ν) :=
-  (isAlgEnvSeq_arrayMeasure alg ν).hasLaw_action_zero
+lemma hasLaw_action_zero (alg : Algorithm Unit 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] :
+    HasLaw (action alg 0) (alg.p0 ()) (arrayMeasure ν) := by
+  have h : HasCondDistrib (action alg 0) (fun _ : probSpace 𝓐 𝓡 ↦ ()) alg.p0 (arrayMeasure ν) :=
+    (isAlgEnvSeq_arrayMeasure alg ν).hasCondDistrib_action_zero
+  exact h.hasLaw_of_const'
 
-lemma hasCondDistrib_reward_zero (alg : Algorithm 𝓐 𝓡) (ν : Kernel 𝓐 𝓡) [IsMarkovKernel ν] :
+lemma hasCondDistrib_reward_zero (alg : Algorithm Unit 𝓐 𝓡) (ν : Kernel 𝓐 𝓡)
+    [IsMarkovKernel ν] :
     HasCondDistrib (reward alg 0) (action alg 0) ν (arrayMeasure ν) := by
   have h := (isAlgEnvSeq_arrayMeasure alg ν).hasCondDistrib_feedback_zero
-  rwa [ν0_stationaryEnv] at h
+  rw [ν0_stationaryEnv] at h
+  simpa using hasCondDistrib_prodMk_left_unique_iff.mp h
 
 end Laws
 
diff --git a/LeanMachineLearning/Online/Bandit/Regret.lean b/LeanMachineLearning/Online/Bandit/Regret.lean
index d5e4a96d..58c5683a 100644
--- a/LeanMachineLearning/Online/Bandit/Regret.lean
+++ b/LeanMachineLearning/Online/Bandit/Regret.lean
@@ -28,9 +28,10 @@ open scoped ENNReal NNReal
 
 namespace Bandits
 
-variable {𝓐 Ω : Type*} [DecidableEq 𝓐] {m𝓐 : MeasurableSpace 𝓐} {mΩ : MeasurableSpace Ω}
+variable {𝓞 𝓐 Ω : Type*} [DecidableEq 𝓐] {m𝓞 : MeasurableSpace 𝓞} {m𝓐 : MeasurableSpace 𝓐}
+  {mΩ : MeasurableSpace Ω}
   {ν : Kernel 𝓐 ℝ}
-  {A : ℕ → Ω → 𝓐} {R : ℕ → Ω → ℝ}
+  {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {R : ℕ → Ω → ℝ}
   {ω : Ω} {m n t : ℕ} {a : 𝓐}
 
 /-- Gap of an action `a`: difference between the highest mean of the actions and the mean of `a`. -/
@@ -91,8 +92,8 @@ lemma integral_regret_eq_sum_gap_mul_integral_pullCount
 with positive gap. -/
 lemma integral_regret_le_of_forall_integral_pullCount_le
     [StandardBorelSpace 𝓐] [Fintype 𝓐] {P : Measure Ω} [IsProbabilityMeasure P]
-    {alg : Algorithm 𝓐 ℝ} {env : Environment 𝓐 ℝ} {B : 𝓐 → ℝ}
-    (h : IsAlgEnvSeq A R alg env P)
+    {alg : Algorithm 𝓞 𝓐 ℝ} {env : Environment 𝓞 𝓐 ℝ} {B : 𝓐 → ℝ}
+    (h : IsAlgEnvSeq O A R alg env P)
     (h_le : ∀ a, gap ν a ≠ 0 → ∫ ω, (pullCount A a n ω : ℝ) ∂P ≤ B a) :
     P[regret ν A n] ≤ ∑ a, gap ν a * B a := by
   have hA := h.measurable_action
diff --git a/LeanMachineLearning/Online/Bandit/RewardByCountMeasure.lean b/LeanMachineLearning/Online/Bandit/RewardByCountMeasure.lean
index 6c75680f..902d6c53 100644
--- a/LeanMachineLearning/Online/Bandit/RewardByCountMeasure.lean
+++ b/LeanMachineLearning/Online/Bandit/RewardByCountMeasure.lean
@@ -20,9 +20,9 @@ open scoped ENNReal NNReal
 namespace Bandits
 
 variable {𝓐 Ω : Type*} {m𝓐 : MeasurableSpace 𝓐} {mΩ : MeasurableSpace Ω} [DecidableEq 𝓐]
-  {A : ℕ → Ω → 𝓐} {R : ℕ → Ω → ℝ} {P : Measure Ω} [IsProbabilityMeasure P]
-  {alg : Algorithm 𝓐 ℝ} {ν : Kernel 𝓐 ℝ} [IsMarkovKernel ν]
-  {h_inter : IsAlgEnvSeq A R alg (stationaryEnv ν) P}
+  {O : ℕ → Ω → Unit} {A : ℕ → Ω → 𝓐} {R : ℕ → Ω → ℝ} {P : Measure Ω} [IsProbabilityMeasure P]
+  {alg : Algorithm Unit 𝓐 ℝ} {ν : Kernel 𝓐 ℝ} [IsMarkovKernel ν]
+  {h_inter : IsAlgEnvSeq O A R alg (stationaryEnv ν) P}
 
 local notation "𝔓" => P.prod (streamMeasure ν)
 
@@ -35,7 +35,7 @@ notation "𝓛[" Y " | " X " ← " x "; " μ "]" => Measure.map Y (μ[|X ⁻¹'
 
 omit [DecidableEq 𝓐] in
 lemma condDistrib_reward'' [Countable 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (n : ℕ) :
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) (n : ℕ) :
     𝓛[fun ω ↦ R n ω.1 | fun ω ↦ A n ω.1; 𝔓] =ᵐ[(𝔓).map (fun ω ↦ A n ω.1)] ν := by
   have hA := h.measurable_action
   have hR := h.measurable_feedback
@@ -56,7 +56,7 @@ variable [StandardBorelSpace 𝓐]
 
 omit [DecidableEq 𝓐] in
 lemma reward_cond_action [Countable 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (a : 𝓐) (n : ℕ)
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) (a : 𝓐) (n : ℕ)
     (hμa : (𝔓).map (fun ω ↦ A n ω.1) {a} ≠ 0) :
     𝓛[fun ω ↦ R n ω.1 | fun ω ↦ A n ω.1 ← a; 𝔓] = ν a := by
   have hA := h.measurable_action
@@ -74,20 +74,21 @@ lemma reward_cond_action [Countable 𝓐]
 variable [Nonempty 𝓐]
 
 lemma condIndepFun_reward_stepsUntil_action' [StandardBorelSpace Ω]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (a : 𝓐) (m n : ℕ) :
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) (a : 𝓐) (m n : ℕ) :
     R n ⟂ᵢ[A n, h.measurable_action n; P] {ω | stepsUntil A a m ω = ↑n}.indicator (fun _ ↦ 1) := by
   -- the indicator of `stepsUntil ... = n` is a function of `hist (n-1)` and `action n`.
   -- It thus suffices to use the independence of `reward n` and `hist (n-1)` conditionally
   -- on `action n`.
   have hA := h.measurable_action
   have hR := h.measurable_feedback
-  have h_indep : R n ⟂ᵢ[A n, hA n; P] fun ω ↦ (history A R n ω, A n ω) :=
+  have h_indep : R n ⟂ᵢ[A n, hA n; P]
+      fun ω ↦ ((history O A R n ω, O n ω), A n ω) :=
     IsAlgEnvSeq.condIndepFun_feedback_history_action_action h n
   refine h_indep.of_measurable_right (hX := hA n) ?_
-  exact measurable_comap_indicator_stepsUntil_eq R a m n
+  exact measurable_comap_indicator_stepsUntil_eq O R a m n
 
 lemma condIndepFun_reward_stepsUntil_action [StandardBorelSpace Ω] [Countable 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     (a : 𝓐) (m n : ℕ) :
     CondIndepFun (m𝓐.comap (fun ω ↦ A n ω.1)) ((h.measurable_action n).comp measurable_fst).comap_le
       (fun ω ↦ R n ω.1) ({ω | stepsUntil A a m ω.1 = ↑n}.indicator (fun _ ↦ 1)) 𝔓 := by
@@ -98,7 +99,7 @@ lemma condIndepFun_reward_stepsUntil_action [StandardBorelSpace Ω] [Countable 
     (condIndepFun_reward_stepsUntil_action' h a m n)
 
 lemma reward_cond_stepsUntil [StandardBorelSpace Ω] [Countable 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (a : 𝓐) (m n : ℕ)
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) (a : 𝓐) (m n : ℕ)
     (hm : m ≠ 0) (hμn : 𝔓 ((fun ω ↦ stepsUntil A a m ω.1) ⁻¹' {↑n}) ≠ 0) :
     𝓛[fun ω ↦ R n ω.1 | fun ω ↦ stepsUntil A a m ω.1 ← ↑n; 𝔓] = ν a := by
   have hA := h.measurable_action
@@ -144,7 +145,7 @@ lemma reward_cond_stepsUntil [StandardBorelSpace Ω] [Countable 𝓐]
 /-- The conditional distribution of the reward received at the `m`-th pull of action `a`
 given the time at which number of pulls is `m` is the constant kernel with value `ν a`. -/
 lemma condDistrib_rewardByCount_stepsUntil [StandardBorelSpace Ω] [Countable 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (a : 𝓐) (m : ℕ) (hm : m ≠ 0) :
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) (a : 𝓐) (m : ℕ) (hm : m ≠ 0) :
     condDistrib (rewardByCount A R a m) (fun ω ↦ stepsUntil A a m ω.1) 𝔓
       =ᵐ[(𝔓).map (fun ω ↦ stepsUntil A a m ω.1)] Kernel.const _ (ν a) := by
   have hA := h.measurable_action
@@ -177,7 +178,7 @@ lemma condDistrib_rewardByCount_stepsUntil [StandardBorelSpace Ω] [Countable 
 
 /-- The reward received at the `m`-th pull of action `a` has law `ν a`. -/
 lemma hasLaw_rewardByCount [StandardBorelSpace Ω] [Countable 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (a : 𝓐) (m : ℕ) (hm : m ≠ 0) :
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) (a : 𝓐) (m : ℕ) (hm : m ≠ 0) :
     HasLaw (rewardByCount A R a m) (ν a) 𝔓 where
   aemeasurable :=
     (measurable_rewardByCount h.measurable_action h.measurable_feedback a m).aemeasurable
@@ -200,7 +201,7 @@ lemma hasLaw_rewardByCount [StandardBorelSpace Ω] [Countable 𝓐]
       simp
 
 lemma identDistrib_rewardByCount [StandardBorelSpace Ω] [Countable 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (a : 𝓐) (n m : ℕ)
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) (a : 𝓐) (n m : ℕ)
     (hn : n ≠ 0) (hm : m ≠ 0) :
     IdentDistrib (rewardByCount A R a n) (rewardByCount A R a m) 𝔓 𝔓 where
   aemeasurable_fst :=
@@ -210,7 +211,7 @@ lemma identDistrib_rewardByCount [StandardBorelSpace Ω] [Countable 𝓐]
   map_eq := by rw [(hasLaw_rewardByCount h a n hn).map_eq, (hasLaw_rewardByCount h a m hm).map_eq]
 
 lemma identDistrib_rewardByCount_id [StandardBorelSpace Ω] [Countable 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (a : 𝓐) (n : ℕ) (hn : n ≠ 0) :
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) (a : 𝓐) (n : ℕ) (hn : n ≠ 0) :
     IdentDistrib (rewardByCount A R a n) id 𝔓 (ν a) where
   aemeasurable_fst :=
     (measurable_rewardByCount h.measurable_action h.measurable_feedback a n).aemeasurable
@@ -218,7 +219,7 @@ lemma identDistrib_rewardByCount_id [StandardBorelSpace Ω] [Countable 𝓐]
   map_eq := by rw [(hasLaw_rewardByCount h a n hn).map_eq, Measure.map_id]
 
 lemma identDistrib_rewardByCount_eval [StandardBorelSpace Ω] [Countable 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (a : 𝓐) (n m : ℕ) (hn : n ≠ 0) :
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) (a : 𝓐) (n m : ℕ) (hn : n ≠ 0) :
     IdentDistrib (rewardByCount A R a n) (fun ω ↦ ω m a) 𝔓 (streamMeasure ν) :=
   (identDistrib_rewardByCount_id h a n hn).trans
     (identDistrib_eval_eval_id_streamMeasure ν m a).symm
@@ -284,24 +285,24 @@ lemma indepFun_update_rewardByCountUntil_eval [Countable 𝓐] (hA : ∀ n, Meas
 /-- Conditionally on the event that the action at time `n` is `b` and that `b` was pulled `k`
 times before, the reward at time `n` is independent of the history before time `n` and of the
 action at time `n`. -/
-lemma indepFun_history_reward_cond (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+lemma indepFun_history_reward_cond (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     (n : ℕ) (b : 𝓐) (k : ℕ) :
-    (fun x ↦ (history A R n x, A n x))
+    (fun x ↦ ((history O A R n x, O n x), A n x))
       ⟂ᵢ[P[|{x | A n x = b ∧ pullCount A b n x = k}]] R n := by
-  rw [setOf_action_eq_and_pullCount_eq_eq_preimage (R' := R)]
+  rw [setOf_action_eq_and_pullCount_eq_eq_preimage (O := O) (R' := R)]
   exact h.indepFun_history_action_feedback_cond_stationaryEnv n
     (measurableSet_snd_eq_and_pullCount'_eq n b k) fun u hu ↦ hu.1
 
 /-- Conditionally on the event that the action at time `t` is `b` and that `b` was pulled `k`
 times before, the reward at time `t` has law `ν b`. -/
-lemma hasLaw_reward_cond (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (t : ℕ) (b : 𝓐) (k : ℕ)
+lemma hasLaw_reward_cond (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) (t : ℕ) (b : 𝓐) (k : ℕ)
     (hP : P {x | A t x = b ∧ pullCount A b t x = k} ≠ 0) :
     HasLaw (R t) (ν b) (P[|{x | A t x = b ∧ pullCount A b t x = k}]) := by
-  rw [setOf_action_eq_and_pullCount_eq_eq_preimage (R' := R)] at hP ⊢
+  rw [setOf_action_eq_and_pullCount_eq_eq_preimage (O := O) (R' := R)] at hP ⊢
   exact h.hasLaw_feedback_cond_stationaryEnv t (measurableSet_snd_eq_and_pullCount'_eq t b k)
     (fun u hu ↦ hu.1) hP
 
-lemma hasLaw_reward_cond_prod (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (t : ℕ) (b : 𝓐)
+lemma hasLaw_reward_cond_prod (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) (t : ℕ) (b : 𝓐)
     (k : ℕ) (hP : P {x | A t x = b ∧ pullCount A b t x = k} ≠ 0) :
     HasLaw (fun ω ↦ R t ω.1) (ν b)
       ((P[|{x | A t x = b ∧ pullCount A b t x = k}]).prod (streamMeasure ν)) :=
@@ -312,7 +313,7 @@ variable [Countable 𝓐]
 /-- Conditionally on the event that the action at time `t` is `b` and that `b` was pulled `k`
 times before, the array `rewardByCountUntil A R t` with the entry `(b, k)` erased is independent of
 the reward at time `t`. -/
-lemma indepFun_update_rewardByCountUntil_reward (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+lemma indepFun_update_rewardByCountUntil_reward (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     (t : ℕ) (b : 𝓐) (k : ℕ) :
     (fun ω ↦ Function.update (rewardByCountUntil A R t ω) (b, k) 0)
       ⟂ᵢ[(P[|{x | A t x = b ∧ pullCount A b t x = k}]).prod (streamMeasure ν)]
@@ -330,20 +331,22 @@ lemma indepFun_update_rewardByCountUntil_reward (h : IsAlgEnvSeq A R alg (statio
   refine Measurable.comp measurable_update_left ?_
   refine measurable_rewardByCountUntil_of t (fun i hi ↦ ?_) (fun i hi ↦ ?_) ?_
   · exact measurable_comp_comap
-      (fun ω : Ω × (ℕ → 𝓐 → ℝ) ↦ ((history A R t ω.1, A t ω.1), ω.2))
-      (g := fun v : ((Fin t → 𝓐 × ℝ) × 𝓐) × (ℕ → 𝓐 → ℝ) ↦ (v.1.1 ⟨i, hi⟩).1) (by fun_prop)
+      (fun ω : Ω × (ℕ → 𝓐 → ℝ) ↦ (((history O A R t ω.1, O t ω.1), A t ω.1), ω.2))
+      (g := fun v : ((Hist Unit 𝓐 ℝ t × Unit) × 𝓐) × (ℕ → 𝓐 → ℝ) ↦ (v.1.1.1 ⟨i, hi⟩).action)
+      (by fun_prop)
   · exact measurable_comp_comap
-      (fun ω : Ω × (ℕ → 𝓐 → ℝ) ↦ ((history A R t ω.1, A t ω.1), ω.2))
-      (g := fun v : ((Fin t → 𝓐 × ℝ) × 𝓐) × (ℕ → 𝓐 → ℝ) ↦ (v.1.1 ⟨i, hi⟩).2) (by fun_prop)
+      (fun ω : Ω × (ℕ → 𝓐 → ℝ) ↦ (((history O A R t ω.1, O t ω.1), A t ω.1), ω.2))
+      (g := fun v : ((Hist Unit 𝓐 ℝ t × Unit) × 𝓐) × (ℕ → 𝓐 → ℝ) ↦ (v.1.1.1 ⟨i, hi⟩).feedback)
+      (by fun_prop)
   · exact measurable_comp_comap
-      (fun ω : Ω × (ℕ → 𝓐 → ℝ) ↦ ((history A R t ω.1, A t ω.1), ω.2))
-      (g := fun v : ((Fin t → 𝓐 × ℝ) × 𝓐) × (ℕ → 𝓐 → ℝ) ↦ v.2) measurable_snd
+      (fun ω : Ω × (ℕ → 𝓐 → ℝ) ↦ (((history O A R t ω.1, O t ω.1), A t ω.1), ω.2))
+      (g := fun v : ((Hist Unit 𝓐 ℝ t × Unit) × 𝓐) × (ℕ → 𝓐 → ℝ) ↦ v.2) measurable_snd
 
 /-- Conditionally on the event that the action at time `t` is `b` and that `b` was pulled `k`
 times before, the arrays `rewardByCountUntil A R (t + 1)` and `rewardByCountUntil A R t` have the
 same law: they differ only in the entry `(b, k)`, which is `R t` in the first and an auxiliary
 reward in the second, and both are independent of the rest of the array with law `ν b`. -/
-lemma identDistrib_rewardByCountUntil_add_one_cond (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+lemma identDistrib_rewardByCountUntil_add_one_cond (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     (t : ℕ) (b : 𝓐) (k : ℕ) :
     IdentDistrib (rewardByCountUntil A R (t + 1)) (rewardByCountUntil A R t)
       ((P[|{x | A t x = b ∧ pullCount A b t x = k}]).prod (streamMeasure ν))
@@ -399,7 +402,7 @@ lemma identDistrib_rewardByCountUntil_add_one_cond (h : IsAlgEnvSeq A R alg (sta
     (IdentDistrib.of_ae_eq (measurable_rewardByCountUntil hA hR _).aemeasurable h2).symm
 
 /-- The law of `rewardByCountUntil A R t` under `𝔓` does not depend on `t`. -/
-lemma identDistrib_rewardByCountUntil_add_one (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+lemma identDistrib_rewardByCountUntil_add_one (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     (t : ℕ) :
     IdentDistrib (rewardByCountUntil A R (t + 1)) (rewardByCountUntil A R t) 𝔓 𝔓 := by
   have hA := h.measurable_action
@@ -424,7 +427,7 @@ lemma identDistrib_rewardByCountUntil_add_one (h : IsAlgEnvSeq A R alg (stationa
   exact identDistrib_rewardByCountUntil_add_one_cond h t p.1 p.2
 
 /-- The law of `rewardByCountUntil A R t` under `𝔓` is `⨂ (a, m), ν a`, for all `t`. -/
-lemma hasLaw_rewardByCountUntil (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (t : ℕ) :
+lemma hasLaw_rewardByCountUntil (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) (t : ℕ) :
     HasLaw (rewardByCountUntil A R t) (Measure.infinitePi fun p : 𝓐 × ℕ ↦ ν p.1) 𝔓 := by
   induction t with
   | zero => exact hasLaw_rewardByCountUntil_zero P
@@ -432,7 +435,7 @@ lemma hasLaw_rewardByCountUntil (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (
 
 /-- The array of rewards by count `(a, m) ↦ rewardByCount A R a (m + 1)` has law
 `⨂ (a, m), ν a`: its entries are independent, and the entry `(a, m)` has law `ν a`. -/
-lemma hasLaw_rewardByCount_infinitePi (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) :
+lemma hasLaw_rewardByCount_infinitePi (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) :
     HasLaw (fun ω (p : 𝓐 × ℕ) ↦ rewardByCount A R p.1 (p.2 + 1) ω)
       (Measure.infinitePi fun p : 𝓐 × ℕ ↦ ν p.1) 𝔓 := by
   have hY : Measurable fun ω (p : 𝓐 × ℕ) ↦ rewardByCount A R p.1 (p.2 + 1) ω :=
@@ -444,14 +447,14 @@ lemma hasLaw_rewardByCount_infinitePi (h : IsAlgEnvSeq A R alg (stationaryEnv ν
     (hasLaw_rewardByCountUntil h) eventually_rewardByCountUntil_eq
 
 /-- The reward received at the `(m + 1)`-th pull of action `a` has law `ν a`. -/
-lemma hasLaw_rewardByCount_add_one (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+lemma hasLaw_rewardByCount_add_one (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     (a : 𝓐) (m : ℕ) :
     HasLaw (rewardByCount A R a (m + 1)) (ν a) 𝔓 :=
   (hasLaw_eval_infinitePi (fun p : 𝓐 × ℕ ↦ ν p.1) (a, m)).comp (hasLaw_rewardByCount_infinitePi h)
 
 /-- The rewards by count `rewardByCount A R a (m + 1)` are independent over all actions `a` and
 all counts `m`. -/
-lemma iIndepFun_rewardByCount_add_one (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) :
+lemma iIndepFun_rewardByCount_add_one (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) :
     iIndepFun (fun (p : 𝓐 × ℕ) ω ↦ rewardByCount A R p.1 (p.2 + 1) ω) 𝔓 :=
   (iIndepFun_iff_hasLaw_Pi_infinitePi
     (X := fun (p : 𝓐 × ℕ) ω ↦ rewardByCount A R p.1 (p.2 + 1) ω) (μ := fun p : 𝓐 × ℕ ↦ ν p.1)
@@ -460,7 +463,7 @@ lemma iIndepFun_rewardByCount_add_one (h : IsAlgEnvSeq A R alg (stationaryEnv ν
 
 /-- The rewards by count `rewardByCount A R a m` for `m ≠ 0` are independent over all actions `a`
 and all counts `m`. -/
-lemma iIndepFun_rewardByCount (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) :
+lemma iIndepFun_rewardByCount (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) :
     iIndepFun (fun (p : {p : 𝓐 × ℕ // p.2 ≠ 0}) ω ↦ rewardByCount A R p.1.1 p.1.2 ω) 𝔓 := by
   have h_eq : (fun (p : {p : 𝓐 × ℕ // p.2 ≠ 0}) ω ↦ rewardByCount A R p.1.1 p.1.2 ω)
       = fun p ω ↦ rewardByCount A R p.1.1 (p.1.2 - 1 + 1) ω := by
@@ -474,14 +477,14 @@ lemma iIndepFun_rewardByCount (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) :
 
 /-- For each action `a`, the rewards by count `(rewardByCount A R a (m + 1))_m` are independent
 (and by `hasLaw_rewardByCount_add_one` identically distributed with law `ν a`). -/
-lemma iIndepFun_rewardByCount_add_one_action (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+lemma iIndepFun_rewardByCount_add_one_action (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     (a : 𝓐) :
     iIndepFun (fun m ω ↦ rewardByCount A R a (m + 1) ω) 𝔓 :=
   (iIndepFun_rewardByCount_add_one h).precomp (g := fun m ↦ (a, m))
     fun _ _ hmn ↦ (Prod.mk.inj hmn).2
 
 /-- Two distinct rewards by count are independent. -/
-lemma indepFun_rewardByCount (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+lemma indepFun_rewardByCount (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     {a b : 𝓐} {m n : ℕ} (hm : m ≠ 0) (hn : n ≠ 0) (hne : (a, m) ≠ (b, n)) :
     rewardByCount A R a m ⟂ᵢ[𝔓] rewardByCount A R b n :=
   (iIndepFun_rewardByCount h).indepFun (i := ⟨(a, m), hm⟩) (j := ⟨(b, n), hn⟩)
diff --git a/LeanMachineLearning/Online/Bandit/SumRewards.lean b/LeanMachineLearning/Online/Bandit/SumRewards.lean
index 4d9fc081..1719cfd0 100644
--- a/LeanMachineLearning/Online/Bandit/SumRewards.lean
+++ b/LeanMachineLearning/Online/Bandit/SumRewards.lean
@@ -24,7 +24,7 @@ namespace ArrayModel
 
 variable {𝓐 : Type*} {m𝓐 : MeasurableSpace 𝓐} [DecidableEq 𝓐] [Countable 𝓐]
   [StandardBorelSpace 𝓐] [Nonempty 𝓐]
-  {alg : Algorithm 𝓐 ℝ} {ν : Kernel 𝓐 ℝ} [IsMarkovKernel ν]
+  {alg : Algorithm Unit 𝓐 ℝ} {ν : Kernel 𝓐 ℝ} [IsMarkovKernel ν]
 
 local notation "A" => action alg
 local notation "R" => reward alg
@@ -134,36 +134,39 @@ end ArrayModel
 variable {𝓐 Ω Ω' : Type*} [DecidableEq 𝓐] {m𝓐 : MeasurableSpace 𝓐} {mΩ : MeasurableSpace Ω}
   {mΩ' : MeasurableSpace Ω'}
   {P : Measure Ω} [IsProbabilityMeasure P] {P' : Measure Ω'} [IsProbabilityMeasure P']
-  {alg : Algorithm 𝓐 ℝ} {ν : Kernel 𝓐 ℝ} [IsMarkovKernel ν]
-  {A : ℕ → Ω → 𝓐} {R : ℕ → Ω → ℝ} {A₂ : ℕ → Ω' → 𝓐} {R₂ : ℕ → Ω' → ℝ}
+  {alg : Algorithm Unit 𝓐 ℝ} {ν : Kernel 𝓐 ℝ} [IsMarkovKernel ν]
+  {O : ℕ → Ω → Unit} {A : ℕ → Ω → 𝓐} {R : ℕ → Ω → ℝ}
+  {O₂ : ℕ → Ω' → Unit} {A₂ : ℕ → Ω' → 𝓐} {R₂ : ℕ → Ω' → ℝ}
   {ω : Ω} {m n t : ℕ} {a : 𝓐}
 
 lemma sumRewards_eq_comp :
     sumRewards A R a n =
-     (fun p ↦ ∑ i ∈ range n, if (p i).1 = a then (p i).2 else 0) ∘ (trajectory A R) := by
+     (fun p : ℕ → Round Unit 𝓐 ℝ ↦
+        ∑ i ∈ range n, if (p i).action = a then (p i).feedback else 0) ∘ (trajectory O A R) := by
   ext
   simp [sumRewards, trajectory]
   grind
 
 lemma pullCount_eq_comp :
     pullCount A a n =
-      (fun p ↦ ∑ i ∈ range n, if (p i).1 = a then 1 else 0) ∘ (trajectory A R) := by
+      (fun p : ℕ → Round Unit 𝓐 ℝ ↦ ∑ i ∈ range n, if (p i).action = a then 1 else 0) ∘
+        (trajectory O A R) := by
   ext
   simp [pullCount, trajectory]
   rfl
 
 -- todo: write those lemmas with IdentDistrib instead of equality of maps
 lemma _root_.Learning.IsAlgEnvSeq.law_sumRewards_unique [MeasurableSingletonClass 𝓐]
-    (h1 : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
-    (h2 : IsAlgEnvSeq A₂ R₂ alg (stationaryEnv ν) P') :
+    (h1 : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
+    (h2 : IsAlgEnvSeq O₂ A₂ R₂ alg (stationaryEnv ν) P') :
     P.map (sumRewards A R a n) = P'.map (sumRewards A₂ R₂ a n) := by
   have hA := h1.measurable_action
   have hR := h1.measurable_feedback
   have hA2 := h2.measurable_action
   have hR2 := h2.measurable_feedback
   have h_unique := isAlgEnvSeq_unique h1 h2
-  rw [sumRewards_eq_comp, sumRewards_eq_comp, ← Measure.map_map, h_unique, Measure.map_map,
-    ← sumRewards_eq_comp]
+  rw [sumRewards_eq_comp (O := O), sumRewards_eq_comp (O := O₂), ← Measure.map_map, h_unique,
+    Measure.map_map, ← sumRewards_eq_comp (O := O₂)]
   · refine measurable_sum _ fun i hi ↦ Measurable.ite ?_ (by fun_prop) (by fun_prop)
     exact (measurableSet_singleton _).preimage (by fun_prop)
   · fun_prop
@@ -172,12 +175,14 @@ lemma _root_.Learning.IsAlgEnvSeq.law_sumRewards_unique [MeasurableSingletonClas
   · fun_prop
 
 lemma _root_.Learning.IsAlgEnvSeq.law_pullCount_sumRewards_unique' [MeasurableSingletonClass 𝓐]
-    (h1 : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
-    (h2 : IsAlgEnvSeq A₂ R₂ alg (stationaryEnv ν) P') :
+    (h1 : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
+    (h2 : IsAlgEnvSeq O₂ A₂ R₂ alg (stationaryEnv ν) P') :
     IdentDistrib (fun ω a ↦ (pullCount A a n ω, sumRewards A R a n ω))
       (fun ω a ↦ (pullCount A₂ a n ω, sumRewards A₂ R₂ a n ω)) P P' := by
+  have hO := h1.measurable_obs
   have hA := h1.measurable_action
   have hR := h1.measurable_feedback
+  have hO2 := h2.measurable_obs
   have hA2 := h2.measurable_action
   have hR2 := h2.measurable_feedback
   constructor
@@ -188,8 +193,9 @@ lemma _root_.Learning.IsAlgEnvSeq.law_pullCount_sumRewards_unique' [MeasurableSi
     rw [measurable_pi_iff]
     exact fun a ↦ Measurable.prod (by fun_prop) (measurable_sumRewards hA2 hR2 _ _)
   have h_unique := isAlgEnvSeq_unique h1 h2
-  let f := fun (p : ℕ → 𝓐 × ℝ ) (a : 𝓐) ↦ (∑ i ∈ range n, if (p i).1 = a then 1 else 0,
-    ∑ i ∈ range n, if (p i).1 = a then (p i).2 else 0)
+  let f := fun (p : ℕ → Round Unit 𝓐 ℝ) (a : 𝓐) ↦
+    (∑ i ∈ range n, if (p i).action = a then 1 else 0,
+      ∑ i ∈ range n, if (p i).action = a then (p i).feedback else 0)
   have hf : Measurable f := by
     rw [measurable_pi_iff]
     intro a
@@ -201,44 +207,42 @@ lemma _root_.Learning.IsAlgEnvSeq.law_pullCount_sumRewards_unique' [MeasurableSi
       refine measurable_sum _ fun i hi ↦ Measurable.ite ?_ (by fun_prop) (by fun_prop)
       exact (measurableSet_singleton _).preimage (by fun_prop)
   have h_eq_comp : (fun ω a ↦ (pullCount A a n ω, sumRewards A R a n ω))
-      = f ∘ (trajectory A R) := by
+      = f ∘ (trajectory O A R) := by
     ext ω a : 2
-    rw [pullCount_eq_comp (R := R), sumRewards_eq_comp]
+    rw [pullCount_eq_comp (O := O) (R := R), sumRewards_eq_comp (O := O)]
     grind
   have h_eq_comp2 : (fun ω a ↦ (pullCount A₂ a n ω, sumRewards A₂ R₂ a n ω))
-      = f ∘ (trajectory A₂ R₂) := by
+      = f ∘ (trajectory O₂ A₂ R₂) := by
     ext ω a : 2
-    rw [pullCount_eq_comp (R := R₂), sumRewards_eq_comp]
+    rw [pullCount_eq_comp (O := O₂) (R := R₂), sumRewards_eq_comp (O := O₂)]
     grind
   rw [h_eq_comp, h_eq_comp2, ← Measure.map_map hf, h_unique, Measure.map_map hf,
     ← h_eq_comp2]
-  · rw [measurable_pi_iff]
-    exact fun n ↦ Measurable.prodMk (hA2 n) (hR2 n)
-  · rw [measurable_pi_iff]
-    exact fun n ↦ Measurable.prodMk (hA n) (hR n)
+  · fun_prop
+  · fun_prop
 
 lemma _root_.Learning.IsAlgEnvSeq.law_pullCount_sumRewards_unique [MeasurableSingletonClass 𝓐]
-    (h1 : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
-    (h2 : IsAlgEnvSeq A₂ R₂ alg (stationaryEnv ν) P') :
+    (h1 : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
+    (h2 : IsAlgEnvSeq O₂ A₂ R₂ alg (stationaryEnv ν) P') :
     P.map (fun ω ↦ (pullCount A a n ω, sumRewards A R a n ω)) =
       P'.map (fun ω ↦ (pullCount A₂ a n ω, sumRewards A₂ R₂ a n ω)) :=
   ((h1.law_pullCount_sumRewards_unique' h2 (n := n)).comp (u := fun f ↦ f a) (by fun_prop)).map_eq
 
 lemma _root_.Learning.IsAlgEnvSeq.identDistrib_pullCount_sumRewards [MeasurableSingletonClass 𝓐]
-    (h1 : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
-    (h2 : IsAlgEnvSeq A₂ R₂ alg (stationaryEnv ν) P') :
+    (h1 : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
+    (h2 : IsAlgEnvSeq O₂ A₂ R₂ alg (stationaryEnv ν) P') :
     IdentDistrib (fun ω n a ↦ (pullCount A a n ω, sumRewards A R a n ω))
       (fun ω' n a ↦ (pullCount A₂ a n ω', sumRewards A₂ R₂ a n ω')) P P' := by
-  let f (τ : ℕ → 𝓐 × ℝ) (n : ℕ) (a : 𝓐) : ℕ × ℝ :=
-    (∑ i ∈ range n, if (τ i).1 = a then 1 else 0,
-     ∑ i ∈ range n, if (τ i).1 = a then (τ i).2 else 0)
+  let f (τ : ℕ → Round Unit 𝓐 ℝ) (n : ℕ) (a : 𝓐) : ℕ × ℝ :=
+    (∑ i ∈ range n, if (τ i).action = a then 1 else 0,
+     ∑ i ∈ range n, if (τ i).action = a then (τ i).feedback else 0)
   have hc1 : (fun ω n a ↦ (pullCount A a n ω, sumRewards A R a n ω)) =
-      f ∘ (trajectory A R) := by
+      f ∘ (trajectory O A R) := by
     ext ω n a : 3
     simp_rw [Function.comp, f, pullCount, card_filter, sumRewards, trajectory]
     rfl
   have hc2 : (fun ω' n a ↦ (pullCount A₂ a n ω', sumRewards A₂ R₂ a n ω')) =
-      f ∘ (trajectory A₂ R₂) := by
+      f ∘ (trajectory O₂ A₂ R₂) := by
     ext ω' n a : 3
     simp_rw [Function.comp, f, pullCount, card_filter, sumRewards, trajectory]
     rfl
@@ -259,7 +263,7 @@ variable [Nonempty 𝓐]
 
 -- this is what we will use for UCB
 lemma prob_pullCount_prod_sumRewards_mem_le [Countable 𝓐] [MeasurableSingletonClass 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     {s : Set (ℕ × ℝ)} [DecidablePred (· ∈ Prod.fst '' s)] (hs : MeasurableSet s) :
     P {ω | (pullCount A a n ω, sumRewards A R a n ω) ∈ s} ≤
       ∑ k ∈ (range (n + 1)).filter (· ∈ Prod.fst '' s),
@@ -281,7 +285,7 @@ lemma prob_pullCount_prod_sumRewards_mem_le [Countable 𝓐] [MeasurableSingleto
     ArrayModel.prob_pullCount_prod_sumRewards_mem_le a n hs
 
 lemma prob_pullCount_mem_and_sumRewards_mem_le [Countable 𝓐] [MeasurableSingletonClass 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     {s : Set ℕ} [DecidablePred (· ∈ s)] (hs : MeasurableSet s) {B : Set ℝ} (hB : MeasurableSet B) :
     P {ω | pullCount A a n ω ∈ s ∧ sumRewards A R a n ω ∈ B} ≤
       ∑ k ∈ (range (n + 1)).filter (· ∈ s),
@@ -300,7 +304,7 @@ lemma prob_pullCount_mem_and_sumRewards_mem_le [Countable 𝓐] [MeasurableSingl
     simp [hk.2.1]
 
 lemma prob_sumRewards_mem_le [Countable 𝓐] [MeasurableSingletonClass 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     {B : Set ℝ} (hB : MeasurableSet B) :
     P (sumRewards A R a n ⁻¹' B) ≤
       ∑ k ∈ range (n + 1), streamMeasure ν {ω | ∑ i ∈ range k, ω i a ∈ B} := by
@@ -311,7 +315,7 @@ lemma prob_sumRewards_mem_le [Countable 𝓐] [MeasurableSingletonClass 𝓐]
   rfl
 
 lemma prob_pullCount_eq_and_sumRewards_mem_le [Countable 𝓐] [MeasurableSingletonClass 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     {m : ℕ} (hm : m ≤ n) {B : Set ℝ} (hB : MeasurableSet B) :
     P {ω | pullCount A a n ω = m ∧ sumRewards A R a n ω ∈ B} ≤
       streamMeasure ν {ω | ∑ i ∈ range m, ω i a ∈ B} := by
@@ -320,7 +324,7 @@ lemma prob_pullCount_eq_and_sumRewards_mem_le [Countable 𝓐] [MeasurableSingle
   simpa [hm'] using h_le
 
 lemma prob_exists_pullCount_eq_and_sumRewards_mem_le [Countable 𝓐] [MeasurableSingletonClass 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) (a : 𝓐) (m : ℕ) {B : Set ℝ}
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) (a : 𝓐) (m : ℕ) {B : Set ℝ}
     (hB : MeasurableSet B) :
     P {ω | ∃ n, pullCount A a n ω = m ∧ sumRewards A R a n ω ∈ B} ≤
       streamMeasure ν {ω | ∑ i ∈ range m, ω i a ∈ B} :=
@@ -337,7 +341,7 @@ lemma prob_exists_pullCount_eq_and_sumRewards_mem_le [Countable 𝓐] [Measurabl
     _ ≤ _ := ArrayModel.prob_exists_pullCount_eq_and_sumRewards_mem_le a m hB
 
 lemma probReal_sumRewards_le_sumRewards_le [Fintype 𝓐] [MeasurableSingletonClass 𝓐]
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P)
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P)
     (a : 𝓐) (n m₁ m₂ : ℕ) :
     P.real {ω | pullCount A (bestArm ν) n ω = m₁ ∧ pullCount A a n ω = m₂ ∧
         sumRewards A R (bestArm ν) n ω ≤ sumRewards A R a n ω} ≤
@@ -436,7 +440,7 @@ end StreamMeasure
 
 lemma prob_sumRewards_sub_pullCount_mul_ge_le [Countable 𝓐] [MeasurableSingletonClass 𝓐]
     {σ2 : ℝ≥0} (hσ2 : 0 < σ2) (ha : HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a))
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) {δ : ℝ} (hδ : 0 < δ) :
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) {δ : ℝ} (hδ : 0 < δ) :
     P {ω | ∃ t < n, pullCount A a t ω ≠ 0 ∧ √(2 * pullCount A a t ω * σ2 * Real.log (1 / δ)) ≤
       sumRewards A R a t ω - pullCount A a t ω * (ν a)[id]} ≤ ENNReal.ofReal ((n - 1) * δ) :=
   let B (m : ℕ) := {x : ℝ | √(2 * m * σ2 * Real.log (1 / δ)) ≤ x - m * (ν a)[id]}
@@ -470,7 +474,7 @@ lemma prob_sumRewards_sub_pullCount_mul_ge_le [Countable 𝓐] [MeasurableSingle
 
 lemma prob_sumRewards_sub_pullCount_mul_le_le [Countable 𝓐] [MeasurableSingletonClass 𝓐]
     {σ2 : ℝ≥0} (hσ2 : 0 < σ2) (ha : HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a))
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) {δ : ℝ} (hδ : 0 < δ) :
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) {δ : ℝ} (hδ : 0 < δ) :
     P {ω | ∃ t < n, pullCount A a t ω ≠ 0 ∧
       sumRewards A R a t ω - pullCount A a t ω * (ν a)[id] ≤
         -√(2 * pullCount A a t ω * σ2 * Real.log (1 / δ))} ≤ ENNReal.ofReal ((n - 1) * δ) :=
@@ -505,7 +509,7 @@ lemma prob_sumRewards_sub_pullCount_mul_le_le [Countable 𝓐] [MeasurableSingle
 
 lemma prob_sumRewards_sub_pullCount_mul_ge_le_of_Fintype [Fintype 𝓐] [MeasurableSingletonClass 𝓐]
     {σ2 : ℝ≥0} (hσ2 : 0 < σ2) (hν : ∀ a, HasSubgaussianMGF (fun x ↦ x - (ν a)[id]) σ2 (ν a))
-    (h : IsAlgEnvSeq A R alg (stationaryEnv ν) P) {δ : ℝ} (hδ : 0 < δ) :
+    (h : IsAlgEnvSeq O A R alg (stationaryEnv ν) P) {δ : ℝ} (hδ : 0 < δ) :
     P {ω | ∃ a, ∃ t < n, pullCount A a t ω ≠ 0 ∧
         √(2 * pullCount A a t ω * σ2 * Real.log (1 / δ)) ≤
           sumRewards A R a t ω - pullCount A a t ω * (ν a)[id]} ≤
@@ -671,8 +675,8 @@ namespace Learning.IsBayesAlgEnvSeq
 
 variable {𝓔 Ω : Type*} [MeasurableSpace 𝓔] [MeasurableSpace Ω]
 variable {K : ℕ} [Nonempty (Fin K)]
-variable {Q : Measure 𝓔} {κ : Kernel (𝓔 × Fin K) ℝ} [IsMarkovKernel κ]
-variable {alg : Algorithm (Fin K) ℝ}
+variable {Q : Measure 𝓔} [IsProbabilityMeasure Q] {κ : Kernel (𝓔 × Fin K) ℝ} [IsMarkovKernel κ]
+variable {alg : Algorithm Unit (Fin K) ℝ}
 variable {E : Ω → 𝓔} {A : ℕ → Ω → (Fin K)} {R : ℕ → Ω → ℝ}
 variable {P : Measure Ω} [IsProbabilityMeasure P]
 
@@ -699,19 +703,20 @@ lemma prob_empMean_sub_actionMean_ge_le (h : IsBayesAlgEnvSeq Q κ alg E A R P)
   have := h.measurable_param
   have := h.measurable_action
   have := h.measurable_feedback
-  let S := {(e, τ) | ∃ a, ∃ t < n, pullCount IT.action a t τ ≠ 0 ∧
+  let S : Set (𝓔 × (ℕ → Round Unit (Fin K) ℝ)) :=
+    {(e, τ) | ∃ a, ∃ t < n, pullCount IT.action a t τ ≠ 0 ∧
     √(2 * pullCount IT.action a t τ * σ2 * Real.log (1 / δ)) ≤
       sumRewards IT.action IT.feedback a t τ - pullCount IT.action a t τ * actionMean κ id a e}
   calc
-    _ ≤ (P.map (fun ω ↦ (E ω, trajectory A R ω))) S := by
+    _ ≤ (P.map (fun ω ↦ (E ω, trajectory (noObs Ω) A R ω))) S := by
         rw [Measure.map_apply (by fun_prop) (by measurability)]
         apply measure_mono
         intro ω ⟨t, ht, a, hpc, hle⟩
         rw [empMean] at hle
         exact ⟨a, t, ht, hpc, sqrt_two_mul_le_sub hpc hle⟩
-    _ = (P.map E ⊗ₘ condDistrib (trajectory A R) E P) S := by
+    _ = (P.map E ⊗ₘ condDistrib (trajectory (noObs Ω) A R) E P) S := by
         rw [← compProd_map_condDistrib (by fun_prop)]
-    _ = ∫⁻ e, condDistrib (trajectory A R) E P e (Prod.mk e ⁻¹' S) ∂(P.map E) :=
+    _ = ∫⁻ e, condDistrib (trajectory (noObs Ω) A R) E P e (Prod.mk e ⁻¹' S) ∂(P.map E) :=
         Measure.compProd_apply (by measurability)
     _ ≤ ∫⁻ e, ENNReal.ofReal (Fintype.card (Fin K) * (n - 1) * δ) ∂(P.map E) := by
         apply lintegral_mono_ae
@@ -738,20 +743,21 @@ lemma prob_empMean_bestAction_sub_actionMean_le_le (h : IsBayesAlgEnvSeq Q κ al
   have := h.measurable_param
   have := h.measurable_action
   have := h.measurable_feedback
-  let S := {(e, τ) | ∃ t < n, pullCount IT.action (bestAction κ id e) t τ ≠ 0 ∧
+  let S : Set (𝓔 × (ℕ → Round Unit (Fin K) ℝ)) :=
+    {(e, τ) | ∃ t < n, pullCount IT.action (bestAction κ id e) t τ ≠ 0 ∧
     sumRewards IT.action IT.feedback (bestAction κ id e) t τ -
         pullCount IT.action (bestAction κ id e) t τ * actionMean κ id (bestAction κ id e) e ≤
           -√(2 * pullCount IT.action (bestAction κ id e) t τ * σ2 * Real.log (1 / δ))}
   calc
-    _ ≤ (P.map (fun ω ↦ (E ω, trajectory A R ω))) S := by
+    _ ≤ (P.map (fun ω ↦ (E ω, trajectory (noObs Ω) A R ω))) S := by
         rw [Measure.map_apply (by fun_prop) (by measurability)]
         apply measure_mono
         intro ω ⟨t, ht, hpc, hle⟩
         rw [empMean] at hle
         exact ⟨t, ht, hpc, sub_le_neg_sqrt_two_mul hpc hle⟩
-    _ = (P.map E ⊗ₘ condDistrib (trajectory A R) E P) S := by
+    _ = (P.map E ⊗ₘ condDistrib (trajectory (noObs Ω) A R) E P) S := by
         rw [← compProd_map_condDistrib (by fun_prop)]
-    _ = ∫⁻ e, condDistrib (trajectory A R) E P e (Prod.mk e ⁻¹' S) ∂(P.map E) :=
+    _ = ∫⁻ e, condDistrib (trajectory (noObs Ω) A R) E P e (Prod.mk e ⁻¹' S) ∂(P.map E) :=
         Measure.compProd_apply (by measurability)
     _ ≤ ∫⁻ e, ENNReal.ofReal ((n - 1) * δ) ∂(P.map E) := by
         apply lintegral_mono_ae
diff --git a/LeanMachineLearning/SequentialLearning/ActionIndicator.lean b/LeanMachineLearning/SequentialLearning/ActionIndicator.lean
index 6c17a2d6..5b5debc8 100644
--- a/LeanMachineLearning/SequentialLearning/ActionIndicator.lean
+++ b/LeanMachineLearning/SequentialLearning/ActionIndicator.lean
@@ -33,8 +33,9 @@ open MeasureTheory ProbabilityTheory Filter Finset
 
 namespace Learning
 
-variable {Ω 𝓐 𝓨 : Type*} {mΩ : MeasurableSpace Ω} {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨}
-  [MeasurableSingletonClass 𝓐] {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {P : Measure Ω}
+variable {Ω 𝓞 𝓐 𝓨 : Type*} {mΩ : MeasurableSpace Ω} {m𝓞 : MeasurableSpace 𝓞}
+  {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨}
+  [MeasurableSingletonClass 𝓐] {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {P : Measure Ω}
 
 /-- The `{0,1}`-valued assignment indicator of action `k`:
 `actionIndicator A k n ω = 𝟙{A n ω = k}`. -/
@@ -93,16 +94,16 @@ lemma integrable_actionIndicator (P : Measure Ω) [IsFiniteMeasure P]
 
 /-- The action indicator is adapted to the history filtration: whether action `k` was chosen at `n`
 is known at time `n`. -/
-lemma IsAlgEnvSeq.adapted_actionIndicator {alg : Algorithm 𝓐 𝓨} {env : Environment 𝓐 𝓨}
-    [IsFiniteMeasure P] (h : IsAlgEnvSeq A Y alg env P) (k : 𝓐) :
+lemma IsAlgEnvSeq.adapted_actionIndicator {alg : Algorithm 𝓞 𝓐 𝓨} {env : Environment 𝓞 𝓐 𝓨}
+    [IsFiniteMeasure P] (h : IsAlgEnvSeq O A Y alg env P) (k : 𝓐) :
     Adapted h.filtration (actionIndicator A k) :=
   fun _ ↦ Measurable.indicator measurable_const (h.adapted_action _ (measurableSet_singleton k))
 
 /-- The action indicator is adapted to the history+action filtration: whether action `k` was chosen
 at `n` is known once we know the action at `n`. -/
 lemma IsAlgEnvSeq.adapted_actionIndicator_filtrationAction
-    {alg : Algorithm 𝓐 𝓨} {env : Environment 𝓐 𝓨}
-    [IsFiniteMeasure P] (h : IsAlgEnvSeq A Y alg env P) (k : 𝓐) :
+    {alg : Algorithm 𝓞 𝓐 𝓨} {env : Environment 𝓞 𝓐 𝓨}
+    [IsFiniteMeasure P] (h : IsAlgEnvSeq O A Y alg env P) (k : 𝓐) :
     Adapted h.filtrationAction (actionIndicator A k) :=
   fun _ ↦ Measurable.indicator measurable_const
     (h.adapted_action_filtrationAction _ (measurableSet_singleton k))
diff --git a/LeanMachineLearning/SequentialLearning/Algorithm.lean b/LeanMachineLearning/SequentialLearning/Algorithm.lean
index 7f95e258..c711163c 100644
--- a/LeanMachineLearning/SequentialLearning/Algorithm.lean
+++ b/LeanMachineLearning/SequentialLearning/Algorithm.lean
@@ -12,27 +12,32 @@ public import LeanMachineLearning.ForMathlib.Probability.Kernel.IonescuTulcea.Tr
 # Algorithms and environments
 
 We define structures for stochastic, sequential algorithms and environments, and the notion of an
-algorithm-environment sequence, which is a sequence of actions and feedbacks generated by
-an algorithm interacting with an environment.
-
-At time `n`, the algorithm has observed the `n` action-feedback pairs at times `0, ..., n - 1`.
-That history is an element of `Fin n → 𝓐 × 𝓨`. The algorithm then chooses an action `A n`
-according to its policy, and the environment returns a feedback `Y n` given the history and `A n`.
-In particular, the first action is chosen according to the policy at time `0`, applied to the
-empty history (the unique element of `Fin 0 → 𝓐 × 𝓨`).
+algorithm-environment sequence, which is a sequence of observations, actions and feedbacks generated
+by an algorithm interacting with an environment.
+
+A round of interaction consists of an observation in `𝓞`, an action in `𝓐` and a feedback in `𝓨`,
+in that order: it is an element of `Round 𝓞 𝓐 𝓨 = 𝓞 × 𝓐 × 𝓨`.
+At time `n`, the algorithm has seen the `n` complete rounds at times `0, ..., n - 1`.
+That history is an element of `Hist 𝓞 𝓐 𝓨 n = Fin n → Round 𝓞 𝓐 𝓨`. The environment then draws
+an observation `O n` given the history, the algorithm chooses an action `A n` according to its
+policy applied to the history and `O n`, and the environment returns a feedback `Y n` given the
+history, `O n` and `A n`.
+In particular, the first observation is drawn according to the observation kernel at time `0`,
+applied to the empty history (the unique element of `Hist 𝓞 𝓐 𝓨 0`).
 
 ## Main definitions
 
-* `Algorithm 𝓐 𝓨`: a stochastic, sequential algorithm.
-* `Environment 𝓐 𝓨`: a stochastic environment.
-* `history A Y n`: the history before time `n`, a random variable with values in `Fin n → 𝓐 × 𝓨`.
-* `IsAlgEnvSeq A 𝓨 alg env P`: an algorithm-environment sequence. That is, a sequence of
-  actions `A` and feedback `Y` that have the correct conditional distributions to be generated by
-  an algorithm `alg` interacting with an environment `env`, defined on a probability space `(Ω, P)`.
-* `IsAlgEnvSeqUntil A Y alg env P N`: `A` and `Y` form an algorithm-environment sequence for the
-  times `n < N`.
-* `prod_left alg`: an `Algorithm 𝓐 (𝓧 × 𝓨)` obtained from an algorithm `alg : Algorithm 𝓐 𝓨` by
-  ignoring the `𝓧` component of each observation.
+* `Round 𝓞 𝓐 𝓨`: one round of interaction, an observation-action-feedback triple.
+* `Hist 𝓞 𝓐 𝓨 n`: a history of `n` complete rounds.
+* `Algorithm 𝓞 𝓐 𝓨`: a stochastic, sequential algorithm.
+* `Environment 𝓞 𝓐 𝓨`: a stochastic environment.
+* `history O A Y n`: the history before time `n`, a random variable with values in `Hist 𝓞 𝓐 𝓨 n`.
+* `IsAlgEnvSeq O A Y alg env P`: an algorithm-environment sequence. That is, a sequence of
+  observations `O`, actions `A` and feedbacks `Y` that have the correct conditional distributions
+  to be generated by an algorithm `alg` interacting with an environment `env`, defined on a
+  probability space `(Ω, P)`.
+* `IsAlgEnvSeqUntil O A Y alg env P N`: `O`, `A` and `Y` form an algorithm-environment sequence for
+  the times `n < N`.
 
 -/
 
@@ -42,149 +47,237 @@ open MeasureTheory ProbabilityTheory Filter Real Finset
 
 open scoped ENNReal NNReal
 
+-- TODO: this belongs to Mathlib.
+instance Prod.instUnique {α β : Type*} [Unique α] [Unique β] : Unique (α × β) where
+  default := (default, default)
+  uniq p := by simp [Prod.ext_iff, Unique.eq_default]
+
 namespace Learning
 
-variable {𝓐 𝓨 Ω : Type*} {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨} {mΩ : MeasurableSpace Ω}
+variable {𝓞 𝓐 𝓨 Ω : Type*} {m𝓞 : MeasurableSpace 𝓞} {m𝓐 : MeasurableSpace 𝓐}
+  {m𝓨 : MeasurableSpace 𝓨} {mΩ : MeasurableSpace Ω}
 
-/-- A stochastic, sequential algorithm. -/
-structure Algorithm (𝓐 𝓨 : Type*) [MeasurableSpace 𝓐] [MeasurableSpace 𝓨] where
-  /-- Policy or sampling rule: distribution of the action at time `n` given the history of the
-  `n` previous action-feedback pairs. -/
-  policy : (n : ℕ) → Kernel (Fin n → 𝓐 × 𝓨) 𝓐
-  /-- The policy is a Markov kernel. -/
-  [h_policy : ∀ n, IsMarkovKernel (policy n)]
+/-- One round of interaction: an observation, then an action, then a feedback. -/
+abbrev Round (𝓞 𝓐 𝓨 : Type*) := 𝓞 × 𝓐 × 𝓨
+
+/-- The observation of a round. -/
+def Round.obs (r : Round 𝓞 𝓐 𝓨) : 𝓞 := r.1
+/-- The action of a round. -/
+def Round.action (r : Round 𝓞 𝓐 𝓨) : 𝓐 := r.2.1
+/-- The feedback of a round. -/
+def Round.feedback (r : Round 𝓞 𝓐 𝓨) : 𝓨 := r.2.2
+
+namespace Round
 
-instance (alg : Algorithm 𝓐 𝓨) (n : ℕ) : IsMarkovKernel (alg.policy n) := alg.h_policy n
+@[simp] lemma obs_mk (o : 𝓞) (a : 𝓐) (y : 𝓨) : Round.obs (o, a, y) = o := rfl
+@[simp] lemma action_mk (o : 𝓞) (a : 𝓐) (y : 𝓨) : Round.action (o, a, y) = a := rfl
+@[simp] lemma feedback_mk (o : 𝓞) (a : 𝓐) (y : 𝓨) : Round.feedback (o, a, y) = y := rfl
+
+@[simp]
+lemma mk_obs_action_feedback (r : Round 𝓞 𝓐 𝓨) : (r.obs, r.action, r.feedback) = r := rfl
+
+lemma obs_eq_fst (r : Round 𝓞 𝓐 𝓨) : r.obs = r.1 := rfl
+lemma action_eq_snd_fst (r : Round 𝓞 𝓐 𝓨) : r.action = r.2.1 := rfl
+lemma feedback_eq_snd_snd (r : Round 𝓞 𝓐 𝓨) : r.feedback = r.2.2 := rfl
+
+@[fun_prop]
+lemma measurable_obs : Measurable (Round.obs (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨)) := measurable_fst
 
-/-- Distribution of the first action: the policy at time `0` applied to the empty history. -/
-def Algorithm.p0 (alg : Algorithm 𝓐 𝓨) : Measure 𝓐 :=
-  alg.policy 0 default
+@[fun_prop]
+lemma measurable_action : Measurable (Round.action (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨)) :=
+  measurable_snd.fst
+
+@[fun_prop]
+lemma measurable_feedback : Measurable (Round.feedback (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨)) :=
+  measurable_snd.snd
+
+end Round
+
+/-- History of `n` complete rounds; `n = 0` is the empty history. -/
+abbrev Hist (𝓞 𝓐 𝓨 : Type*) (n : ℕ) := Fin n → Round 𝓞 𝓐 𝓨
+
+/-- A stochastic, sequential algorithm.
+At each round, it sees an observation in `𝓞`, then takes an action in `𝓐`, and finally receives
+feedback in `𝓨`. The action is a random function of the past rounds and the current observation. -/
+@[ext]
+structure Algorithm (𝓞 𝓐 𝓨 : Type*) [MeasurableSpace 𝓞] [MeasurableSpace 𝓐] [MeasurableSpace 𝓨]
+    where
+  /-- Law of the action of round `n` given the past rounds and the current observation. -/
+  policy : (n : ℕ) → Kernel (Hist 𝓞 𝓐 𝓨 n × 𝓞) 𝓐
+  /-- The policy is a Markov kernel. -/
+  [isMarkovKernel_policy : ∀ n, IsMarkovKernel (policy n)]
+
+instance (alg : Algorithm 𝓞 𝓐 𝓨) (n : ℕ) : IsMarkovKernel (alg.policy n) :=
+  alg.isMarkovKernel_policy n
+
+/-- A stochastic environment.
+At each round, an observation is drawn prior to the algorithm taking an action. Then the environment
+provides feedback based on the observation and the action. -/
+@[ext]
+structure Environment (𝓞 𝓐 𝓨 : Type*) [MeasurableSpace 𝓞] [MeasurableSpace 𝓐] [MeasurableSpace 𝓨]
+    where
+  /-- Law of the observation of round `n` given the past rounds. -/
+  obs : (n : ℕ) → Kernel (Hist 𝓞 𝓐 𝓨 n) 𝓞
+  /-- Law of the feedback of round `n` given the past rounds, the observation and the action. -/
+  feedback : (n : ℕ) → Kernel ((Hist 𝓞 𝓐 𝓨 n × 𝓞) × 𝓐) 𝓨
+  /-- The observation kernel is a Markov kernel. -/
+  [isMarkovKernel_obs : ∀ n, IsMarkovKernel (obs n)]
+  /-- The feedback kernel is a Markov kernel. -/
+  [isMarkovKernel_feedback : ∀ n, IsMarkovKernel (feedback n)]
+
+instance (env : Environment 𝓞 𝓐 𝓨) (n : ℕ) : IsMarkovKernel (env.obs n) := env.isMarkovKernel_obs n
+instance (env : Environment 𝓞 𝓐 𝓨) (n : ℕ) : IsMarkovKernel (env.feedback n) :=
+  env.isMarkovKernel_feedback n
+
+/-- Distribution of the first observation: the observation kernel at time `0` applied to the empty
+history. -/
+def Environment.obs0 (env : Environment 𝓞 𝓐 𝓨) : Measure 𝓞 :=
+  env.obs 0 default
 deriving IsProbabilityMeasure
 
-lemma Algorithm.p0_def (alg : Algorithm 𝓐 𝓨) : alg.p0 = alg.policy 0 default := rfl
+lemma Environment.obs0_def (env : Environment 𝓞 𝓐 𝓨) : env.obs0 = env.obs 0 default := rfl
 
-lemma Algorithm.policy_zero (alg : Algorithm 𝓐 𝓨) (h : Fin 0 → 𝓐 × 𝓨) :
-    alg.policy 0 h = alg.p0 := by
+lemma Environment.obs_zero (env : Environment 𝓞 𝓐 𝓨) (h : Hist 𝓞 𝓐 𝓨 0) :
+    env.obs 0 h = env.obs0 := by
   rw [Unique.eq_default h]
   rfl
 
-/-- An algorithm with observations in `𝓧 × 𝓨` obtained from an algorithm with observations in `𝓨`
-by ignoring the `𝓧` component of each observation. -/
-@[simps]
-def Algorithm.prodLeft (𝓧 : Type*) [MeasurableSpace 𝓧] (alg : Algorithm 𝓐 𝓨) :
-    Algorithm 𝓐 (𝓧 × 𝓨) where
-  policy n := (alg.policy n).comap (fun h i ↦ ((h i).1, (h i).2.2)) (by fun_prop)
+/-- Distribution of the first action given the first observation: the policy at time `0` applied to
+the empty history. -/
+noncomputable def Algorithm.p0 (alg : Algorithm 𝓞 𝓐 𝓨) : Kernel 𝓞 𝓐 :=
+  (alg.policy 0).sectR default
+deriving IsMarkovKernel
 
-@[simp]
-lemma Algorithm.prodLeft_p0 (𝓧 : Type*) [MeasurableSpace 𝓧] (alg : Algorithm 𝓐 𝓨) :
-    (alg.prodLeft 𝓧).p0 = alg.p0 := by
-  rw [p0_def, prodLeft_policy, Kernel.comap_apply, policy_zero]
-
-/-- A stochastic environment. -/
-structure Environment (𝓐 𝓨 : Type*) [MeasurableSpace 𝓐] [MeasurableSpace 𝓨] where
-  /-- Distribution of the feedback at time `n` as function of the history of the `n` previous
-  action-feedback pairs and of the action at time `n`. -/
-  feedback : (n : ℕ) → Kernel ((Fin n → 𝓐 × 𝓨) × 𝓐) 𝓨
-  /-- The feedback kernels are Markov kernels. -/
-  [h_feedback : ∀ n, IsMarkovKernel (feedback n)]
-
-instance (env : Environment 𝓐 𝓨) (n : ℕ) : IsMarkovKernel (env.feedback n) := env.h_feedback n
-
-/-- Distribution of the first feedback given the first action: the feedback kernel at time `0`
-applied to the empty history. -/
-noncomputable def Environment.ν0 (env : Environment 𝓐 𝓨) : Kernel 𝓐 𝓨 :=
-  (env.feedback 0).sectR default
+lemma Algorithm.p0_def (alg : Algorithm 𝓞 𝓐 𝓨) : alg.p0 = (alg.policy 0).sectR default := rfl
+
+lemma Algorithm.p0_apply (alg : Algorithm 𝓞 𝓐 𝓨) (o : 𝓞) :
+    alg.p0 o = alg.policy 0 (default, o) := rfl
+
+lemma Algorithm.policy_zero (alg : Algorithm 𝓞 𝓐 𝓨) (h : Hist 𝓞 𝓐 𝓨 0) (o : 𝓞) :
+    alg.policy 0 (h, o) = alg.p0 o := by
+  rw [Unique.eq_default h]
+  rfl
+
+/-- Distribution of the first feedback given the first observation and action: the feedback kernel
+at time `0` applied to the empty history. -/
+noncomputable def Environment.ν0 (env : Environment 𝓞 𝓐 𝓨) : Kernel (𝓞 × 𝓐) 𝓨 :=
+  (env.feedback 0).comap (fun p ↦ ((default, p.1), p.2)) (by fun_prop)
 deriving IsMarkovKernel
 
-lemma Environment.ν0_def (env : Environment 𝓐 𝓨) : env.ν0 = (env.feedback 0).sectR default := rfl
+lemma Environment.ν0_def (env : Environment 𝓞 𝓐 𝓨) :
+    env.ν0 = (env.feedback 0).comap (fun p ↦ ((default, p.1), p.2)) (by fun_prop) := rfl
 
-lemma Environment.ν0_apply (env : Environment 𝓐 𝓨) (a : 𝓐) :
-    env.ν0 a = env.feedback 0 (default, a) := rfl
+lemma Environment.ν0_apply (env : Environment 𝓞 𝓐 𝓨) (o : 𝓞) (a : 𝓐) :
+    env.ν0 (o, a) = env.feedback 0 ((default, o), a) := rfl
 
-lemma Environment.feedback_zero (env : Environment 𝓐 𝓨) (h : Fin 0 → 𝓐 × 𝓨) (a : 𝓐) :
-    env.feedback 0 (h, a) = env.ν0 a := by
+lemma Environment.feedback_zero (env : Environment 𝓞 𝓐 𝓨) (h : Hist 𝓞 𝓐 𝓨 0) (o : 𝓞) (a : 𝓐) :
+    env.feedback 0 ((h, o), a) = env.ν0 (o, a) := by
   rw [Unique.eq_default h]
   rfl
 
-/-- Kernel describing the distribution of the action-feedback pair at time `n` given the history
-before `n`. -/
+/-- Kernel describing the distribution of the round at time `n` given the history before `n`. -/
 noncomputable
-def stepKernel (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) (n : ℕ) :
-    Kernel (Fin n → 𝓐 × 𝓨) (𝓐 × 𝓨) :=
-  alg.policy n ⊗ₖ env.feedback n
+def stepKernel (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) (n : ℕ) :
+    Kernel (Hist 𝓞 𝓐 𝓨 n) (Round 𝓞 𝓐 𝓨) :=
+  env.obs n ⊗ₖ (alg.policy n ⊗ₖ env.feedback n)
 deriving IsMarkovKernel
 
-lemma stepKernel_def (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) (n : ℕ) :
-    stepKernel alg env n = alg.policy n ⊗ₖ env.feedback n := rfl
+lemma stepKernel_def (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) (n : ℕ) :
+    stepKernel alg env n = env.obs n ⊗ₖ (alg.policy n ⊗ₖ env.feedback n) := rfl
 
 @[simp]
-lemma fst_stepKernel (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) (n : ℕ) :
-    (stepKernel alg env n).fst = alg.policy n := by
+lemma fst_stepKernel (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) (n : ℕ) :
+    (stepKernel alg env n).fst = env.obs n := by
   rw [stepKernel, Kernel.fst_compProd]
 
-lemma stepKernel_zero (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) (h : Fin 0 → 𝓐 × 𝓨) :
-    stepKernel alg env 0 h = alg.p0 ⊗ₘ env.ν0 := by
+lemma stepKernel_zero (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) (h : Hist 𝓞 𝓐 𝓨 0) :
+    stepKernel alg env 0 h = env.obs0 ⊗ₘ (alg.p0 ⊗ₖ env.ν0) := by
   rw [Unique.eq_default h, stepKernel, Kernel.compProd_apply_eq_compProd_sectR]
+  congr 1
+  ext o s hs
+  rw [Kernel.sectR_apply, Kernel.compProd_apply hs, Kernel.compProd_apply hs]
   rfl
 
 section IsAlgEnvSeq
 
-variable {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {alg : Algorithm 𝓐 𝓨} {env : Environment 𝓐 𝓨}
+variable {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
+    {alg : Algorithm 𝓞 𝓐 𝓨} {env : Environment 𝓞 𝓐 𝓨}
     {P : Measure Ω} [IsFiniteMeasure P] {N : ℕ}
 
-/-- Step of the algorithm-environment sequence: the action-feedback pair at time `n`. -/
-def step (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) (n : ℕ) (ω : Ω) : 𝓐 × 𝓨 :=
-  (A n ω, Y n ω)
+/-- Step of the algorithm-environment sequence: the round at time `n`. -/
+def step (O : ℕ → Ω → 𝓞) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) (n : ℕ) (ω : Ω) : Round 𝓞 𝓐 𝓨 :=
+  (O n ω, A n ω, Y n ω)
+
+@[simp] lemma obs_step (n : ℕ) (ω : Ω) : (step O A Y n ω).obs = O n ω := rfl
+@[simp] lemma action_step (n : ℕ) (ω : Ω) : (step O A Y n ω).action = A n ω := rfl
+@[simp] lemma feedback_step (n : ℕ) (ω : Ω) : (step O A Y n ω).feedback = Y n ω := rfl
 
 @[fun_prop]
-lemma measurable_step (n : ℕ) (hA : Measurable (A n)) (hY : Measurable (Y n)) :
-    Measurable (step A Y n) := by
+lemma measurable_step (n : ℕ) (hO : Measurable (O n)) (hA : Measurable (A n))
+    (hY : Measurable (Y n)) :
+    Measurable (step O A Y n) := by
   unfold step
   fun_prop
 
-/-- A random variable that gives the sequence of action-feedback pairs. -/
-def trajectory (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) (ω : Ω) : ℕ → 𝓐 × 𝓨 := fun n ↦ (A n ω, Y n ω)
+/-- A random variable that gives the sequence of rounds. -/
+def trajectory (O : ℕ → Ω → 𝓞) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) (ω : Ω) : ℕ → Round 𝓞 𝓐 𝓨 :=
+  fun n ↦ (O n ω, A n ω, Y n ω)
 
 @[fun_prop]
-lemma measurable_trajectory {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} (hA : ∀ n, Measurable (A n))
-    (hR : ∀ n, Measurable (Y n)) : Measurable (trajectory A Y) := by
+lemma measurable_trajectory {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
+    (hO : ∀ n, Measurable (O n)) (hA : ∀ n, Measurable (A n))
+    (hR : ∀ n, Measurable (Y n)) : Measurable (trajectory O A Y) := by
   unfold trajectory
   fun_prop
 
-/-- History of the algorithm-environment sequence before time `n`: the action-feedback pairs at
+/-- History of the algorithm-environment sequence before time `n`: the rounds at
 times `0, ..., n - 1`. -/
-def history (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) (n : ℕ) (ω : Ω) : Fin n → 𝓐 × 𝓨 :=
-  fun i ↦ (A i ω, Y i ω)
+def history (O : ℕ → Ω → 𝓞) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) (n : ℕ) (ω : Ω) : Hist 𝓞 𝓐 𝓨 n :=
+  fun i ↦ (O i ω, A i ω, Y i ω)
 
-lemma history_apply (n : ℕ) (ω : Ω) (i : Fin n) : history A Y n ω i = (A i ω, Y i ω) := rfl
+lemma history_apply (n : ℕ) (ω : Ω) (i : Fin n) :
+    history O A Y n ω i = (O i ω, A i ω, Y i ω) := rfl
 
 @[simp]
-lemma history_zero : history A Y 0 = fun _ ↦ (default : Fin 0 → 𝓐 × 𝓨) := Unique.eq_default _
+lemma history_zero : history O A Y 0 = fun _ ↦ (default : Hist 𝓞 𝓐 𝓨 0) := Unique.eq_default _
 
 @[fun_prop]
-lemma measurable_history (hA : ∀ n, Measurable (A n))
+lemma measurable_history (hO : ∀ n, Measurable (O n)) (hA : ∀ n, Measurable (A n))
     (hY : ∀ n, Measurable (Y n)) (n : ℕ) :
-    Measurable (history A Y n) := by
+    Measurable (history O A Y n) := by
   unfold history
   fun_prop
 
+lemma history_congr {O' : ℕ → Ω → 𝓞} {A' : ℕ → Ω → 𝓐} {Y' : ℕ → Ω → 𝓨} {P : Measure Ω}
+    (hO : ∀ n, O' n =ᵐ[P] O n) (hA : ∀ n, A' n =ᵐ[P] A n) (hY : ∀ n, Y' n =ᵐ[P] Y n) (n : ℕ) :
+    history O' A' Y' n =ᵐ[P] history O A Y n := by
+  have h : ∀ᵐ ω ∂P, ∀ i : Fin n, (O' i ω = O i ω ∧ A' i ω = A i ω) ∧ Y' i ω = Y i ω := by
+    rw [ae_all_iff]
+    exact fun i ↦ ((hO i).and (hA i)).and (hY i)
+  filter_upwards [h] with ω hω
+  funext i
+  rw [history_apply, history_apply, (hω i).1.1, (hω i).1.2, (hω i).2]
+
 lemma eval_comp_history (n : ℕ) :
-    (fun x ↦ x (Fin.last n)) ∘ (history A Y (n + 1)) = step A Y n := rfl
+    (fun x ↦ x (Fin.last n)) ∘ (history O A Y (n + 1)) = step O A Y n := rfl
 
-lemma fst_eval_comp_history (n : ℕ) :
-    (fun x ↦ (x (Fin.last n)).1) ∘ (history A Y (n + 1)) = A n := rfl
+lemma obs_eval_comp_history (n : ℕ) :
+    (fun x ↦ (x (Fin.last n)).obs) ∘ (history O A Y (n + 1)) = O n := rfl
 
-lemma snd_eval_comp_history (n : ℕ) :
-    (fun x ↦ (x (Fin.last n)).2) ∘ (history A Y (n + 1)) = Y n := rfl
+lemma action_eval_comp_history (n : ℕ) :
+    (fun x ↦ (x (Fin.last n)).action) ∘ (history O A Y (n + 1)) = A n := rfl
+
+lemma feedback_eval_comp_history (n : ℕ) :
+    (fun x ↦ (x (Fin.last n)).feedback) ∘ (history O A Y (n + 1)) = Y n := rfl
 
 /-- The history before time `m` is a restriction of the history before time `n ≥ m`. -/
 lemma history_eq_comp_history {m n : ℕ} (hmn : m ≤ n) :
-    history A Y m = (fun h (i : Fin m) ↦ h (Fin.castLE hmn i)) ∘ history A Y n := rfl
+    history O A Y m = (fun h (i : Fin m) ↦ h (Fin.castLE hmn i)) ∘ history O A Y n := rfl
 
 lemma history_succ (n : ℕ) :
-    history A Y (n + 1) =
-      (MeasurableEquiv.finSuccProd (𝓐 × 𝓨) n).symm ∘
-        (fun ω ↦ (history A Y n ω, step A Y n ω)) := by
+    history O A Y (n + 1) =
+      (MeasurableEquiv.finSuccProd (Round 𝓞 𝓐 𝓨) n).symm ∘
+        (fun ω ↦ (history O A Y n ω, step O A Y n ω)) := by
   funext ω
   simp only [Function.comp_apply, MeasurableEquiv.finSuccProd_symm_apply]
   funext i
@@ -192,270 +285,415 @@ lemma history_succ (n : ℕ) :
   · simp [history, step]
   · simp [history]
 
-/-- An algorithm-environment sequence: a sequence of actions and feedbacks generated
+/-- An algorithm-environment sequence: a sequence of observations, actions and feedbacks generated
 by an algorithm interacting with an environment. -/
 structure IsAlgEnvSeq
-    (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨)
+    (O : ℕ → Ω → 𝓞) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨)
+    (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨)
     (P : Measure Ω) [IsFiniteMeasure P] : Prop where
+  /-- The observation sequence is measurable. -/
+  measurable_obs n : Measurable (O n) := by fun_prop
   /-- The action sequence is measurable. -/
   measurable_action n : Measurable (A n) := by fun_prop
   /-- The feedback sequence is measurable. -/
   measurable_feedback n : Measurable (Y n) := by fun_prop
-  /-- The action at time `n` has the correct conditional distribution given the history. -/
+  /-- The observation at time `n` has the correct conditional distribution given the history. -/
+  hasCondDistrib_obs n :
+    HasCondDistrib (O n) (history O A Y n) (env.obs n) P
+  /-- The action at time `n` has the correct conditional distribution given the history and the
+  observation at time `n`. -/
   hasCondDistrib_action n :
-    HasCondDistrib (A n) (history A Y n) (alg.policy n) P
-  /-- The feedback at time `n` has the correct conditional distribution given the history and
-  the action at time `n`. -/
+    HasCondDistrib (A n) (fun ω ↦ (history O A Y n ω, O n ω)) (alg.policy n) P
+  /-- The feedback at time `n` has the correct conditional distribution given the history, the
+  observation and the action at time `n`. -/
   hasCondDistrib_feedback n :
-    HasCondDistrib (Y n) (fun ω ↦ (history A Y n ω, A n ω)) (env.feedback n) P
+    HasCondDistrib (Y n) (fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) (env.feedback n) P
 
-/-- An algorithm-environment sequence until time `N`: a sequence of actions and feedbacks such that
-the actions and feedbacks at times `n < N` are generated by an algorithm interacting with
-an environment. In particular, the law of `history A Y N` is determined. -/
+/-- An algorithm-environment sequence until time `N`: a sequence of observations, actions and
+feedbacks such that the rounds at times `n < N` are generated by an algorithm interacting with
+an environment. In particular, the law of `history O A Y N` is determined. -/
 structure IsAlgEnvSeqUntil
-    (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨)
+    (O : ℕ → Ω → 𝓞) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨)
+    (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨)
     (P : Measure Ω) [IsFiniteMeasure P] (N : ℕ) : Prop where
+  /-- The observation sequence is measurable. -/
+  measurable_obs n : Measurable (O n) := by fun_prop
   /-- The action sequence is measurable. -/
   measurable_action n : Measurable (A n) := by fun_prop
   /-- The feedback sequence is measurable. -/
   measurable_feedback n : Measurable (Y n) := by fun_prop
-  /-- The action at time `n < N` has the correct conditional distribution given the history. -/
+  /-- The observation at time `n < N` has the correct conditional distribution given the history. -/
+  hasCondDistrib_obs n (hn : n < N) :
+    HasCondDistrib (O n) (history O A Y n) (env.obs n) P
+  /-- The action at time `n < N` has the correct conditional distribution given the history and the
+  observation at time `n`. -/
   hasCondDistrib_action n (hn : n < N) :
-    HasCondDistrib (A n) (history A Y n) (alg.policy n) P
-  /-- The feedback at time `n < N` has the correct conditional distribution given the history and
-  the action at time `n`. -/
+    HasCondDistrib (A n) (fun ω ↦ (history O A Y n ω, O n ω)) (alg.policy n) P
+  /-- The feedback at time `n < N` has the correct conditional distribution given the history, the
+  observation and the action at time `n`. -/
   hasCondDistrib_feedback n (hn : n < N) :
-    HasCondDistrib (Y n) (fun ω ↦ (history A Y n ω, A n ω)) (env.feedback n) P
+    HasCondDistrib (Y n) (fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) (env.feedback n) P
 
-lemma IsAlgEnvSeqUntil.mono (h : IsAlgEnvSeqUntil A Y alg env P N) {N' : ℕ} (hN : N' ≤ N) :
-    IsAlgEnvSeqUntil A Y alg env P N' where
+lemma IsAlgEnvSeqUntil.mono (h : IsAlgEnvSeqUntil O A Y alg env P N) {N' : ℕ} (hN : N' ≤ N) :
+    IsAlgEnvSeqUntil O A Y alg env P N' where
+  measurable_obs := h.measurable_obs
   measurable_action := h.measurable_action
   measurable_feedback := h.measurable_feedback
+  hasCondDistrib_obs n hn := h.hasCondDistrib_obs n (hn.trans_le hN)
   hasCondDistrib_action n hn := h.hasCondDistrib_action n (hn.trans_le hN)
   hasCondDistrib_feedback n hn := h.hasCondDistrib_feedback n (hn.trans_le hN)
 
-lemma IsAlgEnvSeq.isAlgEnvSeqUntil (h : IsAlgEnvSeq A Y alg env P) (N : ℕ) :
-    IsAlgEnvSeqUntil A Y alg env P N where
+lemma IsAlgEnvSeq.isAlgEnvSeqUntil (h : IsAlgEnvSeq O A Y alg env P) (N : ℕ) :
+    IsAlgEnvSeqUntil O A Y alg env P N where
+  measurable_obs := h.measurable_obs
   measurable_action := h.measurable_action
   measurable_feedback := h.measurable_feedback
+  hasCondDistrib_obs n _ := h.hasCondDistrib_obs n
   hasCondDistrib_action n _ := h.hasCondDistrib_action n
   hasCondDistrib_feedback n _ := h.hasCondDistrib_feedback n
 
+/-- `IsAlgEnvSeq` only depends on the almost everywhere equivalence classes of the processes. -/
+lemma IsAlgEnvSeq.congr {O' : ℕ → Ω → 𝓞} {A' : ℕ → Ω → 𝓐} {Y' : ℕ → Ω → 𝓨}
+    (h : IsAlgEnvSeq O A Y alg env P)
+    (hO' : ∀ n, Measurable (O' n)) (hA' : ∀ n, Measurable (A' n)) (hY' : ∀ n, Measurable (Y' n))
+    (hO : ∀ n, O' n =ᵐ[P] O n) (hA : ∀ n, A' n =ᵐ[P] A n) (hY : ∀ n, Y' n =ᵐ[P] Y n) :
+    IsAlgEnvSeq O' A' Y' alg env P where
+  measurable_obs := hO'
+  measurable_action := hA'
+  measurable_feedback := hY'
+  hasCondDistrib_obs n :=
+    HasCondDistrib.congr (h.hasCondDistrib_obs n) (history_congr hO hA hY n) (hO n)
+  hasCondDistrib_action n := by
+    refine HasCondDistrib.congr (h.hasCondDistrib_action n) ?_ (hA n)
+    filter_upwards [history_congr hO hA hY n, hO n] with ω h1 h2
+    rw [h1, h2]
+  hasCondDistrib_feedback n := by
+    refine HasCondDistrib.congr (h.hasCondDistrib_feedback n) ?_ (hY n)
+    filter_upwards [history_congr hO hA hY n, hO n, hA n] with ω h1 h2 h3
+    rw [h1, h2, h3]
+
 lemma isAlgEnvSeq_iff_forall_isAlgEnvSeqUntil :
-    IsAlgEnvSeq A Y alg env P ↔ ∀ N, IsAlgEnvSeqUntil A Y alg env P N where
+    IsAlgEnvSeq O A Y alg env P ↔ ∀ N, IsAlgEnvSeqUntil O A Y alg env P N where
   mp h N := h.isAlgEnvSeqUntil N
   mpr h := {
+    measurable_obs := (h 0).measurable_obs
     measurable_action := (h 0).measurable_action
     measurable_feedback := (h 0).measurable_feedback
+    hasCondDistrib_obs n := (h (n + 1)).hasCondDistrib_obs n n.lt_succ_self
     hasCondDistrib_action n := (h (n + 1)).hasCondDistrib_action n n.lt_succ_self
     hasCondDistrib_feedback n := (h (n + 1)).hasCondDistrib_feedback n n.lt_succ_self }
 
-lemma IsAlgEnvSeq.measurable_step (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    Measurable (step A Y n) := by
+lemma IsAlgEnvSeq.measurable_step (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    Measurable (step O A Y n) := by
+  have hO := h.measurable_obs
   have hA := h.measurable_action
   have hY := h.measurable_feedback
   fun_prop
 
-lemma IsAlgEnvSeq.measurable_history (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    Measurable (history A Y n) := by
+lemma IsAlgEnvSeq.measurable_history (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    Measurable (history O A Y n) := by
+  have hO := h.measurable_obs
   have hA := h.measurable_action
   have hY := h.measurable_feedback
   fun_prop
 
-lemma IsAlgEnvSeq.measurable_trajectory (h : IsAlgEnvSeq A Y alg env P) :
-    Measurable (trajectory A Y) :=
-  Learning.measurable_trajectory h.measurable_action h.measurable_feedback
+lemma IsAlgEnvSeq.measurable_trajectory (h : IsAlgEnvSeq O A Y alg env P) :
+    Measurable (trajectory O A Y) :=
+  Learning.measurable_trajectory h.measurable_obs h.measurable_action h.measurable_feedback
 
-lemma IsAlgEnvSeqUntil.measurable_step (h : IsAlgEnvSeqUntil A Y alg env P N) (n : ℕ) :
-    Measurable (step A Y n) := by
+lemma IsAlgEnvSeqUntil.measurable_step (h : IsAlgEnvSeqUntil O A Y alg env P N) (n : ℕ) :
+    Measurable (step O A Y n) := by
+  have hO := h.measurable_obs
   have hA := h.measurable_action
   have hY := h.measurable_feedback
   fun_prop
 
-lemma IsAlgEnvSeqUntil.measurable_history (h : IsAlgEnvSeqUntil A Y alg env P N) (n : ℕ) :
-    Measurable (history A Y n) := by
+lemma IsAlgEnvSeqUntil.measurable_history (h : IsAlgEnvSeqUntil O A Y alg env P N) (n : ℕ) :
+    Measurable (history O A Y n) := by
+  have hO := h.measurable_obs
   have hA := h.measurable_action
   have hY := h.measurable_feedback
   fun_prop
 
-lemma IsAlgEnvSeq.hasCondDistrib_step (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    HasCondDistrib (step A Y n) (history A Y n) (stepKernel alg env n) P :=
-  HasCondDistrib.prod (h.hasCondDistrib_action n) (h.hasCondDistrib_feedback n)
+lemma IsAlgEnvSeq.hasCondDistrib_step (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    HasCondDistrib (step O A Y n) (history O A Y n) (stepKernel alg env n) P :=
+  (h.hasCondDistrib_obs n).prod
+    ((h.hasCondDistrib_action n).prod (h.hasCondDistrib_feedback n))
 
-lemma IsAlgEnvSeqUntil.hasCondDistrib_step (h : IsAlgEnvSeqUntil A Y alg env P N)
+lemma IsAlgEnvSeqUntil.hasCondDistrib_step (h : IsAlgEnvSeqUntil O A Y alg env P N)
     (n : ℕ) (hn : n < N) :
-    HasCondDistrib (step A Y n) (history A Y n) (stepKernel alg env n) P :=
-  HasCondDistrib.prod (h.hasCondDistrib_action n hn) (h.hasCondDistrib_feedback n hn)
+    HasCondDistrib (step O A Y n) (history O A Y n) (stepKernel alg env n) P :=
+  (h.hasCondDistrib_obs n hn).prod
+    ((h.hasCondDistrib_action n hn).prod (h.hasCondDistrib_feedback n hn))
 
 section Zero
 
 /-! ### Laws at time `0`
 
-At time `0` the history is the unique element of `Fin 0 → 𝓐 × 𝓨`: conditioning on it is the same
+At time `0` the history is the unique element of `Hist 𝓞 𝓐 𝓨 0`: conditioning on it is the same
 as not conditioning. Note that those results need `P` to be a probability measure. -/
 
 variable [IsProbabilityMeasure P]
 
 omit [IsFiniteMeasure P] in
 /-- The history before time `0` is a constant. -/
-lemma hasLaw_history_zero (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) :
-    HasLaw (history A Y 0) (Measure.dirac default) P where
+lemma hasLaw_history_zero (O : ℕ → Ω → 𝓞) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) :
+    HasLaw (history O A Y 0) (Measure.dirac default) P where
   aemeasurable := by rw [history_zero]; exact aemeasurable_const
   map_eq := by rw [history_zero, Measure.map_const, measure_univ, one_smul]
 
-lemma IsAlgEnvSeqUntil.hasLaw_action_zero (h : IsAlgEnvSeqUntil A Y alg env P N) (hN : 0 < N) :
-    HasLaw (A 0) alg.p0 P := by
-  have h0 := h.hasCondDistrib_action 0 hN
+lemma IsAlgEnvSeqUntil.hasLaw_obs_zero (h : IsAlgEnvSeqUntil O A Y alg env P N) (hN : 0 < N) :
+    HasLaw (O 0) env.obs0 P := by
+  have h0 := h.hasCondDistrib_obs 0 hN
   rw [history_zero] at h0
   exact h0.hasLaw_of_const'
 
-lemma IsAlgEnvSeq.hasLaw_action_zero (h : IsAlgEnvSeq A Y alg env P) :
-    HasLaw (A 0) alg.p0 P :=
-  (h.isAlgEnvSeqUntil 1).hasLaw_action_zero zero_lt_one
+lemma IsAlgEnvSeq.hasLaw_obs_zero (h : IsAlgEnvSeq O A Y alg env P) :
+    HasLaw (O 0) env.obs0 P :=
+  (h.isAlgEnvSeqUntil 1).hasLaw_obs_zero zero_lt_one
+
+omit [IsProbabilityMeasure P] in
+lemma IsAlgEnvSeqUntil.hasCondDistrib_action_zero (h : IsAlgEnvSeqUntil O A Y alg env P N)
+    (hN : 0 < N) :
+    HasCondDistrib (A 0) (O 0) alg.p0 P :=
+  hasCondDistrib_prodMk_left_unique_iff.mp (h.hasCondDistrib_action 0 hN)
+
+omit [IsProbabilityMeasure P] in
+lemma IsAlgEnvSeq.hasCondDistrib_action_zero (h : IsAlgEnvSeq O A Y alg env P) :
+    HasCondDistrib (A 0) (O 0) alg.p0 P :=
+  (h.isAlgEnvSeqUntil 1).hasCondDistrib_action_zero zero_lt_one
 
 omit [IsProbabilityMeasure P] in
-lemma IsAlgEnvSeqUntil.hasCondDistrib_feedback_zero (h : IsAlgEnvSeqUntil A Y alg env P N)
+lemma IsAlgEnvSeqUntil.hasCondDistrib_feedback_zero (h : IsAlgEnvSeqUntil O A Y alg env P N)
     (hN : 0 < N) :
-    HasCondDistrib (Y 0) (A 0) env.ν0 P :=
-  hasCondDistrib_prodMk_left_unique_iff.mp (h.hasCondDistrib_feedback 0 hN)
+    HasCondDistrib (Y 0) (fun ω ↦ (O 0 ω, A 0 ω)) env.ν0 P := by
+  have h0 := h.hasCondDistrib_feedback 0 hN
+  rw [history_zero] at h0
+  exact h0.of_measurableEmbedding_comp_right
+    ((measurableEmbedding_prodMk_left (default : Hist 𝓞 𝓐 𝓨 0)).prodMap .id)
 
 omit [IsProbabilityMeasure P] in
-lemma IsAlgEnvSeq.hasCondDistrib_feedback_zero (h : IsAlgEnvSeq A Y alg env P) :
-    HasCondDistrib (Y 0) (A 0) env.ν0 P :=
+lemma IsAlgEnvSeq.hasCondDistrib_feedback_zero (h : IsAlgEnvSeq O A Y alg env P) :
+    HasCondDistrib (Y 0) (fun ω ↦ (O 0 ω, A 0 ω)) env.ν0 P :=
   (h.isAlgEnvSeqUntil 1).hasCondDistrib_feedback_zero zero_lt_one
 
-lemma IsAlgEnvSeqUntil.hasLaw_step_zero (h : IsAlgEnvSeqUntil A Y alg env P N) (hN : 0 < N) :
-    HasLaw (step A Y 0) (alg.p0 ⊗ₘ env.ν0) P := by
+lemma IsAlgEnvSeqUntil.hasLaw_step_zero (h : IsAlgEnvSeqUntil O A Y alg env P N) (hN : 0 < N) :
+    HasLaw (step O A Y 0) (env.obs0 ⊗ₘ (alg.p0 ⊗ₖ env.ν0)) P := by
   have h0 := h.hasCondDistrib_step 0 hN
   rw [history_zero] at h0
   rw [← stepKernel_zero alg env default]
   exact h0.hasLaw_of_const'
 
-lemma IsAlgEnvSeq.hasLaw_step_zero (h : IsAlgEnvSeq A Y alg env P) :
-    HasLaw (step A Y 0) (alg.p0 ⊗ₘ env.ν0) P :=
+lemma IsAlgEnvSeq.hasLaw_step_zero (h : IsAlgEnvSeq O A Y alg env P) :
+    HasLaw (step O A Y 0) (env.obs0 ⊗ₘ (alg.p0 ⊗ₖ env.ν0)) P :=
   (h.isAlgEnvSeqUntil 1).hasLaw_step_zero zero_lt_one
 
 end Zero
 
-lemma IsAlgEnvSeq.hasLaw_action_comp (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    HasLaw (A n) (alg.policy n ∘ₘ (P.map (history A Y n))) P :=
+lemma IsAlgEnvSeq.hasLaw_obs_comp (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    HasLaw (O n) (env.obs n ∘ₘ (P.map (history O A Y n))) P :=
+  HasCondDistrib.hasLaw_comp (h.hasCondDistrib_obs n)
+
+lemma IsAlgEnvSeq.hasLaw_action_comp (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    HasLaw (A n) (alg.policy n ∘ₘ (P.map fun ω ↦ (history O A Y n ω, O n ω))) P :=
   HasCondDistrib.hasLaw_comp (h.hasCondDistrib_action n)
 
-lemma IsAlgEnvSeq.hasLaw_feedback_comp (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    HasLaw (Y n) ((env.feedback n) ∘ₘ (P.map fun ω ↦ (history A Y n ω, A n ω))) P :=
+lemma IsAlgEnvSeq.hasLaw_feedback_comp (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    HasLaw (Y n)
+      ((env.feedback n) ∘ₘ (P.map fun ω ↦ ((history O A Y n ω, O n ω), A n ω))) P :=
   HasCondDistrib.hasLaw_comp (h.hasCondDistrib_feedback n)
 
-lemma IsAlgEnvSeq.hasLaw_step_comp (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    HasLaw (step A Y n) (stepKernel alg env n ∘ₘ (P.map (history A Y n))) P :=
+lemma IsAlgEnvSeq.hasLaw_step_comp (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    HasLaw (step O A Y n) (stepKernel alg env n ∘ₘ (P.map (history O A Y n))) P :=
   HasCondDistrib.hasLaw_comp (h.hasCondDistrib_step n)
 
-/-- Conditionally on the event `A 0 = b`, the first feedback has law `env.ν0 b`. -/
-lemma IsAlgEnvSeq.hasLaw_feedback_zero_cond [MeasurableSingletonClass 𝓐]
-    (h : IsAlgEnvSeq A Y alg env P) {b : 𝓐} (hP : P (A 0 ⁻¹' {b}) ≠ 0) :
-    HasLaw (Y 0) (env.ν0 b) P[|A 0 ⁻¹' {b}] :=
+/-- Conditionally on the event `(O 0, A 0) = p`, the first feedback has law `env.ν0 p`. -/
+lemma IsAlgEnvSeq.hasLaw_feedback_zero_cond [MeasurableSingletonClass 𝓞]
+    [MeasurableSingletonClass 𝓐] (h : IsAlgEnvSeq O A Y alg env P) {p : 𝓞 × 𝓐}
+    (hP : P ((fun ω ↦ (O 0 ω, A 0 ω)) ⁻¹' {p}) ≠ 0) :
+    HasLaw (Y 0) (env.ν0 p) P[|(fun ω ↦ (O 0 ω, A 0 ω)) ⁻¹' {p}] :=
   h.hasCondDistrib_feedback_zero.hasLaw_cond (h.measurable_feedback 0)
-    (measurableSet_singleton b) (fun a ha ↦ by rw [Set.mem_singleton_iff.1 ha]) hP
+    (measurableSet_singleton p) (fun a ha ↦ by rw [Set.mem_singleton_iff.1 ha]) hP
 
 section Filtration
 
 namespace IsAlgEnvSeq
 
 /-- Filtration generated by the history up to time `n` (included): `h.filtration n` is the
-σ-algebra generated by `history A Y (n + 1)`, that is by the action-feedback pairs at times
-`0, ..., n`. -/
-def filtration (h : IsAlgEnvSeq A Y alg env P) :
+σ-algebra generated by `history O A Y (n + 1)`, that is by the rounds at times `0, ..., n`. -/
+def filtration (h : IsAlgEnvSeq O A Y alg env P) :
     Filtration ℕ mΩ where
-  seq n := MeasurableSpace.comap (history A Y (n + 1)) inferInstance
+  seq n := MeasurableSpace.comap (history O A Y (n + 1)) inferInstance
   mono' i j hij := by
     simp only
     rw [← measurable_iff_comap_le, history_eq_comp_history (Nat.succ_le_succ hij)]
     exact measurable_comp_comap _ (by fun_prop)
   le' i := by
     rw [← measurable_iff_comap_le]
-    exact Learning.measurable_history h.measurable_action h.measurable_feedback _
+    exact Learning.measurable_history h.measurable_obs h.measurable_action h.measurable_feedback _
 
-lemma filtration_eq_comap (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    h.filtration n = MeasurableSpace.comap (history A Y (n + 1)) inferInstance := rfl
+lemma filtration_eq_comap (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    h.filtration n = MeasurableSpace.comap (history O A Y (n + 1)) inferInstance := rfl
 
-lemma measurable_history_succ_filtration (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    Measurable[h.filtration n] (history A Y (n + 1)) :=
+lemma measurable_history_succ_filtration (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    Measurable[h.filtration n] (history O A Y (n + 1)) :=
   measurable_iff_comap_le.mpr le_rfl
 
-lemma adapted_history (h : IsAlgEnvSeq A Y alg env P) :
-    Adapted h.filtration (history A Y) := by
+lemma adapted_history (h : IsAlgEnvSeq O A Y alg env P) :
+    Adapted h.filtration (history O A Y) := by
   intro n
   rw [history_eq_comp_history n.le_succ]
   exact measurable_comp_comap _ (by fun_prop)
 
-lemma adapted_step (h : IsAlgEnvSeq A Y alg env P) :
-    Adapted h.filtration (step A Y) := by
+lemma adapted_step (h : IsAlgEnvSeq O A Y alg env P) :
+    Adapted h.filtration (step O A Y) := by
   intro n
   rw [← eval_comp_history]
   exact measurable_comp_comap _ (by fun_prop)
 
-lemma adapted_action (h : IsAlgEnvSeq A Y alg env P) :
+lemma adapted_obs (h : IsAlgEnvSeq O A Y alg env P) :
+    Adapted h.filtration O := by
+  intro n
+  rw [← obs_eval_comp_history (O := O) (A := A) (Y := Y) n]
+  exact measurable_comp_comap _ (by fun_prop)
+
+lemma adapted_action (h : IsAlgEnvSeq O A Y alg env P) :
     Adapted h.filtration A := by
   intro n
-  rw [← fst_eval_comp_history (A := A) (Y := Y) n]
+  rw [← action_eval_comp_history (O := O) (A := A) (Y := Y) n]
   exact measurable_comp_comap _ (by fun_prop)
 
-lemma adapted_feedback (h : IsAlgEnvSeq A Y alg env P) :
+lemma adapted_feedback (h : IsAlgEnvSeq O A Y alg env P) :
     Adapted h.filtration Y := by
   intro n
-  rw [← snd_eval_comp_history (A := A) (Y := Y) n]
+  rw [← feedback_eval_comp_history (O := O) (A := A) (Y := Y) n]
   exact measurable_comp_comap _ (by fun_prop)
 
-/-- Filtration generated by the history before time `n` together with the action at time `n`. -/
-def filtrationAction (h : IsAlgEnvSeq A Y alg env P) :
+/-- Filtration generated by the history before time `n` together with the observation at
+time `n`. -/
+def filtrationObs (h : IsAlgEnvSeq O A Y alg env P) :
     Filtration ℕ mΩ where
-  seq n := MeasurableSpace.comap (fun ω ↦ (history A Y n ω, A n ω)) inferInstance
+  seq n := MeasurableSpace.comap (fun ω ↦ (history O A Y n ω, O n ω)) inferInstance
   mono' n m hnm := by
     simp only
     rw [← measurable_iff_comap_le]
     rcases eq_or_lt_of_le hnm with rfl | hlt
     · exact measurable_iff_comap_le.mpr le_rfl
-    have : (fun ω ↦ (history A Y n ω, A n ω)) =
-        (fun p : (Fin m → 𝓐 × 𝓨) × 𝓐 ↦
-          (fun i : Fin n ↦ p.1 (Fin.castLE hnm i), (p.1 ⟨n, hlt⟩).1)) ∘
-        (fun ω ↦ (history A Y m ω, A m ω)) := rfl
+    have : (fun ω ↦ (history O A Y n ω, O n ω)) =
+        (fun p : Hist 𝓞 𝓐 𝓨 m × 𝓞 ↦
+          (fun i : Fin n ↦ p.1 (Fin.castLE hnm i), (p.1 ⟨n, hlt⟩).obs)) ∘
+        (fun ω ↦ (history O A Y m ω, O m ω)) := rfl
     rw [this]
     exact measurable_comp_comap _ (by fun_prop)
   le' n := by
     rw [← measurable_iff_comap_le]
-    exact (Learning.measurable_history h.measurable_action h.measurable_feedback n).prodMk
-      (h.measurable_action n)
+    exact (Learning.measurable_history h.measurable_obs h.measurable_action
+      h.measurable_feedback n).prodMk (h.measurable_obs n)
+
+lemma filtrationObs_eq_comap (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    h.filtrationObs n =
+      MeasurableSpace.comap (fun ω ↦ (history O A Y n ω, O n ω)) inferInstance := rfl
 
-lemma filtrationAction_eq_comap (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
+/-- Filtration generated by the history before time `n` together with the observation and the
+action at time `n`. -/
+def filtrationAction (h : IsAlgEnvSeq O A Y alg env P) :
+    Filtration ℕ mΩ where
+  seq n := MeasurableSpace.comap (fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) inferInstance
+  mono' n m hnm := by
+    simp only
+    rw [← measurable_iff_comap_le]
+    rcases eq_or_lt_of_le hnm with rfl | hlt
+    · exact measurable_iff_comap_le.mpr le_rfl
+    have : (fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) =
+        (fun p : (Hist 𝓞 𝓐 𝓨 m × 𝓞) × 𝓐 ↦
+          ((fun i : Fin n ↦ p.1.1 (Fin.castLE hnm i), (p.1.1 ⟨n, hlt⟩).obs),
+            (p.1.1 ⟨n, hlt⟩).action)) ∘
+        (fun ω ↦ ((history O A Y m ω, O m ω), A m ω)) := rfl
+    rw [this]
+    exact measurable_comp_comap _ (by fun_prop)
+  le' n := by
+    rw [← measurable_iff_comap_le]
+    exact ((Learning.measurable_history h.measurable_obs h.measurable_action
+      h.measurable_feedback n).prodMk (h.measurable_obs n)).prodMk (h.measurable_action n)
+
+lemma filtrationAction_eq_comap (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
     h.filtrationAction n =
-      MeasurableSpace.comap (fun ω ↦ (history A Y n ω, A n ω)) inferInstance := rfl
+      MeasurableSpace.comap (fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) inferInstance := rfl
 
-lemma filtrationAction_zero_eq_comap (h : IsAlgEnvSeq A Y alg env P) :
-    h.filtrationAction 0 = MeasurableSpace.comap (A 0) inferInstance := by
-  rw [filtrationAction_eq_comap, history_zero]
+lemma filtrationObs_zero_eq_comap (h : IsAlgEnvSeq O A Y alg env P) :
+    h.filtrationObs 0 = MeasurableSpace.comap (O 0) inferInstance := by
+  rw [filtrationObs_eq_comap, history_zero]
   refine le_antisymm ?_ ?_
   · rw [← measurable_iff_comap_le]
     exact measurable_const.prodMk (measurable_iff_comap_le.mpr le_rfl)
   · rw [← measurable_iff_comap_le]
     exact measurable_snd.comp (measurable_iff_comap_le.mpr le_rfl)
 
-lemma measurable_history_filtrationAction (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    Measurable[h.filtrationAction n] (history A Y n) :=
+lemma filtrationAction_zero_eq_comap (h : IsAlgEnvSeq O A Y alg env P) :
+    h.filtrationAction 0 =
+      MeasurableSpace.comap (fun ω ↦ (O 0 ω, A 0 ω)) inferInstance := by
+  rw [filtrationAction_eq_comap, history_zero]
+  refine le_antisymm ?_ ?_
+  · rw [← measurable_iff_comap_le]
+    exact ((measurable_const.prodMk
+      (measurable_fst.comp (measurable_iff_comap_le.mpr le_rfl)))).prodMk
+      (measurable_snd.comp (measurable_iff_comap_le.mpr le_rfl))
+  · rw [← measurable_iff_comap_le]
+    exact ((measurable_snd.comp measurable_fst).comp
+      (measurable_iff_comap_le.mpr le_rfl)).prodMk
+      (measurable_snd.comp (measurable_iff_comap_le.mpr le_rfl))
+
+lemma measurable_history_filtrationObs (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    Measurable[h.filtrationObs n] (history O A Y n) :=
   measurable_fst.comp (measurable_iff_comap_le.mpr le_rfl)
 
-lemma filtration_le_filtrationAction_succ (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
+@[fun_prop]
+lemma measurable_history_filtrationAction (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    Measurable[h.filtrationAction n] (history O A Y n) :=
+  (measurable_fst.comp measurable_fst).comp (measurable_iff_comap_le.mpr le_rfl)
+
+lemma filtrationObs_le_filtrationAction (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    h.filtrationObs n ≤ h.filtrationAction n := by
+  rw [filtrationObs_eq_comap, filtrationAction_eq_comap, ← measurable_iff_comap_le]
+  exact measurable_fst.comp (measurable_iff_comap_le.mpr le_rfl)
+
+lemma filtration_le_filtrationObs_succ (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    h.filtration n ≤ h.filtrationObs (n + 1) :=
+  measurable_iff_comap_le.mp (h.measurable_history_filtrationObs (n + 1))
+
+lemma filtration_le_filtrationAction_succ (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
     h.filtration n ≤ h.filtrationAction (n + 1) :=
   measurable_iff_comap_le.mp (h.measurable_history_filtrationAction (n + 1))
 
-lemma filtrationAction_le_filtration (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
+lemma filtrationAction_le_filtration (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
     h.filtrationAction n ≤ h.filtration n := by
   rw [filtrationAction_eq_comap, ← measurable_iff_comap_le]
-  exact (h.adapted_history n).prodMk (h.adapted_action n)
+  exact ((h.adapted_history n).prodMk (h.adapted_obs n)).prodMk (h.adapted_action n)
 
-lemma adapted_action_filtrationAction (h : IsAlgEnvSeq A Y alg env P) :
+lemma filtrationObs_le_filtration (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    h.filtrationObs n ≤ h.filtration n :=
+  (h.filtrationObs_le_filtrationAction n).trans (h.filtrationAction_le_filtration n)
+
+lemma adapted_obs_filtrationObs (h : IsAlgEnvSeq O A Y alg env P) :
+    Adapted h.filtrationObs O := fun _ ↦
+  measurable_snd.comp (measurable_iff_comap_le.mpr le_rfl)
+
+lemma adapted_obs_filtrationAction (h : IsAlgEnvSeq O A Y alg env P) :
+    Adapted h.filtrationAction O := fun _ ↦
+  (measurable_snd.comp measurable_fst).comp (measurable_iff_comap_le.mpr le_rfl)
+
+lemma adapted_action_filtrationAction (h : IsAlgEnvSeq O A Y alg env P) :
     Adapted h.filtrationAction A := fun _ ↦
   measurable_snd.comp (measurable_iff_comap_le.mpr le_rfl)
 
-lemma measurable_feedback_filtrationAction_of_lt (h : IsAlgEnvSeq A Y alg env P)
+lemma measurable_feedback_filtrationObs_of_lt (h : IsAlgEnvSeq O A Y alg env P)
+    {m n : ℕ} (hmn : m < n) :
+    Measurable[h.filtrationObs n] (Y m) := by
+  obtain ⟨j, rfl⟩ : ∃ j, n = j + 1 := ⟨n - 1, by lia⟩
+  exact (h.adapted_feedback.measurable_le (by lia)).mono
+    (filtration_le_filtrationObs_succ h j) le_rfl
+
+lemma measurable_feedback_filtrationAction_of_lt (h : IsAlgEnvSeq O A Y alg env P)
     {m n : ℕ} (hmn : m < n) :
     Measurable[h.filtrationAction n] (Y m) := by
   obtain ⟨j, rfl⟩ : ∃ j, n = j + 1 := ⟨n - 1, by lia⟩
@@ -468,4 +706,47 @@ end Filtration
 
 end IsAlgEnvSeq
 
+section NoObservation
+
+/-! ### Environments without observations
+
+An environment with `𝓞 = Unit` provides no information to the algorithm before it takes its action:
+the algorithm only sees the past rounds. Since `Unit` carries a unique probability measure, the
+observation kernels of such an environment are all equal to `Kernel.const _ (Measure.dirac ())`,
+and the observation process of an algorithm-environment sequence is `noObs`. -/
+
+/-- Every Markov kernel with codomain `Unit` is the constant kernel at `Measure.dirac ()`. -/
+lemma Kernel.eq_const_dirac_unit {α : Type*} {mα : MeasurableSpace α} (κ : Kernel α Unit)
+    [IsMarkovKernel κ] :
+    κ = Kernel.const α (Measure.dirac ()) := by
+  ext a s hs
+  rw [Kernel.const_apply]
+  rcases Set.eq_empty_or_nonempty s with rfl | ⟨u, hu⟩
+  · simp
+  · have hs_univ : s = Set.univ := Set.eq_univ_of_forall fun x ↦ by rwa [Subsingleton.elim x u]
+    simp [hs_univ]
+
+/-- A random variable with values in `Unit` admits any Markov kernel as conditional
+distribution. -/
+lemma hasCondDistrib_unit {α : Type*} {mα : MeasurableSpace α} {P : Measure Ω}
+    [IsProbabilityMeasure P] {X : Ω → α} (hX : AEMeasurable X P) (U : Ω → Unit)
+    (κ : Kernel α Unit) [IsMarkovKernel κ] :
+    HasCondDistrib U X κ P := by
+  have hU : U = fun _ ↦ () := funext fun _ ↦ rfl
+  subst hU
+  refine HasLaw.mk (hX.prodMk aemeasurable_const) ?_
+  rw [Kernel.eq_const_dirac_unit κ, Measure.compProd_const, Measure.prod_dirac,
+    AEMeasurable.map_map_of_aemeasurable (by fun_prop) hX]
+  rfl
+
+/-- The observation process of an algorithm-environment sequence without observations. -/
+def noObs (Ω : Type*) : ℕ → Ω → Unit := fun _ _ ↦ ()
+
+@[simp] lemma noObs_apply (n : ℕ) (ω : Ω) : noObs Ω n ω = () := rfl
+
+@[fun_prop]
+lemma measurable_noObs (n : ℕ) : Measurable (noObs Ω n) := measurable_const
+
+end NoObservation
+
 end Learning
diff --git a/LeanMachineLearning/SequentialLearning/AlgorithmDensity.lean b/LeanMachineLearning/SequentialLearning/AlgorithmDensity.lean
index de6d7c96..4ea64b4e 100644
--- a/LeanMachineLearning/SequentialLearning/AlgorithmDensity.lean
+++ b/LeanMachineLearning/SequentialLearning/AlgorithmDensity.lean
@@ -46,39 +46,41 @@ open scoped ENNReal
 
 namespace Learning
 
-variable {𝓐 𝓨 : Type*} [MeasurableSpace 𝓐] [MeasurableSpace 𝓨]
+variable {𝓞 𝓐 𝓨 : Type*} [MeasurableSpace 𝓞] [MeasurableSpace 𝓐] [MeasurableSpace 𝓨]
 
 namespace Algorithm
 
 /-- For every time and history, the distribution over actions according to `alg` is absolutely
 continuous with respect to the distribution over actions according to `alg₀`. -/
-structure AbsolutelyContinuous (alg alg₀ : Algorithm 𝓐 𝓨) : Prop where
+structure AbsolutelyContinuous (alg alg₀ : Algorithm 𝓞 𝓐 𝓨) : Prop where
   policy n h : alg.policy n h ≪ alg₀.policy n h
 
 @[inherit_doc AbsolutelyContinuous]
 scoped notation:50 alg " ≪ₐ " alg₀ => AbsolutelyContinuous alg alg₀
 
-lemma AbsolutelyContinuous.p0 {alg alg₀ : Algorithm 𝓐 𝓨} (h : alg ≪ₐ alg₀) : alg.p0 ≪ alg₀.p0 :=
-  h.policy 0 default
+lemma AbsolutelyContinuous.p0 {alg alg₀ : Algorithm 𝓞 𝓐 𝓨} (h : alg ≪ₐ alg₀) (o : 𝓞) :
+    alg.p0 o ≪ alg₀.p0 o :=
+  h.policy 0 (default, o)
 
 /-- If the algorithm `alg` is absolutely continuous with respect to the algorithm `alg₀` and they
 are both interacting with the same environment, then the law of the history before time `n` under
 `alg` is the law of the history before time `n` under `alg₀` with density `alg.density alg₀ n`. -/
 noncomputable
-def density [MeasurableSpace.CountablyGenerated 𝓐] (alg alg₀ : Algorithm 𝓐 𝓨) :
-    (n : ℕ) → (Fin n → 𝓐 × 𝓨) → ℝ≥0∞
+def density [MeasurableSpace.CountablyGenerated 𝓐] (alg alg₀ : Algorithm 𝓞 𝓐 𝓨) :
+    (n : ℕ) → Hist 𝓞 𝓐 𝓨 n → ℝ≥0∞
   | 0, _ => 1
   | n + 1, h =>
-    let p := MeasurableEquiv.finSuccProd (𝓐 × 𝓨) n h
-    alg.density alg₀ n p.1 * (alg.policy n).rnDeriv (alg₀.policy n) p.1 p.2.1
+    let p := MeasurableEquiv.finSuccProd (Round 𝓞 𝓐 𝓨) n h
+    alg.density alg₀ n p.1 * (alg.policy n).rnDeriv (alg₀.policy n) (p.1, p.2.obs) p.2.action
 
 @[simp]
-lemma density_zero [MeasurableSpace.CountablyGenerated 𝓐] (alg alg₀ : Algorithm 𝓐 𝓨)
-    (h : Fin 0 → 𝓐 × 𝓨) :
+lemma density_zero [MeasurableSpace.CountablyGenerated 𝓐] (alg alg₀ : Algorithm 𝓞 𝓐 𝓨)
+    (h : Hist 𝓞 𝓐 𝓨 0) :
     alg.density alg₀ 0 h = 1 := rfl
 
 @[fun_prop]
-lemma measurable_density [MeasurableSpace.CountablyGenerated 𝓐] (alg alg₀ : Algorithm 𝓐 𝓨) (n : ℕ) :
+lemma measurable_density [MeasurableSpace.CountablyGenerated 𝓐]
+    (alg alg₀ : Algorithm 𝓞 𝓐 𝓨) (n : ℕ) :
     Measurable (alg.density alg₀ n) := by
   induction n with
   | zero => simp_rw [density]; fun_prop
@@ -91,21 +93,21 @@ open scoped Algorithm
 namespace IsAlgEnvSeq
 
 variable {Ω : Type*} [MeasurableSpace Ω]
-variable {alg : Algorithm 𝓐 𝓨} {env : Environment 𝓐 𝓨}
-variable {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
+variable {alg : Algorithm 𝓞 𝓐 𝓨} {env : Environment 𝓞 𝓐 𝓨}
+variable {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
 variable {P : Measure Ω} [IsProbabilityMeasure P]
 
 variable {Ω₀ : Type*} [MeasurableSpace Ω₀]
-variable {alg₀ : Algorithm 𝓐 𝓨}
-variable {A₀ : ℕ → Ω₀ → 𝓐} {Y₀ : ℕ → Ω₀ → 𝓨}
+variable {alg₀ : Algorithm 𝓞 𝓐 𝓨}
+variable {O₀ : ℕ → Ω₀ → 𝓞} {A₀ : ℕ → Ω₀ → 𝓐} {Y₀ : ℕ → Ω₀ → 𝓨}
 variable {P₀ : Measure Ω₀} [IsProbabilityMeasure P₀]
 
-lemma absolutelyContinuous_map_history (h : IsAlgEnvSeq A Y alg env P)
-    (h₀ : IsAlgEnvSeq A₀ Y₀ alg₀ env P₀) (hc : alg ≪ₐ alg₀) (n : ℕ) :
-    P.map (history A Y n) ≪ P₀.map (history A₀ Y₀ n) := by
+lemma absolutelyContinuous_map_history (h : IsAlgEnvSeq O A Y alg env P)
+    (h₀ : IsAlgEnvSeq O₀ A₀ Y₀ alg₀ env P₀) (hc : alg ≪ₐ alg₀) (n : ℕ) :
+    P.map (history O A Y n) ≪ P₀.map (history O₀ A₀ Y₀ n) := by
   induction n with
   | zero =>
-    rw [(hasLaw_history_zero A Y).map_eq, (hasLaw_history_zero A₀ Y₀).map_eq]
+    rw [(hasLaw_history_zero O A Y).map_eq, (hasLaw_history_zero O₀ A₀ Y₀).map_eq]
   | succ n ih =>
     simp_rw [history_succ]
     rw [← Measure.map_map (by fun_prop), ← Measure.map_map (by fun_prop)]
@@ -115,24 +117,40 @@ lemma absolutelyContinuous_map_history (h : IsAlgEnvSeq A Y alg env P)
     apply Measure.AbsolutelyContinuous.map _ (by fun_prop)
     rw [(h.hasCondDistrib_step n).map_eq, (h₀.hasCondDistrib_step n).map_eq]
     apply Measure.AbsolutelyContinuous.compProd ih
-    filter_upwards with h' using Measure.AbsolutelyContinuous.compProd_left_apply (hc.policy n h') _
+    filter_upwards with h'
+    rw [stepKernel_def, stepKernel_def, Kernel.compProd_apply_eq_compProd_sectR,
+      Kernel.compProd_apply_eq_compProd_sectR]
+    refine Measure.AbsolutelyContinuous.compProd_right ?_
+    filter_upwards with o
+    simp only [Kernel.sectR_apply]
+    exact Measure.AbsolutelyContinuous.compProd_left_apply (hc.policy n (h', o)) _
 
 variable [MeasurableSpace.CountablyGenerated 𝓐]
 
-lemma hasLaw_history_withDensity (h : IsAlgEnvSeq A Y alg env P)
-    (h₀ : IsAlgEnvSeq A₀ Y₀ alg₀ env P₀) (hc : alg ≪ₐ alg₀) (n : ℕ) : HasLaw (history A Y n)
-      ((P₀.map (history A₀ Y₀ n)).withDensity (alg.density alg₀ n)) P where
+lemma hasLaw_history_withDensity (h : IsAlgEnvSeq O A Y alg env P)
+    (h₀ : IsAlgEnvSeq O₀ A₀ Y₀ alg₀ env P₀) (hc : alg ≪ₐ alg₀) (n : ℕ) : HasLaw (history O A Y n)
+      ((P₀.map (history O₀ A₀ Y₀ n)).withDensity (alg.density alg₀ n)) P where
   aemeasurable := (h.measurable_history n).aemeasurable
   map_eq := by
     induction n with
     | zero =>
-      rw [(hasLaw_history_zero A Y).map_eq, (hasLaw_history_zero A₀ Y₀).map_eq,
+      rw [(hasLaw_history_zero O A Y).map_eq, (hasLaw_history_zero O₀ A₀ Y₀).map_eq,
         show alg.density alg₀ 0 = 1 from rfl, withDensity_one]
     | succ n ih =>
-      let ρ h' (ar : 𝓐 × 𝓨) := Kernel.rnDeriv (alg.policy n) (alg₀.policy n) h' ar.1
+      let ρ h' (r : Round 𝓞 𝓐 𝓨) :=
+        Kernel.rnDeriv (alg.policy n) (alg₀.policy n) (h', r.obs) r.action
       have hs : stepKernel alg env n = (stepKernel alg₀ env n).withDensity ρ := by
-        rw [stepKernel, ← Kernel.withDensity_rnDeriv_eq' (hc.policy n)]
-        exact Kernel.compProd_withDensity_left (Kernel.measurable_rnDeriv _ _)
+        have h_inner : alg.policy n ⊗ₖ env.feedback n
+            = (alg₀.policy n ⊗ₖ env.feedback n).withDensity
+              (fun p ar ↦ Kernel.rnDeriv (alg.policy n) (alg₀.policy n) p ar.1) := by
+          conv_lhs => rw [← Kernel.withDensity_rnDeriv_eq' (hc.policy n)]
+          exact Kernel.compProd_withDensity_left (Kernel.measurable_rnDeriv _ _)
+        have h_sf : IsSFiniteKernel ((alg₀.policy n ⊗ₖ env.feedback n).withDensity
+            (fun p ar ↦ Kernel.rnDeriv (alg.policy n) (alg₀.policy n) p ar.1)) := by
+          rw [← h_inner]
+          infer_instance
+        rw [stepKernel_def alg env n, h_inner, Kernel.compProd_withDensity_right (by fun_prop)]
+        rfl
       have : IsMarkovKernel ((stepKernel alg₀ env n).withDensity ρ) := by
         rw [← hs]
         infer_instance
diff --git a/LeanMachineLearning/SequentialLearning/AlgorithmDensityBayes.lean b/LeanMachineLearning/SequentialLearning/AlgorithmDensityBayes.lean
index ad4b00c4..e345f31f 100644
--- a/LeanMachineLearning/SequentialLearning/AlgorithmDensityBayes.lean
+++ b/LeanMachineLearning/SequentialLearning/AlgorithmDensityBayes.lean
@@ -43,23 +43,23 @@ namespace IsBayesAlgEnvSeq
 variable {𝓐 𝓨 : Type*} [MeasurableSpace 𝓐] [MeasurableSpace 𝓨]
 variable {𝓔 : Type*} [MeasurableSpace 𝓔]
 variable [StandardBorelSpace 𝓐] [Nonempty 𝓐] [StandardBorelSpace 𝓨] [Nonempty 𝓨]
-variable {Q : Measure 𝓔}
+variable {Q : Measure 𝓔} [IsProbabilityMeasure Q]
 variable {κ : Kernel (𝓔 × 𝓐) 𝓨} [IsMarkovKernel κ]
 
 variable {Ω : Type*} [MeasurableSpace Ω]
 variable {E : Ω → 𝓔} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
-variable {alg : Algorithm 𝓐 𝓨}
+variable {alg : Algorithm Unit 𝓐 𝓨}
 variable {P : Measure Ω} [IsProbabilityMeasure P]
 
 variable {Ω₀ : Type*} [MeasurableSpace Ω₀]
 variable {E₀ : Ω₀ → 𝓔} {A₀ : ℕ → Ω₀ → 𝓐} {Y₀ : ℕ → Ω₀ → 𝓨}
-variable {alg₀ : Algorithm 𝓐 𝓨}
+variable {alg₀ : Algorithm Unit 𝓐 𝓨}
 variable {P₀ : Measure Ω₀} [IsProbabilityMeasure P₀]
 
 lemma condDistrib_history_eq_condDistrib_hist_withDensity (h : IsBayesAlgEnvSeq Q κ alg E A Y P)
     (h₀ : IsBayesAlgEnvSeq Q κ alg₀ E₀ A₀ Y₀ P₀) (hc : alg ≪ₐ alg₀) (n : ℕ) :
-    condDistrib (history A Y n) E P =ᵐ[Q]
-      ((condDistrib (history A₀ Y₀ n) E₀ P₀).withDensity
+    condDistrib (history (noObs Ω) A Y n) E P =ᵐ[Q]
+      ((condDistrib (history (noObs Ω₀) A₀ Y₀ n) E₀ P₀).withDensity
         (fun _ ↦ alg.density alg₀ n)) := by
     filter_upwards [h.ae_IsAlgEnvSeq, h₀.ae_IsAlgEnvSeq, h.hasLaw_IT_hist n, h₀.hasLaw_IT_hist n]
       with _ hae hae₀ he he₀
@@ -68,10 +68,13 @@ lemma condDistrib_history_eq_condDistrib_hist_withDensity (h : IsBayesAlgEnvSeq
 
 lemma hasLaw_history_withDensity (h : IsBayesAlgEnvSeq Q κ alg E A Y P)
     (h₀ : IsBayesAlgEnvSeq Q κ alg₀ E₀ A₀ Y₀ P₀) (hc : alg ≪ₐ alg₀) (n : ℕ) :
-    HasLaw (history A Y n)
-      ((P₀.map (history A₀ Y₀ n)).withDensity (alg.density alg₀ n)) P where
-  aemeasurable := (measurable_history h.measurable_action h.measurable_feedback n).aemeasurable
+    HasLaw (history (noObs Ω) A Y n)
+      ((P₀.map (history (noObs Ω₀) A₀ Y₀ n)).withDensity (alg.density alg₀ n)) P where
+  aemeasurable := (measurable_history (fun _ ↦ measurable_const) h.measurable_action
+    h.measurable_feedback n).aemeasurable
   map_eq := by
+    have hO : ∀ n, Measurable (noObs Ω n) := fun _ ↦ measurable_const
+    have hO₀ : ∀ n, Measurable (noObs Ω₀ n) := fun _ ↦ measurable_const
     have hA := h.measurable_action
     have hY := h.measurable_feedback
     have hA₀ := h₀.measurable_action
@@ -84,14 +87,16 @@ lemma hasLaw_history_withDensity (h : IsBayesAlgEnvSeq Q κ alg E A Y P)
           ← h₀.hasLaw_env.map_eq, condDistrib_comp_map hE₀.aemeasurable (by fun_prop)]
 
 variable [StandardBorelSpace 𝓔] [Nonempty 𝓔]
-variable [IsProbabilityMeasure Q]
 
 lemma hasCondDistrib_env_history (h : IsBayesAlgEnvSeq Q κ alg E A Y P)
     (h₀ : IsBayesAlgEnvSeq Q κ alg₀ E₀ A₀ Y₀ P₀) (hc : alg ≪ₐ alg₀) (n : ℕ) :
-    HasCondDistrib E (history A Y n) (condDistrib E₀ (history A₀ Y₀ n) P₀) P where
-  aemeasurable := ((measurable_history h.measurable_action
+    HasCondDistrib E (history (noObs Ω) A Y n)
+      (condDistrib E₀ (history (noObs Ω₀) A₀ Y₀ n) P₀) P where
+  aemeasurable := ((measurable_history (fun _ ↦ measurable_const) h.measurable_action
     h.measurable_feedback n).prodMk h.measurable_param).aemeasurable
   map_eq := by
+    have hO : ∀ n, Measurable (noObs Ω n) := fun _ ↦ measurable_const
+    have hO₀ : ∀ n, Measurable (noObs Ω₀ n) := fun _ ↦ measurable_const
     have hA := h.measurable_action
     have hY := h.measurable_feedback
     have hA₀ := h₀.measurable_action
diff --git a/LeanMachineLearning/SequentialLearning/Algorithms/RandomSampling/Basic.lean b/LeanMachineLearning/SequentialLearning/Algorithms/RandomSampling/Basic.lean
index ebaa27d9..7ade0abe 100644
--- a/LeanMachineLearning/SequentialLearning/Algorithms/RandomSampling/Basic.lean
+++ b/LeanMachineLearning/SequentialLearning/Algorithms/RandomSampling/Basic.lean
@@ -34,7 +34,8 @@ open scoped Topology
 
 namespace Learning
 
-variable {𝓐 𝓨 Ω : Type*} {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨}
+variable {𝓞 𝓐 𝓨 Ω : Type*} {m𝓞 : MeasurableSpace 𝓞} {m𝓐 : MeasurableSpace 𝓐}
+  {m𝓨 : MeasurableSpace 𝓨}
   {mΩ : MeasurableSpace Ω} {μ : Measure 𝓐} [IsProbabilityMeasure μ]
   {P : Measure Ω} [IsProbabilityMeasure P]
 
@@ -42,21 +43,23 @@ open Set in
 /-- The _Random Sampling_ algorithm, which samples from a fixed probability
 measure at each iteration. -/
 @[simps]
-noncomputable def randomSampling (μ : Measure 𝓐) [IsProbabilityMeasure μ] : Algorithm 𝓐 𝓨 where
+noncomputable def randomSampling (μ : Measure 𝓐) [IsProbabilityMeasure μ] :
+    Algorithm 𝓞 𝓐 𝓨 where
   policy _ := Kernel.const _ μ
 
 namespace randomSampling
 
-variable {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {env : Environment 𝓐 𝓨}
+variable {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {env : Environment 𝓞 𝓐 𝓨}
 
 /-- Each action follows the distribution μ. -/
-lemma hasLaw_action (h : IsAlgEnvSeq A Y (randomSampling μ) env P) (n : ℕ) :
+lemma hasLaw_action (h : IsAlgEnvSeq O A Y (randomSampling μ) env P) (n : ℕ) :
     HasLaw (A n) μ P :=
   (h.hasCondDistrib_action n).hasLaw_of_const
 
 /-- Actions are mutually independent. -/
-lemma iIndep_action (h : IsAlgEnvSeq A Y (randomSampling μ) env P) :
+lemma iIndep_action (h : IsAlgEnvSeq O A Y (randomSampling μ) env P) :
     iIndepFun A P := by
+  have hO := h.measurable_obs
   have hA := h.measurable_action
   rw [iIndepFun_nat_iff_forall_indepFun (by fun_prop)]
   intro n
@@ -64,10 +67,11 @@ lemma iIndep_action (h : IsAlgEnvSeq A Y (randomSampling μ) env P) :
   simp only [randomSampling_policy, Measure.compProd_const] at map_eq
   have law_eq : P.map (A (n + 1)) = μ := (hasLaw_action h (n + 1)).map_eq
   rw [← law_eq, ← indepFun_iff_map_prod_eq_prod_map_map] at map_eq
-  · change A (n + 1) ⟂ᵢ[P] (fun (f : Fin (n + 1) → 𝓐 × 𝓨) (i : Iic n) ↦
-      (f ⟨i.1, Nat.lt_succ_of_le (mem_Iic.mp i.2)⟩).1) ∘ (history A Y (n + 1))
+  · change A (n + 1) ⟂ᵢ[P] (fun (p : Hist 𝓞 𝓐 𝓨 (n + 1) × 𝓞) (i : Iic n) ↦
+      (p.1 ⟨i.1, Nat.lt_succ_of_le (mem_Iic.mp i.2)⟩).action) ∘
+      (fun ω ↦ (history O A Y (n + 1) ω, O (n + 1) ω))
     refine map_eq.symm.comp measurable_id (by fun_prop)
-  · exact (h.measurable_history (n + 1)).aemeasurable
+  · exact ((h.measurable_history (n + 1)).prodMk (h.measurable_obs (n + 1))).aemeasurable
   · exact (h.measurable_action (n + 1)).aemeasurable
 
 end randomSampling
diff --git a/LeanMachineLearning/SequentialLearning/Algorithms/RandomSampling/Tendsto.lean b/LeanMachineLearning/SequentialLearning/Algorithms/RandomSampling/Tendsto.lean
index c23e291e..65778126 100644
--- a/LeanMachineLearning/SequentialLearning/Algorithms/RandomSampling/Tendsto.lean
+++ b/LeanMachineLearning/SequentialLearning/Algorithms/RandomSampling/Tendsto.lean
@@ -42,7 +42,8 @@ namespace Learning.randomSampling
 
 variable {𝓐 𝓨 Ω : Type*} {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨}
   {mΩ : MeasurableSpace Ω} {μ : Measure 𝓐} [IsProbabilityMeasure μ] {P : Measure Ω}
-  [IsProbabilityMeasure P] {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {env : Environment 𝓐 𝓨}
+  [IsProbabilityMeasure P] {O : ℕ → Ω → Unit} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
+  {env : Environment Unit 𝓐 𝓨}
   {f : 𝓐 → 𝓨} {hf : Measurable f}
 
 section rewards
@@ -50,7 +51,7 @@ section rewards
 variable [StandardBorelSpace 𝓨] [Nonempty 𝓨]
 
 /-- Each reward follows the distribution μ.map f. -/
-lemma hasLaw_feeback (h : IsAlgEnvSeq A Y (randomSampling μ) (evalEnv f hf) P) (n : ℕ) :
+lemma hasLaw_feeback (h : IsAlgEnvSeq O A Y (randomSampling μ) (evalEnv f hf) P) (n : ℕ) :
     HasLaw (Y n) (μ.map f) P := by
   refine HasLaw.congr ?_ (feedback_evalEnv_ae_eq_eval_action h n)
   have hA := h.measurable_action n
@@ -58,7 +59,7 @@ lemma hasLaw_feeback (h : IsAlgEnvSeq A Y (randomSampling μ) (evalEnv f hf) P)
   rw [← Measure.map_map hf hA, (hasLaw_action h n).map_eq]
 
 /-- Rewards are mutually independent. -/
-lemma iIndep_feedback (h : IsAlgEnvSeq A Y (randomSampling μ) (evalEnv f hf) P) :
+lemma iIndep_feedback (h : IsAlgEnvSeq O A Y (randomSampling μ) (evalEnv f hf) P) :
     iIndepFun Y P :=
   have (n : ℕ) : f ∘ A n =ᵐ[P] Y n :=
     (feedback_evalEnv_ae_eq_eval_action h n).symm
@@ -70,10 +71,10 @@ variable [PseudoMetricSpace 𝓐] [SecondCountableTopology 𝓐] [OpensMeasurabl
   [μ.IsOpenPosMeasure]
 
 /-- The minimum distance from sampled actions to any point tends to zero. -/
-theorem action_tendsto_any (h : IsAlgEnvSeq A Y (randomSampling μ) (evalEnv f hf) P) (a : 𝓐)
+theorem action_tendsto_any (h : IsAlgEnvSeq O A Y (randomSampling μ) (evalEnv f hf) P) (a : 𝓐)
     {ε : ℝ} (hε : 0 < ε) :
     Tendsto (fun i => P {x | ε ≤ (fun (j : Iic i) ↦ dist (A j.1 x) a).min}) atTop (𝓝 0) := by
-  set randomSampling_alg := randomSampling (𝓨 := 𝓨) μ
+  set randomSampling_alg := randomSampling (𝓞 := Unit) (𝓨 := 𝓨) μ
   refine tendsto_zero_of_le (g := fun n ↦ P (⋂ i ∈ Iic n, {x | ε ≤ dist (A i x) a})) ?_ ?_
   · have inter_prod (n : ℕ) : P (⋂ j ∈ Iic n, {x | ε ≤ dist (A j x) a}) =
         ∏ j ∈ Iic n, P {x | ε ≤ dist (A j x) a} := by
@@ -117,7 +118,7 @@ variable [PseudoMetricSpace 𝓨] [BorelSpace 𝓨] (hfc : Continuous f)
 
 /-- The minimum distance from image of actions to any function value tends to zero. -/
 lemma image_action_tendsto_any
-    (h : IsAlgEnvSeq A Y (randomSampling μ) (evalEnv f hfc.measurable) P)
+    (h : IsAlgEnvSeq O A Y (randomSampling μ) (evalEnv f hfc.measurable) P)
     (a : 𝓐) {ε : ℝ} (hε : 0 < ε) :
     Tendsto (fun i => P {x | ε ≤ (fun (j : Iic i) ↦
       dist (f (A j.1 x)) (f a)).min}) atTop (𝓝 0) := by
@@ -139,7 +140,7 @@ lemma image_action_tendsto_any
 variable [StandardBorelSpace 𝓨] [Nonempty 𝓨]
 
 /-- The minimum distance from rewards to any function value tends to zero. -/
-lemma feedback_tendsto_any (h : IsAlgEnvSeq A Y (randomSampling μ) (evalEnv f hfc.measurable) P)
+lemma feedback_tendsto_any (h : IsAlgEnvSeq O A Y (randomSampling μ) (evalEnv f hfc.measurable) P)
     (a : 𝓐) {ε : ℝ} (hε : 0 < ε) :
     Tendsto (fun i => P {x | ε ≤ (fun (j : Iic i) ↦ dist (Y j.1 x) (f a)).min}) atTop (𝓝 0) := by
   convert image_action_tendsto_any hfc h a hε using 2 with n
@@ -154,7 +155,7 @@ lemma feedback_tendsto_any (h : IsAlgEnvSeq A Y (randomSampling μ) (evalEnv f h
 variable {R : ℕ → Ω → ℝ} {f : 𝓐 → ℝ} (hfc : Continuous f) {a : 𝓐}
 
 /-- The minimum image action converges to the function's global minimum. -/
-lemma tendsto_min₀ (h : IsAlgEnvSeq A R (randomSampling μ) (evalEnv f hfc.measurable) P)
+lemma tendsto_min₀ (h : IsAlgEnvSeq O A R (randomSampling μ) (evalEnv f hfc.measurable) P)
     (hf_min : ∀ x, f a ≤ f x) :
     TendstoInMeasure P (fun n ω ↦ (fun (i : Iic n) ↦ f (A i.1 ω)).min) atTop (fun _ ↦ f a) := by
   rw [tendstoInMeasure_iff_dist]
@@ -175,7 +176,7 @@ lemma tendsto_min₀ (h : IsAlgEnvSeq A R (randomSampling μ) (evalEnv f hfc.mea
   grind
 
 /-- The minimum reward converges to the function's global minimum. -/
-lemma tendsto_min (h : IsAlgEnvSeq A R (randomSampling μ) (evalEnv f hfc.measurable) P)
+lemma tendsto_min (h : IsAlgEnvSeq O A R (randomSampling μ) (evalEnv f hfc.measurable) P)
     (hf_min : ∀ x, f a ≤ f x) :
     TendstoInMeasure P (fun n ω ↦ (fun (i : Iic n) ↦ R i.1 ω).min) atTop (fun _ ↦ f a) := by
   refine TendstoInMeasure.congr_left (fun n ↦ ?_) <| tendsto_min₀ hfc h hf_min
@@ -183,7 +184,7 @@ lemma tendsto_min (h : IsAlgEnvSeq A R (randomSampling μ) (evalEnv f hfc.measur
   rw [← hω]
 
 /-- The maximum image action converges to the function's global maximum. -/
-lemma tendsto_max₀ (h : IsAlgEnvSeq A R (randomSampling μ) (evalEnv f hfc.measurable) P)
+lemma tendsto_max₀ (h : IsAlgEnvSeq O A R (randomSampling μ) (evalEnv f hfc.measurable) P)
     (hf_max : ∀ x, f x ≤ f a) :
     TendstoInMeasure P (fun n ω ↦ (fun (i : Iic n) ↦ f (A i.1 ω)).max) atTop (fun _ ↦ f a) := by
   rw [tendstoInMeasure_iff_dist]
@@ -205,7 +206,7 @@ lemma tendsto_max₀ (h : IsAlgEnvSeq A R (randomSampling μ) (evalEnv f hfc.mea
   grind
 
 /-- The maximum reward converges to the function's global maximum. -/
-lemma tendsto_max (h : IsAlgEnvSeq A R (randomSampling μ) (evalEnv f hfc.measurable) P)
+lemma tendsto_max (h : IsAlgEnvSeq O A R (randomSampling μ) (evalEnv f hfc.measurable) P)
     (hf_max : ∀ x, f x ≤ f a) :
     TendstoInMeasure P (fun n ω ↦ (fun (i : Iic n) ↦ R i.1 ω).max) atTop (fun _ ↦ f a) := by
   refine TendstoInMeasure.congr_left (fun n ↦ ?_) <| tendsto_max₀ hfc h hf_max
diff --git a/LeanMachineLearning/SequentialLearning/Algorithms/RoundRobin.lean b/LeanMachineLearning/SequentialLearning/Algorithms/RoundRobin.lean
index c3ddd464..6f021c9e 100644
--- a/LeanMachineLearning/SequentialLearning/Algorithms/RoundRobin.lean
+++ b/LeanMachineLearning/SequentialLearning/Algorithms/RoundRobin.lean
@@ -70,7 +70,7 @@ end Aux
 
 namespace Learning
 
-variable {𝓨 : Type*} {m𝓨 : MeasurableSpace 𝓨} {K : ℕ}
+variable {𝓞 𝓨 : Type*} {m𝓞 : MeasurableSpace 𝓞} {m𝓨 : MeasurableSpace 𝓨} {K : ℕ}
 
 section AlgorithmDefinition
 
@@ -80,7 +80,7 @@ def RoundRobin.nextAction (hK : 0 < K) (n : ℕ) : Fin K := ⟨n % K, Nat.mod_lt
 
 /-- The Round-Robin algorithm: deterministic algorithm that chooses action `n % K` at time `n`. -/
 noncomputable
-def roundRobinAlgorithm (hK : 0 < K) : Algorithm (Fin K) 𝓨 :=
+def roundRobinAlgorithm (hK : 0 < K) : Algorithm 𝓞 (Fin K) 𝓨 :=
   detAlgorithm (fun n _ ↦ RoundRobin.nextAction hK n) (by fun_prop)
 
 end AlgorithmDefinition
@@ -90,17 +90,17 @@ namespace RoundRobin
 variable {hK : 0 < K} {ν : Kernel (Fin K) 𝓨} [IsMarkovKernel ν]
   {Ω : Type*} {mΩ : MeasurableSpace Ω}
   {P : Measure Ω} [IsProbabilityMeasure P]
-  {A : ℕ → Ω → Fin K} {Y : ℕ → Ω → 𝓨}
+  {O : ℕ → Ω → Unit} {A : ℕ → Ω → Fin K} {Y : ℕ → Ω → 𝓨}
 
 /-- The action chosen at time `n` is the action `n % K`. -/
 lemma action_ae_eq (n : ℕ)
-    (h : IsAlgEnvSeqUntil A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P (n + 1)) :
+    (h : IsAlgEnvSeqUntil O A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P (n + 1)) :
     A n =ᵐ[P] fun _ ↦ ⟨n % K, Nat.mod_lt _ hK⟩ := by
   have : Nonempty (Fin K) := Fin.pos_iff_nonempty.mp hK
   exact h.action_detAlgorithm_ae_eq n.lt_succ_self
 
 lemma action_zero
-    (h : IsAlgEnvSeqUntil A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P 1) :
+    (h : IsAlgEnvSeqUntil O A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P 1) :
     A 0 =ᵐ[P] fun _ ↦ ⟨0, hK⟩ := by
   filter_upwards [action_ae_eq 0 h] with ω hω
   rw [hω]
@@ -108,7 +108,7 @@ lemma action_zero
 
 /-- At time `K * m`, the number of times each action is chosen is equal to `m`. -/
 lemma pullCount_mul (m : ℕ)
-    (h : IsAlgEnvSeqUntil A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P (K * m))
+    (h : IsAlgEnvSeqUntil O A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P (K * m))
     (a : Fin K) :
     pullCount A a (K * m) =ᵐ[P] fun _ ↦ m := by
   rw [Filter.EventuallyEq]
@@ -124,14 +124,14 @@ lemma pullCount_mul (m : ℕ)
   _ = m := sum_mod_range_mul hK m a
 
 lemma pullCount_eq_one
-    (h : IsAlgEnvSeqUntil A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P K) (a : Fin K) :
+    (h : IsAlgEnvSeqUntil O A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P K) (a : Fin K) :
     pullCount A a K =ᵐ[P] fun _ ↦ 1 := by
   suffices pullCount A a (K * 1) =ᵐ[P] fun _ ↦ 1 by simpa using this
-  refine pullCount_mul 1 (P := P) (ν := ν) (Y := Y) (hK := hK) ?_ a
+  refine pullCount_mul 1 (P := P) (ν := ν) (O := O) (Y := Y) (hK := hK) ?_ a
   simpa
 
 lemma time_gt_of_pullCount_gt_one
-    (h : IsAlgEnvSeqUntil A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P K) (a : Fin K) :
+    (h : IsAlgEnvSeqUntil O A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P K) (a : Fin K) :
     ∀ᵐ ω ∂P, ∀ n, 1 < pullCount A a n ω → K < n := by
   filter_upwards [pullCount_eq_one h a] with h h_eq n hn
   rw [← h_eq] at hn
@@ -139,7 +139,7 @@ lemma time_gt_of_pullCount_gt_one
   exact hn.not_ge (pullCount_mono _ h_lt _)
 
 lemma pullCount_pos_of_time_ge
-    (h : IsAlgEnvSeqUntil A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P K) :
+    (h : IsAlgEnvSeqUntil O A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P K) :
     ∀ᵐ ω ∂P, ∀ n, K ≤ n → ∀ b : Fin K, 0 < pullCount A b n ω := by
   have h_ae a := pullCount_eq_one h a
   simp_rw [Filter.EventuallyEq, ← ae_all_iff] at h_ae
@@ -149,7 +149,7 @@ lemma pullCount_pos_of_time_ge
   exact pullCount_mono _ hn _
 
 lemma pullCount_pos_of_pullCount_gt_one
-    (h : IsAlgEnvSeqUntil A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P K) (a : Fin K) :
+    (h : IsAlgEnvSeqUntil O A Y (roundRobinAlgorithm hK) (stationaryEnv ν) P K) (a : Fin K) :
     ∀ᵐ ω ∂P, ∀ n, 1 < pullCount A a n ω → ∀ b : Fin K, 0 < pullCount A b n ω := by
   filter_upwards [time_gt_of_pullCount_gt_one h a, pullCount_pos_of_time_ge h] with ω h1 h2 n h_gt a
   exact h2 n (h1 n h_gt).le a
diff --git a/LeanMachineLearning/SequentialLearning/Algorithms/Uniform.lean b/LeanMachineLearning/SequentialLearning/Algorithms/Uniform.lean
index a2efb83b..7c6f3c4d 100644
--- a/LeanMachineLearning/SequentialLearning/Algorithms/Uniform.lean
+++ b/LeanMachineLearning/SequentialLearning/Algorithms/Uniform.lean
@@ -32,13 +32,15 @@ open scoped Algorithm
 
 namespace Learning
 
-variable {𝓐 𝓨 : Type*} {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨}
+variable {𝓞 𝓐 𝓨 : Type*} {m𝓞 : MeasurableSpace 𝓞} {m𝓐 : MeasurableSpace 𝓐}
+  {m𝓨 : MeasurableSpace 𝓨}
 
 /-- The Uniform algorithm: actions are chosen uniformly at random. -/
 noncomputable
-def uniformAlgorithm [Finite 𝓐] [Nonempty 𝓐] : Algorithm 𝓐 𝓨 := randomSampling (uniformOn Set.univ)
+def uniformAlgorithm [Finite 𝓐] [Nonempty 𝓐] : Algorithm 𝓞 𝓐 𝓨 :=
+  randomSampling (uniformOn Set.univ)
 
-lemma absolutelyContinuous_uniformAlgorithm [Finite 𝓐] [Nonempty 𝓐] {alg : Algorithm 𝓐 𝓨} :
+lemma absolutelyContinuous_uniformAlgorithm [Finite 𝓐] [Nonempty 𝓐] {alg : Algorithm 𝓞 𝓐 𝓨} :
     alg ≪ₐ uniformAlgorithm where
   policy n h := Measure.absolutelyContinuous_of_measure_singleton_ne_zero
     (by simp [uniformAlgorithm, uniformOn, ← pos_iff_ne_zero, cond_pos_of_inter_ne_zero])
diff --git a/LeanMachineLearning/SequentialLearning/BayesStationaryEnv.lean b/LeanMachineLearning/SequentialLearning/BayesStationaryEnv.lean
index 97b40c6a..779a6a71 100644
--- a/LeanMachineLearning/SequentialLearning/BayesStationaryEnv.lean
+++ b/LeanMachineLearning/SequentialLearning/BayesStationaryEnv.lean
@@ -5,36 +5,47 @@ Authors: Paulo Rauber, Rémy Degenne
 -/
 module
 
+public import LeanMachineLearning.SequentialLearning.Comap
 public import LeanMachineLearning.SequentialLearning.IonescuTulceaSpace
 public import LeanMachineLearning.SequentialLearning.StationaryEnv
 
 /-!
 # Bayesian stationary environments
 
-This file defines the structure `IsBayesAlgEnvSeq` and provides its basic properties.
+A Bayesian stationary environment is an environment that draws a parameter `e : 𝓔` from a prior
+`Q` before the first round and then behaves like the stationary environment
+`stationaryEnv (κ.sectR e)`. Concretely, `bayesStationaryEnv Q κ : Environment 𝓔 𝓐 𝓨` announces
+an observation `e` at every round and runs against `alg.comapObs (fun _ ↦ ())`, for
+an `alg : Algorithm Unit 𝓐 𝓨`, an algorithm that does not use the observation.
 
 ## Main definitions
 
-* `IsBayesAlgEnvSeq Q κ alg E A Y P`: states that there is a measure `P : Measure Ω` such
-  that the parameter `E : Ω → 𝓔` has law `Q` and that the sequences of actions `A : ℕ → Ω → 𝓐`
-  and feedbacks `Y : ℕ → Ω → 𝓨` are generated by the algorithm `alg : Algorithm 𝓐 𝓨` interacting
-  with an underlying environment that depends on `E` and `κ` (`stationaryEnv (κ.sectR (E ω))`).
+* `bayesStationaryEnv Q κ`: the environment that draws a parameter from `Q` before the first
+  round, announces it as the observation of every round, and returns feedback `κ (e, a)` when the
+  parameter is `e` and the action is `a`.
+* `IsBayesAlgEnvSeq Q κ alg E A Y P`: states that the parameter `E : Ω → 𝓔` has law `Q` and that
+  the sequences of actions `A : ℕ → Ω → 𝓐` and feedbacks `Y : ℕ → Ω → 𝓨` are generated by the
+  algorithm `alg : Algorithm Unit 𝓐 𝓨` interacting with `bayesStationaryEnv Q κ`, which it sees
+  through `Algorithm.comapObs (fun _ ↦ ())`. Equivalently, `A` and `Y` are generated by `alg`
+  interacting with the stationary environment `stationaryEnv (κ.sectR (E ω))`.
 * `bayesTrajMeasure Q κ alg`: for any choice of probability measure `Q : Measure 𝓔`, Markov kernel
-  `κ : Kernel (𝓔 × 𝓐) 𝓨`, and algorithm `alg : Algorithm 𝓐 𝓨`, provides a probability measure
-  `P : Measure (ℕ → 𝓐 × 𝓔 × 𝓨)` on a space that carries `E`, `A`, and `Y` such that
+  `κ : Kernel (𝓔 × 𝓐) 𝓨`, and algorithm `alg : Algorithm Unit 𝓐 𝓨`, provides a probability measure
+  `P : Measure (ℕ → Round 𝓔 𝓐 𝓨)` on a space that carries `E`, `A`, and `Y` such that
   `IsBayesAlgEnvSeq Q κ alg E A Y P`.
-* `bayesTrajMeasurePosterior Q κ alg n`: a `Kernel (Fin n → 𝓐 × 𝓨) 𝓔` that represents the
-  posterior over `E` given the history before time `n` (the `n` first action-feedback pairs) under
+* `bayesTrajMeasurePosterior Q κ alg n`: a `Kernel (Hist Unit 𝓐 𝓨 n) 𝓔` that represents the
+  posterior over `E` given the history before time `n` (the `n` first rounds) under
   the prior `Q` and the algorithm `alg`, assuming that the kernel `κ` specifies how `E` gives rise
   to the underlying (stationary) environment.
   See also `LeanMachineLearning/SequentialLearning/AlgorithmDensityBayes.lean`.
 
 ## Main results
 
+* `IsAlgEnvSeq.isBayesAlgEnvSeq`: a run of `alg.comapObs (fun _ ↦ ())` against
+  `bayesStationaryEnv Q κ` is a Bayesian algorithm-environment sequence for the announced parameter.
 * `ae_IsAlgEnvSeq h`: if `h : IsBayesAlgEnvSeq Q κ alg E A Y P`, for `Q`-almost every `e : 𝓔`,
-  `IsAlgEnvSeq A' Y' alg (stationaryEnv (κ.sectR e)) (condDistrib (trajectory A Y) E P e)` for some
-  sequence of actions `A' : ℕ → (ℕ → 𝓐 × 𝓨) → 𝓐` and sequence of feedbacks
-  `Y' : ℕ → (ℕ → 𝓐 × 𝓨) → 𝓨`. Intuitively, if the observable trajectory is generated by an
+  `IsAlgEnvSeq O' A' Y' alg (stationaryEnv (κ.sectR e)) (condDistrib (trajectory _ A Y) E P e)` for
+  some sequence of actions `A' : ℕ → (ℕ → Round Unit 𝓐 𝓨) → 𝓐` and sequence of feedbacks
+  `Y' : ℕ → (ℕ → Round Unit 𝓐 𝓨) → 𝓨`. Intuitively, if the observable trajectory is generated by an
   underlying parameter `e : 𝓔`, the measure that carries the `IsBayesAlgEnvSeq` structure reveals a
   measure that carries an `IsAlgEnvSeq` structure under the environment `stationaryEnv (κ.sectR e)`
   and the same algorithm. This allows transferring results from the `IsAlgEnvSeq` structure to the
@@ -52,217 +63,268 @@ namespace Learning
 variable {𝓔 𝓐 𝓨 Ω : Type*}
 variable [MeasurableSpace 𝓔] [MeasurableSpace 𝓐] [MeasurableSpace 𝓨] [MeasurableSpace Ω]
 
+section BayesEnv
+
+/-- The environment that draws a parameter `e : 𝓔` from `Q` before the first round, announces it as
+the observation of every round, and returns a feedback drawn from `κ (e, a)` when the action is
+`a`. The algorithm is meant to ignore the announced parameter, that is, to run through
+`Algorithm.comapObs (fun _ ↦ ())`. -/
+noncomputable
+def bayesStationaryEnv (Q : Measure 𝓔) [IsProbabilityMeasure Q] (κ : Kernel (𝓔 × 𝓐) 𝓨)
+    [IsMarkovKernel κ] : Environment 𝓔 𝓐 𝓨 where
+  obs
+    | 0 => Kernel.const _ Q
+    | _ + 1 => Kernel.deterministic (fun h ↦ (h 0).obs) (by fun_prop)
+  feedback _ := κ.comap (fun p ↦ (p.1.2, p.2)) (by fun_prop)
+  isMarkovKernel_obs n := by cases n <;> infer_instance
+
+variable {Q : Measure 𝓔} [IsProbabilityMeasure Q] {κ : Kernel (𝓔 × 𝓐) 𝓨} [IsMarkovKernel κ]
+
+@[simp]
+lemma obs_bayesStationaryEnv_zero : (bayesStationaryEnv Q κ).obs 0 = Kernel.const _ Q := rfl
+
+@[simp]
+lemma obs_bayesStationaryEnv_succ (n : ℕ) :
+    (bayesStationaryEnv Q κ).obs (n + 1)
+      = Kernel.deterministic (fun h : Hist 𝓔 𝓐 𝓨 (n + 1) ↦ (h 0).obs) (by fun_prop) := rfl
+
+@[simp]
+lemma feedback_bayesStationaryEnv (n : ℕ) :
+    (bayesStationaryEnv Q κ).feedback n = κ.comap (fun p ↦ (p.1.2, p.2)) (by fun_prop) := rfl
+
+@[simp]
+lemma obs0_bayesStationaryEnv : (bayesStationaryEnv Q κ).obs0 = Q := rfl
+
+@[simp]
+lemma ν0_bayesStationaryEnv : (bayesStationaryEnv Q κ).ν0 = κ := rfl
+
+end BayesEnv
+
+/-- Insert an announced parameter `e` as the observation of every round of an observable history. -/
+def announceHist (e : 𝓔) {n : ℕ} (h : Hist Unit 𝓐 𝓨 n) : Hist 𝓔 𝓐 𝓨 n :=
+  fun i ↦ (e, (h i).action, (h i).feedback)
+
+@[fun_prop]
+lemma measurable_announceHist (n : ℕ) :
+    Measurable (fun p : 𝓔 × Hist Unit 𝓐 𝓨 n ↦ announceHist p.1 p.2) := by
+  unfold announceHist
+  fun_prop
+
 /-- `IsBayesAlgEnvSeq Q κ alg E A Y P` states that there is a measure `P : Measure Ω` such
-  that the parameter `E : Ω → 𝓔` has law `Q` and that the sequences of actions `A : ℕ → Ω → 𝓐`
-  and feedbacks `Y : ℕ → Ω → 𝓨` are generated by the algorithm `alg : Algorithm 𝓐 𝓨` interacting
-  with an underlying environment that depends on `E` and `κ` (`stationaryEnv (κ.sectR (E ω))`). -/
-structure IsBayesAlgEnvSeq
-    (Q : Measure 𝓔) (κ : Kernel (𝓔 × 𝓐) 𝓨) (alg : Algorithm 𝓐 𝓨)
+that the parameter `E : Ω → 𝓔` has law `Q` and that the sequences of actions `A : ℕ → Ω → 𝓐`
+and feedbacks `Y : ℕ → Ω → 𝓨` are generated by the algorithm `alg : Algorithm Unit 𝓐 𝓨`
+interacting with an underlying environment that depends on `E` and `κ`
+(`stationaryEnv (κ.sectR (E ω))`).
+
+This is `IsAlgEnvSeq` for the announcing environment `bayesStationaryEnv Q κ` and the algorithm
+`alg.comapObs (fun _ ↦ ())` that ignores the announced parameter: the observation at every round
+is `E ω`. -/
+def IsBayesAlgEnvSeq (Q : Measure 𝓔) [IsProbabilityMeasure Q] (κ : Kernel (𝓔 × 𝓐) 𝓨)
+    [IsMarkovKernel κ] (alg : Algorithm Unit 𝓐 𝓨)
     (E : Ω → 𝓔) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨)
-    (P : Measure Ω) [IsFiniteMeasure P] : Prop where
-  measurable_param : Measurable E := by fun_prop
-  measurable_action n : Measurable (A n) := by fun_prop
-  measurable_feedback n : Measurable (Y n) := by fun_prop
-  hasLaw_env : HasLaw E Q P
-  /-- The action at time `n` has the correct conditional distribution given the parameter and the
-  history: it depends only on the history. -/
-  hasCondDistrib_action n :
-    HasCondDistrib (A n) (fun ω ↦ (E ω, history A Y n ω)) ((alg.policy n).prodMkLeft _) P
-  /-- The feedback at time `n` has the correct conditional distribution given the history, the
-  parameter and the action at time `n`: it depends only on the parameter and the action. -/
-  hasCondDistrib_feedback n :
-    HasCondDistrib (Y n) (fun ω ↦ (history A Y n ω, E ω, A n ω)) (κ.prodMkLeft _) P
+    (P : Measure Ω) [IsProbabilityMeasure P] : Prop :=
+  IsAlgEnvSeq (fun _ ↦ E) A Y (alg.comapObs (fun _ : 𝓔 ↦ ())) (bayesStationaryEnv Q κ) P
 
 namespace IsBayesAlgEnvSeq
 
-variable {Q : Measure 𝓔} {κ : Kernel (𝓔 × 𝓐) 𝓨} {alg : Algorithm 𝓐 𝓨}
-variable {E : Ω → 𝓔} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
-variable {P : Measure Ω} [IsFiniteMeasure P]
+variable {Q : Measure 𝓔} [IsProbabilityMeasure Q] {κ : Kernel (𝓔 × 𝓐) 𝓨} [IsMarkovKernel κ]
+  {alg : Algorithm Unit 𝓐 𝓨} {P : Measure Ω} [IsProbabilityMeasure P]
+  {E : Ω → 𝓔} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
+
+/-- A Bayesian algorithm-environment sequence is an algorithm-environment sequence for the
+announcing environment `bayesStationaryEnv Q κ`. -/
+lemma isAlgEnvSeq (h : IsBayesAlgEnvSeq Q κ alg E A Y P) :
+    IsAlgEnvSeq (fun _ ↦ E) A Y (alg.comapObs (fun _ : 𝓔 ↦ ())) (bayesStationaryEnv Q κ) P := h
+
+lemma measurable_param (h : IsBayesAlgEnvSeq Q κ alg E A Y P) : Measurable E :=
+  h.isAlgEnvSeq.measurable_obs 0
+
+lemma measurable_action (h : IsBayesAlgEnvSeq Q κ alg E A Y P) (n : ℕ) : Measurable (A n) :=
+  h.isAlgEnvSeq.measurable_action n
+
+lemma measurable_feedback (h : IsBayesAlgEnvSeq Q κ alg E A Y P) (n : ℕ) : Measurable (Y n) :=
+  h.isAlgEnvSeq.measurable_feedback n
+
+/-- The parameter has law `Q`. -/
+lemma hasLaw_env (h : IsBayesAlgEnvSeq Q κ alg E A Y P) : HasLaw E Q P := by
+  have h0 := h.isAlgEnvSeq.hasCondDistrib_obs 0
+  rw [history_zero] at h0
+  exact h0.hasLaw_of_const'
+
+/-- The action at time `n` has the correct conditional distribution given the parameter and the
+history: it depends only on the history. -/
+lemma hasCondDistrib_action (h : IsBayesAlgEnvSeq Q κ alg E A Y P) (n : ℕ) :
+    HasCondDistrib (A n) (fun ω ↦ (E ω, (history (noObs Ω) A Y n ω, noObs Ω n ω)))
+      ((alg.policy n).prodMkLeft _) P :=
+  HasCondDistrib.comp_right
+    (f := fun p : Hist 𝓔 𝓐 𝓨 n × 𝓔 ↦ (p.2, (Hist.mapObs (fun _ ↦ ()) p.1, ())))
+    (hf := by fun_prop)
+    (Z := fun ω ↦ (history (fun _ ↦ E) A Y n ω, E ω))
+    (h.isAlgEnvSeq.hasCondDistrib_action n)
+
+/-- The feedback at time `n` has the correct conditional distribution given the history, the
+parameter and the action at time `n`: it depends only on the parameter and the action. -/
+lemma hasCondDistrib_feedback (h : IsBayesAlgEnvSeq Q κ alg E A Y P) (n : ℕ) :
+    HasCondDistrib (Y n) (fun ω ↦ ((history (noObs Ω) A Y n ω, noObs Ω n ω), (E ω, A n ω)))
+      (κ.prodMkLeft _) P :=
+  HasCondDistrib.comp_right
+    (f := fun p : (Hist 𝓔 𝓐 𝓨 n × 𝓔) × 𝓐 ↦
+      ((Hist.mapObs (fun _ ↦ ()) p.1.1, ()), (p.1.2, p.2)))
+    (hf := by fun_prop)
+    (Z := fun ω ↦ ((history (fun _ ↦ E) A Y n ω, E ω), A n ω))
+    (h.isAlgEnvSeq.hasCondDistrib_feedback n)
 
 lemma hasCondDistrib_action' (h : IsBayesAlgEnvSeq Q κ alg E A Y P) (n : ℕ) :
-    HasCondDistrib (A n) (history A Y n) (alg.policy n) P :=
+    HasCondDistrib (A n) (fun ω ↦ (history (noObs Ω) A Y n ω, noObs Ω n ω)) (alg.policy n) P :=
   (h.hasCondDistrib_action n).comp_right
 
-lemma hasCondDistrib_feedback' [IsFiniteKernel κ] (h : IsBayesAlgEnvSeq Q κ alg E A Y P) (n : ℕ) :
+lemma hasCondDistrib_feedback' (h : IsBayesAlgEnvSeq Q κ alg E A Y P) (n : ℕ) :
     HasCondDistrib (Y n) (fun ω ↦ (E ω, A n ω)) κ P :=
   (h.hasCondDistrib_feedback n).comp_right
 
-lemma hasLaw_action_zero [IsProbabilityMeasure P] (h : IsBayesAlgEnvSeq Q κ alg E A Y P) :
-    HasLaw (A 0) alg.p0 P := by
-  have h0 := h.hasCondDistrib_action' 0
-  rw [history_zero] at h0
+lemma hasLaw_action_zero (h : IsBayesAlgEnvSeq Q κ alg E A Y P) :
+    HasLaw (A 0) (alg.p0 ()) P := by
+  have h0 : HasCondDistrib (A 0)
+      (fun _ : Ω ↦ ((default : Hist Unit 𝓐 𝓨 0), ())) (alg.policy 0) P := by
+    have h1 := h.hasCondDistrib_action' 0
+    rwa [history_zero] at h1
   exact h0.hasLaw_of_const'
 
-/-- The first action is independent of the parameter `E`, and has law `alg.p0`. -/
+/-- The first action is independent of the parameter `E`, and has law `alg.p0 ()`. -/
 lemma hasCondDistrib_action_zero (h : IsBayesAlgEnvSeq Q κ alg E A Y P) :
-    HasCondDistrib (A 0) E (Kernel.const _ alg.p0) P :=
+    HasCondDistrib (A 0) E (Kernel.const _ (alg.p0 ())) P :=
   hasCondDistrib_prodMk_right_unique_iff.mp (h.hasCondDistrib_action 0)
 
-variable [StandardBorelSpace 𝓐] [Nonempty 𝓐] [StandardBorelSpace 𝓨] [Nonempty 𝓨]
-
-omit [StandardBorelSpace 𝓐] [Nonempty 𝓐] [StandardBorelSpace 𝓨] [Nonempty 𝓨] in
 /-- The posterior over the parameter given the empty history is the prior. -/
-lemma condDistrib_param_history_zero [StandardBorelSpace 𝓔] [Nonempty 𝓔] [IsProbabilityMeasure P]
-    [IsFiniteMeasure Q] (h : IsBayesAlgEnvSeq Q κ alg E A Y P) :
-    condDistrib E (history A Y 0) P = Kernel.const _ Q := by
+lemma condDistrib_param_history_zero [StandardBorelSpace 𝓔] [Nonempty 𝓔]
+    (h : IsBayesAlgEnvSeq Q κ alg E A Y P) :
+    condDistrib E (history (noObs Ω) A Y 0) P = Kernel.const _ Q := by
   ext x : 1
   rw [Unique.eq_default x, history_zero]
-  have hc : HasCondDistrib E (fun _ : Ω ↦ (default : Fin 0 → 𝓐 × 𝓨)) (Kernel.const _ Q) P :=
+  have hc : HasCondDistrib E (fun _ : Ω ↦ (default : Hist Unit 𝓐 𝓨 0)) (Kernel.const _ Q) P :=
     h.hasLaw_env.hasCondDistrib_const
   have h_ae := hc.condDistrib_eq
   rw [Measure.map_const, measure_univ, one_smul, Filter.EventuallyEq,
     ae_dirac_iff Subsingleton.measurableSet] at h_ae
   exact h_ae
 
+variable [StandardBorelSpace 𝓐] [Nonempty 𝓐] [StandardBorelSpace 𝓨] [Nonempty 𝓨]
+
+lemma hasCondDistrib_IT_obs (_h : IsBayesAlgEnvSeq Q κ alg E A Y P) (n : ℕ) :
+    ∀ᵐ e ∂Q, HasCondDistrib (IT.obs n) (IT.hist n) (Kernel.const _ (Measure.dirac ()))
+      (condDistrib (trajectory (noObs Ω) A Y) E P e) :=
+  ae_of_all _ fun _ ↦ hasCondDistrib_unit (IT.measurable_hist n).aemeasurable _ _
+
 lemma hasCondDistrib_IT_action (h : IsBayesAlgEnvSeq Q κ alg E A Y P) (n : ℕ) :
-    ∀ᵐ e ∂Q, HasCondDistrib (IT.action n) (IT.hist n) (alg.policy n)
-      (condDistrib (trajectory A Y) E P e) := by
+    ∀ᵐ e ∂Q, HasCondDistrib (IT.action n) (fun τ ↦ (IT.hist n τ, IT.obs n τ)) (alg.policy n)
+      (condDistrib (trajectory (noObs Ω) A Y) E P e) := by
   rw [← h.hasLaw_env.map_eq]
   filter_upwards [(h.hasCondDistrib_action n).hasCondDistrib_sectR
-    (IT.measurable_hist n) (IT.measurable_action n)
-    (measurable_trajectory h.measurable_action h.measurable_feedback).aemeasurable] with _ he
+    ((IT.measurable_hist n).prodMk (IT.measurable_obs n)) (IT.measurable_action n)
+    (measurable_trajectory (O := noObs Ω) (fun _ ↦ measurable_const) h.measurable_action
+      h.measurable_feedback).aemeasurable] with _ he
   rwa [Kernel.sectR_prodMkLeft] at he
 
-lemma hasCondDistrib_IT_feedback [IsFiniteKernel κ] (h : IsBayesAlgEnvSeq Q κ alg E A Y P)
-    (n : ℕ) :
-    ∀ᵐ e ∂Q, HasCondDistrib (IT.feedback n) (fun τ ↦ (IT.hist n τ, IT.action n τ))
-      ((κ.sectR e).prodMkLeft _) (condDistrib (trajectory A Y) E P e) := by
+lemma hasCondDistrib_IT_feedback (h : IsBayesAlgEnvSeq Q κ alg E A Y P) (n : ℕ) :
+    ∀ᵐ e ∂Q, HasCondDistrib (IT.feedback n)
+      (fun τ ↦ ((IT.hist n τ, IT.obs n τ), IT.action n τ))
+      ((κ.sectR e).prodMkLeft _) (condDistrib (trajectory (noObs Ω) A Y) E P e) := by
   rw [← h.hasLaw_env.map_eq]
-  have hc : HasCondDistrib (Y n) (fun ω ↦ (E ω, history A Y n ω, A n ω))
-      (κ.comap (fun (e, _, a) ↦ (e, a)) (by fun_prop)) P :=
+  have hc : HasCondDistrib (Y n)
+      (fun ω ↦ (E ω, ((history (noObs Ω) A Y n ω, noObs Ω n ω), A n ω)))
+      (κ.comap (fun p : 𝓔 × ((Hist Unit 𝓐 𝓨 n × Unit) × 𝓐) ↦ (p.1, p.2.2)) (by fun_prop)) P :=
     (h.hasCondDistrib_feedback n).measurableEquiv_comp_right (MeasurableEquiv.prodAssoc.symm.trans
       ((MeasurableEquiv.prodCongr .prodComm (.refl _)).trans .prodAssoc))
-  exact hc.hasCondDistrib_sectR ((IT.measurable_hist n).prodMk (IT.measurable_action n))
+  exact hc.hasCondDistrib_sectR
+    (((IT.measurable_hist n).prodMk (IT.measurable_obs n)).prodMk (IT.measurable_action n))
     (IT.measurable_feedback n)
-    (measurable_trajectory h.measurable_action h.measurable_feedback).aemeasurable
+    (measurable_trajectory (O := noObs Ω) (fun _ ↦ measurable_const) h.measurable_action
+      h.measurable_feedback).aemeasurable
 
 lemma hasLaw_IT_hist (h : IsBayesAlgEnvSeq Q κ alg E A Y P) (n : ℕ) :
-    ∀ᵐ e ∂Q, HasLaw (IT.hist n) (condDistrib (history A Y n) E P e)
-      (condDistrib (trajectory A Y) E P e) := by
-  rw [← h.hasLaw_env.map_eq, show history A Y n = IT.hist n ∘ trajectory A Y from rfl]
+    ∀ᵐ e ∂Q, HasLaw (IT.hist n) (condDistrib (history (noObs Ω) A Y n) E P e)
+      (condDistrib (trajectory (noObs Ω) A Y) E P e) := by
+  rw [← h.hasLaw_env.map_eq, show history (noObs Ω) A Y n
+    = IT.hist (𝓞 := Unit) (𝓐 := 𝓐) (𝓨 := 𝓨) n ∘ trajectory (noObs Ω) A Y from rfl]
   filter_upwards [condDistrib_comp E
-    (measurable_trajectory h.measurable_action h.measurable_feedback).aemeasurable
-    (IT.measurable_hist n)] with _ he
+    (measurable_trajectory (O := noObs Ω) (fun _ ↦ measurable_const) h.measurable_action
+      h.measurable_feedback).aemeasurable
+    (IT.measurable_hist (𝓞 := Unit) (𝓐 := 𝓐) (𝓨 := 𝓨) n)] with _ he
   exact ⟨(IT.measurable_hist n).aemeasurable, by
     rw [← Kernel.map_apply _ (IT.measurable_hist n), he]⟩
 
-lemma ae_IsAlgEnvSeq [IsMarkovKernel κ] (h : IsBayesAlgEnvSeq Q κ alg E A Y P) :
-    ∀ᵐ e ∂Q, IsAlgEnvSeq IT.action IT.feedback alg (stationaryEnv (κ.sectR e))
-      (condDistrib (trajectory A Y) E P e) := by
-  filter_upwards [ae_all_iff.2 (hasCondDistrib_IT_action h),
-    ae_all_iff.2 (hasCondDistrib_IT_feedback h)] with _ hA hR
-  exact ⟨IT.measurable_action, IT.measurable_feedback, hA, hR⟩
+lemma ae_IsAlgEnvSeq (h : IsBayesAlgEnvSeq Q κ alg E A Y P) :
+    ∀ᵐ e ∂Q, IsAlgEnvSeq IT.obs IT.action IT.feedback alg (stationaryEnv (κ.sectR e))
+      (condDistrib (trajectory (noObs Ω) A Y) E P e) := by
+  filter_upwards [ae_all_iff.2 (hasCondDistrib_IT_obs h),
+    ae_all_iff.2 (hasCondDistrib_IT_action h),
+    ae_all_iff.2 (hasCondDistrib_IT_feedback h)] with _ hO hA hR
+  exact ⟨IT.measurable_obs, IT.measurable_action, IT.measurable_feedback, hO, hA, hR⟩
 
 end IsBayesAlgEnvSeq
 
 section IsAlgEnvSeq
 
-/-- An environment with observations in `𝓔 × 𝓨`. The first element `e` of an observation is
-sampled from `Q` at time `0` and then remains constant: at time `n + 1` it is read from the first
-observation in the history. The second element of an observation is sampled from `κ (e, a)`, where
-`a` is the corresponding action. -/
-noncomputable
-def bayesStationaryEnv (Q : Measure 𝓔) [IsProbabilityMeasure Q] (κ : Kernel (𝓔 × 𝓐) 𝓨)
-    [IsMarkovKernel κ] : Environment 𝓐 (𝓔 × 𝓨) where
-  feedback
-    | 0 => ((Kernel.const _ Q) ⊗ₖ κ.swapLeft).prodMkLeft _
-    | n + 1 =>
-      let g : (Fin (n + 1) → 𝓐 × 𝓔 × 𝓨) × 𝓐 → 𝓔 × 𝓐 := fun (h, a) ↦ ((h 0).2.1, a)
-      (Kernel.deterministic (Prod.fst ∘ g) (by fun_prop)) ×ₖ (κ.comap g (by fun_prop))
-  h_feedback n := by cases n <;> infer_instance
-
 variable {Q : Measure 𝓔} [IsProbabilityMeasure Q] {κ : Kernel (𝓔 × 𝓐) 𝓨} [IsMarkovKernel κ]
-
-lemma bayesStationaryEnv_feedback_zero :
-    (bayesStationaryEnv Q κ).feedback 0 = ((Kernel.const _ Q) ⊗ₖ κ.swapLeft).prodMkLeft _ := rfl
-
-lemma bayesStationaryEnv_feedback_succ (n : ℕ) :
-    (bayesStationaryEnv Q κ).feedback (n + 1) =
-      (Kernel.deterministic (Prod.fst ∘ (fun (p : (Fin (n + 1) → 𝓐 × 𝓔 × 𝓨) × 𝓐) ↦
-        ((p.1 0).2.1, p.2))) (by fun_prop)) ×ₖ
-        (κ.comap (fun p ↦ ((p.1 0).2.1, p.2)) (by fun_prop)) := rfl
-
-@[simp]
-lemma ν0_bayesStationaryEnv : (bayesStationaryEnv Q κ).ν0 = (Kernel.const _ Q) ⊗ₖ κ.swapLeft := by
-  rw [Environment.ν0_def, bayesStationaryEnv_feedback_zero, Kernel.sectR_prodMkLeft]
-
-variable {alg : Algorithm 𝓐 𝓨} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓔 × 𝓨}
+variable {alg : Algorithm Unit 𝓐 𝓨} {O : ℕ → Ω → 𝓔} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
 variable {P : Measure Ω} [IsProbabilityMeasure P]
 
-lemma IsAlgEnvSeq.isBayesAlgEnvSeq
-    (h : IsAlgEnvSeq A Y (alg.prodLeft 𝓔) (bayesStationaryEnv Q κ) P) :
-    IsBayesAlgEnvSeq Q κ alg (fun ω ↦ (Y 0 ω).1) A (fun n ω ↦ (Y n ω).2) P where
-  measurable_param := (h.measurable_feedback 0).fst
-  measurable_action := h.measurable_action
-  measurable_feedback n := (h.measurable_feedback n).snd
-  hasLaw_env := by
-    apply HasCondDistrib.hasLaw_of_const
-    simpa [Kernel.fst_compProd] using h.hasCondDistrib_feedback_zero.fst
-  hasCondDistrib_action n := by
-    cases n with
-    | zero =>
-      -- At time `0` the history is empty: `E` is the parameter component of `Y 0`, which is
-      -- independent of `A 0`.
-      have hc : HasCondDistrib (fun ω ↦ (Y 0 ω).1) (A 0) (Kernel.const _ Q) P := by
-        simpa [Kernel.fst_compProd] using h.hasCondDistrib_feedback_zero.fst
-      have hc' : HasCondDistrib (A 0) (fun ω ↦ (Y 0 ω).1) (Kernel.const _ alg.p0) P := by
-        simpa [h.hasLaw_action_zero.map_eq] using hc.const_map_of_const
-      exact hasCondDistrib_prodMk_right_unique_iff.mpr hc'
-    | succ n =>
-      let f : (Fin (n + 1) → 𝓐 × 𝓔 × 𝓨) → 𝓔 × (Fin (n + 1) → 𝓐 × 𝓨) :=
-        fun h ↦ ((h 0).2.1, fun i ↦ ((h i).1, (h i).2.2))
-      have hc : HasCondDistrib (A (n + 1)) (history A Y (n + 1))
-          (((alg.policy (n + 1)).comap Prod.snd (by fun_prop)).comap f (by fun_prop)) P :=
-        h.hasCondDistrib_action (n + 1)
-      exact hc.comp_right (f := f)
-  hasCondDistrib_feedback n := by
-    cases n with
-    | zero =>
-      have hc : HasCondDistrib (Y 0) (A 0) ((Kernel.const _ Q) ⊗ₖ κ.swapLeft) P := by
-        simpa using h.hasCondDistrib_feedback_zero
-      exact hasCondDistrib_prodMk_left_unique_iff.mpr
-        (hc.of_compProd.measurableEquiv_comp_right MeasurableEquiv.prodComm)
-    | succ n =>
-      let f : (Fin (n + 1) → 𝓐 × 𝓔 × 𝓨) × 𝓐 → (Fin (n + 1) → 𝓐 × 𝓨) × 𝓔 × 𝓐 :=
-        fun p ↦ ((fun i ↦ ((p.1 i).1, (p.1 i).2.2)), (p.1 0).2.1, p.2)
-      have hc : HasCondDistrib (fun ω ↦ (Y (n + 1) ω).2)
-          (fun ω ↦ (history A Y (n + 1) ω, A (n + 1) ω))
-          ((Kernel.prodMkLeft (Fin (n + 1) → 𝓐 × 𝓨) κ).comap f (by fun_prop)) P := by
-        simpa [bayesStationaryEnv_feedback_succ, Kernel.prodMkLeft, ← Kernel.comap_comp_right,
-          Function.comp_def] using (h.hasCondDistrib_feedback (n + 1)).snd
-      exact hc.comp_right
+/-- Under `bayesStationaryEnv Q κ`, the announced parameter is almost surely the same at every
+round. -/
+lemma IsAlgEnvSeq.ae_obs_eq_obs_zero [StandardBorelSpace 𝓔]
+    (h : IsAlgEnvSeq O A Y (alg.comapObs (fun _ : 𝓔 ↦ ())) (bayesStationaryEnv Q κ) P) (n : ℕ) :
+    O 0 =ᵐ[P] O n := by
+  cases n with
+  | zero => rfl
+  | succ n =>
+    have h1 := h.hasCondDistrib_obs (n + 1)
+    rw [obs_bayesStationaryEnv_succ] at h1
+    have h2 := ae_eq_of_hasCondDistrib_deterministic (by fun_prop)
+      (h.measurable_history (n + 1)).aemeasurable (h.measurable_obs (n + 1)).aemeasurable h1
+    filter_upwards [h2] with ω hω
+    rw [hω]
+    simp [history_apply]
+
+/-- A run of `alg.comapObs (fun _ ↦ ())` against the announcing environment
+`bayesStationaryEnv Q κ` is a Bayesian algorithm-environment sequence for the announced
+parameter. -/
+lemma IsAlgEnvSeq.isBayesAlgEnvSeq [StandardBorelSpace 𝓔]
+    (h : IsAlgEnvSeq O A Y (alg.comapObs (fun _ : 𝓔 ↦ ())) (bayesStationaryEnv Q κ) P) :
+    IsBayesAlgEnvSeq Q κ alg (O 0) A Y P :=
+  h.congr (fun _ ↦ h.measurable_obs 0) h.measurable_action h.measurable_feedback
+    h.ae_obs_eq_obs_zero (fun _ ↦ .rfl) (fun _ ↦ .rfl)
 
 end IsAlgEnvSeq
 
 namespace IT
 
 /-- A measure `P` on a measurable space that carries random variables `E`, `A`, and `Y` such that
-`IsBayesAlgEnvSeq Q κ alg E A Y P`. -/
+`IsBayesAlgEnvSeq Q κ alg E A Y P`. The parameter is the observation of the first round,
+`IT.obs 0`. -/
 noncomputable
 def bayesTrajMeasure (Q : Measure 𝓔) [IsProbabilityMeasure Q] (κ : Kernel (𝓔 × 𝓐) 𝓨)
-    [IsMarkovKernel κ] (alg : Algorithm 𝓐 𝓨) : Measure (ℕ → 𝓐 × 𝓔 × 𝓨) :=
-  trajMeasure (alg.prodLeft 𝓔) (bayesStationaryEnv Q κ)
+    [IsMarkovKernel κ] (alg : Algorithm Unit 𝓐 𝓨) : Measure (ℕ → Round 𝓔 𝓐 𝓨) :=
+  trajMeasure (alg.comapObs (fun _ : 𝓔 ↦ ())) (bayesStationaryEnv Q κ)
 deriving IsProbabilityMeasure
 
-lemma isBayesAlgEnvSeq_bayesTrajMeasure
+lemma isBayesAlgEnvSeq_bayesTrajMeasure [StandardBorelSpace 𝓔]
     (Q : Measure 𝓔) [IsProbabilityMeasure Q] (κ : Kernel (𝓔 × 𝓐) 𝓨) [IsMarkovKernel κ]
-    (alg : Algorithm 𝓐 𝓨) :
-    IsBayesAlgEnvSeq Q κ alg (fun ω ↦ (ω 0).2.1) action (fun n ω ↦ (ω n).2.2)
-       (bayesTrajMeasure Q κ alg) := (isAlgEnvSeq_trajMeasure _ _).isBayesAlgEnvSeq
+    (alg : Algorithm Unit 𝓐 𝓨) :
+    IsBayesAlgEnvSeq Q κ alg (obs 0) action feedback (bayesTrajMeasure Q κ alg) :=
+  (isAlgEnvSeq_trajMeasure _ _).isBayesAlgEnvSeq
 
 /-- A kernel that represents the posterior over `E` given the history before time `n`. -/
 noncomputable
 def bayesTrajMeasurePosterior [StandardBorelSpace 𝓔] [Nonempty 𝓔]
     (Q : Measure 𝓔) [IsProbabilityMeasure Q] (κ : Kernel (𝓔 × 𝓐) 𝓨) [IsMarkovKernel κ]
-    (alg : Algorithm 𝓐 𝓨) (n : ℕ) : Kernel (Fin n → 𝓐 × 𝓨) 𝓔 :=
-  condDistrib (fun ω ↦ (ω 0).2.1) (history action (fun n ω ↦ (ω n).2.2) n)
-    (bayesTrajMeasure Q κ alg)
+    (alg : Algorithm Unit 𝓐 𝓨) (n : ℕ) : Kernel (Hist Unit 𝓐 𝓨 n) 𝓔 :=
+  condDistrib (obs 0) (history (noObs _) action feedback n) (bayesTrajMeasure Q κ alg)
 deriving IsMarkovKernel
 
 /-- The posterior given the empty history is the prior. -/
 lemma bayesTrajMeasurePosterior_zero [StandardBorelSpace 𝓔] [Nonempty 𝓔]
     (Q : Measure 𝓔) [IsProbabilityMeasure Q] (κ : Kernel (𝓔 × 𝓐) 𝓨) [IsMarkovKernel κ]
-    (alg : Algorithm 𝓐 𝓨) :
+    (alg : Algorithm Unit 𝓐 𝓨) :
     bayesTrajMeasurePosterior Q κ alg 0 = Kernel.const _ Q :=
   (isBayesAlgEnvSeq_bayesTrajMeasure Q κ alg).condDistrib_param_history_zero
 
diff --git a/LeanMachineLearning/SequentialLearning/Comap.lean b/LeanMachineLearning/SequentialLearning/Comap.lean
new file mode 100644
index 00000000..78392f5a
--- /dev/null
+++ b/LeanMachineLearning/SequentialLearning/Comap.lean
@@ -0,0 +1,522 @@
+/-
+Copyright (c) 2026 Rémy Degenne. All rights reserved.
+Released under Apache 2.0 license as described in the file LICENSE.
+Authors: Rémy Degenne
+-/
+module
+
+public import LeanMachineLearning.SequentialLearning.Algorithm
+
+/-!
+# Transport of algorithms and environments
+
+## Main definitions
+
+* `Round.map fo fa fy`, `Hist.map fo fa fy`: round-wise transport of a round and a history
+  along maps of the observation, the action and the feedback, with the
+  special cases `mapObs`, `mapAction` and `mapFeedback` that transport a single component.
+* `Algorithm.comap alg F hF`: the algorithm that transforms the pair (past rounds, current
+  observation) by the measurable map `F n` at round `n` before applying the policy of `alg`.
+* `Algorithm.comapObs alg f`: the algorithm that sees `f o` when the observation is `o`, both in
+  the current round and in the past rounds.
+* `Algorithm.comapFeedback alg g`: the algorithm that sees `g y` when the feedback of a past round
+  is `y`.
+* `Environment.comap env F hF f hf`: the environment that reads the summary `F n` of the past
+  rounds and reads `f a` when the algorithm plays `a` in the current round.
+* `Environment.comapAction env f`: the environment that reads `f a` when the algorithm plays `a`,
+  both in the current round and in the past rounds.
+* `Algorithm.congr alg e𝓞 e𝓐 e𝓨`, `Environment.congr env e𝓞 e𝓐 e𝓨`: relabelling of the
+  observations, the actions and the feedbacks of a player along measurable equivalences.
+
+## Main statements
+
+* `IsAlgEnvSeq.hasCondDistrib_action_comapObs`, `IsAlgEnvSeq.hasCondDistrib_action_comapFeedback`:
+  in a run of `alg.comapObs f` (resp. `alg.comapFeedback g`) against any environment, the
+  conditional distribution of the action given the transported history and the transported
+  observation is `alg.policy n`.
+* `IsAlgEnvSeq.hasCondDistrib_obs_comapAction`, `IsAlgEnvSeq.hasCondDistrib_feedback_comapAction`:
+  in a run against `env.comapAction f`, the observations and feedbacks have the conditional
+  distributions of a run of `env` on the transported actions.
+
+-/
+
+@[expose] public section
+
+open MeasureTheory ProbabilityTheory
+
+namespace Learning
+
+variable {𝓞 𝓞' 𝓞'' 𝓐 𝓐' 𝓐'' 𝓨 𝓨' 𝓨'' Ω : Type*}
+  {m𝓞 : MeasurableSpace 𝓞} {m𝓞' : MeasurableSpace 𝓞'} {m𝓞'' : MeasurableSpace 𝓞''}
+  {m𝓐 : MeasurableSpace 𝓐} {m𝓐' : MeasurableSpace 𝓐'} {m𝓐'' : MeasurableSpace 𝓐''}
+  {m𝓨 : MeasurableSpace 𝓨} {m𝓨' : MeasurableSpace 𝓨'} {m𝓨'' : MeasurableSpace 𝓨''}
+  {mΩ : MeasurableSpace Ω}
+  {fo : 𝓞 → 𝓞'} {fa : 𝓐 → 𝓐'} {fy : 𝓨 → 𝓨'} {go : 𝓞' → 𝓞''} {ga : 𝓐' → 𝓐''} {gy : 𝓨' → 𝓨''}
+
+section Map
+
+/-- Transport a round along maps of the observation, the action and the feedback. -/
+def Round.map (fo : 𝓞 → 𝓞') (fa : 𝓐 → 𝓐') (fy : 𝓨 → 𝓨') (r : Round 𝓞 𝓐 𝓨) : Round 𝓞' 𝓐' 𝓨' :=
+  (fo r.obs, fa r.action, fy r.feedback)
+
+/-- Transport a history round-wise. -/
+def Hist.map (fo : 𝓞 → 𝓞') (fa : 𝓐 → 𝓐') (fy : 𝓨 → 𝓨') {n : ℕ} (h : Hist 𝓞 𝓐 𝓨 n) :
+    Hist 𝓞' 𝓐' 𝓨' n :=
+  fun i ↦ Round.map fo fa fy (h i)
+
+@[simp] lemma Round.obs_map (r : Round 𝓞 𝓐 𝓨) : (Round.map fo fa fy r).obs = fo r.obs := rfl
+@[simp] lemma Round.action_map (r : Round 𝓞 𝓐 𝓨) :
+    (Round.map fo fa fy r).action = fa r.action := rfl
+@[simp] lemma Round.feedback_map (r : Round 𝓞 𝓐 𝓨) :
+    (Round.map fo fa fy r).feedback = fy r.feedback := rfl
+
+@[simp] lemma Hist.map_apply {n : ℕ} (h : Hist 𝓞 𝓐 𝓨 n) (i : Fin n) :
+    Hist.map fo fa fy h i = Round.map fo fa fy (h i) := rfl
+
+@[simp] lemma Round.map_id : Round.map (id : 𝓞 → 𝓞) (id : 𝓐 → 𝓐) (id : 𝓨 → 𝓨) = id := rfl
+
+@[simp] lemma Hist.map_id {n : ℕ} :
+    Hist.map (id : 𝓞 → 𝓞) (id : 𝓐 → 𝓐) (id : 𝓨 → 𝓨) (n := n) = id := rfl
+
+lemma Round.map_comp (r : Round 𝓞 𝓐 𝓨) :
+    Round.map go ga gy (Round.map fo fa fy r) = Round.map (go ∘ fo) (ga ∘ fa) (gy ∘ fy) r := rfl
+
+lemma Hist.map_comp {n : ℕ} (h : Hist 𝓞 𝓐 𝓨 n) :
+    Hist.map go ga gy (Hist.map fo fa fy h) = Hist.map (go ∘ fo) (ga ∘ fa) (gy ∘ fy) h := rfl
+
+@[fun_prop]
+lemma Round.measurable_map (hfo : Measurable fo) (hfa : Measurable fa) (hfy : Measurable fy) :
+    Measurable (Round.map fo fa fy) := by unfold Round.map; fun_prop
+
+@[fun_prop]
+lemma Hist.measurable_map (hfo : Measurable fo) (hfa : Measurable fa) (hfy : Measurable fy)
+    (n : ℕ) :
+    Measurable (Hist.map fo fa fy (n := n)) := by unfold Hist.map; fun_prop
+
+/-- Transport the observations of a round. -/
+abbrev Round.mapObs (f : 𝓞 → 𝓞') (r : Round 𝓞 𝓐 𝓨) : Round 𝓞' 𝓐 𝓨 := Round.map f id id r
+
+/-- Transport the observations of a history. -/
+abbrev Hist.mapObs (f : 𝓞 → 𝓞') {n : ℕ} (h : Hist 𝓞 𝓐 𝓨 n) : Hist 𝓞' 𝓐 𝓨 n :=
+  Hist.map f id id h
+
+/-- Transport the actions of a round. -/
+abbrev Round.mapAction (f : 𝓐 → 𝓐') (r : Round 𝓞 𝓐 𝓨) : Round 𝓞 𝓐' 𝓨 := Round.map id f id r
+
+/-- Transport the actions of a history. -/
+abbrev Hist.mapAction (f : 𝓐 → 𝓐') {n : ℕ} (h : Hist 𝓞 𝓐 𝓨 n) : Hist 𝓞 𝓐' 𝓨 n :=
+  Hist.map id f id h
+
+/-- Transport the feedback of a round. -/
+abbrev Round.mapFeedback (f : 𝓨 → 𝓨') (r : Round 𝓞 𝓐 𝓨) : Round 𝓞 𝓐 𝓨' := Round.map id id f r
+
+/-- Transport the feedback of a history. -/
+abbrev Hist.mapFeedback (f : 𝓨 → 𝓨') {n : ℕ} (h : Hist 𝓞 𝓐 𝓨 n) : Hist 𝓞 𝓐 𝓨' n :=
+  Hist.map id id f h
+
+end Map
+
+section Comap
+
+/-- The algorithm with observations in `𝓞'` and feedbacks in `𝓨'` obtained from
+`alg : Algorithm 𝓞 𝓐 𝓨` by transforming the pair (past rounds, current observation) by `F n` at
+each round `n` before applying the policy of `alg`.
+
+This is the primitive transport operation on algorithms: `Algorithm.comapObs` and
+`Algorithm.comapFeedback` are the special cases in which `F n` is a round-wise map of the
+observation and of the feedback. -/
+def Algorithm.comap (alg : Algorithm 𝓞 𝓐 𝓨)
+    (F : (n : ℕ) → Hist 𝓞' 𝓐 𝓨' n × 𝓞' → Hist 𝓞 𝓐 𝓨 n × 𝓞) (hF : ∀ n, Measurable (F n)) :
+    Algorithm 𝓞' 𝓐 𝓨' where
+  policy n := (alg.policy n).comap (F n) (hF n)
+
+@[simp]
+lemma Algorithm.policy_comap (alg : Algorithm 𝓞 𝓐 𝓨)
+    {F : (n : ℕ) → Hist 𝓞' 𝓐 𝓨' n × 𝓞' → Hist 𝓞 𝓐 𝓨 n × 𝓞} (hF : ∀ n, Measurable (F n)) (n : ℕ) :
+    (alg.comap F hF).policy n = (alg.policy n).comap (F n) (hF n) := rfl
+
+@[simp]
+lemma Algorithm.p0_comap (alg : Algorithm 𝓞 𝓐 𝓨)
+    {F : (n : ℕ) → Hist 𝓞' 𝓐 𝓨' n × 𝓞' → Hist 𝓞 𝓐 𝓨 n × 𝓞} (hF : ∀ n, Measurable (F n)) :
+    (alg.comap F hF).p0
+      = alg.p0.comap (fun o ↦ (F 0 (default, o)).2) (((hF 0).comp measurable_prodMk_left).snd) := by
+  ext o : 1
+  rw [p0_apply, policy_comap, Kernel.comap_apply, alg.policy_zero, Kernel.comap_apply]
+
+@[simp]
+lemma Algorithm.comap_id (alg : Algorithm 𝓞 𝓐 𝓨) :
+    alg.comap (fun _ ↦ id) (fun _ ↦ measurable_id) = alg := rfl
+
+lemma Algorithm.comap_comap (alg : Algorithm 𝓞 𝓐 𝓨)
+    {F : (n : ℕ) → Hist 𝓞' 𝓐 𝓨' n × 𝓞' → Hist 𝓞 𝓐 𝓨 n × 𝓞} (hF : ∀ n, Measurable (F n))
+    {G : (n : ℕ) → Hist 𝓞'' 𝓐 𝓨'' n × 𝓞'' → Hist 𝓞' 𝓐 𝓨' n × 𝓞'} (hG : ∀ n, Measurable (G n)) :
+    (alg.comap F hF).comap G hG =
+      alg.comap (fun n ↦ F n ∘ G n) fun n ↦ (hF n).comp (hG n) := rfl
+
+section ComapObs
+
+variable {f : 𝓞' → 𝓞}
+
+/-- The algorithm that sees `f o` when the observation is `o`, both in the current round and in the
+past rounds. -/
+def Algorithm.comapObs (alg : Algorithm 𝓞 𝓐 𝓨) (f : 𝓞' → 𝓞)
+    (hf : Measurable f := by fun_prop) : Algorithm 𝓞' 𝓐 𝓨 :=
+  alg.comap (fun _ p ↦ (Hist.mapObs f p.1, f p.2)) fun _ ↦ by fun_prop
+
+lemma Algorithm.comapObs_def (alg : Algorithm 𝓞 𝓐 𝓨) (hf : Measurable f) :
+    alg.comapObs f hf = alg.comap (fun _ p ↦ (Hist.mapObs f p.1, f p.2)) fun _ ↦ by fun_prop := rfl
+
+@[simp]
+lemma Algorithm.policy_comapObs (alg : Algorithm 𝓞 𝓐 𝓨) (hf : Measurable f) (n : ℕ) :
+    (alg.comapObs f hf).policy n
+      = (alg.policy n).comap (fun p ↦ (Hist.mapObs f p.1, f p.2)) (by fun_prop) := rfl
+
+@[simp]
+lemma Algorithm.p0_comapObs (alg : Algorithm 𝓞 𝓐 𝓨) (hf : Measurable f) :
+    (alg.comapObs f hf).p0 = alg.p0.comap f hf := by
+  ext o : 1
+  rw [p0_apply, policy_comapObs, Kernel.comap_apply, alg.policy_zero, Kernel.comap_apply]
+
+@[simp]
+lemma Algorithm.comapObs_id (alg : Algorithm 𝓞 𝓐 𝓨) : alg.comapObs id measurable_id = alg := rfl
+
+lemma Algorithm.comapObs_comapObs (alg : Algorithm 𝓞 𝓐 𝓨) (hf : Measurable f)
+    {g : 𝓞'' → 𝓞'} (hg : Measurable g) :
+    (alg.comapObs f hf).comapObs g hg = alg.comapObs (f ∘ g) (hf.comp hg) := rfl
+
+end ComapObs
+
+section ComapFeedback
+
+variable {g : 𝓨' → 𝓨}
+
+/-- The algorithm that sees `g y` when the feedback of a past round is `y`. -/
+def Algorithm.comapFeedback (alg : Algorithm 𝓞 𝓐 𝓨) (g : 𝓨' → 𝓨)
+    (hg : Measurable g := by fun_prop) : Algorithm 𝓞 𝓐 𝓨' :=
+  alg.comap (fun _ p ↦ (Hist.mapFeedback g p.1, p.2)) fun _ ↦ by fun_prop
+
+lemma Algorithm.comapFeedback_def (alg : Algorithm 𝓞 𝓐 𝓨) (hg : Measurable g) :
+    alg.comapFeedback g hg
+      = alg.comap (fun _ p ↦ (Hist.mapFeedback g p.1, p.2)) fun _ ↦ by fun_prop := rfl
+
+@[simp]
+lemma Algorithm.policy_comapFeedback (alg : Algorithm 𝓞 𝓐 𝓨) (hg : Measurable g) (n : ℕ) :
+    (alg.comapFeedback g hg).policy n
+      = (alg.policy n).comap (fun p ↦ (Hist.mapFeedback g p.1, p.2)) (by fun_prop) := rfl
+
+@[simp]
+lemma Algorithm.p0_comapFeedback (alg : Algorithm 𝓞 𝓐 𝓨) (hg : Measurable g) :
+    (alg.comapFeedback g hg).p0 = alg.p0 := by
+  ext o : 1
+  rw [p0_apply, policy_comapFeedback, Kernel.comap_apply, alg.policy_zero, p0_apply]
+
+@[simp]
+lemma Algorithm.comapFeedback_id (alg : Algorithm 𝓞 𝓐 𝓨) :
+    alg.comapFeedback id measurable_id = alg := rfl
+
+lemma Algorithm.comapFeedback_comapFeedback (alg : Algorithm 𝓞 𝓐 𝓨)
+    {g : 𝓨' → 𝓨} (hg : Measurable g) {g' : 𝓨'' → 𝓨'} (hg' : Measurable g') :
+    (alg.comapFeedback g hg).comapFeedback g' hg' = alg.comapFeedback (g ∘ g') (hg.comp hg') := rfl
+
+end ComapFeedback
+
+/-- Transporting the observations and the feedbacks of an algorithm are independent operations. -/
+lemma Algorithm.comapObs_comapFeedback_comm (alg : Algorithm 𝓞 𝓐 𝓨)
+    {f : 𝓞' → 𝓞} (hf : Measurable f) {g : 𝓨' → 𝓨} (hg : Measurable g) :
+    (alg.comapFeedback g hg).comapObs f hf = (alg.comapObs f hf).comapFeedback g hg := rfl
+
+section ComapAction
+
+variable {F : (n : ℕ) → Hist 𝓞 𝓐' 𝓨 n → Hist 𝓞 𝓐 𝓨 n} {f : 𝓐' → 𝓐}
+
+/-- The environment that reads the summary `F n` of the past rounds and reads `f a` when the
+algorithm plays `a` in the current round.
+
+This is the primitive transport operation on environments, dual to `Algorithm.comap`:
+`Environment.comapAction` is the special case in which `F n` is the round-wise map of the actions.
+Only the action can change type, since the observations and the feedbacks are outputs of the
+environment; `F n` can nonetheless forget or summarize the past rounds, as an environment that
+reads only the last round does. -/
+def Environment.comap (env : Environment 𝓞 𝓐 𝓨)
+    (F : (n : ℕ) → Hist 𝓞 𝓐' 𝓨 n → Hist 𝓞 𝓐 𝓨 n) (hF : ∀ n, Measurable (F n))
+    (f : 𝓐' → 𝓐) (hf : Measurable f) : Environment 𝓞 𝓐' 𝓨 where
+  obs n := (env.obs n).comap (F n) (hF n)
+  feedback n := (env.feedback n).comap (fun p ↦ ((F n p.1.1, p.1.2), f p.2)) (by fun_prop)
+
+@[simp]
+lemma Environment.obs_comap (env : Environment 𝓞 𝓐 𝓨)
+    (hF : ∀ n, Measurable (F n)) (hf : Measurable f) (n : ℕ) :
+    (env.comap F hF f hf).obs n = (env.obs n).comap (F n) (hF n) := rfl
+
+@[simp]
+lemma Environment.feedback_comap (env : Environment 𝓞 𝓐 𝓨) (hF : ∀ n, Measurable (F n))
+    (hf : Measurable f) (n : ℕ) :
+    (env.comap F hF f hf).feedback n
+      = (env.feedback n).comap (fun p ↦ ((F n p.1.1, p.1.2), f p.2)) (by fun_prop) := rfl
+
+@[simp]
+lemma Environment.obs0_comap (env : Environment 𝓞 𝓐 𝓨) (hF : ∀ n, Measurable (F n))
+    (hf : Measurable f) :
+    (env.comap F hF f hf).obs0 = env.obs0 := by
+  rw [Environment.obs0_def, obs_comap, Kernel.comap_apply, env.obs_zero]
+
+@[simp]
+lemma Environment.ν0_comap (env : Environment 𝓞 𝓐 𝓨) (hF : ∀ n, Measurable (F n))
+    (hf : Measurable f) :
+    (env.comap F hF f hf).ν0 = env.ν0.comap (fun p ↦ (p.1, f p.2)) (by fun_prop) := by
+  ext p : 1
+  rw [Environment.ν0_apply, feedback_comap, Kernel.comap_apply, env.feedback_zero,
+    Kernel.comap_apply]
+
+@[simp]
+lemma Environment.comap_id (env : Environment 𝓞 𝓐 𝓨) :
+    env.comap (fun _ ↦ id) (fun _ ↦ measurable_id) id measurable_id = env := rfl
+
+lemma Environment.comap_comp (env : Environment 𝓞 𝓐 𝓨) (hF : ∀ n, Measurable (F n))
+    (hf : Measurable f)
+    {G : (n : ℕ) → Hist 𝓞 𝓐'' 𝓨 n → Hist 𝓞 𝓐' 𝓨 n} (hG : ∀ n, Measurable (G n))
+    {g : 𝓐'' → 𝓐'} (hg : Measurable g) :
+    (env.comap F hF f hf).comap G hG g hg
+      = env.comap (fun n ↦ F n ∘ G n) (fun n ↦ (hF n).comp (hG n)) (f ∘ g) (hf.comp hg) := rfl
+
+/-- The environment that reads `f a` when the algorithm plays `a`, both in the current round and in
+the past rounds. -/
+def Environment.comapAction (env : Environment 𝓞 𝓐 𝓨) (f : 𝓐' → 𝓐)
+    (hf : Measurable f := by fun_prop) : Environment 𝓞 𝓐' 𝓨 :=
+  env.comap (fun _ ↦ Hist.mapAction f) (fun _ ↦ by fun_prop) f hf
+
+lemma Environment.comapAction_def (env : Environment 𝓞 𝓐 𝓨) (f : 𝓐' → 𝓐) (hf : Measurable f) :
+    env.comapAction f hf = env.comap (fun _ ↦ Hist.mapAction f) (fun _ ↦ by fun_prop) f hf := rfl
+
+@[simp]
+lemma Environment.obs_comapAction (env : Environment 𝓞 𝓐 𝓨) (hf : Measurable f)
+    (n : ℕ) :
+    (env.comapAction f hf).obs n = (env.obs n).comap (Hist.mapAction f) (by fun_prop) := rfl
+
+@[simp]
+lemma Environment.feedback_comapAction (env : Environment 𝓞 𝓐 𝓨) (hf : Measurable f) (n : ℕ) :
+    (env.comapAction f hf).feedback n = (env.feedback n).comap
+      (fun p ↦ ((Hist.mapAction f p.1.1, p.1.2), f p.2)) (by fun_prop) := rfl
+
+@[simp]
+lemma Environment.obs0_comapAction (env : Environment 𝓞 𝓐 𝓨) (hf : Measurable f) :
+    (env.comapAction f hf).obs0 = env.obs0 := by
+  rw [Environment.obs0_def, obs_comapAction, Kernel.comap_apply, env.obs_zero]
+
+@[simp]
+lemma Environment.ν0_comapAction (env : Environment 𝓞 𝓐 𝓨) (hf : Measurable f) :
+    (env.comapAction f hf).ν0 = env.ν0.comap (fun p ↦ (p.1, f p.2)) (by fun_prop) := by
+  ext p : 1
+  rw [Environment.ν0_apply, feedback_comapAction, Kernel.comap_apply, env.feedback_zero,
+    Kernel.comap_apply]
+
+@[simp]
+lemma Environment.comapAction_id (env : Environment 𝓞 𝓐 𝓨) :
+    env.comapAction id measurable_id = env := rfl
+
+lemma Environment.comapAction_comp (env : Environment 𝓞 𝓐 𝓨) (hf : Measurable f)
+    (g : 𝓐'' → 𝓐') (hg : Measurable g) :
+    (env.comapAction f hf).comapAction g hg = env.comapAction (f ∘ g) (hf.comp hg) := rfl
+
+end ComapAction
+
+end Comap
+
+section Congr
+
+/-- Relabelling of the observations, the actions and the feedbacks of an algorithm along measurable
+equivalences. -/
+noncomputable def Algorithm.congr (alg : Algorithm 𝓞 𝓐 𝓨) (e𝓞 : 𝓞 ≃ᵐ 𝓞') (e𝓐 : 𝓐 ≃ᵐ 𝓐')
+    (e𝓨 : 𝓨 ≃ᵐ 𝓨') : Algorithm 𝓞' 𝓐' 𝓨' where
+  policy n := ((alg.policy n).map e𝓐).comap
+    (fun p ↦ (Hist.map e𝓞.symm e𝓐.symm e𝓨.symm p.1, e𝓞.symm p.2)) (by fun_prop)
+  isMarkovKernel_policy n := by
+    have : IsMarkovKernel ((alg.policy n).map e𝓐) := Kernel.IsMarkovKernel.map _ e𝓐.measurable
+    infer_instance
+
+@[simp]
+lemma Algorithm.policy_congr (alg : Algorithm 𝓞 𝓐 𝓨) (e𝓞 : 𝓞 ≃ᵐ 𝓞') (e𝓐 : 𝓐 ≃ᵐ 𝓐')
+    (e𝓨 : 𝓨 ≃ᵐ 𝓨') (n : ℕ) :
+    (alg.congr e𝓞 e𝓐 e𝓨).policy n = ((alg.policy n).map e𝓐).comap
+      (fun p ↦ (Hist.map e𝓞.symm e𝓐.symm e𝓨.symm p.1, e𝓞.symm p.2)) (by fun_prop) := rfl
+
+@[simp]
+lemma Algorithm.p0_congr (alg : Algorithm 𝓞 𝓐 𝓨) (e𝓞 : 𝓞 ≃ᵐ 𝓞') (e𝓐 : 𝓐 ≃ᵐ 𝓐')
+    (e𝓨 : 𝓨 ≃ᵐ 𝓨') :
+    (alg.congr e𝓞 e𝓐 e𝓨).p0 = (alg.p0.map e𝓐).comap e𝓞.symm e𝓞.symm.measurable := by
+  ext o : 1
+  rw [p0_apply, policy_congr, Kernel.comap_apply, Kernel.map_apply _ e𝓐.measurable,
+    alg.policy_zero, Kernel.comap_apply, Kernel.map_apply _ e𝓐.measurable]
+
+@[simp]
+lemma Algorithm.congr_refl (alg : Algorithm 𝓞 𝓐 𝓨) :
+    alg.congr (.refl 𝓞) (.refl 𝓐) (.refl 𝓨) = alg := by
+  ext n : 2
+  simp [MeasurableEquiv.symm_refl, MeasurableEquiv.coe_refl]
+
+lemma Algorithm.congr_congr (alg : Algorithm 𝓞 𝓐 𝓨) (e𝓞 : 𝓞 ≃ᵐ 𝓞') (e𝓐 : 𝓐 ≃ᵐ 𝓐')
+    (e𝓨 : 𝓨 ≃ᵐ 𝓨') (f𝓞 : 𝓞' ≃ᵐ 𝓞'') (f𝓐 : 𝓐' ≃ᵐ 𝓐'') (f𝓨 : 𝓨' ≃ᵐ 𝓨'') :
+    (alg.congr e𝓞 e𝓐 e𝓨).congr f𝓞 f𝓐 f𝓨
+      = alg.congr (e𝓞.trans f𝓞) (e𝓐.trans f𝓐) (e𝓨.trans f𝓨) := by
+  ext n : 2
+  ext p : 1
+  rw [policy_congr, Kernel.comap_apply, Kernel.map_apply _ f𝓐.measurable, policy_congr,
+    Kernel.comap_apply, Kernel.map_apply _ e𝓐.measurable,
+    Measure.map_map f𝓐.measurable e𝓐.measurable, policy_congr, Kernel.comap_apply,
+    Kernel.map_apply _ (e𝓐.trans f𝓐).measurable]
+  rfl
+
+@[simp]
+lemma Algorithm.congr_symm (alg : Algorithm 𝓞 𝓐 𝓨) (e𝓞 : 𝓞 ≃ᵐ 𝓞') (e𝓐 : 𝓐 ≃ᵐ 𝓐')
+    (e𝓨 : 𝓨 ≃ᵐ 𝓨') :
+    (alg.congr e𝓞 e𝓐 e𝓨).congr e𝓞.symm e𝓐.symm e𝓨.symm = alg := by
+  rw [congr_congr, MeasurableEquiv.self_trans_symm, MeasurableEquiv.self_trans_symm,
+    MeasurableEquiv.self_trans_symm, congr_refl]
+
+/-- Relabelling of the observations, the actions and the feedbacks of an environment along
+measurable equivalences. See also `Algorithm.congr`. -/
+noncomputable def Environment.congr (env : Environment 𝓞 𝓐 𝓨) (e𝓞 : 𝓞 ≃ᵐ 𝓞') (e𝓐 : 𝓐 ≃ᵐ 𝓐')
+    (e𝓨 : 𝓨 ≃ᵐ 𝓨') :
+    Environment 𝓞' 𝓐' 𝓨' where
+  obs n := ((env.obs n).map e𝓞).comap (Hist.map e𝓞.symm e𝓐.symm e𝓨.symm) (by fun_prop)
+  feedback n := ((env.feedback n).map e𝓨).comap
+    (fun p ↦ ((Hist.map e𝓞.symm e𝓐.symm e𝓨.symm p.1.1, e𝓞.symm p.1.2), e𝓐.symm p.2)) (by fun_prop)
+  isMarkovKernel_obs n := by
+    have : IsMarkovKernel ((env.obs n).map e𝓞) := Kernel.IsMarkovKernel.map _ e𝓞.measurable
+    infer_instance
+  isMarkovKernel_feedback n := by
+    have : IsMarkovKernel ((env.feedback n).map e𝓨) := Kernel.IsMarkovKernel.map _ e𝓨.measurable
+    infer_instance
+
+@[simp]
+lemma Environment.obs_congr (env : Environment 𝓞 𝓐 𝓨) (e𝓞 : 𝓞 ≃ᵐ 𝓞') (e𝓐 : 𝓐 ≃ᵐ 𝓐')
+    (e𝓨 : 𝓨 ≃ᵐ 𝓨') (n : ℕ) :
+    (env.congr e𝓞 e𝓐 e𝓨).obs n
+      = ((env.obs n).map e𝓞).comap (Hist.map e𝓞.symm e𝓐.symm e𝓨.symm) (by fun_prop) := rfl
+
+@[simp]
+lemma Environment.feedback_congr (env : Environment 𝓞 𝓐 𝓨) (e𝓞 : 𝓞 ≃ᵐ 𝓞') (e𝓐 : 𝓐 ≃ᵐ 𝓐')
+    (e𝓨 : 𝓨 ≃ᵐ 𝓨') (n : ℕ) :
+    (env.congr e𝓞 e𝓐 e𝓨).feedback n = ((env.feedback n).map e𝓨).comap
+      (fun p ↦ ((Hist.map e𝓞.symm e𝓐.symm e𝓨.symm p.1.1, e𝓞.symm p.1.2), e𝓐.symm p.2))
+      (by fun_prop) := rfl
+
+@[simp]
+lemma Environment.obs0_congr (env : Environment 𝓞 𝓐 𝓨) (e𝓞 : 𝓞 ≃ᵐ 𝓞') (e𝓐 : 𝓐 ≃ᵐ 𝓐')
+    (e𝓨 : 𝓨 ≃ᵐ 𝓨') :
+    (env.congr e𝓞 e𝓐 e𝓨).obs0 = env.obs0.map e𝓞 := by
+  rw [Environment.obs0_def, obs_congr, Kernel.comap_apply, Kernel.map_apply _ e𝓞.measurable,
+    env.obs_zero]
+
+@[simp]
+lemma Environment.ν0_congr (env : Environment 𝓞 𝓐 𝓨) (e𝓞 : 𝓞 ≃ᵐ 𝓞') (e𝓐 : 𝓐 ≃ᵐ 𝓐')
+    (e𝓨 : 𝓨 ≃ᵐ 𝓨') :
+    (env.congr e𝓞 e𝓐 e𝓨).ν0
+      = (env.ν0.map e𝓨).comap (fun p ↦ (e𝓞.symm p.1, e𝓐.symm p.2)) (by fun_prop) := by
+  ext p : 1
+  rw [Environment.ν0_apply, feedback_congr, Kernel.comap_apply,
+    Kernel.map_apply _ e𝓨.measurable, env.feedback_zero, Kernel.comap_apply,
+    Kernel.map_apply _ e𝓨.measurable]
+
+@[simp]
+lemma Environment.congr_refl (env : Environment 𝓞 𝓐 𝓨) :
+    env.congr (.refl 𝓞) (.refl 𝓐) (.refl 𝓨) = env := by
+  ext n : 2
+  · simp [MeasurableEquiv.symm_refl, MeasurableEquiv.coe_refl]
+  · simp [MeasurableEquiv.symm_refl, MeasurableEquiv.coe_refl]
+
+lemma Environment.congr_congr (env : Environment 𝓞 𝓐 𝓨) (e𝓞 : 𝓞 ≃ᵐ 𝓞') (e𝓐 : 𝓐 ≃ᵐ 𝓐')
+    (e𝓨 : 𝓨 ≃ᵐ 𝓨') (f𝓞 : 𝓞' ≃ᵐ 𝓞'') (f𝓐 : 𝓐' ≃ᵐ 𝓐'') (f𝓨 : 𝓨' ≃ᵐ 𝓨'') :
+    (env.congr e𝓞 e𝓐 e𝓨).congr f𝓞 f𝓐 f𝓨
+      = env.congr (e𝓞.trans f𝓞) (e𝓐.trans f𝓐) (e𝓨.trans f𝓨) := by
+  ext n : 2
+  · ext p : 1
+    rw [obs_congr, Kernel.comap_apply, Kernel.map_apply _ f𝓞.measurable, obs_congr,
+      Kernel.comap_apply, Kernel.map_apply _ e𝓞.measurable,
+      Measure.map_map f𝓞.measurable e𝓞.measurable, obs_congr, Kernel.comap_apply,
+      Kernel.map_apply _ (e𝓞.trans f𝓞).measurable]
+    rfl
+  · ext p : 1
+    rw [feedback_congr, Kernel.comap_apply, Kernel.map_apply _ f𝓨.measurable, feedback_congr,
+      Kernel.comap_apply, Kernel.map_apply _ e𝓨.measurable,
+      Measure.map_map f𝓨.measurable e𝓨.measurable, feedback_congr, Kernel.comap_apply,
+      Kernel.map_apply _ (e𝓨.trans f𝓨).measurable]
+    rfl
+
+@[simp]
+lemma Environment.congr_symm (env : Environment 𝓞 𝓐 𝓨) (e𝓞 : 𝓞 ≃ᵐ 𝓞') (e𝓐 : 𝓐 ≃ᵐ 𝓐')
+    (e𝓨 : 𝓨 ≃ᵐ 𝓨') :
+    (env.congr e𝓞 e𝓐 e𝓨).congr e𝓞.symm e𝓐.symm e𝓨.symm = env := by
+  rw [congr_congr, MeasurableEquiv.self_trans_symm, MeasurableEquiv.self_trans_symm,
+    MeasurableEquiv.self_trans_symm, congr_refl]
+
+end Congr
+
+section Runs
+
+variable {alg : Algorithm 𝓞 𝓐 𝓨} {P : Measure Ω} [IsFiniteMeasure P]
+  {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
+
+namespace IsAlgEnvSeq
+
+section ComapObs
+
+variable {env : Environment 𝓞' 𝓐 𝓨} {f : 𝓞' → 𝓞} {hf : Measurable f} {O : ℕ → Ω → 𝓞'}
+
+/-- The algorithm does not use the part of the observation that it ignores: the conditional
+distribution of its action given the transported history and observation is its own policy. -/
+lemma hasCondDistrib_action_comapObs (h : IsAlgEnvSeq O A Y (alg.comapObs f hf) env P) (n : ℕ) :
+    HasCondDistrib (A n)
+      (fun ω ↦ (history (fun n ω ↦ f (O n ω)) A Y n ω, f (O n ω))) (alg.policy n) P :=
+  HasCondDistrib.comp_right (f := fun p : Hist 𝓞' 𝓐 𝓨 n × 𝓞' ↦ (Hist.mapObs f p.1, f p.2))
+    (hf := by fun_prop) (h.hasCondDistrib_action n)
+
+end ComapObs
+
+section ComapFeedback
+
+variable {env : Environment 𝓞 𝓐 𝓨'} {g : 𝓨' → 𝓨} {hg : Measurable g} {O : ℕ → Ω → 𝓞}
+  {Y' : ℕ → Ω → 𝓨'}
+
+/-- The algorithm does not use the part of the past feedbacks that it ignores: the conditional
+distribution of its action given the transported history and the observation is its own policy. -/
+lemma hasCondDistrib_action_comapFeedback
+    (h : IsAlgEnvSeq O A Y' (alg.comapFeedback g hg) env P) (n : ℕ) :
+    HasCondDistrib (A n)
+      (fun ω ↦ (history O A (fun n ω ↦ g (Y' n ω)) n ω, O n ω)) (alg.policy n) P :=
+  HasCondDistrib.comp_right (f := fun p : Hist 𝓞 𝓐 𝓨' n × 𝓞 ↦ (Hist.mapFeedback g p.1, p.2))
+    (hf := by fun_prop) (h.hasCondDistrib_action n)
+
+end ComapFeedback
+
+section ComapAction
+
+variable {env : Environment 𝓞 𝓐 𝓨} {f : 𝓐' → 𝓐} {hf : Measurable f} {O : ℕ → Ω → 𝓞}
+  {A' : ℕ → Ω → 𝓐'}
+
+/-- The environment does not use the part of the action that it ignores: the conditional
+distribution of the observation given the transported history is its own observation kernel. -/
+lemma hasCondDistrib_obs_comapAction {alg : Algorithm 𝓞 𝓐' 𝓨}
+    (h : IsAlgEnvSeq O A' Y alg (env.comapAction f hf) P) (n : ℕ) :
+    HasCondDistrib (O n) (history O (fun n ω ↦ f (A' n ω)) Y n) (env.obs n) P :=
+  HasCondDistrib.comp_right (f := Hist.mapAction (𝓞 := 𝓞) (𝓨 := 𝓨) f (n := n))
+    (hf := by fun_prop) (h.hasCondDistrib_obs n)
+
+/-- The environment does not use the part of the action that it ignores: the conditional
+distribution of the feedback given the transported history, the observation and the transported
+action is its own feedback kernel. -/
+lemma hasCondDistrib_feedback_comapAction {alg : Algorithm 𝓞 𝓐' 𝓨}
+    (h : IsAlgEnvSeq O A' Y alg (env.comapAction f hf) P) (n : ℕ) :
+    HasCondDistrib (Y n)
+      (fun ω ↦ ((history O (fun n ω ↦ f (A' n ω)) Y n ω, O n ω), f (A' n ω))) (env.feedback n) P :=
+  HasCondDistrib.comp_right
+    (f := fun p : (Hist 𝓞 𝓐' 𝓨 n × 𝓞) × 𝓐' ↦ ((Hist.mapAction f p.1.1, p.1.2), f p.2))
+    (hf := by fun_prop) (h.hasCondDistrib_feedback n)
+
+end ComapAction
+
+end IsAlgEnvSeq
+
+end Runs
+
+end Learning
diff --git a/LeanMachineLearning/SequentialLearning/Deterministic.lean b/LeanMachineLearning/SequentialLearning/Deterministic.lean
index 913659ce..66784d36 100644
--- a/LeanMachineLearning/SequentialLearning/Deterministic.lean
+++ b/LeanMachineLearning/SequentialLearning/Deterministic.lean
@@ -12,8 +12,8 @@ public import LeanMachineLearning.SequentialLearning.Algorithm
 # Deterministic algorithms and environments
 
 A deterministic algorithm chooses its action in a deterministic way. That is, that action is given
-by a measurable function of the history instead of a general Markov kernel.
-Similarly, a deterministic environment gives feedback in a deterministic way.
+by a measurable function of the history and of the current observation instead of a general Markov
+kernel. Similarly, a deterministic environment gives feedback in a deterministic way.
 
 ## Main definitions
 
@@ -26,19 +26,19 @@ measurable functions.
 * `IsDeterministicAlg alg`: a typeclass expressing that the algorithm `alg` is deterministic.
 * `IsDeterministicEnv env`: a typeclass expressing that the environment `env` is deterministic.
 * `nextAction alg n`: the function that gives the action of a deterministic algorithm `alg`
-  at step `n`, as a function of the history before `n`.
-* `actionZero alg`: the initial action of a deterministic algorithm `alg`. This is
-  `nextAction alg 0` applied to the empty history.
+  at step `n`, as a function of the history before `n` and of the observation at step `n`.
+* `actionZero alg`: the initial action of a deterministic algorithm `alg`, as a function of the
+  first observation. This is `nextAction alg 0` applied to the empty history.
 * `feedbackFun env n`: the function that gives the feedback of a deterministic environment `env`
-  at step `n`, as a function of the history and the current action.
+  at step `n`, as a function of the history, the current observation and the current action.
 * `feedbackFunZero env`: the function that gives the initial feedback of a deterministic
   environment `env`. This is `feedbackFun env 0` applied to the empty history.
 
 * `detAlgorithm nextA h_next`: a deterministic algorithm that chooses its action
   according to the measurable function `nextA` (with proof of measurability `h_next`).
-  The initial action is `nextA 0 default`.
-* `detEnvironment f hf`: a deterministic environment that gives feedback according to the
-  measurable function `f` (with proof of measurability `hf`).
+  The initial action is `fun o ↦ nextA 0 (default, o)`.
+* `detEnvironment obs f hf`: a deterministic environment with observation kernels `obs`, that gives
+  feedback according to the measurable function `f` (with proof of measurability `hf`).
 
 -/
 
@@ -50,159 +50,177 @@ open scoped ENNReal NNReal
 
 namespace Learning
 
-variable {𝓐 𝓨 : Type*} {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨}
+variable {𝓞 𝓐 𝓨 : Type*} {m𝓞 : MeasurableSpace 𝓞} {m𝓐 : MeasurableSpace 𝓐}
+  {m𝓨 : MeasurableSpace 𝓨}
 
 /-- An algorithm is deterministic if its actions are determined by measurable functions of the
-history (and not possibly random kernels). -/
-class IsDeterministicAlg (alg : Algorithm 𝓐 𝓨) : Prop where
-  exists_nextAction n : ∃ (nextAction : (Fin n → 𝓐 × 𝓨) → 𝓐) (h_meas : Measurable nextAction),
+history and of the current observation (and not possibly random kernels). -/
+class IsDeterministicAlg (alg : Algorithm 𝓞 𝓐 𝓨) : Prop where
+  exists_nextAction n : ∃ (nextAction : (Hist 𝓞 𝓐 𝓨 n × 𝓞) → 𝓐) (h_meas : Measurable nextAction),
     alg.policy n = Kernel.deterministic nextAction h_meas
 
-/-- The action of a deterministic algorithm at step `n`, as a function of the history before `n`. -/
+/-- The action of a deterministic algorithm at step `n`, as a function of the history before `n`
+and of the observation at step `n`. -/
 noncomputable
-def nextAction (alg : Algorithm 𝓐 𝓨) [h_det : IsDeterministicAlg alg] (n : ℕ) :
-    (Fin n → 𝓐 × 𝓨) → 𝓐 :=
+def nextAction (alg : Algorithm 𝓞 𝓐 𝓨) [h_det : IsDeterministicAlg alg] (n : ℕ) :
+    (Hist 𝓞 𝓐 𝓨 n × 𝓞) → 𝓐 :=
   (h_det.exists_nextAction n).choose
 
-/-- The initial action of a deterministic algorithm. -/
+/-- The initial action of a deterministic algorithm, as a function of the first observation. -/
 noncomputable
-def actionZero (alg : Algorithm 𝓐 𝓨) [IsDeterministicAlg alg] : 𝓐 :=
-  nextAction alg 0 default
+def actionZero (alg : Algorithm 𝓞 𝓐 𝓨) [IsDeterministicAlg alg] : 𝓞 → 𝓐 :=
+  fun o ↦ nextAction alg 0 (default, o)
 
 @[fun_prop]
-lemma measurable_nextAction (alg : Algorithm 𝓐 𝓨) [IsDeterministicAlg alg] (n : ℕ) :
+lemma measurable_nextAction (alg : Algorithm 𝓞 𝓐 𝓨) [IsDeterministicAlg alg] (n : ℕ) :
     Measurable (nextAction alg n) :=
   (IsDeterministicAlg.exists_nextAction n).choose_spec.choose
 
-lemma policy_eq_deterministic (alg : Algorithm 𝓐 𝓨) [h_det : IsDeterministicAlg alg] (n : ℕ) :
+@[fun_prop]
+lemma measurable_actionZero (alg : Algorithm 𝓞 𝓐 𝓨) [IsDeterministicAlg alg] :
+    Measurable (actionZero alg) :=
+  (measurable_nextAction alg 0).comp (measurable_const.prodMk measurable_id)
+
+lemma policy_eq_deterministic (alg : Algorithm 𝓞 𝓐 𝓨) [h_det : IsDeterministicAlg alg] (n : ℕ) :
     alg.policy n = Kernel.deterministic (nextAction alg n) (measurable_nextAction alg n) :=
   (IsDeterministicAlg.exists_nextAction n).choose_spec.choose_spec
 
-lemma nextAction_zero (alg : Algorithm 𝓐 𝓨) [IsDeterministicAlg alg] (h : Fin 0 → 𝓐 × 𝓨) :
-    nextAction alg 0 h = actionZero alg := by
+lemma nextAction_zero (alg : Algorithm 𝓞 𝓐 𝓨) [IsDeterministicAlg alg] (h : Hist 𝓞 𝓐 𝓨 0)
+    (o : 𝓞) :
+    nextAction alg 0 (h, o) = actionZero alg o := by
   rw [Unique.eq_default h]
   rfl
 
-lemma p0_eq_dirac (alg : Algorithm 𝓐 𝓨) [IsDeterministicAlg alg] :
-    alg.p0 = Measure.dirac (actionZero alg) := by
-  rw [Algorithm.p0_def, policy_eq_deterministic, Kernel.deterministic_apply]
+lemma p0_eq_deterministic (alg : Algorithm 𝓞 𝓐 𝓨) [IsDeterministicAlg alg] :
+    alg.p0 = Kernel.deterministic (actionZero alg) (measurable_actionZero alg) := by
+  ext o : 1
+  rw [Algorithm.p0_apply, policy_eq_deterministic, Kernel.deterministic_apply,
+    Kernel.deterministic_apply]
   rfl
 
 namespace IsDeterministicAlg
 
 variable {Ω : Type*} {mΩ : MeasurableSpace Ω}
-  {alg : Algorithm 𝓐 𝓨} {env : Environment 𝓐 𝓨} {P : Measure Ω} [IsFiniteMeasure P]
-  {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {n N : ℕ}
+  {alg : Algorithm 𝓞 𝓐 𝓨} {env : Environment 𝓞 𝓐 𝓨} {P : Measure Ω} [IsFiniteMeasure P]
+  {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {n N : ℕ}
 
 lemma action_ae_eq_of_IsAlgEnvSeqUntil [MeasurableEq 𝓐]
-    [h_det : IsDeterministicAlg alg] (h : IsAlgEnvSeqUntil A Y alg env P N) (hn : n < N) :
-    A n =ᵐ[P] fun ω ↦ nextAction alg n (history A Y n ω) := by
+    [h_det : IsDeterministicAlg alg] (h : IsAlgEnvSeqUntil O A Y alg env P N) (hn : n < N) :
+    A n =ᵐ[P] fun ω ↦ nextAction alg n (history O A Y n ω, O n ω) := by
   have h_eq := (h.hasCondDistrib_action n hn)
   rw [policy_eq_deterministic alg n] at h_eq
+  have hO := h.measurable_obs
+  have hA := h.measurable_action
+  have hY := h.measurable_feedback
   exact ae_eq_of_hasCondDistrib_deterministic (measurable_nextAction _ _) (by fun_prop)
     (by fun_prop) h_eq
 
 lemma action_zero_of_IsAlgEnvSeqUntil [MeasurableEq 𝓐] [h_det : IsDeterministicAlg alg]
-    (h : IsAlgEnvSeqUntil A Y alg env P N) (hN : 0 < N) :
-    A 0 =ᵐ[P] fun _ ↦ actionZero alg := by
+    (h : IsAlgEnvSeqUntil O A Y alg env P N) (hN : 0 < N) :
+    A 0 =ᵐ[P] fun ω ↦ actionZero alg (O 0 ω) := by
   filter_upwards [action_ae_eq_of_IsAlgEnvSeqUntil h hN] with ω hω
   rw [hω, nextAction_zero]
 
-lemma hasLaw_action_zero_of_IsAlgEnvSeqUntil [IsProbabilityMeasure P]
-    [h_det : IsDeterministicAlg alg] (h : IsAlgEnvSeqUntil A Y alg env P N) (hN : 0 < N) :
-    HasLaw (A 0) (Measure.dirac (actionZero alg)) P where
-  aemeasurable := have hA := h.measurable_action; by fun_prop
-  map_eq := (h.hasLaw_action_zero hN).map_eq.trans (p0_eq_dirac alg)
+lemma hasCondDistrib_action_zero_of_IsAlgEnvSeqUntil [h_det : IsDeterministicAlg alg]
+    (h : IsAlgEnvSeqUntil O A Y alg env P N) (hN : 0 < N) :
+    HasCondDistrib (A 0) (O 0)
+      (Kernel.deterministic (actionZero alg) (measurable_actionZero alg)) P := by
+  rw [← p0_eq_deterministic]
+  exact h.hasCondDistrib_action_zero hN
 
-lemma hasLaw_action_zero [IsProbabilityMeasure P] [h_det : IsDeterministicAlg alg]
-    (h : IsAlgEnvSeq A Y alg env P) :
-    HasLaw (A 0) (Measure.dirac (actionZero alg)) P :=
-  hasLaw_action_zero_of_IsAlgEnvSeqUntil (h.isAlgEnvSeqUntil 1) zero_lt_one
+lemma hasCondDistrib_action_zero [h_det : IsDeterministicAlg alg]
+    (h : IsAlgEnvSeq O A Y alg env P) :
+    HasCondDistrib (A 0) (O 0)
+      (Kernel.deterministic (actionZero alg) (measurable_actionZero alg)) P :=
+  hasCondDistrib_action_zero_of_IsAlgEnvSeqUntil (h.isAlgEnvSeqUntil 1) zero_lt_one
 
 lemma action_ae_eq [MeasurableEq 𝓐] [h_det : IsDeterministicAlg alg]
-    (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    A n =ᵐ[P] fun ω ↦ nextAction alg n (history A Y n ω) :=
+    (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    A n =ᵐ[P] fun ω ↦ nextAction alg n (history O A Y n ω, O n ω) :=
   action_ae_eq_of_IsAlgEnvSeqUntil (h.isAlgEnvSeqUntil (n + 1)) n.lt_succ_self
 
 lemma action_zero_ae_eq [MeasurableEq 𝓐] [h_det : IsDeterministicAlg alg]
-    (h : IsAlgEnvSeq A Y alg env P) :
-    A 0 =ᵐ[P] fun _ ↦ actionZero alg :=
+    (h : IsAlgEnvSeq O A Y alg env P) :
+    A 0 =ᵐ[P] fun ω ↦ actionZero alg (O 0 ω) :=
   action_zero_of_IsAlgEnvSeqUntil (h.isAlgEnvSeqUntil 1) zero_lt_one
 
 lemma action_ae_all_eq [MeasurableEq 𝓐] [h_det : IsDeterministicAlg alg]
-    (h : IsAlgEnvSeq A Y alg env P) :
-    ∀ᵐ ω ∂P, ∀ n, A n ω = nextAction alg n (history A Y n ω) :=
+    (h : IsAlgEnvSeq O A Y alg env P) :
+    ∀ᵐ ω ∂P, ∀ n, A n ω = nextAction alg n (history O A Y n ω, O n ω) :=
   ae_all_iff.mpr (action_ae_eq h)
 
 end IsDeterministicAlg
 
 /-- An environment is deterministic if its feedbacks are determined by measurable functions of
-the history and the action (and not possibly random kernels). -/
-class IsDeterministicEnv (env : Environment 𝓐 𝓨) : Prop where
-  exists_f : ∀ n, ∃ (f : ((Fin n → 𝓐 × 𝓨) × 𝓐) → 𝓨) (hf : Measurable f),
+the history, the observation and the action (and not possibly random kernels). -/
+class IsDeterministicEnv (env : Environment 𝓞 𝓐 𝓨) : Prop where
+  exists_f : ∀ n, ∃ (f : ((Hist 𝓞 𝓐 𝓨 n × 𝓞) × 𝓐) → 𝓨) (hf : Measurable f),
     env.feedback n = Kernel.deterministic f hf
 
 /-- The feedback function of a deterministic environment at step `n`. -/
 noncomputable
-def feedbackFun (env : Environment 𝓐 𝓨) [h_det : IsDeterministicEnv env] (n : ℕ) :
-    ((Fin n → 𝓐 × 𝓨) × 𝓐) → 𝓨 :=
+def feedbackFun (env : Environment 𝓞 𝓐 𝓨) [h_det : IsDeterministicEnv env] (n : ℕ) :
+    ((Hist 𝓞 𝓐 𝓨 n × 𝓞) × 𝓐) → 𝓨 :=
   (h_det.exists_f n).choose
 
 @[fun_prop]
-lemma measurable_feedbackFun (env : Environment 𝓐 𝓨) [IsDeterministicEnv env] (n : ℕ) :
+lemma measurable_feedbackFun (env : Environment 𝓞 𝓐 𝓨) [IsDeterministicEnv env] (n : ℕ) :
     Measurable (feedbackFun env n) :=
   (IsDeterministicEnv.exists_f n).choose_spec.choose
 
-lemma feedback_eq_deterministic (env : Environment 𝓐 𝓨) [IsDeterministicEnv env] (n : ℕ) :
+lemma feedback_eq_deterministic (env : Environment 𝓞 𝓐 𝓨) [IsDeterministicEnv env] (n : ℕ) :
     env.feedback n = Kernel.deterministic (feedbackFun env n) (measurable_feedbackFun env n) :=
   (IsDeterministicEnv.exists_f n).choose_spec.choose_spec
 
-/-- The initial feedback function of a deterministic environment. -/
+/-- The initial feedback function of a deterministic environment, as a function of the first
+observation and the first action. -/
 noncomputable
-def feedbackFunZero (env : Environment 𝓐 𝓨) [IsDeterministicEnv env] : 𝓐 → 𝓨 :=
-  fun a ↦ feedbackFun env 0 (default, a)
+def feedbackFunZero (env : Environment 𝓞 𝓐 𝓨) [IsDeterministicEnv env] : 𝓞 × 𝓐 → 𝓨 :=
+  fun p ↦ feedbackFun env 0 ((default, p.1), p.2)
 
 @[fun_prop]
-lemma measurable_feedbackFunZero (env : Environment 𝓐 𝓨) [IsDeterministicEnv env] :
+lemma measurable_feedbackFunZero (env : Environment 𝓞 𝓐 𝓨) [IsDeterministicEnv env] :
     Measurable (feedbackFunZero env) :=
-  (measurable_feedbackFun env 0).comp (measurable_const.prodMk measurable_id)
+  (measurable_feedbackFun env 0).comp
+    ((measurable_const.prodMk measurable_fst).prodMk measurable_snd)
 
-lemma feedbackFun_zero (env : Environment 𝓐 𝓨) [IsDeterministicEnv env] (h : Fin 0 → 𝓐 × 𝓨)
-    (a : 𝓐) :
-    feedbackFun env 0 (h, a) = feedbackFunZero env a := by
+lemma feedbackFun_zero (env : Environment 𝓞 𝓐 𝓨) [IsDeterministicEnv env] (h : Hist 𝓞 𝓐 𝓨 0)
+    (o : 𝓞) (a : 𝓐) :
+    feedbackFun env 0 ((h, o), a) = feedbackFunZero env (o, a) := by
   rw [Unique.eq_default h]
   rfl
 
-lemma ν0_eq_deterministic (env : Environment 𝓐 𝓨) [IsDeterministicEnv env] :
+lemma ν0_eq_deterministic (env : Environment 𝓞 𝓐 𝓨) [IsDeterministicEnv env] :
     env.ν0 = Kernel.deterministic (feedbackFunZero env) (measurable_feedbackFunZero env) := by
-  ext a : 1
-  rw [Environment.ν0_apply, feedback_eq_deterministic, Kernel.deterministic_apply,
+  ext p : 1
+  rw [Environment.ν0_def, Kernel.comap_apply, feedback_eq_deterministic, Kernel.deterministic_apply,
     Kernel.deterministic_apply]
   rfl
 
 namespace IsDeterministicEnv
 
 variable {Ω : Type*} {mΩ : MeasurableSpace Ω}
-  {alg : Algorithm 𝓐 𝓨} {env : Environment 𝓐 𝓨} {P : Measure Ω} [IsFiniteMeasure P]
-  {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
+  {alg : Algorithm 𝓞 𝓐 𝓨} {env : Environment 𝓞 𝓐 𝓨} {P : Measure Ω} [IsFiniteMeasure P]
+  {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
 
 lemma hasCondDistrib_feedback [h_det : IsDeterministicEnv env]
-    (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    HasCondDistrib (Y n) (fun ω ↦ (history A Y n ω, A n ω))
+    (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    HasCondDistrib (Y n) (fun ω ↦ ((history O A Y n ω, O n ω), A n ω))
       (Kernel.deterministic (feedbackFun env n) (measurable_feedbackFun env n)) P := by
   rw [← feedback_eq_deterministic]
   exact h.hasCondDistrib_feedback n
 
 lemma hasCondDistrib_feedback_zero [h_det : IsDeterministicEnv env]
-    (h : IsAlgEnvSeq A Y alg env P) :
-    HasCondDistrib (Y 0) (A 0)
+    (h : IsAlgEnvSeq O A Y alg env P) :
+    HasCondDistrib (Y 0) (fun ω ↦ (O 0 ω, A 0 ω))
       (Kernel.deterministic (feedbackFunZero env) (measurable_feedbackFunZero env)) P := by
   rw [← ν0_eq_deterministic]
   exact h.hasCondDistrib_feedback_zero
 
 lemma feedback_ae_eq [MeasurableEq 𝓨] [h_det : IsDeterministicEnv env]
-    (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    Y n =ᵐ[P] fun ω ↦ feedbackFun env n (history A Y n ω, A n ω) := by
+    (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    Y n =ᵐ[P] fun ω ↦ feedbackFun env n ((history O A Y n ω, O n ω), A n ω) := by
+  have hO := h.measurable_obs
   have hA := h.measurable_action
   have hY := h.measurable_feedback
   exact ae_eq_of_hasCondDistrib_deterministic (measurable_feedbackFun _ _) (by fun_prop)
@@ -210,25 +228,31 @@ lemma feedback_ae_eq [MeasurableEq 𝓨] [h_det : IsDeterministicEnv env]
 
 end IsDeterministicEnv
 
-variable {nextA : (n : ℕ) → (Fin n → 𝓐 × 𝓨) → 𝓐} {h_next : ∀ n, Measurable (nextA n)}
-  {env : Environment 𝓐 𝓨}
-  {f : (n : ℕ) → ((Fin n → 𝓐 × 𝓨) × 𝓐) → 𝓨} {hf : ∀ n, Measurable (f n)}
+variable {nextA : (n : ℕ) → (Hist 𝓞 𝓐 𝓨 n × 𝓞) → 𝓐} {h_next : ∀ n, Measurable (nextA n)}
+  {env : Environment 𝓞 𝓐 𝓨}
+  {obs : (n : ℕ) → Kernel (Hist 𝓞 𝓐 𝓨 n) 𝓞} [∀ n, IsMarkovKernel (obs n)]
+  {f : (n : ℕ) → ((Hist 𝓞 𝓐 𝓨 n × 𝓞) × 𝓐) → 𝓨} {hf : ∀ n, Measurable (f n)}
 
 /-- A deterministic algorithm, which chooses the action given by the function `nextA`.
-The initial action is `nextA 0 default`. -/
+The initial action is `fun o ↦ nextA 0 (default, o)`. -/
 @[simps]
 noncomputable
-def detAlgorithm (nextA : (n : ℕ) → (Fin n → 𝓐 × 𝓨) → 𝓐)
+def detAlgorithm (nextA : (n : ℕ) → (Hist 𝓞 𝓐 𝓨 n × 𝓞) → 𝓐)
     (h_next : ∀ n, Measurable (nextA n)) :
-    Algorithm 𝓐 𝓨 where
+    Algorithm 𝓞 𝓐 𝓨 where
   policy n := Kernel.deterministic (nextA n) (h_next n)
 
 instance : IsDeterministicAlg (detAlgorithm nextA h_next) where
   exists_nextAction n := ⟨nextA n, h_next n, rfl⟩
 
 @[simp]
-lemma p0_detAlgorithm : (detAlgorithm nextA h_next).p0 = Measure.dirac (nextA 0 default) := by
-  rw [Algorithm.p0_def, detAlgorithm_policy, Kernel.deterministic_apply]
+lemma p0_detAlgorithm :
+    (detAlgorithm nextA h_next).p0
+      = Kernel.deterministic (fun o ↦ nextA 0 (default, o))
+        ((h_next 0).comp (measurable_const.prodMk measurable_id)) := by
+  ext o : 1
+  rw [Algorithm.p0_apply, detAlgorithm_policy, Kernel.deterministic_apply,
+    Kernel.deterministic_apply]
 
 @[simp]
 lemma nextAction_detAlgorithm [MeasurableSpace.SeparatesPoints 𝓐] (n : ℕ) :
@@ -238,54 +262,68 @@ lemma nextAction_detAlgorithm [MeasurableSpace.SeparatesPoints 𝓐] (n : ℕ) :
 
 @[simp]
 lemma actionZero_detAlgorithm [MeasurableSpace.SeparatesPoints 𝓐] :
-    actionZero (detAlgorithm nextA h_next) = nextA 0 default := by
-  rw [actionZero, nextAction_detAlgorithm]
+    actionZero (detAlgorithm nextA h_next) = fun o ↦ nextA 0 (default, o) := by
+  unfold actionZero
+  rw [nextAction_detAlgorithm]
 
 /-- A deterministic environment, where the feedback is given by evaluating
 fixed measurable functions. -/
-noncomputable def detEnvironment
-    (f : (n : ℕ) → ((Fin n → 𝓐 × 𝓨) × 𝓐) → 𝓨) (hf : ∀ n, Measurable (f n)) :
-    Environment 𝓐 𝓨 where
+noncomputable def detEnvironment (obs : (n : ℕ) → Kernel (Hist 𝓞 𝓐 𝓨 n) 𝓞)
+    [∀ n, IsMarkovKernel (obs n)]
+    (f : (n : ℕ) → ((Hist 𝓞 𝓐 𝓨 n × 𝓞) × 𝓐) → 𝓨) (hf : ∀ n, Measurable (f n)) :
+    Environment 𝓞 𝓐 𝓨 where
+  obs := obs
   feedback n := (Kernel.deterministic (f n) (hf n))
 
-instance : IsDeterministicEnv (detEnvironment f hf) where
+@[simp]
+lemma obs_detEnvironment (n : ℕ) : (detEnvironment obs f hf).obs n = obs n := rfl
+
+@[simp]
+lemma feedback_detEnvironment (n : ℕ) :
+    (detEnvironment obs f hf).feedback n = Kernel.deterministic (f n) (hf n) := rfl
+
+instance : IsDeterministicEnv (detEnvironment obs f hf) where
   exists_f n := ⟨f n, hf n, rfl⟩
 
 @[simp]
 lemma feedbackFun_detEnvironment [MeasurableSpace.SeparatesPoints 𝓨] (n : ℕ) :
-    feedbackFun (detEnvironment f hf) n = f n := by
-  simpa [detEnvironment] using (feedback_eq_deterministic (detEnvironment f hf) n).symm
+    feedbackFun (detEnvironment obs f hf) n = f n := by
+  simpa [detEnvironment] using (feedback_eq_deterministic (detEnvironment obs f hf) n).symm
 
 @[simp]
 lemma feedbackFunZero_detEnvironment [MeasurableSpace.SeparatesPoints 𝓨] :
-    feedbackFunZero (detEnvironment f hf) = fun a ↦ f 0 (default, a) := by
+    feedbackFunZero (detEnvironment obs f hf) = fun p ↦ f 0 ((default, p.1), p.2) := by
   unfold feedbackFunZero
   rw [feedbackFun_detEnvironment]
 
 namespace IsAlgEnvSeq
 
 variable {Ω : Type*} {mΩ : MeasurableSpace Ω}
-  {alg : Algorithm 𝓐 𝓨} {ν : Kernel 𝓐 𝓨} [IsMarkovKernel ν]
-  {P : Measure Ω} [IsProbabilityMeasure P] {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
-
-lemma hasLaw_action_zero_detAlgorithm [MeasurableEq 𝓐]
-    (h : IsAlgEnvSeq A Y (detAlgorithm nextA h_next) env P) :
-    HasLaw (A 0) (Measure.dirac (nextA 0 default)) P := by
-  simpa using IsDeterministicAlg.hasLaw_action_zero h
+  {alg : Algorithm 𝓞 𝓐 𝓨} {ν : Kernel (𝓞 × 𝓐) 𝓨} [IsMarkovKernel ν]
+  {P : Measure Ω} [IsProbabilityMeasure P]
+  {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
+
+lemma hasCondDistrib_action_zero_detAlgorithm
+    (h : IsAlgEnvSeq O A Y (detAlgorithm nextA h_next) env P) :
+    HasCondDistrib (A 0) (O 0)
+      (Kernel.deterministic (fun o ↦ nextA 0 (default, o))
+        ((h_next 0).comp (measurable_const.prodMk measurable_id))) P := by
+  rw [← p0_detAlgorithm]
+  exact h.hasCondDistrib_action_zero
 
 lemma action_detAlgorithm_ae_eq [MeasurableEq 𝓐]
-    (h : IsAlgEnvSeq A Y (detAlgorithm nextA h_next) env P) (n : ℕ) :
-    A n =ᵐ[P] fun ω ↦ nextA n (history A Y n ω) :=
+    (h : IsAlgEnvSeq O A Y (detAlgorithm nextA h_next) env P) (n : ℕ) :
+    A n =ᵐ[P] fun ω ↦ nextA n (history O A Y n ω, O n ω) :=
   (IsDeterministicAlg.action_ae_eq h n).trans (by simp)
 
 lemma action_zero_detAlgorithm [MeasurableEq 𝓐]
-    (h : IsAlgEnvSeq A Y (detAlgorithm nextA h_next) env P) :
-    A 0 =ᵐ[P] fun _ ↦ nextA 0 default :=
+    (h : IsAlgEnvSeq O A Y (detAlgorithm nextA h_next) env P) :
+    A 0 =ᵐ[P] fun ω ↦ nextA 0 (default, O 0 ω) :=
   (IsDeterministicAlg.action_zero_ae_eq h).trans (by simp)
 
 lemma action_detAlgorithm_ae_all_eq [MeasurableEq 𝓐]
-    (h : IsAlgEnvSeq A Y (detAlgorithm nextA h_next) env P) :
-    ∀ᵐ ω ∂P, ∀ n, A n ω = nextA n (history A Y n ω) :=
+    (h : IsAlgEnvSeq O A Y (detAlgorithm nextA h_next) env P) :
+    ∀ᵐ ω ∂P, ∀ n, A n ω = nextA n (history O A Y n ω, O n ω) :=
   ae_all_iff.mpr (action_detAlgorithm_ae_eq h)
 
 end IsAlgEnvSeq
@@ -293,22 +331,26 @@ end IsAlgEnvSeq
 namespace IsAlgEnvSeqUntil
 
 variable {Ω : Type*} {mΩ : MeasurableSpace Ω}
-  {alg : Algorithm 𝓐 𝓨} {ν : Kernel 𝓐 𝓨} [IsMarkovKernel ν]
-  {P : Measure Ω} [IsProbabilityMeasure P] {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {N n : ℕ}
-
-lemma hasLaw_action_zero_detAlgorithm [MeasurableEq 𝓐]
-    (h : IsAlgEnvSeqUntil A Y (detAlgorithm nextA h_next) env P N) (hN : 0 < N) :
-    HasLaw (A 0) (Measure.dirac (nextA 0 default)) P := by
-  simpa using IsDeterministicAlg.hasLaw_action_zero_of_IsAlgEnvSeqUntil h hN
+  {alg : Algorithm 𝓞 𝓐 𝓨} {ν : Kernel (𝓞 × 𝓐) 𝓨} [IsMarkovKernel ν]
+  {P : Measure Ω} [IsProbabilityMeasure P]
+  {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {N n : ℕ}
+
+lemma hasCondDistrib_action_zero_detAlgorithm
+    (h : IsAlgEnvSeqUntil O A Y (detAlgorithm nextA h_next) env P N) (hN : 0 < N) :
+    HasCondDistrib (A 0) (O 0)
+      (Kernel.deterministic (fun o ↦ nextA 0 (default, o))
+        ((h_next 0).comp (measurable_const.prodMk measurable_id))) P := by
+  rw [← p0_detAlgorithm]
+  exact h.hasCondDistrib_action_zero hN
 
 lemma action_detAlgorithm_ae_eq [MeasurableEq 𝓐]
-    (h : IsAlgEnvSeqUntil A Y (detAlgorithm nextA h_next) env P N) (hn : n < N) :
-    A n =ᵐ[P] fun ω ↦ nextA n (history A Y n ω) :=
+    (h : IsAlgEnvSeqUntil O A Y (detAlgorithm nextA h_next) env P N) (hn : n < N) :
+    A n =ᵐ[P] fun ω ↦ nextA n (history O A Y n ω, O n ω) :=
   (IsDeterministicAlg.action_ae_eq_of_IsAlgEnvSeqUntil h hn).trans (by simp)
 
 lemma action_zero_detAlgorithm [MeasurableEq 𝓐]
-    (h : IsAlgEnvSeqUntil A Y (detAlgorithm nextA h_next) env P N) (hN : 0 < N) :
-    A 0 =ᵐ[P] fun _ ↦ nextA 0 default :=
+    (h : IsAlgEnvSeqUntil O A Y (detAlgorithm nextA h_next) env P N) (hN : 0 < N) :
+    A 0 =ᵐ[P] fun ω ↦ nextA 0 (default, O 0 ω) :=
   (IsDeterministicAlg.action_zero_of_IsAlgEnvSeqUntil h hN).trans (by simp)
 
 end IsAlgEnvSeqUntil
diff --git a/LeanMachineLearning/SequentialLearning/DivergenceDecomposition.lean b/LeanMachineLearning/SequentialLearning/DivergenceDecomposition.lean
index 4cb2be59..a45faebd 100644
--- a/LeanMachineLearning/SequentialLearning/DivergenceDecomposition.lean
+++ b/LeanMachineLearning/SequentialLearning/DivergenceDecomposition.lean
@@ -15,19 +15,19 @@ public import LeanMachineLearning.SequentialLearning.StationaryEnv
 # The divergence decomposition
 
 Let `alg`, `alg'` be algorithms, `env`, `env'` be environments, and consider two
-algorithm-environment sequences `(X, Y)` and `(X', Y')` of `alg` against `env` and of `alg'`
+algorithm-environment sequences `(O, A, Y)` and `(O', A', Y')` of `alg` against `env` and of `alg'`
 against `env'`, on arbitrary probability spaces `(Ω, P)` and `(Ω', P')`. The Kullback-Leibler
 divergence between the laws of the histories of the first `M` rounds is the sum, over the rounds
 `t < M`, of the conditional divergences of the step at round `t` given the first `t` rounds.
 Note that both arguments of the conditional term use the law of the *first* history,
 so that term measures only how the two step kernels differ.
-The same identity holds for the whole trajectory `trajectory X Y : Ω → (ℕ → 𝓐 × 𝓨)`, with a series
-in place of the finite sum.
+The same identity holds for the whole trajectory `trajectory O A Y : Ω → (ℕ → Round 𝓞 𝓐 𝓨)`, with
+a series in place of the finite sum.
 
 For a single algorithm run against two stationary environments with reward kernels `κ` and
-`κ'`, the two step kernels share the policy and differ only in the reward kernel, so the
-conditional divergence of a step is the conditional divergence of the reward given the played
-action.
+`κ'`, the two step kernels share the observation kernel and the policy and differ only in the
+reward kernel, so the conditional divergence of a step is the conditional divergence of the reward
+given the played action.
 This is the *divergence decomposition* of bandit lower bounds.
 
 ## Main statements
@@ -49,17 +49,18 @@ open scoped ENNReal RealInnerProductSpace ENat
 
 namespace Learning
 
-variable {𝓐 𝓨 : Type*} {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨}
+variable {𝓞 𝓐 𝓨 : Type*} {m𝓞 : MeasurableSpace 𝓞} {m𝓐 : MeasurableSpace 𝓐}
+  {m𝓨 : MeasurableSpace 𝓨}
   {Ω Ω' : Type*} {mΩ : MeasurableSpace Ω} {mΩ' : MeasurableSpace Ω'}
   {P : Measure Ω} {P' : Measure Ω'} [IsProbabilityMeasure P] [IsProbabilityMeasure P']
-  {X : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {X' : ℕ → Ω' → 𝓐} {Y' : ℕ → Ω' → 𝓨}
-  {alg alg' : Algorithm 𝓐 𝓨} {env env' : Environment 𝓐 𝓨}
-  {κ κ' : Kernel 𝓐 𝓨} [IsMarkovKernel κ] [IsMarkovKernel κ']
+  {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
+  {O' : ℕ → Ω' → 𝓞} {A' : ℕ → Ω' → 𝓐} {Y' : ℕ → Ω' → 𝓨}
+  {alg alg' : Algorithm 𝓞 𝓐 𝓨} {env env' : Environment 𝓞 𝓐 𝓨}
 
 section
 
-variable {α β γ : Type*} {mα : MeasurableSpace α} {mβ : MeasurableSpace β}
-  {mγ : MeasurableSpace γ} {μ : Measure α} [IsFiniteMeasure μ]
+variable {α β γ δ : Type*} {mα : MeasurableSpace α} {mβ : MeasurableSpace β}
+  {mγ : MeasurableSpace γ} {mδ : MeasurableSpace δ} {μ : Measure α} [IsFiniteMeasure μ]
 
 /-- The divergence of one step of a policy/reward decomposition, in composition-product form:
 the policy `π` is shared and the reward kernels `κ`, `η` (which ignore the history) differ, so
@@ -74,21 +75,38 @@ lemma klDiv_compProd_compProd_prodMkLeft_eq_klDiv_comp_compProd (μ : Measure α
     Measure.compProd_assoc, ← Measure.snd_compProd, Measure.snd]
   exact klDiv_compProd_comap _ _ _ measurable_snd
 
+/-- The divergence of one step of an observation/policy/reward decomposition, in
+composition-product form: the observation kernel `o` and the policy `π` are shared and the reward
+kernels `κ`, `η` (which ignore the history and the observation) differ, so the divergence is the
+conditional divergence of the reward kernels given the played action, whose law is
+`π ∘ₘ (μ ⊗ₘ o)`. -/
+lemma klDiv_compProd_compProd_compProd_prodMkLeft_eq_klDiv_comp_compProd (μ : Measure α)
+    [IsFiniteMeasure μ] (o : Kernel α β) [IsMarkovKernel o] (π : Kernel (α × β) γ)
+    [IsMarkovKernel π] (κ η : Kernel γ δ) [IsFiniteKernel κ] [IsFiniteKernel η] :
+    klDiv (μ ⊗ₘ (o ⊗ₖ (π ⊗ₖ κ.prodMkLeft (α × β))))
+        (μ ⊗ₘ (o ⊗ₖ (π ⊗ₖ η.prodMkLeft (α × β)))) =
+      klDiv ((π ∘ₘ (μ ⊗ₘ o)) ⊗ₘ κ) ((π ∘ₘ (μ ⊗ₘ o)) ⊗ₘ η) := by
+  rw [← klDiv_map_measurableEquiv _ _ MeasurableEquiv.prodAssoc.symm, Measure.compProd_assoc,
+    Measure.compProd_assoc]
+  exact klDiv_compProd_compProd_prodMkLeft_eq_klDiv_comp_compProd _ _ _ _
+
 end
 
 /-- **Chain rule for histories.** For two algorithms `alg`, `alg'` run against two environments
 `env`, `env'`, the divergence between the laws of the histories of the first `M` rounds is
 the sum over the rounds `t < M` of the conditional divergences of the step at round `t` given
 the first `t` rounds. -/
-lemma IsAlgEnvSeq.klDiv_map_history_stepKernel (h : IsAlgEnvSeq X Y alg env P)
-    (h' : IsAlgEnvSeq X' Y' alg' env' P') (M : ℕ) :
-    klDiv (P.map (history X Y M)) (P'.map (history X' Y' M)) =
+lemma IsAlgEnvSeq.klDiv_map_history_stepKernel (h : IsAlgEnvSeq O A Y alg env P)
+    (h' : IsAlgEnvSeq O' A' Y' alg' env' P') (M : ℕ) :
+    klDiv (P.map (history O A Y M)) (P'.map (history O' A' Y' M)) =
       ∑ t ∈ range M,
-        klDiv (P.map (history X Y t) ⊗ₘ stepKernel alg env t)
-          (P.map (history X Y t) ⊗ₘ stepKernel alg' env' t) := by
-  have hX := h.measurable_action
+        klDiv (P.map (history O A Y t) ⊗ₘ stepKernel alg env t)
+          (P.map (history O A Y t) ⊗ₘ stepKernel alg' env' t) := by
+  have hO := h.measurable_obs
+  have hA := h.measurable_action
   have hY := h.measurable_feedback
-  have hX' := h'.measurable_action
+  have hO' := h'.measurable_obs
+  have hA' := h'.measurable_action
   have hY' := h'.measurable_feedback
   induction M with
   | zero => simp
@@ -100,21 +118,22 @@ lemma IsAlgEnvSeq.klDiv_map_history_stepKernel (h : IsAlgEnvSeq X Y alg env P)
 
 /-- The divergence between the laws of two trajectories is the supremum over `n` of the divergences
 between the laws of the histories up to time `n`. -/
-lemma klDiv_map_trajectory_eq_iSup (hX : ∀ n, Measurable (X n)) (hY : ∀ n, Measurable (Y n))
-    (hX' : ∀ n, Measurable (X' n)) (hY' : ∀ n, Measurable (Y' n)) :
-    klDiv (P.map (trajectory X Y)) (P'.map (trajectory X' Y')) =
-      ⨆ n, klDiv (P.map (history X Y n)) (P'.map (history X' Y' n)) := by
-  have hg : ∀ n, Measurable fun f : ℕ → 𝓐 × 𝓨 ↦ fun i : Fin n ↦ f i.1 := fun n ↦
+lemma klDiv_map_trajectory_eq_iSup (hO : ∀ n, Measurable (O n)) (hA : ∀ n, Measurable (A n))
+    (hY : ∀ n, Measurable (Y n)) (hO' : ∀ n, Measurable (O' n)) (hA' : ∀ n, Measurable (A' n))
+    (hY' : ∀ n, Measurable (Y' n)) :
+    klDiv (P.map (trajectory O A Y)) (P'.map (trajectory O' A' Y')) =
+      ⨆ n, klDiv (P.map (history O A Y n)) (P'.map (history O' A' Y' n)) := by
+  have hg : ∀ n, Measurable fun f : ℕ → Round 𝓞 𝓐 𝓨 ↦ fun i : Fin n ↦ f i.1 := fun n ↦
     measurable_pi_lambda _ fun i ↦ measurable_pi_apply i.1
   rw [klDiv_eq_iSup_map hg ?_ MeasurableSpace.iSup_comap_restrictFin]
   · refine iSup_congr fun n ↦ ?_
-    rw [Measure.map_map (hg n) (measurable_trajectory hX hY),
-      Measure.map_map (hg n) (measurable_trajectory hX' hY')]
+    rw [Measure.map_map (hg n) (measurable_trajectory hO hA hY),
+      Measure.map_map (hg n) (measurable_trajectory hO' hA' hY')]
     rfl
   · intro n m hnm
-    have : (fun f : ℕ → 𝓐 × 𝓨 ↦ fun i : Fin n ↦ f i.1) =
-        (fun h : Fin m → 𝓐 × 𝓨 ↦ fun i : Fin n ↦ h (Fin.castLE hnm i)) ∘
-          fun f : ℕ → 𝓐 × 𝓨 ↦ fun i : Fin m ↦ f i.1 := rfl
+    have : (fun f : ℕ → Round 𝓞 𝓐 𝓨 ↦ fun i : Fin n ↦ f i.1) =
+        (fun h : Fin m → Round 𝓞 𝓐 𝓨 ↦ fun i : Fin n ↦ h (Fin.castLE hnm i)) ∘
+          fun f : ℕ → Round 𝓞 𝓐 𝓨 ↦ fun i : Fin m ↦ f i.1 := rfl
     beta_reduce
     rw [this, ← MeasurableSpace.comap_comp]
     exact MeasurableSpace.comap_mono (measurable_pi_lambda _ fun i ↦
@@ -124,57 +143,59 @@ lemma klDiv_map_trajectory_eq_iSup (hX : ∀ n, Measurable (X n)) (hY : ∀ n, M
 two environments `env` and `env'`, the divergence between the laws of the trajectories is
 the series over the rounds `t` of the conditional divergences of the step at round `t` given
 the first `t` rounds. -/
-lemma IsAlgEnvSeq.klDiv_map_trajectory_stepKernel (h : IsAlgEnvSeq X Y alg env P)
-    (h' : IsAlgEnvSeq X' Y' alg' env' P') :
-    klDiv (P.map (trajectory X Y)) (P'.map (trajectory X' Y')) =
-      ∑' t : ℕ, klDiv (P.map (history X Y t) ⊗ₘ stepKernel alg env t)
-        (P.map (history X Y t) ⊗ₘ stepKernel alg' env' t) := by
-  have hX := h.measurable_action
-  have hY := h.measurable_feedback
-  have hX' := h'.measurable_action
-  have hY' := h'.measurable_feedback
-  rw [klDiv_map_trajectory_eq_iSup hX hY hX' hY', ENNReal.tsum_eq_iSup_nat]
+lemma IsAlgEnvSeq.klDiv_map_trajectory_stepKernel (h : IsAlgEnvSeq O A Y alg env P)
+    (h' : IsAlgEnvSeq O' A' Y' alg' env' P') :
+    klDiv (P.map (trajectory O A Y)) (P'.map (trajectory O' A' Y')) =
+      ∑' t : ℕ, klDiv (P.map (history O A Y t) ⊗ₘ stepKernel alg env t)
+        (P.map (history O A Y t) ⊗ₘ stepKernel alg' env' t) := by
+  rw [klDiv_map_trajectory_eq_iSup h.measurable_obs h.measurable_action h.measurable_feedback
+    h'.measurable_obs h'.measurable_action h'.measurable_feedback, ENNReal.tsum_eq_iSup_nat]
   exact iSup_congr fun n ↦ h.klDiv_map_history_stepKernel h' n
 
 section StationaryEnv
 
+variable {O : ℕ → Ω → Unit} {O' : ℕ → Ω' → Unit} {alg : Algorithm Unit 𝓐 𝓨}
+  {κ κ' : Kernel 𝓐 𝓨} [IsMarkovKernel κ] [IsMarkovKernel κ']
+
 /-- Chain rule for histories of a single algorithm versus two stationary environments. -/
-lemma IsAlgEnvSeq.klDiv_map_history_compProd (h : IsAlgEnvSeq X Y alg (stationaryEnv κ) P)
-    (h' : IsAlgEnvSeq X' Y' alg (stationaryEnv κ') P') (M : ℕ) :
-    klDiv (P.map (history X Y M)) (P'.map (history X' Y' M)) =
-      ∑ t ∈ range M, klDiv (P.map (X t) ⊗ₘ κ) (P.map (X t) ⊗ₘ κ') := by
+lemma IsAlgEnvSeq.klDiv_map_history_compProd (h : IsAlgEnvSeq O A Y alg (stationaryEnv κ) P)
+    (h' : IsAlgEnvSeq O' A' Y' alg (stationaryEnv κ') P') (M : ℕ) :
+    klDiv (P.map (history O A Y M)) (P'.map (history O' A' Y' M)) =
+      ∑ t ∈ range M, klDiv (P.map (A t) ⊗ₘ κ) (P.map (A t) ⊗ₘ κ') := by
   rw [h.klDiv_map_history_stepKernel h']
   refine sum_congr rfl fun t _ ↦ ?_
+  have h_obs := (h.hasCondDistrib_obs t).map_eq
+  rw [obs_stationaryEnv] at h_obs
   rw [stepKernel_stationaryEnv, stepKernel_stationaryEnv,
-    klDiv_compProd_compProd_prodMkLeft_eq_klDiv_comp_compProd,
+    klDiv_compProd_compProd_compProd_prodMkLeft_eq_klDiv_comp_compProd, ← h_obs,
     ← (h.hasCondDistrib_action t).hasLaw_comp.map_eq]
 
 /-- Chain rule for histories of a single algorithm versus two stationary environments. -/
 lemma IsAlgEnvSeq.klDiv_map_history [MeasurableSpace.CountablyGenerated 𝓨]
-    (h : IsAlgEnvSeq X Y alg (stationaryEnv κ) P)
-    (h' : IsAlgEnvSeq X' Y' alg (stationaryEnv κ') P') (M : ℕ) :
-    klDiv (P.map (history X Y M)) (P'.map (history X' Y' M)) =
-      ∑ t ∈ range M, ∫⁻ ω, klDiv (κ (X t ω)) (κ' (X t ω)) ∂P := by
+    (h : IsAlgEnvSeq O A Y alg (stationaryEnv κ) P)
+    (h' : IsAlgEnvSeq O' A' Y' alg (stationaryEnv κ') P') (M : ℕ) :
+    klDiv (P.map (history O A Y M)) (P'.map (history O' A' Y' M)) =
+      ∑ t ∈ range M, ∫⁻ ω, klDiv (κ (A t ω)) (κ' (A t ω)) ∂P := by
   rw [h.klDiv_map_history_compProd h']
   refine sum_congr rfl fun t _ ↦ ?_
   rw [klDiv_compProd_right_eq_lintegral,
     lintegral_map (measurable_klDiv_kernel κ κ') (h.measurable_action t)]
 
 /-- Chain rule for trajectories of a single algorithm versus two stationary environments. -/
-lemma IsAlgEnvSeq.klDiv_map_trajectory_compProd (h : IsAlgEnvSeq X Y alg (stationaryEnv κ) P)
-    (h' : IsAlgEnvSeq X' Y' alg (stationaryEnv κ') P') :
-    klDiv (P.map (trajectory X Y)) (P'.map (trajectory X' Y')) =
-      ∑' t : ℕ, klDiv (P.map (X t) ⊗ₘ κ) (P.map (X t) ⊗ₘ κ') := by
-  rw [klDiv_map_trajectory_eq_iSup h.measurable_action h.measurable_feedback
-    h'.measurable_action h'.measurable_feedback, ENNReal.tsum_eq_iSup_nat]
+lemma IsAlgEnvSeq.klDiv_map_trajectory_compProd (h : IsAlgEnvSeq O A Y alg (stationaryEnv κ) P)
+    (h' : IsAlgEnvSeq O' A' Y' alg (stationaryEnv κ') P') :
+    klDiv (P.map (trajectory O A Y)) (P'.map (trajectory O' A' Y')) =
+      ∑' t : ℕ, klDiv (P.map (A t) ⊗ₘ κ) (P.map (A t) ⊗ₘ κ') := by
+  rw [klDiv_map_trajectory_eq_iSup h.measurable_obs h.measurable_action h.measurable_feedback
+    h'.measurable_obs h'.measurable_action h'.measurable_feedback, ENNReal.tsum_eq_iSup_nat]
   exact iSup_congr fun n ↦ h.klDiv_map_history_compProd h' n
 
 /-- Chain rule for trajectories of a single algorithm versus two stationary environments. -/
 lemma IsAlgEnvSeq.klDiv_map_trajectory [MeasurableSpace.CountablyGenerated 𝓨]
-    (h : IsAlgEnvSeq X Y alg (stationaryEnv κ) P)
-    (h' : IsAlgEnvSeq X' Y' alg (stationaryEnv κ') P') :
-    klDiv (P.map (trajectory X Y)) (P'.map (trajectory X' Y')) =
-      ∑' t : ℕ, ∫⁻ ω, klDiv (κ (X t ω)) (κ' (X t ω)) ∂P := by
+    (h : IsAlgEnvSeq O A Y alg (stationaryEnv κ) P)
+    (h' : IsAlgEnvSeq O' A' Y' alg (stationaryEnv κ') P') :
+    klDiv (P.map (trajectory O A Y)) (P'.map (trajectory O' A' Y')) =
+      ∑' t : ℕ, ∫⁻ ω, klDiv (κ (A t ω)) (κ' (A t ω)) ∂P := by
   rw [h.klDiv_map_trajectory_compProd h']
   refine tsum_congr fun t ↦ ?_
   rw [klDiv_compProd_right_eq_lintegral,
diff --git a/LeanMachineLearning/SequentialLearning/EvaluationEnv.lean b/LeanMachineLearning/SequentialLearning/EvaluationEnv.lean
index 9aac0774..fdae6c06 100644
--- a/LeanMachineLearning/SequentialLearning/EvaluationEnv.lean
+++ b/LeanMachineLearning/SequentialLearning/EvaluationEnv.lean
@@ -69,30 +69,31 @@ lemma feedbackFun_onlineEvalEnv [MeasurableSpace.SeparatesPoints 𝓨] (n : ℕ)
 
 @[simp]
 lemma feedbackFunZero_onlineEvalEnv [MeasurableSpace.SeparatesPoints 𝓨] :
-    feedbackFunZero (onlineEvalEnv g hg) = g 0 := by
+    feedbackFunZero (onlineEvalEnv g hg) = fun p ↦ g 0 p.2 := by
   unfold feedbackFunZero
   rw [feedbackFun_onlineEvalEnv]
 
 section OnlineEvalEnv
 
-variable {Ω : Type*} {mΩ : MeasurableSpace Ω} {alg : Algorithm 𝓐 𝓨}
+variable {Ω : Type*} {mΩ : MeasurableSpace Ω} {alg : Algorithm Unit 𝓐 𝓨}
   {g : ℕ → 𝓐 → 𝓨} {hg : ∀ n, Measurable (g n)}
-  {P : Measure Ω} [IsProbabilityMeasure P] {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
+  {P : Measure Ω} [IsProbabilityMeasure P]
+  {O : ℕ → Ω → Unit} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
 
 lemma hascondDistrib_feedback_onlineEvalEnv
-    (h : IsAlgEnvSeq A Y alg (onlineEvalEnv g hg) P) (n : ℕ) :
+    (h : IsAlgEnvSeq O A Y alg (onlineEvalEnv g hg) P) (n : ℕ) :
     HasCondDistrib (Y n) (A n) (Kernel.deterministic (g n) (hg n)) P := by
   simpa using IsObliviousEnv.hasCondDistrib_feedback h n
 
 lemma feedback_onlineEvalEnv_ae_eq_eval_action [StandardBorelSpace 𝓨] [Nonempty 𝓨]
-    (h : IsAlgEnvSeq A Y alg (onlineEvalEnv g hg) P) (n : ℕ) :
+    (h : IsAlgEnvSeq O A Y alg (onlineEvalEnv g hg) P) (n : ℕ) :
     Y n =ᵐ[P] g n ∘ A n :=
   ae_eq_of_condDistrib_eq_deterministic (hg n) (h.measurable_action n).aemeasurable
     (h.measurable_feedback n).aemeasurable
     (hascondDistrib_feedback_onlineEvalEnv h n).condDistrib_eq
 
 lemma forall_feedback_onlineEvalEnv_ae_eq_eval_action [StandardBorelSpace 𝓨] [Nonempty 𝓨]
-    (h : IsAlgEnvSeq A Y alg (onlineEvalEnv g hg) P) :
+    (h : IsAlgEnvSeq O A Y alg (onlineEvalEnv g hg) P) :
     ∀ᵐ ω ∂P, ∀ n, Y n ω = g n (A n ω) := by
   rw [ae_all_iff]
   intro n
@@ -114,7 +115,7 @@ lemma feedbackCondAction_evalEnv (n : ℕ) :
 
 @[simp]
 lemma feedbackFunZero_evalEnv [MeasurableSpace.SeparatesPoints 𝓨] :
-    feedbackFunZero (evalEnv f hf) = f := by simp [evalEnv]
+    feedbackFunZero (evalEnv f hf) = fun p ↦ f p.2 := by simp [evalEnv]
 
 @[simp]
 lemma feedbackFun_evalEnv [MeasurableSpace.SeparatesPoints 𝓨] (n : ℕ) :
@@ -122,24 +123,26 @@ lemma feedbackFun_evalEnv [MeasurableSpace.SeparatesPoints 𝓨] (n : ℕ) :
 
 section EvalEnv
 
-variable {Ω : Type*} {mΩ : MeasurableSpace Ω} {alg : Algorithm 𝓐 𝓨} {f : 𝓐 → 𝓨} {hf : Measurable f}
-  {P : Measure Ω} [IsProbabilityMeasure P] {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
+variable {Ω : Type*} {mΩ : MeasurableSpace Ω} {alg : Algorithm Unit 𝓐 𝓨}
+  {f : 𝓐 → 𝓨} {hf : Measurable f}
+  {P : Measure Ω} [IsProbabilityMeasure P]
+  {O : ℕ → Ω → Unit} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
 
-lemma hascondDistrib_feedback_evalEnv (h : IsAlgEnvSeq A Y alg (evalEnv f hf) P) (n : ℕ) :
+lemma hascondDistrib_feedback_evalEnv (h : IsAlgEnvSeq O A Y alg (evalEnv f hf) P) (n : ℕ) :
     HasCondDistrib (Y n) (A n) (Kernel.deterministic f hf) P := by
   simpa using IsObliviousEnv.hasCondDistrib_feedback h n
 
 lemma feedback_evalEnv_ae_eq_eval_action [StandardBorelSpace 𝓨] [Nonempty 𝓨]
-  (h : IsAlgEnvSeq A Y alg (evalEnv f hf) P) (n : ℕ) :
+  (h : IsAlgEnvSeq O A Y alg (evalEnv f hf) P) (n : ℕ) :
     Y n =ᵐ[P] f ∘ A n := feedback_onlineEvalEnv_ae_eq_eval_action h n
 
 lemma forall_feedback_evalEnv_ae_eq_eval_action [StandardBorelSpace 𝓨] [Nonempty 𝓨]
-    (h : IsAlgEnvSeq A Y alg (evalEnv f hf) P) :
+    (h : IsAlgEnvSeq O A Y alg (evalEnv f hf) P) :
     ∀ᵐ ω ∂P, ∀ n, Y n ω = f (A n ω) := forall_feedback_onlineEvalEnv_ae_eq_eval_action h
 
 open Finset in
 lemma feedback_evalEnv_ae_eq_eval_action_comp {β : Type*} [StandardBorelSpace 𝓨] [Nonempty 𝓨]
-    (h : IsAlgEnvSeq A Y alg (evalEnv f hf) P) {n : ℕ} (g : (Iic n → 𝓨) → β) :
+    (h : IsAlgEnvSeq O A Y alg (evalEnv f hf) P) {n : ℕ} (g : (Iic n → 𝓨) → β) :
     ∀ᵐ ω ∂P, g (fun i ↦ Y i ω) = g (fun i ↦ f (A i ω)) := by
   filter_upwards [forall_feedback_evalEnv_ae_eq_eval_action h] with ω hω
   simp_rw [hω]
diff --git a/LeanMachineLearning/SequentialLearning/FeedbackMartingale.lean b/LeanMachineLearning/SequentialLearning/FeedbackMartingale.lean
index 34aedb10..df1f7815 100644
--- a/LeanMachineLearning/SequentialLearning/FeedbackMartingale.lean
+++ b/LeanMachineLearning/SequentialLearning/FeedbackMartingale.lean
@@ -21,128 +21,132 @@ open scoped ENNReal
 
 namespace Learning
 
-variable {Ω 𝓐 𝓨 : Type*} {mΩ : MeasurableSpace Ω} {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨}
+variable {Ω 𝓞 𝓐 𝓨 : Type*} {mΩ : MeasurableSpace Ω} {m𝓞 : MeasurableSpace 𝓞}
+  {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨}
   [NormedAddCommGroup 𝓨] [NormedSpace ℝ 𝓨]
   {P : Measure Ω} [IsFiniteMeasure P]
-  {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {alg : Algorithm 𝓐 𝓨} {env : Environment 𝓐 𝓨}
+  {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
+  {alg : Algorithm 𝓞 𝓐 𝓨} {env : Environment 𝓞 𝓐 𝓨}
 
 /-- The sum of noise terms for action `k`.
 This is the martingale part of `sumRewards A Y k` for the filtration
 `IsAlgEnvSeq.filtrationAction`. -/
 noncomputable
-def noiseSum (env : Environment 𝓐 𝓨) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) (k : 𝓐) (n : ℕ) (ω : Ω) : 𝓨 :=
-  ∑ m ∈ range n, {ω | A m ω = k}.indicator (fun ω ↦ Y m ω - env.means A Y (A m ω) m ω) ω
+def noiseSum (env : Environment 𝓞 𝓐 𝓨) (O : ℕ → Ω → 𝓞) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨)
+    (k : 𝓐) (n : ℕ) (ω : Ω) : 𝓨 :=
+  ∑ m ∈ range n, {ω | A m ω = k}.indicator (fun ω ↦ Y m ω - env.means O A Y (A m ω) m ω) ω
 
 /-- The sum of mean terms for action `k`.
 This is the predictable part of `sumRewards A Y k` for the filtration
 `IsAlgEnvSeq.filtrationAction`. -/
 noncomputable
-def meanSum (env : Environment 𝓐 𝓨) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) (k : 𝓐) (n : ℕ) (ω : Ω) : 𝓨 :=
-  ∑ m ∈ range n, {ω | A m ω = k}.indicator (fun ω ↦ env.means A Y (A m ω) m ω) ω
+def meanSum (env : Environment 𝓞 𝓐 𝓨) (O : ℕ → Ω → 𝓞) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨)
+    (k : 𝓐) (n : ℕ) (ω : Ω) : 𝓨 :=
+  ∑ m ∈ range n, {ω | A m ω = k}.indicator (fun ω ↦ env.means O A Y (A m ω) m ω) ω
 
 lemma noiseSum_add_meanSum' (k : 𝓐) (n : ℕ) (ω : Ω) :
-    noiseSum env A Y k n ω + meanSum env A Y k n ω =
+    noiseSum env O A Y k n ω + meanSum env O A Y k n ω =
       ∑ m ∈ range n, {ω | A m ω = k}.indicator (Y m) ω := by
   simp only [noiseSum, meanSum, ← sum_add_distrib]
   congr with m
   by_cases h : A m ω = k <;> simp [h]
 
 lemma noiseSum_add_meanSum [DecidableEq 𝓐] (k : 𝓐) (n : ℕ) (ω : Ω) :
-    noiseSum env A Y k n ω + meanSum env A Y k n ω = sumRewards A Y k n ω := by
+    noiseSum env O A Y k n ω + meanSum env O A Y k n ω = sumRewards A Y k n ω := by
   unfold sumRewards
   rw [noiseSum_add_meanSum' k n ω]
   congr with m
   by_cases h : A m ω = k <;> simp [h]
 
 @[simp]
-lemma noiseSum_zero (k : 𝓐) : noiseSum env A Y k 0 = fun _ ↦ 0 := by unfold noiseSum; simp
+lemma noiseSum_zero (k : 𝓐) : noiseSum env O A Y k 0 = fun _ ↦ 0 := by unfold noiseSum; simp
 
 @[simp]
-lemma meanSum_zero (k : 𝓐) : meanSum env A Y k 0 = fun _ ↦ 0 := by unfold meanSum; simp
+lemma meanSum_zero (k : 𝓐) : meanSum env O A Y k 0 = fun _ ↦ 0 := by unfold meanSum; simp
 
 lemma noiseSum_succ (k : 𝓐) (n : ℕ) :
-    noiseSum env A Y k (n + 1) = noiseSum env A Y k n +
-      {ω | A n ω = k}.indicator (fun ω ↦ Y n ω - env.means A Y (A n ω) n ω) := by
+    noiseSum env O A Y k (n + 1) = noiseSum env O A Y k n +
+      {ω | A n ω = k}.indicator (fun ω ↦ Y n ω - env.means O A Y (A n ω) n ω) := by
   ext ω
   simp [noiseSum, Finset.sum_range_succ]
 
 lemma noiseSum_succ_sub (k : 𝓐) (n : ℕ) (ω : Ω) :
-    noiseSum env A Y k (n + 1) ω - noiseSum env A Y k n ω
-      = {ω | A n ω = k}.indicator (fun ω ↦ Y n ω - env.means A Y (A n ω) n ω) ω := by
+    noiseSum env O A Y k (n + 1) ω - noiseSum env O A Y k n ω
+      = {ω | A n ω = k}.indicator (fun ω ↦ Y n ω - env.means O A Y (A n ω) n ω) ω := by
   simp [noiseSum_succ]
 
 lemma meanSum_succ (k : 𝓐) (n : ℕ) :
-    meanSum env A Y k (n + 1) = meanSum env A Y k n +
-      {ω | A n ω = k}.indicator (fun ω ↦ env.means A Y (A n ω) n ω) := by
+    meanSum env O A Y k (n + 1) = meanSum env O A Y k n +
+      {ω | A n ω = k}.indicator (fun ω ↦ env.means O A Y (A n ω) n ω) := by
   ext ω
   simp [meanSum, Finset.sum_range_succ]
 
 lemma meanSum_succ_sub (k : 𝓐) (n : ℕ) (ω : Ω) :
-    meanSum env A Y k (n + 1) ω - meanSum env A Y k n ω
-      = {ω | A n ω = k}.indicator (fun ω ↦ env.means A Y (A n ω) n ω) ω := by
+    meanSum env O A Y k (n + 1) ω - meanSum env O A Y k n ω
+      = {ω | A n ω = k}.indicator (fun ω ↦ env.means O A Y (A n ω) n ω) ω := by
   simp [meanSum_succ]
 
 variable [MeasurableSingletonClass 𝓐] [SecondCountableTopology 𝓨]
 
 @[fun_prop]
 lemma IsAlgEnvSeq.integrable_noiseSum_increment [OpensMeasurableSpace 𝓨]
-    {m : ℕ} (h : IsAlgEnvSeq A Y alg env P) (hint : Integrable (Y m) P) (k : 𝓐) :
+    {m : ℕ} (h : IsAlgEnvSeq O A Y alg env P) (hint : Integrable (Y m) P) (k : 𝓐) :
     Integrable (fun ω ↦ {ω | A m ω = k}.indicator
-      (fun ω ↦ Y m ω - env.means A Y (A m ω) m ω) ω) P := by
+      (fun ω ↦ Y m ω - env.means O A Y (A m ω) m ω) ω) P := by
   exact (hint.sub (h.integrable_means_action hint)).indicator
     (h.measurable_action _ (measurableSet_singleton k))
 
 @[fun_prop]
 lemma IsAlgEnvSeq.integrable_meanSum_increment [OpensMeasurableSpace 𝓨]
-    {m : ℕ} (h : IsAlgEnvSeq A Y alg env P) (hint : Integrable (Y m) P) (k : 𝓐) :
-    Integrable (fun ω ↦ {ω | A m ω = k}.indicator (fun ω ↦ env.means A Y (A m ω) m ω) ω) P := by
+    {m : ℕ} (h : IsAlgEnvSeq O A Y alg env P) (hint : Integrable (Y m) P) (k : 𝓐) :
+    Integrable (fun ω ↦ {ω | A m ω = k}.indicator (fun ω ↦ env.means O A Y (A m ω) m ω) ω) P := by
   exact (h.integrable_means_action hint).indicator
     (h.measurable_action _ (measurableSet_singleton k))
 
 @[fun_prop]
 lemma IsAlgEnvSeq.integrable_noiseSum [OpensMeasurableSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) (hint : ∀ n, Integrable (Y n) P) (k : 𝓐) (n : ℕ) :
-    Integrable (noiseSum env A Y k n) P :=
+    (h : IsAlgEnvSeq O A Y alg env P) (hint : ∀ n, Integrable (Y n) P) (k : 𝓐) (n : ℕ) :
+    Integrable (noiseSum env O A Y k n) P :=
   integrable_finsetSum _ fun m _ ↦ h.integrable_noiseSum_increment (hint m) k
 
 @[fun_prop]
 lemma IsAlgEnvSeq.integrable_meanSum [OpensMeasurableSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) (hint : ∀ n, Integrable (Y n) P) (k : 𝓐) (n : ℕ) :
-    Integrable (meanSum env A Y k n) P :=
+    (h : IsAlgEnvSeq O A Y alg env P) (hint : ∀ n, Integrable (Y n) P) (k : 𝓐) (n : ℕ) :
+    Integrable (meanSum env O A Y k n) P :=
   integrable_finsetSum _ fun m _ ↦ h.integrable_meanSum_increment (hint m) k
 
 lemma IsAlgEnvSeq.memLp_noiseSum_increment [BorelSpace 𝓨]
-    {m : ℕ} (k : 𝓐) (h : IsAlgEnvSeq A Y alg env P) {p : ℝ≥0∞} (hp1 : 1 ≤ p) (hp_top : p ≠ ∞)
+    {m : ℕ} (k : 𝓐) (h : IsAlgEnvSeq O A Y alg env P) {p : ℝ≥0∞} (hp1 : 1 ≤ p) (hp_top : p ≠ ∞)
     (hY : MemLp (Y m) p P) :
-    MemLp ({ω | A m ω = k}.indicator (fun ω ↦ Y m ω - env.means A Y (A m ω) m ω)) p P := by
+    MemLp ({ω | A m ω = k}.indicator (fun ω ↦ Y m ω - env.means O A Y (A m ω) m ω)) p P := by
   refine (hY.sub ?_).indicator (h.measurable_action _ (measurableSet_singleton k))
   exact h.memLp_means_action hp1 hp_top hY
 
 lemma IsAlgEnvSeq.memLp_meanSum_increment [BorelSpace 𝓨]
-    {m : ℕ} (k : 𝓐) (h : IsAlgEnvSeq A Y alg env P) {p : ℝ≥0∞} (hp1 : 1 ≤ p) (hp_top : p ≠ ∞)
+    {m : ℕ} (k : 𝓐) (h : IsAlgEnvSeq O A Y alg env P) {p : ℝ≥0∞} (hp1 : 1 ≤ p) (hp_top : p ≠ ∞)
     (hY : MemLp (Y m) p P) :
-    MemLp ({ω | A m ω = k}.indicator (fun ω ↦ env.means A Y (A m ω) m ω)) p P := by
+    MemLp ({ω | A m ω = k}.indicator (fun ω ↦ env.means O A Y (A m ω) m ω)) p P := by
   exact (h.memLp_means_action hp1 hp_top hY).indicator
     (h.measurable_action _ (measurableSet_singleton k))
 
 lemma IsAlgEnvSeq.memLp_noiseSum [BorelSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) {p : ℝ≥0∞} (hp1 : 1 ≤ p) (hp_top : p ≠ ∞)
+    (h : IsAlgEnvSeq O A Y alg env P) {p : ℝ≥0∞} (hp1 : 1 ≤ p) (hp_top : p ≠ ∞)
     (hY : ∀ n, MemLp (Y n) p P) (k : 𝓐) (n : ℕ) :
-    MemLp (noiseSum env A Y k n) p P :=
+    MemLp (noiseSum env O A Y k n) p P :=
   memLp_finsetSum _ fun m _ ↦ memLp_noiseSum_increment k h hp1 hp_top (hY m)
 
 lemma IsAlgEnvSeq.memLp_meanSum [BorelSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) {p : ℝ≥0∞} (hp1 : 1 ≤ p) (hp_top : p ≠ ∞)
+    (h : IsAlgEnvSeq O A Y alg env P) {p : ℝ≥0∞} (hp1 : 1 ≤ p) (hp_top : p ≠ ∞)
     (hY : ∀ n, MemLp (Y n) p P) (k : 𝓐) (n : ℕ) :
-    MemLp (meanSum env A Y k n) p P :=
+    MemLp (meanSum env O A Y k n) p P :=
   memLp_finsetSum _ fun m _ ↦ memLp_meanSum_increment k h hp1 hp_top (hY m)
 
 section Martingale
 
 variable [BorelSpace 𝓨]
 
-lemma IsAlgEnvSeq.adapted_noiseSum (h : IsAlgEnvSeq A Y alg env P) (k : 𝓐) :
-    Adapted h.filtrationAction (noiseSum env A Y k) := by
+lemma IsAlgEnvSeq.adapted_noiseSum (h : IsAlgEnvSeq O A Y alg env P) (k : 𝓐) :
+    Adapted h.filtrationAction (noiseSum env O A Y k) := by
   refine fun n ↦ Finset.measurable_fun_sum _ fun m hm ↦ ?_
   have hAm : Measurable[h.filtrationAction n] (A m) :=
     h.adapted_action_filtrationAction.measurable_le (by grind)
@@ -151,12 +155,12 @@ lemma IsAlgEnvSeq.adapted_noiseSum (h : IsAlgEnvSeq A Y alg env P) (k : 𝓐) :
   refine (hYm.sub ?_).indicator (hAm (measurableSet_singleton k))
   exact h.adapted_means_filtrationAction.measurable_le (by grind)
 
-lemma IsAlgEnvSeq.stronglyAdapted_noiseSum (h : IsAlgEnvSeq A Y alg env P) (k : 𝓐) :
-    StronglyAdapted h.filtrationAction (noiseSum env A Y k) :=
+lemma IsAlgEnvSeq.stronglyAdapted_noiseSum (h : IsAlgEnvSeq O A Y alg env P) (k : 𝓐) :
+    StronglyAdapted h.filtrationAction (noiseSum env O A Y k) :=
   (adapted_noiseSum h k).stronglyAdapted
 
-lemma IsAlgEnvSeq.isStronglyPredictable_meanSum (h : IsAlgEnvSeq A Y alg env P) (k : 𝓐) :
-    IsStronglyPredictable h.filtrationAction (meanSum env A Y k) := by
+lemma IsAlgEnvSeq.isStronglyPredictable_meanSum (h : IsAlgEnvSeq O A Y alg env P) (k : 𝓐) :
+    IsStronglyPredictable h.filtrationAction (meanSum env O A Y k) := by
   refine .of_measurable_add_one ?_ fun n ↦ ?_
   · simp only [meanSum_zero]
     fun_prop
@@ -167,12 +171,13 @@ lemma IsAlgEnvSeq.isStronglyPredictable_meanSum (h : IsAlgEnvSeq A Y alg env P)
     exact (h.stronglyAdapted_means_filtrationAction m).mono (h.filtrationAction.mono (by grind))
 
 lemma IsAlgEnvSeq.condExp_noiseSum_increment [CompleteSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) (k : 𝓐) (i : ℕ) (hint : Integrable (Y i) P) :
-    P[{ω | A i ω = k}.indicator (fun ω ↦ Y i ω - env.means A Y (A i ω) i ω) | h.filtrationAction i]
-      =ᵐ[P] 0 := by
+    (h : IsAlgEnvSeq O A Y alg env P) (k : 𝓐) (i : ℕ) (hint : Integrable (Y i) P) :
+    P[{ω | A i ω = k}.indicator (fun ω ↦ Y i ω - env.means O A Y (A i ω) i ω)
+        | h.filtrationAction i] =ᵐ[P] 0 := by
   let c : Ω → ℝ := actionIndicator A k i
-  let g : Ω → 𝓨 := fun ω ↦ Y i ω - env.means A Y (A i ω) i ω
-  have h_smul : c • g = {ω | A i ω = k}.indicator (fun ω ↦ Y i ω - env.means A Y (A i ω) i ω) := by
+  let g : Ω → 𝓨 := fun ω ↦ Y i ω - env.means O A Y (A i ω) i ω
+  have h_smul : c • g
+      = {ω | A i ω = k}.indicator (fun ω ↦ Y i ω - env.means O A Y (A i ω) i ω) := by
     ext ω
     by_cases hω : A i ω = k <;> simp [c, g, actionIndicator, hω]
   have hAG : Measurable[h.filtrationAction i] (A i) := h.adapted_action_filtrationAction i
@@ -199,15 +204,15 @@ lemma IsAlgEnvSeq.condExp_noiseSum_increment [CompleteSpace 𝓨]
   · simp [c, actionIndicator, hak]
 
 lemma IsAlgEnvSeq.martingale_noiseSum [CompleteSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) (hint : ∀ n, Integrable (Y n) P) (k : 𝓐) :
-    Martingale (noiseSum env A Y k) h.filtrationAction P := by
-  have hInt : ∀ n, Integrable (noiseSum env A Y k n) P := h.integrable_noiseSum (hint) k
+    (h : IsAlgEnvSeq O A Y alg env P) (hint : ∀ n, Integrable (Y n) P) (k : 𝓐) :
+    Martingale (noiseSum env O A Y k) h.filtrationAction P := by
+  have hInt : ∀ n, Integrable (noiseSum env O A Y k n) P := h.integrable_noiseSum (hint) k
   refine martingale_nat (h.stronglyAdapted_noiseSum k) hInt fun i ↦ ?_
   rw [noiseSum_succ]
   symm
   have hadd := condExp_add (hInt i)
     (integrable_noiseSum_increment h (hint i) k) (h.filtrationAction i)
-  have hself : P[noiseSum env A Y k i | h.filtrationAction i] = noiseSum env A Y k i :=
+  have hself : P[noiseSum env O A Y k i | h.filtrationAction i] = noiseSum env O A Y k i :=
     condExp_of_stronglyMeasurable (h.filtrationAction.le i)
       (h.stronglyAdapted_noiseSum k i) (hInt i)
   have hincr := condExp_noiseSum_increment h k i (hint i)
diff --git a/LeanMachineLearning/SequentialLearning/FiniteActions.lean b/LeanMachineLearning/SequentialLearning/FiniteActions.lean
index a59ca328..b0ca865b 100644
--- a/LeanMachineLearning/SequentialLearning/FiniteActions.lean
+++ b/LeanMachineLearning/SequentialLearning/FiniteActions.lean
@@ -28,11 +28,12 @@ open MeasureTheory Finset Learning
 
 namespace Learning
 
-variable {𝓐 R Ω : Type*} {m𝓐 : MeasurableSpace 𝓐} {mR : MeasurableSpace R} {mΩ : MeasurableSpace Ω}
+variable {𝓞 𝓐 R Ω : Type*} {m𝓞 : MeasurableSpace 𝓞} {m𝓐 : MeasurableSpace 𝓐}
+  {mR : MeasurableSpace R} {mΩ : MeasurableSpace Ω}
   [DecidableEq 𝓐]
-  {alg : Algorithm 𝓐 R} {env : Environment 𝓐 R}
+  {alg : Algorithm 𝓞 𝓐 R} {env : Environment 𝓞 𝓐 R}
   {P : Measure Ω} [IsProbabilityMeasure P]
-  {A : ℕ → Ω → 𝓐} {R' : ℕ → Ω → R}
+  {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {R' : ℕ → Ω → R}
   {a : 𝓐} {m n t : ℕ} {ω : Ω}
 
 section PullCount
@@ -45,7 +46,7 @@ def pullCount (A : ℕ → Ω → 𝓐) (a : 𝓐) (t : ℕ) (ω : Ω) : ℕ :=
 /-- Number of pulls of arm `a` in the history before time `n`.
 This is the number of entries in `h` in which the arm is `a`. -/
 noncomputable
-def pullCount' (n : ℕ) (h : Fin n → 𝓐 × R) (a : 𝓐) := #{s | (h s).1 = a}
+def pullCount' (n : ℕ) (h : Hist 𝓞 𝓐 R n) (a : 𝓐) := #{s | (h s).action = a}
 
 @[simp]
 lemma pullCount_zero (a : 𝓐) : pullCount A a 0 = 0 := by ext; simp [pullCount]
@@ -84,23 +85,23 @@ lemma pullCount_add_one :
 lemma pullCount_eq_sum (a : 𝓐) (t : ℕ) (ω : Ω) :
     pullCount A a t ω = ∑ s ∈ range t, if A s ω = a then 1 else 0 := by simp [pullCount]
 
-lemma pullCount'_eq_sum (n : ℕ) (h : Fin n → 𝓐 × R) (a : 𝓐) :
-    pullCount' n h a = ∑ s : Fin n, if (h s).1 = a then 1 else 0 := by simp [pullCount']
+lemma pullCount'_eq_sum (n : ℕ) (h : Hist 𝓞 𝓐 R n) (a : 𝓐) :
+    pullCount' n h a = ∑ s : Fin n, if (h s).action = a then 1 else 0 := by simp [pullCount']
 
 lemma pullCount_eq_pullCount' {n : ℕ} {ω : Ω} :
-    pullCount A a n ω = pullCount' n (fun i ↦ (A i ω, R' i ω)) a := by
+    pullCount A a n ω = pullCount' n (history O A R' n ω) a := by
   rw [pullCount_eq_sum, pullCount'_eq_sum]
   exact (Fin.sum_univ_eq_sum_range (fun i ↦ if A i ω = a then 1 else 0) n).symm
 
 /-- `pullCount A a n` is a function of the history before time `n`. -/
 lemma pullCount_eq_comp_history (a : 𝓐) (n : ℕ) :
-    pullCount A a n = (fun h : Fin n → 𝓐 × R ↦ pullCount' n h a) ∘ history A R' n := by
+    pullCount A a n = (fun h : Hist 𝓞 𝓐 R n ↦ pullCount' n h a) ∘ history O A R' n := by
   ext ω
   exact pullCount_eq_pullCount'
 
 lemma pullCount'_mono {n m : ℕ} (hnm : n ≤ m) :
-    pullCount' n (fun i ↦ (A i ω, R' i ω)) a ≤ pullCount' m (fun i ↦ (A i ω, R' i ω)) a := by
-  rw [← pullCount_eq_pullCount', ← pullCount_eq_pullCount']
+    pullCount' n (history O A R' n ω) a ≤ pullCount' m (history O A R' m ω) a := by
+  rw [← pullCount_eq_pullCount' (O := O), ← pullCount_eq_pullCount' (O := O)]
   exact pullCount_mono a hnm _
 
 lemma pullCount_le (a : 𝓐) (t : ℕ) (ω : Ω) : pullCount A a t ω ≤ t :=
@@ -201,38 +202,39 @@ lemma measurable_uncurry_pullCount_comp [Countable 𝓐] [MeasurableSingletonCla
 
 @[fun_prop]
 lemma measurable_pullCount' [MeasurableSingletonClass 𝓐] (n : ℕ) (a : 𝓐) :
-    Measurable (fun h : Fin n → 𝓐 × R ↦ pullCount' n h a) := by
+    Measurable (fun h : Hist 𝓞 𝓐 R n ↦ pullCount' n h a) := by
   simp_rw [pullCount'_eq_sum]
-  have h_meas s : Measurable (fun (h : Fin n → 𝓐 × R) ↦ if (h s).1 = a then 1 else 0) := by
+  have h_meas s : Measurable (fun (h : Hist 𝓞 𝓐 R n) ↦ if (h s).action = a then 1 else 0) := by
     refine Measurable.ite ?_ (by fun_prop) (by fun_prop)
     exact (measurableSet_singleton _).preimage (by fun_prop)
   fun_prop
 
 @[fun_prop]
 lemma measurable_uncurry_pullCount' [MeasurableEq 𝓐] (n : ℕ) :
-    Measurable (fun p : (Fin n → 𝓐 × R) × 𝓐 ↦ pullCount' n p.1 p.2) := by
+    Measurable (fun p : Hist 𝓞 𝓐 R n × 𝓐 ↦ pullCount' n p.1 p.2) := by
   simp_rw [pullCount'_eq_sum]
-  have h_meas s : Measurable (fun h : (Fin n → 𝓐 × R) × 𝓐 ↦ if (h.1 s).1 = h.2 then 1 else 0) := by
+  have h_meas s : Measurable
+      (fun h : Hist 𝓞 𝓐 R n × 𝓐 ↦ if (h.1 s).action = h.2 then 1 else 0) := by
     refine Measurable.ite ?_ (by fun_prop) (by fun_prop)
     exact measurableSet_eq_fun (by fun_prop) (by fun_prop)
   fun_prop
 
 lemma adapted_pullCount_add_one [MeasurableSingletonClass 𝓐]
-    (h : IsAlgEnvSeq A R' alg env P) (a : 𝓐) :
+    (h : IsAlgEnvSeq O A R' alg env P) (a : 𝓐) :
     Adapted h.filtration (fun n ↦ pullCount A a (n + 1)) := by
   intro n
   change Measurable[h.filtration n] (pullCount A a (n + 1))
-  rw [measurable_iff_comap_le, h.filtration_eq_comap, pullCount_eq_comp_history (R' := R'),
+  rw [measurable_iff_comap_le, h.filtration_eq_comap, pullCount_eq_comp_history (O := O) (R' := R'),
     ← measurable_iff_comap_le]
   exact measurable_comp_comap _ (measurable_pullCount' (n + 1) a)
 
 lemma stronglyAdapted_pullCount_add_one [MeasurableSingletonClass 𝓐]
-    (h : IsAlgEnvSeq A R' alg env P) (a : 𝓐) :
+    (h : IsAlgEnvSeq O A R' alg env P) (a : 𝓐) :
     StronglyAdapted h.filtration (fun n ↦ pullCount A a (n + 1)) :=
   (adapted_pullCount_add_one h a).stronglyAdapted
 
 lemma isStronglyPredictable_pullCount [MeasurableSingletonClass 𝓐]
-    (h : IsAlgEnvSeq A R' alg env P) (a : 𝓐) :
+    (h : IsAlgEnvSeq O A R' alg env P) (a : 𝓐) :
     IsStronglyPredictable h.filtration (pullCount A a) := by
   rw [IsStronglyPredictable.iff_measurable_add_one]
   refine ⟨?_, stronglyAdapted_pullCount_add_one h a⟩
@@ -247,19 +249,20 @@ lemma measurableSet_action_eq_and_pullCount_eq [MeasurableSingletonClass 𝓐]
 
 lemma measurableSet_snd_eq_and_pullCount'_eq [MeasurableSingletonClass 𝓐]
     (n : ℕ) (b : 𝓐) (k : ℕ) :
-    MeasurableSet {u : (Fin n → 𝓐 × R) × 𝓐 | u.2 = b ∧ pullCount' n u.1 b = k} :=
+    MeasurableSet {u : (Hist 𝓞 𝓐 R n × 𝓞) × 𝓐 | u.2 = b ∧ pullCount' n u.1.1 b = k} :=
   ((measurableSet_singleton _).preimage measurable_snd).inter
-    ((measurableSet_singleton _).preimage ((measurable_pullCount' n b).comp measurable_fst))
+    ((measurableSet_singleton _).preimage
+      ((measurable_pullCount' n b).comp (measurable_fst.comp measurable_fst)))
 
 /-- The event that the action at time `n` is `b` and that `b` was pulled `k` times before is
-a preimage by `(history A R' n, A n)`. -/
+a preimage by `((history O A R' n, O n), A n)`. -/
 lemma setOf_action_eq_and_pullCount_eq_eq_preimage (n : ℕ) (b : 𝓐) (k : ℕ) :
     {x | A n x = b ∧ pullCount A b n x = k}
-      = (fun x ↦ (history A R' n x, A n x)) ⁻¹' {u | u.2 = b ∧ pullCount' n u.1 b = k} := by
+      = (fun x ↦ ((history O A R' n x, O n x), A n x))
+        ⁻¹' {u | u.2 = b ∧ pullCount' n u.1.1 b = k} := by
   ext x
   simp only [Set.mem_ofPred_eq, Set.mem_preimage]
-  rw [pullCount_eq_pullCount' (R' := R')]
-  rfl
+  rw [pullCount_eq_pullCount' (O := O) (R' := R')]
 
 lemma integrable_pullCount [MeasurableSingletonClass 𝓐]
     (hA : ∀ n, Measurable (A n)) (a : 𝓐) (n : ℕ) :
@@ -527,7 +530,7 @@ lemma stepsUntil_eq_congr {ω' : Ω} (h_eq : ∀ i ≤ n, A i ω = A i ω') :
 section Measurability
 
 lemma isStoppingTime_stepsUntil [MeasurableSingletonClass 𝓐]
-    (h : IsAlgEnvSeq A R' alg env P) (a : 𝓐) (hm : m ≠ 0) :
+    (h : IsAlgEnvSeq O A R' alg env P) (a : 𝓐) (hm : m ≠ 0) :
     IsStoppingTime h.filtration (stepsUntil A a m) := by
   rw [stepsUntil_eq_leastGE _ hm]
   refine StronglyAdapted.isStoppingTime_leastGE _ fun n ↦ ?_
@@ -576,9 +579,9 @@ lemma measurable_stepsUntil' [MeasurableSingletonClass 𝓐]
   (measurable_stepsUntil hA a m).comp measurable_fst
 
 lemma measurable_comap_indicator_stepsUntil_eq [MeasurableSingletonClass 𝓐]
-    (R' : ℕ → Ω → R) (a : 𝓐) (m n : ℕ) :
+    (O : ℕ → Ω → 𝓞) (R' : ℕ → Ω → R) (a : 𝓐) (m n : ℕ) :
     Measurable[MeasurableSpace.comap
-        (fun ω : Ω ↦ (history A R' n ω, A n ω)) inferInstance]
+        (fun ω : Ω ↦ ((history O A R' n ω, O n ω), A n ω)) inferInstance]
       ({ω | stepsUntil A a m ω = ↑n}.indicator fun _ ↦ 1) := by
   by_cases hm : m = 0
   · simp only [hm]
@@ -604,38 +607,42 @@ lemma measurable_comap_indicator_stepsUntil_eq [MeasurableSingletonClass 𝓐]
   refine ((measurableSet_singleton _).preimage ?_).inter ((measurableSet_singleton _).preimage ?_)
   · rw [measurable_iff_comap_le, Prod.instMeasurableSpace, MeasurableSpace.comap_prodMk]
     exact le_sup_of_le_right le_rfl
-  · rw [measurable_iff_comap_le, Prod.instMeasurableSpace, MeasurableSpace.comap_prodMk]
-    refine le_sup_of_le_left ?_
-    rw [← measurable_iff_comap_le, pullCount_eq_comp_history (R' := R')]
-    exact measurable_comp_comap _ (measurable_pullCount' n a)
+  · have h_comp : pullCount A a n
+        = (fun p : (Hist 𝓞 𝓐 R n × 𝓞) × 𝓐 ↦ pullCount' n p.1.1 a) ∘
+          (fun ω ↦ ((history O A R' n ω, O n ω), A n ω)) := by
+      rw [pullCount_eq_comp_history (O := O) (R' := R')]
+      rfl
+    rw [h_comp]
+    exact measurable_comp_comap _ (by fun_prop)
 
 lemma measurable_indicator_stepsUntil_eq [MeasurableSingletonClass 𝓐]
-    (h : IsAlgEnvSeq A R' alg env P) (a : 𝓐) (m n : ℕ) :
+    (h : IsAlgEnvSeq O A R' alg env P) (a : 𝓐) (m n : ℕ) :
     Measurable ({ω : Ω | stepsUntil A a m ω = ↑n}.indicator fun _ ↦ 1) := by
-  refine (measurable_comap_indicator_stepsUntil_eq (mR := mR) R' a m n).mono ?_ le_rfl
+  refine (measurable_comap_indicator_stepsUntil_eq (m𝓞 := m𝓞) (mR := mR) O R' a m n).mono ?_ le_rfl
   refine Measurable.comap_le ?_
+  have hO := h.measurable_obs
   have hA := h.measurable_action
   have hR' := h.measurable_feedback
   fun_prop
 
 lemma measurableSet_stepsUntil_eq [MeasurableSingletonClass 𝓐]
-    (R' : ℕ → Ω → R) (a : 𝓐) (m n : ℕ) :
-    MeasurableSet[MeasurableSpace.comap (fun ω : Ω ↦ (history A R' n ω, A n ω))
-        inferInstance]
+    (O : ℕ → Ω → 𝓞) (R' : ℕ → Ω → R) (a : 𝓐) (m n : ℕ) :
+    MeasurableSet[MeasurableSpace.comap
+        (fun ω : Ω ↦ ((history O A R' n ω, O n ω), A n ω)) inferInstance]
       {ω : Ω | stepsUntil A a m ω = ↑n} := by
   let mProd := MeasurableSpace.comap
-    (fun ω : Ω ↦ (history A R' n ω, A n ω)) inferInstance
+    (fun ω : Ω ↦ ((history O A R' n ω, O n ω), A n ω)) inferInstance
   suffices Measurable[mProd] ({ω | stepsUntil A a m ω = ↑n}.indicator fun x ↦ 1) by
     rwa [measurable_indicator_const_iff] at this
-  exact measurable_comap_indicator_stepsUntil_eq R' a m n
+  exact measurable_comap_indicator_stepsUntil_eq O R' a m n
 
 /-- `stepsUntil a m` is a stopping time with respect to the filtration `filtrationAction`. -/
 lemma isStoppingTime_stepsUntil_filtrationAction [MeasurableSingletonClass 𝓐]
-    (h : IsAlgEnvSeq A R' alg env P) (a : 𝓐) (m : ℕ) :
+    (h : IsAlgEnvSeq O A R' alg env P) (a : 𝓐) (m : ℕ) :
     IsStoppingTime h.filtrationAction (stepsUntil A a m) := by
   refine isStoppingTime_of_measurableSet_eq fun n ↦ ?_
   rw [h.filtrationAction_eq_comap n]
-  exact measurableSet_stepsUntil_eq R' a m n
+  exact measurableSet_stepsUntil_eq O R' a m n
 
 end Measurability
 
@@ -874,11 +881,11 @@ lemma sum_comp_pullCount [Fintype 𝓐] [AddCommMonoid R] (f : ℕ → R) (t : 
     · simp [sum_range_succ]
     · simp
 
-lemma sum_pullCount' [Fintype 𝓐] (n : ℕ) (h : Fin n → 𝓐 × ℝ) : ∑ a, pullCount' n h a = n := by
+lemma sum_pullCount' [Fintype 𝓐] (n : ℕ) (h : Hist 𝓞 𝓐 ℝ n) : ∑ a, pullCount' n h a = n := by
   simp_rw [pullCount'_eq_sum]
   rw [Finset.sum_comm]
-  have hcol (s : Fin n) : ∑ a, (if (h s).1 = a then (1 : ℕ) else 0) = 1 := by
-    simp [Finset.sum_ite_eq univ (h s).1 (fun _ ↦ (1 : ℕ))]
+  have hcol (s : Fin n) : ∑ a, (if (h s).action = a then (1 : ℕ) else 0) = 1 := by
+    simp [Finset.sum_ite_eq univ (h s).action (fun _ ↦ (1 : ℕ))]
   simp [hcol]
 
 end Learning
diff --git a/LeanMachineLearning/SequentialLearning/IonescuTulceaSpace.lean b/LeanMachineLearning/SequentialLearning/IonescuTulceaSpace.lean
index 411429af..515c6874 100644
--- a/LeanMachineLearning/SequentialLearning/IonescuTulceaSpace.lean
+++ b/LeanMachineLearning/SequentialLearning/IonescuTulceaSpace.lean
@@ -17,11 +17,11 @@ The main ingredient of the construction is the Ionescu-Tulcea theorem.
 
 ## Main statements
 
-* `isAlgEnvSeq_unique`: the law of the sequence of actions and observations generated by an
+* `isAlgEnvSeq_unique`: the law of the sequence of rounds generated by an
   algorithm-environment pair is unique: it does not depend on the probability space used.
-  If `A₁`, `R₁` and `A₂`, `R₂` are two algorithm-environment sequences generated by the same
-  algorithm-environment pair on probability spaces `(Ω, P)` and `(Ω', P')`, then
-  `P.map (fun ω n ↦ (A₁ n ω, R₁ n ω)) = P'.map (fun ω n ↦ (A₂ n ω, R₂ n ω))`.
+  If `O₁`, `A₁`, `R₁` and `O₂`, `A₂`, `R₂` are two algorithm-environment sequences generated by the
+  same algorithm-environment pair on probability spaces `(Ω, P)` and `(Ω', P')`, then
+  `P.map (trajectory O₁ A₁ R₁) = P'.map (trajectory O₂ A₂ R₂)`.
 
 -/
 
@@ -33,151 +33,171 @@ open scoped ENNReal NNReal
 
 namespace Learning
 
-variable {𝓐 𝓨 Ω : Type*} {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨} {mΩ : MeasurableSpace Ω}
+variable {𝓞 𝓐 𝓨 Ω : Type*} {m𝓞 : MeasurableSpace 𝓞} {m𝓐 : MeasurableSpace 𝓐}
+  {m𝓨 : MeasurableSpace 𝓨} {mΩ : MeasurableSpace Ω}
 
-/-- Measure on the sequence of actions and observations generated by the algorithm/environment. -/
+/-- Measure on the sequence of rounds generated by the algorithm/environment. -/
 noncomputable
-def trajMeasure (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) :
-    Measure (ℕ → 𝓐 × 𝓨) :=
+def trajMeasure (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) :
+    Measure (ℕ → Round 𝓞 𝓐 𝓨) :=
   Kernel.trajMeasureFin (stepKernel alg env)
 deriving IsProbabilityMeasure
 
 namespace IT
 
-/-- Action and feedback at step `n`. -/
-def step (n : ℕ) (h : ℕ → 𝓐 × 𝓨) : 𝓐 × 𝓨 := h n
+/-- The round at step `n`. -/
+def step (n : ℕ) (h : ℕ → Round 𝓞 𝓐 𝓨) : Round 𝓞 𝓐 𝓨 := h n
+
+/-- `obs n` is the observation at time `n`. This is a random variable on the measurable space
+`ℕ → Round 𝓞 𝓐 𝓨`. -/
+def obs (n : ℕ) (h : ℕ → Round 𝓞 𝓐 𝓨) : 𝓞 := (h n).obs
 
 /-- `action n` is the action pulled at time `n`. This is a random variable on the measurable space
-`ℕ → 𝓐 × 𝓨`. -/
-def action (n : ℕ) (h : ℕ → 𝓐 × 𝓨) : 𝓐 := (h n).1
+`ℕ → Round 𝓞 𝓐 𝓨`. -/
+def action (n : ℕ) (h : ℕ → Round 𝓞 𝓐 𝓨) : 𝓐 := (h n).action
 
 /-- `feedback n` is the feedback at time `n`. This is a random variable on the measurable space
-`ℕ → 𝓐 × 𝓨`. -/
-def feedback (n : ℕ) (h : ℕ → 𝓐 × 𝓨) : 𝓨 := (h n).2
+`ℕ → Round 𝓞 𝓐 𝓨`. -/
+def feedback (n : ℕ) (h : ℕ → Round 𝓞 𝓐 𝓨) : 𝓨 := (h n).feedback
 
-/-- `hist n` is the history before time `n`: the action-feedback pairs at times `0, ..., n - 1`.
-This is a random variable on the measurable space `ℕ → 𝓐 × 𝓨`. -/
-def hist (n : ℕ) (h : ℕ → 𝓐 × 𝓨) : Fin n → 𝓐 × 𝓨 := fun i ↦ h i
+/-- `hist n` is the history before time `n`: the rounds at times `0, ..., n - 1`.
+This is a random variable on the measurable space `ℕ → Round 𝓞 𝓐 𝓨`. -/
+def hist (n : ℕ) (h : ℕ → Round 𝓞 𝓐 𝓨) : Hist 𝓞 𝓐 𝓨 n := fun i ↦ h i
 
-lemma fst_comp_step (n : ℕ) : Prod.fst ∘ step (𝓐 := 𝓐) (𝓨 := 𝓨) n = action n := rfl
+lemma fst_comp_step (n : ℕ) : Prod.fst ∘ step (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨) n = obs n := rfl
 
-lemma history_action_feedback (n : ℕ) :
-    history (action (𝓐 := 𝓐) (𝓨 := 𝓨)) feedback n = hist n := rfl
+lemma history_obs_action_feedback (n : ℕ) :
+    history (obs (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨)) action feedback n = hist n := rfl
 
 @[simp]
-lemma hist_zero : hist (𝓐 := 𝓐) (𝓨 := 𝓨) 0 = fun _ ↦ default := Unique.eq_default _
+lemma hist_zero : hist (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨) 0 = fun _ ↦ default := Unique.eq_default _
 
 @[fun_prop]
-lemma measurable_step (n : ℕ) : Measurable (step n (𝓐 := 𝓐) (𝓨 := 𝓨)) := by
+lemma measurable_step (n : ℕ) : Measurable (step n (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨)) := by
   unfold step; fun_prop
 
 @[fun_prop]
-lemma measurable_step_prod : Measurable (fun p : ℕ × (ℕ → 𝓐 × 𝓨) ↦ step p.1 p.2) :=
+lemma measurable_step_prod : Measurable (fun p : ℕ × (ℕ → Round 𝓞 𝓐 𝓨) ↦ step p.1 p.2) :=
+  measurable_from_prod_countable_right fun n ↦ (by simp only; fun_prop)
+
+@[fun_prop]
+lemma measurable_obs (n : ℕ) : Measurable (obs n (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨)) := by
+  unfold obs; fun_prop
+
+@[fun_prop]
+lemma measurable_obs_prod : Measurable (fun p : ℕ × (ℕ → Round 𝓞 𝓐 𝓨) ↦ obs p.1 p.2) :=
   measurable_from_prod_countable_right fun n ↦ (by simp only; fun_prop)
 
 @[fun_prop]
-lemma measurable_action (n : ℕ) : Measurable (action n (𝓐 := 𝓐) (𝓨 := 𝓨)) := by
+lemma measurable_action (n : ℕ) : Measurable (action n (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨)) := by
   unfold action; fun_prop
 
 @[fun_prop]
-lemma measurable_action_prod : Measurable (fun p : ℕ × (ℕ → 𝓐 × 𝓨) ↦ action p.1 p.2) :=
+lemma measurable_action_prod : Measurable (fun p : ℕ × (ℕ → Round 𝓞 𝓐 𝓨) ↦ action p.1 p.2) :=
   measurable_from_prod_countable_right fun n ↦ (by simp only; fun_prop)
 
 @[fun_prop]
-lemma measurable_feedback (n : ℕ) : Measurable (feedback n (𝓐 := 𝓐) (𝓨 := 𝓨)) := by
+lemma measurable_feedback (n : ℕ) : Measurable (feedback n (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨)) := by
   unfold feedback; fun_prop
 
 @[fun_prop]
-lemma measurable_feedback_prod : Measurable (fun p : ℕ × (ℕ → 𝓐 × 𝓨) ↦ feedback p.1 p.2) :=
+lemma measurable_feedback_prod : Measurable (fun p : ℕ × (ℕ → Round 𝓞 𝓐 𝓨) ↦ feedback p.1 p.2) :=
   measurable_from_prod_countable_right fun n ↦ (by simp only; fun_prop)
 
 @[fun_prop]
-lemma measurable_hist (n : ℕ) : Measurable (hist n (𝓐 := 𝓐) (𝓨 := 𝓨)) := by unfold hist; fun_prop
+lemma measurable_hist (n : ℕ) : Measurable (hist n (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨)) := by
+  unfold hist; fun_prop
 
 lemma hist_succ_eq_comp_frestrictLe (n : ℕ) :
-    hist (𝓐 := 𝓐) (𝓨 := 𝓨) (n + 1) =
-      (MeasurableEquiv.finSuccPiIic (fun _ ↦ 𝓐 × 𝓨) n).symm ∘ Preorder.frestrictLe n := rfl
+    hist (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨) (n + 1) =
+      (MeasurableEquiv.finSuccPiIic (fun _ ↦ Round 𝓞 𝓐 𝓨) n).symm ∘ Preorder.frestrictLe n := rfl
 
 lemma frestrictLe_eq_comp_hist (n : ℕ) :
-    Preorder.frestrictLe («π» := fun _ ↦ 𝓐 × 𝓨) n =
-      (MeasurableEquiv.finSuccPiIic (fun _ ↦ 𝓐 × 𝓨) n) ∘ hist (n + 1) := rfl
+    Preorder.frestrictLe («π» := fun _ ↦ Round 𝓞 𝓐 𝓨) n =
+      (MeasurableEquiv.finSuccPiIic (fun _ ↦ Round 𝓞 𝓐 𝓨) n) ∘ hist (n + 1) := rfl
 
 lemma step_eq_eval_comp_hist (n : ℕ) :
-    step (𝓐 := 𝓐) (𝓨 := 𝓨) n = (fun x ↦ x (Fin.last n)) ∘ (hist (n + 1)) := rfl
+    step (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨) n = (fun x ↦ x (Fin.last n)) ∘ (hist (n + 1)) := rfl
+
+lemma obs_eq_eval_comp_hist (n : ℕ) :
+    obs (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨) n = (fun x ↦ (x (Fin.last n)).obs) ∘ (hist (n + 1)) := rfl
 
 lemma action_eq_eval_comp_hist (n : ℕ) :
-    action (𝓐 := 𝓐) (𝓨 := 𝓨) n = (fun x ↦ (x (Fin.last n)).1) ∘ (hist (n + 1)) := rfl
+    action (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨) n = (fun x ↦ (x (Fin.last n)).action) ∘ (hist (n + 1)) := rfl
 
 lemma feedback_eq_eval_comp_hist (n : ℕ) :
-    feedback (𝓐 := 𝓐) (𝓨 := 𝓨) n = (fun x ↦ (x (Fin.last n)).2) ∘ (hist (n + 1)) := rfl
+    feedback (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨) n =
+      (fun x ↦ (x (Fin.last n)).feedback) ∘ (hist (n + 1)) := rfl
 
 /-- The history before time `m` is a restriction of the history before time `n ≥ m`. -/
 lemma hist_eq_comp_hist {m n : ℕ} (hmn : m ≤ n) :
-    hist (𝓐 := 𝓐) (𝓨 := 𝓨) m = (fun h (i : Fin m) ↦ h (Fin.castLE hmn i)) ∘ hist n := rfl
+    hist (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨) m = (fun h (i : Fin m) ↦ h (Fin.castLE hmn i)) ∘ hist n := rfl
 
 end IT
 
 section ModelEquivalence
 
 variable {Ω Ω' : Type*} {mΩ : MeasurableSpace Ω} {mΩ' : MeasurableSpace Ω'}
-  {alg : Algorithm 𝓐 𝓨} {env : Environment 𝓐 𝓨}
+  {alg : Algorithm 𝓞 𝓐 𝓨} {env : Environment 𝓞 𝓐 𝓨}
   {P : Measure Ω} [IsProbabilityMeasure P] {P' : Measure Ω'} [IsProbabilityMeasure P']
-  {A₁ : ℕ → Ω → 𝓐} {R₁ : ℕ → Ω → 𝓨} {A₂ : ℕ → Ω' → 𝓐} {R₂ : ℕ → Ω' → 𝓨} {N : ℕ}
+  {O₁ : ℕ → Ω → 𝓞} {A₁ : ℕ → Ω → 𝓐} {R₁ : ℕ → Ω → 𝓨}
+  {O₂ : ℕ → Ω' → 𝓞} {A₂ : ℕ → Ω' → 𝓐} {R₂ : ℕ → Ω' → 𝓨} {N : ℕ}
 
-lemma IsAlgEnvSeq.map_trajectory (h : IsAlgEnvSeq A₁ R₁ alg env P) :
-    P.map (trajectory A₁ R₁) = trajMeasure alg env :=
-  (Kernel.hasLaw_trajMeasureFin (X := fun _ ↦ 𝓐 × 𝓨) (Y := step A₁ R₁) (P := P)
+lemma IsAlgEnvSeq.map_trajectory (h : IsAlgEnvSeq O₁ A₁ R₁ alg env P) :
+    P.map (trajectory O₁ A₁ R₁) = trajMeasure alg env :=
+  (Kernel.hasLaw_trajMeasureFin (X := fun _ ↦ Round 𝓞 𝓐 𝓨) (Y := step O₁ A₁ R₁) (P := P)
     (κ' := stepKernel alg env) h.measurable_step h.hasCondDistrib_step).map_eq
 
-lemma IsAlgEnvSeq.hasLaw_trajectory (h : IsAlgEnvSeq A₁ R₁ alg env P) :
-    HasLaw (trajectory A₁ R₁) (trajMeasure alg env) P where
+lemma IsAlgEnvSeq.hasLaw_trajectory (h : IsAlgEnvSeq O₁ A₁ R₁ alg env P) :
+    HasLaw (trajectory O₁ A₁ R₁) (trajMeasure alg env) P where
   aemeasurable := h.measurable_trajectory.aemeasurable
   map_eq := h.map_trajectory
 
-lemma IsAlgEnvSeqUntil.map_history (h : IsAlgEnvSeqUntil A₁ R₁ alg env P N) :
-    P.map (history A₁ R₁ N) = (trajMeasure alg env).map (IT.hist N) :=
-  Kernel.eq_trajMeasureFin_map (X := fun _ ↦ 𝓐 × 𝓨) (Y := step A₁ R₁) (P := P) (N := N)
+lemma IsAlgEnvSeqUntil.map_history (h : IsAlgEnvSeqUntil O₁ A₁ R₁ alg env P N) :
+    P.map (history O₁ A₁ R₁ N) = (trajMeasure alg env).map (IT.hist N) :=
+  Kernel.eq_trajMeasureFin_map (X := fun _ ↦ Round 𝓞 𝓐 𝓨) (Y := step O₁ A₁ R₁) (P := P) (N := N)
     (κ' := stepKernel alg env) h.measurable_step h.hasCondDistrib_step
 
-lemma IsAlgEnvSeq.map_history (h : IsAlgEnvSeq A₁ R₁ alg env P) (n : ℕ) :
-    P.map (history A₁ R₁ n) = (trajMeasure alg env).map (IT.hist n) :=
+lemma IsAlgEnvSeq.map_history (h : IsAlgEnvSeq O₁ A₁ R₁ alg env P) (n : ℕ) :
+    P.map (history O₁ A₁ R₁ n) = (trajMeasure alg env).map (IT.hist n) :=
   (h.isAlgEnvSeqUntil n).map_history
 
-/-- The law of the sequence of actions and observations generated by an algorithm-environment pair
+/-- The law of the sequence of rounds generated by an algorithm-environment pair
 is unique: it does not depend on the probability space used. -/
-lemma isAlgEnvSeq_unique (h1 : IsAlgEnvSeq A₁ R₁ alg env P)
-    (h2 : IsAlgEnvSeq A₂ R₂ alg env P') :
-    P.map (trajectory A₁ R₁) = P'.map (trajectory A₂ R₂) := by
+lemma isAlgEnvSeq_unique (h1 : IsAlgEnvSeq O₁ A₁ R₁ alg env P)
+    (h2 : IsAlgEnvSeq O₂ A₂ R₂ alg env P') :
+    P.map (trajectory O₁ A₁ R₁) = P'.map (trajectory O₂ A₂ R₂) := by
   rw [h1.map_trajectory, h2.map_trajectory]
 
-/-- The law of the sequence of actions and observations generated by an algorithm-environment pair
+/-- The law of the sequence of rounds generated by an algorithm-environment pair
 is unique: it does not depend on the probability space used. -/
-theorem IsAlgEnvSeq.identDistrib_trajectory (h1 : IsAlgEnvSeq A₁ R₁ alg env P)
-    (h2 : IsAlgEnvSeq A₂ R₂ alg env P') :
-    IdentDistrib (trajectory A₁ R₁) (trajectory A₂ R₂) P P' where
+theorem IsAlgEnvSeq.identDistrib_trajectory (h1 : IsAlgEnvSeq O₁ A₁ R₁ alg env P)
+    (h2 : IsAlgEnvSeq O₂ A₂ R₂ alg env P') :
+    IdentDistrib (trajectory O₁ A₁ R₁) (trajectory O₂ A₂ R₂) P P' where
   aemeasurable_fst := h1.measurable_trajectory.aemeasurable
   aemeasurable_snd := h2.measurable_trajectory.aemeasurable
   map_eq := isAlgEnvSeq_unique h1 h2
 
 /-- The law of the history before time `N` is determined by the algorithm-environment pair,
 provided that the sequence is an algorithm-environment sequence until time `N`. -/
-lemma isAlgEnvSeqUntil_unique (h1 : IsAlgEnvSeqUntil A₁ R₁ alg env P N)
-    (h2 : IsAlgEnvSeqUntil A₂ R₂ alg env P' N) :
-    P.map (history A₁ R₁ N) = P'.map (history A₂ R₂ N) := by
+lemma isAlgEnvSeqUntil_unique (h1 : IsAlgEnvSeqUntil O₁ A₁ R₁ alg env P N)
+    (h2 : IsAlgEnvSeqUntil O₂ A₂ R₂ alg env P' N) :
+    P.map (history O₁ A₁ R₁ N) = P'.map (history O₂ A₂ R₂ N) := by
   rw [h1.map_history, h2.map_history]
 
 end ModelEquivalence
 
 namespace IT
 
-/-- Filtration of the algorithm Seq: `IT.filtration 𝓐 𝓨 n` is the σ-algebra generated by the
-action-feedback pairs at times `0, ..., n`, that is by `hist (n + 1)`. -/
-protected def filtration (𝓐 𝓨 : Type*) [MeasurableSpace 𝓐] [MeasurableSpace 𝓨] :
-    Filtration ℕ (inferInstance : MeasurableSpace (ℕ → 𝓐 × 𝓨)) :=
-  MeasureTheory.Filtration.piLE (X := fun _ ↦ 𝓐 × 𝓨)
+/-- Filtration of the algorithm Seq: `IT.filtration 𝓞 𝓐 𝓨 n` is the σ-algebra generated by the
+rounds at times `0, ..., n`, that is by `hist (n + 1)`. -/
+protected def filtration (𝓞 𝓐 𝓨 : Type*) [MeasurableSpace 𝓞] [MeasurableSpace 𝓐]
+    [MeasurableSpace 𝓨] :
+    Filtration ℕ (inferInstance : MeasurableSpace (ℕ → Round 𝓞 𝓐 𝓨)) :=
+  MeasureTheory.Filtration.piLE (X := fun _ ↦ Round 𝓞 𝓐 𝓨)
 
 lemma filtration_eq_comap (n : ℕ) :
-    IT.filtration 𝓐 𝓨 n = MeasurableSpace.comap (hist (n + 1)) inferInstance := by
+    IT.filtration 𝓞 𝓐 𝓨 n = MeasurableSpace.comap (hist (n + 1)) inferInstance := by
   simp only [IT.filtration, Filtration.piLE_eq_comap_frestrictLe]
   refine le_antisymm ?_ ?_
   · rw [← measurable_iff_comap_le, frestrictLe_eq_comp_hist]
@@ -185,46 +205,54 @@ lemma filtration_eq_comap (n : ℕ) :
   · rw [← measurable_iff_comap_le, hist_succ_eq_comp_frestrictLe]
     exact measurable_comp_comap _ (by fun_prop)
 
-lemma adapted_step : Adapted (IT.filtration 𝓐 𝓨) (step (𝓐 := 𝓐) (𝓨 := 𝓨)) := by
+lemma adapted_step : Adapted (IT.filtration 𝓞 𝓐 𝓨) (step (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨)) := by
   intro n
   rw [filtration_eq_comap, step_eq_eval_comp_hist]
   exact measurable_comp_comap _ (by fun_prop)
 
+lemma adapted_obs : Adapted (IT.filtration 𝓞 𝓐 𝓨) obs := by
+  intro n
+  rw [filtration_eq_comap, obs_eq_eval_comp_hist]
+  exact measurable_comp_comap _ (by fun_prop)
+
 lemma measurable_hist_succ_filtration (n : ℕ) :
-    Measurable[IT.filtration 𝓐 𝓨 n] (hist (n + 1)) := by
+    Measurable[IT.filtration 𝓞 𝓐 𝓨 n] (hist (n + 1)) := by
   rw [filtration_eq_comap]
   exact measurable_iff_comap_le.mpr le_rfl
 
-lemma adapted_hist : Adapted (IT.filtration 𝓐 𝓨) hist := by
+lemma adapted_hist : Adapted (IT.filtration 𝓞 𝓐 𝓨) hist := by
   intro n
   rw [filtration_eq_comap, hist_eq_comp_hist n.le_succ]
   exact measurable_comp_comap _ (by fun_prop)
 
-lemma adapted_action : Adapted (IT.filtration 𝓐 𝓨) action := by
+lemma adapted_action : Adapted (IT.filtration 𝓞 𝓐 𝓨) action := by
   intro n
   rw [filtration_eq_comap, action_eq_eval_comp_hist]
   exact measurable_comp_comap _ (by fun_prop)
 
-lemma adapted_feedback : Adapted (IT.filtration 𝓐 𝓨) feedback := by
+lemma adapted_feedback : Adapted (IT.filtration 𝓞 𝓐 𝓨) feedback := by
   intro n
   rw [filtration_eq_comap, feedback_eq_eval_comp_hist]
   exact measurable_comp_comap _ (by fun_prop)
 
 section FiltrationAction
 
-/-- Filtration generated by the history before time `n` together with the action at time `n`. -/
-def filtrationAction (𝓐 𝓨 : Type*) [MeasurableSpace 𝓐] [MeasurableSpace 𝓨] :
-    Filtration ℕ (inferInstance : MeasurableSpace (ℕ → 𝓐 × 𝓨)) where
-  seq n := MeasurableSpace.comap (fun ω ↦ (hist n ω, action n ω)) inferInstance
+/-- Filtration generated by the history before time `n` together with the observation and the
+action at time `n`. -/
+def filtrationAction (𝓞 𝓐 𝓨 : Type*) [MeasurableSpace 𝓞] [MeasurableSpace 𝓐]
+    [MeasurableSpace 𝓨] :
+    Filtration ℕ (inferInstance : MeasurableSpace (ℕ → Round 𝓞 𝓐 𝓨)) where
+  seq n := MeasurableSpace.comap (fun ω ↦ ((hist n ω, obs n ω), action n ω)) inferInstance
   mono' n m hnm := by
     simp only
     rw [← measurable_iff_comap_le]
     rcases eq_or_lt_of_le hnm with rfl | hlt
     · exact measurable_iff_comap_le.mpr le_rfl
-    have : (fun ω ↦ (hist (𝓐 := 𝓐) (𝓨 := 𝓨) n ω, action n ω)) =
-        (fun p : (Fin m → 𝓐 × 𝓨) × 𝓐 ↦
-          (fun i : Fin n ↦ p.1 (Fin.castLE hnm i), (p.1 ⟨n, hlt⟩).1)) ∘
-        (fun ω ↦ (hist m ω, action m ω)) := rfl
+    have : (fun ω ↦ ((hist (𝓞 := 𝓞) (𝓐 := 𝓐) (𝓨 := 𝓨) n ω, obs n ω), action n ω)) =
+        (fun p : (Hist 𝓞 𝓐 𝓨 m × 𝓞) × 𝓐 ↦
+          ((fun i : Fin n ↦ p.1.1 (Fin.castLE hnm i), (p.1.1 ⟨n, hlt⟩).obs),
+            (p.1.1 ⟨n, hlt⟩).action)) ∘
+        (fun ω ↦ ((hist m ω, obs m ω), action m ω)) := rfl
     rw [this]
     exact measurable_comp_comap _ (by fun_prop)
   le' n := by
@@ -232,108 +260,105 @@ def filtrationAction (𝓐 𝓨 : Type*) [MeasurableSpace 𝓐] [MeasurableSpace
     fun_prop
 
 lemma filtrationAction_eq_comap (n : ℕ) :
-    filtrationAction 𝓐 𝓨 n =
-      MeasurableSpace.comap (fun ω ↦ (hist n ω, action n ω)) inferInstance := rfl
+    filtrationAction 𝓞 𝓐 𝓨 n =
+      MeasurableSpace.comap (fun ω ↦ ((hist n ω, obs n ω), action n ω)) inferInstance := rfl
 
 lemma filtrationAction_zero_eq_comap :
-    filtrationAction 𝓐 𝓨 0 = MeasurableSpace.comap (action 0) inferInstance := by
+    filtrationAction 𝓞 𝓐 𝓨 0 =
+      MeasurableSpace.comap (fun ω ↦ (obs 0 ω, action 0 ω)) inferInstance := by
   rw [filtrationAction_eq_comap, hist_zero]
   refine le_antisymm ?_ ?_
   · rw [← measurable_iff_comap_le]
-    exact measurable_const.prodMk (measurable_iff_comap_le.mpr le_rfl)
+    exact (measurable_const.prodMk
+      (measurable_fst.comp (measurable_iff_comap_le.mpr le_rfl))).prodMk
+      (measurable_snd.comp (measurable_iff_comap_le.mpr le_rfl))
   · rw [← measurable_iff_comap_le]
-    exact measurable_snd.comp (measurable_iff_comap_le.mpr le_rfl)
+    exact ((measurable_snd.comp measurable_fst).comp
+      (measurable_iff_comap_le.mpr le_rfl)).prodMk
+      (measurable_snd.comp (measurable_iff_comap_le.mpr le_rfl))
 
 lemma measurable_hist_filtrationAction (n : ℕ) :
-    Measurable[filtrationAction 𝓐 𝓨 n] (hist n) :=
-  measurable_fst.comp (measurable_iff_comap_le.mpr le_rfl)
+    Measurable[filtrationAction 𝓞 𝓐 𝓨 n] (hist n) :=
+  (measurable_fst.comp measurable_fst).comp (measurable_iff_comap_le.mpr le_rfl)
 
 lemma filtration_le_filtrationAction_add_one (n : ℕ) :
-    IT.filtration 𝓐 𝓨 n ≤ filtrationAction 𝓐 𝓨 (n + 1) := by
+    IT.filtration 𝓞 𝓐 𝓨 n ≤ filtrationAction 𝓞 𝓐 𝓨 (n + 1) := by
   rw [filtration_eq_comap]
   exact measurable_iff_comap_le.mp (measurable_hist_filtrationAction (n + 1))
 
 lemma filtration_le_filtrationAction {m n : ℕ} (h : n < m) :
-    IT.filtration 𝓐 𝓨 n ≤ filtrationAction 𝓐 𝓨 m := by
+    IT.filtration 𝓞 𝓐 𝓨 n ≤ filtrationAction 𝓞 𝓐 𝓨 m := by
   have h' : n + 1 ≤ m := by grind
-  exact (filtration_le_filtrationAction_add_one n).trans ((filtrationAction 𝓐 𝓨).mono h')
+  exact (filtration_le_filtrationAction_add_one n).trans ((filtrationAction 𝓞 𝓐 𝓨).mono h')
 
 lemma filtrationAction_le_filtration_self (n : ℕ) :
-    filtrationAction 𝓐 𝓨 n ≤ IT.filtration 𝓐 𝓨 n := by
+    filtrationAction 𝓞 𝓐 𝓨 n ≤ IT.filtration 𝓞 𝓐 𝓨 n := by
   rw [filtrationAction_eq_comap, ← measurable_iff_comap_le]
-  exact (adapted_hist n).prodMk (adapted_action n)
+  exact ((adapted_hist n).prodMk (adapted_obs n)).prodMk (adapted_action n)
 
 lemma filtrationAction_le_filtration {m n : ℕ} (h : m ≤ n) :
-    filtrationAction 𝓐 𝓨 m ≤ IT.filtration 𝓐 𝓨 n :=
-  (filtrationAction_le_filtration_self m).trans ((IT.filtration 𝓐 𝓨).mono h)
+    filtrationAction 𝓞 𝓐 𝓨 m ≤ IT.filtration 𝓞 𝓐 𝓨 n :=
+  (filtrationAction_le_filtration_self m).trans ((IT.filtration 𝓞 𝓐 𝓨).mono h)
+
+lemma measurable_obs_filtrationAction (n : ℕ) :
+    Measurable[filtrationAction 𝓞 𝓐 𝓨 n] (obs n) :=
+  (measurable_snd.comp measurable_fst).comp (measurable_iff_comap_le.mpr le_rfl)
 
 lemma measurable_action_filtrationAction (n : ℕ) :
-    Measurable[filtrationAction 𝓐 𝓨 n] (action n) :=
+    Measurable[filtrationAction 𝓞 𝓐 𝓨 n] (action n) :=
   measurable_snd.comp (measurable_iff_comap_le.mpr le_rfl)
 
 end FiltrationAction
 
 section Laws
 
-lemma hasCondDistrib_step (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) (n : ℕ) :
+lemma hasCondDistrib_step (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) (n : ℕ) :
     HasCondDistrib (step n) (hist n) (stepKernel alg env n) (trajMeasure alg env) :=
   Kernel.hasCondDistrib_trajMeasureFin n
 
-lemma hasCondDistrib_action (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) (n : ℕ) :
-    HasCondDistrib (action n) (hist n) (alg.policy n) (trajMeasure alg env) := by
+lemma hasCondDistrib_obs (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) (n : ℕ) :
+    HasCondDistrib (obs n) (hist n) (env.obs n) (trajMeasure alg env) := by
   rw [← fst_comp_step, ← fst_stepKernel, Kernel.fst_eq]
   exact HasCondDistrib.comp_left (hasCondDistrib_step alg env n) measurable_fst
 
-lemma hasCondDistrib_feedback (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) (n : ℕ) :
-    HasCondDistrib (feedback n) (fun ω ↦ (hist n ω, action n ω)) (env.feedback n)
+/-- The action and the feedback at time `n`, given the history and the observation at time `n`. -/
+lemma hasCondDistrib_action_feedback (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) (n : ℕ) :
+    HasCondDistrib (fun ω ↦ (action n ω, feedback n ω)) (fun ω ↦ (hist n ω, obs n ω))
+      (alg.policy n ⊗ₖ env.feedback n) (trajMeasure alg env) :=
+  HasCondDistrib.of_compProd (hasCondDistrib_step alg env n)
+
+lemma hasCondDistrib_action (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) (n : ℕ) :
+    HasCondDistrib (action n) (fun ω ↦ (hist n ω, obs n ω)) (alg.policy n)
       (trajMeasure alg env) := by
-  have h_step := hasCondDistrib_step alg env n
-  have h_action := hasCondDistrib_action alg env n
-  refine ⟨by fun_prop, ?_⟩
-  rw [h_action.map_eq, ← Measure.compProd_assoc, ← stepKernel, ← h_step.map_eq,
-    Measure.map_map (by fun_prop) (by fun_prop)]
-  rfl
-
-lemma isAlgEnvSeq_trajMeasure (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) :
-    IsAlgEnvSeq action feedback alg env (trajMeasure alg env) where
+  have h := (hasCondDistrib_action_feedback alg env n).fst
+  rwa [Kernel.fst_compProd] at h
+
+lemma hasCondDistrib_feedback (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) (n : ℕ) :
+    HasCondDistrib (feedback n) (fun ω ↦ ((hist n ω, obs n ω), action n ω)) (env.feedback n)
+      (trajMeasure alg env) :=
+  HasCondDistrib.of_compProd (hasCondDistrib_action_feedback alg env n)
+
+lemma isAlgEnvSeq_trajMeasure (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) :
+    IsAlgEnvSeq obs action feedback alg env (trajMeasure alg env) where
+  hasCondDistrib_obs n := hasCondDistrib_obs alg env n
   hasCondDistrib_action n := hasCondDistrib_action alg env n
   hasCondDistrib_feedback n := hasCondDistrib_feedback alg env n
 
-lemma hasLaw_step_zero (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) :
-    HasLaw (step 0) (alg.p0 ⊗ₘ env.ν0) (trajMeasure alg env) :=
+lemma hasLaw_step_zero (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) :
+    HasLaw (step 0) (env.obs0 ⊗ₘ (alg.p0 ⊗ₖ env.ν0)) (trajMeasure alg env) :=
   (isAlgEnvSeq_trajMeasure alg env).hasLaw_step_zero
 
-lemma hasLaw_action_zero (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) :
-    HasLaw (action 0) alg.p0 (trajMeasure alg env) :=
-  (isAlgEnvSeq_trajMeasure alg env).hasLaw_action_zero
+lemma hasLaw_obs_zero (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) :
+    HasLaw (obs 0) env.obs0 (trajMeasure alg env) :=
+  (isAlgEnvSeq_trajMeasure alg env).hasLaw_obs_zero
 
-lemma hasCondDistrib_feedback_zero (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) :
-    HasCondDistrib (feedback 0) (action 0) env.ν0 (trajMeasure alg env) :=
-  (isAlgEnvSeq_trajMeasure alg env).hasCondDistrib_feedback_zero
+lemma hasCondDistrib_action_zero (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) :
+    HasCondDistrib (action 0) (obs 0) alg.p0 (trajMeasure alg env) :=
+  (isAlgEnvSeq_trajMeasure alg env).hasCondDistrib_action_zero
 
-lemma condDistrib_feedback_zero [StandardBorelSpace 𝓨] [Nonempty 𝓨]
-    (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) :
-    condDistrib (feedback 0) (action 0) (trajMeasure alg env)
-      =ᵐ[(trajMeasure alg env).map (action 0)] env.ν0 :=
-  (hasCondDistrib_feedback_zero alg env).condDistrib_eq
-
-lemma condDistrib_step [StandardBorelSpace 𝓐] [Nonempty 𝓐] [StandardBorelSpace 𝓨] [Nonempty 𝓨]
-    (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) (n : ℕ) :
-    condDistrib (step n) (hist n) (trajMeasure alg env)
-      =ᵐ[(trajMeasure alg env).map (hist n)] stepKernel alg env n :=
-  (hasCondDistrib_step alg env n).condDistrib_eq
-
-lemma condDistrib_action [StandardBorelSpace 𝓐] [Nonempty 𝓐]
-    (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) (n : ℕ) :
-    condDistrib (action n) (hist n) (trajMeasure alg env)
-      =ᵐ[(trajMeasure alg env).map (hist n)] alg.policy n :=
-  (hasCondDistrib_action alg env n).condDistrib_eq
-
-lemma condDistrib_feedback [StandardBorelSpace 𝓨] [Nonempty 𝓨]
-    (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨) (n : ℕ) :
-    condDistrib (feedback n) (fun ω ↦ (hist n ω, action n ω)) (trajMeasure alg env)
-      =ᵐ[(trajMeasure alg env).map (fun ω ↦ (hist n ω, action n ω))] env.feedback n :=
-  (hasCondDistrib_feedback alg env n).condDistrib_eq
+lemma hasCondDistrib_feedback_zero (alg : Algorithm 𝓞 𝓐 𝓨) (env : Environment 𝓞 𝓐 𝓨) :
+    HasCondDistrib (feedback 0) (fun ω ↦ (obs 0 ω, action 0 ω)) env.ν0 (trajMeasure alg env) :=
+  (isAlgEnvSeq_trajMeasure alg env).hasCondDistrib_feedback_zero
 
 end Laws
 
diff --git a/LeanMachineLearning/SequentialLearning/Means.lean b/LeanMachineLearning/SequentialLearning/Means.lean
index f583c1f5..6207a34f 100644
--- a/LeanMachineLearning/SequentialLearning/Means.lean
+++ b/LeanMachineLearning/SequentialLearning/Means.lean
@@ -45,102 +45,106 @@ end ProbabilityTheory
 
 namespace Learning
 
-variable {Ω 𝓐 𝓨 : Type*} {mΩ : MeasurableSpace Ω} {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨}
+variable {Ω 𝓞 𝓐 𝓨 : Type*} {mΩ : MeasurableSpace Ω} {m𝓞 : MeasurableSpace 𝓞}
+  {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨}
   [NormedAddCommGroup 𝓨] [NormedSpace ℝ 𝓨]
-  {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {P : Measure Ω} [IsFiniteMeasure P]
-  {alg : Algorithm 𝓐 𝓨} {env : Environment 𝓐 𝓨}
+  {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {P : Measure Ω} [IsFiniteMeasure P]
+  {alg : Algorithm 𝓞 𝓐 𝓨} {env : Environment 𝓞 𝓐 𝓨}
 
 /-- The kernel that gives the measure of the feedback distribution as a function of the action
 chosen at time `n`. -/
-noncomputable def Environment.measure (env : Environment 𝓐 𝓨) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨)
-    (n : ℕ) (ω : Ω) : Kernel 𝓐 𝓨 :=
-  (env.feedback n).sectR (history A Y n ω)
+noncomputable def Environment.measure (env : Environment 𝓞 𝓐 𝓨) (O : ℕ → Ω → 𝓞) (A : ℕ → Ω → 𝓐)
+    (Y : ℕ → Ω → 𝓨) (n : ℕ) (ω : Ω) : Kernel 𝓐 𝓨 :=
+  (env.feedback n).sectR (history O A Y n ω, O n ω)
 
 /-- The means of the feedback distribution as a function of the action chosen at time `n`. -/
-noncomputable def Environment.means (env : Environment 𝓐 𝓨) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨)
-    (k : 𝓐) (n : ℕ) (ω : Ω) : 𝓨 :=
-  (env.measure A Y n ω k)[id]
+noncomputable def Environment.means (env : Environment 𝓞 𝓐 𝓨) (O : ℕ → Ω → 𝓞) (A : ℕ → Ω → 𝓐)
+    (Y : ℕ → Ω → 𝓨) (k : 𝓐) (n : ℕ) (ω : Ω) : 𝓨 :=
+  (env.measure O A Y n ω k)[id]
 
 @[simp]
-lemma means_zero (env : Environment 𝓐 𝓨) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨)
+lemma means_zero (env : Environment 𝓞 𝓐 𝓨) (O : ℕ → Ω → 𝓞) (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨)
     (k : 𝓐) (ω : Ω) :
-    env.means A Y k 0 ω = (env.ν0 k)[id] := by
+    env.means O A Y k 0 ω = (env.ν0 (O 0 ω, k))[id] := by
   simp [Environment.means, Environment.measure, Environment.feedback_zero]
 
 @[simp]
-lemma means_of_isObliviousEnv [IsObliviousEnv env] (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨)
-    (k : 𝓐) (n : ℕ) (ω : Ω) :
-    env.means A Y k n ω = (feedbackCondAction env n k)[id] := by
+lemma means_of_isObliviousEnv [IsObliviousEnv env] (O : ℕ → Ω → 𝓞) (A : ℕ → Ω → 𝓐)
+    (Y : ℕ → Ω → 𝓨) (k : 𝓐) (n : ℕ) (ω : Ω) :
+    env.means O A Y k n ω = (feedbackCondAction env n k)[id] := by
   simp [Environment.means, Environment.measure, feedback_eq_feedbackCondAction]
 
 lemma means_obliviousEnv (ν : ℕ → Kernel 𝓐 𝓨) [∀ n, IsMarkovKernel (ν n)]
-    (k : 𝓐) (n : ℕ) (ω : Ω) :
-    (obliviousEnv ν).means A Y k n ω = (ν n k)[id] := by simp
+    {O : ℕ → Ω → Unit} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} (k : 𝓐) (n : ℕ) (ω : Ω) :
+    (obliviousEnv ν).means O A Y k n ω = (ν n k)[id] := by simp
 
-lemma means_stationaryEnv (ν : Kernel 𝓐 𝓨) [IsMarkovKernel ν] (k : 𝓐) (n : ℕ) (ω : Ω) :
-    (stationaryEnv ν).means A Y k n ω = (ν k)[id] := by simp
+lemma means_stationaryEnv (ν : Kernel 𝓐 𝓨) [IsMarkovKernel ν]
+    {O : ℕ → Ω → Unit} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} (k : 𝓐) (n : ℕ) (ω : Ω) :
+    (stationaryEnv ν).means O A Y k n ω = (ν k)[id] := by simp
 
 @[fun_prop]
 lemma IsAlgEnvSeq.stronglyMeasurable_means [SecondCountableTopology 𝓨] [OpensMeasurableSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) (k : 𝓐) (n : ℕ) :
-    StronglyMeasurable (env.means A Y k n) := by
+    (h : IsAlgEnvSeq O A Y alg env P) (k : 𝓐) (n : ℕ) :
+    StronglyMeasurable (env.means O A Y k n) := by
   unfold Environment.means
-  have h_eq ω : env.measure A Y n ω k =
-      (env.feedback n ∘ₖ Kernel.deterministic (fun ω ↦ (history A Y n ω, k))
-        ((h.measurable_history n).prodMk (by fun_prop))) ω := by
+  have hO := h.measurable_obs
+  have h_eq ω : env.measure O A Y n ω k =
+      (env.feedback n ∘ₖ Kernel.deterministic (fun ω ↦ ((history O A Y n ω, O n ω), k))
+        (((h.measurable_history n).prodMk (h.measurable_obs n)).prodMk (by fun_prop))) ω := by
     simp [Environment.measure, Kernel.comp_deterministic_eq_comap]
   simp_rw [h_eq]
   fun_prop
 
 @[fun_prop]
 lemma IsAlgEnvSeq.measurable_means [SecondCountableTopology 𝓨] [BorelSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) (k : 𝓐) (n : ℕ) :
-    Measurable (env.means A Y k n) :=
+    (h : IsAlgEnvSeq O A Y alg env P) (k : 𝓐) (n : ℕ) :
+    Measurable (env.means O A Y k n) :=
   (h.stronglyMeasurable_means k n).measurable
 
 lemma IsAlgEnvSeq.adapted_means_filtrationAction [SecondCountableTopology 𝓨] [BorelSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) :
-    Adapted h.filtrationAction (fun n ω ↦ env.means A Y (A n ω) n ω) := by
+    (h : IsAlgEnvSeq O A Y alg env P) :
+    Adapted h.filtrationAction (fun n ω ↦ env.means O A Y (A n ω) n ω) := by
   intro n
   simp only [Environment.means, Environment.measure, Kernel.sectR_apply, id_eq]
   change Measurable[h.filtrationAction n]
-    ((fun ω ↦ ∫ x, x ∂(env.feedback n ω)) ∘ (fun ω ↦ (history A Y n ω, A n ω)))
+    ((fun ω ↦ ∫ x, x ∂(env.feedback n ω)) ∘ (fun ω ↦ ((history O A Y n ω, O n ω), A n ω)))
   rw [IsAlgEnvSeq.filtrationAction_eq_comap]
   exact measurable_comp_comap _ stronglyMeasurable_id.integral_kernel.measurable
 
 lemma IsAlgEnvSeq.stronglyAdapted_means_filtrationAction [SecondCountableTopology 𝓨] [BorelSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) :
-    StronglyAdapted h.filtrationAction (fun n ω ↦ env.means A Y (A n ω) n ω) :=
+    (h : IsAlgEnvSeq O A Y alg env P) :
+    StronglyAdapted h.filtrationAction (fun n ω ↦ env.means O A Y (A n ω) n ω) :=
   (h.adapted_means_filtrationAction).stronglyAdapted
 
 lemma IsAlgEnvSeq.adapted_means [SecondCountableTopology 𝓨] [BorelSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) :
-    Adapted h.filtration (fun n ω ↦ env.means A Y (A n ω) n ω) :=
+    (h : IsAlgEnvSeq O A Y alg env P) :
+    Adapted h.filtration (fun n ω ↦ env.means O A Y (A n ω) n ω) :=
   fun n ↦ (h.adapted_means_filtrationAction n).mono (h.filtrationAction_le_filtration n) le_rfl
 
 omit [NormedSpace ℝ 𝓨] in
 lemma IsAlgEnvSeq.condExp_feedback_comp {𝓩 : Type*} [NormedAddCommGroup 𝓩] [NormedSpace ℝ 𝓩]
     [CompleteSpace 𝓩] [StandardBorelSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) (n : ℕ)
+    (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ)
     {g : 𝓨 → 𝓩} (hg : StronglyMeasurable g) (hint : Integrable (fun ω ↦ g (Y n ω)) P) :
     P[fun ω ↦ g (Y n ω) | h.filtrationAction n] =ᵐ[P]
-      fun ω ↦ (env.feedback n (history A Y n ω, A n ω))[g] := by
-  have hX : Measurable (fun ω ↦ (history A Y n ω, A n ω)) :=
-    (h.measurable_history n).prodMk (h.measurable_action n)
+      fun ω ↦ (env.feedback n ((history O A Y n ω, O n ω), A n ω))[g] := by
+  have hX : Measurable (fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) :=
+    ((h.measurable_history n).prodMk (h.measurable_obs n)).prodMk (h.measurable_action n)
   rw [h.filtrationAction_eq_comap n]
   exact (h.hasCondDistrib_feedback n).condExp_comp_eq hX hg hint
 
 lemma IsAlgEnvSeq.condExp_feedback [BorelSpace 𝓨] [SecondCountableTopology 𝓨] [CompleteSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) (n : ℕ)
+    (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ)
     (hint : Integrable (Y n) P) :
-    P[Y n | h.filtrationAction n] =ᵐ[P] fun ω ↦ env.means A Y (A n ω) n ω :=
+    P[Y n | h.filtrationAction n] =ᵐ[P] fun ω ↦ env.means O A Y (A n ω) n ω :=
   condExp_feedback_comp h n stronglyMeasurable_id hint
 
 lemma IsAlgEnvSeq.memLp_means_action [SecondCountableTopology 𝓨] [BorelSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) {n : ℕ} {p : ℝ≥0∞} (hp1 : 1 ≤ p) (hp_top : p ≠ ∞)
+    (h : IsAlgEnvSeq O A Y alg env P) {n : ℕ} {p : ℝ≥0∞} (hp1 : 1 ≤ p) (hp_top : p ≠ ∞)
     (hint : MemLp (Y n) p P) :
-    MemLp (fun ω ↦ env.means A Y (A n ω) n ω) p P := by
+    MemLp (fun ω ↦ env.means O A Y (A n ω) n ω) p P := by
   have hp0 : p ≠ 0 := by positivity
+  have hO := h.measurable_obs
   have hA := h.measurable_action
   have h_hist := h.measurable_history
   have hint' : MemLp id p (P.map (Y n)) := by
@@ -149,9 +153,9 @@ lemma IsAlgEnvSeq.memLp_means_action [SecondCountableTopology 𝓨] [BorelSpace
   simp only [id_eq]
   rw [(h.hasLaw_feedback_comp n).map_eq, Measure.memLp_comp_iff hp0 hp_top (by fun_prop)] at hint'
   have hint'' := hint'.2.comp_aemeasurable (by fun_prop)
-  have h_eq ω : (env.feedback n) (history A Y n ω, A n ω) =
+  have h_eq ω : (env.feedback n) ((history O A Y n ω, O n ω), A n ω) =
       (env.feedback n ∘ₖ
-        Kernel.deterministic (fun ω ↦ (history A Y n ω, A n ω)) (by fun_prop)) ω := by
+        Kernel.deterministic (fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) (by fun_prop)) ω := by
     simp [Kernel.comp_deterministic_eq_comap]
   rw [← integrable_norm_rpow_iff _ hp0 hp_top]
   swap
@@ -165,13 +169,15 @@ lemma IsAlgEnvSeq.memLp_means_action [SecondCountableTopology 𝓨] [BorelSpace
     simp_rw [Kernel.sectR_apply, h_eq]
     exact StronglyMeasurable.integral_kernel (by fun_prop)
   · simp only [Real.norm_eq_abs, Function.comp_apply, Kernel.sectR_apply]
-    filter_upwards [ae_of_ae_map ((h_hist n).prodMk (hA n)).aemeasurable hint'.1] with ω hω
+    filter_upwards [ae_of_ae_map (((h_hist n).prodMk (hO n)).prodMk (hA n)).aemeasurable hint'.1]
+      with ω hω
     rw [abs_of_nonneg (by positivity)]
     exact norm_integral_rpow_le_integral_norm_rpow hp1 hp_top hω
 
 lemma IsAlgEnvSeq.integrable_means_action [SecondCountableTopology 𝓨] [OpensMeasurableSpace 𝓨]
-    (h : IsAlgEnvSeq A Y alg env P) {n : ℕ} (hint : Integrable (Y n) P) :
-    Integrable (fun ω ↦ env.means A Y (A n ω) n ω) P := by
+    (h : IsAlgEnvSeq O A Y alg env P) {n : ℕ} (hint : Integrable (Y n) P) :
+    Integrable (fun ω ↦ env.means O A Y (A n ω) n ω) P := by
+  have hO := h.measurable_obs
   have hA := h.measurable_action
   have h_hist := h.measurable_history
   have hint' : Integrable id (P.map (Y n)) := by
@@ -183,9 +189,9 @@ lemma IsAlgEnvSeq.integrable_means_action [SecondCountableTopology 𝓨] [OpensM
   simp only [id_eq] at hint''
   refine Integrable.mono' hint'' ?_ ?_
   · refine StronglyMeasurable.aestronglyMeasurable ?_
-    have h_eq ω : (env.feedback n) (history A Y n ω, A n ω) =
+    have h_eq ω : (env.feedback n) ((history O A Y n ω, O n ω), A n ω) =
         (env.feedback n ∘ₖ
-          Kernel.deterministic (fun ω ↦ (history A Y n ω, A n ω)) (by fun_prop)) ω := by
+          Kernel.deterministic (fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) (by fun_prop)) ω := by
       simp [Kernel.comp_deterministic_eq_comap]
     simp_rw [Kernel.sectR_apply, h_eq]
     exact StronglyMeasurable.integral_kernel (by fun_prop)
diff --git a/LeanMachineLearning/SequentialLearning/StationaryEnv.lean b/LeanMachineLearning/SequentialLearning/StationaryEnv.lean
index f4108acd..b652c7e5 100644
--- a/LeanMachineLearning/SequentialLearning/StationaryEnv.lean
+++ b/LeanMachineLearning/SequentialLearning/StationaryEnv.lean
@@ -12,14 +12,15 @@ public import LeanMachineLearning.SequentialLearning.Algorithm
 # Oblivious and stationary environments
 
 An oblivious environment is an environment in which the distribution of the next feedback depends
-only on the last action (and not on the past history).
+only on the last action (and not on the past history nor on the current observation).
 If the kernel that gives the distribution of the next feedback given the last action is the same at
 every time step, then we say that the environment is stationary.
 
 ## Main definitions
 
 We define a `Prop`-valued typeclass `IsObliviousEnv` to express that an environment is oblivious,
-and we define two constructors for oblivious environments.
+and we define two constructors for oblivious environments. Those constructors build environments
+without observations, that is with observation type `Unit`.
 
 Typeclass and related definitions:
 * `IsObliviousEnv env`: the environment `env` is oblivious.
@@ -27,12 +28,12 @@ Typeclass and related definitions:
   given the action at time `n` in an oblivious environment `env`.
 
 Constructors for oblivious environments:
-* `obliviousEnv ν`: an oblivious environment, in which the distribution of the next feedback depends
-  only on the last action, but in a possibly time-dependent manner, and is given by a sequence of
-  Markov kernels `ν : ℕ → Kernel 𝓐 𝓨`.
-* `stationaryEnv ν`: a stationary environment, in which the distribution of the next feedback
-  depends only on the last action (and not on the past history), and is given by a Markov kernel
-  `ν : Kernel 𝓐 𝓨`.
+* `obliviousEnv ν`: an oblivious environment without observations, in which the distribution of the
+  next feedback depends only on the last action, but in a possibly time-dependent manner, and is
+  given by a sequence of Markov kernels `ν : ℕ → Kernel 𝓐 𝓨`.
+* `stationaryEnv ν`: a stationary environment without observations, in which the distribution of
+  the next feedback depends only on the last action (and not on the past history), and is given by
+  a Markov kernel `ν : Kernel 𝓐 𝓨`.
 
 -/
 
@@ -44,58 +45,62 @@ open scoped ENNReal NNReal
 
 namespace Learning
 
-variable {𝓐 𝓨 : Type*} {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨}
+variable {𝓞 𝓐 𝓨 : Type*} {m𝓞 : MeasurableSpace 𝓞} {m𝓐 : MeasurableSpace 𝓐}
+  {m𝓨 : MeasurableSpace 𝓨}
 
 /-- An environment is oblivious if the distribution of the next feedback depends only on
-the last action and not on the past history. -/
-class IsObliviousEnv (env : Environment 𝓐 𝓨) : Prop where
+the last action and not on the past history nor on the current observation. -/
+class IsObliviousEnv (env : Environment 𝓞 𝓐 𝓨) : Prop where
   exists_eq_prodMkLeft : ∃ ν : ℕ → Kernel 𝓐 𝓨, (∀ n, IsMarkovKernel (ν n)) ∧
     (∀ n, env.feedback n = (ν n).prodMkLeft _)
 
 /-- The kernel representing the conditional distribution of the feedback given the action
 at time `n` in an oblivious environment. -/
 noncomputable
-def feedbackCondAction (env : Environment 𝓐 𝓨) [h_obl : IsObliviousEnv env] (n : ℕ) : Kernel 𝓐 𝓨 :=
+def feedbackCondAction (env : Environment 𝓞 𝓐 𝓨) [h_obl : IsObliviousEnv env] (n : ℕ) :
+    Kernel 𝓐 𝓨 :=
   h_obl.exists_eq_prodMkLeft.choose n
 
-instance (env : Environment 𝓐 𝓨) [IsObliviousEnv env] (n : ℕ) :
+instance (env : Environment 𝓞 𝓐 𝓨) [IsObliviousEnv env] (n : ℕ) :
     IsMarkovKernel (feedbackCondAction env n) :=
   IsObliviousEnv.exists_eq_prodMkLeft.choose_spec.1 n
 
-lemma feedback_eq_feedbackCondAction (env : Environment 𝓐 𝓨) [IsObliviousEnv env] (n : ℕ) :
+lemma feedback_eq_feedbackCondAction (env : Environment 𝓞 𝓐 𝓨) [IsObliviousEnv env] (n : ℕ) :
     env.feedback n = (feedbackCondAction env n).prodMkLeft _ :=
   IsObliviousEnv.exists_eq_prodMkLeft.choose_spec.2 n
 
-lemma ν0_eq_feedbackCondAction (env : Environment 𝓐 𝓨) [IsObliviousEnv env] :
-    env.ν0 = feedbackCondAction env 0 := by
-  rw [Environment.ν0_def, feedback_eq_feedbackCondAction, Kernel.sectR_prodMkLeft]
+lemma ν0_eq_feedbackCondAction (env : Environment 𝓞 𝓐 𝓨) [IsObliviousEnv env] :
+    env.ν0 = (feedbackCondAction env 0).prodMkLeft 𝓞 := by
+  ext p : 1
+  rw [Environment.ν0_def, Kernel.comap_apply, feedback_eq_feedbackCondAction,
+    Kernel.prodMkLeft_apply, Kernel.prodMkLeft_apply]
 
 namespace IsObliviousEnv
 
 variable {Ω : Type*} {mΩ : MeasurableSpace Ω}
-  {alg : Algorithm 𝓐 𝓨} {env : Environment 𝓐 𝓨} {P : Measure Ω} [IsFiniteMeasure P]
-  {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {n N : ℕ}
+  {alg : Algorithm 𝓞 𝓐 𝓨} {env : Environment 𝓞 𝓐 𝓨} {P : Measure Ω} [IsFiniteMeasure P]
+  {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨} {n N : ℕ}
   {ν : ℕ → Kernel 𝓐 𝓨} [∀ n, IsMarkovKernel (ν n)]
 
 lemma hasCondDistrib_feedback_history_action [IsObliviousEnv env]
-    (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    HasCondDistrib (Y n) (fun ω ↦ (history A Y n ω, A n ω))
+    (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    HasCondDistrib (Y n) (fun ω ↦ ((history O A Y n ω, O n ω), A n ω))
       ((feedbackCondAction env n).prodMkLeft _) P := by
   rw [← feedback_eq_feedbackCondAction]
   exact h.hasCondDistrib_feedback n
 
-lemma hasCondDistrib_feedback [IsObliviousEnv env] (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
+lemma hasCondDistrib_feedback [IsObliviousEnv env] (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
     HasCondDistrib (Y n) (A n) (feedbackCondAction env n) P :=
   (hasCondDistrib_feedback_history_action h n).comp_right
 
 /-- Conditionally on an event determined by the history before time `n` and the action at time
 `n`, on which that action is equal to `b`, the feedback at time `n` has law
 `feedbackCondAction env n b`. -/
-lemma hasLaw_feedback_cond [IsObliviousEnv env] (h : IsAlgEnvSeq A Y alg env P) (n : ℕ)
-    {s : Set ((Fin n → 𝓐 × 𝓨) × 𝓐)} (hs : MeasurableSet s) {b : 𝓐} (hsb : ∀ u ∈ s, u.2 = b)
-    (hP : P ((fun ω ↦ (history A Y n ω, A n ω)) ⁻¹' s) ≠ 0) :
+lemma hasLaw_feedback_cond [IsObliviousEnv env] (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ)
+    {s : Set ((Hist 𝓞 𝓐 𝓨 n × 𝓞) × 𝓐)} (hs : MeasurableSet s) {b : 𝓐} (hsb : ∀ u ∈ s, u.2 = b)
+    (hP : P ((fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) ⁻¹' s) ≠ 0) :
     HasLaw (Y n) (feedbackCondAction env n b)
-      P[|(fun ω ↦ (history A Y n ω, A n ω)) ⁻¹' s] := by
+      P[|(fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) ⁻¹' s] := by
   refine (hasCondDistrib_feedback_history_action h n).hasLaw_cond (h.measurable_feedback _) hs
     (fun u hu ↦ ?_) hP
   rw [Kernel.prodMkLeft_apply, hsb u hu]
@@ -104,23 +109,26 @@ lemma hasLaw_feedback_cond [IsObliviousEnv env] (h : IsAlgEnvSeq A Y alg env P)
 `n`, on which that action is constant, the feedback at time `n` is independent of the
 history before time `n` and of the action at time `n`. -/
 lemma indepFun_history_action_feedback_cond [IsObliviousEnv env]
-    (h : IsAlgEnvSeq A Y alg env P) (n : ℕ)
-    {s : Set ((Fin n → 𝓐 × 𝓨) × 𝓐)} (hs : MeasurableSet s) {b : 𝓐} (hsb : ∀ u ∈ s, u.2 = b) :
-    (fun ω ↦ (history A Y n ω, A n ω))
-      ⟂ᵢ[P[|(fun ω ↦ (history A Y n ω, A n ω)) ⁻¹' s]] Y n := by
+    (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ)
+    {s : Set ((Hist 𝓞 𝓐 𝓨 n × 𝓞) × 𝓐)} (hs : MeasurableSet s) {b : 𝓐} (hsb : ∀ u ∈ s, u.2 = b) :
+    (fun ω ↦ ((history O A Y n ω, O n ω), A n ω))
+      ⟂ᵢ[P[|(fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) ⁻¹' s]] Y n := by
+  have hO := h.measurable_obs
   have hA := h.measurable_action
   have hY := h.measurable_feedback
   refine (hasCondDistrib_feedback_history_action h n).indepFun_cond (by fun_prop) hs
     (η := feedbackCondAction env n b) fun u hu ↦ ?_
   rw [Kernel.prodMkLeft_apply, hsb u hu]
 
-variable [StandardBorelSpace 𝓐] [Nonempty 𝓐] [StandardBorelSpace 𝓨] [Nonempty 𝓨]
+variable [StandardBorelSpace 𝓞] [Nonempty 𝓞] [StandardBorelSpace 𝓐] [Nonempty 𝓐]
+  [StandardBorelSpace 𝓨] [Nonempty 𝓨]
 
-/-- The feedback at time `n` is conditionally independent of the history before time `n`
-given the action at time `n`. -/
+/-- The feedback at time `n` is conditionally independent of the history before time `n` and of
+the observation at time `n`, given the action at time `n`. -/
 lemma condIndepFun_feedback_history_action [StandardBorelSpace Ω]
-        [IsObliviousEnv env] (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    Y n ⟂ᵢ[A n, h.measurable_action _ ; P] history A Y n := by
+        [IsObliviousEnv env] (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    Y n ⟂ᵢ[A n, h.measurable_action _ ; P] (fun ω ↦ (history O A Y n ω, O n ω)) := by
+  have hO := h.measurable_obs
   have hA := h.measurable_action
   have hY := h.measurable_feedback
   refine condIndepFun_of_exists_condDistrib_prod_ae_eq_prodMkLeft
@@ -131,29 +139,38 @@ lemma condIndepFun_feedback_history_action [StandardBorelSpace Ω]
   exact h.hasCondDistrib_feedback n
 
 lemma condIndepFun_feedback_history_action_action [StandardBorelSpace Ω]
-    [IsObliviousEnv env] (h : IsAlgEnvSeq A Y alg env P) (n : ℕ) :
-    Y n ⟂ᵢ[A n, h.measurable_action n; P] (fun ω ↦ (history A Y n ω, A n ω)) := by
-  have h_indep : Y n ⟂ᵢ[A n, h.measurable_action n; P] history A Y n :=
-    condIndepFun_feedback_history_action h n
+    [IsObliviousEnv env] (h : IsAlgEnvSeq O A Y alg env P) (n : ℕ) :
+    Y n ⟂ᵢ[A n, h.measurable_action n; P] (fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) := by
+  have h_indep : Y n ⟂ᵢ[A n, h.measurable_action n; P]
+      (fun ω ↦ (history O A Y n ω, O n ω)) := condIndepFun_feedback_history_action h n
+  have hO := h.measurable_obs
   have hA := h.measurable_action
   have hY := h.measurable_feedback
   exact h_indep.prod_right (by fun_prop) (by fun_prop) (by fun_prop)
 
 end IsObliviousEnv
 
-/-- An oblivious environment, in which the distribution of the next feedback depends only on
-the last action, but in a possibly time-dependent manner. -/
+/-- An oblivious environment without observations, in which the distribution of the next feedback
+depends only on the last action, but in a possibly time-dependent manner. -/
 @[simps]
-def obliviousEnv (ν : ℕ → Kernel 𝓐 𝓨) [∀ n, IsMarkovKernel (ν n)] : Environment 𝓐 𝓨 where
+noncomputable
+def obliviousEnv (ν : ℕ → Kernel 𝓐 𝓨) [∀ n, IsMarkovKernel (ν n)] : Environment Unit 𝓐 𝓨 where
+  obs _ := Kernel.const _ (Measure.dirac ())
   feedback n := (ν n).prodMkLeft _
 
 lemma feedback_obliviousEnv (ν : ℕ → Kernel 𝓐 𝓨) [∀ n, IsMarkovKernel (ν n)] (n : ℕ) :
     (obliviousEnv ν).feedback n = (ν n).prodMkLeft _ := rfl
 
+@[simp]
+lemma obs0_obliviousEnv (ν : ℕ → Kernel 𝓐 𝓨) [∀ n, IsMarkovKernel (ν n)] :
+    (obliviousEnv ν).obs0 = Measure.dirac () := rfl
+
 @[simp]
 lemma ν0_obliviousEnv (ν : ℕ → Kernel 𝓐 𝓨) [∀ n, IsMarkovKernel (ν n)] :
-    (obliviousEnv ν).ν0 = ν 0 := by
-  rw [Environment.ν0_def, feedback_obliviousEnv, Kernel.sectR_prodMkLeft]
+    (obliviousEnv ν).ν0 = (ν 0).prodMkLeft Unit := by
+  ext p : 1
+  rw [Environment.ν0_def, Kernel.comap_apply, feedback_obliviousEnv, Kernel.prodMkLeft_apply,
+    Kernel.prodMkLeft_apply]
 
 instance (ν : ℕ → Kernel 𝓐 𝓨) [∀ n, IsMarkovKernel (ν n)] :
     IsObliviousEnv (obliviousEnv ν) where
@@ -170,25 +187,38 @@ lemma feedbackCondAction_obliviousEnv (ν : ℕ → Kernel 𝓐 𝓨) [hν : ∀
   · refine absurd (hν 0) ?_
     simp only [Subsingleton.eq_zero ν, Pi.zero_apply]
     exact Kernel.not_isMarkovKernel_zero
-  have : Nonempty (Fin n → 𝓐 × 𝓨) := ⟨fun _ ↦ (h𝓐.some, hR.some)⟩
+  have : Nonempty (Hist Unit 𝓐 𝓨 n × Unit) := ⟨(fun _ ↦ ((), h𝓐.some, hR.some), ())⟩
   have h_eq := feedback_eq_feedbackCondAction (obliviousEnv ν) n
   rw [feedback_obliviousEnv, Kernel.prodMkLeft_inj] at h_eq
   exact h_eq.symm
 
-/-- A stationary environment, in which the distribution of the next feedback depends only on the
-last action. -/
-def stationaryEnv (ν : Kernel 𝓐 𝓨) [IsMarkovKernel ν] : Environment 𝓐 𝓨 := obliviousEnv fun _ ↦ ν
+/-- A stationary environment without observations, in which the distribution of the next feedback
+depends only on the last action. -/
+noncomputable
+def stationaryEnv (ν : Kernel 𝓐 𝓨) [IsMarkovKernel ν] : Environment Unit 𝓐 𝓨 :=
+  obliviousEnv fun _ ↦ ν
+
+@[simp]
+lemma obs_stationaryEnv (ν : Kernel 𝓐 𝓨) [IsMarkovKernel ν] (n : ℕ) :
+    (stationaryEnv ν).obs n = Kernel.const _ (Measure.dirac ()) := rfl
 
 @[simp]
 lemma feedback_stationaryEnv (ν : Kernel 𝓐 𝓨) [IsMarkovKernel ν] (n : ℕ) :
     (stationaryEnv ν).feedback n = ν.prodMkLeft _ := rfl
 
-lemma stepKernel_stationaryEnv (alg : Algorithm 𝓐 𝓨) (η : Kernel 𝓐 𝓨) [IsMarkovKernel η] (n : ℕ) :
-    stepKernel alg (stationaryEnv η) n = alg.policy n ⊗ₖ η.prodMkLeft _ := by
-  rw [stepKernel, feedback_stationaryEnv]
+lemma stepKernel_stationaryEnv (alg : Algorithm Unit 𝓐 𝓨) (η : Kernel 𝓐 𝓨) [IsMarkovKernel η]
+    (n : ℕ) :
+    stepKernel alg (stationaryEnv η) n
+      = Kernel.const _ (Measure.dirac ()) ⊗ₖ (alg.policy n ⊗ₖ η.prodMkLeft _) := by
+  rw [stepKernel_def, obs_stationaryEnv, feedback_stationaryEnv]
+
+@[simp]
+lemma obs0_stationaryEnv (ν : Kernel 𝓐 𝓨) [IsMarkovKernel ν] :
+    (stationaryEnv ν).obs0 = Measure.dirac () := rfl
 
 @[simp]
-lemma ν0_stationaryEnv (ν : Kernel 𝓐 𝓨) [IsMarkovKernel ν] : (stationaryEnv ν).ν0 = ν :=
+lemma ν0_stationaryEnv (ν : Kernel 𝓐 𝓨) [IsMarkovKernel ν] :
+    (stationaryEnv ν).ν0 = ν.prodMkLeft Unit :=
   ν0_obliviousEnv _
 
 instance (ν : Kernel 𝓐 𝓨) [IsMarkovKernel ν] : IsObliviousEnv (stationaryEnv ν) where
@@ -199,60 +229,64 @@ lemma feedbackCondAction_stationaryEnv (ν : Kernel 𝓐 𝓨) [hν : IsMarkovKe
     feedbackCondAction (stationaryEnv ν) n = ν := feedbackCondAction_obliviousEnv _ _
 
 variable {Ω : Type*} {mΩ : MeasurableSpace Ω}
-  {alg : Algorithm 𝓐 𝓨} {ν : Kernel 𝓐 𝓨} [IsMarkovKernel ν]
-  {P : Measure Ω} [IsProbabilityMeasure P] {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
+  {alg : Algorithm Unit 𝓐 𝓨} {ν : Kernel 𝓐 𝓨} [IsMarkovKernel ν]
+  {P : Measure Ω} [IsProbabilityMeasure P]
+  {O : ℕ → Ω → Unit} {A : ℕ → Ω → 𝓐} {Y : ℕ → Ω → 𝓨}
 
 namespace IsAlgEnvSeq
 
 /-- The conditional distribution of the feedback at time `n` given the action at time `n`
 is `ν n`. -/
 lemma hasCondDistrib_feedback_obliviousEnv {ν : ℕ → Kernel 𝓐 𝓨} [∀ n, IsMarkovKernel (ν n)]
-    (h : IsAlgEnvSeq A Y alg (obliviousEnv ν) P) (n : ℕ) :
+    (h : IsAlgEnvSeq O A Y alg (obliviousEnv ν) P) (n : ℕ) :
     HasCondDistrib (Y n) (A n) (ν n) P := by
   simpa using IsObliviousEnv.hasCondDistrib_feedback h n
 
 /-- The conditional distribution of the feedback at time `n` given the action at time `n` is `ν`. -/
 lemma hasCondDistrib_feedback_stationaryEnv
-    (h : IsAlgEnvSeq A Y alg (stationaryEnv ν) P) (n : ℕ) :
+    (h : IsAlgEnvSeq O A Y alg (stationaryEnv ν) P) (n : ℕ) :
     HasCondDistrib (Y n) (A n) ν P :=
   hasCondDistrib_feedback_obliviousEnv h n
 
 /-- The conditional distribution of the feedback at time `n` given the action at time `n` is `ν`. -/
 lemma condDistrib_feedback_stationaryEnv [StandardBorelSpace 𝓨] [Nonempty 𝓨]
-    (h : IsAlgEnvSeq A Y alg (stationaryEnv ν) P) (n : ℕ) :
+    (h : IsAlgEnvSeq O A Y alg (stationaryEnv ν) P) (n : ℕ) :
     condDistrib (Y n) (A n) P =ᵐ[P.map (A n)] ν :=
   (hasCondDistrib_feedback_stationaryEnv h n).condDistrib_eq
 
 /-- Conditionally on an event determined by the history before time `n` and the action at time
 `n`, on which that action is equal to `b`, the feedback at time `n` has law `ν b`. -/
-lemma hasLaw_feedback_cond_stationaryEnv (h : IsAlgEnvSeq A Y alg (stationaryEnv ν) P) (n : ℕ)
-    {s : Set ((Fin n → 𝓐 × 𝓨) × 𝓐)} (hs : MeasurableSet s) {b : 𝓐} (hsb : ∀ u ∈ s, u.2 = b)
-    (hP : P ((fun ω ↦ (history A Y n ω, A n ω)) ⁻¹' s) ≠ 0) :
-    HasLaw (Y n) (ν b) P[|(fun ω ↦ (history A Y n ω, A n ω)) ⁻¹' s] := by
+lemma hasLaw_feedback_cond_stationaryEnv (h : IsAlgEnvSeq O A Y alg (stationaryEnv ν) P) (n : ℕ)
+    {s : Set ((Hist Unit 𝓐 𝓨 n × Unit) × 𝓐)} (hs : MeasurableSet s) {b : 𝓐}
+    (hsb : ∀ u ∈ s, u.2 = b)
+    (hP : P ((fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) ⁻¹' s) ≠ 0) :
+    HasLaw (Y n) (ν b) P[|(fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) ⁻¹' s] := by
   simpa using IsObliviousEnv.hasLaw_feedback_cond h n hs hsb hP
 
 /-- Conditionally on an event determined by the history before time `n` and the action at time
 `n`, on which that action is constant, the feedback at time `n` is independent of the
 history before time `n` and of the action at time `n`. -/
 lemma indepFun_history_action_feedback_cond_stationaryEnv
-    (h : IsAlgEnvSeq A Y alg (stationaryEnv ν) P) (n : ℕ)
-    {s : Set ((Fin n → 𝓐 × 𝓨) × 𝓐)} (hs : MeasurableSet s) {b : 𝓐} (hsb : ∀ u ∈ s, u.2 = b) :
-    (fun ω ↦ (history A Y n ω, A n ω))
-      ⟂ᵢ[P[|(fun ω ↦ (history A Y n ω, A n ω)) ⁻¹' s]] Y n :=
+    (h : IsAlgEnvSeq O A Y alg (stationaryEnv ν) P) (n : ℕ)
+    {s : Set ((Hist Unit 𝓐 𝓨 n × Unit) × 𝓐)} (hs : MeasurableSet s) {b : 𝓐}
+    (hsb : ∀ u ∈ s, u.2 = b) :
+    (fun ω ↦ ((history O A Y n ω, O n ω), A n ω))
+      ⟂ᵢ[P[|(fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) ⁻¹' s]] Y n :=
   IsObliviousEnv.indepFun_history_action_feedback_cond h n hs hsb
 
 /-- The feedback at time `n` is conditionally independent of the history before time `n`
 given the action at time `n`. -/
 lemma condIndepFun_feedback_history_action [StandardBorelSpace Ω]
     [StandardBorelSpace 𝓐] [Nonempty 𝓐] [StandardBorelSpace 𝓨] [Nonempty 𝓨]
-    (h : IsAlgEnvSeq A Y alg (stationaryEnv ν) P) (n : ℕ) :
-    Y n ⟂ᵢ[A n, h.measurable_action _ ; P] history A Y n :=
+    (h : IsAlgEnvSeq O A Y alg (stationaryEnv ν) P) (n : ℕ) :
+    Y n ⟂ᵢ[A n, h.measurable_action _ ; P] (fun ω ↦ (history O A Y n ω, O n ω)) :=
   IsObliviousEnv.condIndepFun_feedback_history_action h n
 
 lemma condIndepFun_feedback_history_action_action [StandardBorelSpace Ω]
     [StandardBorelSpace 𝓐] [Nonempty 𝓐] [StandardBorelSpace 𝓨] [Nonempty 𝓨]
-    (h : IsAlgEnvSeq A Y alg (stationaryEnv ν) P) (n : ℕ) :
-    Y n ⟂ᵢ[A n, h.measurable_action n; P] (fun ω ↦ (history A Y n ω, A n ω)) :=
+    (h : IsAlgEnvSeq O A Y alg (stationaryEnv ν) P) (n : ℕ) :
+    Y n ⟂ᵢ[A n, h.measurable_action n; P]
+      (fun ω ↦ ((history O A Y n ω, O n ω), A n ω)) :=
   IsObliviousEnv.condIndepFun_feedback_history_action_action h n
 
 end IsAlgEnvSeq
diff --git a/LeanMachineLearning/SequentialLearning/SumRewards.lean b/LeanMachineLearning/SequentialLearning/SumRewards.lean
index b02ad3d3..820cd4ef 100644
--- a/LeanMachineLearning/SequentialLearning/SumRewards.lean
+++ b/LeanMachineLearning/SequentialLearning/SumRewards.lean
@@ -17,10 +17,11 @@ open MeasureTheory Finset Learning
 
 namespace Learning
 
-variable {𝓐 𝓨 Ω : Type*} {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨} {mΩ : MeasurableSpace Ω}
+variable {𝓞 𝓐 𝓨 Ω : Type*} {m𝓞 : MeasurableSpace 𝓞} {m𝓐 : MeasurableSpace 𝓐}
+  {m𝓨 : MeasurableSpace 𝓨} {mΩ : MeasurableSpace Ω}
   [DecidableEq 𝓐] [AddCommGroup 𝓨]
   {P : Measure Ω} [IsProbabilityMeasure P]
-  {A : ℕ → Ω → 𝓐} {R : ℕ → Ω → 𝓨}
+  {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → 𝓐} {R : ℕ → Ω → 𝓨}
   {a : 𝓐} {m n t : ℕ} {ω : Ω}
 
 /-- Sum of rewards obtained when pulling action `a` up to time `t` (exclusive). -/
@@ -29,8 +30,8 @@ noncomputable def sumRewards (A : ℕ → Ω → 𝓐) (R : ℕ → Ω → 𝓨)
 
 /-- Sum of rewards of arm `a` in the history before time `n`. -/
 noncomputable
-def sumRewards' (n : ℕ) (h : Fin n → 𝓐 × 𝓨) (a : 𝓐) :=
-  ∑ s, if (h s).1 = a then (h s).2 else 0
+def sumRewards' (n : ℕ) (h : Hist 𝓞 𝓐 𝓨 n) (a : 𝓐) :=
+  ∑ s, if (h s).action = a then (h s).feedback else 0
 
 /-- Empirical mean reward obtained when pulling action `a` up to time `t` (exclusive). -/
 noncomputable
@@ -39,7 +40,7 @@ def empMean (A : ℕ → Ω → 𝓐) (R : ℕ → Ω → ℝ) (a : 𝓐) (t : 
 
 /-- Empirical mean of arm `a` in the history before time `n`. -/
 noncomputable
-def empMean' (n : ℕ) (h : Fin n → 𝓐 × ℝ) (a : 𝓐) :=
+def empMean' (n : ℕ) (h : Hist 𝓞 𝓐 ℝ n) (a : 𝓐) :=
   sumRewards' n h a / pullCount' n h a
 
 @[simp]
@@ -86,23 +87,23 @@ lemma sum_rewardByCount_eq_sumRewards {R : ℕ → Ω → 𝓨} (a : 𝓐) (t :
       rwa [pullCount_eq_pullCount_of_action_ne hta, sum_range_succ, ite_eq_right hta, add_zero]
 
 lemma sumRewards_eq_sumRewards' {R : ℕ → Ω → 𝓨} {n : ℕ} {ω : Ω} :
-    sumRewards A R a n ω = sumRewards' n (fun i ↦ (A i ω, R i ω)) a :=
+    sumRewards A R a n ω = sumRewards' n (history O A R n ω) a :=
   (Fin.sum_univ_eq_sum_range (fun i ↦ if A i ω = a then R i ω else 0) n).symm
 
 /-- `sumRewards A R a n` is a function of the history before time `n`. -/
 lemma sumRewards_eq_comp_history {R : ℕ → Ω → 𝓨} (a : 𝓐) (n : ℕ) :
-    sumRewards A R a n = (fun h : Fin n → 𝓐 × 𝓨 ↦ sumRewards' n h a) ∘ history A R n := by
+    sumRewards A R a n = (fun h : Hist 𝓞 𝓐 𝓨 n ↦ sumRewards' n h a) ∘ history O A R n := by
   ext ω
   exact sumRewards_eq_sumRewards'
 
 lemma empMean_eq_empMean' {R : ℕ → Ω → ℝ} {n : ℕ} {ω : Ω} :
-    empMean A R a n ω = empMean' n (fun i ↦ (A i ω, R i ω)) a := by
+    empMean A R a n ω = empMean' n (history O A R n ω) a := by
   unfold empMean empMean'
-  rw [sumRewards_eq_sumRewards', pullCount_eq_pullCount']
+  rw [sumRewards_eq_sumRewards' (O := O), pullCount_eq_pullCount' (O := O)]
 
 /-- `empMean A R a n` is a function of the history before time `n`. -/
 lemma empMean_eq_comp_history {R : ℕ → Ω → ℝ} (a : 𝓐) (n : ℕ) :
-    empMean A R a n = (fun h : Fin n → 𝓐 × ℝ ↦ empMean' n h a) ∘ history A R n := by
+    empMean A R a n = (fun h : Hist 𝓞 𝓐 ℝ n ↦ empMean' n h a) ∘ history O A R n := by
   ext ω
   exact empMean_eq_empMean'
 
@@ -158,37 +159,38 @@ lemma measurable_uncurry_empMean_comp [Countable 𝓐] [MeasurableSingletonClass
 
 @[fun_prop]
 lemma measurable_sumRewards' [MeasurableSingletonClass 𝓐] [MeasurableAdd₂ 𝓨] (n : ℕ) (a : 𝓐) :
-    Measurable (sumRewards' (𝓨 := 𝓨) n · a) := by
+    Measurable (sumRewards' (𝓞 := 𝓞) (𝓨 := 𝓨) n · a) := by
   simp_rw [sumRewards']
-  have h_meas s : Measurable (fun (h : Fin n → 𝓐 × 𝓨) ↦ if (h s).1 = a then (h s).2 else 0) := by
+  have h_meas s : Measurable
+      (fun (h : Hist 𝓞 𝓐 𝓨 n) ↦ if (h s).action = a then (h s).feedback else 0) := by
     refine Measurable.ite ?_ (by fun_prop) (by fun_prop)
     exact (measurableSet_singleton _).preimage (by fun_prop)
   fun_prop
 
 @[fun_prop]
 lemma measurable_uncurry_sumRewards' [MeasurableEq 𝓐] [MeasurableAdd₂ 𝓨] (n : ℕ) :
-    Measurable (fun p : (Fin n → 𝓐 × 𝓨) × 𝓐 ↦ sumRewards' n p.1 p.2) := by
+    Measurable (fun p : Hist 𝓞 𝓐 𝓨 n × 𝓐 ↦ sumRewards' n p.1 p.2) := by
   simp_rw [sumRewards']
-  have h_meas s : Measurable (fun p : (Fin n → 𝓐 × 𝓨) × 𝓐 ↦
-      if (p.1 s).1 = p.2 then (p.1 s).2 else 0) := by
+  have h_meas s : Measurable (fun p : Hist 𝓞 𝓐 𝓨 n × 𝓐 ↦
+      if (p.1 s).action = p.2 then (p.1 s).feedback else 0) := by
     refine Measurable.ite ?_ (by fun_prop) (by fun_prop)
     exact measurableSet_eq_fun (by fun_prop) (by fun_prop)
   fun_prop
 
 @[fun_prop]
 lemma measurable_empMean' [MeasurableSingletonClass 𝓐] (n : ℕ) (a : 𝓐) :
-    Measurable (empMean' n · a) := by unfold empMean'; fun_prop
+    Measurable (empMean' (𝓞 := 𝓞) n · a) := by unfold empMean'; fun_prop
 
 @[fun_prop]
 lemma measurable_uncurry_empMean' [MeasurableEq 𝓐] (n : ℕ) :
-    Measurable (fun p : (Fin n → 𝓐 × ℝ) × 𝓐 ↦ empMean' n p.1 p.2) := by unfold empMean'; fun_prop
+    Measurable (fun p : Hist 𝓞 𝓐 ℝ n × 𝓐 ↦ empMean' n p.1 p.2) := by unfold empMean'; fun_prop
 
 variable [MeasurableSingletonClass 𝓐]
 
 lemma IsAlgEnvSeq.isStronglyPredictable_sumRewards {𝓨 : Type*} {_ : MeasurableSpace 𝓨}
     [NormedAddCommGroup 𝓨] [OpensMeasurableSpace 𝓨] [SecondCountableTopology 𝓨]
-    {R : ℕ → Ω → 𝓨} {alg : Algorithm 𝓐 𝓨} {env : Environment 𝓐 𝓨}
-    (h : IsAlgEnvSeq A R alg env P) (a : 𝓐) :
+    {R : ℕ → Ω → 𝓨} {alg : Algorithm 𝓞 𝓐 𝓨} {env : Environment 𝓞 𝓐 𝓨}
+    (h : IsAlgEnvSeq O A R alg env P) (a : 𝓐) :
     IsStronglyPredictable h.filtration (sumRewards A R a) := by
   rw [IsStronglyPredictable.iff_measurable_add_one]
   constructor
@@ -206,8 +208,8 @@ lemma IsAlgEnvSeq.isStronglyPredictable_sumRewards {𝓨 : Type*} {_ : Measurabl
 
 lemma IsAlgEnvSeq.stronglyAdapted_sumRewards_add_one {𝓨 : Type*} {_ : MeasurableSpace 𝓨}
     [NormedAddCommGroup 𝓨] [OpensMeasurableSpace 𝓨] [SecondCountableTopology 𝓨]
-    {R : ℕ → Ω → 𝓨} {alg : Algorithm 𝓐 𝓨} {env : Environment 𝓐 𝓨}
-    (h : IsAlgEnvSeq A R alg env P) (a : 𝓐) :
+    {R : ℕ → Ω → 𝓨} {alg : Algorithm 𝓞 𝓐 𝓨} {env : Environment 𝓞 𝓐 𝓨}
+    (h : IsAlgEnvSeq O A R alg env P) (a : 𝓐) :
     StronglyAdapted h.filtration (fun n ↦ sumRewards A R a (n + 1)) := by
   have h_predictable := h.isStronglyPredictable_sumRewards a
   rw [IsStronglyPredictable.iff_measurable_add_one] at h_predictable
@@ -216,14 +218,14 @@ lemma IsAlgEnvSeq.stronglyAdapted_sumRewards_add_one {𝓨 : Type*} {_ : Measura
 -- TODO: give a direct proof, without a topology
 lemma IsAlgEnvSeq.adapted_sumRewards_add_one {𝓨 : Type*} {_ : MeasurableSpace 𝓨}
     [NormedAddCommGroup 𝓨] [BorelSpace 𝓨] [SecondCountableTopology 𝓨]
-    {R : ℕ → Ω → 𝓨} {alg : Algorithm 𝓐 𝓨} {env : Environment 𝓐 𝓨}
-    (h : IsAlgEnvSeq A R alg env P) (a : 𝓐) :
+    {R : ℕ → Ω → 𝓨} {alg : Algorithm 𝓞 𝓐 𝓨} {env : Environment 𝓞 𝓐 𝓨}
+    (h : IsAlgEnvSeq O A R alg env P) (a : 𝓐) :
     Adapted h.filtration (fun n ↦ sumRewards A R a (n + 1)) :=
   (h.stronglyAdapted_sumRewards_add_one a).adapted
 
 lemma IsAlgEnvSeq.isStronglyPredictable_empMean {R' : ℕ → Ω → ℝ}
-    {alg : Algorithm 𝓐 ℝ} {env : Environment 𝓐 ℝ}
-    (h : IsAlgEnvSeq A R' alg env P) (a : 𝓐) :
+    {alg : Algorithm 𝓞 𝓐 ℝ} {env : Environment 𝓞 𝓐 ℝ}
+    (h : IsAlgEnvSeq O A R' alg env P) (a : 𝓐) :
     IsStronglyPredictable h.filtration (empMean A R' a) := by
   unfold empMean
   refine StronglyMeasurable.div ?_ ?_
@@ -232,16 +234,16 @@ lemma IsAlgEnvSeq.isStronglyPredictable_empMean {R' : ℕ → Ω → ℝ}
     fun_prop
 
 lemma IsAlgEnvSeq.stronglyAdapted_empMean_add_one
-    {R' : ℕ → Ω → ℝ} {alg : Algorithm 𝓐 ℝ} {env : Environment 𝓐 ℝ}
-    (h : IsAlgEnvSeq A R' alg env P) (a : 𝓐) :
+    {R' : ℕ → Ω → ℝ} {alg : Algorithm 𝓞 𝓐 ℝ} {env : Environment 𝓞 𝓐 ℝ}
+    (h : IsAlgEnvSeq O A R' alg env P) (a : 𝓐) :
     StronglyAdapted h.filtration (fun n ↦ empMean A R' a (n + 1)) := by
   have h_predictable := h.isStronglyPredictable_empMean a
   rw [IsStronglyPredictable.iff_measurable_add_one] at h_predictable
   exact h_predictable.2
 
 lemma IsAlgEnvSeq.adapted_empMean_add_one {R' : ℕ → Ω → ℝ}
-    {alg : Algorithm 𝓐 ℝ} {env : Environment 𝓐 ℝ}
-    (h : IsAlgEnvSeq A R' alg env P) (a : 𝓐) :
+    {alg : Algorithm 𝓞 𝓐 ℝ} {env : Environment 𝓞 𝓐 ℝ}
+    (h : IsAlgEnvSeq O A R' alg env P) (a : 𝓐) :
     Adapted h.filtration (fun n ↦ empMean A R' a (n + 1)) :=
   (h.stronglyAdapted_empMean_add_one a).adapted