Skip to content

Rework the algorithm/environment classes and constructors - #254

Open
RemyDegenne wants to merge 9 commits into
mainfrom
rename
Open

RemyDegenne wants to merge 9 commits into
mainfrom
rename

Conversation

@RemyDegenne

@RemyDegenne RemyDegenne commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator

AI disclorure: the new list of names and plan for the organisation of algorithms and environments were done manually. The changes in the code to follow the plan (mostly renames) are done with AI.

See the new naming-convention README at LeanMachineLearning/SequentialLearning/README.md . It gives the current names for algorithms or environments that don't depend on some parts of the problem, as well as proposed names for not yet implemented classes and constructors (marked MISSING).

  • Add Markov algorithms, that depend only on the observation (the state in an MDP) but not the history.
  • Allow oblivious and stationary environments to have non-Unit observations. The old Unit-restricted constructors are now Environment.bandit and Environment.banditSeq.

Renames:

Kind Old New
Class IsDeterministicAlg Algorithm.IsDeterministic
Class IsDeterministicEnv Environment.HasDeterministicFeedback
Class IsObliviousEnv Environment.IsOblivious (meaning changed, see above)
Accessor nextAction, actionZero Algorithm.nextAction, Algorithm.actionZero
Accessor feedbackFun, feedbackFunZero Environment.feedbackFun, Environment.feedbackFunZero
Time zero Algorithm.p0 Algorithm.policyZero
Time zero Environment.obs0 Environment.obsZero
Time zero Environment.ν0 Environment.feedbackZero
Constructor detAlgorithm Algorithm.deterministic
Constructor randomSampling Algorithm.const
Constructor detEnvironment Environment.detFeedback
Constructor obliviousEnv ν (no observations) Environment.banditSeq ν
Constructor stationaryEnv ν (no observations) Environment.bandit ν
Constructor onlineEvalEnv Environment.evalSeq (now built on banditSeq)
Constructor evalEnv Environment.eval

Notable breaking changes: the names Environment.stationary and Environment.oblivious still exist, but now mean the versions with observations. Code that used the old observation-free environments should switch to Environment.bandit and Environment.banditSeq.

@RemyDegenne
RemyDegenne marked this pull request as ready for review October 3, 2026 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant