Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,3 +99,75 @@ When modernizing existing libEnsemble scripts (functionality tests, regression t
- **Mandatory Fields**: Ensure `gen_specs["in"]` or `gen_specs["persis_in"]` includes at least one field (e.g., `["sim_id"]`) if feedback is sent back to the generator, to satisfy the allocator's requirements.
- **gest-api Simulators**: The gest-api pattern also applies to simulators. Set `SimSpecs.simulator` to a callable with signature `(input_dict: dict, **kwargs) -> dict` instead of providing a `sim_f`. libEnsemble automatically wraps it with `gest_api_sim` from `libensemble.sim_funcs.gest_api_wrapper` and handles all NumPy conversions. `SimSpecs.inputs` and `SimSpecs.outputs` can be derived automatically when `SimSpecs.vocs` is provided.
- **`safe_mode` is opt-in**: `libE_specs["safe_mode"]` defaults to `False`, meaning protected History fields (`gen_worker`, `gen_started_time`, `gen_ended_time`, `sim_worker`, `sim_started`, `sim_started_time`, `sim_ended`, `sim_ended_time`, `gen_informed`, `gen_informed_time`, `kill_sent`) are freely overwritable by default. Set `safe_mode=True` to enable protection. Overwriting these fields without understanding their purpose may crash libEnsemble.
- **Pre-generated samples**: Scripts that previously used the ``give_pregenerated_work`` allocator (with no generator) should be migrated to use ``PreloadedSampleGenerator`` from ``libensemble.gen_classes.preloaded``. Pass it as ``GenSpecs(generator=PreloadedSampleGenerator(H0))`` and use the default ``AllocSpecs()``. The generator serves the pre-loaded points via ``suggest()`` and returns an empty list when exhausted, triggering normal ensemble shutdown.

Deprecation Policy
------------------

This section describes the standard process for deprecating **any** libEnsemble feature
(allocation functions, generator classes, public API, parameters, etc.).

**Warning category**

Always use ``LibEnsembleDeprecationWarning`` — a custom subclass of ``DeprecationWarning``
importable from ``libensemble._deprecation``. Never emit bare ``DeprecationWarning``
directly. The custom subclass lets users and downstream libraries filter libEnsemble
deprecations independently::

from libensemble._deprecation import LibEnsembleDeprecationWarning
warnings.filterwarnings("error", category=LibEnsembleDeprecationWarning)

**Emit the warning**

Emit the warning at the earliest point of use (module import, class instantiation, or
function call — whichever the user is most likely to see). Use the ``warn_deprecated()``
helper from the same module when the standard message format is sufficient::

import warnings
from libensemble._deprecation import LibEnsembleDeprecationWarning

warnings.warn(
"libensemble.<module>.<name> is deprecated as of libEnsemble X.Y "
"and will be removed in X.Z. Use <replacement> instead. "
"See https://libensemble.readthedocs.io/... for migration guidance.",
LibEnsembleDeprecationWarning,
stacklevel=2, # points to the caller's import/call site
)

**Docstring banner**

Add a ``.. deprecated:: X.Y`` directive at the top of the deprecated object's docstring,
naming the replacement and the removal version::

def my_old_function(...):
"""
.. deprecated:: 2.0
``my_old_function`` is deprecated and will be removed in libEnsemble 2.1.
Use :func:`libensemble.module.my_new_function` instead.
...
"""

**Sphinx docs**

In the relevant ``.rst`` file, move the deprecated item to a "Deprecated" section (or
subsection) at the bottom of the page and prefix its ``automodule``/``autofunction`` block
with a ``.. deprecated:: X.Y`` admonition and a ``.. warning::`` summarising all items
in the section together with migration guidance. See
``docs/function_guides/allocator.rst`` for a reference example.

**Pytest noise suppression**

Add a ``filterwarnings`` rule to ``[tool.pytest.ini_options]`` in ``pyproject.toml`` so
that tests of deprecated-but-not-yet-removed code do not produce noisy output::

[tool.pytest.ini_options]
filterwarnings = [
"ignore::libensemble._deprecation.LibEnsembleDeprecationWarning",
]

Remove this rule when the deprecated code is deleted.

**Timeline**

The standard window is: soft-deprecate in release N, hard-remove (delete code + tests) in N+1.
Tests that exclusively cover deprecated features are deleted in the removal release, not before.
2 changes: 1 addition & 1 deletion docs/examples/calling_scripts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,6 @@ paired with a gest-api ``simulator`` callable.
:language: python
:caption: tests/regression_tests/test_asktell_aposmm_nlopt.py
:linenos:
:end-at: workflow.exit_criteria = ExitCriteria(sim_max=2000, wallclock_max=600)
:end-at: H, _, _ = workflow.run(sim_max=3000, wallclock_max=600)

.. _regression tests: https://github.com/Libensemble/libensemble/tree/develop/libensemble/tests/regression_tests
93 changes: 69 additions & 24 deletions docs/function_guides/allocator.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ We encourage experimenting with:

.. dropdown:: Example

.. literalinclude:: ../../libensemble/alloc_funcs/fast_alloc.py
:caption: libensemble.alloc_funcs.fast_alloc.give_sim_work_first
.. literalinclude:: ../../libensemble/alloc_funcs/give_sim_work_first.py
:caption: libensemble.alloc_funcs.give_sim_work_first.give_sim_work_first

The ``alloc_f`` function definition resembles::

Expand Down Expand Up @@ -184,56 +184,101 @@ give_sim_work_first
:language: python
:linenos:

fast_alloc
----------
.. automodule:: fast_alloc
:members:
:undoc-members:
persistent_aposmm_alloc
-----------------------
.. automodule:: persistent_aposmm_alloc
:members:
:undoc-members:

give_pregenerated_work
----------------------
.. automodule:: give_pregenerated_work
:members:
:undoc-members:

.. dropdown:: :underline:`fast_alloc.py`
.. _deprecated-alloc-label:

.. literalinclude:: ../../libensemble/alloc_funcs/fast_alloc.py
:language: python
:linenos:
Deprecated Allocation Functions
================================

start_persistent_local_opt_gens
-------------------------------
.. automodule:: start_persistent_local_opt_gens
.. warning::

The following allocation functions are **deprecated as of libEnsemble 2.0** and will be
**removed in libEnsemble 2.1**. They emit a :class:`~libensemble._deprecation.LibEnsembleDeprecationWarning`
on import.

**Migration guidance:**

- Functions that managed non-persistent generators (``fast_alloc``, ``fast_alloc_and_pausing``,
``only_one_gen_alloc``) should be replaced with
:func:`~libensemble.alloc_funcs.give_sim_work_first.give_sim_work_first` or the default
:func:`~libensemble.alloc_funcs.start_only_persistent.only_persistent_gens` with a
persistent generator.
- APOSMM-adjacent functions (``start_persistent_local_opt_gens``, ``start_fd_persistent``)
should migrate to
:func:`~libensemble.alloc_funcs.persistent_aposmm_alloc.persistent_aposmm_alloc`.
- ``inverse_bayes_allocf`` should be replaced with the default ``only_persistent_gens``
combined with a persistent generator that implements the required batch/subbatch logic.

fast_alloc
----------
.. deprecated:: 2.0
Use :func:`~libensemble.alloc_funcs.give_sim_work_first.give_sim_work_first` or the default
:func:`~libensemble.alloc_funcs.start_only_persistent.only_persistent_gens` instead.
Will be removed in libEnsemble 2.1.

.. automodule:: fast_alloc
:members:
:undoc-members:

fast_alloc_and_pausing
----------------------
.. deprecated:: 2.0
Use the default :func:`~libensemble.alloc_funcs.start_only_persistent.only_persistent_gens`
with a persistent generator instead. Will be removed in libEnsemble 2.1.

.. automodule:: fast_alloc_and_pausing
:members:
:undoc-members:

only_one_gen_alloc
------------------
.. deprecated:: 2.0
Use :func:`~libensemble.alloc_funcs.give_sim_work_first.give_sim_work_first` with
``num_active_gens=1``, or the default
:func:`~libensemble.alloc_funcs.start_only_persistent.only_persistent_gens` instead.
Will be removed in libEnsemble 2.1.

.. automodule:: only_one_gen_alloc
:members:
:undoc-members:

start_fd_persistent
-------------------
.. deprecated:: 2.0
Use the default :func:`~libensemble.alloc_funcs.start_only_persistent.only_persistent_gens`
with a persistent generator instead. Will be removed in libEnsemble 2.1.

.. automodule:: start_fd_persistent
:members:
:undoc-members:

persistent_aposmm_alloc
-----------------------
.. automodule:: persistent_aposmm_alloc
:members:
:undoc-members:
start_persistent_local_opt_gens
-------------------------------
.. deprecated:: 2.0
Use :func:`~libensemble.alloc_funcs.persistent_aposmm_alloc.persistent_aposmm_alloc`
instead. Will be removed in libEnsemble 2.1.

give_pregenerated_work
----------------------
.. automodule:: give_pregenerated_work
:members:
:undoc-members:
.. automodule:: start_persistent_local_opt_gens
:members:
:undoc-members:

inverse_bayes_allocf
--------------------
.. deprecated:: 2.0
Use the default :func:`~libensemble.alloc_funcs.start_only_persistent.only_persistent_gens`
with a persistent generator instead. Will be removed in libEnsemble 2.1.

.. automodule:: inverse_bayes_allocf
:members:
:undoc-members:
4 changes: 2 additions & 2 deletions docs/platforms/aurora.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ simulations for each worker:

.. code-block:: python

# Instruct libEnsemble to exit after this many simulations
ensemble.exit_criteria = ExitCriteria(sim_max=nsim_workers * 2)
# Run ensemble; exit after this many simulations
ensemble.run(sim_max=nsim_workers * 2)

Now grab an interactive session on two nodes (or use the batch script at
``../submission_scripts/submit_pbs_aurora.sh``)::
Expand Down
5 changes: 2 additions & 3 deletions docs/tutorials/aposmm_tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ libEnsemble classes, APOSMM, and our simulator callable:
from libensemble import Ensemble
from libensemble.gen_classes import APOSMM
from gest_api.vocs import VOCS
from libensemble.specs import SimSpecs, GenSpecs, ExitCriteria
from libensemble.specs import SimSpecs, GenSpecs

APOSMM supports a wide variety of external optimizers. The ``rc.aposmm_optimizers``
statement above indicates to APOSMM which optimization method package to use,
Expand Down Expand Up @@ -156,9 +156,8 @@ Finally, we configure the simulation function, exit criteria, and run the workfl
:linenos:

workflow.sim_specs = SimSpecs(simulator=six_hump_camel_func, vocs=vocs)
workflow.exit_criteria = ExitCriteria(sim_max=2000)

H, _, _ = workflow.run()
H, _, _ = workflow.run(sim_max=2000)

if workflow.is_manager:
# We can map our variables back to an array for easy printing
Expand Down
11 changes: 4 additions & 7 deletions docs/tutorials/gpcam_tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ If you wish to make your own functions based on the above, those can be imported
from pprint import pprint

from libensemble import Ensemble
from libensemble.specs import LibeSpecs, GenSpecs, SimSpecs, AllocSpecs, ExitCriteria
from libensemble.specs import LibeSpecs, GenSpecs, SimSpecs, AllocSpecs

# If importing from libensemble
from libensemble.gen_funcs.persistent_gpCAM import persistent_gpCAM
Expand Down Expand Up @@ -256,15 +256,12 @@ If you wish to make your own functions based on the above, those can be imported
user={"async_return": False}, # False = batch returns
)

exit_criteria = ExitCriteria(sim_max=num_batches * batch_size)

# Initialize and run the ensemble.
# Initialize the ensemble.
ensemble = Ensemble(
libE_specs=libE_specs,
sim_specs=sim_specs,
gen_specs=gen_specs,
alloc_specs=alloc_specs,
exit_criteria=exit_criteria,
)

At the end of our calling script we run the ensemble.
Expand All @@ -275,7 +272,7 @@ At the end of our calling script we run the ensemble.
cleanup()
ensemble.persis_info = {}

H, persis_info, flag = ensemble.run() # Start the ensemble. Blocks until completion.
H, persis_info, flag = ensemble.run(sim_max=num_batches * batch_size) # Start the ensemble. Blocks until completion.
ensemble.save_output("H_array", append_attrs=False) # Save H (history of all evaluated points) to file
pprint(H[["sim_id", "x", "f"]][:16]) # See first 16 results

Expand All @@ -293,7 +290,7 @@ To see how the accuracy of the surrogate model improves, we can use previously e
cleanup()
ensemble.persis_info = {}

H, persis_info, flag = ensemble.run()
H, persis_info, flag = ensemble.run(sim_max=num_batches * batch_size)
print(persis_info)

Viewing model progression
Expand Down
9 changes: 3 additions & 6 deletions docs/tutorials/xopt_bayesian_gen.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Imports

from libensemble import Ensemble
from libensemble.alloc_funcs.start_only_persistent import only_persistent_gens as alloc_f
from libensemble.specs import AllocSpecs, ExitCriteria, GenSpecs, LibeSpecs, SimSpecs
from libensemble.specs import AllocSpecs, GenSpecs, LibeSpecs, SimSpecs

Simulator Function
------------------
Expand Down Expand Up @@ -93,17 +93,15 @@ The simulator is a simple callable function that takes a dictionary of inputs an
)

alloc_specs = AllocSpecs(alloc_f=alloc_f)
exit_criteria = ExitCriteria(sim_max=12)

workflow = Ensemble(
libE_specs=libE_specs,
sim_specs=sim_specs,
alloc_specs=alloc_specs,
gen_specs=gen_specs,
exit_criteria=exit_criteria,
)

H, _, _ = workflow.run()
H, _, _ = workflow.run(sim_max=12)

if workflow.is_manager:
print(f"Completed {len(H)} simulations")
Expand Down Expand Up @@ -158,10 +156,9 @@ Reset generator and change to libEnsemble-style simulator:
sim_specs=sim_specs,
alloc_specs=alloc_specs,
gen_specs=gen_specs,
exit_criteria=exit_criteria,
)

H, _, _ = workflow.run()
H, _, _ = workflow.run(sim_max=12)

if workflow.is_manager:
print(f"Completed {len(H)} simulations")
Expand Down
38 changes: 38 additions & 0 deletions libensemble/_deprecation.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
"""
Deprecation utilities for libEnsemble.
"""

import warnings


class LibEnsembleDeprecationWarning(DeprecationWarning):
"""Warning category for deprecated libEnsemble features.

Subclass of :class:`DeprecationWarning` so users can filter libEnsemble
deprecations independently::

import warnings
from libensemble._deprecation import LibEnsembleDeprecationWarning
warnings.filterwarnings("error", category=LibEnsembleDeprecationWarning)
"""


def warn_deprecated(name: str, replacement: str, removal_version: str = "2.1") -> None:
"""Emit a :class:`LibEnsembleDeprecationWarning` for a deprecated feature.

Parameters
----------
name:
Dotted module or object path (e.g. ``"libensemble.alloc_funcs.fast_alloc"``).
replacement:
Human-readable description of the recommended replacement.
removal_version:
The libEnsemble version in which the feature will be removed.
"""
warnings.warn(
f"{name} is deprecated as of libEnsemble 2.0 "
f"and will be removed in {removal_version}. "
f"Use {replacement} instead.",
LibEnsembleDeprecationWarning,
stacklevel=3,
)
12 changes: 12 additions & 0 deletions libensemble/alloc_funcs/fast_alloc.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,20 @@
from libensemble._deprecation import warn_deprecated
from libensemble.tools.alloc_support import AllocSupport, InsufficientFreeResources

warn_deprecated(
name="libensemble.alloc_funcs.fast_alloc",
replacement="libensemble.alloc_funcs.give_sim_work_first.give_sim_work_first "
"or the default only_persistent_gens with a persistent generator",
)


def give_sim_work_first(W, H, sim_specs, gen_specs, alloc_specs, persis_info, libE_info):
"""
.. deprecated:: 2.0
``fast_alloc.give_sim_work_first`` is deprecated and will be removed in libEnsemble 2.1.
Use :func:`libensemble.alloc_funcs.give_sim_work_first.give_sim_work_first` or the
default ``only_persistent_gens`` (with a persistent generator) instead.

This allocation function gives (in order) entries in ``H`` to idle workers
to evaluate in the simulation function. The fields in ``sim_specs["in"]``
are given. If all entries in `H` have been given a be evaluated, a worker
Expand Down
Loading