Skip to content

[Docs] A model's status.observedGeneration and lastReconcileTime advance only when its status changes - #213

Open
github-actions[bot] wants to merge 2 commits into
mainfrom
codex/nightly-docs-a44a953a78699864
Open

github-actions[bot] wants to merge 2 commits into
mainfrom
codex/nightly-docs-a44a953a78699864

Conversation

@github-actions

@github-actions github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

What this PR does

I edited my BaseModel's displayName and status.observedGeneration still lags metadata.generation, and a healthy Ready model's lastReconcileTime is hours old — is the model controller stuck?

Why we need it

Source change: ome-projects/ome@e8cc64f

From commit 50 (#653, '[API] Add conditions + PVC reasons to model status'), whose patch adds ObservedGeneration and LastReconcileTime to ModelStatusSpec (patch lines 441-453, read today) with the descriptions that now render into the generated API reference ('If this lags behind .metadata.generation, the controller has not yet observed the latest spec edit'; 'Liveness signal — if it falls behind by more than the reconcile period, the controller is wedged or not scheduled'). Current code verified today: pkg/apis/ome/v1beta1/basemodel_types.go:491-503 carries those comments (rendered at reference/api/ome.v1beta1.md:8263-8282, generated, must not be edited); the ONE stamping helper shared.StampObservedReconcile (pkg/controller/v1beta1/basemodel/shared/shared.go:187-196) has exactly two call sites, both inside status writes: backends/pernode/status.go:140-170, whose updateModelStatusWithRetry returns at :148-152 BEFORE stamping whenever nodesReady, nodesFailed and the computed state are all unchanged, and backends/pvc/pvc.go:574-589, which stamps on every PVC-backend status write; basemodel/controller.go itself writes no status. So for an hf://, oci://, modelpack://, local:// or vendor:// model the two fields advance only when the node lists or the state change: an edit base-models.md:166 classifies as 'displayName, modelCapabilities or other descriptive metadata' ('Nodes keep their copy as it is') leaves observedGeneration behind metadata.generation indefinitely with nothing wrong, and a quiet healthy Ready model's lastReconcileTime goes stale — so the generated reference's 'controller is wedged' liveness reading misleads exactly the operator who checks it. A pvc:// model behaves differently: its backend rewrites status as it re-checks the claim and Job (including the TTL Job-recreation cycle documented at serve-models-from-pvc.md:299), so its stamps advance regularly. Docs gap verified today: base-models.md's Model lifecycle section (:211-223) documents only status.state, nodesReady and nodesFailed; a repo-wide grep for observedGeneration and lastReconcileTime under src/lib/content finds no authored mention for BaseModel/ClusterBaseModel (matches are ISVC, quota, WorkloadCluster, TrafficMap pages and the generated reference; benchmarks.md:197 is BenchmarkJob's own field). One independent concern: the freshness contract of the two status fields one commit added, as a short addition to the Model lifecycle section that owns model status. Distinct from merged #186 (the InferenceService-side ome.io/template-hash / observedTemplateHash analog on a different resource and page) and from the PVC condition reasons already documented at serve-models-from-pvc.md:276-298.

Scope: model-storage / model-status-freshness-fields. Other concerns are deferred.

How to test

  • Passed the documentation path and size guard (under 1000 added plus deleted lines; no file-count limit).
  • Passed an independent accuracy and single-concern review.
  • Passed git diff --check and website content/link tests, type checks, lint and production build.

Checklist

  • Every commit is signed off (git commit -s)
  • pnpm lint && pnpm check && pnpm test && pnpm build passes (run by the publisher on an isolated copy)

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 3bc061ad-4cef-43a1-a602-4e91491061e4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor Author

Documentation maintenance: ready. Unsuccessful content rounds: 0/3. Operational attempts: 0/3.

Head: 654000ddbcad98e7395c507943aa1ad0d182c0c0; reviewed main: 2d7041091d7a485b4b8c51ff3d6b2d09b17aa3f1.

The diff adds one subsection ("Status freshness") to base-models.md's Model lifecycle section, documenting the freshness contract of status.observedGeneration and status.lastReconcileTime — a single concern on the page that owns model status. Every claim verifies against the pinned OME code: the fields exist on ModelStatusSpec (basemodel_types.go:491-503) with the documented JSON names used in the kubectl example; StampObservedReconcile (shared/shared.go:187-196) sets ObservedGeneration to the current generation and LastReconcileTime to now, and its only two call sites are inside status writes; the per-node writer (pernode/status.go:148-152) early-returns before stamping when nodesReady, nodesFailed and the computed state are unchanged, and the agent's update handler (scout.go:436-455) diffs only labels, annotations and storage/download inputs, so a displayName edit (spec-inline, bumps generation) triggers no status write and observedGeneration lags indefinitely; the PVC writer (pvc.go:574-589) stamps unconditionally on every status write, which recurs via the Job TTL (default 3600) deletion firing the Owns(Job) watch and Job recreation — the cycle the linked serve-models-from-pvc.md#troubleshooting section documents. Both link anchors resolve, the style follows writing-docs.md (sentence-case heading, paired bash/output blocks with the guide's custom-columns pattern, relative .md links), no since badges or release comparisons are added, and no generated, nav or redirect files are touched. The context contains no review threads, so none are marked addressed.

Workflow evidence

Human review threads and CODEOWNER approval remain under repository policy.

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants