Link the TRL example and the multi-harness article from harbor_env

[sergiopaniego]

Summary

envs/harbor_env/README.md (and its docs page) never said how to train on these rollouts. This adds a short "Training with TRL" section after the Quick Start that links:

• TRL's examples/async_grpo_harbor, the complete training script, with a Hugging Face Jobs launcher;
• TRL's OpenEnv guide;
• The ultimate guide to multi-harness RL, which trains one policy across four harnesses through this environment, its runnable FineEnvs tutorial, and its collection.

The TRL guide is linked without an anchor, because its harness section is being renamed in huggingface/trl#7457. The FineEnvs tutorial link goes live when adithya-s-k/FineEnvs#29 merges.

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation
• New environment
• Refactoring

Alignment Checklist

Before submitting, verify:

• I have read .claude/docs/PRINCIPLES.md and this PR aligns with our principles
• I have checked .claude/docs/INVARIANTS.md and no invariants are violated
• I have run /pre-submit-pr (or bash .claude/hooks/lint.sh and tests) and addressed all issues

RFC Status

• Not required (bug fix, docs, minor refactoring)
• RFC exists: #___
• RFC needed (will create before merge)

Test Plan

scripts/sync_env_docs.py --check passes. All links return 200 except the FineEnvs tutorial, which lands with adithya-s-k/FineEnvs#29.

Claude Code Review

N/A. This PR was written with AI assistance (Claude Code) and reviewed by me.

---

Note

Low Risk
Documentation-only edits with external links; no runtime or API behavior changes.

Overview
Adds a Training with TRL section to the Harbor docs (envs/harbor_env/README.md and the synced docs/source/environments/harbor.md), placed after Quick Start and before The web UI.

The section briefly describes training on Harbor rollouts with TRL’s AsyncGRPOTrainer (HarnessRolloutWorker, HarborSessionFactory, per-session TrainingTrace) and points readers to the TRL async_grpo_harbor example, TRL’s OpenEnv guide, and the multi-harness RL Space plus FineEnvs tutorial and Hub collection.

^{Reviewed by Cursor Bugbot for commit 06c4af3. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Alignment Review Report

Automated Checks

• Lint: N/A (docs-only change; no Python files touched, and uv/.venv aren't available in this sandbox to run ruff/usort anyway)
• Debug code: CLEAN - check-debug.sh found print/TODO hits only in pre-existing files (test_local_docker_provider.py, mcp_client.py, uv_provider.py, rollout.py); none in the files this PR touches

Open RFCs Context

• RFC 012 ("Harbor capture purpose, provider fidelity and live-session ownership", In Review) defines the exact TrainingTrace/export_training_contract contract that this PR's new "Training with TRL" section references by name. The added docs are consistent with — not in conflict with — that RFC's terminology.
• RFC 010 and RFC 011 are Draft but unrelated to harbor_env docs.
• No other open RFC touches the Harbor docs page or TRL integration.

Tier 1: Fixes Required

None. The diff only adds an identical 14-line "Training with TRL" section with three external links to both docs/source/environments/harbor.md and envs/harbor_env/README.md. I verified:

• The two files stay byte-identical for the new section (ran scripts/sync_env_docs.py --check → "All environment stubs are present and up to date").
• The new ## Training with TRL heading matches the existing ##-level sectioning used elsewhere in the page.
• No code, API, or invariant-relevant files are touched.

Tier 2: Alignment Discussion

Principle Conflicts

None identified. This is a pure documentation addition (external links to a TRL training example, a TRL guide, and a multi-harness RL writeup) — no code paths, no API surface, no reward/training logic change. It doesn't touch the Gym-like/MCP boundary, client-server separation, or reward-in-environment invariants.

RFC Conflicts

None identified. The referenced concepts (AsyncGRPOTrainer, HarnessRolloutWorker, HarborSessionFactory, TrainingTrace) align with the training-contract terminology already defined in RFC 012 (In Review) rather than contradicting it.

Summary

• 0 mechanical issues to fix
• 0 alignment points for human review
• 0 RFC conflicts to discuss

This is a safe, mergeable docs-only change.

  

_{Sent by Cursor Automation: Pre-review}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.