fix(config): reject FSDP strategies that transformers' FSDP2 default cannot run, and warn about two ignored knobs

CHANGELOG.md

@@ -68,6 +68,12 @@ All notable changes to ForgeLM are documented here.
 
 ### Fixed
 
+- `distributed.fsdp_strategy: no_shard` and `hybrid_shard` are rejected when the config loads (exit 1). They were documented, passed `--dry-run`,
+  and then failed with exit 2 at trainer construction (`fsdp=hybrid_shard is only available with FSDP1`), because transformers 5.10+ runs
+  FSDP2 by default. `fsdp_auto_wrap: false` and `fsdp_backward_prefetch: backward_post` are still accepted but change nothing under FSDP2;
+  ForgeLM now logs a warning when either is set, and the field descriptions, reference pages, user manuals and `config_template.yaml`
+  say so. Moving to FSDP2 settings (`fsdp=True` plus an explicit `fsdp_config`) is not part of this change; the string form is
+  deprecated in transformers and slated for removal in v5.20 (#104).
 - Fix the KTO bullet on the Alignment overview user-manual page (English and Turkish): it named the
   fields `(prompt, response, good?)`, but the KTO loader reads `prompt`, `completion` and a boolean
   `label` (#965).

config_template.yaml

@@ -167,7 +167,7 @@ data:
 #   strategy: "deepspeed"           # "deepspeed" or "fsdp"
 #   deepspeed_config: "zero2"       # preset: "zero2", "zero3", "zero3_offload" or path to JSON
 #   # --- FSDP options (when strategy: "fsdp") ---
-#   # fsdp_strategy: "full_shard"   # "full_shard", "shard_grad_op", "no_shard", "hybrid_shard"
+#   # fsdp_strategy: "full_shard"   # "full_shard" or "shard_grad_op" (FSDP2 supports no other)
 #   # fsdp_auto_wrap: true
 #   # fsdp_offload: false
 

docs/reference/configuration-tr.md

@@ -398,10 +398,10 @@ kaynağı kaydedilmemiştir.
 |------|-----|-----------|----------|
 | `strategy` | string | `null` | `"deepspeed"` veya `"fsdp"` (null = tek GPU) |
 | `deepspeed_config` | string | `null` | Ön ayar (`"zero2"`, `"zero3"`, `"zero3_offload"`) veya JSON yolu |
-| `fsdp_strategy` | string | `"full_shard"` | `"full_shard"`, `"shard_grad_op"`, `"hybrid_shard"`, `"no_shard"` |
-| `fsdp_auto_wrap` | bool | `true` | Transformer katmanlarını otomatik sar |
+| `fsdp_strategy` | string | `"full_shard"` | `"full_shard"` veya `"shard_grad_op"` (`"hybrid_shard"` ve `"no_shard"` reddedilir: FSDP2 bunları desteklemez) |
+| `fsdp_auto_wrap` | bool | `true` | Transformer katmanlarını otomatik sar (transformers 5.10+'ta yok sayılır, katmanlar hep sarılır) |
 | `fsdp_offload` | bool | `false` | Parametreleri CPU'ya taşı |
-| `fsdp_backward_prefetch` | string | `"backward_pre"` | `"backward_pre"` veya `"backward_post"` |
+| `fsdp_backward_prefetch` | string | `"backward_pre"` | `"backward_pre"` veya `"backward_post"` (transformers 5.10+'ta yok sayılır; FSDP2 prefetch politikası almaz) |
 | `fsdp_state_dict_type` | string | `"FULL_STATE_DICT"` | `"FULL_STATE_DICT"` veya `"SHARDED_STATE_DICT"` |
 
 ## `synthetic` (İsteğe bağlı — Sentetik Veri Üretimi)

docs/reference/configuration.md

@@ -310,10 +310,10 @@ silently extend the retention horizon by re-using a stale workspace.
 |-------|------|---------|-------------|
 | `strategy` | string | `null` | `"deepspeed"` or `"fsdp"` (null = single GPU) |
 | `deepspeed_config` | string | `null` | Preset (`"zero2"`, `"zero3"`, `"zero3_offload"`) or JSON path |
-| `fsdp_strategy` | string | `"full_shard"` | `"full_shard"`, `"shard_grad_op"`, `"hybrid_shard"`, `"no_shard"` |
-| `fsdp_auto_wrap` | bool | `true` | Auto-wrap transformer layers |
+| `fsdp_strategy` | string | `"full_shard"` | `"full_shard"` or `"shard_grad_op"` (`"hybrid_shard"` and `"no_shard"` are rejected: FSDP2 does not support them) |
+| `fsdp_auto_wrap` | bool | `true` | Auto-wrap transformer layers (ignored on transformers 5.10+, which always wraps them) |
 | `fsdp_offload` | bool | `false` | Offload parameters to CPU |
-| `fsdp_backward_prefetch` | string | `"backward_pre"` | `"backward_pre"` or `"backward_post"` |
+| `fsdp_backward_prefetch` | string | `"backward_pre"` | `"backward_pre"` or `"backward_post"` (ignored on transformers 5.10+, whose FSDP2 takes no prefetch policy) |
 | `fsdp_state_dict_type` | string | `"FULL_STATE_DICT"` | `"FULL_STATE_DICT"` or `"SHARDED_STATE_DICT"` |
 
 ---

docs/reference/distributed_training-tr.md

@@ -120,21 +120,23 @@ PyTorch Fully Sharded Data Parallel (FSDP), DeepSpeed'e alternatif olarak PyTorc
 |----------|----------|-----------------|
 | `full_shard` | Parametreleri, gradyanları ve optimizer durumlarını böl | En Yüksek |
 | `shard_grad_op` | Yalnızca gradyanları ve optimizer durumlarını böl | Orta |
-| `hybrid_shard` | Düğüm içi tam bölme, düğümler arası çoğaltma | Çoklu Düğüm |
-| `no_shard` | Standart DDP (bölme yok) | Yok |
+
+`hybrid_shard` ve `no_shard` **kabul edilmez**: `fsdp_strategy` `full_shard` ya da `shard_grad_op` olmalıdır. ForgeLM'in gerektirdiği transformers 5.10 ve sonrası varsayılan olarak FSDP2 çalıştırır ve FSDP2 bu ikisini de desteklemez: eskiden `--dry-run`'ı geçip model yüklendikten sonra exit 2 ile (`fsdp=hybrid_shard is only available with FSDP1`) patlıyorlardı, şimdi doğrulamada exit 1 ile reddediliyorlar.
 
 ### Kullanım
 
 ```yaml
 distributed:
   strategy: "fsdp"
   fsdp_strategy: "full_shard"
-  fsdp_auto_wrap: true                    # Auto-wrap transformer layers
+  fsdp_auto_wrap: true                    # transformers 5.10+'ta yok sayılır (FSDP2 transformer katmanlarını hep sarar)
   fsdp_offload: false                     # Offload parameters AND gradients (between forward/backward) to CPU
-  fsdp_backward_prefetch: "backward_pre"  # Prefetch strategy
+  fsdp_backward_prefetch: "backward_pre"  # transformers 5.10+'ta yok sayılır (FSDP2 prefetch politikası almaz)
   fsdp_state_dict_type: "FULL_STATE_DICT" # State dict handling
 ```
 
+`fsdp_auto_wrap: false` ve `fsdp_backward_prefetch: backward_post` hâlâ kabul edilir, ancak FSDP2'de hiçbir şeyi değiştirmezler; bu yüzden ikisinden biri varsayılanından farklı ayarlandığında ForgeLM bir uyarı loglar.
+
 ### FSDP'yi DeepSpeed yerine ne zaman seçmeli
 
 - **FSDP**: Native PyTorch, ekstra bağımlılık yok, daha basit kurulum,

docs/reference/distributed_training.md

@@ -118,21 +118,23 @@ PyTorch Fully Sharded Data Parallel (FSDP) is an alternative to DeepSpeed that's
 |----------|-------------|----------------|
 | `full_shard` | Shard parameters, gradients, and optimizer states | Highest |
 | `shard_grad_op` | Shard gradients and optimizer states only | Medium |
-| `hybrid_shard` | Full shard within nodes, replicate across nodes | Multi-node |
-| `no_shard` | Standard DDP (no sharding) | None |
+
+`hybrid_shard` and `no_shard` are **not accepted**: `fsdp_strategy` must be `full_shard` or `shard_grad_op`. transformers 5.10 and later, which ForgeLM requires, run FSDP2 by default, and FSDP2 supports neither: they used to pass `--dry-run` and then fail with exit 2 (`fsdp=hybrid_shard is only available with FSDP1`) after the model was loaded, and now fail validation with exit 1.
 
 ### Usage
 
 ```yaml
 distributed:
   strategy: "fsdp"
   fsdp_strategy: "full_shard"
-  fsdp_auto_wrap: true          # Auto-wrap transformer layers
+  fsdp_auto_wrap: true          # Ignored on transformers 5.10+ (FSDP2 always wraps transformer layers)
   fsdp_offload: false           # Offload parameters AND gradients (between forward/backward) to CPU
-  fsdp_backward_prefetch: "backward_pre"  # Prefetch strategy
+  fsdp_backward_prefetch: "backward_pre"  # Ignored on transformers 5.10+ (FSDP2 takes no prefetch policy)
   fsdp_state_dict_type: "FULL_STATE_DICT" # State dict handling
 ```
 
+`fsdp_auto_wrap: false` and `fsdp_backward_prefetch: backward_post` are still accepted, but under FSDP2 they change nothing, so ForgeLM logs a warning when either is set away from its default.
+
 ### When to Choose FSDP over DeepSpeed
 
 - **FSDP**: Native PyTorch, no extra dependency, simpler setup, good for most use cases

docs/usermanuals/en/reference/configuration.md

@@ -229,10 +229,10 @@ The merge field is `method` (not `algorithm`), and there is no `dare_ties` choic
 distributed:
   strategy: null                              # null (single-GPU, no distributed wrapping) | deepspeed | fsdp
   deepspeed_config: null                      # path to a DeepSpeed JSON, or preset name: zero2 | zero3 | zero3_offload
-  fsdp_strategy: "full_shard"                 # full_shard | shard_grad_op | no_shard | hybrid_shard
-  fsdp_auto_wrap: true                        # auto-wrap transformer layers (recommended)
+  fsdp_strategy: "full_shard"                 # full_shard | shard_grad_op
+  fsdp_auto_wrap: true                        # ignored on transformers 5.10+ (FSDP2 always wraps transformer layers)
   fsdp_offload: false                         # offload FSDP parameters to CPU between forward/backward
-  fsdp_backward_prefetch: "backward_pre"      # backward_pre | backward_post
+  fsdp_backward_prefetch: "backward_pre"      # backward_pre | backward_post; ignored on transformers 5.10+
   fsdp_state_dict_type: "FULL_STATE_DICT"     # FULL_STATE_DICT | SHARDED_STATE_DICT
 ```
 

docs/usermanuals/en/training/distributed.md

@@ -111,12 +111,14 @@ FSDP shards similarly to ZeRO-3 but uses PyTorch's native FullyShardedDataParall
 ```yaml
 distributed:
   strategy: "fsdp"
-  fsdp_strategy: "full_shard"             # full_shard | shard_grad_op | no_shard | hybrid_shard
-  fsdp_auto_wrap: true                    # auto-wrap transformer layers (recommended)
+  fsdp_strategy: "full_shard"             # full_shard | shard_grad_op
+  fsdp_auto_wrap: true                    # ignored on transformers 5.10+ (FSDP2 always wraps transformer layers)
   fsdp_offload: false                     # offload parameters to CPU between forward/backward
   fsdp_state_dict_type: "FULL_STATE_DICT" # FULL_STATE_DICT | SHARDED_STATE_DICT
 ```
 
+`no_shard` and `hybrid_shard` are rejected when the config loads: transformers 5.10+, which ForgeLM requires, runs FSDP2, and FSDP2 supports neither. `fsdp_auto_wrap: false` and `fsdp_backward_prefetch: backward_post` are still accepted but change nothing under FSDP2, so ForgeLM logs a warning when either is set.
+
 `DistributedConfig` has no `fsdp_auto_wrap_policy` or `fsdp_offload_params` field — auto-wrap is the plain boolean `fsdp_auto_wrap`, and CPU offload is `fsdp_offload` (no `_params` suffix).
 
 ## Gradient accumulation

docs/usermanuals/tr/reference/configuration.md

@@ -231,10 +231,10 @@ Merge alanı `algorithm` değil `method`'dur, `dare_ties` seçeneği, `base_mode
 distributed:
   strategy: null                              # null (tek GPU, dağıtık sarmalama yok) | deepspeed | fsdp
   deepspeed_config: null                      # bir DeepSpeed JSON'a yol, veya preset adı: zero2 | zero3 | zero3_offload
-  fsdp_strategy: "full_shard"                 # full_shard | shard_grad_op | no_shard | hybrid_shard
-  fsdp_auto_wrap: true                        # transformer katmanlarını otomatik sarmala (önerilir)
+  fsdp_strategy: "full_shard"                 # full_shard | shard_grad_op
+  fsdp_auto_wrap: true                        # transformers 5.10+'ta yok sayılır (FSDP2 transformer katmanlarını hep sarar)
   fsdp_offload: false                         # forward/backward arasında FSDP parametrelerini CPU'ya offload et
-  fsdp_backward_prefetch: "backward_pre"      # backward_pre | backward_post
+  fsdp_backward_prefetch: "backward_pre"      # backward_pre | backward_post; transformers 5.10+'ta yok sayılır
   fsdp_state_dict_type: "FULL_STATE_DICT"     # FULL_STATE_DICT | SHARDED_STATE_DICT
 ```
 

docs/usermanuals/tr/training/distributed.md

@@ -111,12 +111,14 @@ FSDP, ZeRO-3 gibi sharder ama PyTorch'un yerli FullyShardedDataParallel'ini kull
 ```yaml
 distributed:
   strategy: "fsdp"
-  fsdp_strategy: "full_shard"             # full_shard | shard_grad_op | no_shard | hybrid_shard
-  fsdp_auto_wrap: true                    # transformer katmanlarını otomatik sar (önerilir)
+  fsdp_strategy: "full_shard"             # full_shard | shard_grad_op
+  fsdp_auto_wrap: true                    # transformers 5.10+'ta yok sayılır (FSDP2 transformer katmanlarını hep sarar)
   fsdp_offload: false                     # forward/backward arasında parametreleri CPU'ya boşalt
   fsdp_state_dict_type: "FULL_STATE_DICT" # FULL_STATE_DICT | SHARDED_STATE_DICT
 ```
 
+`no_shard` ve `hybrid_shard` config yüklenirken reddedilir: ForgeLM'in gerektirdiği transformers 5.10+ FSDP2 çalıştırır ve FSDP2 bu ikisini desteklemez. `fsdp_auto_wrap: false` ve `fsdp_backward_prefetch: backward_post` hâlâ kabul edilir ancak FSDP2'de hiçbir şeyi değiştirmez; bu yüzden ikisinden biri ayarlandığında ForgeLM bir uyarı loglar.
+
 `DistributedConfig`'te `fsdp_auto_wrap_policy` veya `fsdp_offload_params` alanı yoktur — auto-wrap düz bir `fsdp_auto_wrap` boolean'ıdır, CPU offload ise `fsdp_offload`'dur (`_params` eki yoktur).
 
 ## Gradient accumulation

forgelm/cli/_config_load.py

@@ -55,3 +55,27 @@ def _apply_offline_flag(config, offline_arg: bool) -> None:
         os.environ["TRANSFORMERS_OFFLINE"] = "1"
         os.environ["HF_DATASETS_OFFLINE"] = "1"
         logger.info("Offline mode enabled. All HF Hub network calls are disabled.")
+
+
+def _warn_inert_fsdp_options(config) -> None:
+    """Say so when an FSDP knob that transformers' FSDP2 default ignores is set away from its default.
+
+    ``fsdp_auto_wrap: false`` and ``fsdp_backward_prefetch: backward_post`` are still accepted, but under
+    transformers 5.10+ (FSDP2) the transformer-layer wrap policy is always applied and no backward prefetch is
+    passed on, so the run behaves as if the defaults had been kept.  (This lives here, not in the schema, to keep
+    ``config.py`` within its deferred-split line budget.)
+    """
+    dist = getattr(config, "distributed", None)
+    if getattr(dist, "strategy", None) != "fsdp":
+        return
+    if not dist.fsdp_auto_wrap:
+        logger.warning(
+            "distributed.fsdp_auto_wrap=false has no effect: transformers 5.10+ runs FSDP2, which always wraps "
+            "transformer layers."
+        )
+    if dist.fsdp_backward_prefetch != "backward_pre":
+        logger.warning(
+            "distributed.fsdp_backward_prefetch=%r has no effect: transformers 5.10+ runs FSDP2, which does not take "
+            "a backward-prefetch policy.",
+            dist.fsdp_backward_prefetch,
+        )

forgelm/cli/_dispatch.py

@@ -10,7 +10,7 @@
 import sys
 
 from .._strict_json import dumps_strict
-from ._config_load import _apply_offline_flag, _load_config_or_exit
+from ._config_load import _apply_offline_flag, _load_config_or_exit, _warn_inert_fsdp_options
 from ._exit_codes import EXIT_CONFIG_ERROR, EXIT_SUCCESS, EXIT_TRAINING_ERROR, _clamp_exit_code
 from ._logging import _setup_logging, logger
 from ._no_train_modes import _maybe_run_no_train_mode
@@ -241,6 +241,7 @@ def _main_inner() -> None:
 
     config = _load_config_or_exit(args.config, json_output)
     _apply_offline_flag(config, args.offline)
+    _warn_inert_fsdp_options(config)
 
     if config.pipeline is not None:
         _dispatch_pipeline_mode(config, args, json_output)

forgelm/config.py

@@ -770,17 +770,17 @@ class DistributedConfig(BaseModel):
         default=None,
         description="DeepSpeed config: filesystem path to a DS JSON OR preset name (`zero2`, `zero3`, `zero3_offload`).",
     )
-    fsdp_strategy: Literal["full_shard", "shard_grad_op", "no_shard", "hybrid_shard"] = Field(
+    fsdp_strategy: Literal["full_shard", "shard_grad_op"] = Field(
         default="full_shard",
-        description="FSDP sharding strategy.  `full_shard` is the production default; `hybrid_shard` for multi-node intra-node sharding.",
+        description="FSDP sharding strategy.  `full_shard` is the production default; `shard_grad_op` shards gradients and optimizer states only.  `no_shard` and `hybrid_shard` are not accepted: transformers 5.10+ runs FSDP2, which supports neither.",
     )
-    fsdp_auto_wrap: bool = Field(default=True, description="FSDP: auto-wrap transformer layers (recommended).")
+    fsdp_auto_wrap: bool = Field(default=True, description="FSDP auto-wrap; ignored by FSDP2 (transformers 5.10+).")
     fsdp_offload: bool = Field(
         default=False, description="FSDP: offload parameters to CPU between forward and backward (slower, less VRAM)."
     )
     fsdp_backward_prefetch: Literal["backward_pre", "backward_post"] = Field(
         default="backward_pre",
-        description="FSDP backward-prefetch policy.  `backward_pre` overlaps comm + compute; `backward_post` is more memory-conservative.",
+        description="FSDP backward-prefetch policy.  Not honoured by transformers 5.10+ (FSDP2 prefetches on its own); `backward_post` logs a warning.",
     )
     fsdp_state_dict_type: Literal["FULL_STATE_DICT", "SHARDED_STATE_DICT"] = Field(
         default="FULL_STATE_DICT",