diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d44dd02..4f1765d2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,15 @@ All notable changes to ForgeLM are documented here. ### Fixed +- Say that the DeepSpeed preset names (`zero2`, `zero3`, `zero3_offload`) work only from a source checkout. They resolve to + `configs/deepspeed/.json` beside the package or in the current directory, and neither the wheel nor the Docker image ships + that folder, so on a `pip install` the names fail with a configuration error (and the unset-`deepspeed_config` default is + `zero2`). The configuration and distributed-training references, the troubleshooting entry (its sample error is now the real + message), the enterprise guide, the user manuals, `config_template.yaml` and the field description now say so and point at a + JSON path; the template's `torchrun … forgelm` launch line is `-m forgelm`, the only form that runs the checkout. The + unverified "`zero3_offload` fits 70B on 8×24 GB" sentence is gone, and the `zero2` preset's own start-up failure is noted + (#410). Shipping the presets in the package, and a test that resolves one from an installed wheel, are code changes and stay + open on the issues (#110, #111, #115). - Fix the KTO bullet on the Alignment overview user-manual page (English and Turkish): it named the fields `(prompt, response, good?)`, but the KTO loader reads `prompt`, `completion` and a boolean `label` (#965). diff --git a/config_template.yaml b/config_template.yaml index 8aec1398..e06e01f9 100644 --- a/config_template.yaml +++ b/config_template.yaml @@ -162,10 +162,10 @@ data: # dare_seed: 42 # DARE: RNG seed for the random drop mask # Distributed training (multi-GPU). Requires: pip install forgelm[distributed] -# Launch with: torchrun --nproc_per_node=N forgelm --config my_config.yaml +# Launch with: torchrun --nproc_per_node=N -m forgelm --config my_config.yaml # distributed: # strategy: "deepspeed" # "deepspeed" or "fsdp" -# deepspeed_config: "zero2" # preset: "zero2", "zero3", "zero3_offload" or path to JSON +# deepspeed_config: "ds.json" # path to a DeepSpeed JSON (the presets "zero2", "zero3", "zero3_offload" need a source checkout) # # --- FSDP options (when strategy: "fsdp") --- # # fsdp_strategy: "full_shard" # "full_shard", "shard_grad_op", "no_shard", "hybrid_shard" # # fsdp_auto_wrap: true diff --git a/docs/guides/enterprise_deployment.md b/docs/guides/enterprise_deployment.md index f9c70e60..c56d6f2d 100644 --- a/docs/guides/enterprise_deployment.md +++ b/docs/guides/enterprise_deployment.md @@ -121,7 +121,7 @@ ForgeLM sets `HF_HUB_OFFLINE=1`, `TRANSFORMERS_OFFLINE=1`, `HF_DATASETS_OFFLINE= ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # or "zero3", "zero3_offload" + deepspeed_config: "/etc/forgelm/ds_zero2.json" # your own DeepSpeed JSON; the zero2/zero3/zero3_offload preset names need a source checkout ``` ```bash diff --git a/docs/guides/troubleshooting-tr.md b/docs/guides/troubleshooting-tr.md index c4cb2408..29e6331c 100644 --- a/docs/guides/troubleshooting-tr.md +++ b/docs/guides/troubleshooting-tr.md @@ -363,11 +363,12 @@ nvidia-smi ### DeepSpeed Config bulunamadı ```text -FileNotFoundError: DeepSpeed preset 'zero2' not found +DeepSpeed preset 'zero2' not found at /configs/deepspeed/zero2.json ``` -ForgeLM proje kökünden çalıştırdığınızdan emin olun ya da mutlak yol -kullanın: +Ön ayar adları yalnızca bir kaynak checkout'unda çözülür (paketin yanındaki ya da geçerli dizindeki bir +`configs/deepspeed/` dizini); `pip install` veya Docker imajı bunları içermez. ForgeLM proje kökünden çalıştırın ya da +bir DeepSpeed JSON dosyasının yolunu kullanın: ```yaml distributed: diff --git a/docs/guides/troubleshooting.md b/docs/guides/troubleshooting.md index 7533eedb..9549875b 100644 --- a/docs/guides/troubleshooting.md +++ b/docs/guides/troubleshooting.md @@ -344,10 +344,12 @@ nvidia-smi ### DeepSpeed Config Not Found ```text -FileNotFoundError: DeepSpeed preset 'zero2' not found +DeepSpeed preset 'zero2' not found at /configs/deepspeed/zero2.json ``` -Ensure you're running from the ForgeLM project root, or use an absolute path: +The preset names resolve only from a source checkout (a `configs/deepspeed/` directory beside the package or in the +current directory); a `pip install` or the Docker image does not contain them. Run from the ForgeLM project root, or +use the path of a DeepSpeed JSON file: ```yaml distributed: diff --git a/docs/reference/configuration-tr.md b/docs/reference/configuration-tr.md index b78fa07f..1c512eaa 100644 --- a/docs/reference/configuration-tr.md +++ b/docs/reference/configuration-tr.md @@ -397,7 +397,7 @@ kaynağı kaydedilmemiştir. | Alan | Tip | Varsayılan | Açıklama | |------|-----|-----------|----------| | `strategy` | string | `null` | `"deepspeed"` veya `"fsdp"` (null = tek GPU) | -| `deepspeed_config` | string | `null` | Ön ayar (`"zero2"`, `"zero3"`, `"zero3_offload"`) veya JSON yolu | +| `deepspeed_config` | string | `null` | JSON yolu ya da yalnızca kaynak checkout'unda çözülen (pip veya Docker kurulumunda çözülmeyen) bir ön ayar adı (`"zero2"`, `"zero3"`, `"zero3_offload"`) | | `fsdp_strategy` | string | `"full_shard"` | `"full_shard"`, `"shard_grad_op"`, `"hybrid_shard"`, `"no_shard"` | | `fsdp_auto_wrap` | bool | `true` | Transformer katmanlarını otomatik sar | | `fsdp_offload` | bool | `false` | Parametreleri CPU'ya taşı | diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 0f83d16e..fa533638 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -309,7 +309,7 @@ silently extend the retention horizon by re-using a stale workspace. | Field | Type | Default | Description | |-------|------|---------|-------------| | `strategy` | string | `null` | `"deepspeed"` or `"fsdp"` (null = single GPU) | -| `deepspeed_config` | string | `null` | Preset (`"zero2"`, `"zero3"`, `"zero3_offload"`) or JSON path | +| `deepspeed_config` | string | `null` | JSON path, or a preset name (`"zero2"`, `"zero3"`, `"zero3_offload"`) that resolves only from a source checkout, not from a pip or Docker install | | `fsdp_strategy` | string | `"full_shard"` | `"full_shard"`, `"shard_grad_op"`, `"hybrid_shard"`, `"no_shard"` | | `fsdp_auto_wrap` | bool | `true` | Auto-wrap transformer layers | | `fsdp_offload` | bool | `false` | Offload parameters to CPU | diff --git a/docs/reference/distributed_training-tr.md b/docs/reference/distributed_training-tr.md index 32e6378f..bee3d2ef 100644 --- a/docs/reference/distributed_training-tr.md +++ b/docs/reference/distributed_training-tr.md @@ -49,7 +49,16 @@ torchrun --nproc_per_node=4 -m forgelm.cli --config my_config.yaml ## DeepSpeed Yapılandırması -ForgeLM, üç yerleşik DeepSpeed ön ayarı sunar. `deepspeed_config` parametresine bu isimlerden birini veya kendi JSON dosyanızın yolunu verin. +Depo, `configs/deepspeed/` altında üç DeepSpeed ön ayarı barındırır. `deepspeed_config` parametresine bunların adlarından birini veya kendi JSON dosyanızın yolunu verin. + +**Ön ayar adları yalnızca bir kaynak checkout'unda çalışır.** Kurulu paketin yanındaki ya da geçerli dizindeki +`configs/deepspeed/.json` dosyasına çözülürler. `pip install forgelm` wheel'i ve Docker imajı bu dosyaları içermez; +orada bir ön ayar adı yapılandırma hatasıyla (exit 1) başarısız olur. Deponun `configs/deepspeed/` klasöründeki JSON +dosyalarını kopyalayın ya da kendinizinkini yazın ve `deepspeed_config`'i onun yoluna ayarlayın. + +**Bilinen sorun ([#410](https://github.com/HodeTech/ForgeLM/issues/410)):** `deepspeed_config` ayarlanmadığında da +kullanılan `zero2` ön ayarı `allgather_bucket_size` değerini `"auto"` yapar; transformers'ın DeepSpeed entegrasyonu bu +değeri doldurmaz, bu yüzden onunla bir ZeRO-2 koşusu durur. Düzelene kadar kendi ZeRO-2 JSON'unuzu kullanın. ### Ön Ayarlar (Presets) @@ -64,8 +73,8 @@ ForgeLM, üç yerleşik DeepSpeed ön ayarı sunar. `deepspeed_config` parametre ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # Ön ayar adı - # deepspeed_config: "./my_ds_config.json" # Veya özel dosya yolu + deepspeed_config: "./my_ds_config.json" # Kendi JSON'unuzun yolu (her kurulumda çalışır) + # deepspeed_config: "zero2" # Ön ayar adı: yalnızca kaynak checkout'u ``` ### ZeRO Aşama Karşılaştırması diff --git a/docs/reference/distributed_training.md b/docs/reference/distributed_training.md index b83b29f9..8d9f6f29 100644 --- a/docs/reference/distributed_training.md +++ b/docs/reference/distributed_training.md @@ -49,7 +49,16 @@ torchrun --nproc_per_node=4 -m forgelm.cli --config my_config.yaml ## DeepSpeed Configuration -ForgeLM provides three built-in DeepSpeed presets. Set `deepspeed_config` to one of these names, or provide a path to your own JSON file. +The repository ships three DeepSpeed presets under `configs/deepspeed/`. Set `deepspeed_config` to one of their names, or provide a path to your own JSON file. + +**The preset names work only from a source checkout.** They resolve to `configs/deepspeed/.json` beside the +installed package or in the current directory. A `pip install forgelm` wheel and the Docker image do not contain those +files, so there a preset name fails with a configuration error (exit 1). Copy the JSON files from the repository's +`configs/deepspeed/` folder, or write your own, and set `deepspeed_config` to its path. + +**Known issue ([#410](https://github.com/HodeTech/ForgeLM/issues/410)):** the `zero2` preset, which is also used when +`deepspeed_config` is unset, sets `allgather_bucket_size` to `"auto"`, a value transformers' DeepSpeed integration does +not fill in, so a ZeRO-2 run with it aborts. Use your own ZeRO-2 JSON until that is fixed. ### Presets @@ -64,8 +73,8 @@ ForgeLM provides three built-in DeepSpeed presets. Set `deepspeed_config` to one ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # Preset name - # deepspeed_config: "./my_ds_config.json" # Or custom path + deepspeed_config: "./my_ds_config.json" # Path to your own JSON (works on every install) + # deepspeed_config: "zero2" # Preset name: source checkout only ``` ### ZeRO Stage Comparison diff --git a/docs/usermanuals/en/reference/configuration.md b/docs/usermanuals/en/reference/configuration.md index c4ba73c2..82e25c66 100644 --- a/docs/usermanuals/en/reference/configuration.md +++ b/docs/usermanuals/en/reference/configuration.md @@ -228,7 +228,7 @@ The merge field is `method` (not `algorithm`), and there is no `dare_ties` choic ```yaml distributed: strategy: null # null (single-GPU, no distributed wrapping) | deepspeed | fsdp - deepspeed_config: null # path to a DeepSpeed JSON, or preset name: zero2 | zero3 | zero3_offload + deepspeed_config: null # path to a DeepSpeed JSON (preset names zero2 | zero3 | zero3_offload need a source checkout) fsdp_strategy: "full_shard" # full_shard | shard_grad_op | no_shard | hybrid_shard fsdp_auto_wrap: true # auto-wrap transformer layers (recommended) fsdp_offload: false # offload FSDP parameters to CPU between forward/backward diff --git a/docs/usermanuals/en/training/distributed.md b/docs/usermanuals/en/training/distributed.md index 7aa622ff..41f23b79 100644 --- a/docs/usermanuals/en/training/distributed.md +++ b/docs/usermanuals/en/training/distributed.md @@ -66,7 +66,7 @@ ZeRO-2 shards the optimiser state (the heaviest VRAM component for adaptive opti ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # preset name (or a filesystem path to a DeepSpeed JSON) + deepspeed_config: "./ds_zero2.json" # a filesystem path to a DeepSpeed JSON (preset names such as "zero2" need a source checkout) training: gradient_accumulation_steps: 4 @@ -95,7 +95,7 @@ training: gradient_accumulation_steps: 8 ``` -The `zero3_offload` preset fits 70B on 8×24 GB by offloading optimiser state and parameters to CPU. ZeRO-Infinity's NVMe offload isn't a built-in preset — point `deepspeed_config` at a custom DeepSpeed JSON with `offload_param`/`offload_optimizer` set to `device: nvme` for that case. +The `zero3_offload` preset offloads optimiser state and parameters to CPU, trading speed for GPU memory; whether a given model fits depends on your GPU memory and CPU RAM. The preset names `zero2`, `zero3` and `zero3_offload` resolve only from a source checkout (the JSON files live in the repository's `configs/deepspeed/` folder, not in the pip wheel or the Docker image), and the `zero2` preset is currently broken ([#410](https://github.com/HodeTech/ForgeLM/issues/410)), so give `deepspeed_config` the path of a JSON file you control. ZeRO-Infinity's NVMe offload isn't a preset — point `deepspeed_config` at a custom DeepSpeed JSON with `offload_param`/`offload_optimizer` set to `device: nvme` for that case. | Model | GPUs | ZeRO-3 + offload? | |---|---|---| diff --git a/docs/usermanuals/tr/reference/configuration.md b/docs/usermanuals/tr/reference/configuration.md index 39f8e44b..4ac4d99c 100644 --- a/docs/usermanuals/tr/reference/configuration.md +++ b/docs/usermanuals/tr/reference/configuration.md @@ -230,7 +230,7 @@ Merge alanı `algorithm` değil `method`'dur, `dare_ties` seçeneği, `base_mode ```yaml distributed: strategy: null # null (tek GPU, dağıtık sarmalama yok) | deepspeed | fsdp - deepspeed_config: null # bir DeepSpeed JSON'a yol, veya preset adı: zero2 | zero3 | zero3_offload + deepspeed_config: null # bir DeepSpeed JSON'a yol (preset adları zero2 | zero3 | zero3_offload kaynak checkout'u ister) fsdp_strategy: "full_shard" # full_shard | shard_grad_op | no_shard | hybrid_shard fsdp_auto_wrap: true # transformer katmanlarını otomatik sarmala (önerilir) fsdp_offload: false # forward/backward arasında FSDP parametrelerini CPU'ya offload et diff --git a/docs/usermanuals/tr/training/distributed.md b/docs/usermanuals/tr/training/distributed.md index 4dcfd239..53d2ae78 100644 --- a/docs/usermanuals/tr/training/distributed.md +++ b/docs/usermanuals/tr/training/distributed.md @@ -66,7 +66,7 @@ ZeRO-2 optimizer state'i sharder (Adam gibi adaptif optimizer'larda en ağır VR ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # preset adı (veya bir DeepSpeed JSON dosya yolu) + deepspeed_config: "./ds_zero2.json" # bir DeepSpeed JSON dosya yolu ("zero2" gibi preset adları kaynak checkout'u ister) training: gradient_accumulation_steps: 4 @@ -95,7 +95,7 @@ training: gradient_accumulation_steps: 8 ``` -`zero3_offload` preset'i, optimizer state ve parametreleri CPU'ya boşaltarak 70B'i 8×24 GB'a sığdırır. ZeRO-Infinity'nin NVMe offload'u hazır bir preset değildir — bunun için `deepspeed_config`'i, `offload_param`/`offload_optimizer` alanları `device: nvme` olarak ayarlanmış özel bir DeepSpeed JSON dosyasına yönlendirin. +`zero3_offload` preset'i optimizer state ve parametreleri CPU'ya boşaltır, hızdan GPU belleği kazandırır; belirli bir modelin sığıp sığmayacağı GPU belleğinize ve CPU RAM'inize bağlıdır. `zero2`, `zero3` ve `zero3_offload` preset adları yalnızca kaynak checkout'unda çözülür (JSON dosyaları pip wheel'inde ya da Docker imajında değil, deponun `configs/deepspeed/` klasöründedir) ve `zero2` preset'i şu an bozuktur ([#410](https://github.com/HodeTech/ForgeLM/issues/410)); bu yüzden `deepspeed_config`'e kontrolünüzdeki bir JSON dosyasının yolunu verin. ZeRO-Infinity'nin NVMe offload'u bir preset değildir — bunun için `deepspeed_config`'i, `offload_param`/`offload_optimizer` alanları `device: nvme` olarak ayarlanmış özel bir DeepSpeed JSON dosyasına yönlendirin. | Model | GPU | ZeRO-3 + offload? | |---|---|---| diff --git a/forgelm/config.py b/forgelm/config.py index 4114f2df..135a6deb 100644 --- a/forgelm/config.py +++ b/forgelm/config.py @@ -768,7 +768,7 @@ class DistributedConfig(BaseModel): ) deepspeed_config: Optional[str] = Field( default=None, - description="DeepSpeed config: filesystem path to a DS JSON OR preset name (`zero2`, `zero3`, `zero3_offload`).", + description="DeepSpeed config: filesystem path to a DS JSON. The preset names (`zero2`, `zero3`, `zero3_offload`) resolve only from a source checkout, not a pip or Docker install.", ) fsdp_strategy: Literal["full_shard", "shard_grad_op", "no_shard", "hybrid_shard"] = Field( default="full_shard",