Repository navigation
添加面向新同学的 Daft 参数调优文档 - #20
Merged
Merged
Conversation
Beginner-friendly Chinese doc: explains the four Daft concepts that matter here (runner, partition, morsel, actor UDF), how env knobs are wired through configure_daft_runner, a parameter reference grouped by concern, a symptom-based troubleshooting table, and a recommended tuning order with a large-run .env starting point.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
大规模在 Ray 上跑 analyze/ingest 时有不少参数可调(并行度、morsel、LLM 并发、S3、写出布局),但这些参数的含义和调整方向依赖对 Daft 执行模型的理解,对不熟悉 Daft 的同学门槛较高,容易调错方向(比如 MinIO 限流时反而去加重试)。
改动
新增
docs/daft-tuning.md,面向对 Daft 原理不太熟的同学:configure_daft_runner()/daft_io_config()接通,改.env重跑即可;新增 Daft 参数必须接线,否则是死配置。.env起点模板。说明
文档描述的参数接通行为依赖 #18(
configure_daft_runner接通执行配置、ANALYZE_NUM_PARTITIONS等),建议在 #18 之后合并。验证
纯文档改动,无代码变更。