Skip to content

Commit b41f313

Browse files
authored
[docs](load) update strict mode description and examples (#3545)
## Versions - [x] dev - [x] 4.x - [ ] 3.x - [ ] 2.1 ## Languages - [x] Chinese - [x] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built
1 parent 1ff4d05 commit b41f313

File tree

4 files changed

+70
-16
lines changed

4 files changed

+70
-16
lines changed

docs/data-operate/import/handling-messy-data.md

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ The system employs different strategies based on the strict mode setting:
3535
| ----------------- | -------------------- | ----------------------------- | ----------- | ------ |
3636
| NULL | \N | NULL | ON/OFF | NULL |
3737
| Non-NULL | "abc" or 2000 | NULL | ON | Invalid (Filtered) |
38-
| Non-NULL | "abc" | NULL | OFF | NULL |
38+
| Non-NULL | "abc" or 2000 | NULL | OFF | NULL |
3939
| Non-NULL | 1 | 1 | ON/OFF | Loaded Successfully |
4040

4141
:::tip
@@ -51,14 +51,28 @@ The system employs different strategies based on the strict mode setting:
5151
| NULL | \N | NULL | ON/OFF | NULL |
5252
| Non-NULL | aaa | NULL | ON | Invalid (Filtered) |
5353
| Non-NULL | aaa | NULL | OFF | NULL |
54-
| Non-NULL | 1 or 10 | 1 or 10 | ON/OFF | Loaded Successfully |
54+
| Non-NULL | 10 | NULL (overflow) | ON | Filtered |
55+
| Non-NULL | 10 | NULL (overflow) | OFF | NULL |
5556

5657
:::tip
5758
1. The column in the table allows NULL values
5859

5960
2. `abc` becomes NULL after conversion to Decimal due to type issues. When strict mode is ON, such data will be filtered out. When OFF, NULL will be loaded.
6061

61-
3. Although `10` exceeds the range, since its type meets decimal requirements, strict mode does not affect it.
62+
3. `10` is a value outside the range of Decimal(1, 0) and will be converted to NULL. It will be filtered out if strict mode is enabled. If strict mode is disabled, `null` will be imported.
63+
:::
64+
65+
**3. Example of a column type char(10)**
66+
67+
| Original data type | Original Data Example | Value after conversion to char(10) | Strict mode | Result |
68+
| ------------------ | --------------------- | ---------------------------------- | ---------- | ------ |
69+
| NULL | \N | NULL | ON/OFF | NULL |
70+
| Non-NULL | a1234567890 | a1234567890 | ON | Too long, filtered |
71+
| Non-NULL | a1234567890 | a1234567890 | OFF | a123456789 (truncated) |
72+
73+
:::tip
74+
75+
1. The column in the table allows NULL values
6276
:::
6377

6478
### Enable Strict Mode
@@ -183,5 +197,5 @@ SET insert_max_filter_ratio = 0.1;
183197
INSERT INTO test_table FROM S3/HDFS/LOCAL();```
184198
185199
:::tip
186-
For Insert Into statements, `insert_max_filter_ratio` only takes effect when `enable_insert_strict = false`, and only applies to `INSERT INTO FROM S3/HDFS/LOCAL()` syntax. The default value is 1.0, which means that all abnormal data are allowed to be filtered.
200+
For Insert Into statements, `insert_max_filter_ratio` only takes effect when `enable_insert_strict = false`. The default value is 1.0, which means that all abnormal data are allowed to be filtered.
187201
:::

i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/handling-messy-data.md

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@
3333
| ------------ | ------------- | --------------------- | ---------- | ---------------- |
3434
| 空值 | \N | NULL | 开启或关闭 | NULL |
3535
| 非空值 | "abc" or 2000 | NULL | 开启 | 非法值(被过滤) |
36-
| 非空值 | "abc" | NULL | 关闭 | NULL |
36+
| 非空值 | "abc" or 2000 | NULL | 关闭 | NULL |
3737
| 非空值 | 1 | 1 | 开启或关闭 | 正确导入 |
3838

3939
:::tip
@@ -49,14 +49,27 @@
4949
| 空值 | \N | null | 开启或关闭 | NULL |
5050
| 非空值 | aaa | NULL | 开启 | 非法值(被过滤) |
5151
| 非空值 | aaa | NULL | 关闭 | NULL |
52-
| 非空值 | 1 or 10 | 1 or 10 | 开启或关闭 | 正确导入 |
52+
| 非空值 | 10 | NULL(溢出) | 开启 | 被过滤 |
53+
| 非空值 | 10 | NULL(溢出) | 关闭 | NULL |
5354

5455
:::tip
5556
1. 表中的列允许导入空值
5657

5758
2. `abc` 在转换为 Decimal 后,会因类型问题变为 NULL。在严格模式开启的情况下,这类数据将会被过滤。而如果是关闭状态,则会导入 `null`
5859

59-
3. `10` 虽然是一个超过范围的值,但是因为其类型符合 decimal 的要求,所以严格模式对其不产生影响。
60+
3. `10` 是一个超过Decimal(1, 0)范围的值,会被转换为NULL,在严格模式开启的情况下将会被过滤。严格模式关闭状态下,则会导入 `null`
61+
:::
62+
63+
**3. 以列类型为 char(10) 举例**
64+
65+
| 原始数据类型 | 原始数据举例 | 转换为 char(10) 后的值 | 严格模式 | 结果 |
66+
| ------------ | ------------ | --------------------- | ---------- | ---------------- |
67+
| 空值 | \N | null | 开启或关闭 | NULL |
68+
| 非空值 | a1234567890 | a1234567890 | 开启 | 超长,被过滤 |
69+
| 非空值 | a1234567890 | a1234567890 | 关闭 | a123456789 (截断) |
70+
71+
:::tip
72+
1. 表中的列允许导入空值
6073
:::
6174

6275
### 开启严格模式
@@ -182,5 +195,5 @@ INSERT INTO test_table FROM S3/HDFS/LOCAL();
182195
```
183196

184197
:::tip
185-
对于 Insert Into 语句,`insert_max_filter_ratio` 仅在 `enable_insert_strict = false` 时生效,且只适用于 `INSERT INTO FROM S3/HDFS/LOCAL()` 语法。默认值为 1.0,表示允许所有异常数据被过滤。
198+
对于 Insert Into 语句,`insert_max_filter_ratio` 仅在 `enable_insert_strict = false` 时生效。默认值为 1.0,表示允许所有异常数据被过滤。
186199
:::

i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/data-operate/import/handling-messy-data.md

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@
3333
| ------------ | ------------- | --------------------- | ---------- | ---------------- |
3434
| 空值 | \N | NULL | 开启或关闭 | NULL |
3535
| 非空值 | "abc" or 2000 | NULL | 开启 | 非法值(被过滤) |
36-
| 非空值 | "abc" | NULL | 关闭 | NULL |
36+
| 非空值 | "abc" or 2000 | NULL | 关闭 | NULL |
3737
| 非空值 | 1 | 1 | 开启或关闭 | 正确导入 |
3838

3939
:::tip
@@ -49,14 +49,27 @@
4949
| 空值 | \N | null | 开启或关闭 | NULL |
5050
| 非空值 | aaa | NULL | 开启 | 非法值(被过滤) |
5151
| 非空值 | aaa | NULL | 关闭 | NULL |
52-
| 非空值 | 1 or 10 | 1 or 10 | 开启或关闭 | 正确导入 |
52+
| 非空值 | 10 | NULL(溢出) | 开启 | 被过滤 |
53+
| 非空值 | 10 | NULL(溢出) | 关闭 | NULL |
5354

5455
:::tip
5556
1. 表中的列允许导入空值
5657

5758
2. `abc` 在转换为 Decimal 后,会因类型问题变为 NULL。在严格模式开启的情况下,这类数据将会被过滤。而如果是关闭状态,则会导入 `null`
5859

59-
3. `10` 虽然是一个超过范围的值,但是因为其类型符合 decimal 的要求,所以严格模式对其不产生影响。
60+
3. `10` 是一个超过Decimal(1, 0)范围的值,会被转换为NULL,在严格模式开启的情况下将会被过滤。严格模式关闭状态下,则会导入 `null`
61+
:::
62+
63+
**3. 以列类型为 char(10) 举例**
64+
65+
| 原始数据类型 | 原始数据举例 | 转换为 char(10) 后的值 | 严格模式 | 结果 |
66+
| ------------ | ------------ | --------------------- | ---------- | ---------------- |
67+
| 空值 | \N | null | 开启或关闭 | NULL |
68+
| 非空值 | a1234567890 | a1234567890 | 开启 | 超长,被过滤 |
69+
| 非空值 | a1234567890 | a1234567890 | 关闭 | a123456789 (截断) |
70+
71+
:::tip
72+
1. 表中的列允许导入空值
6073
:::
6174

6275
### 开启严格模式
@@ -182,5 +195,5 @@ INSERT INTO test_table FROM S3/HDFS/LOCAL();
182195
```
183196

184197
:::tip
185-
对于 Insert Into 语句,`insert_max_filter_ratio` 仅在 `enable_insert_strict = false` 时生效,且只适用于 `INSERT INTO FROM S3/HDFS/LOCAL()` 语法。默认值为 1.0,表示允许所有异常数据被过滤。
198+
对于 Insert Into 语句,`insert_max_filter_ratio` 仅在 `enable_insert_strict = false` 时生效。默认值为 1.0,表示允许所有异常数据被过滤。
186199
:::

versioned_docs/version-4.x/data-operate/import/handling-messy-data.md

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ The system employs different strategies based on the strict mode setting:
3535
| ----------------- | -------------------- | ----------------------------- | ----------- | ------ |
3636
| NULL | \N | NULL | ON/OFF | NULL |
3737
| Non-NULL | "abc" or 2000 | NULL | ON | Invalid (Filtered) |
38-
| Non-NULL | "abc" | NULL | OFF | NULL |
38+
| Non-NULL | "abc" or 2000 | NULL | OFF | NULL |
3939
| Non-NULL | 1 | 1 | ON/OFF | Loaded Successfully |
4040

4141
:::tip
@@ -51,14 +51,28 @@ The system employs different strategies based on the strict mode setting:
5151
| NULL | \N | NULL | ON/OFF | NULL |
5252
| Non-NULL | aaa | NULL | ON | Invalid (Filtered) |
5353
| Non-NULL | aaa | NULL | OFF | NULL |
54-
| Non-NULL | 1 or 10 | 1 or 10 | ON/OFF | Loaded Successfully |
54+
| Non-NULL | 10 | NULL (overflow) | ON | Filtered |
55+
| Non-NULL | 10 | NULL (overflow) | OFF | NULL |
5556

5657
:::tip
5758
1. The column in the table allows NULL values
5859

5960
2. `abc` becomes NULL after conversion to Decimal due to type issues. When strict mode is ON, such data will be filtered out. When OFF, NULL will be loaded.
6061

61-
3. Although `10` exceeds the range, since its type meets decimal requirements, strict mode does not affect it.
62+
3. `10` is a value outside the range of Decimal(1, 0) and will be converted to NULL. It will be filtered out if strict mode is enabled. If strict mode is disabled, `null` will be imported.
63+
:::
64+
65+
**3. Example of a column type char(10)**
66+
67+
| Original data type | Original Data Example | Value after conversion to char(10) | Strict mode | Result |
68+
| ------------------ | --------------------- | ---------------------------------- | ---------- | ------ |
69+
| NULL | \N | NULL | ON/OFF | NULL |
70+
| Non-NULL | a1234567890 | a1234567890 | ON | Too long, filtered |
71+
| Non-NULL | a1234567890 | a1234567890 | OFF | a123456789 (truncated) |
72+
73+
:::tip
74+
75+
1. The column in the table allows NULL values
6276
:::
6377

6478
### Enable Strict Mode
@@ -183,5 +197,5 @@ SET insert_max_filter_ratio = 0.1;
183197
INSERT INTO test_table FROM S3/HDFS/LOCAL();```
184198
185199
:::tip
186-
For Insert Into statements, `insert_max_filter_ratio` only takes effect when `enable_insert_strict = false`, and only applies to `INSERT INTO FROM S3/HDFS/LOCAL()` syntax. The default value is 1.0, which means that all abnormal data are allowed to be filtered.
200+
For Insert Into statements, `insert_max_filter_ratio` only takes effect when `enable_insert_strict = false`. The default value is 1.0, which means that all abnormal data are allowed to be filtered.
187201
:::

0 commit comments

Comments
 (0)