From 9a26616b698f7a216ba9cb3e128f168aaa06917c Mon Sep 17 00:00:00 2001 From: leto-bbq Date: Thu, 6 Aug 2026 10:07:43 +0800 Subject: [PATCH] docs: update tree-model streaming framework documentation --- .../Tree/User-Manual/Streaming_apache.md | 18 +++++++++--------- .../V1.3.x/User-Manual/Streaming_apache.md | 18 +++++++++--------- .../dev-1.3/User-Manual/Streaming_apache.md | 18 +++++++++--------- .../latest/User-Manual/Streaming_apache.md | 18 +++++++++--------- .../Tree/User-Manual/Streaming_apache.md | 18 +++++++++--------- .../V1.3.x/User-Manual/Streaming_apache.md | 18 +++++++++--------- .../dev-1.3/User-Manual/Streaming_apache.md | 18 +++++++++--------- .../latest/User-Manual/Streaming_apache.md | 18 +++++++++--------- 8 files changed, 72 insertions(+), 72 deletions(-) diff --git a/src/UserGuide/Master/Tree/User-Manual/Streaming_apache.md b/src/UserGuide/Master/Tree/User-Manual/Streaming_apache.md index 586dcbb62..7624d3e40 100644 --- a/src/UserGuide/Master/Tree/User-Manual/Streaming_apache.md +++ b/src/UserGuide/Master/Tree/User-Manual/Streaming_apache.md @@ -64,17 +64,17 @@ The design of user programming interfaces for stream processing plugins follows In the user programming interface of stream processing plugins, events abstract the write operations of database data. Events are captured by the local stream processing engine and passed sequentially through the three stages of stream processing, namely Pipe Source, Pipe Processor, and Pipe Sink plugins. User logic is triggered and executed within these three plugins. -To accommodate both low-latency stream processing in low-load scenarios and high-throughput stream processing in high-load scenarios at the edge, the stream processing engine dynamically chooses the processing objects from operation logs and data files. Therefore, the user programming interface for stream processing requires the user to provide the handling logic for two types of events: TabletInsertionEvent for operation log write events and TsFileInsertionEvent for data file write events. +To accommodate both low-latency stream processing in low-load scenarios and high-throughput stream processing in high-load scenarios at the edge, the stream processing engine dynamically chooses whether to process real-time writes or data files. Therefore, the user programming interface for stream processing requires users to provide the handling logic for two types of events: Tablet write events (`TabletInsertionEvent`) and TsFile write events (`TsFileInsertionEvent`). -#### **TabletInsertionEvent** +#### **Tablet Write Event (`TabletInsertionEvent`)** -The TabletInsertionEvent is a high-level data abstraction for user write requests, which provides the ability to manipulate the underlying data of the write request by providing a unified operation interface. +The Tablet write event (`TabletInsertionEvent`) is a high-level data abstraction for user write requests. By providing a unified operation interface, it enables users to process the written data. -For different database deployments, the underlying storage structure corresponding to the operation log write event is different. For stand-alone deployment scenarios, the operation log write event is an encapsulation of write-ahead log (WAL) entries; for distributed deployment scenarios, the operation log write event is an encapsulation of individual node consensus protocol operation log entries. +Since V2.0.5, the stream processing framework obtains real-time written data directly from write requests and encapsulates it as a `TabletInsertionEvent`, instead of reading the data from operation logs. This change does not affect the `TabletInsertionEvent` interface or the usage of existing plugins. -For write operations generated by different write request interfaces of the database, the data structure of the request structure corresponding to the operation log write event is also different.IoTDB provides many write interfaces such as InsertRecord, InsertRecords, InsertTablet, InsertTablets, and so on, and each kind of write request uses a completely different serialisation method to generate a write request. completely different serialisation methods and generate different binary entries. +For write operations generated through different database write interfaces, the data structures corresponding to Tablet write events also differ. IoTDB provides multiple write interfaces, such as `InsertRecord`, `InsertRecords`, `InsertTablet`, and `InsertTablets`. The data structures of different write requests also vary. -The existence of operation log write events provides users with a unified view of data operations, which shields the implementation differences of the underlying data structures, greatly reduces the programming threshold for users, and improves the ease of use of the functionality. +Tablet write events provide users with a unified view of data operations, hiding implementation differences among underlying data structures, greatly lowering the programming barrier, and improving usability. ```java /** TabletInsertionEvent is used to define the event of data insertion. */ @@ -98,9 +98,9 @@ public interface TabletInsertionEvent extends Event { } ``` -#### **TsFileInsertionEvent** +#### **TsFile Write Event (`TsFileInsertionEvent`)** -The TsFileInsertionEvent represents a high-level abstraction of the database's disk flush operation and is a collection of multiple TabletInsertionEvents. +The TsFile write event (`TsFileInsertionEvent`) is a high-level abstraction of database file flush operations and a collection of multiple Tablet write events (`TabletInsertionEvent`). IoTDB's storage engine is based on the LSM (Log-Structured Merge) structure. When data is written, the write operations are first flushed to log-structured files, while the written data is also stored in memory. When the memory reaches its capacity limit, a flush operation is triggered, converting the data in memory into a database file while deleting the previously written log entries. During the conversion from memory data to database file data, two compression processes, encoding compression and universal compression, are applied. As a result, the data in the database file occupies less space compared to the original data in memory. @@ -110,7 +110,7 @@ In summary, the data file write event appears in the event stream of stream proc 1. Historical data extraction: Before a stream processing task starts, all persisted write data exists in the form of TsFiles. When collecting historical data at the beginning of a stream processing task, the historical data is abstracted as TsFileInsertionEvent. -2. Real-time data extraction: During the execution of a stream processing task, if the speed of processing the log entries representing real-time operations is slower than the rate of write requests, the unprocessed log entries will be persisted to disk in the form of TsFiles. When these data are extracted by the stream processing engine, they are abstracted as TsFileInsertionEvent. +2. Real-time data extraction: While a stream processing task is running, real-time writes are usually processed as `TabletInsertionEvent`s. When unprocessed data accumulates, the stream processing engine can extract data in batches from TsFiles that have already been flushed to disk and process it as `TsFileInsertionEvent`s. ```java /** diff --git a/src/UserGuide/V1.3.x/User-Manual/Streaming_apache.md b/src/UserGuide/V1.3.x/User-Manual/Streaming_apache.md index bfb49b91d..084bddf6f 100644 --- a/src/UserGuide/V1.3.x/User-Manual/Streaming_apache.md +++ b/src/UserGuide/V1.3.x/User-Manual/Streaming_apache.md @@ -64,17 +64,17 @@ The design of user programming interfaces for stream processing plugins follows In the user programming interface of stream processing plugins, events abstract the write operations of database data. Events are captured by the local stream processing engine and passed sequentially through the three stages of stream processing, namely Pipe Source, Pipe Processor, and Pipe Sink plugins. User logic is triggered and executed within these three plugins. -To accommodate both low-latency stream processing in low-load scenarios and high-throughput stream processing in high-load scenarios at the edge, the stream processing engine dynamically chooses the processing objects from operation logs and data files. Therefore, the user programming interface for stream processing requires the user to provide the handling logic for two types of events: TabletInsertionEvent for operation log write events and TsFileInsertionEvent for data file write events. +To accommodate both low-latency stream processing in low-load scenarios and high-throughput stream processing in high-load scenarios at the edge, the stream processing engine dynamically chooses whether to process real-time writes or data files. Therefore, the user programming interface for stream processing requires users to provide the handling logic for two types of events: Tablet write events (`TabletInsertionEvent`) and TsFile write events (`TsFileInsertionEvent`). -#### **TabletInsertionEvent** +#### **Tablet Write Event (`TabletInsertionEvent`)** -The TabletInsertionEvent is a high-level data abstraction for user write requests, which provides the ability to manipulate the underlying data of the write request by providing a unified operation interface. +The Tablet write event (`TabletInsertionEvent`) is a high-level data abstraction for user write requests. By providing a unified operation interface, it enables users to process the written data. -For different database deployments, the underlying storage structure corresponding to the operation log write event is different. For stand-alone deployment scenarios, the operation log write event is an encapsulation of write-ahead log (WAL) entries; for distributed deployment scenarios, the operation log write event is an encapsulation of individual node consensus protocol operation log entries. +Since V1.3.5, the stream processing framework obtains real-time written data directly from write requests and encapsulates it as a `TabletInsertionEvent`, instead of reading the data from operation logs. This change does not affect the `TabletInsertionEvent` interface or the usage of existing plugins. -For write operations generated by different write request interfaces of the database, the data structure of the request structure corresponding to the operation log write event is also different.IoTDB provides many write interfaces such as InsertRecord, InsertRecords, InsertTablet, InsertTablets, and so on, and each kind of write request uses a completely different serialisation method to generate a write request. completely different serialisation methods and generate different binary entries. +For write operations generated through different database write interfaces, the data structures corresponding to Tablet write events also differ. IoTDB provides multiple write interfaces, such as `InsertRecord`, `InsertRecords`, `InsertTablet`, and `InsertTablets`. The data structures of different write requests also vary. -The existence of operation log write events provides users with a unified view of data operations, which shields the implementation differences of the underlying data structures, greatly reduces the programming threshold for users, and improves the ease of use of the functionality. +Tablet write events provide users with a unified view of data operations, hiding implementation differences among underlying data structures, greatly lowering the programming barrier, and improving usability. ```java /** TabletInsertionEvent is used to define the event of data insertion. */ @@ -98,9 +98,9 @@ public interface TabletInsertionEvent extends Event { } ``` -#### **TsFileInsertionEvent** +#### **TsFile Write Event (`TsFileInsertionEvent`)** -The TsFileInsertionEvent represents a high-level abstraction of the database's disk flush operation and is a collection of multiple TabletInsertionEvents. +The TsFile write event (`TsFileInsertionEvent`) is a high-level abstraction of database file flush operations and a collection of multiple Tablet write events (`TabletInsertionEvent`). IoTDB's storage engine is based on the LSM (Log-Structured Merge) structure. When data is written, the write operations are first flushed to log-structured files, while the written data is also stored in memory. When the memory reaches its capacity limit, a flush operation is triggered, converting the data in memory into a database file while deleting the previously written log entries. During the conversion from memory data to database file data, two compression processes, encoding compression and universal compression, are applied. As a result, the data in the database file occupies less space compared to the original data in memory. @@ -110,7 +110,7 @@ In summary, the data file write event appears in the event stream of stream proc 1. Historical data extraction: Before a stream processing task starts, all persisted write data exists in the form of TsFiles. When collecting historical data at the beginning of a stream processing task, the historical data is abstracted as TsFileInsertionEvent. -2. Real-time data extraction: During the execution of a stream processing task, if the speed of processing the log entries representing real-time operations is slower than the rate of write requests, the unprocessed log entries will be persisted to disk in the form of TsFiles. When these data are extracted by the stream processing engine, they are abstracted as TsFileInsertionEvent. +2. Real-time data extraction: While a stream processing task is running, real-time writes are usually processed as `TabletInsertionEvent`s. When unprocessed data accumulates, the stream processing engine can extract data in batches from TsFiles that have already been flushed to disk and process it as `TsFileInsertionEvent`s. ```java /** diff --git a/src/UserGuide/dev-1.3/User-Manual/Streaming_apache.md b/src/UserGuide/dev-1.3/User-Manual/Streaming_apache.md index bfb49b91d..084bddf6f 100644 --- a/src/UserGuide/dev-1.3/User-Manual/Streaming_apache.md +++ b/src/UserGuide/dev-1.3/User-Manual/Streaming_apache.md @@ -64,17 +64,17 @@ The design of user programming interfaces for stream processing plugins follows In the user programming interface of stream processing plugins, events abstract the write operations of database data. Events are captured by the local stream processing engine and passed sequentially through the three stages of stream processing, namely Pipe Source, Pipe Processor, and Pipe Sink plugins. User logic is triggered and executed within these three plugins. -To accommodate both low-latency stream processing in low-load scenarios and high-throughput stream processing in high-load scenarios at the edge, the stream processing engine dynamically chooses the processing objects from operation logs and data files. Therefore, the user programming interface for stream processing requires the user to provide the handling logic for two types of events: TabletInsertionEvent for operation log write events and TsFileInsertionEvent for data file write events. +To accommodate both low-latency stream processing in low-load scenarios and high-throughput stream processing in high-load scenarios at the edge, the stream processing engine dynamically chooses whether to process real-time writes or data files. Therefore, the user programming interface for stream processing requires users to provide the handling logic for two types of events: Tablet write events (`TabletInsertionEvent`) and TsFile write events (`TsFileInsertionEvent`). -#### **TabletInsertionEvent** +#### **Tablet Write Event (`TabletInsertionEvent`)** -The TabletInsertionEvent is a high-level data abstraction for user write requests, which provides the ability to manipulate the underlying data of the write request by providing a unified operation interface. +The Tablet write event (`TabletInsertionEvent`) is a high-level data abstraction for user write requests. By providing a unified operation interface, it enables users to process the written data. -For different database deployments, the underlying storage structure corresponding to the operation log write event is different. For stand-alone deployment scenarios, the operation log write event is an encapsulation of write-ahead log (WAL) entries; for distributed deployment scenarios, the operation log write event is an encapsulation of individual node consensus protocol operation log entries. +Since V1.3.5, the stream processing framework obtains real-time written data directly from write requests and encapsulates it as a `TabletInsertionEvent`, instead of reading the data from operation logs. This change does not affect the `TabletInsertionEvent` interface or the usage of existing plugins. -For write operations generated by different write request interfaces of the database, the data structure of the request structure corresponding to the operation log write event is also different.IoTDB provides many write interfaces such as InsertRecord, InsertRecords, InsertTablet, InsertTablets, and so on, and each kind of write request uses a completely different serialisation method to generate a write request. completely different serialisation methods and generate different binary entries. +For write operations generated through different database write interfaces, the data structures corresponding to Tablet write events also differ. IoTDB provides multiple write interfaces, such as `InsertRecord`, `InsertRecords`, `InsertTablet`, and `InsertTablets`. The data structures of different write requests also vary. -The existence of operation log write events provides users with a unified view of data operations, which shields the implementation differences of the underlying data structures, greatly reduces the programming threshold for users, and improves the ease of use of the functionality. +Tablet write events provide users with a unified view of data operations, hiding implementation differences among underlying data structures, greatly lowering the programming barrier, and improving usability. ```java /** TabletInsertionEvent is used to define the event of data insertion. */ @@ -98,9 +98,9 @@ public interface TabletInsertionEvent extends Event { } ``` -#### **TsFileInsertionEvent** +#### **TsFile Write Event (`TsFileInsertionEvent`)** -The TsFileInsertionEvent represents a high-level abstraction of the database's disk flush operation and is a collection of multiple TabletInsertionEvents. +The TsFile write event (`TsFileInsertionEvent`) is a high-level abstraction of database file flush operations and a collection of multiple Tablet write events (`TabletInsertionEvent`). IoTDB's storage engine is based on the LSM (Log-Structured Merge) structure. When data is written, the write operations are first flushed to log-structured files, while the written data is also stored in memory. When the memory reaches its capacity limit, a flush operation is triggered, converting the data in memory into a database file while deleting the previously written log entries. During the conversion from memory data to database file data, two compression processes, encoding compression and universal compression, are applied. As a result, the data in the database file occupies less space compared to the original data in memory. @@ -110,7 +110,7 @@ In summary, the data file write event appears in the event stream of stream proc 1. Historical data extraction: Before a stream processing task starts, all persisted write data exists in the form of TsFiles. When collecting historical data at the beginning of a stream processing task, the historical data is abstracted as TsFileInsertionEvent. -2. Real-time data extraction: During the execution of a stream processing task, if the speed of processing the log entries representing real-time operations is slower than the rate of write requests, the unprocessed log entries will be persisted to disk in the form of TsFiles. When these data are extracted by the stream processing engine, they are abstracted as TsFileInsertionEvent. +2. Real-time data extraction: While a stream processing task is running, real-time writes are usually processed as `TabletInsertionEvent`s. When unprocessed data accumulates, the stream processing engine can extract data in batches from TsFiles that have already been flushed to disk and process it as `TsFileInsertionEvent`s. ```java /** diff --git a/src/UserGuide/latest/User-Manual/Streaming_apache.md b/src/UserGuide/latest/User-Manual/Streaming_apache.md index 586dcbb62..7624d3e40 100644 --- a/src/UserGuide/latest/User-Manual/Streaming_apache.md +++ b/src/UserGuide/latest/User-Manual/Streaming_apache.md @@ -64,17 +64,17 @@ The design of user programming interfaces for stream processing plugins follows In the user programming interface of stream processing plugins, events abstract the write operations of database data. Events are captured by the local stream processing engine and passed sequentially through the three stages of stream processing, namely Pipe Source, Pipe Processor, and Pipe Sink plugins. User logic is triggered and executed within these three plugins. -To accommodate both low-latency stream processing in low-load scenarios and high-throughput stream processing in high-load scenarios at the edge, the stream processing engine dynamically chooses the processing objects from operation logs and data files. Therefore, the user programming interface for stream processing requires the user to provide the handling logic for two types of events: TabletInsertionEvent for operation log write events and TsFileInsertionEvent for data file write events. +To accommodate both low-latency stream processing in low-load scenarios and high-throughput stream processing in high-load scenarios at the edge, the stream processing engine dynamically chooses whether to process real-time writes or data files. Therefore, the user programming interface for stream processing requires users to provide the handling logic for two types of events: Tablet write events (`TabletInsertionEvent`) and TsFile write events (`TsFileInsertionEvent`). -#### **TabletInsertionEvent** +#### **Tablet Write Event (`TabletInsertionEvent`)** -The TabletInsertionEvent is a high-level data abstraction for user write requests, which provides the ability to manipulate the underlying data of the write request by providing a unified operation interface. +The Tablet write event (`TabletInsertionEvent`) is a high-level data abstraction for user write requests. By providing a unified operation interface, it enables users to process the written data. -For different database deployments, the underlying storage structure corresponding to the operation log write event is different. For stand-alone deployment scenarios, the operation log write event is an encapsulation of write-ahead log (WAL) entries; for distributed deployment scenarios, the operation log write event is an encapsulation of individual node consensus protocol operation log entries. +Since V2.0.5, the stream processing framework obtains real-time written data directly from write requests and encapsulates it as a `TabletInsertionEvent`, instead of reading the data from operation logs. This change does not affect the `TabletInsertionEvent` interface or the usage of existing plugins. -For write operations generated by different write request interfaces of the database, the data structure of the request structure corresponding to the operation log write event is also different.IoTDB provides many write interfaces such as InsertRecord, InsertRecords, InsertTablet, InsertTablets, and so on, and each kind of write request uses a completely different serialisation method to generate a write request. completely different serialisation methods and generate different binary entries. +For write operations generated through different database write interfaces, the data structures corresponding to Tablet write events also differ. IoTDB provides multiple write interfaces, such as `InsertRecord`, `InsertRecords`, `InsertTablet`, and `InsertTablets`. The data structures of different write requests also vary. -The existence of operation log write events provides users with a unified view of data operations, which shields the implementation differences of the underlying data structures, greatly reduces the programming threshold for users, and improves the ease of use of the functionality. +Tablet write events provide users with a unified view of data operations, hiding implementation differences among underlying data structures, greatly lowering the programming barrier, and improving usability. ```java /** TabletInsertionEvent is used to define the event of data insertion. */ @@ -98,9 +98,9 @@ public interface TabletInsertionEvent extends Event { } ``` -#### **TsFileInsertionEvent** +#### **TsFile Write Event (`TsFileInsertionEvent`)** -The TsFileInsertionEvent represents a high-level abstraction of the database's disk flush operation and is a collection of multiple TabletInsertionEvents. +The TsFile write event (`TsFileInsertionEvent`) is a high-level abstraction of database file flush operations and a collection of multiple Tablet write events (`TabletInsertionEvent`). IoTDB's storage engine is based on the LSM (Log-Structured Merge) structure. When data is written, the write operations are first flushed to log-structured files, while the written data is also stored in memory. When the memory reaches its capacity limit, a flush operation is triggered, converting the data in memory into a database file while deleting the previously written log entries. During the conversion from memory data to database file data, two compression processes, encoding compression and universal compression, are applied. As a result, the data in the database file occupies less space compared to the original data in memory. @@ -110,7 +110,7 @@ In summary, the data file write event appears in the event stream of stream proc 1. Historical data extraction: Before a stream processing task starts, all persisted write data exists in the form of TsFiles. When collecting historical data at the beginning of a stream processing task, the historical data is abstracted as TsFileInsertionEvent. -2. Real-time data extraction: During the execution of a stream processing task, if the speed of processing the log entries representing real-time operations is slower than the rate of write requests, the unprocessed log entries will be persisted to disk in the form of TsFiles. When these data are extracted by the stream processing engine, they are abstracted as TsFileInsertionEvent. +2. Real-time data extraction: While a stream processing task is running, real-time writes are usually processed as `TabletInsertionEvent`s. When unprocessed data accumulates, the stream processing engine can extract data in batches from TsFiles that have already been flushed to disk and process it as `TsFileInsertionEvent`s. ```java /** diff --git a/src/zh/UserGuide/Master/Tree/User-Manual/Streaming_apache.md b/src/zh/UserGuide/Master/Tree/User-Manual/Streaming_apache.md index 0a9ce0a34..6f4eda3ea 100644 --- a/src/zh/UserGuide/Master/Tree/User-Manual/Streaming_apache.md +++ b/src/zh/UserGuide/Master/Tree/User-Manual/Streaming_apache.md @@ -63,17 +63,17 @@ Pipe Source 用于抽取数据，Pipe Processor 用于处理数据，Pipe Sink 在流处理插件的用户编程接口中，事件是数据库数据写入操作的抽象。事件由单机流处理引擎捕获，按照流处理三个阶段的流程，依次传递至 PipeSource 插件，PipeProcessor 插件和 PipeSink 插件，并依次在三个插件中触发用户逻辑的执行。 -为了兼顾端侧低负载场景下的流处理低延迟和端侧高负载场景下的流处理高吞吐，流处理引擎会动态地在操作日志和数据文件中选择处理对象，因此，流处理的用户编程接口要求用户提供下列两类事件的处理逻辑：操作日志写入事件 TabletInsertionEvent 和数据文件写入事件 TsFileInsertionEvent。 +为了兼顾端侧低负载场景下的流处理低延迟和端侧高负载场景下的流处理高吞吐，流处理引擎会动态选择处理实时写入数据或数据文件。因此，流处理的用户编程接口要求用户提供下列两类事件的处理逻辑：Tablet 写入事件 `TabletInsertionEvent` 和 TsFile 写入事件 `TsFileInsertionEvent`。 -#### **操作日志写入事件（TabletInsertionEvent）** +#### **Tablet 写入事件（TabletInsertionEvent）** -操作日志写入事件（TabletInsertionEvent）是对用户写入请求的高层数据抽象，它通过提供统一的操作接口，为用户提供了操纵写入请求底层数据的能力。 +Tablet 写入事件（`TabletInsertionEvent`）是对用户写入请求的高层数据抽象，它通过提供统一的操作接口，为用户提供了处理写入数据的能力。 -对于不同的数据库部署方式，操作日志写入事件对应的底层存储结构是不一样的。对于单机部署的场景，操作日志写入事件是对写前日志（WAL）条目的封装；对于分布式部署的场景，操作日志写入事件是对单个节点共识协议操作日志条目的封装。 +自 V2.0.5 起，流处理框架会直接从写入请求中获取实时写入数据，并将其封装为 `TabletInsertionEvent`，不再从操作日志中读取数据。该变化不影响 `TabletInsertionEvent` 接口以及已有插件的使用方式。 -对于数据库不同写入请求接口生成的写入操作，操作日志写入事件对应的请求结构体的数据结构也是不一样的。IoTDB 提供了 InsertRecord、InsertRecords、InsertTablet、InsertTablets 等众多的写入接口，每一种写入请求都使用了完全不同的序列化方式，生成的二进制条目也不尽相同。 +对于数据库不同写入接口生成的写入操作，Tablet 写入事件对应的数据结构也是不一样的。IoTDB 提供了 `InsertRecord`、`InsertRecords`、`InsertTablet`、`InsertTablets` 等多种写入接口，不同写入请求的数据结构也不尽相同。 -操作日志写入事件的存在，为用户提供了一种统一的数据操作视图，它屏蔽了底层数据结构的实现差异，极大地降低了用户的编程门槛，提升了功能的易用性。 +Tablet 写入事件为用户提供了一种统一的数据操作视图，它屏蔽了底层数据结构的实现差异，极大地降低了用户的编程门槛，提升了功能的易用性。 ```java /** TabletInsertionEvent is used to define the event of data insertion. */ @@ -97,9 +97,9 @@ public interface TabletInsertionEvent extends Event { } ``` -#### **数据文件写入事件（TsFileInsertionEvent）** +#### **TsFile 写入事件（TsFileInsertionEvent）** -数据文件写入事件（TsFileInsertionEvent） 是对数据库文件落盘操作的高层抽象，它是若干操作日志写入事件（TabletInsertionEvent）的数据集合。 +TsFile 写入事件（`TsFileInsertionEvent`）是对数据库文件落盘操作的高层抽象，它是若干 Tablet 写入事件（`TabletInsertionEvent`）的数据集合。 IoTDB 的存储引擎是 LSM 结构的。数据写入时会先将写入操作落盘到日志结构的文件里，同时将写入数据保存在内存里。当内存达到控制上限，则会触发刷盘行为，即将内存中的数据转换为数据库文件，同时删除之前预写的操作日志。当内存中的数据转换为数据库文件中的数据时，会经过编码压缩和通用压缩两次压缩处理，因此数据库文件的数据相比内存中的原始数据占用的空间更少。 @@ -109,7 +109,7 @@ IoTDB 的存储引擎是 LSM 结构的。数据写入时会先将写入操作落 （1）历史数据抽取：一个流处理任务开始前，所有已经落盘的写入数据都会以 TsFile 的形式存在。一个流处理任务开始后，采集历史数据时，历史数据将以 TsFileInsertionEvent 作为抽象； -（2）实时数据抽取：一个流处理任务进行时，当数据流中实时处理操作日志写入事件的速度慢于写入请求速度一定进度之后，未来得及处理的操作日志写入事件会被被持久化至磁盘，以 TsFile 的形式存在，这一些数据被流处理引擎抽取到后，会以 TsFileInsertionEvent 作为抽象。 +（2）实时数据抽取：一个流处理任务运行时，实时写入的数据通常以 `TabletInsertionEvent` 的形式进行处理。当待处理的数据出现积压时，流处理引擎可以从已经落盘的 TsFile 中批量抽取数据，并以 `TsFileInsertionEvent` 的形式进行处理。 ```java /** diff --git a/src/zh/UserGuide/V1.3.x/User-Manual/Streaming_apache.md b/src/zh/UserGuide/V1.3.x/User-Manual/Streaming_apache.md index 38811fa30..0bf8b0074 100644 --- a/src/zh/UserGuide/V1.3.x/User-Manual/Streaming_apache.md +++ b/src/zh/UserGuide/V1.3.x/User-Manual/Streaming_apache.md @@ -63,17 +63,17 @@ Pipe Source 用于抽取数据，Pipe Processor 用于处理数据，Pipe Sink 在流处理插件的用户编程接口中，事件是数据库数据写入操作的抽象。事件由单机流处理引擎捕获，按照流处理三个阶段的流程，依次传递至 PipeSource 插件，PipeProcessor 插件和 PipeSink 插件，并依次在三个插件中触发用户逻辑的执行。 -为了兼顾端侧低负载场景下的流处理低延迟和端侧高负载场景下的流处理高吞吐，流处理引擎会动态地在操作日志和数据文件中选择处理对象，因此，流处理的用户编程接口要求用户提供下列两类事件的处理逻辑：操作日志写入事件 TabletInsertionEvent 和数据文件写入事件 TsFileInsertionEvent。 +为了兼顾端侧低负载场景下的流处理低延迟和端侧高负载场景下的流处理高吞吐，流处理引擎会动态选择处理实时写入数据或数据文件。因此，流处理的用户编程接口要求用户提供下列两类事件的处理逻辑：Tablet 写入事件 `TabletInsertionEvent` 和 TsFile 写入事件 `TsFileInsertionEvent`。 -#### **操作日志写入事件（TabletInsertionEvent）** +#### **Tablet 写入事件（TabletInsertionEvent）** -操作日志写入事件（TabletInsertionEvent）是对用户写入请求的高层数据抽象，它通过提供统一的操作接口，为用户提供了操纵写入请求底层数据的能力。 +Tablet 写入事件（`TabletInsertionEvent`）是对用户写入请求的高层数据抽象，它通过提供统一的操作接口，为用户提供了处理写入数据的能力。 -对于不同的数据库部署方式，操作日志写入事件对应的底层存储结构是不一样的。对于单机部署的场景，操作日志写入事件是对写前日志（WAL）条目的封装；对于分布式部署的场景，操作日志写入事件是对单个节点共识协议操作日志条目的封装。 +自 V1.3.5 起，流处理框架会直接从写入请求中获取实时写入数据，并将其封装为 `TabletInsertionEvent`，不再从操作日志中读取数据。该变化不影响 `TabletInsertionEvent` 接口以及已有插件的使用方式。 -对于数据库不同写入请求接口生成的写入操作，操作日志写入事件对应的请求结构体的数据结构也是不一样的。IoTDB 提供了 InsertRecord、InsertRecords、InsertTablet、InsertTablets 等众多的写入接口，每一种写入请求都使用了完全不同的序列化方式，生成的二进制条目也不尽相同。 +对于数据库不同写入接口生成的写入操作，Tablet 写入事件对应的数据结构也是不一样的。IoTDB 提供了 `InsertRecord`、`InsertRecords`、`InsertTablet`、`InsertTablets` 等多种写入接口，不同写入请求的数据结构也不尽相同。 -操作日志写入事件的存在，为用户提供了一种统一的数据操作视图，它屏蔽了底层数据结构的实现差异，极大地降低了用户的编程门槛，提升了功能的易用性。 +Tablet 写入事件为用户提供了一种统一的数据操作视图，它屏蔽了底层数据结构的实现差异，极大地降低了用户的编程门槛，提升了功能的易用性。 ```java /** TabletInsertionEvent is used to define the event of data insertion. */ @@ -97,9 +97,9 @@ public interface TabletInsertionEvent extends Event { } ``` -#### **数据文件写入事件（TsFileInsertionEvent）** +#### **TsFile 写入事件（TsFileInsertionEvent）** -数据文件写入事件（TsFileInsertionEvent） 是对数据库文件落盘操作的高层抽象，它是若干操作日志写入事件（TabletInsertionEvent）的数据集合。 +TsFile 写入事件（`TsFileInsertionEvent`）是对数据库文件落盘操作的高层抽象，它是若干 Tablet 写入事件（`TabletInsertionEvent`）的数据集合。 IoTDB 的存储引擎是 LSM 结构的。数据写入时会先将写入操作落盘到日志结构的文件里，同时将写入数据保存在内存里。当内存达到控制上限，则会触发刷盘行为，即将内存中的数据转换为数据库文件，同时删除之前预写的操作日志。当内存中的数据转换为数据库文件中的数据时，会经过编码压缩和通用压缩两次压缩处理，因此数据库文件的数据相比内存中的原始数据占用的空间更少。 @@ -109,7 +109,7 @@ IoTDB 的存储引擎是 LSM 结构的。数据写入时会先将写入操作落 （1）历史数据抽取：一个流处理任务开始前，所有已经落盘的写入数据都会以 TsFile 的形式存在。一个流处理任务开始后，采集历史数据时，历史数据将以 TsFileInsertionEvent 作为抽象； -（2）实时数据抽取：一个流处理任务进行时，当数据流中实时处理操作日志写入事件的速度慢于写入请求速度一定进度之后，未来得及处理的操作日志写入事件会被被持久化至磁盘，以 TsFile 的形式存在，这一些数据被流处理引擎抽取到后，会以 TsFileInsertionEvent 作为抽象。 +（2）实时数据抽取：一个流处理任务运行时，实时写入的数据通常以 `TabletInsertionEvent` 的形式进行处理。当待处理的数据出现积压时，流处理引擎可以从已经落盘的 TsFile 中批量抽取数据，并以 `TsFileInsertionEvent` 的形式进行处理。 ```java /** diff --git a/src/zh/UserGuide/dev-1.3/User-Manual/Streaming_apache.md b/src/zh/UserGuide/dev-1.3/User-Manual/Streaming_apache.md index 38811fa30..0bf8b0074 100644 --- a/src/zh/UserGuide/dev-1.3/User-Manual/Streaming_apache.md +++ b/src/zh/UserGuide/dev-1.3/User-Manual/Streaming_apache.md @@ -63,17 +63,17 @@ Pipe Source 用于抽取数据，Pipe Processor 用于处理数据，Pipe Sink 在流处理插件的用户编程接口中，事件是数据库数据写入操作的抽象。事件由单机流处理引擎捕获，按照流处理三个阶段的流程，依次传递至 PipeSource 插件，PipeProcessor 插件和 PipeSink 插件，并依次在三个插件中触发用户逻辑的执行。 -为了兼顾端侧低负载场景下的流处理低延迟和端侧高负载场景下的流处理高吞吐，流处理引擎会动态地在操作日志和数据文件中选择处理对象，因此，流处理的用户编程接口要求用户提供下列两类事件的处理逻辑：操作日志写入事件 TabletInsertionEvent 和数据文件写入事件 TsFileInsertionEvent。 +为了兼顾端侧低负载场景下的流处理低延迟和端侧高负载场景下的流处理高吞吐，流处理引擎会动态选择处理实时写入数据或数据文件。因此，流处理的用户编程接口要求用户提供下列两类事件的处理逻辑：Tablet 写入事件 `TabletInsertionEvent` 和 TsFile 写入事件 `TsFileInsertionEvent`。 -#### **操作日志写入事件（TabletInsertionEvent）** +#### **Tablet 写入事件（TabletInsertionEvent）** -操作日志写入事件（TabletInsertionEvent）是对用户写入请求的高层数据抽象，它通过提供统一的操作接口，为用户提供了操纵写入请求底层数据的能力。 +Tablet 写入事件（`TabletInsertionEvent`）是对用户写入请求的高层数据抽象，它通过提供统一的操作接口，为用户提供了处理写入数据的能力。 -对于不同的数据库部署方式，操作日志写入事件对应的底层存储结构是不一样的。对于单机部署的场景，操作日志写入事件是对写前日志（WAL）条目的封装；对于分布式部署的场景，操作日志写入事件是对单个节点共识协议操作日志条目的封装。 +自 V1.3.5 起，流处理框架会直接从写入请求中获取实时写入数据，并将其封装为 `TabletInsertionEvent`，不再从操作日志中读取数据。该变化不影响 `TabletInsertionEvent` 接口以及已有插件的使用方式。 -对于数据库不同写入请求接口生成的写入操作，操作日志写入事件对应的请求结构体的数据结构也是不一样的。IoTDB 提供了 InsertRecord、InsertRecords、InsertTablet、InsertTablets 等众多的写入接口，每一种写入请求都使用了完全不同的序列化方式，生成的二进制条目也不尽相同。 +对于数据库不同写入接口生成的写入操作，Tablet 写入事件对应的数据结构也是不一样的。IoTDB 提供了 `InsertRecord`、`InsertRecords`、`InsertTablet`、`InsertTablets` 等多种写入接口，不同写入请求的数据结构也不尽相同。 -操作日志写入事件的存在，为用户提供了一种统一的数据操作视图，它屏蔽了底层数据结构的实现差异，极大地降低了用户的编程门槛，提升了功能的易用性。 +Tablet 写入事件为用户提供了一种统一的数据操作视图，它屏蔽了底层数据结构的实现差异，极大地降低了用户的编程门槛，提升了功能的易用性。 ```java /** TabletInsertionEvent is used to define the event of data insertion. */ @@ -97,9 +97,9 @@ public interface TabletInsertionEvent extends Event { } ``` -#### **数据文件写入事件（TsFileInsertionEvent）** +#### **TsFile 写入事件（TsFileInsertionEvent）** -数据文件写入事件（TsFileInsertionEvent） 是对数据库文件落盘操作的高层抽象，它是若干操作日志写入事件（TabletInsertionEvent）的数据集合。 +TsFile 写入事件（`TsFileInsertionEvent`）是对数据库文件落盘操作的高层抽象，它是若干 Tablet 写入事件（`TabletInsertionEvent`）的数据集合。 IoTDB 的存储引擎是 LSM 结构的。数据写入时会先将写入操作落盘到日志结构的文件里，同时将写入数据保存在内存里。当内存达到控制上限，则会触发刷盘行为，即将内存中的数据转换为数据库文件，同时删除之前预写的操作日志。当内存中的数据转换为数据库文件中的数据时，会经过编码压缩和通用压缩两次压缩处理，因此数据库文件的数据相比内存中的原始数据占用的空间更少。 @@ -109,7 +109,7 @@ IoTDB 的存储引擎是 LSM 结构的。数据写入时会先将写入操作落 （1）历史数据抽取：一个流处理任务开始前，所有已经落盘的写入数据都会以 TsFile 的形式存在。一个流处理任务开始后，采集历史数据时，历史数据将以 TsFileInsertionEvent 作为抽象； -（2）实时数据抽取：一个流处理任务进行时，当数据流中实时处理操作日志写入事件的速度慢于写入请求速度一定进度之后，未来得及处理的操作日志写入事件会被被持久化至磁盘，以 TsFile 的形式存在，这一些数据被流处理引擎抽取到后，会以 TsFileInsertionEvent 作为抽象。 +（2）实时数据抽取：一个流处理任务运行时，实时写入的数据通常以 `TabletInsertionEvent` 的形式进行处理。当待处理的数据出现积压时，流处理引擎可以从已经落盘的 TsFile 中批量抽取数据，并以 `TsFileInsertionEvent` 的形式进行处理。 ```java /** diff --git a/src/zh/UserGuide/latest/User-Manual/Streaming_apache.md b/src/zh/UserGuide/latest/User-Manual/Streaming_apache.md index 0a9ce0a34..6f4eda3ea 100644 --- a/src/zh/UserGuide/latest/User-Manual/Streaming_apache.md +++ b/src/zh/UserGuide/latest/User-Manual/Streaming_apache.md @@ -63,17 +63,17 @@ Pipe Source 用于抽取数据，Pipe Processor 用于处理数据，Pipe Sink 在流处理插件的用户编程接口中，事件是数据库数据写入操作的抽象。事件由单机流处理引擎捕获，按照流处理三个阶段的流程，依次传递至 PipeSource 插件，PipeProcessor 插件和 PipeSink 插件，并依次在三个插件中触发用户逻辑的执行。 -为了兼顾端侧低负载场景下的流处理低延迟和端侧高负载场景下的流处理高吞吐，流处理引擎会动态地在操作日志和数据文件中选择处理对象，因此，流处理的用户编程接口要求用户提供下列两类事件的处理逻辑：操作日志写入事件 TabletInsertionEvent 和数据文件写入事件 TsFileInsertionEvent。 +为了兼顾端侧低负载场景下的流处理低延迟和端侧高负载场景下的流处理高吞吐，流处理引擎会动态选择处理实时写入数据或数据文件。因此，流处理的用户编程接口要求用户提供下列两类事件的处理逻辑：Tablet 写入事件 `TabletInsertionEvent` 和 TsFile 写入事件 `TsFileInsertionEvent`。 -#### **操作日志写入事件（TabletInsertionEvent）** +#### **Tablet 写入事件（TabletInsertionEvent）** -操作日志写入事件（TabletInsertionEvent）是对用户写入请求的高层数据抽象，它通过提供统一的操作接口，为用户提供了操纵写入请求底层数据的能力。 +Tablet 写入事件（`TabletInsertionEvent`）是对用户写入请求的高层数据抽象，它通过提供统一的操作接口，为用户提供了处理写入数据的能力。 -对于不同的数据库部署方式，操作日志写入事件对应的底层存储结构是不一样的。对于单机部署的场景，操作日志写入事件是对写前日志（WAL）条目的封装；对于分布式部署的场景，操作日志写入事件是对单个节点共识协议操作日志条目的封装。 +自 V2.0.5 起，流处理框架会直接从写入请求中获取实时写入数据，并将其封装为 `TabletInsertionEvent`，不再从操作日志中读取数据。该变化不影响 `TabletInsertionEvent` 接口以及已有插件的使用方式。 -对于数据库不同写入请求接口生成的写入操作，操作日志写入事件对应的请求结构体的数据结构也是不一样的。IoTDB 提供了 InsertRecord、InsertRecords、InsertTablet、InsertTablets 等众多的写入接口，每一种写入请求都使用了完全不同的序列化方式，生成的二进制条目也不尽相同。 +对于数据库不同写入接口生成的写入操作，Tablet 写入事件对应的数据结构也是不一样的。IoTDB 提供了 `InsertRecord`、`InsertRecords`、`InsertTablet`、`InsertTablets` 等多种写入接口，不同写入请求的数据结构也不尽相同。 -操作日志写入事件的存在，为用户提供了一种统一的数据操作视图，它屏蔽了底层数据结构的实现差异，极大地降低了用户的编程门槛，提升了功能的易用性。 +Tablet 写入事件为用户提供了一种统一的数据操作视图，它屏蔽了底层数据结构的实现差异，极大地降低了用户的编程门槛，提升了功能的易用性。 ```java /** TabletInsertionEvent is used to define the event of data insertion. */ @@ -97,9 +97,9 @@ public interface TabletInsertionEvent extends Event { } ``` -#### **数据文件写入事件（TsFileInsertionEvent）** +#### **TsFile 写入事件（TsFileInsertionEvent）** -数据文件写入事件（TsFileInsertionEvent） 是对数据库文件落盘操作的高层抽象，它是若干操作日志写入事件（TabletInsertionEvent）的数据集合。 +TsFile 写入事件（`TsFileInsertionEvent`）是对数据库文件落盘操作的高层抽象，它是若干 Tablet 写入事件（`TabletInsertionEvent`）的数据集合。 IoTDB 的存储引擎是 LSM 结构的。数据写入时会先将写入操作落盘到日志结构的文件里，同时将写入数据保存在内存里。当内存达到控制上限，则会触发刷盘行为，即将内存中的数据转换为数据库文件，同时删除之前预写的操作日志。当内存中的数据转换为数据库文件中的数据时，会经过编码压缩和通用压缩两次压缩处理，因此数据库文件的数据相比内存中的原始数据占用的空间更少。 @@ -109,7 +109,7 @@ IoTDB 的存储引擎是 LSM 结构的。数据写入时会先将写入操作落 （1）历史数据抽取：一个流处理任务开始前，所有已经落盘的写入数据都会以 TsFile 的形式存在。一个流处理任务开始后，采集历史数据时，历史数据将以 TsFileInsertionEvent 作为抽象； -（2）实时数据抽取：一个流处理任务进行时，当数据流中实时处理操作日志写入事件的速度慢于写入请求速度一定进度之后，未来得及处理的操作日志写入事件会被被持久化至磁盘，以 TsFile 的形式存在，这一些数据被流处理引擎抽取到后，会以 TsFileInsertionEvent 作为抽象。 +（2）实时数据抽取：一个流处理任务运行时，实时写入的数据通常以 `TabletInsertionEvent` 的形式进行处理。当待处理的数据出现积压时，流处理引擎可以从已经落盘的 TsFile 中批量抽取数据，并以 `TsFileInsertionEvent` 的形式进行处理。 ```java /**