From 6fa22dbcc3fca2e67f967d5cb3dbebfaf0ac057e Mon Sep 17 00:00:00 2001 From: Leto_b Date: Mon, 2 Mar 2026 14:57:06 +0800 Subject: [PATCH] MQTT service and REST service JAR packages are externalized from 2082 --- src/.vuepress/sidebar/V2.0.x/en-Table.ts | 4 +- src/.vuepress/sidebar/V2.0.x/en-Tree.ts | 2 +- src/.vuepress/sidebar/V2.0.x/zh-Table.ts | 2 +- src/.vuepress/sidebar/V2.0.x/zh-Tree.ts | 2 +- .../sidebar_timecho/V2.0.x/en-Table.ts | 4 +- .../sidebar_timecho/V2.0.x/en-Tree.ts | 2 +- .../sidebar_timecho/V2.0.x/zh-Table.ts | 2 +- .../sidebar_timecho/V2.0.x/zh-Tree.ts | 2 +- .../Master/Table/API/Programming-MQTT.md | 245 +------------ .../Table/API/Programming-MQTT_apache.md | 259 +++++++++++++ .../Table/API/Programming-MQTT_timecho.md | 261 ++++++++++++++ src/UserGuide/Master/Table/API/RestAPI-V1.md | 323 +---------------- .../Master/Table/API/RestAPI-V1_apache.md | 338 +++++++++++++++++ .../Master/Table/API/RestAPI-V1_timecho.md | 340 ++++++++++++++++++ .../Basic-Concept/Write-Updata-Data_apache.md | 2 +- .../Write-Updata-Data_timecho.md | 2 +- .../IoTDB-Package_timecho.md | 4 +- .../Master/Tree/API/Programming-MQTT.md | 278 +------------- .../Tree/API/Programming-MQTT_apache.md | 293 +++++++++++++++ .../Tree/API/Programming-MQTT_timecho.md | 294 +++++++++++++++ .../Master/Tree/API/RestServiceV1_timecho.md | 2 + .../Master/Tree/API/RestServiceV2_timecho.md | 2 + .../Tree/Basic-Concept/Write-Data_apache.md | 2 +- .../Tree/Basic-Concept/Write-Data_timecho.md | 2 +- .../IoTDB-Package_timecho.md | 4 +- .../latest-Table/API/Programming-MQTT.md | 244 +------------ .../API/Programming-MQTT_apache.md | 259 +++++++++++++ .../API/Programming-MQTT_timecho.md | 261 ++++++++++++++ src/UserGuide/latest-Table/API/RestAPI-V1.md | 323 +---------------- .../latest-Table/API/RestAPI-V1_apache.md | 338 +++++++++++++++++ .../latest-Table/API/RestAPI-V1_timecho.md | 340 ++++++++++++++++++ .../Basic-Concept/Write-Updata-Data_apache.md | 2 +- .../Write-Updata-Data_timecho.md | 2 +- .../IoTDB-Package_timecho.md | 4 +- src/UserGuide/latest/API/Programming-MQTT.md | 278 +------------- .../latest/API/Programming-MQTT_apache.md | 293 +++++++++++++++ .../latest/API/Programming-MQTT_timecho.md | 294 +++++++++++++++ .../latest/API/RestServiceV1_timecho.md | 2 + .../latest/API/RestServiceV2_timecho.md | 2 + .../latest/Basic-Concept/Write-Data_apache.md | 2 +- .../Basic-Concept/Write-Data_timecho.md | 2 +- .../IoTDB-Package_timecho.md | 4 +- .../Master/Table/API/Programming-MQTT.md | 247 +------------ .../Table/API/Programming-MQTT_apache.md | 262 ++++++++++++++ .../Table/API/Programming-MQTT_timecho.md | 264 ++++++++++++++ .../Master/Table/API/RestServiceV1.md | 319 +--------------- .../Master/Table/API/RestServiceV1_apache.md | 334 +++++++++++++++++ .../Master/Table/API/RestServiceV1_timecho.md | 336 +++++++++++++++++ .../IoTDB-Package_timecho.md | 2 + .../Master/Tree/API/Programming-MQTT.md | 278 +------------- .../Tree/API/Programming-MQTT_apache.md | 293 +++++++++++++++ .../Tree/API/Programming-MQTT_timecho.md | 295 +++++++++++++++ .../Master/Tree/API/RestServiceV1_timecho.md | 2 + .../Master/Tree/API/RestServiceV2_timecho.md | 2 + .../Tree/Basic-Concept/Write-Data_apache.md | 2 +- .../Tree/Basic-Concept/Write-Data_timecho.md | 2 +- .../IoTDB-Package_timecho.md | 4 +- .../latest-Table/API/Programming-MQTT.md | 247 +------------ .../API/Programming-MQTT_apache.md | 262 ++++++++++++++ .../API/Programming-MQTT_timecho.md | 264 ++++++++++++++ .../latest-Table/API/RestServiceV1.md | 319 +--------------- .../latest-Table/API/RestServiceV1_apache.md | 334 +++++++++++++++++ .../latest-Table/API/RestServiceV1_timecho.md | 336 +++++++++++++++++ .../IoTDB-Package_timecho.md | 2 + .../UserGuide/latest/API/Programming-MQTT.md | 278 +------------- .../latest/API/Programming-MQTT_apache.md | 293 +++++++++++++++ .../latest/API/Programming-MQTT_timecho.md | 295 +++++++++++++++ .../latest/API/RestServiceV1_timecho.md | 2 + .../latest/API/RestServiceV2_timecho.md | 2 + .../latest/Basic-Concept/Write-Data_apache.md | 2 +- .../Basic-Concept/Write-Data_timecho.md | 2 +- .../IoTDB-Package_timecho.md | 2 + 72 files changed, 7245 insertions(+), 3358 deletions(-) create mode 100644 src/UserGuide/Master/Table/API/Programming-MQTT_apache.md create mode 100644 src/UserGuide/Master/Table/API/Programming-MQTT_timecho.md create mode 100644 src/UserGuide/Master/Table/API/RestAPI-V1_apache.md create mode 100644 src/UserGuide/Master/Table/API/RestAPI-V1_timecho.md create mode 100644 src/UserGuide/Master/Tree/API/Programming-MQTT_apache.md create mode 100644 src/UserGuide/Master/Tree/API/Programming-MQTT_timecho.md create mode 100644 src/UserGuide/latest-Table/API/Programming-MQTT_apache.md create mode 100644 src/UserGuide/latest-Table/API/Programming-MQTT_timecho.md create mode 100644 src/UserGuide/latest-Table/API/RestAPI-V1_apache.md create mode 100644 src/UserGuide/latest-Table/API/RestAPI-V1_timecho.md create mode 100644 src/UserGuide/latest/API/Programming-MQTT_apache.md create mode 100644 src/UserGuide/latest/API/Programming-MQTT_timecho.md create mode 100644 src/zh/UserGuide/Master/Table/API/Programming-MQTT_apache.md create mode 100644 src/zh/UserGuide/Master/Table/API/Programming-MQTT_timecho.md create mode 100644 src/zh/UserGuide/Master/Table/API/RestServiceV1_apache.md create mode 100644 src/zh/UserGuide/Master/Table/API/RestServiceV1_timecho.md create mode 100644 src/zh/UserGuide/Master/Tree/API/Programming-MQTT_apache.md create mode 100644 src/zh/UserGuide/Master/Tree/API/Programming-MQTT_timecho.md create mode 100644 src/zh/UserGuide/latest-Table/API/Programming-MQTT_apache.md create mode 100644 src/zh/UserGuide/latest-Table/API/Programming-MQTT_timecho.md create mode 100644 src/zh/UserGuide/latest-Table/API/RestServiceV1_apache.md create mode 100644 src/zh/UserGuide/latest-Table/API/RestServiceV1_timecho.md create mode 100644 src/zh/UserGuide/latest/API/Programming-MQTT_apache.md create mode 100644 src/zh/UserGuide/latest/API/Programming-MQTT_timecho.md diff --git a/src/.vuepress/sidebar/V2.0.x/en-Table.ts b/src/.vuepress/sidebar/V2.0.x/en-Table.ts index cd15c118e..b5171d4c9 100644 --- a/src/.vuepress/sidebar/V2.0.x/en-Table.ts +++ b/src/.vuepress/sidebar/V2.0.x/en-Table.ts @@ -174,8 +174,8 @@ export const enSidebar = { { text: 'GO Native API', link: 'Programming-Go-Native-API_apache' }, { text: 'C# Native API', link: 'Programming-CSharp-Native-API_apache' }, { text: 'JDBC', link: 'Programming-JDBC_apache' }, - { text: 'MQTT Protocol', link: 'Programming-MQTT' }, - { text: 'RESTAPI V1 ', link: 'RestAPI-V1' }, + { text: 'MQTT Protocol', link: 'Programming-MQTT_apache' }, + { text: 'RESTAPI V1 ', link: 'RestAPI-V1_apache' }, ], }, { diff --git a/src/.vuepress/sidebar/V2.0.x/en-Tree.ts b/src/.vuepress/sidebar/V2.0.x/en-Tree.ts index a6d09c31c..7ca6aa4c4 100644 --- a/src/.vuepress/sidebar/V2.0.x/en-Tree.ts +++ b/src/.vuepress/sidebar/V2.0.x/en-Tree.ts @@ -197,7 +197,7 @@ export const enSidebar = { { text: 'Node.js Native API', link: 'Programming-NodeJS-Native-API' }, { text: 'Rust Native API', link: 'Programming-Rust-Native-API' }, { text: 'JDBC', link: 'Programming-JDBC_apache' }, - { text: 'MQTT Protocol', link: 'Programming-MQTT' }, + { text: 'MQTT Protocol', link: 'Programming-MQTT_apache' }, { text: 'REST API', collapsible: true, diff --git a/src/.vuepress/sidebar/V2.0.x/zh-Table.ts b/src/.vuepress/sidebar/V2.0.x/zh-Table.ts index 7ac6579ac..6208f43ef 100644 --- a/src/.vuepress/sidebar/V2.0.x/zh-Table.ts +++ b/src/.vuepress/sidebar/V2.0.x/zh-Table.ts @@ -171,7 +171,7 @@ export const zhSidebar = { { text: 'GO原生接口', link: 'Programming-Go-Native-API_apache' }, { text: 'C#原生接口', link: 'Programming-CSharp-Native-API_apache' }, { text: 'JDBC', link: 'Programming-JDBC_apache' }, - { text: 'MQTT协议', link: 'Programming-MQTT' }, + { text: 'MQTT协议', link: 'Programming-MQTT_apache' }, { text: 'RESTAPI V1 ', link: 'RestServiceV1' }, ], }, diff --git a/src/.vuepress/sidebar/V2.0.x/zh-Tree.ts b/src/.vuepress/sidebar/V2.0.x/zh-Tree.ts index 2dd161c3a..a3d4ce57e 100644 --- a/src/.vuepress/sidebar/V2.0.x/zh-Tree.ts +++ b/src/.vuepress/sidebar/V2.0.x/zh-Tree.ts @@ -188,7 +188,7 @@ export const zhSidebar = { { text: 'Node.js原生接口', link: 'Programming-NodeJS-Native-API' }, { text: 'Rust原生接口', link: 'Programming-Rust-Native-API' }, { text: 'JDBC', link: 'Programming-JDBC_apache' }, - { text: 'MQTT协议', link: 'Programming-MQTT' }, + { text: 'MQTT协议', link: 'Programming-MQTT_apache' }, { text: 'REST API', collapsible: true, diff --git a/src/.vuepress/sidebar_timecho/V2.0.x/en-Table.ts b/src/.vuepress/sidebar_timecho/V2.0.x/en-Table.ts index 0b33f4719..852f2c2d2 100644 --- a/src/.vuepress/sidebar_timecho/V2.0.x/en-Table.ts +++ b/src/.vuepress/sidebar_timecho/V2.0.x/en-Table.ts @@ -202,8 +202,8 @@ export const enSidebar = { { text: 'GO Native API', link: 'Programming-Go-Native-API_timecho' }, { text: 'C# Native API', link: 'Programming-CSharp-Native-API_timecho' }, { text: 'JDBC', link: 'Programming-JDBC_timecho' }, - { text: 'MQTT Protocol', link: 'Programming-MQTT' }, - { text: 'RESTAPI V1 ', link: 'RestAPI-V1' }, + { text: 'MQTT Protocol', link: 'Programming-MQTT_timecho' }, + { text: 'RESTAPI V1 ', link: 'RestAPI-V1_timecho' }, ], }, { diff --git a/src/.vuepress/sidebar_timecho/V2.0.x/en-Tree.ts b/src/.vuepress/sidebar_timecho/V2.0.x/en-Tree.ts index 1a2fb46dc..e3268e213 100644 --- a/src/.vuepress/sidebar_timecho/V2.0.x/en-Tree.ts +++ b/src/.vuepress/sidebar_timecho/V2.0.x/en-Tree.ts @@ -222,7 +222,7 @@ export const enSidebar = { { text: 'Node.js Native API', link: 'Programming-NodeJS-Native-API' }, { text: 'Rust Native API', link: 'Programming-Rust-Native-API' }, { text: 'JDBC', link: 'Programming-JDBC_timecho' }, - { text: 'MQTT Protocol', link: 'Programming-MQTT' }, + { text: 'MQTT Protocol', link: 'Programming-MQTT_timecho' }, { text: 'OPC UA Protocol', link: 'Programming-OPC-UA_timecho' }, { text: 'OPC DA Protocol', link: 'Programming-OPC-DA_timecho' }, { diff --git a/src/.vuepress/sidebar_timecho/V2.0.x/zh-Table.ts b/src/.vuepress/sidebar_timecho/V2.0.x/zh-Table.ts index 340e66fb1..746d1dc46 100644 --- a/src/.vuepress/sidebar_timecho/V2.0.x/zh-Table.ts +++ b/src/.vuepress/sidebar_timecho/V2.0.x/zh-Table.ts @@ -193,7 +193,7 @@ export const zhSidebar = { { text: 'GO原生接口', link: 'Programming-Go-Native-API_timecho' }, { text: 'C#原生接口', link: 'Programming-CSharp-Native-API_timecho' }, { text: 'JDBC', link: 'Programming-JDBC_timecho' }, - { text: 'MQTT协议', link: 'Programming-MQTT' }, + { text: 'MQTT协议', link: 'Programming-MQTT_timecho' }, { text: 'RESTAPI V1 ', link: 'RestServiceV1' }, ], }, diff --git a/src/.vuepress/sidebar_timecho/V2.0.x/zh-Tree.ts b/src/.vuepress/sidebar_timecho/V2.0.x/zh-Tree.ts index 59c88217b..66cdded38 100644 --- a/src/.vuepress/sidebar_timecho/V2.0.x/zh-Tree.ts +++ b/src/.vuepress/sidebar_timecho/V2.0.x/zh-Tree.ts @@ -204,7 +204,7 @@ export const zhSidebar = { { text: 'Node.js原生接口', link: 'Programming-NodeJS-Native-API' }, { text: 'Rust', link: 'Programming-Rust-Native-API' }, { text: 'JDBC', link: 'Programming-JDBC_timecho' }, - { text: 'MQTT协议', link: 'Programming-MQTT' }, + { text: 'MQTT协议', link: 'Programming-MQTT_timecho' }, { text: 'OPC UA协议', link: 'Programming-OPC-UA_timecho' }, { text: 'OPC DA协议', link: 'Programming-OPC-DA_timecho' }, { diff --git a/src/UserGuide/Master/Table/API/Programming-MQTT.md b/src/UserGuide/Master/Table/API/Programming-MQTT.md index 0ea8d2e15..0a6a38616 100644 --- a/src/UserGuide/Master/Table/API/Programming-MQTT.md +++ b/src/UserGuide/Master/Table/API/Programming-MQTT.md @@ -1,3 +1,6 @@ +--- +redirectTo: Programming-MQTT_apache.html +--- -# MQTT Protocol - -## 1. Overview - -MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for IoT and low-bandwidth environments. It operates on a Publish/Subscribe (Pub/Sub) model, enabling efficient and reliable bidirectional communication between devices. Its core objectives are low power consumption, minimal bandwidth usage, and high real-time performance, making it ideal for unstable networks or resource-constrained scenarios (e.g., sensors, mobile devices). - -IoTDB provides deep integration with the MQTT protocol, fully compliant with MQTT v3.1 (OASIS International Standard). The IoTDB server includes a built-in high-performance MQTT Broker module, eliminating the need for third-party middleware. Devices can directly write time-series data into the IoTDB storage engine via MQTT messages. - -![](/img/mqtt-table-en-1.png) - - -## 2. Configuration - -By default, the IoTDB MQTT service loads configurations from `${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`. - -| **Property** | **Description** | **Default** | -| ------------------------ | ---------------------------------------------------------------------------------------------------------------------- | ------------------- | -| `enable_mqtt_service` | Enable/ disable the MQTT service. | FALSE | -| `mqtt_host` | Host address bound to the MQTT service. | 127.0.0.1 | -| `mqtt_port` | Port bound to the MQTT service. | 1883 | -| `mqtt_handler_pool_size` | Thread pool size for processing MQTT messages. | 1 | -| **`mqtt_payload_formatter`** | **Formatting method for MQTT message payloads. ​**​**Options: `json` (tree model), `line` (table model).** | **json** | -| `mqtt_max_message_size` | Maximum allowed MQTT message size (bytes). | 1048576 | - -## 3. Write Protocol - -* Line Protocol Syntax - -```JavaScript -[,=[,=]][ =[,=]] =[,=] [] -``` - -* Example - -```JavaScript -myMeasurement,tag1=value1,tag2=value2 attr1=value1,attr2=value2 fieldKey="fieldValue" 1556813561098000000 -``` - -![](/img/mqtt-table-en-2.png) - -## 4. Naming Conventions - -* Database Name - -The first segment of the MQTT topic (split by `/`) is used as the database name. - -```Properties -topic: stock/Legacy -databaseName: stock - - -topic: stock/Legacy/# -databaseName:stock -``` - -* Table Name - -The table name is derived from the `` in the line protocol. - -* Type Identifiers - -| Filed Value | IoTDB Data Type | -|--------------------------------------------------------------------| ----------------- | -| 1
1.12 | DOUBLE | -| 1`f`
1.12`f` | FLOAT | -| 1`i`
123`i` | INT64 | -| 1`u`
123`u` | INT64 | -| 1`i32`
123`i32` | INT32 | -| `"xxx"` | TEXT | -| `t`,`T`,`true`,`True`,`TRUE`
`f`,`F`,`false`,`False`,`FALSE` | BOOLEAN | - - -## 5. Coding Examples -The following is an example which a mqtt client send messages to IoTDB server. - - ```java -MQTT mqtt = new MQTT(); -mqtt.setHost("127.0.0.1", 1883); -mqtt.setUserName("root"); -mqtt.setPassword("root"); - -BlockingConnection connection = mqtt.blockingConnection(); -String DATABASE = "myMqttTest"; -connection.connect(); - -String payload = - "test1,tag1=t1,tag2=t2 attr3=a5,attr4=a4 field1=\"fieldValue1\",field2=1i,field3=1u 1"; -connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -payload = "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 2"; -connection.publish(DATABASE, payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -payload = "# It's a remark\n " + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 6"; - connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); - Thread.sleep(10); - -//batch write example -payload = - "test1,tag1=t1,tag2=t2 field7=t,field8=T,field9=true 3 \n " - + "test1,tag1=t1,tag2=t2 field7=f,field8=F,field9=FALSE 4"; -connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -//batch write example -payload = - "test1,tag1=t1,tag2=t2 attr1=a1,attr2=a2 field1=\"fieldValue1\",field2=1i,field3=1u 4 \n " - + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 5"; -connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -connection.disconnect(); - ``` - - - -## 6. Customize your MQTT Message Format - -If you do not like the above Line format, you can customize your MQTT Message format by just writing several lines -of codes. An example can be found in [example/mqtt-customize](https://github.com/apache/iotdb/tree/master/example/mqtt-customize) project. - -Steps: -1. Create a java project, and add dependency: -```xml - - org.apache.iotdb - iotdb-server - 2.0.4-SNAPSHOT - -``` -2. Define your implementation which implements `org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` - e.g., - -```java -package org.apache.iotdb.mqtt.server; - -import io.netty.buffer.ByteBuf; -import org.apache.iotdb.db.protocol.mqtt.Message; -import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; - -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -public class CustomizedLinePayloadFormatter implements PayloadFormatter { - - @Override - public List format(String topic, ByteBuf payload) { - // Suppose the payload is a line format - if (payload == null) { - return null; - } - - String line = payload.toString(StandardCharsets.UTF_8); - // parse data from the line and generate Messages and put them into List ret - List ret = new ArrayList<>(); - // this is just an example, so we just generate some Messages directly - for (int i = 0; i < 3; i++) { - long ts = i; - TableMessage message = new TableMessage(); - - // Parsing Database Name - message.setDatabase("db" + i); - - //Parsing Table Names - message.setTable("t" + i); - - // Parsing Tags - List tagKeys = new ArrayList<>(); - tagKeys.add("tag1" + i); - tagKeys.add("tag2" + i); - List tagValues = new ArrayList<>(); - tagValues.add("t_value1" + i); - tagValues.add("t_value2" + i); - message.setTagKeys(tagKeys); - message.setTagValues(tagValues); - - // Parsing Attributes - List attributeKeys = new ArrayList<>(); - List attributeValues = new ArrayList<>(); - attributeKeys.add("attr1" + i); - attributeKeys.add("attr2" + i); - attributeValues.add("a_value1" + i); - attributeValues.add("a_value2" + i); - message.setAttributeKeys(attributeKeys); - message.setAttributeValues(attributeValues); - - // Parsing Fields - List fields = Arrays.asList("field1" + i, "field2" + i); - List dataTypes = Arrays.asList(TSDataType.FLOAT, TSDataType.FLOAT); - List values = Arrays.asList("4.0" + i, "5.0" + i); - message.setFields(fields); - message.setDataTypes(dataTypes); - message.setValues(values); - - //// Parsing timestamp - message.setTimestamp(ts); - ret.add(message); - } - return ret; - } - - @Override - public String getName() { - // set the value of mqtt_payload_formatter in iotdb-system.properties as the following string: - return "CustomizedLine"; - } -} -``` -3. modify the file in `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter`: - clean the file and put your implementation class name into the file. - In this example, the content is: `org.apache.iotdb.mqtt.server.CustomizedLinePayloadFormatter` -4. compile your implementation as a jar file: `mvn package -DskipTests` - - -Then, in your server: -1. Create ${IOTDB_HOME}/ext/mqtt/ folder, and put the jar into this folder. -2. Update configuration to enable MQTT service. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) -3. Set the value of `mqtt_payload_formatter` in `conf/iotdb-system.properties` as the value of getName() in your implementation - , in this example, the value is `CustomizedLine` -4. Launch the IoTDB server. -5. Now IoTDB will use your implementation to parse the MQTT message. - -More: the message format can be anything you want. For example, if it is a binary format, -just use `payload.forEachByte()` or `payload.array` to get bytes content. - - -## 7. Caution - -To avoid compatibility issues caused by a default client_id, always explicitly supply a unique, non-empty client_id in every MQTT client. -Behavior varies when the client_id is missing or empty. Common examples: -1. Explicitly sending an empty string -• MQTTX: When client_id="", IoTDB silently discards the message. -• mosquitto_pub: When client_id="", IoTDB receives the message normally. -2. Omitting client_id entirely -• MQTTX: IoTDB accepts the message. -• mosquitto_pub: IoTDB rejects the connection. -Therefore, explicitly assigning a unique, non-empty client_id is the simplest way to eliminate these discrepancies and ensure reliable message delivery. \ No newline at end of file +--> \ No newline at end of file diff --git a/src/UserGuide/Master/Table/API/Programming-MQTT_apache.md b/src/UserGuide/Master/Table/API/Programming-MQTT_apache.md new file mode 100644 index 000000000..c7e4e438c --- /dev/null +++ b/src/UserGuide/Master/Table/API/Programming-MQTT_apache.md @@ -0,0 +1,259 @@ + +# MQTT Protocol + +## 1. Overview + +MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for IoT and low-bandwidth environments. It operates on a Publish/Subscribe (Pub/Sub) model, enabling efficient and reliable bidirectional communication between devices. Its core objectives are low power consumption, minimal bandwidth usage, and high real-time performance, making it ideal for unstable networks or resource-constrained scenarios (e.g., sensors, mobile devices). + +IoTDB provides deep integration with the MQTT protocol, fully compliant with MQTT v3.1 (OASIS International Standard). The IoTDB server includes a built-in high-performance MQTT Broker module, eliminating the need for third-party middleware. Devices can directly write time-series data into the IoTDB storage engine via MQTT messages. + +![](/img/mqtt-table-en-1.png) + + +## 2. Configuration + +By default, the IoTDB MQTT service loads configurations from `${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`. + +| **Property** | **Description** | **Default** | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------- | ------------------- | +| `enable_mqtt_service` | Enable/ disable the MQTT service. | FALSE | +| `mqtt_host` | Host address bound to the MQTT service. | 127.0.0.1 | +| `mqtt_port` | Port bound to the MQTT service. | 1883 | +| `mqtt_handler_pool_size` | Thread pool size for processing MQTT messages. | 1 | +| **`mqtt_payload_formatter`** | **Formatting method for MQTT message payloads. ​**​**Options: `json` (tree mode), `line` (table mode).** | **json** | +| `mqtt_max_message_size` | Maximum allowed MQTT message size (bytes). | 1048576 | + +## 3. Write Protocol + +* Line Protocol Syntax + +```JavaScript +[,=[,=]][ =[,=]] =[,=] [] +``` + +* Example + +```JavaScript +myMeasurement,tag1=value1,tag2=value2 attr1=value1,attr2=value2 fieldKey="fieldValue" 1556813561098000000 +``` + +![](/img/mqtt-table-en-2.png) + +## 4. Naming Conventions + +* Database Name + +The first segment of the MQTT topic (split by `/`) is used as the database name. + +```Properties +topic: stock/Legacy +databaseName: stock + + +topic: stock/Legacy/# +databaseName:stock +``` + +* Table Name + +The table name is derived from the `` in the line protocol. + +* Type Identifiers + +| Filed Value | IoTDB Data Type | +|--------------------------------------------------------------------| ----------------- | +| 1
1.12 | DOUBLE | +| 1`f`
1.12`f` | FLOAT | +| 1`i`
123`i` | INT64 | +| 1`u`
123`u` | INT64 | +| 1`i32`
123`i32` | INT32 | +| `"xxx"` | TEXT | +| `t`,`T`,`true`,`True`,`TRUE`
`f`,`F`,`false`,`False`,`FALSE` | BOOLEAN | + + +## 5. Coding Examples +The following is an example which a mqtt client send messages to IoTDB server. + + ```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +String DATABASE = "myMqttTest"; +connection.connect(); + +String payload = + "test1,tag1=t1,tag2=t2 attr3=a5,attr4=a4 field1=\"fieldValue1\",field2=1i,field3=1u 1"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 2"; +connection.publish(DATABASE, payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "# It's a remark\n " + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 6"; + connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); + Thread.sleep(10); + +//batch write example +payload = + "test1,tag1=t1,tag2=t2 field7=t,field8=T,field9=true 3 \n " + + "test1,tag1=t1,tag2=t2 field7=f,field8=F,field9=FALSE 4"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +//batch write example +payload = + "test1,tag1=t1,tag2=t2 attr1=a1,attr2=a2 field1=\"fieldValue1\",field2=1i,field3=1u 4 \n " + + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 5"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +connection.disconnect(); + ``` + + + +## 6. Customize your MQTT Message Format + +If you do not like the above Line format, you can customize your MQTT Message format by just writing several lines +of codes. An example can be found in [example/mqtt-customize](https://github.com/apache/iotdb/tree/master/example/mqtt-customize) project. + +Steps: +1. Create a java project, and add dependency: +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. Define your implementation which implements `org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` + e.g., + +```java +package org.apache.iotdb.mqtt.server; + +import io.netty.buffer.ByteBuf; +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +public class CustomizedLinePayloadFormatter implements PayloadFormatter { + + @Override + public List format(String topic, ByteBuf payload) { + // Suppose the payload is a line format + if (payload == null) { + return null; + } + + String line = payload.toString(StandardCharsets.UTF_8); + // parse data from the line and generate Messages and put them into List ret + List ret = new ArrayList<>(); + // this is just an example, so we just generate some Messages directly + for (int i = 0; i < 3; i++) { + long ts = i; + TableMessage message = new TableMessage(); + + // Parsing Database Name + message.setDatabase("db" + i); + + //Parsing Table Names + message.setTable("t" + i); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add("tag1" + i); + tagKeys.add("tag2" + i); + List tagValues = new ArrayList<>(); + tagValues.add("t_value1" + i); + tagValues.add("t_value2" + i); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add("attr1" + i); + attributeKeys.add("attr2" + i); + attributeValues.add("a_value1" + i); + attributeValues.add("a_value2" + i); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList("field1" + i, "field2" + i); + List dataTypes = Arrays.asList(TSDataType.FLOAT, TSDataType.FLOAT); + List values = Arrays.asList("4.0" + i, "5.0" + i); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + //// Parsing timestamp + message.setTimestamp(ts); + ret.add(message); + } + return ret; + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-system.properties as the following string: + return "CustomizedLine"; + } +} +``` +3. modify the file in `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter`: + clean the file and put your implementation class name into the file. + In this example, the content is: `org.apache.iotdb.mqtt.server.CustomizedLinePayloadFormatter` +4. compile your implementation as a jar file: `mvn package -DskipTests` + + +Then, in your server: +1. Create ${IOTDB_HOME}/ext/mqtt/ folder, and put the jar into this folder. +2. Update configuration to enable MQTT service. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. Set the value of `mqtt_payload_formatter` in `conf/iotdb-system.properties` as the value of getName() in your implementation + , in this example, the value is `CustomizedLine` +4. Launch the IoTDB server. +5. Now IoTDB will use your implementation to parse the MQTT message. + +More: the message format can be anything you want. For example, if it is a binary format, +just use `payload.forEachByte()` or `payload.array` to get bytes content. + +## 7. Caution + +To avoid compatibility issues caused by a default client_id, always explicitly supply a unique, non-empty client_id in every MQTT client. +Behavior varies when the client_id is missing or empty. Common examples: +1. Explicitly sending an empty string +• MQTTX: When client_id="", IoTDB silently discards the message. +• mosquitto_pub: When client_id="", IoTDB receives the message normally. +2. Omitting client_id entirely +• MQTTX: IoTDB accepts the message. +• mosquitto_pub: IoTDB rejects the connection. +Therefore, explicitly assigning a unique, non-empty client_id is the simplest way to eliminate these discrepancies and ensure reliable message delivery. \ No newline at end of file diff --git a/src/UserGuide/Master/Table/API/Programming-MQTT_timecho.md b/src/UserGuide/Master/Table/API/Programming-MQTT_timecho.md new file mode 100644 index 000000000..3c6170a2f --- /dev/null +++ b/src/UserGuide/Master/Table/API/Programming-MQTT_timecho.md @@ -0,0 +1,261 @@ + +# MQTT Protocol + +## 1. Overview + +MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for IoT and low-bandwidth environments. It operates on a Publish/Subscribe (Pub/Sub) model, enabling efficient and reliable bidirectional communication between devices. Its core objectives are low power consumption, minimal bandwidth usage, and high real-time performance, making it ideal for unstable networks or resource-constrained scenarios (e.g., sensors, mobile devices). + +IoTDB provides deep integration with the MQTT protocol, fully compliant with MQTT v3.1 (OASIS International Standard). The IoTDB server includes a built-in high-performance MQTT Broker module, eliminating the need for third-party middleware. Devices can directly write time-series data into the IoTDB storage engine via MQTT messages. + +![](/img/mqtt-table-en-1.png) + +Note: As of version V2.0.8, the TimechoDB installation package does not include the MQTT service JAR file by default. Please contact the Timecho team to obtain the JAR file before using this service, and place it in the `timechodb_home/lib` or `timechodb_home/ext/external_service` directory. + + +## 2. Configuration + +By default, the IoTDB MQTT service loads configurations from `${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`. + +| **Property** | **Description** | **Default** | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------- | ------------------- | +| `enable_mqtt_service` | Enable/ disable the MQTT service. | FALSE | +| `mqtt_host` | Host address bound to the MQTT service. | 127.0.0.1 | +| `mqtt_port` | Port bound to the MQTT service. | 1883 | +| `mqtt_handler_pool_size` | Thread pool size for processing MQTT messages. | 1 | +| **`mqtt_payload_formatter`** | **Formatting method for MQTT message payloads. ​**​**Options: `json` (tree mode), `line` (table mode).** | **json** | +| `mqtt_max_message_size` | Maximum allowed MQTT message size (bytes). | 1048576 | + +## 3. Write Protocol + +* Line Protocol Syntax + +```JavaScript +[,=[,=]][ =[,=]] =[,=] [] +``` + +* Example + +```JavaScript +myMeasurement,tag1=value1,tag2=value2 attr1=value1,attr2=value2 fieldKey="fieldValue" 1556813561098000000 +``` + +![](/img/mqtt-table-en-2.png) + +## 4. Naming Conventions + +* Database Name + +The first segment of the MQTT topic (split by `/`) is used as the database name. + +```Properties +topic: stock/Legacy +databaseName: stock + + +topic: stock/Legacy/# +databaseName:stock +``` + +* Table Name + +The table name is derived from the `` in the line protocol. + +* Type Identifiers + +| Filed Value | IoTDB Data Type | +|--------------------------------------------------------------------| ----------------- | +| 1
1.12 | DOUBLE | +| 1`f`
1.12`f` | FLOAT | +| 1`i`
123`i` | INT64 | +| 1`u`
123`u` | INT64 | +| 1`i32`
123`i32` | INT32 | +| `"xxx"` | TEXT | +| `t`,`T`,`true`,`True`,`TRUE`
`f`,`F`,`false`,`False`,`FALSE` | BOOLEAN | + + +## 5. Coding Examples +The following is an example which a mqtt client send messages to IoTDB server. + + ```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +String DATABASE = "myMqttTest"; +connection.connect(); + +String payload = + "test1,tag1=t1,tag2=t2 attr3=a5,attr4=a4 field1=\"fieldValue1\",field2=1i,field3=1u 1"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 2"; +connection.publish(DATABASE, payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "# It's a remark\n " + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 6"; + connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); + Thread.sleep(10); + +//batch write example +payload = + "test1,tag1=t1,tag2=t2 field7=t,field8=T,field9=true 3 \n " + + "test1,tag1=t1,tag2=t2 field7=f,field8=F,field9=FALSE 4"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +//batch write example +payload = + "test1,tag1=t1,tag2=t2 attr1=a1,attr2=a2 field1=\"fieldValue1\",field2=1i,field3=1u 4 \n " + + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 5"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +connection.disconnect(); + ``` + + + +## 6. Customize your MQTT Message Format + +If you do not like the above Line format, you can customize your MQTT Message format by just writing several lines +of codes. An example can be found in [example/mqtt-customize](https://github.com/apache/iotdb/tree/master/example/mqtt-customize) project. + +Steps: +1. Create a java project, and add dependency: +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. Define your implementation which implements `org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` + e.g., + +```java +package org.apache.iotdb.mqtt.server; + +import io.netty.buffer.ByteBuf; +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +public class CustomizedLinePayloadFormatter implements PayloadFormatter { + + @Override + public List format(String topic, ByteBuf payload) { + // Suppose the payload is a line format + if (payload == null) { + return null; + } + + String line = payload.toString(StandardCharsets.UTF_8); + // parse data from the line and generate Messages and put them into List ret + List ret = new ArrayList<>(); + // this is just an example, so we just generate some Messages directly + for (int i = 0; i < 3; i++) { + long ts = i; + TableMessage message = new TableMessage(); + + // Parsing Database Name + message.setDatabase("db" + i); + + //Parsing Table Names + message.setTable("t" + i); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add("tag1" + i); + tagKeys.add("tag2" + i); + List tagValues = new ArrayList<>(); + tagValues.add("t_value1" + i); + tagValues.add("t_value2" + i); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add("attr1" + i); + attributeKeys.add("attr2" + i); + attributeValues.add("a_value1" + i); + attributeValues.add("a_value2" + i); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList("field1" + i, "field2" + i); + List dataTypes = Arrays.asList(TSDataType.FLOAT, TSDataType.FLOAT); + List values = Arrays.asList("4.0" + i, "5.0" + i); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + //// Parsing timestamp + message.setTimestamp(ts); + ret.add(message); + } + return ret; + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-system.properties as the following string: + return "CustomizedLine"; + } +} +``` +3. modify the file in `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter`: + clean the file and put your implementation class name into the file. + In this example, the content is: `org.apache.iotdb.mqtt.server.CustomizedLinePayloadFormatter` +4. compile your implementation as a jar file: `mvn package -DskipTests` + + +Then, in your server: +1. Create ${IOTDB_HOME}/ext/mqtt/ folder, and put the jar into this folder. +2. Update configuration to enable MQTT service. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. Set the value of `mqtt_payload_formatter` in `conf/iotdb-system.properties` as the value of getName() in your implementation + , in this example, the value is `CustomizedLine` +4. Launch the IoTDB server. +5. Now IoTDB will use your implementation to parse the MQTT message. + +More: the message format can be anything you want. For example, if it is a binary format, +just use `payload.forEachByte()` or `payload.array` to get bytes content. + +## 7. Caution + +To avoid compatibility issues caused by a default client_id, always explicitly supply a unique, non-empty client_id in every MQTT client. +Behavior varies when the client_id is missing or empty. Common examples: +1. Explicitly sending an empty string +• MQTTX: When client_id="", IoTDB silently discards the message. +• mosquitto_pub: When client_id="", IoTDB receives the message normally. +2. Omitting client_id entirely +• MQTTX: IoTDB accepts the message. +• mosquitto_pub: IoTDB rejects the connection. +Therefore, explicitly assigning a unique, non-empty client_id is the simplest way to eliminate these discrepancies and ensure reliable message delivery. \ No newline at end of file diff --git a/src/UserGuide/Master/Table/API/RestAPI-V1.md b/src/UserGuide/Master/Table/API/RestAPI-V1.md index babf3ac98..c3c6f6554 100644 --- a/src/UserGuide/Master/Table/API/RestAPI-V1.md +++ b/src/UserGuide/Master/Table/API/RestAPI-V1.md @@ -1,3 +1,6 @@ +--- +redirectTo: RestAPI-V1_apache.html +--- -# RestAPI V1 - -IoTDB's RESTful service can be used for querying, writing, and management operations. It uses the OpenAPI standard to define interfaces and generate frameworks. - -## 1. Enabling RESTful Service - -The RESTful service is disabled by default. To enable it, locate the `conf/iotdb-system.properties` file in the IoTDB installation directory, set `enable_rest_service` to `true`, and then restart the datanode process. - -```Properties -enable_rest_service=true -``` - -## 2. Authentication - -Except for the health check endpoint `/ping`, the RESTful service uses basic authentication. Each URL request must include the header `'Authorization':'Basic' + base64.encode(username + ':' + password)`. - -In the example, the username is `root` and the password is `root`. The corresponding Basic authentication header format is: - -```Properties -Authorization : Basic cm9vdDpyb290 -``` - -- If the username or password authentication fails, the following information is returned: - - - HTTP status code: 801 - - - Response body: - - ```JSON - {"code":801,"message":"WRONG_LOGIN_PASSWORD"} - ``` - -- If the `Authorization` header is not set, the following information is returned: - - - HTTP status code: 800 - - - Response body: - - ```JSON - {"code":800,"message":"INIT_AUTH_ERROR"} - ``` - -## 3. Interface Definitions - -### 3.1 Ping - -The `/ping` endpoint can be used for online service health checks. - -- Request Method: GET - -- Request Path: `http://ip:port/ping` - -- Example Request: - - ```Bash - curl http://127.0.0.1:18080/ping - ``` - -- HTTP Status Codes: - - - `200`: The service is working normally and can accept external requests. - - - `503`: The service is experiencing issues and cannot accept external requests. - - | Parameter Name | Type | Description | - | :------------- | :------ | :--------------- | - | code | integer | Status Code | - | message | string | Code Information | - -- Response Example: - - - When the HTTP status code is `200`: - - ```JSON - { "code": 200, "message": "SUCCESS_STATUS"} - ``` - - - When the HTTP status code is `503`: - - ```JSON - { "code": 500, "message": "thrift service is unavailable"} - ``` - -**Note**: The `/ping` endpoint does not require authentication. - -### 3.2 Query Interface - -- Request Path: `/rest/table/v1/query` - -- Request Method: POST - -- Request Format: - - - Header: `application/json` - - - Request Parameters: - - | Parameter Name | Type | Required | Description | - | :------------- | :----- | :------- | :----------------------------------------------------------- | - | `database` | string | Yes | Database name | - | `sql` | string | Yes | SQL query | - | `row_limit` | int | No | Maximum number of rows to return in a single query. If not set, the default value from the configuration file (`rest_query_default_row_size_limit`) is used. If the result set exceeds this limit, status code `411` is returned. | - -- Response Format: - - | Parameter Name | Type | Description | - | :------------- | :---- | :----------------------------------------------------------- | - | `column_names` | array | Column names | - | `data_types` | array | Data types of each column | - | `values` | array | A 2D array where the first dimension represents rows, and the second dimension represents columns. Each element corresponds to a column, with the same length as `column_names`. | - -- Example Request: - - ```Bash - curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","sql":"select s1,s2,s3 from test_table"}' http://127.0.0.1:18080/rest/table/v1/query - ``` - -- Example Response: - - ```JSON - { - "column_names": [ - "s1", - "s2", - "s3" - ], - "data_types": [ - "STRING", - "BOOLEAN", - "INT32" - ], - "values": [ - [ - "a11", - true, - 2024 - ], - [ - "a11", - false, - 2025 - ] - ] - } - ``` - -### 3.3 Non-Query Interface - -- Request Path: `/rest/table/v1/nonQuery` - -- Request Method: POST - -- Request Format: - - - Header: `application/json` - - - Request Parameters: - - | Parameter Name | Type | Required | Description | - | :------------- | :----- | :------- | :------------ | - | `sql` | string | Yes | SQL statement | - | `database` | string | No | Database name | - -- Response Format: - - | Parameter Name | Type | Description | - | :------------- | :------ | :---------- | - | `code` | integer | Status code | - | `message` | string | Message | - -- Example Requests: - - - Create a database: - - ```Bash - curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"create database test","database":""}' http://127.0.0.1:18080/rest/table/v1/nonQuery - ``` - - - Create a table `test_table` in the `test` database: - - ```Bash - curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"CREATE TABLE table1 (time TIMESTAMP TIME,region STRING TAG,plant_id STRING TAG,device_id STRING TAG,model_id STRING ATTRIBUTE,maintenance STRING ATTRIBUTE,temperature FLOAT FIELD,humidity FLOAT FIELD,status Boolean FIELD,arrival_time TIMESTAMP FIELD) WITH (TTL=31536000000)","database":"test"}' http://127.0.0.1:18080/rest/table/v1/nonQuery - ``` - -- Example Response: - - ```JSON - { - "code": 200, - "message": "SUCCESS_STATUS" - } - ``` - -### 3.4 Batch Write Interface - -- Request Path: `/rest/table/v1/insertTablet` - -- Request Method: POST - -- Request Format: - - - Header: `application/json` - - - Request Parameters: - - | Parameter Name | Type | Required | Description | - | :------------------ | :----- | :------- | :----------------------------------------------------------- | - | `database` | string | Yes | Database name | - | `table` | string | Yes | Table name | - | `column_names` | array | Yes | Column names | - | `column_categories` | array | Yes | Column categories (`TAG`, `FIELD`, `ATTRIBUTE`) | - | `data_types` | array | Yes | Data types | - | `timestamps` | array | Yes | Timestamp column | - | `values` | array | Yes | Value columns. Each column's values can be `null`. A 2D array where the first dimension corresponds to timestamps, and the second dimension corresponds to columns. | - -- Response Format: - - | Parameter Name | Type | Description | - | :------------- | :------ | :---------- | - | `code` | integer | Status code | - | `message` | string | Message | - -- Example Request: - - ```Bash - curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","column_categories":["TAG","FIELD","FIELD"],"timestamps":[1739702535000,1739789055000],"column_names":["s1","s2","s3"],"data_types":["STRING","BOOLEAN","INT32"],"values":[["a11",true,2024],["a11",false,2025]],"table":"test_table"}' http://127.0.0.1:18080/rest/table/v1/insertTablet - ``` - -- Example Response: - - ```JSON - { - "code": 200, - "message": "SUCCESS_STATUS" - } - ``` - -## 4. Configuration - -The configuration file is located in `iotdb-system.properties`. - -- Set `enable_rest_service` to `true` to enable the module, or `false` to disable it. The default value is `false`. - - ```Properties - enable_rest_service=true - ``` - -- Only effective when `enable_rest_service=true`. Set `rest_service_port` to a number (1025~65535) to customize the REST service socket port. The default value is `18080`. - - ```Properties - rest_service_port=18080 - ``` - -- Set `enable_swagger` to `true` to enable Swagger for displaying REST interface information, or `false` to disable it. The default value is `false`. - - ```Properties - enable_swagger=false - ``` - -- The maximum number of rows that can be returned in a single query. If the result set exceeds this limit, only the rows within the limit will be returned, and status code `411` will be returned. - - ```Properties - rest_query_default_row_size_limit=10000 - ``` - -- Expiration time for caching client login information (used to speed up user authentication, in seconds, default is 8 hours). - - ```Properties - cache_expire_in_seconds=28800 - ``` - -- Maximum number of users stored in the cache (default is 100). - - ```Properties - cache_max_num=100 - ``` - -- Initial cache capacity (default is 10). - - ```Properties - cache_init_num=10 - ``` - -- Whether to enable SSL configuration for the REST service. Set `enable_https` to `true` to enable it, or `false` to disable it. The default value is `false`. - - ```Properties - enable_https=false - ``` - -- Path to the `keyStore` (optional). - - ```Properties - key_store_path= - ``` - -- Password for the `keyStore` (optional). - - ```Properties - key_store_pwd= - ``` - -- Path to the `trustStore` (optional). - - ```Properties - trust_store_path="" - ``` - -- Password for the `trustStore` (optional). - - ```Properties - trust_store_pwd="" - ``` - -- SSL timeout time, in seconds. - - ```Properties - idle_timeout_in_seconds=5000 - ``` \ No newline at end of file +--> \ No newline at end of file diff --git a/src/UserGuide/Master/Table/API/RestAPI-V1_apache.md b/src/UserGuide/Master/Table/API/RestAPI-V1_apache.md new file mode 100644 index 000000000..babf3ac98 --- /dev/null +++ b/src/UserGuide/Master/Table/API/RestAPI-V1_apache.md @@ -0,0 +1,338 @@ + +# RestAPI V1 + +IoTDB's RESTful service can be used for querying, writing, and management operations. It uses the OpenAPI standard to define interfaces and generate frameworks. + +## 1. Enabling RESTful Service + +The RESTful service is disabled by default. To enable it, locate the `conf/iotdb-system.properties` file in the IoTDB installation directory, set `enable_rest_service` to `true`, and then restart the datanode process. + +```Properties +enable_rest_service=true +``` + +## 2. Authentication + +Except for the health check endpoint `/ping`, the RESTful service uses basic authentication. Each URL request must include the header `'Authorization':'Basic' + base64.encode(username + ':' + password)`. + +In the example, the username is `root` and the password is `root`. The corresponding Basic authentication header format is: + +```Properties +Authorization : Basic cm9vdDpyb290 +``` + +- If the username or password authentication fails, the following information is returned: + + - HTTP status code: 801 + + - Response body: + + ```JSON + {"code":801,"message":"WRONG_LOGIN_PASSWORD"} + ``` + +- If the `Authorization` header is not set, the following information is returned: + + - HTTP status code: 800 + + - Response body: + + ```JSON + {"code":800,"message":"INIT_AUTH_ERROR"} + ``` + +## 3. Interface Definitions + +### 3.1 Ping + +The `/ping` endpoint can be used for online service health checks. + +- Request Method: GET + +- Request Path: `http://ip:port/ping` + +- Example Request: + + ```Bash + curl http://127.0.0.1:18080/ping + ``` + +- HTTP Status Codes: + + - `200`: The service is working normally and can accept external requests. + + - `503`: The service is experiencing issues and cannot accept external requests. + + | Parameter Name | Type | Description | + | :------------- | :------ | :--------------- | + | code | integer | Status Code | + | message | string | Code Information | + +- Response Example: + + - When the HTTP status code is `200`: + + ```JSON + { "code": 200, "message": "SUCCESS_STATUS"} + ``` + + - When the HTTP status code is `503`: + + ```JSON + { "code": 500, "message": "thrift service is unavailable"} + ``` + +**Note**: The `/ping` endpoint does not require authentication. + +### 3.2 Query Interface + +- Request Path: `/rest/table/v1/query` + +- Request Method: POST + +- Request Format: + + - Header: `application/json` + + - Request Parameters: + + | Parameter Name | Type | Required | Description | + | :------------- | :----- | :------- | :----------------------------------------------------------- | + | `database` | string | Yes | Database name | + | `sql` | string | Yes | SQL query | + | `row_limit` | int | No | Maximum number of rows to return in a single query. If not set, the default value from the configuration file (`rest_query_default_row_size_limit`) is used. If the result set exceeds this limit, status code `411` is returned. | + +- Response Format: + + | Parameter Name | Type | Description | + | :------------- | :---- | :----------------------------------------------------------- | + | `column_names` | array | Column names | + | `data_types` | array | Data types of each column | + | `values` | array | A 2D array where the first dimension represents rows, and the second dimension represents columns. Each element corresponds to a column, with the same length as `column_names`. | + +- Example Request: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","sql":"select s1,s2,s3 from test_table"}' http://127.0.0.1:18080/rest/table/v1/query + ``` + +- Example Response: + + ```JSON + { + "column_names": [ + "s1", + "s2", + "s3" + ], + "data_types": [ + "STRING", + "BOOLEAN", + "INT32" + ], + "values": [ + [ + "a11", + true, + 2024 + ], + [ + "a11", + false, + 2025 + ] + ] + } + ``` + +### 3.3 Non-Query Interface + +- Request Path: `/rest/table/v1/nonQuery` + +- Request Method: POST + +- Request Format: + + - Header: `application/json` + + - Request Parameters: + + | Parameter Name | Type | Required | Description | + | :------------- | :----- | :------- | :------------ | + | `sql` | string | Yes | SQL statement | + | `database` | string | No | Database name | + +- Response Format: + + | Parameter Name | Type | Description | + | :------------- | :------ | :---------- | + | `code` | integer | Status code | + | `message` | string | Message | + +- Example Requests: + + - Create a database: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"create database test","database":""}' http://127.0.0.1:18080/rest/table/v1/nonQuery + ``` + + - Create a table `test_table` in the `test` database: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"CREATE TABLE table1 (time TIMESTAMP TIME,region STRING TAG,plant_id STRING TAG,device_id STRING TAG,model_id STRING ATTRIBUTE,maintenance STRING ATTRIBUTE,temperature FLOAT FIELD,humidity FLOAT FIELD,status Boolean FIELD,arrival_time TIMESTAMP FIELD) WITH (TTL=31536000000)","database":"test"}' http://127.0.0.1:18080/rest/table/v1/nonQuery + ``` + +- Example Response: + + ```JSON + { + "code": 200, + "message": "SUCCESS_STATUS" + } + ``` + +### 3.4 Batch Write Interface + +- Request Path: `/rest/table/v1/insertTablet` + +- Request Method: POST + +- Request Format: + + - Header: `application/json` + + - Request Parameters: + + | Parameter Name | Type | Required | Description | + | :------------------ | :----- | :------- | :----------------------------------------------------------- | + | `database` | string | Yes | Database name | + | `table` | string | Yes | Table name | + | `column_names` | array | Yes | Column names | + | `column_categories` | array | Yes | Column categories (`TAG`, `FIELD`, `ATTRIBUTE`) | + | `data_types` | array | Yes | Data types | + | `timestamps` | array | Yes | Timestamp column | + | `values` | array | Yes | Value columns. Each column's values can be `null`. A 2D array where the first dimension corresponds to timestamps, and the second dimension corresponds to columns. | + +- Response Format: + + | Parameter Name | Type | Description | + | :------------- | :------ | :---------- | + | `code` | integer | Status code | + | `message` | string | Message | + +- Example Request: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","column_categories":["TAG","FIELD","FIELD"],"timestamps":[1739702535000,1739789055000],"column_names":["s1","s2","s3"],"data_types":["STRING","BOOLEAN","INT32"],"values":[["a11",true,2024],["a11",false,2025]],"table":"test_table"}' http://127.0.0.1:18080/rest/table/v1/insertTablet + ``` + +- Example Response: + + ```JSON + { + "code": 200, + "message": "SUCCESS_STATUS" + } + ``` + +## 4. Configuration + +The configuration file is located in `iotdb-system.properties`. + +- Set `enable_rest_service` to `true` to enable the module, or `false` to disable it. The default value is `false`. + + ```Properties + enable_rest_service=true + ``` + +- Only effective when `enable_rest_service=true`. Set `rest_service_port` to a number (1025~65535) to customize the REST service socket port. The default value is `18080`. + + ```Properties + rest_service_port=18080 + ``` + +- Set `enable_swagger` to `true` to enable Swagger for displaying REST interface information, or `false` to disable it. The default value is `false`. + + ```Properties + enable_swagger=false + ``` + +- The maximum number of rows that can be returned in a single query. If the result set exceeds this limit, only the rows within the limit will be returned, and status code `411` will be returned. + + ```Properties + rest_query_default_row_size_limit=10000 + ``` + +- Expiration time for caching client login information (used to speed up user authentication, in seconds, default is 8 hours). + + ```Properties + cache_expire_in_seconds=28800 + ``` + +- Maximum number of users stored in the cache (default is 100). + + ```Properties + cache_max_num=100 + ``` + +- Initial cache capacity (default is 10). + + ```Properties + cache_init_num=10 + ``` + +- Whether to enable SSL configuration for the REST service. Set `enable_https` to `true` to enable it, or `false` to disable it. The default value is `false`. + + ```Properties + enable_https=false + ``` + +- Path to the `keyStore` (optional). + + ```Properties + key_store_path= + ``` + +- Password for the `keyStore` (optional). + + ```Properties + key_store_pwd= + ``` + +- Path to the `trustStore` (optional). + + ```Properties + trust_store_path="" + ``` + +- Password for the `trustStore` (optional). + + ```Properties + trust_store_pwd="" + ``` + +- SSL timeout time, in seconds. + + ```Properties + idle_timeout_in_seconds=5000 + ``` \ No newline at end of file diff --git a/src/UserGuide/Master/Table/API/RestAPI-V1_timecho.md b/src/UserGuide/Master/Table/API/RestAPI-V1_timecho.md new file mode 100644 index 000000000..fffe6872d --- /dev/null +++ b/src/UserGuide/Master/Table/API/RestAPI-V1_timecho.md @@ -0,0 +1,340 @@ + +# RestAPI V1 + +IoTDB's RESTful service can be used for querying, writing, and management operations. It uses the OpenAPI standard to define interfaces and generate frameworks. + +Note: As of version V2.0.8, the TimechoDB installation package does not include the REST service JAR file by default. Please contact the Timecho team to obtain the corresponding JAR file before using this service, and place it in the `timechodb_home/lib` or `timechodb_home/ext/external_service` directory. + +## 1. Enabling RESTful Service + +The RESTful service is disabled by default. To enable it, locate the `conf/iotdb-system.properties` file in the IoTDB installation directory, set `enable_rest_service` to `true`, and then restart the datanode process. + +```Properties +enable_rest_service=true +``` + +## 2. Authentication + +Except for the health check endpoint `/ping`, the RESTful service uses basic authentication. Each URL request must include the header `'Authorization':'Basic' + base64.encode(username + ':' + password)`. + +In the example, the username is `root` and the password is `root`. The corresponding Basic authentication header format is: + +```Properties +Authorization : Basic cm9vdDpyb290 +``` + +- If the username or password authentication fails, the following information is returned: + + - HTTP status code: 801 + + - Response body: + + ```JSON + {"code":801,"message":"WRONG_LOGIN_PASSWORD"} + ``` + +- If the `Authorization` header is not set, the following information is returned: + + - HTTP status code: 800 + + - Response body: + + ```JSON + {"code":800,"message":"INIT_AUTH_ERROR"} + ``` + +## 3. Interface Definitions + +### 3.1 Ping + +The `/ping` endpoint can be used for online service health checks. + +- Request Method: GET + +- Request Path: `http://ip:port/ping` + +- Example Request: + + ```Bash + curl http://127.0.0.1:18080/ping + ``` + +- HTTP Status Codes: + + - `200`: The service is working normally and can accept external requests. + + - `503`: The service is experiencing issues and cannot accept external requests. + + | Parameter Name | Type | Description | + | :------------- | :------ | :--------------- | + | code | integer | Status Code | + | message | string | Code Information | + +- Response Example: + + - When the HTTP status code is `200`: + + ```JSON + { "code": 200, "message": "SUCCESS_STATUS"} + ``` + + - When the HTTP status code is `503`: + + ```JSON + { "code": 500, "message": "thrift service is unavailable"} + ``` + +**Note**: The `/ping` endpoint does not require authentication. + +### 3.2 Query Interface + +- Request Path: `/rest/table/v1/query` + +- Request Method: POST + +- Request Format: + + - Header: `application/json` + + - Request Parameters: + + | Parameter Name | Type | Required | Description | + | :------------- | :----- | :------- | :----------------------------------------------------------- | + | `database` | string | Yes | Database name | + | `sql` | string | Yes | SQL query | + | `row_limit` | int | No | Maximum number of rows to return in a single query. If not set, the default value from the configuration file (`rest_query_default_row_size_limit`) is used. If the result set exceeds this limit, status code `411` is returned. | + +- Response Format: + + | Parameter Name | Type | Description | + | :------------- | :---- | :----------------------------------------------------------- | + | `column_names` | array | Column names | + | `data_types` | array | Data types of each column | + | `values` | array | A 2D array where the first dimension represents rows, and the second dimension represents columns. Each element corresponds to a column, with the same length as `column_names`. | + +- Example Request: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","sql":"select s1,s2,s3 from test_table"}' http://127.0.0.1:18080/rest/table/v1/query + ``` + +- Example Response: + + ```JSON + { + "column_names": [ + "s1", + "s2", + "s3" + ], + "data_types": [ + "STRING", + "BOOLEAN", + "INT32" + ], + "values": [ + [ + "a11", + true, + 2024 + ], + [ + "a11", + false, + 2025 + ] + ] + } + ``` + +### 3.3 Non-Query Interface + +- Request Path: `/rest/table/v1/nonQuery` + +- Request Method: POST + +- Request Format: + + - Header: `application/json` + + - Request Parameters: + + | Parameter Name | Type | Required | Description | + | :------------- | :----- | :------- | :------------ | + | `sql` | string | Yes | SQL statement | + | `database` | string | No | Database name | + +- Response Format: + + | Parameter Name | Type | Description | + | :------------- | :------ | :---------- | + | `code` | integer | Status code | + | `message` | string | Message | + +- Example Requests: + + - Create a database: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"create database test","database":""}' http://127.0.0.1:18080/rest/table/v1/nonQuery + ``` + + - Create a table `test_table` in the `test` database: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"CREATE TABLE table1 (time TIMESTAMP TIME,region STRING TAG,plant_id STRING TAG,device_id STRING TAG,model_id STRING ATTRIBUTE,maintenance STRING ATTRIBUTE,temperature FLOAT FIELD,humidity FLOAT FIELD,status Boolean FIELD,arrival_time TIMESTAMP FIELD) WITH (TTL=31536000000)","database":"test"}' http://127.0.0.1:18080/rest/table/v1/nonQuery + ``` + +- Example Response: + + ```JSON + { + "code": 200, + "message": "SUCCESS_STATUS" + } + ``` + +### 3.4 Batch Write Interface + +- Request Path: `/rest/table/v1/insertTablet` + +- Request Method: POST + +- Request Format: + + - Header: `application/json` + + - Request Parameters: + + | Parameter Name | Type | Required | Description | + | :------------------ | :----- | :------- | :----------------------------------------------------------- | + | `database` | string | Yes | Database name | + | `table` | string | Yes | Table name | + | `column_names` | array | Yes | Column names | + | `column_categories` | array | Yes | Column categories (`TAG`, `FIELD`, `ATTRIBUTE`) | + | `data_types` | array | Yes | Data types | + | `timestamps` | array | Yes | Timestamp column | + | `values` | array | Yes | Value columns. Each column's values can be `null`. A 2D array where the first dimension corresponds to timestamps, and the second dimension corresponds to columns. | + +- Response Format: + + | Parameter Name | Type | Description | + | :------------- | :------ | :---------- | + | `code` | integer | Status code | + | `message` | string | Message | + +- Example Request: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","column_categories":["TAG","FIELD","FIELD"],"timestamps":[1739702535000,1739789055000],"column_names":["s1","s2","s3"],"data_types":["STRING","BOOLEAN","INT32"],"values":[["a11",true,2024],["a11",false,2025]],"table":"test_table"}' http://127.0.0.1:18080/rest/table/v1/insertTablet + ``` + +- Example Response: + + ```JSON + { + "code": 200, + "message": "SUCCESS_STATUS" + } + ``` + +## 4. Configuration + +The configuration file is located in `iotdb-system.properties`. + +- Set `enable_rest_service` to `true` to enable the module, or `false` to disable it. The default value is `false`. + + ```Properties + enable_rest_service=true + ``` + +- Only effective when `enable_rest_service=true`. Set `rest_service_port` to a number (1025~65535) to customize the REST service socket port. The default value is `18080`. + + ```Properties + rest_service_port=18080 + ``` + +- Set `enable_swagger` to `true` to enable Swagger for displaying REST interface information, or `false` to disable it. The default value is `false`. + + ```Properties + enable_swagger=false + ``` + +- The maximum number of rows that can be returned in a single query. If the result set exceeds this limit, only the rows within the limit will be returned, and status code `411` will be returned. + + ```Properties + rest_query_default_row_size_limit=10000 + ``` + +- Expiration time for caching client login information (used to speed up user authentication, in seconds, default is 8 hours). + + ```Properties + cache_expire_in_seconds=28800 + ``` + +- Maximum number of users stored in the cache (default is 100). + + ```Properties + cache_max_num=100 + ``` + +- Initial cache capacity (default is 10). + + ```Properties + cache_init_num=10 + ``` + +- Whether to enable SSL configuration for the REST service. Set `enable_https` to `true` to enable it, or `false` to disable it. The default value is `false`. + + ```Properties + enable_https=false + ``` + +- Path to the `keyStore` (optional). + + ```Properties + key_store_path= + ``` + +- Password for the `keyStore` (optional). + + ```Properties + key_store_pwd= + ``` + +- Path to the `trustStore` (optional). + + ```Properties + trust_store_path="" + ``` + +- Password for the `trustStore` (optional). + + ```Properties + trust_store_pwd="" + ``` + +- SSL timeout time, in seconds. + + ```Properties + idle_timeout_in_seconds=5000 + ``` \ No newline at end of file diff --git a/src/UserGuide/Master/Table/Basic-Concept/Write-Updata-Data_apache.md b/src/UserGuide/Master/Table/Basic-Concept/Write-Updata-Data_apache.md index 193fa718b..d138f2a27 100644 --- a/src/UserGuide/Master/Table/Basic-Concept/Write-Updata-Data_apache.md +++ b/src/UserGuide/Master/Table/Basic-Concept/Write-Updata-Data_apache.md @@ -294,7 +294,7 @@ It costs 0.014s * **TAG** columns are optional. * The number of specified columns can be less than that of the target table. Missing columns will be automatically filled with `NULL` values. * For Java applications, the `INSERT INTO QUERY` statement can be executed using the [executeNonQueryStatement](../API/Programming-Java-Native-API_apache.md#_3-1-itablesession-interface) method. -* For REST API access, the `INSERT INTO QUERY` statement can be executed via the [/rest/table/v1/nonQuery](../API/RestAPI-V1.md#_3-3-non-query-interface) endpoint. +* For REST API access, the `INSERT INTO QUERY` statement can be executed via the [/rest/table/v1/nonQuery](../API/RestAPI-V1_apache.md#_3-3-non-query-interface) endpoint. * `INSERT INTO QUERY` does **not** support the `EXPLAIN` and `EXPLAIN ANALYZE` commands. * To execute the query write-back statement successfully, users must have the following permissions: * The `SELECT` permission on the source tables involved in the query. diff --git a/src/UserGuide/Master/Table/Basic-Concept/Write-Updata-Data_timecho.md b/src/UserGuide/Master/Table/Basic-Concept/Write-Updata-Data_timecho.md index ef146a655..8da2d8545 100644 --- a/src/UserGuide/Master/Table/Basic-Concept/Write-Updata-Data_timecho.md +++ b/src/UserGuide/Master/Table/Basic-Concept/Write-Updata-Data_timecho.md @@ -294,7 +294,7 @@ It costs 0.014s * **TAG** columns are optional. * The number of specified columns can be less than that of the target table. Missing columns will be automatically filled with `NULL` values. * For Java applications, the `INSERT INTO QUERY` statement can be executed using the [executeNonQueryStatement](../API/Programming-Java-Native-API_timecho.md#_3-1-itablesession-interface) method. -* For REST API access, the `INSERT INTO QUERY` statement can be executed via the [/rest/table/v1/nonQuery](../API/RestAPI-V1.md#_3-3-non-query-interface) endpoint. +* For REST API access, the `INSERT INTO QUERY` statement can be executed via the [/rest/table/v1/nonQuery](../API/RestAPI-V1_timecho.md#_3-3-non-query-interface) endpoint. * `INSERT INTO QUERY` does **not** support the `EXPLAIN` and `EXPLAIN ANALYZE` commands. * To execute the query write-back statement successfully, users must have the following permissions: * The `SELECT` permission on the source tables involved in the query. diff --git a/src/UserGuide/Master/Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md b/src/UserGuide/Master/Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md index 3c1742408..a0968c919 100644 --- a/src/UserGuide/Master/Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md +++ b/src/UserGuide/Master/Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md @@ -43,4 +43,6 @@ After unpacking the installation package（`iotdb-enterprise-{version}-bin.zip` | NOTICE | File | Open-source notice file. | | README_ZH.md | File | User manual (Chinese version). | | README.md | File | User manual (English version). | -| RELEASE_NOTES.md | File | Release notes. | \ No newline at end of file +| RELEASE_NOTES.md | File | Release notes. | + +Note: As of version V2.0.8, the TimechoDB installation package does not include the MQTT service and REST service JAR files by default. If you need to use them, please contact the Timecho team to obtain them. \ No newline at end of file diff --git a/src/UserGuide/Master/Tree/API/Programming-MQTT.md b/src/UserGuide/Master/Tree/API/Programming-MQTT.md index f99e0cdda..0a6a38616 100644 --- a/src/UserGuide/Master/Tree/API/Programming-MQTT.md +++ b/src/UserGuide/Master/Tree/API/Programming-MQTT.md @@ -1,3 +1,6 @@ +--- +redirectTo: Programming-MQTT_apache.html +--- -# MQTT Protocol - -## 1. Overview - -MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for IoT and low-bandwidth environments. It operates on a Publish/Subscribe (Pub/Sub) model, enabling efficient and reliable bidirectional communication between devices. Its core objectives are low power consumption, minimal bandwidth usage, and high real-time performance, making it ideal for unstable networks or resource-constrained scenarios (e.g., sensors, mobile devices). - -IoTDB provides deep integration with the MQTT protocol, fully compliant with MQTT v3.1 (OASIS International Standard). The IoTDB server includes a built-in high-performance MQTT Broker module, eliminating the need for third-party middleware. Devices can directly write time-series data into the IoTDB storage engine via MQTT messages. - - - - -## 2. Built-in MQTT Service -The Built-in MQTT Service provide the ability of direct connection to IoTDB through MQTT. It listen the publish messages from MQTT clients - and then write the data into storage immediately. -The MQTT topic corresponds to IoTDB timeseries. -The messages payload can be format to events by `PayloadFormatter` which loaded by java SPI, and the default implementation is `JSONPayloadFormatter`. -The default `json` formatter support two json format and its json array. The following is an MQTT message payload example: - -```json - { - "device":"root.sg.d1", - "timestamp":1586076045524, - "measurements":["s1","s2"], - "values":[0.530635,0.530635] - } -``` -or -```json - { - "device":"root.sg.d1", - "timestamps":[1586076045524,1586076065526], - "measurements":["s1","s2"], - "values":[[0.530635,0.530635], [0.530655,0.530695]] - } -``` -or json array of the above two. - - - -## 3. MQTT Configurations -The IoTDB MQTT service load configurations from `${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties` by default. - -Configurations are as follows: - -| **Property** | **Description** | **Default** | -| ------------------------ | ---------------------------------------------------------------------------------------------------------------------- | ------------------- | -| `enable_mqtt_service` | Enable/ disable the MQTT service. | FALSE | -| `mqtt_host` | Host address bound to the MQTT service. | 127.0.0.1 | -| `mqtt_port` | Port bound to the MQTT service. | 1883 | -| `mqtt_handler_pool_size` | Thread pool size for processing MQTT messages. | 1 | -| **`mqtt_payload_formatter`** | **Formatting method for MQTT message payloads. ​**​**Options: `json` (tree model), `line` (table model).** | **json** | -| `mqtt_max_message_size` | Maximum allowed MQTT message size (bytes). | 1048576 | - -## 4. Coding Examples -The following is an example which a mqtt client send messages to IoTDB server. - -```java -MQTT mqtt = new MQTT(); -mqtt.setHost("127.0.0.1", 1883); -mqtt.setUserName("root"); -mqtt.setPassword("root"); - -BlockingConnection connection = mqtt.blockingConnection(); -connection.connect(); - -Random random = new Random(); -for (int i = 0; i < 10; i++) { - String payload = String.format("{\n" + - "\"device\":\"root.sg.d1\",\n" + - "\"timestamp\":%d,\n" + - "\"measurements\":[\"s1\"],\n" + - "\"values\":[%f]\n" + - "}", System.currentTimeMillis(), random.nextDouble()); - - connection.publish("root.sg.d1.s1", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -} - -connection.disconnect(); - -``` - -## 5. Customize your MQTT Message Format - -In a production environment, each device typically has its own MQTT client, and the message formats of these clients have been pre-defined. If communication is to be carried out in accordance with the MQTT message format supported by IoTDB, a comprehensive upgrade and transformation of all existing clients would be required, which would undoubtedly incur significant costs. However, we can easily achieve customization of the MQTT message format through simple programming means, without the need to modify the clients. -An example can be found in [example/mqtt-customize](https://github.com/apache/iotdb/tree/rc/2.0.1/example/mqtt-customize) project. - -Assuming the MQTT client sends the following message format: -```json - { - "time":1586076045523, - "deviceID":"car_1", - "deviceType":"Gasoline car​​", - "point":"Fuel level​​", - "value":10.0 -} -``` -Or in the form of an array of JSON: -```java -[ - { - "time":1586076045523, - "deviceID":"car_1", - "deviceType":"Gasoline car​​", - "point":"Fuel level", - "value":10.0 - }, - { - "time":1586076045524, - "deviceID":"car_2", - "deviceType":"NEV(new enegry vehicle)", - "point":"Speed", - "value":80.0 - } -] -``` - -Then you can set up the custom MQTT message format through the following steps: - -1. Create a java project, and add dependency: -```xml - - org.apache.iotdb - iotdb-server - 2.0.4-SNAPSHOT - -``` -2. Define your implementation which implements `org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` -e.g., - -```java -package org.apache.iotdb.mqtt.server; - -import org.apache.iotdb.db.protocol.mqtt.Message; -import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; -import org.apache.iotdb.db.protocol.mqtt.TableMessage; - -import com.google.common.collect.Lists; -import com.google.gson.Gson; -import com.google.gson.GsonBuilder; -import com.google.gson.JsonArray; -import com.google.gson.JsonElement; -import com.google.gson.JsonObject; -import com.google.gson.JsonParseException; -import io.netty.buffer.ByteBuf; -import org.apache.commons.lang3.NotImplementedException; -import org.apache.tsfile.enums.TSDataType; - -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -/** - * The Customized JSON payload formatter. one json format supported: { "time":1586076045523, - * "deviceID":"car_1", "deviceType":"NEV", "point":"Speed", "value":80.0 } - */ -public class CustomizedJsonPayloadFormatter implements PayloadFormatter { - private static final String JSON_KEY_TIME = "time"; - private static final String JSON_KEY_DEVICEID = "deviceID"; - private static final String JSON_KEY_DEVICETYPE = "deviceType"; - private static final String JSON_KEY_POINT = "point"; - private static final String JSON_KEY_VALUE = "value"; - private static final Gson GSON = new GsonBuilder().create(); - - @Override - public List format(String topic, ByteBuf payload) { - if (payload == null) { - return new ArrayList<>(); - } - String txt = payload.toString(StandardCharsets.UTF_8); - JsonElement jsonElement = GSON.fromJson(txt, JsonElement.class); - if (jsonElement.isJsonObject()) { - JsonObject jsonObject = jsonElement.getAsJsonObject(); - return formatTableRow(topic, jsonObject); - } else if (jsonElement.isJsonArray()) { - JsonArray jsonArray = jsonElement.getAsJsonArray(); - List messages = new ArrayList<>(); - for (JsonElement element : jsonArray) { - JsonObject jsonObject = element.getAsJsonObject(); - messages.addAll(formatTableRow(topic, jsonObject)); - } - return messages; - } - throw new JsonParseException("payload is invalidate"); - } - - @Override - @Deprecated - public List format(ByteBuf payload) { - throw new NotImplementedException(); - } - - private List formatTableRow(String topic, JsonObject jsonObject) { - TableMessage message = new TableMessage(); - String database = !topic.contains("/") ? topic : topic.substring(0, topic.indexOf("/")); - String table = "test_table"; - - // Parsing Database Name - message.setDatabase((database)); - - // Parsing Table Name - message.setTable(table); - - // Parsing Tags - List tagKeys = new ArrayList<>(); - tagKeys.add(JSON_KEY_DEVICEID); - List tagValues = new ArrayList<>(); - tagValues.add(jsonObject.get(JSON_KEY_DEVICEID).getAsString()); - message.setTagKeys(tagKeys); - message.setTagValues(tagValues); - - // Parsing Attributes - List attributeKeys = new ArrayList<>(); - List attributeValues = new ArrayList<>(); - attributeKeys.add(JSON_KEY_DEVICETYPE); - attributeValues.add(jsonObject.get(JSON_KEY_DEVICETYPE).getAsString()); - message.setAttributeKeys(attributeKeys); - message.setAttributeValues(attributeValues); - - // Parsing Fields - List fields = Arrays.asList(JSON_KEY_POINT); - List dataTypes = Arrays.asList(TSDataType.FLOAT); - List values = Arrays.asList(jsonObject.get(JSON_KEY_VALUE).getAsFloat()); - message.setFields(fields); - message.setDataTypes(dataTypes); - message.setValues(values); - - // Parsing timestamp - message.setTimestamp(jsonObject.get(JSON_KEY_TIME).getAsLong()); - return Lists.newArrayList(message); - } - - @Override - public String getName() { - // set the value of mqtt_payload_formatter in iotdb-common.properties as the following string: - return "CustomizedJson2Table"; - } - - @Override - public String getType() { - return PayloadFormatter.TABLE_TYPE; - } -} -``` -3. modify the file in `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter`: - clean the file and put your implementation class name into the file. - In this example, the content is: `org.apache.iotdb.mqtt.server.CustomizedJsonPayloadFormatter` -4. compile your implementation as a jar file: `mvn package -DskipTests` - - -Then, in your server: -1. Create ${IOTDB_HOME}/ext/mqtt/ folder, and put the jar into this folder. -2. Update configuration to enable MQTT service. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) -3. Set the value of `mqtt_payload_formatter` in `conf/iotdb-system.properties` as the value of getName() in your implementation - , in this example, the value is `CustomizedJson2Table` -4. Launch the IoTDB server. -5. Now IoTDB will use your implementation to parse the MQTT message. - -More: the message format can be anything you want. For example, if it is a binary format, -just use `payload.forEachByte()` or `payload.array` to get bytes content. - - -## 6. Caution - -To avoid compatibility issues caused by a default client_id, always explicitly supply a unique, non-empty client_id in every MQTT client. -Behavior varies when the client_id is missing or empty. Common examples: -1. Explicitly sending an empty string - • MQTTX: When client_id="", IoTDB silently discards the message. - • mosquitto_pub: When client_id="", IoTDB receives the message normally. -2. Omitting client_id entirely - • MQTTX: IoTDB accepts the message. - • mosquitto_pub: IoTDB rejects the connection. - Therefore, explicitly assigning a unique, non-empty client_id is the simplest way to eliminate these discrepancies and ensure reliable message delivery. +--> \ No newline at end of file diff --git a/src/UserGuide/Master/Tree/API/Programming-MQTT_apache.md b/src/UserGuide/Master/Tree/API/Programming-MQTT_apache.md new file mode 100644 index 000000000..f170b502f --- /dev/null +++ b/src/UserGuide/Master/Tree/API/Programming-MQTT_apache.md @@ -0,0 +1,293 @@ + +# MQTT Protocol + +## 1. Overview + +MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for IoT and low-bandwidth environments. It operates on a Publish/Subscribe (Pub/Sub) model, enabling efficient and reliable bidirectional communication between devices. Its core objectives are low power consumption, minimal bandwidth usage, and high real-time performance, making it ideal for unstable networks or resource-constrained scenarios (e.g., sensors, mobile devices). + +IoTDB provides deep integration with the MQTT protocol, fully compliant with MQTT v3.1 (OASIS International Standard). The IoTDB server includes a built-in high-performance MQTT Broker module, eliminating the need for third-party middleware. Devices can directly write time-series data into the IoTDB storage engine via MQTT messages. + + + + +## 2. Built-in MQTT Service +The Built-in MQTT Service provide the ability of direct connection to IoTDB through MQTT. It listen the publish messages from MQTT clients + and then write the data into storage immediately. +The MQTT topic corresponds to IoTDB timeseries. +The messages payload can be format to events by `PayloadFormatter` which loaded by java SPI, and the default implementation is `JSONPayloadFormatter`. +The default `json` formatter support two json format and its json array. The following is an MQTT message payload example: + +```json + { + "device":"root.sg.d1", + "timestamp":1586076045524, + "measurements":["s1","s2"], + "values":[0.530635,0.530635] + } +``` +or +```json + { + "device":"root.sg.d1", + "timestamps":[1586076045524,1586076065526], + "measurements":["s1","s2"], + "values":[[0.530635,0.530635], [0.530655,0.530695]] + } +``` +or json array of the above two. + + + +## 3. MQTT Configurations +The IoTDB MQTT service load configurations from `${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties` by default. + +Configurations are as follows: + +| **Property** | **Description** | **Default** | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------- | ------------------- | +| `enable_mqtt_service` | Enable/ disable the MQTT service. | FALSE | +| `mqtt_host` | Host address bound to the MQTT service. | 127.0.0.1 | +| `mqtt_port` | Port bound to the MQTT service. | 1883 | +| `mqtt_handler_pool_size` | Thread pool size for processing MQTT messages. | 1 | +| **`mqtt_payload_formatter`** | **Formatting method for MQTT message payloads. ​**​**Options: `json` (tree mode), `line` (table mode).** | **json** | +| `mqtt_max_message_size` | Maximum allowed MQTT message size (bytes). | 1048576 | + +## 4. Coding Examples +The following is an example which a mqtt client send messages to IoTDB server. + +```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +connection.connect(); + +Random random = new Random(); +for (int i = 0; i < 10; i++) { + String payload = String.format("{\n" + + "\"device\":\"root.sg.d1\",\n" + + "\"timestamp\":%d,\n" + + "\"measurements\":[\"s1\"],\n" + + "\"values\":[%f]\n" + + "}", System.currentTimeMillis(), random.nextDouble()); + + connection.publish("root.sg.d1.s1", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +} + +connection.disconnect(); + +``` + +## 5. Customize your MQTT Message Format + +In a production environment, each device typically has its own MQTT client, and the message formats of these clients have been pre-defined. If communication is to be carried out in accordance with the MQTT message format supported by IoTDB, a comprehensive upgrade and transformation of all existing clients would be required, which would undoubtedly incur significant costs. However, we can easily achieve customization of the MQTT message format through simple programming means, without the need to modify the clients. +An example can be found in [example/mqtt-customize](https://github.com/apache/iotdb/tree/rc/2.0.1/example/mqtt-customize) project. + +Assuming the MQTT client sends the following message format: +```json + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"Gasoline car​​", + "point":"Fuel level​​", + "value":10.0 +} +``` +Or in the form of an array of JSON: +```java +[ + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"Gasoline car​​", + "point":"Fuel level", + "value":10.0 + }, + { + "time":1586076045524, + "deviceID":"car_2", + "deviceType":"NEV(new enegry vehicle)", + "point":"Speed", + "value":80.0 + } +] +``` + +Then you can set up the custom MQTT message format through the following steps: + +1. Create a java project, and add dependency: +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. Define your implementation which implements `org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` +e.g., + +```java +package org.apache.iotdb.mqtt.server; + +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; +import org.apache.iotdb.db.protocol.mqtt.TableMessage; + +import com.google.common.collect.Lists; +import com.google.gson.Gson; +import com.google.gson.GsonBuilder; +import com.google.gson.JsonArray; +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParseException; +import io.netty.buffer.ByteBuf; +import org.apache.commons.lang3.NotImplementedException; +import org.apache.tsfile.enums.TSDataType; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +/** + * The Customized JSON payload formatter. one json format supported: { "time":1586076045523, + * "deviceID":"car_1", "deviceType":"NEV", "point":"Speed", "value":80.0 } + */ +public class CustomizedJsonPayloadFormatter implements PayloadFormatter { + private static final String JSON_KEY_TIME = "time"; + private static final String JSON_KEY_DEVICEID = "deviceID"; + private static final String JSON_KEY_DEVICETYPE = "deviceType"; + private static final String JSON_KEY_POINT = "point"; + private static final String JSON_KEY_VALUE = "value"; + private static final Gson GSON = new GsonBuilder().create(); + + @Override + public List format(String topic, ByteBuf payload) { + if (payload == null) { + return new ArrayList<>(); + } + String txt = payload.toString(StandardCharsets.UTF_8); + JsonElement jsonElement = GSON.fromJson(txt, JsonElement.class); + if (jsonElement.isJsonObject()) { + JsonObject jsonObject = jsonElement.getAsJsonObject(); + return formatTableRow(topic, jsonObject); + } else if (jsonElement.isJsonArray()) { + JsonArray jsonArray = jsonElement.getAsJsonArray(); + List messages = new ArrayList<>(); + for (JsonElement element : jsonArray) { + JsonObject jsonObject = element.getAsJsonObject(); + messages.addAll(formatTableRow(topic, jsonObject)); + } + return messages; + } + throw new JsonParseException("payload is invalidate"); + } + + @Override + @Deprecated + public List format(ByteBuf payload) { + throw new NotImplementedException(); + } + + private List formatTableRow(String topic, JsonObject jsonObject) { + TableMessage message = new TableMessage(); + String database = !topic.contains("/") ? topic : topic.substring(0, topic.indexOf("/")); + String table = "test_table"; + + // Parsing Database Name + message.setDatabase((database)); + + // Parsing Table Name + message.setTable(table); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add(JSON_KEY_DEVICEID); + List tagValues = new ArrayList<>(); + tagValues.add(jsonObject.get(JSON_KEY_DEVICEID).getAsString()); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add(JSON_KEY_DEVICETYPE); + attributeValues.add(jsonObject.get(JSON_KEY_DEVICETYPE).getAsString()); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList(JSON_KEY_POINT); + List dataTypes = Arrays.asList(TSDataType.FLOAT); + List values = Arrays.asList(jsonObject.get(JSON_KEY_VALUE).getAsFloat()); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + // Parsing timestamp + message.setTimestamp(jsonObject.get(JSON_KEY_TIME).getAsLong()); + return Lists.newArrayList(message); + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-common.properties as the following string: + return "CustomizedJson2Table"; + } + + @Override + public String getType() { + return PayloadFormatter.TABLE_TYPE; + } +} +``` +3. modify the file in `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter`: + clean the file and put your implementation class name into the file. + In this example, the content is: `org.apache.iotdb.mqtt.server.CustomizedJsonPayloadFormatter` +4. compile your implementation as a jar file: `mvn package -DskipTests` + + +Then, in your server: +1. Create ${IOTDB_HOME}/ext/mqtt/ folder, and put the jar into this folder. +2. Update configuration to enable MQTT service. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. Set the value of `mqtt_payload_formatter` in `conf/iotdb-system.properties` as the value of getName() in your implementation + , in this example, the value is `CustomizedJson2Table` +4. Launch the IoTDB server. +5. Now IoTDB will use your implementation to parse the MQTT message. + +More: the message format can be anything you want. For example, if it is a binary format, +just use `payload.forEachByte()` or `payload.array` to get bytes content. + + +## 6. Caution + +To avoid compatibility issues caused by a default client_id, always explicitly supply a unique, non-empty client_id in every MQTT client. +Behavior varies when the client_id is missing or empty. Common examples: +1. Explicitly sending an empty string + • MQTTX: When client_id="", IoTDB silently discards the message. + • mosquitto_pub: When client_id="", IoTDB receives the message normally. +2. Omitting client_id entirely + • MQTTX: IoTDB accepts the message. + • mosquitto_pub: IoTDB rejects the connection. + Therefore, explicitly assigning a unique, non-empty client_id is the simplest way to eliminate these discrepancies and ensure reliable message delivery. diff --git a/src/UserGuide/Master/Tree/API/Programming-MQTT_timecho.md b/src/UserGuide/Master/Tree/API/Programming-MQTT_timecho.md new file mode 100644 index 000000000..56338a087 --- /dev/null +++ b/src/UserGuide/Master/Tree/API/Programming-MQTT_timecho.md @@ -0,0 +1,294 @@ + +# MQTT Protocol + +## 1. Overview + +MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for IoT and low-bandwidth environments. It operates on a Publish/Subscribe (Pub/Sub) model, enabling efficient and reliable bidirectional communication between devices. Its core objectives are low power consumption, minimal bandwidth usage, and high real-time performance, making it ideal for unstable networks or resource-constrained scenarios (e.g., sensors, mobile devices). + +IoTDB provides deep integration with the MQTT protocol, fully compliant with MQTT v3.1 (OASIS International Standard). The IoTDB server includes a built-in high-performance MQTT Broker module, eliminating the need for third-party middleware. Devices can directly write time-series data into the IoTDB storage engine via MQTT messages. + + + +Note: As of version V2.0.8, the TimechoDB installation package does not include the MQTT service JAR file by default. Please contact the Timecho team to obtain the JAR file before using this service, and place it in the `timechodb_home/lib` or `timechodb_home/ext/external_service` directory. + +## 2. Built-in MQTT Service +The Built-in MQTT Service provide the ability of direct connection to IoTDB through MQTT. It listen the publish messages from MQTT clients + and then write the data into storage immediately. +The MQTT topic corresponds to IoTDB timeseries. +The messages payload can be format to events by `PayloadFormatter` which loaded by java SPI, and the default implementation is `JSONPayloadFormatter`. +The default `json` formatter support two json format and its json array. The following is an MQTT message payload example: + +```json + { + "device":"root.sg.d1", + "timestamp":1586076045524, + "measurements":["s1","s2"], + "values":[0.530635,0.530635] + } +``` +or +```json + { + "device":"root.sg.d1", + "timestamps":[1586076045524,1586076065526], + "measurements":["s1","s2"], + "values":[[0.530635,0.530635], [0.530655,0.530695]] + } +``` +or json array of the above two. + + + +## 3. MQTT Configurations +The IoTDB MQTT service load configurations from `${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties` by default. + +Configurations are as follows: + +| **Property** | **Description** | **Default** | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------- | ------------------- | +| `enable_mqtt_service` | Enable/ disable the MQTT service. | FALSE | +| `mqtt_host` | Host address bound to the MQTT service. | 127.0.0.1 | +| `mqtt_port` | Port bound to the MQTT service. | 1883 | +| `mqtt_handler_pool_size` | Thread pool size for processing MQTT messages. | 1 | +| **`mqtt_payload_formatter`** | **Formatting method for MQTT message payloads. ​**​**Options: `json` (tree mode), `line` (table mode).** | **json** | +| `mqtt_max_message_size` | Maximum allowed MQTT message size (bytes). | 1048576 | + +## 4. Coding Examples +The following is an example which a mqtt client send messages to IoTDB server. + +```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +connection.connect(); + +Random random = new Random(); +for (int i = 0; i < 10; i++) { + String payload = String.format("{\n" + + "\"device\":\"root.sg.d1\",\n" + + "\"timestamp\":%d,\n" + + "\"measurements\":[\"s1\"],\n" + + "\"values\":[%f]\n" + + "}", System.currentTimeMillis(), random.nextDouble()); + + connection.publish("root.sg.d1.s1", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +} + +connection.disconnect(); + +``` + +## 5. Customize your MQTT Message Format + +In a production environment, each device typically has its own MQTT client, and the message formats of these clients have been pre-defined. If communication is to be carried out in accordance with the MQTT message format supported by IoTDB, a comprehensive upgrade and transformation of all existing clients would be required, which would undoubtedly incur significant costs. However, we can easily achieve customization of the MQTT message format through simple programming means, without the need to modify the clients. +An example can be found in [example/mqtt-customize](https://github.com/apache/iotdb/tree/rc/2.0.1/example/mqtt-customize) project. + +Assuming the MQTT client sends the following message format: +```json + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"Gasoline car​​", + "point":"Fuel level​​", + "value":10.0 +} +``` +Or in the form of an array of JSON: +```java +[ + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"Gasoline car​​", + "point":"Fuel level", + "value":10.0 + }, + { + "time":1586076045524, + "deviceID":"car_2", + "deviceType":"NEV(new enegry vehicle)", + "point":"Speed", + "value":80.0 + } +] +``` + +Then you can set up the custom MQTT message format through the following steps: + +1. Create a java project, and add dependency: +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. Define your implementation which implements `org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` +e.g., + +```java +package org.apache.iotdb.mqtt.server; + +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; +import org.apache.iotdb.db.protocol.mqtt.TableMessage; + +import com.google.common.collect.Lists; +import com.google.gson.Gson; +import com.google.gson.GsonBuilder; +import com.google.gson.JsonArray; +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParseException; +import io.netty.buffer.ByteBuf; +import org.apache.commons.lang3.NotImplementedException; +import org.apache.tsfile.enums.TSDataType; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +/** + * The Customized JSON payload formatter. one json format supported: { "time":1586076045523, + * "deviceID":"car_1", "deviceType":"NEV", "point":"Speed", "value":80.0 } + */ +public class CustomizedJsonPayloadFormatter implements PayloadFormatter { + private static final String JSON_KEY_TIME = "time"; + private static final String JSON_KEY_DEVICEID = "deviceID"; + private static final String JSON_KEY_DEVICETYPE = "deviceType"; + private static final String JSON_KEY_POINT = "point"; + private static final String JSON_KEY_VALUE = "value"; + private static final Gson GSON = new GsonBuilder().create(); + + @Override + public List format(String topic, ByteBuf payload) { + if (payload == null) { + return new ArrayList<>(); + } + String txt = payload.toString(StandardCharsets.UTF_8); + JsonElement jsonElement = GSON.fromJson(txt, JsonElement.class); + if (jsonElement.isJsonObject()) { + JsonObject jsonObject = jsonElement.getAsJsonObject(); + return formatTableRow(topic, jsonObject); + } else if (jsonElement.isJsonArray()) { + JsonArray jsonArray = jsonElement.getAsJsonArray(); + List messages = new ArrayList<>(); + for (JsonElement element : jsonArray) { + JsonObject jsonObject = element.getAsJsonObject(); + messages.addAll(formatTableRow(topic, jsonObject)); + } + return messages; + } + throw new JsonParseException("payload is invalidate"); + } + + @Override + @Deprecated + public List format(ByteBuf payload) { + throw new NotImplementedException(); + } + + private List formatTableRow(String topic, JsonObject jsonObject) { + TableMessage message = new TableMessage(); + String database = !topic.contains("/") ? topic : topic.substring(0, topic.indexOf("/")); + String table = "test_table"; + + // Parsing Database Name + message.setDatabase((database)); + + // Parsing Table Name + message.setTable(table); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add(JSON_KEY_DEVICEID); + List tagValues = new ArrayList<>(); + tagValues.add(jsonObject.get(JSON_KEY_DEVICEID).getAsString()); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add(JSON_KEY_DEVICETYPE); + attributeValues.add(jsonObject.get(JSON_KEY_DEVICETYPE).getAsString()); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList(JSON_KEY_POINT); + List dataTypes = Arrays.asList(TSDataType.FLOAT); + List values = Arrays.asList(jsonObject.get(JSON_KEY_VALUE).getAsFloat()); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + // Parsing timestamp + message.setTimestamp(jsonObject.get(JSON_KEY_TIME).getAsLong()); + return Lists.newArrayList(message); + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-common.properties as the following string: + return "CustomizedJson2Table"; + } + + @Override + public String getType() { + return PayloadFormatter.TABLE_TYPE; + } +} +``` +3. modify the file in `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter`: + clean the file and put your implementation class name into the file. + In this example, the content is: `org.apache.iotdb.mqtt.server.CustomizedJsonPayloadFormatter` +4. compile your implementation as a jar file: `mvn package -DskipTests` + + +Then, in your server: +1. Create ${IOTDB_HOME}/ext/mqtt/ folder, and put the jar into this folder. +2. Update configuration to enable MQTT service. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. Set the value of `mqtt_payload_formatter` in `conf/iotdb-system.properties` as the value of getName() in your implementation + , in this example, the value is `CustomizedJson2Table` +4. Launch the IoTDB server. +5. Now IoTDB will use your implementation to parse the MQTT message. + +More: the message format can be anything you want. For example, if it is a binary format, +just use `payload.forEachByte()` or `payload.array` to get bytes content. + + +## 6. Caution + +To avoid compatibility issues caused by a default client_id, always explicitly supply a unique, non-empty client_id in every MQTT client. +Behavior varies when the client_id is missing or empty. Common examples: +1. Explicitly sending an empty string + • MQTTX: When client_id="", IoTDB silently discards the message. + • mosquitto_pub: When client_id="", IoTDB receives the message normally. +2. Omitting client_id entirely + • MQTTX: IoTDB accepts the message. + • mosquitto_pub: IoTDB rejects the connection. + Therefore, explicitly assigning a unique, non-empty client_id is the simplest way to eliminate these discrepancies and ensure reliable message delivery. diff --git a/src/UserGuide/Master/Tree/API/RestServiceV1_timecho.md b/src/UserGuide/Master/Tree/API/RestServiceV1_timecho.md index a10bacce8..c85ae256c 100644 --- a/src/UserGuide/Master/Tree/API/RestServiceV1_timecho.md +++ b/src/UserGuide/Master/Tree/API/RestServiceV1_timecho.md @@ -22,6 +22,8 @@ # REST API V1(Not Recommend) IoTDB's RESTful services can be used for query, write, and management operations, using the OpenAPI standard to define interfaces and generate frameworks. +Note: As of version V2.0.8, the TimechoDB installation package does not include the REST service JAR file by default. Please contact the Timecho team to obtain the corresponding JAR file before using this service, and place it in the `timechodb_home/lib` or `timechodb_home/ext/external_service` directory. + ## 1. Enable RESTful Services RESTful services are disabled by default. diff --git a/src/UserGuide/Master/Tree/API/RestServiceV2_timecho.md b/src/UserGuide/Master/Tree/API/RestServiceV2_timecho.md index cda9f3f0d..69c0d5efe 100644 --- a/src/UserGuide/Master/Tree/API/RestServiceV2_timecho.md +++ b/src/UserGuide/Master/Tree/API/RestServiceV2_timecho.md @@ -22,6 +22,8 @@ # REST API V2 IoTDB's RESTful services can be used for query, write, and management operations, using the OpenAPI standard to define interfaces and generate frameworks. +Note: As of version V2.0.8, the TimechoDB installation package does not include the REST service JAR file by default. Please contact the Timecho team to obtain the corresponding JAR file before using this service, and place it in the `timechodb_home/lib` or `timechodb_home/ext/external_service` directory. + ## 1. Enable RESTful Services RESTful services are disabled by default. diff --git a/src/UserGuide/Master/Tree/Basic-Concept/Write-Data_apache.md b/src/UserGuide/Master/Tree/Basic-Concept/Write-Data_apache.md index d89fe30d3..6d42714a6 100644 --- a/src/UserGuide/Master/Tree/Basic-Concept/Write-Data_apache.md +++ b/src/UserGuide/Master/Tree/Basic-Concept/Write-Data_apache.md @@ -177,7 +177,7 @@ Example： ## 4. MQTT WRITE -Refer to [Built-in MQTT Service](../API/Programming-MQTT.md#built-in-mqtt-service) +Refer to [Built-in MQTT Service](../API/Programming-MQTT_apache.md#_2-built-in-mqtt-service) ## 5. BATCH DATA LOAD diff --git a/src/UserGuide/Master/Tree/Basic-Concept/Write-Data_timecho.md b/src/UserGuide/Master/Tree/Basic-Concept/Write-Data_timecho.md index ac2585dbd..8108566d5 100644 --- a/src/UserGuide/Master/Tree/Basic-Concept/Write-Data_timecho.md +++ b/src/UserGuide/Master/Tree/Basic-Concept/Write-Data_timecho.md @@ -177,7 +177,7 @@ Example： ## 4. MQTT WRITE -Refer to [Built-in MQTT Service](../API/Programming-MQTT.md#built-in-mqtt-service) +Refer to [Built-in MQTT Service](../API/Programming-MQTT_timecho.md#_2-built-in-mqtt-service) ## 5. BATCH DATA LOAD diff --git a/src/UserGuide/Master/Tree/Deployment-and-Maintenance/IoTDB-Package_timecho.md b/src/UserGuide/Master/Tree/Deployment-and-Maintenance/IoTDB-Package_timecho.md index 3c1742408..a0968c919 100644 --- a/src/UserGuide/Master/Tree/Deployment-and-Maintenance/IoTDB-Package_timecho.md +++ b/src/UserGuide/Master/Tree/Deployment-and-Maintenance/IoTDB-Package_timecho.md @@ -43,4 +43,6 @@ After unpacking the installation package（`iotdb-enterprise-{version}-bin.zip` | NOTICE | File | Open-source notice file. | | README_ZH.md | File | User manual (Chinese version). | | README.md | File | User manual (English version). | -| RELEASE_NOTES.md | File | Release notes. | \ No newline at end of file +| RELEASE_NOTES.md | File | Release notes. | + +Note: As of version V2.0.8, the TimechoDB installation package does not include the MQTT service and REST service JAR files by default. If you need to use them, please contact the Timecho team to obtain them. \ No newline at end of file diff --git a/src/UserGuide/latest-Table/API/Programming-MQTT.md b/src/UserGuide/latest-Table/API/Programming-MQTT.md index c7e4e438c..0a6a38616 100644 --- a/src/UserGuide/latest-Table/API/Programming-MQTT.md +++ b/src/UserGuide/latest-Table/API/Programming-MQTT.md @@ -1,3 +1,6 @@ +--- +redirectTo: Programming-MQTT_apache.html +--- -# MQTT Protocol - -## 1. Overview - -MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for IoT and low-bandwidth environments. It operates on a Publish/Subscribe (Pub/Sub) model, enabling efficient and reliable bidirectional communication between devices. Its core objectives are low power consumption, minimal bandwidth usage, and high real-time performance, making it ideal for unstable networks or resource-constrained scenarios (e.g., sensors, mobile devices). - -IoTDB provides deep integration with the MQTT protocol, fully compliant with MQTT v3.1 (OASIS International Standard). The IoTDB server includes a built-in high-performance MQTT Broker module, eliminating the need for third-party middleware. Devices can directly write time-series data into the IoTDB storage engine via MQTT messages. - -![](/img/mqtt-table-en-1.png) - - -## 2. Configuration - -By default, the IoTDB MQTT service loads configurations from `${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`. - -| **Property** | **Description** | **Default** | -| ------------------------ | -------------------------------------------------------------------------------------------------------------------- | ------------------- | -| `enable_mqtt_service` | Enable/ disable the MQTT service. | FALSE | -| `mqtt_host` | Host address bound to the MQTT service. | 127.0.0.1 | -| `mqtt_port` | Port bound to the MQTT service. | 1883 | -| `mqtt_handler_pool_size` | Thread pool size for processing MQTT messages. | 1 | -| **`mqtt_payload_formatter`** | **Formatting method for MQTT message payloads. ​**​**Options: `json` (tree mode), `line` (table mode).** | **json** | -| `mqtt_max_message_size` | Maximum allowed MQTT message size (bytes). | 1048576 | - -## 3. Write Protocol - -* Line Protocol Syntax - -```JavaScript -[,=[,=]][ =[,=]] =[,=] [] -``` - -* Example - -```JavaScript -myMeasurement,tag1=value1,tag2=value2 attr1=value1,attr2=value2 fieldKey="fieldValue" 1556813561098000000 -``` - -![](/img/mqtt-table-en-2.png) - -## 4. Naming Conventions - -* Database Name - -The first segment of the MQTT topic (split by `/`) is used as the database name. - -```Properties -topic: stock/Legacy -databaseName: stock - - -topic: stock/Legacy/# -databaseName:stock -``` - -* Table Name - -The table name is derived from the `` in the line protocol. - -* Type Identifiers - -| Filed Value | IoTDB Data Type | -|--------------------------------------------------------------------| ----------------- | -| 1
1.12 | DOUBLE | -| 1`f`
1.12`f` | FLOAT | -| 1`i`
123`i` | INT64 | -| 1`u`
123`u` | INT64 | -| 1`i32`
123`i32` | INT32 | -| `"xxx"` | TEXT | -| `t`,`T`,`true`,`True`,`TRUE`
`f`,`F`,`false`,`False`,`FALSE` | BOOLEAN | - - -## 5. Coding Examples -The following is an example which a mqtt client send messages to IoTDB server. - - ```java -MQTT mqtt = new MQTT(); -mqtt.setHost("127.0.0.1", 1883); -mqtt.setUserName("root"); -mqtt.setPassword("root"); - -BlockingConnection connection = mqtt.blockingConnection(); -String DATABASE = "myMqttTest"; -connection.connect(); - -String payload = - "test1,tag1=t1,tag2=t2 attr3=a5,attr4=a4 field1=\"fieldValue1\",field2=1i,field3=1u 1"; -connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -payload = "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 2"; -connection.publish(DATABASE, payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -payload = "# It's a remark\n " + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 6"; - connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); - Thread.sleep(10); - -//batch write example -payload = - "test1,tag1=t1,tag2=t2 field7=t,field8=T,field9=true 3 \n " - + "test1,tag1=t1,tag2=t2 field7=f,field8=F,field9=FALSE 4"; -connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -//batch write example -payload = - "test1,tag1=t1,tag2=t2 attr1=a1,attr2=a2 field1=\"fieldValue1\",field2=1i,field3=1u 4 \n " - + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 5"; -connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -connection.disconnect(); - ``` - - - -## 6. Customize your MQTT Message Format - -If you do not like the above Line format, you can customize your MQTT Message format by just writing several lines -of codes. An example can be found in [example/mqtt-customize](https://github.com/apache/iotdb/tree/master/example/mqtt-customize) project. - -Steps: -1. Create a java project, and add dependency: -```xml - - org.apache.iotdb - iotdb-server - 2.0.4-SNAPSHOT - -``` -2. Define your implementation which implements `org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` - e.g., - -```java -package org.apache.iotdb.mqtt.server; - -import io.netty.buffer.ByteBuf; -import org.apache.iotdb.db.protocol.mqtt.Message; -import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; - -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -public class CustomizedLinePayloadFormatter implements PayloadFormatter { - - @Override - public List format(String topic, ByteBuf payload) { - // Suppose the payload is a line format - if (payload == null) { - return null; - } - - String line = payload.toString(StandardCharsets.UTF_8); - // parse data from the line and generate Messages and put them into List ret - List ret = new ArrayList<>(); - // this is just an example, so we just generate some Messages directly - for (int i = 0; i < 3; i++) { - long ts = i; - TableMessage message = new TableMessage(); - - // Parsing Database Name - message.setDatabase("db" + i); - - //Parsing Table Names - message.setTable("t" + i); - - // Parsing Tags - List tagKeys = new ArrayList<>(); - tagKeys.add("tag1" + i); - tagKeys.add("tag2" + i); - List tagValues = new ArrayList<>(); - tagValues.add("t_value1" + i); - tagValues.add("t_value2" + i); - message.setTagKeys(tagKeys); - message.setTagValues(tagValues); - - // Parsing Attributes - List attributeKeys = new ArrayList<>(); - List attributeValues = new ArrayList<>(); - attributeKeys.add("attr1" + i); - attributeKeys.add("attr2" + i); - attributeValues.add("a_value1" + i); - attributeValues.add("a_value2" + i); - message.setAttributeKeys(attributeKeys); - message.setAttributeValues(attributeValues); - - // Parsing Fields - List fields = Arrays.asList("field1" + i, "field2" + i); - List dataTypes = Arrays.asList(TSDataType.FLOAT, TSDataType.FLOAT); - List values = Arrays.asList("4.0" + i, "5.0" + i); - message.setFields(fields); - message.setDataTypes(dataTypes); - message.setValues(values); - - //// Parsing timestamp - message.setTimestamp(ts); - ret.add(message); - } - return ret; - } - - @Override - public String getName() { - // set the value of mqtt_payload_formatter in iotdb-system.properties as the following string: - return "CustomizedLine"; - } -} -``` -3. modify the file in `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter`: - clean the file and put your implementation class name into the file. - In this example, the content is: `org.apache.iotdb.mqtt.server.CustomizedLinePayloadFormatter` -4. compile your implementation as a jar file: `mvn package -DskipTests` - - -Then, in your server: -1. Create ${IOTDB_HOME}/ext/mqtt/ folder, and put the jar into this folder. -2. Update configuration to enable MQTT service. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) -3. Set the value of `mqtt_payload_formatter` in `conf/iotdb-system.properties` as the value of getName() in your implementation - , in this example, the value is `CustomizedLine` -4. Launch the IoTDB server. -5. Now IoTDB will use your implementation to parse the MQTT message. - -More: the message format can be anything you want. For example, if it is a binary format, -just use `payload.forEachByte()` or `payload.array` to get bytes content. - -## 7. Caution - -To avoid compatibility issues caused by a default client_id, always explicitly supply a unique, non-empty client_id in every MQTT client. -Behavior varies when the client_id is missing or empty. Common examples: -1. Explicitly sending an empty string -• MQTTX: When client_id="", IoTDB silently discards the message. -• mosquitto_pub: When client_id="", IoTDB receives the message normally. -2. Omitting client_id entirely -• MQTTX: IoTDB accepts the message. -• mosquitto_pub: IoTDB rejects the connection. -Therefore, explicitly assigning a unique, non-empty client_id is the simplest way to eliminate these discrepancies and ensure reliable message delivery. \ No newline at end of file +--> \ No newline at end of file diff --git a/src/UserGuide/latest-Table/API/Programming-MQTT_apache.md b/src/UserGuide/latest-Table/API/Programming-MQTT_apache.md new file mode 100644 index 000000000..c7e4e438c --- /dev/null +++ b/src/UserGuide/latest-Table/API/Programming-MQTT_apache.md @@ -0,0 +1,259 @@ + +# MQTT Protocol + +## 1. Overview + +MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for IoT and low-bandwidth environments. It operates on a Publish/Subscribe (Pub/Sub) model, enabling efficient and reliable bidirectional communication between devices. Its core objectives are low power consumption, minimal bandwidth usage, and high real-time performance, making it ideal for unstable networks or resource-constrained scenarios (e.g., sensors, mobile devices). + +IoTDB provides deep integration with the MQTT protocol, fully compliant with MQTT v3.1 (OASIS International Standard). The IoTDB server includes a built-in high-performance MQTT Broker module, eliminating the need for third-party middleware. Devices can directly write time-series data into the IoTDB storage engine via MQTT messages. + +![](/img/mqtt-table-en-1.png) + + +## 2. Configuration + +By default, the IoTDB MQTT service loads configurations from `${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`. + +| **Property** | **Description** | **Default** | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------- | ------------------- | +| `enable_mqtt_service` | Enable/ disable the MQTT service. | FALSE | +| `mqtt_host` | Host address bound to the MQTT service. | 127.0.0.1 | +| `mqtt_port` | Port bound to the MQTT service. | 1883 | +| `mqtt_handler_pool_size` | Thread pool size for processing MQTT messages. | 1 | +| **`mqtt_payload_formatter`** | **Formatting method for MQTT message payloads. ​**​**Options: `json` (tree mode), `line` (table mode).** | **json** | +| `mqtt_max_message_size` | Maximum allowed MQTT message size (bytes). | 1048576 | + +## 3. Write Protocol + +* Line Protocol Syntax + +```JavaScript +[,=[,=]][ =[,=]] =[,=] [] +``` + +* Example + +```JavaScript +myMeasurement,tag1=value1,tag2=value2 attr1=value1,attr2=value2 fieldKey="fieldValue" 1556813561098000000 +``` + +![](/img/mqtt-table-en-2.png) + +## 4. Naming Conventions + +* Database Name + +The first segment of the MQTT topic (split by `/`) is used as the database name. + +```Properties +topic: stock/Legacy +databaseName: stock + + +topic: stock/Legacy/# +databaseName:stock +``` + +* Table Name + +The table name is derived from the `` in the line protocol. + +* Type Identifiers + +| Filed Value | IoTDB Data Type | +|--------------------------------------------------------------------| ----------------- | +| 1
1.12 | DOUBLE | +| 1`f`
1.12`f` | FLOAT | +| 1`i`
123`i` | INT64 | +| 1`u`
123`u` | INT64 | +| 1`i32`
123`i32` | INT32 | +| `"xxx"` | TEXT | +| `t`,`T`,`true`,`True`,`TRUE`
`f`,`F`,`false`,`False`,`FALSE` | BOOLEAN | + + +## 5. Coding Examples +The following is an example which a mqtt client send messages to IoTDB server. + + ```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +String DATABASE = "myMqttTest"; +connection.connect(); + +String payload = + "test1,tag1=t1,tag2=t2 attr3=a5,attr4=a4 field1=\"fieldValue1\",field2=1i,field3=1u 1"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 2"; +connection.publish(DATABASE, payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "# It's a remark\n " + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 6"; + connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); + Thread.sleep(10); + +//batch write example +payload = + "test1,tag1=t1,tag2=t2 field7=t,field8=T,field9=true 3 \n " + + "test1,tag1=t1,tag2=t2 field7=f,field8=F,field9=FALSE 4"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +//batch write example +payload = + "test1,tag1=t1,tag2=t2 attr1=a1,attr2=a2 field1=\"fieldValue1\",field2=1i,field3=1u 4 \n " + + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 5"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +connection.disconnect(); + ``` + + + +## 6. Customize your MQTT Message Format + +If you do not like the above Line format, you can customize your MQTT Message format by just writing several lines +of codes. An example can be found in [example/mqtt-customize](https://github.com/apache/iotdb/tree/master/example/mqtt-customize) project. + +Steps: +1. Create a java project, and add dependency: +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. Define your implementation which implements `org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` + e.g., + +```java +package org.apache.iotdb.mqtt.server; + +import io.netty.buffer.ByteBuf; +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +public class CustomizedLinePayloadFormatter implements PayloadFormatter { + + @Override + public List format(String topic, ByteBuf payload) { + // Suppose the payload is a line format + if (payload == null) { + return null; + } + + String line = payload.toString(StandardCharsets.UTF_8); + // parse data from the line and generate Messages and put them into List ret + List ret = new ArrayList<>(); + // this is just an example, so we just generate some Messages directly + for (int i = 0; i < 3; i++) { + long ts = i; + TableMessage message = new TableMessage(); + + // Parsing Database Name + message.setDatabase("db" + i); + + //Parsing Table Names + message.setTable("t" + i); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add("tag1" + i); + tagKeys.add("tag2" + i); + List tagValues = new ArrayList<>(); + tagValues.add("t_value1" + i); + tagValues.add("t_value2" + i); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add("attr1" + i); + attributeKeys.add("attr2" + i); + attributeValues.add("a_value1" + i); + attributeValues.add("a_value2" + i); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList("field1" + i, "field2" + i); + List dataTypes = Arrays.asList(TSDataType.FLOAT, TSDataType.FLOAT); + List values = Arrays.asList("4.0" + i, "5.0" + i); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + //// Parsing timestamp + message.setTimestamp(ts); + ret.add(message); + } + return ret; + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-system.properties as the following string: + return "CustomizedLine"; + } +} +``` +3. modify the file in `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter`: + clean the file and put your implementation class name into the file. + In this example, the content is: `org.apache.iotdb.mqtt.server.CustomizedLinePayloadFormatter` +4. compile your implementation as a jar file: `mvn package -DskipTests` + + +Then, in your server: +1. Create ${IOTDB_HOME}/ext/mqtt/ folder, and put the jar into this folder. +2. Update configuration to enable MQTT service. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. Set the value of `mqtt_payload_formatter` in `conf/iotdb-system.properties` as the value of getName() in your implementation + , in this example, the value is `CustomizedLine` +4. Launch the IoTDB server. +5. Now IoTDB will use your implementation to parse the MQTT message. + +More: the message format can be anything you want. For example, if it is a binary format, +just use `payload.forEachByte()` or `payload.array` to get bytes content. + +## 7. Caution + +To avoid compatibility issues caused by a default client_id, always explicitly supply a unique, non-empty client_id in every MQTT client. +Behavior varies when the client_id is missing or empty. Common examples: +1. Explicitly sending an empty string +• MQTTX: When client_id="", IoTDB silently discards the message. +• mosquitto_pub: When client_id="", IoTDB receives the message normally. +2. Omitting client_id entirely +• MQTTX: IoTDB accepts the message. +• mosquitto_pub: IoTDB rejects the connection. +Therefore, explicitly assigning a unique, non-empty client_id is the simplest way to eliminate these discrepancies and ensure reliable message delivery. \ No newline at end of file diff --git a/src/UserGuide/latest-Table/API/Programming-MQTT_timecho.md b/src/UserGuide/latest-Table/API/Programming-MQTT_timecho.md new file mode 100644 index 000000000..3c6170a2f --- /dev/null +++ b/src/UserGuide/latest-Table/API/Programming-MQTT_timecho.md @@ -0,0 +1,261 @@ + +# MQTT Protocol + +## 1. Overview + +MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for IoT and low-bandwidth environments. It operates on a Publish/Subscribe (Pub/Sub) model, enabling efficient and reliable bidirectional communication between devices. Its core objectives are low power consumption, minimal bandwidth usage, and high real-time performance, making it ideal for unstable networks or resource-constrained scenarios (e.g., sensors, mobile devices). + +IoTDB provides deep integration with the MQTT protocol, fully compliant with MQTT v3.1 (OASIS International Standard). The IoTDB server includes a built-in high-performance MQTT Broker module, eliminating the need for third-party middleware. Devices can directly write time-series data into the IoTDB storage engine via MQTT messages. + +![](/img/mqtt-table-en-1.png) + +Note: As of version V2.0.8, the TimechoDB installation package does not include the MQTT service JAR file by default. Please contact the Timecho team to obtain the JAR file before using this service, and place it in the `timechodb_home/lib` or `timechodb_home/ext/external_service` directory. + + +## 2. Configuration + +By default, the IoTDB MQTT service loads configurations from `${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`. + +| **Property** | **Description** | **Default** | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------- | ------------------- | +| `enable_mqtt_service` | Enable/ disable the MQTT service. | FALSE | +| `mqtt_host` | Host address bound to the MQTT service. | 127.0.0.1 | +| `mqtt_port` | Port bound to the MQTT service. | 1883 | +| `mqtt_handler_pool_size` | Thread pool size for processing MQTT messages. | 1 | +| **`mqtt_payload_formatter`** | **Formatting method for MQTT message payloads. ​**​**Options: `json` (tree mode), `line` (table mode).** | **json** | +| `mqtt_max_message_size` | Maximum allowed MQTT message size (bytes). | 1048576 | + +## 3. Write Protocol + +* Line Protocol Syntax + +```JavaScript +[,=[,=]][ =[,=]] =[,=] [] +``` + +* Example + +```JavaScript +myMeasurement,tag1=value1,tag2=value2 attr1=value1,attr2=value2 fieldKey="fieldValue" 1556813561098000000 +``` + +![](/img/mqtt-table-en-2.png) + +## 4. Naming Conventions + +* Database Name + +The first segment of the MQTT topic (split by `/`) is used as the database name. + +```Properties +topic: stock/Legacy +databaseName: stock + + +topic: stock/Legacy/# +databaseName:stock +``` + +* Table Name + +The table name is derived from the `` in the line protocol. + +* Type Identifiers + +| Filed Value | IoTDB Data Type | +|--------------------------------------------------------------------| ----------------- | +| 1
1.12 | DOUBLE | +| 1`f`
1.12`f` | FLOAT | +| 1`i`
123`i` | INT64 | +| 1`u`
123`u` | INT64 | +| 1`i32`
123`i32` | INT32 | +| `"xxx"` | TEXT | +| `t`,`T`,`true`,`True`,`TRUE`
`f`,`F`,`false`,`False`,`FALSE` | BOOLEAN | + + +## 5. Coding Examples +The following is an example which a mqtt client send messages to IoTDB server. + + ```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +String DATABASE = "myMqttTest"; +connection.connect(); + +String payload = + "test1,tag1=t1,tag2=t2 attr3=a5,attr4=a4 field1=\"fieldValue1\",field2=1i,field3=1u 1"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 2"; +connection.publish(DATABASE, payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "# It's a remark\n " + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 6"; + connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); + Thread.sleep(10); + +//batch write example +payload = + "test1,tag1=t1,tag2=t2 field7=t,field8=T,field9=true 3 \n " + + "test1,tag1=t1,tag2=t2 field7=f,field8=F,field9=FALSE 4"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +//batch write example +payload = + "test1,tag1=t1,tag2=t2 attr1=a1,attr2=a2 field1=\"fieldValue1\",field2=1i,field3=1u 4 \n " + + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 5"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +connection.disconnect(); + ``` + + + +## 6. Customize your MQTT Message Format + +If you do not like the above Line format, you can customize your MQTT Message format by just writing several lines +of codes. An example can be found in [example/mqtt-customize](https://github.com/apache/iotdb/tree/master/example/mqtt-customize) project. + +Steps: +1. Create a java project, and add dependency: +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. Define your implementation which implements `org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` + e.g., + +```java +package org.apache.iotdb.mqtt.server; + +import io.netty.buffer.ByteBuf; +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +public class CustomizedLinePayloadFormatter implements PayloadFormatter { + + @Override + public List format(String topic, ByteBuf payload) { + // Suppose the payload is a line format + if (payload == null) { + return null; + } + + String line = payload.toString(StandardCharsets.UTF_8); + // parse data from the line and generate Messages and put them into List ret + List ret = new ArrayList<>(); + // this is just an example, so we just generate some Messages directly + for (int i = 0; i < 3; i++) { + long ts = i; + TableMessage message = new TableMessage(); + + // Parsing Database Name + message.setDatabase("db" + i); + + //Parsing Table Names + message.setTable("t" + i); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add("tag1" + i); + tagKeys.add("tag2" + i); + List tagValues = new ArrayList<>(); + tagValues.add("t_value1" + i); + tagValues.add("t_value2" + i); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add("attr1" + i); + attributeKeys.add("attr2" + i); + attributeValues.add("a_value1" + i); + attributeValues.add("a_value2" + i); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList("field1" + i, "field2" + i); + List dataTypes = Arrays.asList(TSDataType.FLOAT, TSDataType.FLOAT); + List values = Arrays.asList("4.0" + i, "5.0" + i); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + //// Parsing timestamp + message.setTimestamp(ts); + ret.add(message); + } + return ret; + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-system.properties as the following string: + return "CustomizedLine"; + } +} +``` +3. modify the file in `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter`: + clean the file and put your implementation class name into the file. + In this example, the content is: `org.apache.iotdb.mqtt.server.CustomizedLinePayloadFormatter` +4. compile your implementation as a jar file: `mvn package -DskipTests` + + +Then, in your server: +1. Create ${IOTDB_HOME}/ext/mqtt/ folder, and put the jar into this folder. +2. Update configuration to enable MQTT service. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. Set the value of `mqtt_payload_formatter` in `conf/iotdb-system.properties` as the value of getName() in your implementation + , in this example, the value is `CustomizedLine` +4. Launch the IoTDB server. +5. Now IoTDB will use your implementation to parse the MQTT message. + +More: the message format can be anything you want. For example, if it is a binary format, +just use `payload.forEachByte()` or `payload.array` to get bytes content. + +## 7. Caution + +To avoid compatibility issues caused by a default client_id, always explicitly supply a unique, non-empty client_id in every MQTT client. +Behavior varies when the client_id is missing or empty. Common examples: +1. Explicitly sending an empty string +• MQTTX: When client_id="", IoTDB silently discards the message. +• mosquitto_pub: When client_id="", IoTDB receives the message normally. +2. Omitting client_id entirely +• MQTTX: IoTDB accepts the message. +• mosquitto_pub: IoTDB rejects the connection. +Therefore, explicitly assigning a unique, non-empty client_id is the simplest way to eliminate these discrepancies and ensure reliable message delivery. \ No newline at end of file diff --git a/src/UserGuide/latest-Table/API/RestAPI-V1.md b/src/UserGuide/latest-Table/API/RestAPI-V1.md index babf3ac98..c3c6f6554 100644 --- a/src/UserGuide/latest-Table/API/RestAPI-V1.md +++ b/src/UserGuide/latest-Table/API/RestAPI-V1.md @@ -1,3 +1,6 @@ +--- +redirectTo: RestAPI-V1_apache.html +--- -# RestAPI V1 - -IoTDB's RESTful service can be used for querying, writing, and management operations. It uses the OpenAPI standard to define interfaces and generate frameworks. - -## 1. Enabling RESTful Service - -The RESTful service is disabled by default. To enable it, locate the `conf/iotdb-system.properties` file in the IoTDB installation directory, set `enable_rest_service` to `true`, and then restart the datanode process. - -```Properties -enable_rest_service=true -``` - -## 2. Authentication - -Except for the health check endpoint `/ping`, the RESTful service uses basic authentication. Each URL request must include the header `'Authorization':'Basic' + base64.encode(username + ':' + password)`. - -In the example, the username is `root` and the password is `root`. The corresponding Basic authentication header format is: - -```Properties -Authorization : Basic cm9vdDpyb290 -``` - -- If the username or password authentication fails, the following information is returned: - - - HTTP status code: 801 - - - Response body: - - ```JSON - {"code":801,"message":"WRONG_LOGIN_PASSWORD"} - ``` - -- If the `Authorization` header is not set, the following information is returned: - - - HTTP status code: 800 - - - Response body: - - ```JSON - {"code":800,"message":"INIT_AUTH_ERROR"} - ``` - -## 3. Interface Definitions - -### 3.1 Ping - -The `/ping` endpoint can be used for online service health checks. - -- Request Method: GET - -- Request Path: `http://ip:port/ping` - -- Example Request: - - ```Bash - curl http://127.0.0.1:18080/ping - ``` - -- HTTP Status Codes: - - - `200`: The service is working normally and can accept external requests. - - - `503`: The service is experiencing issues and cannot accept external requests. - - | Parameter Name | Type | Description | - | :------------- | :------ | :--------------- | - | code | integer | Status Code | - | message | string | Code Information | - -- Response Example: - - - When the HTTP status code is `200`: - - ```JSON - { "code": 200, "message": "SUCCESS_STATUS"} - ``` - - - When the HTTP status code is `503`: - - ```JSON - { "code": 500, "message": "thrift service is unavailable"} - ``` - -**Note**: The `/ping` endpoint does not require authentication. - -### 3.2 Query Interface - -- Request Path: `/rest/table/v1/query` - -- Request Method: POST - -- Request Format: - - - Header: `application/json` - - - Request Parameters: - - | Parameter Name | Type | Required | Description | - | :------------- | :----- | :------- | :----------------------------------------------------------- | - | `database` | string | Yes | Database name | - | `sql` | string | Yes | SQL query | - | `row_limit` | int | No | Maximum number of rows to return in a single query. If not set, the default value from the configuration file (`rest_query_default_row_size_limit`) is used. If the result set exceeds this limit, status code `411` is returned. | - -- Response Format: - - | Parameter Name | Type | Description | - | :------------- | :---- | :----------------------------------------------------------- | - | `column_names` | array | Column names | - | `data_types` | array | Data types of each column | - | `values` | array | A 2D array where the first dimension represents rows, and the second dimension represents columns. Each element corresponds to a column, with the same length as `column_names`. | - -- Example Request: - - ```Bash - curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","sql":"select s1,s2,s3 from test_table"}' http://127.0.0.1:18080/rest/table/v1/query - ``` - -- Example Response: - - ```JSON - { - "column_names": [ - "s1", - "s2", - "s3" - ], - "data_types": [ - "STRING", - "BOOLEAN", - "INT32" - ], - "values": [ - [ - "a11", - true, - 2024 - ], - [ - "a11", - false, - 2025 - ] - ] - } - ``` - -### 3.3 Non-Query Interface - -- Request Path: `/rest/table/v1/nonQuery` - -- Request Method: POST - -- Request Format: - - - Header: `application/json` - - - Request Parameters: - - | Parameter Name | Type | Required | Description | - | :------------- | :----- | :------- | :------------ | - | `sql` | string | Yes | SQL statement | - | `database` | string | No | Database name | - -- Response Format: - - | Parameter Name | Type | Description | - | :------------- | :------ | :---------- | - | `code` | integer | Status code | - | `message` | string | Message | - -- Example Requests: - - - Create a database: - - ```Bash - curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"create database test","database":""}' http://127.0.0.1:18080/rest/table/v1/nonQuery - ``` - - - Create a table `test_table` in the `test` database: - - ```Bash - curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"CREATE TABLE table1 (time TIMESTAMP TIME,region STRING TAG,plant_id STRING TAG,device_id STRING TAG,model_id STRING ATTRIBUTE,maintenance STRING ATTRIBUTE,temperature FLOAT FIELD,humidity FLOAT FIELD,status Boolean FIELD,arrival_time TIMESTAMP FIELD) WITH (TTL=31536000000)","database":"test"}' http://127.0.0.1:18080/rest/table/v1/nonQuery - ``` - -- Example Response: - - ```JSON - { - "code": 200, - "message": "SUCCESS_STATUS" - } - ``` - -### 3.4 Batch Write Interface - -- Request Path: `/rest/table/v1/insertTablet` - -- Request Method: POST - -- Request Format: - - - Header: `application/json` - - - Request Parameters: - - | Parameter Name | Type | Required | Description | - | :------------------ | :----- | :------- | :----------------------------------------------------------- | - | `database` | string | Yes | Database name | - | `table` | string | Yes | Table name | - | `column_names` | array | Yes | Column names | - | `column_categories` | array | Yes | Column categories (`TAG`, `FIELD`, `ATTRIBUTE`) | - | `data_types` | array | Yes | Data types | - | `timestamps` | array | Yes | Timestamp column | - | `values` | array | Yes | Value columns. Each column's values can be `null`. A 2D array where the first dimension corresponds to timestamps, and the second dimension corresponds to columns. | - -- Response Format: - - | Parameter Name | Type | Description | - | :------------- | :------ | :---------- | - | `code` | integer | Status code | - | `message` | string | Message | - -- Example Request: - - ```Bash - curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","column_categories":["TAG","FIELD","FIELD"],"timestamps":[1739702535000,1739789055000],"column_names":["s1","s2","s3"],"data_types":["STRING","BOOLEAN","INT32"],"values":[["a11",true,2024],["a11",false,2025]],"table":"test_table"}' http://127.0.0.1:18080/rest/table/v1/insertTablet - ``` - -- Example Response: - - ```JSON - { - "code": 200, - "message": "SUCCESS_STATUS" - } - ``` - -## 4. Configuration - -The configuration file is located in `iotdb-system.properties`. - -- Set `enable_rest_service` to `true` to enable the module, or `false` to disable it. The default value is `false`. - - ```Properties - enable_rest_service=true - ``` - -- Only effective when `enable_rest_service=true`. Set `rest_service_port` to a number (1025~65535) to customize the REST service socket port. The default value is `18080`. - - ```Properties - rest_service_port=18080 - ``` - -- Set `enable_swagger` to `true` to enable Swagger for displaying REST interface information, or `false` to disable it. The default value is `false`. - - ```Properties - enable_swagger=false - ``` - -- The maximum number of rows that can be returned in a single query. If the result set exceeds this limit, only the rows within the limit will be returned, and status code `411` will be returned. - - ```Properties - rest_query_default_row_size_limit=10000 - ``` - -- Expiration time for caching client login information (used to speed up user authentication, in seconds, default is 8 hours). - - ```Properties - cache_expire_in_seconds=28800 - ``` - -- Maximum number of users stored in the cache (default is 100). - - ```Properties - cache_max_num=100 - ``` - -- Initial cache capacity (default is 10). - - ```Properties - cache_init_num=10 - ``` - -- Whether to enable SSL configuration for the REST service. Set `enable_https` to `true` to enable it, or `false` to disable it. The default value is `false`. - - ```Properties - enable_https=false - ``` - -- Path to the `keyStore` (optional). - - ```Properties - key_store_path= - ``` - -- Password for the `keyStore` (optional). - - ```Properties - key_store_pwd= - ``` - -- Path to the `trustStore` (optional). - - ```Properties - trust_store_path="" - ``` - -- Password for the `trustStore` (optional). - - ```Properties - trust_store_pwd="" - ``` - -- SSL timeout time, in seconds. - - ```Properties - idle_timeout_in_seconds=5000 - ``` \ No newline at end of file +--> \ No newline at end of file diff --git a/src/UserGuide/latest-Table/API/RestAPI-V1_apache.md b/src/UserGuide/latest-Table/API/RestAPI-V1_apache.md new file mode 100644 index 000000000..babf3ac98 --- /dev/null +++ b/src/UserGuide/latest-Table/API/RestAPI-V1_apache.md @@ -0,0 +1,338 @@ + +# RestAPI V1 + +IoTDB's RESTful service can be used for querying, writing, and management operations. It uses the OpenAPI standard to define interfaces and generate frameworks. + +## 1. Enabling RESTful Service + +The RESTful service is disabled by default. To enable it, locate the `conf/iotdb-system.properties` file in the IoTDB installation directory, set `enable_rest_service` to `true`, and then restart the datanode process. + +```Properties +enable_rest_service=true +``` + +## 2. Authentication + +Except for the health check endpoint `/ping`, the RESTful service uses basic authentication. Each URL request must include the header `'Authorization':'Basic' + base64.encode(username + ':' + password)`. + +In the example, the username is `root` and the password is `root`. The corresponding Basic authentication header format is: + +```Properties +Authorization : Basic cm9vdDpyb290 +``` + +- If the username or password authentication fails, the following information is returned: + + - HTTP status code: 801 + + - Response body: + + ```JSON + {"code":801,"message":"WRONG_LOGIN_PASSWORD"} + ``` + +- If the `Authorization` header is not set, the following information is returned: + + - HTTP status code: 800 + + - Response body: + + ```JSON + {"code":800,"message":"INIT_AUTH_ERROR"} + ``` + +## 3. Interface Definitions + +### 3.1 Ping + +The `/ping` endpoint can be used for online service health checks. + +- Request Method: GET + +- Request Path: `http://ip:port/ping` + +- Example Request: + + ```Bash + curl http://127.0.0.1:18080/ping + ``` + +- HTTP Status Codes: + + - `200`: The service is working normally and can accept external requests. + + - `503`: The service is experiencing issues and cannot accept external requests. + + | Parameter Name | Type | Description | + | :------------- | :------ | :--------------- | + | code | integer | Status Code | + | message | string | Code Information | + +- Response Example: + + - When the HTTP status code is `200`: + + ```JSON + { "code": 200, "message": "SUCCESS_STATUS"} + ``` + + - When the HTTP status code is `503`: + + ```JSON + { "code": 500, "message": "thrift service is unavailable"} + ``` + +**Note**: The `/ping` endpoint does not require authentication. + +### 3.2 Query Interface + +- Request Path: `/rest/table/v1/query` + +- Request Method: POST + +- Request Format: + + - Header: `application/json` + + - Request Parameters: + + | Parameter Name | Type | Required | Description | + | :------------- | :----- | :------- | :----------------------------------------------------------- | + | `database` | string | Yes | Database name | + | `sql` | string | Yes | SQL query | + | `row_limit` | int | No | Maximum number of rows to return in a single query. If not set, the default value from the configuration file (`rest_query_default_row_size_limit`) is used. If the result set exceeds this limit, status code `411` is returned. | + +- Response Format: + + | Parameter Name | Type | Description | + | :------------- | :---- | :----------------------------------------------------------- | + | `column_names` | array | Column names | + | `data_types` | array | Data types of each column | + | `values` | array | A 2D array where the first dimension represents rows, and the second dimension represents columns. Each element corresponds to a column, with the same length as `column_names`. | + +- Example Request: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","sql":"select s1,s2,s3 from test_table"}' http://127.0.0.1:18080/rest/table/v1/query + ``` + +- Example Response: + + ```JSON + { + "column_names": [ + "s1", + "s2", + "s3" + ], + "data_types": [ + "STRING", + "BOOLEAN", + "INT32" + ], + "values": [ + [ + "a11", + true, + 2024 + ], + [ + "a11", + false, + 2025 + ] + ] + } + ``` + +### 3.3 Non-Query Interface + +- Request Path: `/rest/table/v1/nonQuery` + +- Request Method: POST + +- Request Format: + + - Header: `application/json` + + - Request Parameters: + + | Parameter Name | Type | Required | Description | + | :------------- | :----- | :------- | :------------ | + | `sql` | string | Yes | SQL statement | + | `database` | string | No | Database name | + +- Response Format: + + | Parameter Name | Type | Description | + | :------------- | :------ | :---------- | + | `code` | integer | Status code | + | `message` | string | Message | + +- Example Requests: + + - Create a database: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"create database test","database":""}' http://127.0.0.1:18080/rest/table/v1/nonQuery + ``` + + - Create a table `test_table` in the `test` database: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"CREATE TABLE table1 (time TIMESTAMP TIME,region STRING TAG,plant_id STRING TAG,device_id STRING TAG,model_id STRING ATTRIBUTE,maintenance STRING ATTRIBUTE,temperature FLOAT FIELD,humidity FLOAT FIELD,status Boolean FIELD,arrival_time TIMESTAMP FIELD) WITH (TTL=31536000000)","database":"test"}' http://127.0.0.1:18080/rest/table/v1/nonQuery + ``` + +- Example Response: + + ```JSON + { + "code": 200, + "message": "SUCCESS_STATUS" + } + ``` + +### 3.4 Batch Write Interface + +- Request Path: `/rest/table/v1/insertTablet` + +- Request Method: POST + +- Request Format: + + - Header: `application/json` + + - Request Parameters: + + | Parameter Name | Type | Required | Description | + | :------------------ | :----- | :------- | :----------------------------------------------------------- | + | `database` | string | Yes | Database name | + | `table` | string | Yes | Table name | + | `column_names` | array | Yes | Column names | + | `column_categories` | array | Yes | Column categories (`TAG`, `FIELD`, `ATTRIBUTE`) | + | `data_types` | array | Yes | Data types | + | `timestamps` | array | Yes | Timestamp column | + | `values` | array | Yes | Value columns. Each column's values can be `null`. A 2D array where the first dimension corresponds to timestamps, and the second dimension corresponds to columns. | + +- Response Format: + + | Parameter Name | Type | Description | + | :------------- | :------ | :---------- | + | `code` | integer | Status code | + | `message` | string | Message | + +- Example Request: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","column_categories":["TAG","FIELD","FIELD"],"timestamps":[1739702535000,1739789055000],"column_names":["s1","s2","s3"],"data_types":["STRING","BOOLEAN","INT32"],"values":[["a11",true,2024],["a11",false,2025]],"table":"test_table"}' http://127.0.0.1:18080/rest/table/v1/insertTablet + ``` + +- Example Response: + + ```JSON + { + "code": 200, + "message": "SUCCESS_STATUS" + } + ``` + +## 4. Configuration + +The configuration file is located in `iotdb-system.properties`. + +- Set `enable_rest_service` to `true` to enable the module, or `false` to disable it. The default value is `false`. + + ```Properties + enable_rest_service=true + ``` + +- Only effective when `enable_rest_service=true`. Set `rest_service_port` to a number (1025~65535) to customize the REST service socket port. The default value is `18080`. + + ```Properties + rest_service_port=18080 + ``` + +- Set `enable_swagger` to `true` to enable Swagger for displaying REST interface information, or `false` to disable it. The default value is `false`. + + ```Properties + enable_swagger=false + ``` + +- The maximum number of rows that can be returned in a single query. If the result set exceeds this limit, only the rows within the limit will be returned, and status code `411` will be returned. + + ```Properties + rest_query_default_row_size_limit=10000 + ``` + +- Expiration time for caching client login information (used to speed up user authentication, in seconds, default is 8 hours). + + ```Properties + cache_expire_in_seconds=28800 + ``` + +- Maximum number of users stored in the cache (default is 100). + + ```Properties + cache_max_num=100 + ``` + +- Initial cache capacity (default is 10). + + ```Properties + cache_init_num=10 + ``` + +- Whether to enable SSL configuration for the REST service. Set `enable_https` to `true` to enable it, or `false` to disable it. The default value is `false`. + + ```Properties + enable_https=false + ``` + +- Path to the `keyStore` (optional). + + ```Properties + key_store_path= + ``` + +- Password for the `keyStore` (optional). + + ```Properties + key_store_pwd= + ``` + +- Path to the `trustStore` (optional). + + ```Properties + trust_store_path="" + ``` + +- Password for the `trustStore` (optional). + + ```Properties + trust_store_pwd="" + ``` + +- SSL timeout time, in seconds. + + ```Properties + idle_timeout_in_seconds=5000 + ``` \ No newline at end of file diff --git a/src/UserGuide/latest-Table/API/RestAPI-V1_timecho.md b/src/UserGuide/latest-Table/API/RestAPI-V1_timecho.md new file mode 100644 index 000000000..fffe6872d --- /dev/null +++ b/src/UserGuide/latest-Table/API/RestAPI-V1_timecho.md @@ -0,0 +1,340 @@ + +# RestAPI V1 + +IoTDB's RESTful service can be used for querying, writing, and management operations. It uses the OpenAPI standard to define interfaces and generate frameworks. + +Note: As of version V2.0.8, the TimechoDB installation package does not include the REST service JAR file by default. Please contact the Timecho team to obtain the corresponding JAR file before using this service, and place it in the `timechodb_home/lib` or `timechodb_home/ext/external_service` directory. + +## 1. Enabling RESTful Service + +The RESTful service is disabled by default. To enable it, locate the `conf/iotdb-system.properties` file in the IoTDB installation directory, set `enable_rest_service` to `true`, and then restart the datanode process. + +```Properties +enable_rest_service=true +``` + +## 2. Authentication + +Except for the health check endpoint `/ping`, the RESTful service uses basic authentication. Each URL request must include the header `'Authorization':'Basic' + base64.encode(username + ':' + password)`. + +In the example, the username is `root` and the password is `root`. The corresponding Basic authentication header format is: + +```Properties +Authorization : Basic cm9vdDpyb290 +``` + +- If the username or password authentication fails, the following information is returned: + + - HTTP status code: 801 + + - Response body: + + ```JSON + {"code":801,"message":"WRONG_LOGIN_PASSWORD"} + ``` + +- If the `Authorization` header is not set, the following information is returned: + + - HTTP status code: 800 + + - Response body: + + ```JSON + {"code":800,"message":"INIT_AUTH_ERROR"} + ``` + +## 3. Interface Definitions + +### 3.1 Ping + +The `/ping` endpoint can be used for online service health checks. + +- Request Method: GET + +- Request Path: `http://ip:port/ping` + +- Example Request: + + ```Bash + curl http://127.0.0.1:18080/ping + ``` + +- HTTP Status Codes: + + - `200`: The service is working normally and can accept external requests. + + - `503`: The service is experiencing issues and cannot accept external requests. + + | Parameter Name | Type | Description | + | :------------- | :------ | :--------------- | + | code | integer | Status Code | + | message | string | Code Information | + +- Response Example: + + - When the HTTP status code is `200`: + + ```JSON + { "code": 200, "message": "SUCCESS_STATUS"} + ``` + + - When the HTTP status code is `503`: + + ```JSON + { "code": 500, "message": "thrift service is unavailable"} + ``` + +**Note**: The `/ping` endpoint does not require authentication. + +### 3.2 Query Interface + +- Request Path: `/rest/table/v1/query` + +- Request Method: POST + +- Request Format: + + - Header: `application/json` + + - Request Parameters: + + | Parameter Name | Type | Required | Description | + | :------------- | :----- | :------- | :----------------------------------------------------------- | + | `database` | string | Yes | Database name | + | `sql` | string | Yes | SQL query | + | `row_limit` | int | No | Maximum number of rows to return in a single query. If not set, the default value from the configuration file (`rest_query_default_row_size_limit`) is used. If the result set exceeds this limit, status code `411` is returned. | + +- Response Format: + + | Parameter Name | Type | Description | + | :------------- | :---- | :----------------------------------------------------------- | + | `column_names` | array | Column names | + | `data_types` | array | Data types of each column | + | `values` | array | A 2D array where the first dimension represents rows, and the second dimension represents columns. Each element corresponds to a column, with the same length as `column_names`. | + +- Example Request: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","sql":"select s1,s2,s3 from test_table"}' http://127.0.0.1:18080/rest/table/v1/query + ``` + +- Example Response: + + ```JSON + { + "column_names": [ + "s1", + "s2", + "s3" + ], + "data_types": [ + "STRING", + "BOOLEAN", + "INT32" + ], + "values": [ + [ + "a11", + true, + 2024 + ], + [ + "a11", + false, + 2025 + ] + ] + } + ``` + +### 3.3 Non-Query Interface + +- Request Path: `/rest/table/v1/nonQuery` + +- Request Method: POST + +- Request Format: + + - Header: `application/json` + + - Request Parameters: + + | Parameter Name | Type | Required | Description | + | :------------- | :----- | :------- | :------------ | + | `sql` | string | Yes | SQL statement | + | `database` | string | No | Database name | + +- Response Format: + + | Parameter Name | Type | Description | + | :------------- | :------ | :---------- | + | `code` | integer | Status code | + | `message` | string | Message | + +- Example Requests: + + - Create a database: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"create database test","database":""}' http://127.0.0.1:18080/rest/table/v1/nonQuery + ``` + + - Create a table `test_table` in the `test` database: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"CREATE TABLE table1 (time TIMESTAMP TIME,region STRING TAG,plant_id STRING TAG,device_id STRING TAG,model_id STRING ATTRIBUTE,maintenance STRING ATTRIBUTE,temperature FLOAT FIELD,humidity FLOAT FIELD,status Boolean FIELD,arrival_time TIMESTAMP FIELD) WITH (TTL=31536000000)","database":"test"}' http://127.0.0.1:18080/rest/table/v1/nonQuery + ``` + +- Example Response: + + ```JSON + { + "code": 200, + "message": "SUCCESS_STATUS" + } + ``` + +### 3.4 Batch Write Interface + +- Request Path: `/rest/table/v1/insertTablet` + +- Request Method: POST + +- Request Format: + + - Header: `application/json` + + - Request Parameters: + + | Parameter Name | Type | Required | Description | + | :------------------ | :----- | :------- | :----------------------------------------------------------- | + | `database` | string | Yes | Database name | + | `table` | string | Yes | Table name | + | `column_names` | array | Yes | Column names | + | `column_categories` | array | Yes | Column categories (`TAG`, `FIELD`, `ATTRIBUTE`) | + | `data_types` | array | Yes | Data types | + | `timestamps` | array | Yes | Timestamp column | + | `values` | array | Yes | Value columns. Each column's values can be `null`. A 2D array where the first dimension corresponds to timestamps, and the second dimension corresponds to columns. | + +- Response Format: + + | Parameter Name | Type | Description | + | :------------- | :------ | :---------- | + | `code` | integer | Status code | + | `message` | string | Message | + +- Example Request: + + ```Bash + curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","column_categories":["TAG","FIELD","FIELD"],"timestamps":[1739702535000,1739789055000],"column_names":["s1","s2","s3"],"data_types":["STRING","BOOLEAN","INT32"],"values":[["a11",true,2024],["a11",false,2025]],"table":"test_table"}' http://127.0.0.1:18080/rest/table/v1/insertTablet + ``` + +- Example Response: + + ```JSON + { + "code": 200, + "message": "SUCCESS_STATUS" + } + ``` + +## 4. Configuration + +The configuration file is located in `iotdb-system.properties`. + +- Set `enable_rest_service` to `true` to enable the module, or `false` to disable it. The default value is `false`. + + ```Properties + enable_rest_service=true + ``` + +- Only effective when `enable_rest_service=true`. Set `rest_service_port` to a number (1025~65535) to customize the REST service socket port. The default value is `18080`. + + ```Properties + rest_service_port=18080 + ``` + +- Set `enable_swagger` to `true` to enable Swagger for displaying REST interface information, or `false` to disable it. The default value is `false`. + + ```Properties + enable_swagger=false + ``` + +- The maximum number of rows that can be returned in a single query. If the result set exceeds this limit, only the rows within the limit will be returned, and status code `411` will be returned. + + ```Properties + rest_query_default_row_size_limit=10000 + ``` + +- Expiration time for caching client login information (used to speed up user authentication, in seconds, default is 8 hours). + + ```Properties + cache_expire_in_seconds=28800 + ``` + +- Maximum number of users stored in the cache (default is 100). + + ```Properties + cache_max_num=100 + ``` + +- Initial cache capacity (default is 10). + + ```Properties + cache_init_num=10 + ``` + +- Whether to enable SSL configuration for the REST service. Set `enable_https` to `true` to enable it, or `false` to disable it. The default value is `false`. + + ```Properties + enable_https=false + ``` + +- Path to the `keyStore` (optional). + + ```Properties + key_store_path= + ``` + +- Password for the `keyStore` (optional). + + ```Properties + key_store_pwd= + ``` + +- Path to the `trustStore` (optional). + + ```Properties + trust_store_path="" + ``` + +- Password for the `trustStore` (optional). + + ```Properties + trust_store_pwd="" + ``` + +- SSL timeout time, in seconds. + + ```Properties + idle_timeout_in_seconds=5000 + ``` \ No newline at end of file diff --git a/src/UserGuide/latest-Table/Basic-Concept/Write-Updata-Data_apache.md b/src/UserGuide/latest-Table/Basic-Concept/Write-Updata-Data_apache.md index 193fa718b..d138f2a27 100644 --- a/src/UserGuide/latest-Table/Basic-Concept/Write-Updata-Data_apache.md +++ b/src/UserGuide/latest-Table/Basic-Concept/Write-Updata-Data_apache.md @@ -294,7 +294,7 @@ It costs 0.014s * **TAG** columns are optional. * The number of specified columns can be less than that of the target table. Missing columns will be automatically filled with `NULL` values. * For Java applications, the `INSERT INTO QUERY` statement can be executed using the [executeNonQueryStatement](../API/Programming-Java-Native-API_apache.md#_3-1-itablesession-interface) method. -* For REST API access, the `INSERT INTO QUERY` statement can be executed via the [/rest/table/v1/nonQuery](../API/RestAPI-V1.md#_3-3-non-query-interface) endpoint. +* For REST API access, the `INSERT INTO QUERY` statement can be executed via the [/rest/table/v1/nonQuery](../API/RestAPI-V1_apache.md#_3-3-non-query-interface) endpoint. * `INSERT INTO QUERY` does **not** support the `EXPLAIN` and `EXPLAIN ANALYZE` commands. * To execute the query write-back statement successfully, users must have the following permissions: * The `SELECT` permission on the source tables involved in the query. diff --git a/src/UserGuide/latest-Table/Basic-Concept/Write-Updata-Data_timecho.md b/src/UserGuide/latest-Table/Basic-Concept/Write-Updata-Data_timecho.md index ef146a655..8da2d8545 100644 --- a/src/UserGuide/latest-Table/Basic-Concept/Write-Updata-Data_timecho.md +++ b/src/UserGuide/latest-Table/Basic-Concept/Write-Updata-Data_timecho.md @@ -294,7 +294,7 @@ It costs 0.014s * **TAG** columns are optional. * The number of specified columns can be less than that of the target table. Missing columns will be automatically filled with `NULL` values. * For Java applications, the `INSERT INTO QUERY` statement can be executed using the [executeNonQueryStatement](../API/Programming-Java-Native-API_timecho.md#_3-1-itablesession-interface) method. -* For REST API access, the `INSERT INTO QUERY` statement can be executed via the [/rest/table/v1/nonQuery](../API/RestAPI-V1.md#_3-3-non-query-interface) endpoint. +* For REST API access, the `INSERT INTO QUERY` statement can be executed via the [/rest/table/v1/nonQuery](../API/RestAPI-V1_timecho.md#_3-3-non-query-interface) endpoint. * `INSERT INTO QUERY` does **not** support the `EXPLAIN` and `EXPLAIN ANALYZE` commands. * To execute the query write-back statement successfully, users must have the following permissions: * The `SELECT` permission on the source tables involved in the query. diff --git a/src/UserGuide/latest-Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md b/src/UserGuide/latest-Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md index 3c1742408..a0968c919 100644 --- a/src/UserGuide/latest-Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md +++ b/src/UserGuide/latest-Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md @@ -43,4 +43,6 @@ After unpacking the installation package（`iotdb-enterprise-{version}-bin.zip` | NOTICE | File | Open-source notice file. | | README_ZH.md | File | User manual (Chinese version). | | README.md | File | User manual (English version). | -| RELEASE_NOTES.md | File | Release notes. | \ No newline at end of file +| RELEASE_NOTES.md | File | Release notes. | + +Note: As of version V2.0.8, the TimechoDB installation package does not include the MQTT service and REST service JAR files by default. If you need to use them, please contact the Timecho team to obtain them. \ No newline at end of file diff --git a/src/UserGuide/latest/API/Programming-MQTT.md b/src/UserGuide/latest/API/Programming-MQTT.md index f170b502f..0a6a38616 100644 --- a/src/UserGuide/latest/API/Programming-MQTT.md +++ b/src/UserGuide/latest/API/Programming-MQTT.md @@ -1,3 +1,6 @@ +--- +redirectTo: Programming-MQTT_apache.html +--- -# MQTT Protocol - -## 1. Overview - -MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for IoT and low-bandwidth environments. It operates on a Publish/Subscribe (Pub/Sub) model, enabling efficient and reliable bidirectional communication between devices. Its core objectives are low power consumption, minimal bandwidth usage, and high real-time performance, making it ideal for unstable networks or resource-constrained scenarios (e.g., sensors, mobile devices). - -IoTDB provides deep integration with the MQTT protocol, fully compliant with MQTT v3.1 (OASIS International Standard). The IoTDB server includes a built-in high-performance MQTT Broker module, eliminating the need for third-party middleware. Devices can directly write time-series data into the IoTDB storage engine via MQTT messages. - - - - -## 2. Built-in MQTT Service -The Built-in MQTT Service provide the ability of direct connection to IoTDB through MQTT. It listen the publish messages from MQTT clients - and then write the data into storage immediately. -The MQTT topic corresponds to IoTDB timeseries. -The messages payload can be format to events by `PayloadFormatter` which loaded by java SPI, and the default implementation is `JSONPayloadFormatter`. -The default `json` formatter support two json format and its json array. The following is an MQTT message payload example: - -```json - { - "device":"root.sg.d1", - "timestamp":1586076045524, - "measurements":["s1","s2"], - "values":[0.530635,0.530635] - } -``` -or -```json - { - "device":"root.sg.d1", - "timestamps":[1586076045524,1586076065526], - "measurements":["s1","s2"], - "values":[[0.530635,0.530635], [0.530655,0.530695]] - } -``` -or json array of the above two. - - - -## 3. MQTT Configurations -The IoTDB MQTT service load configurations from `${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties` by default. - -Configurations are as follows: - -| **Property** | **Description** | **Default** | -| ------------------------ | -------------------------------------------------------------------------------------------------------------------- | ------------------- | -| `enable_mqtt_service` | Enable/ disable the MQTT service. | FALSE | -| `mqtt_host` | Host address bound to the MQTT service. | 127.0.0.1 | -| `mqtt_port` | Port bound to the MQTT service. | 1883 | -| `mqtt_handler_pool_size` | Thread pool size for processing MQTT messages. | 1 | -| **`mqtt_payload_formatter`** | **Formatting method for MQTT message payloads. ​**​**Options: `json` (tree mode), `line` (table mode).** | **json** | -| `mqtt_max_message_size` | Maximum allowed MQTT message size (bytes). | 1048576 | - -## 4. Coding Examples -The following is an example which a mqtt client send messages to IoTDB server. - -```java -MQTT mqtt = new MQTT(); -mqtt.setHost("127.0.0.1", 1883); -mqtt.setUserName("root"); -mqtt.setPassword("root"); - -BlockingConnection connection = mqtt.blockingConnection(); -connection.connect(); - -Random random = new Random(); -for (int i = 0; i < 10; i++) { - String payload = String.format("{\n" + - "\"device\":\"root.sg.d1\",\n" + - "\"timestamp\":%d,\n" + - "\"measurements\":[\"s1\"],\n" + - "\"values\":[%f]\n" + - "}", System.currentTimeMillis(), random.nextDouble()); - - connection.publish("root.sg.d1.s1", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -} - -connection.disconnect(); - -``` - -## 5. Customize your MQTT Message Format - -In a production environment, each device typically has its own MQTT client, and the message formats of these clients have been pre-defined. If communication is to be carried out in accordance with the MQTT message format supported by IoTDB, a comprehensive upgrade and transformation of all existing clients would be required, which would undoubtedly incur significant costs. However, we can easily achieve customization of the MQTT message format through simple programming means, without the need to modify the clients. -An example can be found in [example/mqtt-customize](https://github.com/apache/iotdb/tree/rc/2.0.1/example/mqtt-customize) project. - -Assuming the MQTT client sends the following message format: -```json - { - "time":1586076045523, - "deviceID":"car_1", - "deviceType":"Gasoline car​​", - "point":"Fuel level​​", - "value":10.0 -} -``` -Or in the form of an array of JSON: -```java -[ - { - "time":1586076045523, - "deviceID":"car_1", - "deviceType":"Gasoline car​​", - "point":"Fuel level", - "value":10.0 - }, - { - "time":1586076045524, - "deviceID":"car_2", - "deviceType":"NEV(new enegry vehicle)", - "point":"Speed", - "value":80.0 - } -] -``` - -Then you can set up the custom MQTT message format through the following steps: - -1. Create a java project, and add dependency: -```xml - - org.apache.iotdb - iotdb-server - 2.0.4-SNAPSHOT - -``` -2. Define your implementation which implements `org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` -e.g., - -```java -package org.apache.iotdb.mqtt.server; - -import org.apache.iotdb.db.protocol.mqtt.Message; -import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; -import org.apache.iotdb.db.protocol.mqtt.TableMessage; - -import com.google.common.collect.Lists; -import com.google.gson.Gson; -import com.google.gson.GsonBuilder; -import com.google.gson.JsonArray; -import com.google.gson.JsonElement; -import com.google.gson.JsonObject; -import com.google.gson.JsonParseException; -import io.netty.buffer.ByteBuf; -import org.apache.commons.lang3.NotImplementedException; -import org.apache.tsfile.enums.TSDataType; - -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -/** - * The Customized JSON payload formatter. one json format supported: { "time":1586076045523, - * "deviceID":"car_1", "deviceType":"NEV", "point":"Speed", "value":80.0 } - */ -public class CustomizedJsonPayloadFormatter implements PayloadFormatter { - private static final String JSON_KEY_TIME = "time"; - private static final String JSON_KEY_DEVICEID = "deviceID"; - private static final String JSON_KEY_DEVICETYPE = "deviceType"; - private static final String JSON_KEY_POINT = "point"; - private static final String JSON_KEY_VALUE = "value"; - private static final Gson GSON = new GsonBuilder().create(); - - @Override - public List format(String topic, ByteBuf payload) { - if (payload == null) { - return new ArrayList<>(); - } - String txt = payload.toString(StandardCharsets.UTF_8); - JsonElement jsonElement = GSON.fromJson(txt, JsonElement.class); - if (jsonElement.isJsonObject()) { - JsonObject jsonObject = jsonElement.getAsJsonObject(); - return formatTableRow(topic, jsonObject); - } else if (jsonElement.isJsonArray()) { - JsonArray jsonArray = jsonElement.getAsJsonArray(); - List messages = new ArrayList<>(); - for (JsonElement element : jsonArray) { - JsonObject jsonObject = element.getAsJsonObject(); - messages.addAll(formatTableRow(topic, jsonObject)); - } - return messages; - } - throw new JsonParseException("payload is invalidate"); - } - - @Override - @Deprecated - public List format(ByteBuf payload) { - throw new NotImplementedException(); - } - - private List formatTableRow(String topic, JsonObject jsonObject) { - TableMessage message = new TableMessage(); - String database = !topic.contains("/") ? topic : topic.substring(0, topic.indexOf("/")); - String table = "test_table"; - - // Parsing Database Name - message.setDatabase((database)); - - // Parsing Table Name - message.setTable(table); - - // Parsing Tags - List tagKeys = new ArrayList<>(); - tagKeys.add(JSON_KEY_DEVICEID); - List tagValues = new ArrayList<>(); - tagValues.add(jsonObject.get(JSON_KEY_DEVICEID).getAsString()); - message.setTagKeys(tagKeys); - message.setTagValues(tagValues); - - // Parsing Attributes - List attributeKeys = new ArrayList<>(); - List attributeValues = new ArrayList<>(); - attributeKeys.add(JSON_KEY_DEVICETYPE); - attributeValues.add(jsonObject.get(JSON_KEY_DEVICETYPE).getAsString()); - message.setAttributeKeys(attributeKeys); - message.setAttributeValues(attributeValues); - - // Parsing Fields - List fields = Arrays.asList(JSON_KEY_POINT); - List dataTypes = Arrays.asList(TSDataType.FLOAT); - List values = Arrays.asList(jsonObject.get(JSON_KEY_VALUE).getAsFloat()); - message.setFields(fields); - message.setDataTypes(dataTypes); - message.setValues(values); - - // Parsing timestamp - message.setTimestamp(jsonObject.get(JSON_KEY_TIME).getAsLong()); - return Lists.newArrayList(message); - } - - @Override - public String getName() { - // set the value of mqtt_payload_formatter in iotdb-common.properties as the following string: - return "CustomizedJson2Table"; - } - - @Override - public String getType() { - return PayloadFormatter.TABLE_TYPE; - } -} -``` -3. modify the file in `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter`: - clean the file and put your implementation class name into the file. - In this example, the content is: `org.apache.iotdb.mqtt.server.CustomizedJsonPayloadFormatter` -4. compile your implementation as a jar file: `mvn package -DskipTests` - - -Then, in your server: -1. Create ${IOTDB_HOME}/ext/mqtt/ folder, and put the jar into this folder. -2. Update configuration to enable MQTT service. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) -3. Set the value of `mqtt_payload_formatter` in `conf/iotdb-system.properties` as the value of getName() in your implementation - , in this example, the value is `CustomizedJson2Table` -4. Launch the IoTDB server. -5. Now IoTDB will use your implementation to parse the MQTT message. - -More: the message format can be anything you want. For example, if it is a binary format, -just use `payload.forEachByte()` or `payload.array` to get bytes content. - - -## 6. Caution - -To avoid compatibility issues caused by a default client_id, always explicitly supply a unique, non-empty client_id in every MQTT client. -Behavior varies when the client_id is missing or empty. Common examples: -1. Explicitly sending an empty string - • MQTTX: When client_id="", IoTDB silently discards the message. - • mosquitto_pub: When client_id="", IoTDB receives the message normally. -2. Omitting client_id entirely - • MQTTX: IoTDB accepts the message. - • mosquitto_pub: IoTDB rejects the connection. - Therefore, explicitly assigning a unique, non-empty client_id is the simplest way to eliminate these discrepancies and ensure reliable message delivery. +--> \ No newline at end of file diff --git a/src/UserGuide/latest/API/Programming-MQTT_apache.md b/src/UserGuide/latest/API/Programming-MQTT_apache.md new file mode 100644 index 000000000..f170b502f --- /dev/null +++ b/src/UserGuide/latest/API/Programming-MQTT_apache.md @@ -0,0 +1,293 @@ + +# MQTT Protocol + +## 1. Overview + +MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for IoT and low-bandwidth environments. It operates on a Publish/Subscribe (Pub/Sub) model, enabling efficient and reliable bidirectional communication between devices. Its core objectives are low power consumption, minimal bandwidth usage, and high real-time performance, making it ideal for unstable networks or resource-constrained scenarios (e.g., sensors, mobile devices). + +IoTDB provides deep integration with the MQTT protocol, fully compliant with MQTT v3.1 (OASIS International Standard). The IoTDB server includes a built-in high-performance MQTT Broker module, eliminating the need for third-party middleware. Devices can directly write time-series data into the IoTDB storage engine via MQTT messages. + + + + +## 2. Built-in MQTT Service +The Built-in MQTT Service provide the ability of direct connection to IoTDB through MQTT. It listen the publish messages from MQTT clients + and then write the data into storage immediately. +The MQTT topic corresponds to IoTDB timeseries. +The messages payload can be format to events by `PayloadFormatter` which loaded by java SPI, and the default implementation is `JSONPayloadFormatter`. +The default `json` formatter support two json format and its json array. The following is an MQTT message payload example: + +```json + { + "device":"root.sg.d1", + "timestamp":1586076045524, + "measurements":["s1","s2"], + "values":[0.530635,0.530635] + } +``` +or +```json + { + "device":"root.sg.d1", + "timestamps":[1586076045524,1586076065526], + "measurements":["s1","s2"], + "values":[[0.530635,0.530635], [0.530655,0.530695]] + } +``` +or json array of the above two. + + + +## 3. MQTT Configurations +The IoTDB MQTT service load configurations from `${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties` by default. + +Configurations are as follows: + +| **Property** | **Description** | **Default** | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------- | ------------------- | +| `enable_mqtt_service` | Enable/ disable the MQTT service. | FALSE | +| `mqtt_host` | Host address bound to the MQTT service. | 127.0.0.1 | +| `mqtt_port` | Port bound to the MQTT service. | 1883 | +| `mqtt_handler_pool_size` | Thread pool size for processing MQTT messages. | 1 | +| **`mqtt_payload_formatter`** | **Formatting method for MQTT message payloads. ​**​**Options: `json` (tree mode), `line` (table mode).** | **json** | +| `mqtt_max_message_size` | Maximum allowed MQTT message size (bytes). | 1048576 | + +## 4. Coding Examples +The following is an example which a mqtt client send messages to IoTDB server. + +```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +connection.connect(); + +Random random = new Random(); +for (int i = 0; i < 10; i++) { + String payload = String.format("{\n" + + "\"device\":\"root.sg.d1\",\n" + + "\"timestamp\":%d,\n" + + "\"measurements\":[\"s1\"],\n" + + "\"values\":[%f]\n" + + "}", System.currentTimeMillis(), random.nextDouble()); + + connection.publish("root.sg.d1.s1", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +} + +connection.disconnect(); + +``` + +## 5. Customize your MQTT Message Format + +In a production environment, each device typically has its own MQTT client, and the message formats of these clients have been pre-defined. If communication is to be carried out in accordance with the MQTT message format supported by IoTDB, a comprehensive upgrade and transformation of all existing clients would be required, which would undoubtedly incur significant costs. However, we can easily achieve customization of the MQTT message format through simple programming means, without the need to modify the clients. +An example can be found in [example/mqtt-customize](https://github.com/apache/iotdb/tree/rc/2.0.1/example/mqtt-customize) project. + +Assuming the MQTT client sends the following message format: +```json + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"Gasoline car​​", + "point":"Fuel level​​", + "value":10.0 +} +``` +Or in the form of an array of JSON: +```java +[ + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"Gasoline car​​", + "point":"Fuel level", + "value":10.0 + }, + { + "time":1586076045524, + "deviceID":"car_2", + "deviceType":"NEV(new enegry vehicle)", + "point":"Speed", + "value":80.0 + } +] +``` + +Then you can set up the custom MQTT message format through the following steps: + +1. Create a java project, and add dependency: +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. Define your implementation which implements `org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` +e.g., + +```java +package org.apache.iotdb.mqtt.server; + +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; +import org.apache.iotdb.db.protocol.mqtt.TableMessage; + +import com.google.common.collect.Lists; +import com.google.gson.Gson; +import com.google.gson.GsonBuilder; +import com.google.gson.JsonArray; +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParseException; +import io.netty.buffer.ByteBuf; +import org.apache.commons.lang3.NotImplementedException; +import org.apache.tsfile.enums.TSDataType; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +/** + * The Customized JSON payload formatter. one json format supported: { "time":1586076045523, + * "deviceID":"car_1", "deviceType":"NEV", "point":"Speed", "value":80.0 } + */ +public class CustomizedJsonPayloadFormatter implements PayloadFormatter { + private static final String JSON_KEY_TIME = "time"; + private static final String JSON_KEY_DEVICEID = "deviceID"; + private static final String JSON_KEY_DEVICETYPE = "deviceType"; + private static final String JSON_KEY_POINT = "point"; + private static final String JSON_KEY_VALUE = "value"; + private static final Gson GSON = new GsonBuilder().create(); + + @Override + public List format(String topic, ByteBuf payload) { + if (payload == null) { + return new ArrayList<>(); + } + String txt = payload.toString(StandardCharsets.UTF_8); + JsonElement jsonElement = GSON.fromJson(txt, JsonElement.class); + if (jsonElement.isJsonObject()) { + JsonObject jsonObject = jsonElement.getAsJsonObject(); + return formatTableRow(topic, jsonObject); + } else if (jsonElement.isJsonArray()) { + JsonArray jsonArray = jsonElement.getAsJsonArray(); + List messages = new ArrayList<>(); + for (JsonElement element : jsonArray) { + JsonObject jsonObject = element.getAsJsonObject(); + messages.addAll(formatTableRow(topic, jsonObject)); + } + return messages; + } + throw new JsonParseException("payload is invalidate"); + } + + @Override + @Deprecated + public List format(ByteBuf payload) { + throw new NotImplementedException(); + } + + private List formatTableRow(String topic, JsonObject jsonObject) { + TableMessage message = new TableMessage(); + String database = !topic.contains("/") ? topic : topic.substring(0, topic.indexOf("/")); + String table = "test_table"; + + // Parsing Database Name + message.setDatabase((database)); + + // Parsing Table Name + message.setTable(table); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add(JSON_KEY_DEVICEID); + List tagValues = new ArrayList<>(); + tagValues.add(jsonObject.get(JSON_KEY_DEVICEID).getAsString()); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add(JSON_KEY_DEVICETYPE); + attributeValues.add(jsonObject.get(JSON_KEY_DEVICETYPE).getAsString()); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList(JSON_KEY_POINT); + List dataTypes = Arrays.asList(TSDataType.FLOAT); + List values = Arrays.asList(jsonObject.get(JSON_KEY_VALUE).getAsFloat()); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + // Parsing timestamp + message.setTimestamp(jsonObject.get(JSON_KEY_TIME).getAsLong()); + return Lists.newArrayList(message); + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-common.properties as the following string: + return "CustomizedJson2Table"; + } + + @Override + public String getType() { + return PayloadFormatter.TABLE_TYPE; + } +} +``` +3. modify the file in `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter`: + clean the file and put your implementation class name into the file. + In this example, the content is: `org.apache.iotdb.mqtt.server.CustomizedJsonPayloadFormatter` +4. compile your implementation as a jar file: `mvn package -DskipTests` + + +Then, in your server: +1. Create ${IOTDB_HOME}/ext/mqtt/ folder, and put the jar into this folder. +2. Update configuration to enable MQTT service. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. Set the value of `mqtt_payload_formatter` in `conf/iotdb-system.properties` as the value of getName() in your implementation + , in this example, the value is `CustomizedJson2Table` +4. Launch the IoTDB server. +5. Now IoTDB will use your implementation to parse the MQTT message. + +More: the message format can be anything you want. For example, if it is a binary format, +just use `payload.forEachByte()` or `payload.array` to get bytes content. + + +## 6. Caution + +To avoid compatibility issues caused by a default client_id, always explicitly supply a unique, non-empty client_id in every MQTT client. +Behavior varies when the client_id is missing or empty. Common examples: +1. Explicitly sending an empty string + • MQTTX: When client_id="", IoTDB silently discards the message. + • mosquitto_pub: When client_id="", IoTDB receives the message normally. +2. Omitting client_id entirely + • MQTTX: IoTDB accepts the message. + • mosquitto_pub: IoTDB rejects the connection. + Therefore, explicitly assigning a unique, non-empty client_id is the simplest way to eliminate these discrepancies and ensure reliable message delivery. diff --git a/src/UserGuide/latest/API/Programming-MQTT_timecho.md b/src/UserGuide/latest/API/Programming-MQTT_timecho.md new file mode 100644 index 000000000..56338a087 --- /dev/null +++ b/src/UserGuide/latest/API/Programming-MQTT_timecho.md @@ -0,0 +1,294 @@ + +# MQTT Protocol + +## 1. Overview + +MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for IoT and low-bandwidth environments. It operates on a Publish/Subscribe (Pub/Sub) model, enabling efficient and reliable bidirectional communication between devices. Its core objectives are low power consumption, minimal bandwidth usage, and high real-time performance, making it ideal for unstable networks or resource-constrained scenarios (e.g., sensors, mobile devices). + +IoTDB provides deep integration with the MQTT protocol, fully compliant with MQTT v3.1 (OASIS International Standard). The IoTDB server includes a built-in high-performance MQTT Broker module, eliminating the need for third-party middleware. Devices can directly write time-series data into the IoTDB storage engine via MQTT messages. + + + +Note: As of version V2.0.8, the TimechoDB installation package does not include the MQTT service JAR file by default. Please contact the Timecho team to obtain the JAR file before using this service, and place it in the `timechodb_home/lib` or `timechodb_home/ext/external_service` directory. + +## 2. Built-in MQTT Service +The Built-in MQTT Service provide the ability of direct connection to IoTDB through MQTT. It listen the publish messages from MQTT clients + and then write the data into storage immediately. +The MQTT topic corresponds to IoTDB timeseries. +The messages payload can be format to events by `PayloadFormatter` which loaded by java SPI, and the default implementation is `JSONPayloadFormatter`. +The default `json` formatter support two json format and its json array. The following is an MQTT message payload example: + +```json + { + "device":"root.sg.d1", + "timestamp":1586076045524, + "measurements":["s1","s2"], + "values":[0.530635,0.530635] + } +``` +or +```json + { + "device":"root.sg.d1", + "timestamps":[1586076045524,1586076065526], + "measurements":["s1","s2"], + "values":[[0.530635,0.530635], [0.530655,0.530695]] + } +``` +or json array of the above two. + + + +## 3. MQTT Configurations +The IoTDB MQTT service load configurations from `${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties` by default. + +Configurations are as follows: + +| **Property** | **Description** | **Default** | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------- | ------------------- | +| `enable_mqtt_service` | Enable/ disable the MQTT service. | FALSE | +| `mqtt_host` | Host address bound to the MQTT service. | 127.0.0.1 | +| `mqtt_port` | Port bound to the MQTT service. | 1883 | +| `mqtt_handler_pool_size` | Thread pool size for processing MQTT messages. | 1 | +| **`mqtt_payload_formatter`** | **Formatting method for MQTT message payloads. ​**​**Options: `json` (tree mode), `line` (table mode).** | **json** | +| `mqtt_max_message_size` | Maximum allowed MQTT message size (bytes). | 1048576 | + +## 4. Coding Examples +The following is an example which a mqtt client send messages to IoTDB server. + +```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +connection.connect(); + +Random random = new Random(); +for (int i = 0; i < 10; i++) { + String payload = String.format("{\n" + + "\"device\":\"root.sg.d1\",\n" + + "\"timestamp\":%d,\n" + + "\"measurements\":[\"s1\"],\n" + + "\"values\":[%f]\n" + + "}", System.currentTimeMillis(), random.nextDouble()); + + connection.publish("root.sg.d1.s1", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +} + +connection.disconnect(); + +``` + +## 5. Customize your MQTT Message Format + +In a production environment, each device typically has its own MQTT client, and the message formats of these clients have been pre-defined. If communication is to be carried out in accordance with the MQTT message format supported by IoTDB, a comprehensive upgrade and transformation of all existing clients would be required, which would undoubtedly incur significant costs. However, we can easily achieve customization of the MQTT message format through simple programming means, without the need to modify the clients. +An example can be found in [example/mqtt-customize](https://github.com/apache/iotdb/tree/rc/2.0.1/example/mqtt-customize) project. + +Assuming the MQTT client sends the following message format: +```json + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"Gasoline car​​", + "point":"Fuel level​​", + "value":10.0 +} +``` +Or in the form of an array of JSON: +```java +[ + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"Gasoline car​​", + "point":"Fuel level", + "value":10.0 + }, + { + "time":1586076045524, + "deviceID":"car_2", + "deviceType":"NEV(new enegry vehicle)", + "point":"Speed", + "value":80.0 + } +] +``` + +Then you can set up the custom MQTT message format through the following steps: + +1. Create a java project, and add dependency: +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. Define your implementation which implements `org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` +e.g., + +```java +package org.apache.iotdb.mqtt.server; + +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; +import org.apache.iotdb.db.protocol.mqtt.TableMessage; + +import com.google.common.collect.Lists; +import com.google.gson.Gson; +import com.google.gson.GsonBuilder; +import com.google.gson.JsonArray; +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParseException; +import io.netty.buffer.ByteBuf; +import org.apache.commons.lang3.NotImplementedException; +import org.apache.tsfile.enums.TSDataType; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +/** + * The Customized JSON payload formatter. one json format supported: { "time":1586076045523, + * "deviceID":"car_1", "deviceType":"NEV", "point":"Speed", "value":80.0 } + */ +public class CustomizedJsonPayloadFormatter implements PayloadFormatter { + private static final String JSON_KEY_TIME = "time"; + private static final String JSON_KEY_DEVICEID = "deviceID"; + private static final String JSON_KEY_DEVICETYPE = "deviceType"; + private static final String JSON_KEY_POINT = "point"; + private static final String JSON_KEY_VALUE = "value"; + private static final Gson GSON = new GsonBuilder().create(); + + @Override + public List format(String topic, ByteBuf payload) { + if (payload == null) { + return new ArrayList<>(); + } + String txt = payload.toString(StandardCharsets.UTF_8); + JsonElement jsonElement = GSON.fromJson(txt, JsonElement.class); + if (jsonElement.isJsonObject()) { + JsonObject jsonObject = jsonElement.getAsJsonObject(); + return formatTableRow(topic, jsonObject); + } else if (jsonElement.isJsonArray()) { + JsonArray jsonArray = jsonElement.getAsJsonArray(); + List messages = new ArrayList<>(); + for (JsonElement element : jsonArray) { + JsonObject jsonObject = element.getAsJsonObject(); + messages.addAll(formatTableRow(topic, jsonObject)); + } + return messages; + } + throw new JsonParseException("payload is invalidate"); + } + + @Override + @Deprecated + public List format(ByteBuf payload) { + throw new NotImplementedException(); + } + + private List formatTableRow(String topic, JsonObject jsonObject) { + TableMessage message = new TableMessage(); + String database = !topic.contains("/") ? topic : topic.substring(0, topic.indexOf("/")); + String table = "test_table"; + + // Parsing Database Name + message.setDatabase((database)); + + // Parsing Table Name + message.setTable(table); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add(JSON_KEY_DEVICEID); + List tagValues = new ArrayList<>(); + tagValues.add(jsonObject.get(JSON_KEY_DEVICEID).getAsString()); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add(JSON_KEY_DEVICETYPE); + attributeValues.add(jsonObject.get(JSON_KEY_DEVICETYPE).getAsString()); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList(JSON_KEY_POINT); + List dataTypes = Arrays.asList(TSDataType.FLOAT); + List values = Arrays.asList(jsonObject.get(JSON_KEY_VALUE).getAsFloat()); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + // Parsing timestamp + message.setTimestamp(jsonObject.get(JSON_KEY_TIME).getAsLong()); + return Lists.newArrayList(message); + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-common.properties as the following string: + return "CustomizedJson2Table"; + } + + @Override + public String getType() { + return PayloadFormatter.TABLE_TYPE; + } +} +``` +3. modify the file in `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter`: + clean the file and put your implementation class name into the file. + In this example, the content is: `org.apache.iotdb.mqtt.server.CustomizedJsonPayloadFormatter` +4. compile your implementation as a jar file: `mvn package -DskipTests` + + +Then, in your server: +1. Create ${IOTDB_HOME}/ext/mqtt/ folder, and put the jar into this folder. +2. Update configuration to enable MQTT service. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. Set the value of `mqtt_payload_formatter` in `conf/iotdb-system.properties` as the value of getName() in your implementation + , in this example, the value is `CustomizedJson2Table` +4. Launch the IoTDB server. +5. Now IoTDB will use your implementation to parse the MQTT message. + +More: the message format can be anything you want. For example, if it is a binary format, +just use `payload.forEachByte()` or `payload.array` to get bytes content. + + +## 6. Caution + +To avoid compatibility issues caused by a default client_id, always explicitly supply a unique, non-empty client_id in every MQTT client. +Behavior varies when the client_id is missing or empty. Common examples: +1. Explicitly sending an empty string + • MQTTX: When client_id="", IoTDB silently discards the message. + • mosquitto_pub: When client_id="", IoTDB receives the message normally. +2. Omitting client_id entirely + • MQTTX: IoTDB accepts the message. + • mosquitto_pub: IoTDB rejects the connection. + Therefore, explicitly assigning a unique, non-empty client_id is the simplest way to eliminate these discrepancies and ensure reliable message delivery. diff --git a/src/UserGuide/latest/API/RestServiceV1_timecho.md b/src/UserGuide/latest/API/RestServiceV1_timecho.md index a10bacce8..c85ae256c 100644 --- a/src/UserGuide/latest/API/RestServiceV1_timecho.md +++ b/src/UserGuide/latest/API/RestServiceV1_timecho.md @@ -22,6 +22,8 @@ # REST API V1(Not Recommend) IoTDB's RESTful services can be used for query, write, and management operations, using the OpenAPI standard to define interfaces and generate frameworks. +Note: As of version V2.0.8, the TimechoDB installation package does not include the REST service JAR file by default. Please contact the Timecho team to obtain the corresponding JAR file before using this service, and place it in the `timechodb_home/lib` or `timechodb_home/ext/external_service` directory. + ## 1. Enable RESTful Services RESTful services are disabled by default. diff --git a/src/UserGuide/latest/API/RestServiceV2_timecho.md b/src/UserGuide/latest/API/RestServiceV2_timecho.md index cda9f3f0d..69c0d5efe 100644 --- a/src/UserGuide/latest/API/RestServiceV2_timecho.md +++ b/src/UserGuide/latest/API/RestServiceV2_timecho.md @@ -22,6 +22,8 @@ # REST API V2 IoTDB's RESTful services can be used for query, write, and management operations, using the OpenAPI standard to define interfaces and generate frameworks. +Note: As of version V2.0.8, the TimechoDB installation package does not include the REST service JAR file by default. Please contact the Timecho team to obtain the corresponding JAR file before using this service, and place it in the `timechodb_home/lib` or `timechodb_home/ext/external_service` directory. + ## 1. Enable RESTful Services RESTful services are disabled by default. diff --git a/src/UserGuide/latest/Basic-Concept/Write-Data_apache.md b/src/UserGuide/latest/Basic-Concept/Write-Data_apache.md index 35ebef8b5..60d6aae88 100644 --- a/src/UserGuide/latest/Basic-Concept/Write-Data_apache.md +++ b/src/UserGuide/latest/Basic-Concept/Write-Data_apache.md @@ -177,7 +177,7 @@ Example： ## 4. MQTT WRITE -Refer to [Built-in MQTT Service](../API/Programming-MQTT.md#built-in-mqtt-service) +Refer to [Built-in MQTT Service](../API/Programming-MQTT_apache.md#_2-built-in-mqtt-service) ## 5. BATCH DATA LOAD diff --git a/src/UserGuide/latest/Basic-Concept/Write-Data_timecho.md b/src/UserGuide/latest/Basic-Concept/Write-Data_timecho.md index 380ab5037..7cbd7d021 100644 --- a/src/UserGuide/latest/Basic-Concept/Write-Data_timecho.md +++ b/src/UserGuide/latest/Basic-Concept/Write-Data_timecho.md @@ -177,7 +177,7 @@ Example： ## 4. MQTT WRITE -Refer to [Built-in MQTT Service](../API/Programming-MQTT.md#built-in-mqtt-service) +Refer to [Built-in MQTT Service](../API/Programming-MQTT_timecho.md#_2-built-in-mqtt-service) ## 5. BATCH DATA LOAD diff --git a/src/UserGuide/latest/Deployment-and-Maintenance/IoTDB-Package_timecho.md b/src/UserGuide/latest/Deployment-and-Maintenance/IoTDB-Package_timecho.md index 3c1742408..a0968c919 100644 --- a/src/UserGuide/latest/Deployment-and-Maintenance/IoTDB-Package_timecho.md +++ b/src/UserGuide/latest/Deployment-and-Maintenance/IoTDB-Package_timecho.md @@ -43,4 +43,6 @@ After unpacking the installation package（`iotdb-enterprise-{version}-bin.zip` | NOTICE | File | Open-source notice file. | | README_ZH.md | File | User manual (Chinese version). | | README.md | File | User manual (English version). | -| RELEASE_NOTES.md | File | Release notes. | \ No newline at end of file +| RELEASE_NOTES.md | File | Release notes. | + +Note: As of version V2.0.8, the TimechoDB installation package does not include the MQTT service and REST service JAR files by default. If you need to use them, please contact the Timecho team to obtain them. \ No newline at end of file diff --git a/src/zh/UserGuide/Master/Table/API/Programming-MQTT.md b/src/zh/UserGuide/Master/Table/API/Programming-MQTT.md index 8f283f862..0a6a38616 100644 --- a/src/zh/UserGuide/Master/Table/API/Programming-MQTT.md +++ b/src/zh/UserGuide/Master/Table/API/Programming-MQTT.md @@ -1,3 +1,6 @@ +--- +redirectTo: Programming-MQTT_apache.html +--- -# MQTT 协议 - -## 1. 概述 - -MQTT 是一种专为物联网（IoT）和低带宽环境设计的轻量级消息传输协议，基于发布/订阅（Pub/Sub）模型，支持设备间高效、可靠的双向通信。其核心目标是低功耗、低带宽消耗和高实时性，尤其适合网络不稳定或资源受限的场景（如传感器、移动设备）。 - -IoTDB 深度集成了 MQTT 协议能力，完整兼容 MQTT v3.1（OASIS 国际标准协议）。IoTDB 服务器内置高性能 MQTT Broker 服务模块，无需第三方中间件，支持设备通过 MQTT 报文将时序数据直接写入 IoTDB 存储引擎。 - -![](/img/mqtt-table-1.png) - -## 2. 配置方式 - -默认情况下，IoTDB MQTT 服务通过`${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`加载配置。 - -具体配置项如下： - -| **名称** | **描述** | **默认** | -|---------------------------| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | -| `enable_mqtt_service` | 是否启用 mqtt 服务 | FALSE | -| `mqtt_host` | mqtt 服务绑定主机 | 127.0.0.1 | -| `mqtt_port` | mqtt 服务绑定端口 | 1883 | -| `mqtt_handler_pool_size` | 处理 mqtt 消息的处理程序池大小 | 1 | -| **`mqtt_payload_formatter`** | **mqtt**​**​ 消息有效负载格式化程序。**​**可选项：**​​**`json`**​**：仅适用于树模型。**​​**`line`**​**：仅适用于表模型。** | **json** | -| `mqtt_max_message_size` | mqtt 消息最大长度（字节） | 1048576 | - -## 3. 写入协议 - -* 行协议语法格式 - -```JavaScript -[,=[,=]][ =[,=]] =[,=] [] -``` - -* 示例 - -```JavaScript -myMeasurement,tag1=value1,tag2=value2 attr1=value1,attr2=value2 fieldKey="fieldValue" 1556813561098000000 -``` - -![](/img/mqtt-table-2.png) - - -## 4. 命名约定 - -* 数据库名称 - -MQTT topic 名称用 `/` 分割后， 第一串内容作为数据库名称。 - -```Properties -topic: stock/Legacy -databaseName: stock - - -topic: stock/Legacy/# -databaseName:stock -``` - -* 表名称 - -表名称使用行协议中的 ``。 - -* 类型标识 - -| Filed 内容 | IoTDB 数据类型 | -|--------------------------------------------------------------| ---------------- | -| 1
1.12 | DOUBLE | -| 1`f`
1.12`f` | FLOAT | -| 1`i`
123`i` | INT64 | -| 1`u`
123`u` | INT64| -| 1`i32`
123`i32` | INT32 | -| `"xxx"` | TEXT | -| `t`,`T`,`true`,`True`,`TRUE`
`f`,`F`,`false`,`False`,`FALSE` | BOOLEAN | - - -## 5. 代码示例 -以下是 mqtt 客户端将消息发送到 IoTDB 服务器的示例。 - - ```java -MQTT mqtt = new MQTT(); -mqtt.setHost("127.0.0.1", 1883); -mqtt.setUserName("root"); -mqtt.setPassword("root"); - -BlockingConnection connection = mqtt.blockingConnection(); -String DATABASE = "myMqttTest"; -connection.connect(); - -String payload = - "test1,tag1=t1,tag2=t2 attr3=a5,attr4=a4 field1=\"fieldValue1\",field2=1i,field3=1u 1"; -connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -payload = "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 2"; -connection.publish(DATABASE, payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -payload = "# It's a remark\n " + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 6"; - connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); - Thread.sleep(10); - -//批量写入示例 -payload = - "test1,tag1=t1,tag2=t2 field7=t,field8=T,field9=true 3 \n " - + "test1,tag1=t1,tag2=t2 field7=f,field8=F,field9=FALSE 4"; -connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -//批量写入示例 -payload = - "test1,tag1=t1,tag2=t2 attr1=a1,attr2=a2 field1=\"fieldValue1\",field2=1i,field3=1u 4 \n " - + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 5"; -connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -connection.disconnect(); - ``` - - -## 6. 自定义 MQTT 消息格式 - -事实上可以通过简单编程来实现 MQTT 消息的格式自定义。 -可以在源码的 [example/mqtt-customize](https://github.com/apache/iotdb/tree/master/example/mqtt-customize) 项目中找到一个简单示例。 - -步骤: -1. 创建一个 Java 项目，增加如下依赖 -```xml - - org.apache.iotdb - iotdb-server - 2.0.4-SNAPSHOT - -``` -2. 创建一个实现类，实现接口 `org.apache.iotdb.db.mqtt.protocol.PayloadFormatter` - -```java -package org.apache.iotdb.mqtt.server; - -import io.netty.buffer.ByteBuf; -import org.apache.iotdb.db.protocol.mqtt.Message; -import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; - -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -public class CustomizedLinePayloadFormatter implements PayloadFormatter { - - @Override - public List format(String topic, ByteBuf payload) { - // Suppose the payload is a line format - if (payload == null) { - return null; - } - - String line = payload.toString(StandardCharsets.UTF_8); - // parse data from the line and generate Messages and put them into List ret - List ret = new ArrayList<>(); - // this is just an example, so we just generate some Messages directly - for (int i = 0; i < 3; i++) { - long ts = i; - TableMessage message = new TableMessage(); - - // Parsing Database Name - message.setDatabase("db" + i); - - //Parsing Table Names - message.setTable("t" + i); - - // Parsing Tags - List tagKeys = new ArrayList<>(); - tagKeys.add("tag1" + i); - tagKeys.add("tag2" + i); - List tagValues = new ArrayList<>(); - tagValues.add("t_value1" + i); - tagValues.add("t_value2" + i); - message.setTagKeys(tagKeys); - message.setTagValues(tagValues); - - // Parsing Attributes - List attributeKeys = new ArrayList<>(); - List attributeValues = new ArrayList<>(); - attributeKeys.add("attr1" + i); - attributeKeys.add("attr2" + i); - attributeValues.add("a_value1" + i); - attributeValues.add("a_value2" + i); - message.setAttributeKeys(attributeKeys); - message.setAttributeValues(attributeValues); - - // Parsing Fields - List fields = Arrays.asList("field1" + i, "field2" + i); - List dataTypes = Arrays.asList(TSDataType.FLOAT, TSDataType.FLOAT); - List values = Arrays.asList("4.0" + i, "5.0" + i); - message.setFields(fields); - message.setDataTypes(dataTypes); - message.setValues(values); - - //// Parsing timestamp - message.setTimestamp(ts); - ret.add(message); - } - return ret; - } - - @Override - public String getName() { - // set the value of mqtt_payload_formatter in iotdb-system.properties as the following string: - return "CustomizedLine"; - } -} -``` - - -3. 修改项目中的 `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` 文件: - 将示例中的文件内容清除，并将刚才的实现类的全名（包名.类名）写入文件中。注意，这个文件中只有一行。 - 在本例中，文件内容为: `org.apache.iotdb.mqtt.server.CustomizedLinePayloadFormatter` -4. 编译项目生成一个 jar 包: `mvn package -DskipTests` - - -在 IoTDB 服务端: -1. 创建 ${IOTDB_HOME}/ext/mqtt/ 文件夹, 将刚才的 jar 包放入此文件夹。 -2. 打开 MQTT 服务参数. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) -3. 用刚才的实现类中的 getName() 方法的返回值 设置为 `conf/iotdb-system.properties` 中 `mqtt_payload_formatter` 的值， - , 在本例中，为 `CustomizedLine` -4. 启动 IoTDB -5. 搞定 - -More: MQTT 协议的消息不限于 line，你还可以用任意二进制。通过如下函数获得： -`payload.forEachByte()` or `payload.array`。 - -## 7. 注意事项 - -为避免因缺省client_id引发的兼容性问题，强烈建议在所有MQTT客户端中始终显式地提供唯一且非空的 client_id。 -不同客户端在client_id缺失或为空时的表现并不一致，常见示例如下： -1. 显式传入空字符串 -• MQTTX：client_id=""时，IoTDB会直接丢弃消息； -• mosquitto_pub：client_id=""时，IoTDB能正常接收消息。 -2. 完全不传client_id -• MQTTX：消息可被IoTDB正常接收； -• mosquitto_pub：IoTDB拒绝连接。 -由此可见，显式指定唯一且非空的client_id是消除上述差异、确保消息可靠投递的最简单做法。 - +--> \ No newline at end of file diff --git a/src/zh/UserGuide/Master/Table/API/Programming-MQTT_apache.md b/src/zh/UserGuide/Master/Table/API/Programming-MQTT_apache.md new file mode 100644 index 000000000..b243ff3eb --- /dev/null +++ b/src/zh/UserGuide/Master/Table/API/Programming-MQTT_apache.md @@ -0,0 +1,262 @@ + +# MQTT 协议 + +## 1. 概述 + +MQTT 是一种专为物联网（IoT）和低带宽环境设计的轻量级消息传输协议，基于发布/订阅（Pub/Sub）模型，支持设备间高效、可靠的双向通信。其核心目标是低功耗、低带宽消耗和高实时性，尤其适合网络不稳定或资源受限的场景（如传感器、移动设备）。 + +IoTDB 深度集成了 MQTT 协议能力，完整兼容 MQTT v3.1（OASIS 国际标准协议）。IoTDB 服务器内置高性能 MQTT Broker 服务模块，无需第三方中间件，支持设备通过 MQTT 报文将时序数据直接写入 IoTDB 存储引擎。 + +![](/img/mqtt-table-1.png) + +## 2. 配置方式 + +默认情况下，IoTDB MQTT 服务通过`${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`加载配置。 + +具体配置项如下： + +| **名称** | **描述** | **默认** | +|---------------------------| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | +| `enable_mqtt_service` | 是否启用 mqtt 服务 | FALSE | +| `mqtt_host` | mqtt 服务绑定主机 | 127.0.0.1 | +| `mqtt_port` | mqtt 服务绑定端口 | 1883 | +| `mqtt_handler_pool_size` | 处理 mqtt 消息的处理程序池大小 | 1 | +| **`mqtt_payload_formatter`** | **mqtt**​**​ 消息有效负载格式化程序。**​**可选项：**​​**`json`**​**：仅适用于树模型。**​​**`line`**​**：仅适用于表模型。** | **json** | +| `mqtt_max_message_size` | mqtt 消息最大长度（字节） | 1048576 | + +## 3. 写入协议 + +* 行协议语法格式 + +```JavaScript +[,=[,=]][ =[,=]] =[,=] [] +``` + +* 示例 + +```JavaScript +myMeasurement,tag1=value1,tag2=value2 attr1=value1,attr2=value2 fieldKey="fieldValue" 1556813561098000000 +``` + +![](/img/mqtt-table-2.png) + + +## 4. 命名约定 + +* 数据库名称 + +MQTT topic 名称用 `/` 分割后， 第一串内容作为数据库名称。 + +```Properties +topic: stock/Legacy +databaseName: stock + + +topic: stock/Legacy/# +databaseName:stock +``` + +* 表名称 + +表名称使用行协议中的 ``。 + +* 类型标识 + +| Filed 内容 | IoTDB 数据类型 | +|--------------------------------------------------------------| ---------------- | +| 1
1.12 | DOUBLE | +| 1`f`
1.12`f` | FLOAT | +| 1`i`
123`i` | INT64 | +| 1`u`
123`u` | INT64| +| 1`i32`
123`i32` | INT32 | +| `"xxx"` | TEXT | +| `t`,`T`,`true`,`True`,`TRUE`
`f`,`F`,`false`,`False`,`FALSE` | BOOLEAN | + + +## 5. 代码示例 +以下是 mqtt 客户端将消息发送到 IoTDB 服务器的示例。 + + ```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +String DATABASE = "myMqttTest"; +connection.connect(); + +String payload = + "test1,tag1=t1,tag2=t2 attr3=a5,attr4=a4 field1=\"fieldValue1\",field2=1i,field3=1u 1"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 2"; +connection.publish(DATABASE, payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "# It's a remark\n " + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 6"; + connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); + Thread.sleep(10); + +//批量写入示例 +payload = + "test1,tag1=t1,tag2=t2 field7=t,field8=T,field9=true 3 \n " + + "test1,tag1=t1,tag2=t2 field7=f,field8=F,field9=FALSE 4"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +//批量写入示例 +payload = + "test1,tag1=t1,tag2=t2 attr1=a1,attr2=a2 field1=\"fieldValue1\",field2=1i,field3=1u 4 \n " + + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 5"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +connection.disconnect(); + ``` + + +## 6. 自定义 MQTT 消息格式 + +事实上可以通过简单编程来实现 MQTT 消息的格式自定义。 +可以在源码的 [example/mqtt-customize](https://github.com/apache/iotdb/tree/master/example/mqtt-customize) 项目中找到一个简单示例。 + +步骤: +1. 创建一个 Java 项目，增加如下依赖 +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. 创建一个实现类，实现接口 `org.apache.iotdb.db.mqtt.protocol.PayloadFormatter` + +```java +package org.apache.iotdb.mqtt.server; + +import io.netty.buffer.ByteBuf; +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +public class CustomizedLinePayloadFormatter implements PayloadFormatter { + + @Override + public List format(String topic, ByteBuf payload) { + // Suppose the payload is a line format + if (payload == null) { + return null; + } + + String line = payload.toString(StandardCharsets.UTF_8); + // parse data from the line and generate Messages and put them into List ret + List ret = new ArrayList<>(); + // this is just an example, so we just generate some Messages directly + for (int i = 0; i < 3; i++) { + long ts = i; + TableMessage message = new TableMessage(); + + // Parsing Database Name + message.setDatabase("db" + i); + + //Parsing Table Names + message.setTable("t" + i); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add("tag1" + i); + tagKeys.add("tag2" + i); + List tagValues = new ArrayList<>(); + tagValues.add("t_value1" + i); + tagValues.add("t_value2" + i); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add("attr1" + i); + attributeKeys.add("attr2" + i); + attributeValues.add("a_value1" + i); + attributeValues.add("a_value2" + i); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList("field1" + i, "field2" + i); + List dataTypes = Arrays.asList(TSDataType.FLOAT, TSDataType.FLOAT); + List values = Arrays.asList("4.0" + i, "5.0" + i); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + //// Parsing timestamp + message.setTimestamp(ts); + ret.add(message); + } + return ret; + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-system.properties as the following string: + return "CustomizedLine"; + } +} +``` + + +3. 修改项目中的 `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` 文件: + 将示例中的文件内容清除，并将刚才的实现类的全名（包名.类名）写入文件中。注意，这个文件中只有一行。 + 在本例中，文件内容为: `org.apache.iotdb.mqtt.server.CustomizedLinePayloadFormatter` +4. 编译项目生成一个 jar 包: `mvn package -DskipTests` + + +在 IoTDB 服务端: +1. 创建 ${IOTDB_HOME}/ext/mqtt/ 文件夹, 将刚才的 jar 包放入此文件夹。 +2. 打开 MQTT 服务参数. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. 用刚才的实现类中的 getName() 方法的返回值 设置为 `conf/iotdb-system.properties` 中 `mqtt_payload_formatter` 的值， + , 在本例中，为 `CustomizedLine` +4. 启动 IoTDB +5. 搞定 + +More: MQTT 协议的消息不限于 line，你还可以用任意二进制。通过如下函数获得： +`payload.forEachByte()` or `payload.array`。 + + +## 7. 注意事项 + +为避免因缺省client_id引发的兼容性问题，强烈建议在所有MQTT客户端中始终显式地提供唯一且非空的 client_id。 +不同客户端在client_id缺失或为空时的表现并不一致，常见示例如下： +1. 显式传入空字符串 +• MQTTX：client_id=""时，IoTDB会直接丢弃消息； +• mosquitto_pub：client_id=""时，IoTDB能正常接收消息。 +2. 完全不传client_id +• MQTTX：消息可被IoTDB正常接收； +• mosquitto_pub：IoTDB拒绝连接。 +由此可见，显式指定唯一且非空的client_id是消除上述差异、确保消息可靠投递的最简单做法。 \ No newline at end of file diff --git a/src/zh/UserGuide/Master/Table/API/Programming-MQTT_timecho.md b/src/zh/UserGuide/Master/Table/API/Programming-MQTT_timecho.md new file mode 100644 index 000000000..7076d466b --- /dev/null +++ b/src/zh/UserGuide/Master/Table/API/Programming-MQTT_timecho.md @@ -0,0 +1,264 @@ + +# MQTT 协议 + +## 1. 概述 + +MQTT 是一种专为物联网（IoT）和低带宽环境设计的轻量级消息传输协议，基于发布/订阅（Pub/Sub）模型，支持设备间高效、可靠的双向通信。其核心目标是低功耗、低带宽消耗和高实时性，尤其适合网络不稳定或资源受限的场景（如传感器、移动设备）。 + +IoTDB 深度集成了 MQTT 协议能力，完整兼容 MQTT v3.1（OASIS 国际标准协议）。IoTDB 服务器内置高性能 MQTT Broker 服务模块，无需第三方中间件，支持设备通过 MQTT 报文将时序数据直接写入 IoTDB 存储引擎。 + +![](/img/mqtt-table-1.png) + +注意，自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 MQTT 服务的 JAR 包。请使用该服务前联系天谋团队获取 JAR 包，并放置于 timechodb_home/lib 或者 timechodb_home/ext/external_service 路径下。 + +## 2. 配置方式 + +默认情况下，IoTDB MQTT 服务通过`${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`加载配置。 + +具体配置项如下： + +| **名称** | **描述** | **默认** | +|---------------------------| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | +| `enable_mqtt_service` | 是否启用 mqtt 服务 | FALSE | +| `mqtt_host` | mqtt 服务绑定主机 | 127.0.0.1 | +| `mqtt_port` | mqtt 服务绑定端口 | 1883 | +| `mqtt_handler_pool_size` | 处理 mqtt 消息的处理程序池大小 | 1 | +| **`mqtt_payload_formatter`** | **mqtt**​**​ 消息有效负载格式化程序。**​**可选项：**​​**`json`**​**：仅适用于树模型。**​​**`line`**​**：仅适用于表模型。** | **json** | +| `mqtt_max_message_size` | mqtt 消息最大长度（字节） | 1048576 | + +## 3. 写入协议 + +* 行协议语法格式 + +```JavaScript +[,=[,=]][ =[,=]] =[,=] [] +``` + +* 示例 + +```JavaScript +myMeasurement,tag1=value1,tag2=value2 attr1=value1,attr2=value2 fieldKey="fieldValue" 1556813561098000000 +``` + +![](/img/mqtt-table-2.png) + + +## 4. 命名约定 + +* 数据库名称 + +MQTT topic 名称用 `/` 分割后， 第一串内容作为数据库名称。 + +```Properties +topic: stock/Legacy +databaseName: stock + + +topic: stock/Legacy/# +databaseName:stock +``` + +* 表名称 + +表名称使用行协议中的 ``。 + +* 类型标识 + +| Filed 内容 | IoTDB 数据类型 | +|--------------------------------------------------------------| ---------------- | +| 1
1.12 | DOUBLE | +| 1`f`
1.12`f` | FLOAT | +| 1`i`
123`i` | INT64 | +| 1`u`
123`u` | INT64| +| 1`i32`
123`i32` | INT32 | +| `"xxx"` | TEXT | +| `t`,`T`,`true`,`True`,`TRUE`
`f`,`F`,`false`,`False`,`FALSE` | BOOLEAN | + + +## 5. 代码示例 +以下是 mqtt 客户端将消息发送到 IoTDB 服务器的示例。 + + ```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +String DATABASE = "myMqttTest"; +connection.connect(); + +String payload = + "test1,tag1=t1,tag2=t2 attr3=a5,attr4=a4 field1=\"fieldValue1\",field2=1i,field3=1u 1"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 2"; +connection.publish(DATABASE, payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "# It's a remark\n " + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 6"; + connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); + Thread.sleep(10); + +//批量写入示例 +payload = + "test1,tag1=t1,tag2=t2 field7=t,field8=T,field9=true 3 \n " + + "test1,tag1=t1,tag2=t2 field7=f,field8=F,field9=FALSE 4"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +//批量写入示例 +payload = + "test1,tag1=t1,tag2=t2 attr1=a1,attr2=a2 field1=\"fieldValue1\",field2=1i,field3=1u 4 \n " + + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 5"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +connection.disconnect(); + ``` + + +## 6. 自定义 MQTT 消息格式 + +事实上可以通过简单编程来实现 MQTT 消息的格式自定义。 +可以在源码的 [example/mqtt-customize](https://github.com/apache/iotdb/tree/master/example/mqtt-customize) 项目中找到一个简单示例。 + +步骤: +1. 创建一个 Java 项目，增加如下依赖 +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. 创建一个实现类，实现接口 `org.apache.iotdb.db.mqtt.protocol.PayloadFormatter` + +```java +package org.apache.iotdb.mqtt.server; + +import io.netty.buffer.ByteBuf; +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +public class CustomizedLinePayloadFormatter implements PayloadFormatter { + + @Override + public List format(String topic, ByteBuf payload) { + // Suppose the payload is a line format + if (payload == null) { + return null; + } + + String line = payload.toString(StandardCharsets.UTF_8); + // parse data from the line and generate Messages and put them into List ret + List ret = new ArrayList<>(); + // this is just an example, so we just generate some Messages directly + for (int i = 0; i < 3; i++) { + long ts = i; + TableMessage message = new TableMessage(); + + // Parsing Database Name + message.setDatabase("db" + i); + + //Parsing Table Names + message.setTable("t" + i); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add("tag1" + i); + tagKeys.add("tag2" + i); + List tagValues = new ArrayList<>(); + tagValues.add("t_value1" + i); + tagValues.add("t_value2" + i); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add("attr1" + i); + attributeKeys.add("attr2" + i); + attributeValues.add("a_value1" + i); + attributeValues.add("a_value2" + i); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList("field1" + i, "field2" + i); + List dataTypes = Arrays.asList(TSDataType.FLOAT, TSDataType.FLOAT); + List values = Arrays.asList("4.0" + i, "5.0" + i); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + //// Parsing timestamp + message.setTimestamp(ts); + ret.add(message); + } + return ret; + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-system.properties as the following string: + return "CustomizedLine"; + } +} +``` + + +3. 修改项目中的 `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` 文件: + 将示例中的文件内容清除，并将刚才的实现类的全名（包名.类名）写入文件中。注意，这个文件中只有一行。 + 在本例中，文件内容为: `org.apache.iotdb.mqtt.server.CustomizedLinePayloadFormatter` +4. 编译项目生成一个 jar 包: `mvn package -DskipTests` + + +在 IoTDB 服务端: +1. 创建 ${IOTDB_HOME}/ext/mqtt/ 文件夹, 将刚才的 jar 包放入此文件夹。 +2. 打开 MQTT 服务参数. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. 用刚才的实现类中的 getName() 方法的返回值 设置为 `conf/iotdb-system.properties` 中 `mqtt_payload_formatter` 的值， + , 在本例中，为 `CustomizedLine` +4. 启动 IoTDB +5. 搞定 + +More: MQTT 协议的消息不限于 line，你还可以用任意二进制。通过如下函数获得： +`payload.forEachByte()` or `payload.array`。 + + +## 7. 注意事项 + +为避免因缺省client_id引发的兼容性问题，强烈建议在所有MQTT客户端中始终显式地提供唯一且非空的 client_id。 +不同客户端在client_id缺失或为空时的表现并不一致，常见示例如下： +1. 显式传入空字符串 +• MQTTX：client_id=""时，IoTDB会直接丢弃消息； +• mosquitto_pub：client_id=""时，IoTDB能正常接收消息。 +2. 完全不传client_id +• MQTTX：消息可被IoTDB正常接收； +• mosquitto_pub：IoTDB拒绝连接。 +由此可见，显式指定唯一且非空的client_id是消除上述差异、确保消息可靠投递的最简单做法。 \ No newline at end of file diff --git a/src/zh/UserGuide/Master/Table/API/RestServiceV1.md b/src/zh/UserGuide/Master/Table/API/RestServiceV1.md index 16fe30443..ff8429cae 100644 --- a/src/zh/UserGuide/Master/Table/API/RestServiceV1.md +++ b/src/zh/UserGuide/Master/Table/API/RestServiceV1.md @@ -1,3 +1,6 @@ +--- +redirectTo: RestServiceV1_apache.html +--- - -# RestAPI V1 - -IoTDB 的 RESTful 服务可用于查询、写入和管理操作，它使用 OpenAPI 标准来定义接口并生成框架。 - -## 1. 开启 RESTful 服务 - -Restful 服务默认情况是关闭的，开启 restful 功能需要找到 IoTDB 安装目录下的`conf/iotdb-system.properties`文件，将 `enable_rest_service` 设置为 `true` ，然后重启 datanode 进程。 - -```Plain - enable_rest_service=true -``` - -## 2. 鉴权 - -除了检活接口 `/ping`，restful服务使用了基础（basic）鉴权，每次 URL 请求都需要在 header 中携带 `'Authorization':'Basic' + base64.encode(username + ':' + password)`。 - -示例中使用的用户名为：`root`，密码为：`root`，对应的 Basic 鉴权 Header 格式为 - -```Plain -Authorization : Basic cm9vdDpyb290 -``` - -- 若用户名密码认证失败，则返回如下信息： -- HTTP 状态码：801 -- 返回结构体如下 - -```JSON -{"code":801,"message":"WRONG_LOGIN_PASSWORD"} -``` - -- 若未设置 `Authorization`，则返回如下信息： -- HTTP 状态码：800 -- 返回结构体如下 - -```Plain -{"code":800,"message":"INIT_AUTH_ERROR"} -``` - -## 3. 接口定义 - -### 3.1 ping - -ping 接口可以用于线上服务检活。 - -请求方式：`GET` - -请求路径：http://ip:port/ping - -请求示例： - -```Plain -curl http://127.0.0.1:18080/ping -``` - -返回的 HTTP 状态码： - -- `200`：当前服务工作正常，可以接收外部请求。 -- `503`：当前服务出现异常，不能接收外部请求。 - -响应参数： - -| 参数名称 | 参数类型 | 参数描述 | -| -------- | -------- | -------- | -| code | integer | 状态码 | -| message | string | 信息提示 | - -响应示例： - -- HTTP 状态码为 `200` 时： - -```JSON -{ "code": 200, "message": "SUCCESS_STATUS"} -``` - -- HTTP 状态码为 `503` 时： - -```JSON -{ "code": 500, "message": "thrift service is unavailable"} -``` - -注意：`ping` 接口访问不需要鉴权。 - -### 3.2 查询接口 - -- 请求地址：`/rest/table/v1/query` - -- 请求方式：post - -- Request 格式 - -请求头:`application/json` - -请求参数说明 - -| 参数名称 | 参数类型 | 是否必填 | 参数描述 | -| --------- | -------- | -------- | ------------------------------------------------------------ | -| database | string | 是 | 数据库名称 | -| sql | string | 是 | | -| row_limit | int | 否 | 一次查询能返回的结果集的最大行数。 如果不设置该参数，将使用配置文件的 `rest_query_default_row_size_limit` 作为默认值。 当返回结果集的行数超出限制时，将返回状态码 `411`。 | - -- Response 格式 - -| 参数名称 | 参数类型 | 参数描述 | -| ------------ | -------- | ------------------------------------------------------------ | -| column_names | array | 列名 | -| data_types | array | 每一列的类型 | -| values | array | 二维数组，第一维与结果集所有行，第二维数组代表结果集的每一行，每一个元素为一列，长度与column_names的长度相同。 | - -- 请求示例 - -```JSON -curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","sql":"select s1,s2,s3 from test_table"}' http://127.0.0.1:18080/rest/table/v1/query -``` - -- 响应示例： - -```JSON -{ - "column_names": [ - "s1", - "s2", - "s3" - ], - "data_types": [ - "STRING", - "BOOLEAN", - "INT32" - ], - "values": [ - [ - "a11", - true, - 2024 - ], - [ - "a11", - false, - 2025 - ] - ] -} -``` - -### 3.3 非查询接口 - -- 请求地址：`/rest/table/v1/nonQuery` - -- 请求方式：post - -- Request 格式 - - - 请求头:`application/json` - - - 请求参数说明 - -| 参数名称 | 参数类型 | 是否必填 | 参数描述 | -| -------- | -------- | -------- | -------- | -| sql | string | 是 | | -| database | string | 否 | 数据库名 | - -- Response 格式 - -| 参数名称 | 参数类型 | 参数描述 | -| -------- | -------- | -------- | -| code | integer | 状态码 | -| message | string | 信息提示 | - -- 请求示例 - -```JSON -#创建数据库 -curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"create database test","database":""}' http://127.0.0.1:18080/rest/table/v1/nonQuery -#在test库中创建表test_table -curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"CREATE TABLE table1 (time TIMESTAMP TIME,region STRING TAG,plant_id STRING TAG,device_id STRING TAG,model_id STRING ATTRIBUTE,maintenance STRING ATTRIBUTE,temperature FLOAT FIELD,humidity FLOAT FIELD,status Boolean FIELD,arrival_time TIMESTAMP FIELD) WITH (TTL=31536000000)","database":"test"}' http://127.0.0.1:18080/rest/table/v1/nonQuery -``` - -- 响应示例： - -```JSON -{ - "code": 200, - "message": "SUCCESS_STATUS" -} -``` - -### 3.4 批量写入接口 - -- 请求地址：`/rest/table/v1/insertTablet` - -- 请求方式：post - -- Request 格式 - -请求头:`application/json` - -请求参数说明 - -| 参数名称 | 参数类型 | 是否必填 | 参数描述 | -| ----------------- | -------- | -------- | ------------------------------------------------------------ | -| database | string | 是 | 数据库名称 | -| table | string | 是 | 表名 | -| column_names | array | 是 | 列名 | -| column_categories | array | 是 | 列类别(TAG,FIELD,*ATTRIBUTE*) | -| data_types | array | 是 | 数据类型 | -| timestamps | array | 是 | 时间列 | -| values | array | 是 | 值列，每一列中的值可以为 `null`二维数组第一层长度跟timestamps长度相同。第二层长度跟column_names长度相同 | - -- Response 格式 - -响应参数: - -| 参数名称 | 参数类型 | 参数描述 | -| -------- | -------- | -------- | -| code | integer | 状态码 | -| message | string | 信息提示 | - -- 请求示例 - -```JSON -curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","column_categories":["TAG","FIELD","FIELD"],"timestamps":[1739702535000,1739789055000],"column_names":["s1","s2","s3"],"data_types":["STRING","BOOLEAN","INT32"],"values":[["a11",true,2024],["a11",false,2025]],"table":"test_table"}' http://127.0.0.1:18080/rest/table/v1/insertTablet -``` - -- 响应示例 - -```JSON -{ - "code": 200, - "message": "SUCCESS_STATUS" -} -``` - - -## 4. 配置 - -配置文件位于 `iotdb-system.properties` 中。 - -- 将 `enable_rest_service` 设置为 `true` 表示启用该模块，而将 `false` 表示禁用该模块。默认情况下，该值为 `false`。 - -```Plain -enable_rest_service=true -``` - -- 仅在 `enable_rest_service=true` 时生效。将 `rest_service_port `设置为数字（1025~65535），以自定义REST服务套接字端口，默认情况下，值为 `18080`。 - -```Plain -rest_service_port=18080 -``` - -- 将 'enable_swagger' 设置 'true' 启用swagger来展示rest接口信息, 而设置为 'false' 关闭该功能. 默认情况下，该值为 `false`。 - -```Plain -enable_swagger=false -``` - -- 一次查询能返回的结果集最大行数。当返回结果集的行数超出参数限制时，您只会得到在行数范围内的结果集，且将得到状态码`411`。 - -```Plain -rest_query_default_row_size_limit=10000 -``` - -- 缓存客户登录信息的过期时间（用于加速用户鉴权的速度，单位为秒，默认是8个小时） - -```Plain -cache_expire_in_seconds=28800 -``` - -- 缓存中存储的最大用户数量（默认是100） - -```Plain -cache_max_num=100 -``` - -- 缓存初始容量（默认是10） - -```Plain -cache_init_num=10 -``` - -- REST Service 是否开启 SSL 配置，将 `enable_https` 设置为 `true` 以启用该模块，而将 `false` 设置为禁用该模块。默认情况下，该值为 `false`。 - -```Plain -enable_https=false -``` - -- keyStore 所在路径（非必填） - -```Plain -key_store_path= -``` - -- keyStore 密码（非必填） - -```Plain -key_store_pwd= -``` - -- trustStore 所在路径（非必填） - -```Plain -trust_store_path="" -``` - -- trustStore 密码（非必填） - -```Plain -trust_store_pwd="" -``` - -- SSL 超时时间，单位为秒 - -```Plain -idle_timeout_in_seconds=5000 -``` \ No newline at end of file +--> \ No newline at end of file diff --git a/src/zh/UserGuide/Master/Table/API/RestServiceV1_apache.md b/src/zh/UserGuide/Master/Table/API/RestServiceV1_apache.md new file mode 100644 index 000000000..16fe30443 --- /dev/null +++ b/src/zh/UserGuide/Master/Table/API/RestServiceV1_apache.md @@ -0,0 +1,334 @@ + + +# RestAPI V1 + +IoTDB 的 RESTful 服务可用于查询、写入和管理操作，它使用 OpenAPI 标准来定义接口并生成框架。 + +## 1. 开启 RESTful 服务 + +Restful 服务默认情况是关闭的，开启 restful 功能需要找到 IoTDB 安装目录下的`conf/iotdb-system.properties`文件，将 `enable_rest_service` 设置为 `true` ，然后重启 datanode 进程。 + +```Plain + enable_rest_service=true +``` + +## 2. 鉴权 + +除了检活接口 `/ping`，restful服务使用了基础（basic）鉴权，每次 URL 请求都需要在 header 中携带 `'Authorization':'Basic' + base64.encode(username + ':' + password)`。 + +示例中使用的用户名为：`root`，密码为：`root`，对应的 Basic 鉴权 Header 格式为 + +```Plain +Authorization : Basic cm9vdDpyb290 +``` + +- 若用户名密码认证失败，则返回如下信息： +- HTTP 状态码：801 +- 返回结构体如下 + +```JSON +{"code":801,"message":"WRONG_LOGIN_PASSWORD"} +``` + +- 若未设置 `Authorization`，则返回如下信息： +- HTTP 状态码：800 +- 返回结构体如下 + +```Plain +{"code":800,"message":"INIT_AUTH_ERROR"} +``` + +## 3. 接口定义 + +### 3.1 ping + +ping 接口可以用于线上服务检活。 + +请求方式：`GET` + +请求路径：http://ip:port/ping + +请求示例： + +```Plain +curl http://127.0.0.1:18080/ping +``` + +返回的 HTTP 状态码： + +- `200`：当前服务工作正常，可以接收外部请求。 +- `503`：当前服务出现异常，不能接收外部请求。 + +响应参数： + +| 参数名称 | 参数类型 | 参数描述 | +| -------- | -------- | -------- | +| code | integer | 状态码 | +| message | string | 信息提示 | + +响应示例： + +- HTTP 状态码为 `200` 时： + +```JSON +{ "code": 200, "message": "SUCCESS_STATUS"} +``` + +- HTTP 状态码为 `503` 时： + +```JSON +{ "code": 500, "message": "thrift service is unavailable"} +``` + +注意：`ping` 接口访问不需要鉴权。 + +### 3.2 查询接口 + +- 请求地址：`/rest/table/v1/query` + +- 请求方式：post + +- Request 格式 + +请求头:`application/json` + +请求参数说明 + +| 参数名称 | 参数类型 | 是否必填 | 参数描述 | +| --------- | -------- | -------- | ------------------------------------------------------------ | +| database | string | 是 | 数据库名称 | +| sql | string | 是 | | +| row_limit | int | 否 | 一次查询能返回的结果集的最大行数。 如果不设置该参数，将使用配置文件的 `rest_query_default_row_size_limit` 作为默认值。 当返回结果集的行数超出限制时，将返回状态码 `411`。 | + +- Response 格式 + +| 参数名称 | 参数类型 | 参数描述 | +| ------------ | -------- | ------------------------------------------------------------ | +| column_names | array | 列名 | +| data_types | array | 每一列的类型 | +| values | array | 二维数组，第一维与结果集所有行，第二维数组代表结果集的每一行，每一个元素为一列，长度与column_names的长度相同。 | + +- 请求示例 + +```JSON +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","sql":"select s1,s2,s3 from test_table"}' http://127.0.0.1:18080/rest/table/v1/query +``` + +- 响应示例： + +```JSON +{ + "column_names": [ + "s1", + "s2", + "s3" + ], + "data_types": [ + "STRING", + "BOOLEAN", + "INT32" + ], + "values": [ + [ + "a11", + true, + 2024 + ], + [ + "a11", + false, + 2025 + ] + ] +} +``` + +### 3.3 非查询接口 + +- 请求地址：`/rest/table/v1/nonQuery` + +- 请求方式：post + +- Request 格式 + + - 请求头:`application/json` + + - 请求参数说明 + +| 参数名称 | 参数类型 | 是否必填 | 参数描述 | +| -------- | -------- | -------- | -------- | +| sql | string | 是 | | +| database | string | 否 | 数据库名 | + +- Response 格式 + +| 参数名称 | 参数类型 | 参数描述 | +| -------- | -------- | -------- | +| code | integer | 状态码 | +| message | string | 信息提示 | + +- 请求示例 + +```JSON +#创建数据库 +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"create database test","database":""}' http://127.0.0.1:18080/rest/table/v1/nonQuery +#在test库中创建表test_table +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"CREATE TABLE table1 (time TIMESTAMP TIME,region STRING TAG,plant_id STRING TAG,device_id STRING TAG,model_id STRING ATTRIBUTE,maintenance STRING ATTRIBUTE,temperature FLOAT FIELD,humidity FLOAT FIELD,status Boolean FIELD,arrival_time TIMESTAMP FIELD) WITH (TTL=31536000000)","database":"test"}' http://127.0.0.1:18080/rest/table/v1/nonQuery +``` + +- 响应示例： + +```JSON +{ + "code": 200, + "message": "SUCCESS_STATUS" +} +``` + +### 3.4 批量写入接口 + +- 请求地址：`/rest/table/v1/insertTablet` + +- 请求方式：post + +- Request 格式 + +请求头:`application/json` + +请求参数说明 + +| 参数名称 | 参数类型 | 是否必填 | 参数描述 | +| ----------------- | -------- | -------- | ------------------------------------------------------------ | +| database | string | 是 | 数据库名称 | +| table | string | 是 | 表名 | +| column_names | array | 是 | 列名 | +| column_categories | array | 是 | 列类别(TAG,FIELD,*ATTRIBUTE*) | +| data_types | array | 是 | 数据类型 | +| timestamps | array | 是 | 时间列 | +| values | array | 是 | 值列，每一列中的值可以为 `null`二维数组第一层长度跟timestamps长度相同。第二层长度跟column_names长度相同 | + +- Response 格式 + +响应参数: + +| 参数名称 | 参数类型 | 参数描述 | +| -------- | -------- | -------- | +| code | integer | 状态码 | +| message | string | 信息提示 | + +- 请求示例 + +```JSON +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","column_categories":["TAG","FIELD","FIELD"],"timestamps":[1739702535000,1739789055000],"column_names":["s1","s2","s3"],"data_types":["STRING","BOOLEAN","INT32"],"values":[["a11",true,2024],["a11",false,2025]],"table":"test_table"}' http://127.0.0.1:18080/rest/table/v1/insertTablet +``` + +- 响应示例 + +```JSON +{ + "code": 200, + "message": "SUCCESS_STATUS" +} +``` + + +## 4. 配置 + +配置文件位于 `iotdb-system.properties` 中。 + +- 将 `enable_rest_service` 设置为 `true` 表示启用该模块，而将 `false` 表示禁用该模块。默认情况下，该值为 `false`。 + +```Plain +enable_rest_service=true +``` + +- 仅在 `enable_rest_service=true` 时生效。将 `rest_service_port `设置为数字（1025~65535），以自定义REST服务套接字端口，默认情况下，值为 `18080`。 + +```Plain +rest_service_port=18080 +``` + +- 将 'enable_swagger' 设置 'true' 启用swagger来展示rest接口信息, 而设置为 'false' 关闭该功能. 默认情况下，该值为 `false`。 + +```Plain +enable_swagger=false +``` + +- 一次查询能返回的结果集最大行数。当返回结果集的行数超出参数限制时，您只会得到在行数范围内的结果集，且将得到状态码`411`。 + +```Plain +rest_query_default_row_size_limit=10000 +``` + +- 缓存客户登录信息的过期时间（用于加速用户鉴权的速度，单位为秒，默认是8个小时） + +```Plain +cache_expire_in_seconds=28800 +``` + +- 缓存中存储的最大用户数量（默认是100） + +```Plain +cache_max_num=100 +``` + +- 缓存初始容量（默认是10） + +```Plain +cache_init_num=10 +``` + +- REST Service 是否开启 SSL 配置，将 `enable_https` 设置为 `true` 以启用该模块，而将 `false` 设置为禁用该模块。默认情况下，该值为 `false`。 + +```Plain +enable_https=false +``` + +- keyStore 所在路径（非必填） + +```Plain +key_store_path= +``` + +- keyStore 密码（非必填） + +```Plain +key_store_pwd= +``` + +- trustStore 所在路径（非必填） + +```Plain +trust_store_path="" +``` + +- trustStore 密码（非必填） + +```Plain +trust_store_pwd="" +``` + +- SSL 超时时间，单位为秒 + +```Plain +idle_timeout_in_seconds=5000 +``` \ No newline at end of file diff --git a/src/zh/UserGuide/Master/Table/API/RestServiceV1_timecho.md b/src/zh/UserGuide/Master/Table/API/RestServiceV1_timecho.md new file mode 100644 index 000000000..70c5eba40 --- /dev/null +++ b/src/zh/UserGuide/Master/Table/API/RestServiceV1_timecho.md @@ -0,0 +1,336 @@ + + +# RestAPI V1 + +IoTDB 的 RESTful 服务可用于查询、写入和管理操作，它使用 OpenAPI 标准来定义接口并生成框架。 + +注意：自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 REST 服务的 JAR 包，请使用该服务前联系天谋团队获取相应的 JAR 包，并放置于 timechodb_home/lib 或者 timechodb_home/ext/external_service 路径下。 + +## 1. 开启 RESTful 服务 + +Restful 服务默认情况是关闭的，开启 restful 功能需要找到 IoTDB 安装目录下的`conf/iotdb-system.properties`文件，将 `enable_rest_service` 设置为 `true` ，然后重启 datanode 进程。 + +```Plain + enable_rest_service=true +``` + +## 2. 鉴权 + +除了检活接口 `/ping`，restful服务使用了基础（basic）鉴权，每次 URL 请求都需要在 header 中携带 `'Authorization':'Basic' + base64.encode(username + ':' + password)`。 + +示例中使用的用户名为：`root`，密码为：`root`，对应的 Basic 鉴权 Header 格式为 + +```Plain +Authorization : Basic cm9vdDpyb290 +``` + +- 若用户名密码认证失败，则返回如下信息： +- HTTP 状态码：801 +- 返回结构体如下 + +```JSON +{"code":801,"message":"WRONG_LOGIN_PASSWORD"} +``` + +- 若未设置 `Authorization`，则返回如下信息： +- HTTP 状态码：800 +- 返回结构体如下 + +```Plain +{"code":800,"message":"INIT_AUTH_ERROR"} +``` + +## 3. 接口定义 + +### 3.1 ping + +ping 接口可以用于线上服务检活。 + +请求方式：`GET` + +请求路径：http://ip:port/ping + +请求示例： + +```Plain +curl http://127.0.0.1:18080/ping +``` + +返回的 HTTP 状态码： + +- `200`：当前服务工作正常，可以接收外部请求。 +- `503`：当前服务出现异常，不能接收外部请求。 + +响应参数： + +| 参数名称 | 参数类型 | 参数描述 | +| -------- | -------- | -------- | +| code | integer | 状态码 | +| message | string | 信息提示 | + +响应示例： + +- HTTP 状态码为 `200` 时： + +```JSON +{ "code": 200, "message": "SUCCESS_STATUS"} +``` + +- HTTP 状态码为 `503` 时： + +```JSON +{ "code": 500, "message": "thrift service is unavailable"} +``` + +注意：`ping` 接口访问不需要鉴权。 + +### 3.2 查询接口 + +- 请求地址：`/rest/table/v1/query` + +- 请求方式：post + +- Request 格式 + +请求头:`application/json` + +请求参数说明 + +| 参数名称 | 参数类型 | 是否必填 | 参数描述 | +| --------- | -------- | -------- | ------------------------------------------------------------ | +| database | string | 是 | 数据库名称 | +| sql | string | 是 | | +| row_limit | int | 否 | 一次查询能返回的结果集的最大行数。 如果不设置该参数，将使用配置文件的 `rest_query_default_row_size_limit` 作为默认值。 当返回结果集的行数超出限制时，将返回状态码 `411`。 | + +- Response 格式 + +| 参数名称 | 参数类型 | 参数描述 | +| ------------ | -------- | ------------------------------------------------------------ | +| column_names | array | 列名 | +| data_types | array | 每一列的类型 | +| values | array | 二维数组，第一维与结果集所有行，第二维数组代表结果集的每一行，每一个元素为一列，长度与column_names的长度相同。 | + +- 请求示例 + +```JSON +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","sql":"select s1,s2,s3 from test_table"}' http://127.0.0.1:18080/rest/table/v1/query +``` + +- 响应示例： + +```JSON +{ + "column_names": [ + "s1", + "s2", + "s3" + ], + "data_types": [ + "STRING", + "BOOLEAN", + "INT32" + ], + "values": [ + [ + "a11", + true, + 2024 + ], + [ + "a11", + false, + 2025 + ] + ] +} +``` + +### 3.3 非查询接口 + +- 请求地址：`/rest/table/v1/nonQuery` + +- 请求方式：post + +- Request 格式 + + - 请求头:`application/json` + + - 请求参数说明 + +| 参数名称 | 参数类型 | 是否必填 | 参数描述 | +| -------- | -------- | -------- | -------- | +| sql | string | 是 | | +| database | string | 否 | 数据库名 | + +- Response 格式 + +| 参数名称 | 参数类型 | 参数描述 | +| -------- | -------- | -------- | +| code | integer | 状态码 | +| message | string | 信息提示 | + +- 请求示例 + +```JSON +#创建数据库 +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"create database test","database":""}' http://127.0.0.1:18080/rest/table/v1/nonQuery +#在test库中创建表test_table +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"CREATE TABLE table1 (time TIMESTAMP TIME,region STRING TAG,plant_id STRING TAG,device_id STRING TAG,model_id STRING ATTRIBUTE,maintenance STRING ATTRIBUTE,temperature FLOAT FIELD,humidity FLOAT FIELD,status Boolean FIELD,arrival_time TIMESTAMP FIELD) WITH (TTL=31536000000)","database":"test"}' http://127.0.0.1:18080/rest/table/v1/nonQuery +``` + +- 响应示例： + +```JSON +{ + "code": 200, + "message": "SUCCESS_STATUS" +} +``` + +### 3.4 批量写入接口 + +- 请求地址：`/rest/table/v1/insertTablet` + +- 请求方式：post + +- Request 格式 + +请求头:`application/json` + +请求参数说明 + +| 参数名称 | 参数类型 | 是否必填 | 参数描述 | +| ----------------- | -------- | -------- | ------------------------------------------------------------ | +| database | string | 是 | 数据库名称 | +| table | string | 是 | 表名 | +| column_names | array | 是 | 列名 | +| column_categories | array | 是 | 列类别(TAG,FIELD,*ATTRIBUTE*) | +| data_types | array | 是 | 数据类型 | +| timestamps | array | 是 | 时间列 | +| values | array | 是 | 值列，每一列中的值可以为 `null`二维数组第一层长度跟timestamps长度相同。第二层长度跟column_names长度相同 | + +- Response 格式 + +响应参数: + +| 参数名称 | 参数类型 | 参数描述 | +| -------- | -------- | -------- | +| code | integer | 状态码 | +| message | string | 信息提示 | + +- 请求示例 + +```JSON +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","column_categories":["TAG","FIELD","FIELD"],"timestamps":[1739702535000,1739789055000],"column_names":["s1","s2","s3"],"data_types":["STRING","BOOLEAN","INT32"],"values":[["a11",true,2024],["a11",false,2025]],"table":"test_table"}' http://127.0.0.1:18080/rest/table/v1/insertTablet +``` + +- 响应示例 + +```JSON +{ + "code": 200, + "message": "SUCCESS_STATUS" +} +``` + + +## 4. 配置 + +配置文件位于 `iotdb-system.properties` 中。 + +- 将 `enable_rest_service` 设置为 `true` 表示启用该模块，而将 `false` 表示禁用该模块。默认情况下，该值为 `false`。 + +```Plain +enable_rest_service=true +``` + +- 仅在 `enable_rest_service=true` 时生效。将 `rest_service_port `设置为数字（1025~65535），以自定义REST服务套接字端口，默认情况下，值为 `18080`。 + +```Plain +rest_service_port=18080 +``` + +- 将 'enable_swagger' 设置 'true' 启用swagger来展示rest接口信息, 而设置为 'false' 关闭该功能. 默认情况下，该值为 `false`。 + +```Plain +enable_swagger=false +``` + +- 一次查询能返回的结果集最大行数。当返回结果集的行数超出参数限制时，您只会得到在行数范围内的结果集，且将得到状态码`411`。 + +```Plain +rest_query_default_row_size_limit=10000 +``` + +- 缓存客户登录信息的过期时间（用于加速用户鉴权的速度，单位为秒，默认是8个小时） + +```Plain +cache_expire_in_seconds=28800 +``` + +- 缓存中存储的最大用户数量（默认是100） + +```Plain +cache_max_num=100 +``` + +- 缓存初始容量（默认是10） + +```Plain +cache_init_num=10 +``` + +- REST Service 是否开启 SSL 配置，将 `enable_https` 设置为 `true` 以启用该模块，而将 `false` 设置为禁用该模块。默认情况下，该值为 `false`。 + +```Plain +enable_https=false +``` + +- keyStore 所在路径（非必填） + +```Plain +key_store_path= +``` + +- keyStore 密码（非必填） + +```Plain +key_store_pwd= +``` + +- trustStore 所在路径（非必填） + +```Plain +trust_store_path="" +``` + +- trustStore 密码（非必填） + +```Plain +trust_store_pwd="" +``` + +- SSL 超时时间，单位为秒 + +```Plain +idle_timeout_in_seconds=5000 +``` \ No newline at end of file diff --git a/src/zh/UserGuide/Master/Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md b/src/zh/UserGuide/Master/Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md index c5d5251f4..e54fb9459 100644 --- a/src/zh/UserGuide/Master/Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md +++ b/src/zh/UserGuide/Master/Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md @@ -43,3 +43,5 @@ | README_ZH.md | 文件 | 使用说明（中文版） | | README.md | 文件 | 使用说明（英文版） | | RELEASE_NOTES.md | 文件 | 版本说明 | + +注意：自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 MQTT 服务 和 REST 服务的 JAR 包。如需使用，请联系天谋团队获取。 \ No newline at end of file diff --git a/src/zh/UserGuide/Master/Tree/API/Programming-MQTT.md b/src/zh/UserGuide/Master/Tree/API/Programming-MQTT.md index 1ce846b1d..0a6a38616 100644 --- a/src/zh/UserGuide/Master/Tree/API/Programming-MQTT.md +++ b/src/zh/UserGuide/Master/Tree/API/Programming-MQTT.md @@ -1,3 +1,6 @@ +--- +redirectTo: Programming-MQTT_apache.html +--- - -# MQTT 协议 - -## 1. 概述 - -MQTT 是一种专为物联网（IoT）和低带宽环境设计的轻量级消息传输协议，基于发布/订阅（Pub/Sub）模型，支持设备间高效、可靠的双向通信。其核心目标是低功耗、低带宽消耗和高实时性，尤其适合网络不稳定或资源受限的场景（如传感器、移动设备）。 - -IoTDB 深度集成了 MQTT 协议能力，完整兼容 MQTT v3.1（OASIS 国际标准协议）。IoTDB 服务器内置高性能 MQTT Broker 服务模块，无需第三方中间件，支持设备通过 MQTT 报文将时序数据直接写入 IoTDB 存储引擎。 - - - -## 2. 内置 MQTT 服务 -内置的 MQTT 服务提供了通过 MQTT 直接连接到 IoTDB 的能力。 它侦听来自 MQTT 客户端的发布消息，然后立即将数据写入存储。 -MQTT 主题与 IoTDB 时间序列相对应。 -消息有效载荷可以由 Java SPI 加载的`PayloadFormatter`格式化为事件，默认实现为`JSONPayloadFormatter` - 默认的`json`格式化程序支持两种 json 格式以及由他们组成的json数组，以下是 MQTT 消息有效负载示例： - -```json - { - "device":"root.sg.d1", - "timestamp":1586076045524, - "measurements":["s1","s2"], - "values":[0.530635,0.530635] - } -``` -或者 -```json - { - "device":"root.sg.d1", - "timestamps":[1586076045524,1586076065526], - "measurements":["s1","s2"], - "values":[[0.530635,0.530635], [0.530655,0.530695]] - } -``` -或者以上两者的JSON数组形式。 - - - -## 3. MQTT 配置 -默认情况下，IoTDB MQTT 服务从`${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`加载配置。 - -配置如下： - -| **名称** | **描述** | **默认** | -|---------------------------| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | -| `enable_mqtt_service` | 是否启用 mqtt 服务 | FALSE | -| `mqtt_host` | mqtt 服务绑定主机 | 127.0.0.1 | -| `mqtt_port` | mqtt 服务绑定端口 | 1883 | -| `mqtt_handler_pool_size` | 处理 mqtt 消息的处理程序池大小 | 1 | -| **`mqtt_payload_formatter`** | **mqtt**​**​ 消息有效负载格式化程序。**​**可选项：**​​**`json`**​**：仅适用于树模型。**​​**`line`**​**：仅适用于表模型。** | **json** | -| `mqtt_max_message_size` | mqtt 消息最大长度（字节） | 1048576 | - - -## 4. 示例代码 -以下是 mqtt 客户端将消息发送到 IoTDB 服务器的示例。 - - ```java -MQTT mqtt = new MQTT(); -mqtt.setHost("127.0.0.1", 1883); -mqtt.setUserName("root"); -mqtt.setPassword("root"); - -BlockingConnection connection = mqtt.blockingConnection(); -connection.connect(); - -Random random = new Random(); -for (int i = 0; i < 10; i++) { - String payload = String.format("{\n" + - "\"device\":\"root.sg.d1\",\n" + - "\"timestamp\":%d,\n" + - "\"measurements\":[\"s1\"],\n" + - "\"values\":[%f]\n" + - "}", System.currentTimeMillis(), random.nextDouble()); - - connection.publish("root.sg.d1.s1", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -} - -connection.disconnect(); - ``` - - -## 5. 自定义 MQTT 消息格式 - -在生产环境中，每个设备通常都配备了自己的 MQTT 客户端，且这些客户端的消息格式已经预先设定。如果按照 IoTDB 所支持的 MQTT 消息格式进行通信，就需要对现有的所有客户端进行全面的升级改造，这无疑会带来较高的成本。然而，我们可以通过简单的编程手段，轻松实现 MQTT 消息格式的自定义，而无需改造客户端。 -可以在源码的 [example/mqtt-customize](https://github.com/apache/iotdb/tree/rc/2.0.1/example/mqtt-customize) 项目中找到一个简单示例。 - -假定mqtt客户端传过来的是以下消息格式： -```json - { - "time":1586076045523, - "deviceID":"car_1", - "deviceType":"油车", - "point":"油量", - "value":10.0 -} -``` -或者JSON的数组形式： -```java -[ - { - "time":1586076045523, - "deviceID":"car_1", - "deviceType":"油车", - "point":"油量", - "value":10.0 - }, - { - "time":1586076045524, - "deviceID":"car_2", - "deviceType":"新能源车", - "point":"速度", - "value":80.0 - } -] -``` - - -则可以通过以下步骤设置设置自定义MQTT消息格式： - -1. 创建一个 Java 项目，增加如下依赖 -```xml - - org.apache.iotdb - iotdb-server - 2.0.4-SNAPSHOT - -``` -2. 创建一个实现类，实现接口 `org.apache.iotdb.db.mqtt.protocol.PayloadFormatter` - -```java -package org.apache.iotdb.mqtt.server; - -import org.apache.iotdb.db.protocol.mqtt.Message; -import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; -import org.apache.iotdb.db.protocol.mqtt.TableMessage; - -import com.google.common.collect.Lists; -import com.google.gson.Gson; -import com.google.gson.GsonBuilder; -import com.google.gson.JsonArray; -import com.google.gson.JsonElement; -import com.google.gson.JsonObject; -import com.google.gson.JsonParseException; -import io.netty.buffer.ByteBuf; -import org.apache.commons.lang3.NotImplementedException; -import org.apache.tsfile.enums.TSDataType; - -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -/** - * The Customized JSON payload formatter. one json format supported: { "time":1586076045523, - * "deviceID":"car_1", "deviceType":"新能源车", "point":"速度", "value":80.0 } - */ -public class CustomizedJsonPayloadFormatter implements PayloadFormatter { - private static final String JSON_KEY_TIME = "time"; - private static final String JSON_KEY_DEVICEID = "deviceID"; - private static final String JSON_KEY_DEVICETYPE = "deviceType"; - private static final String JSON_KEY_POINT = "point"; - private static final String JSON_KEY_VALUE = "value"; - private static final Gson GSON = new GsonBuilder().create(); - - @Override - public List format(String topic, ByteBuf payload) { - if (payload == null) { - return new ArrayList<>(); - } - String txt = payload.toString(StandardCharsets.UTF_8); - JsonElement jsonElement = GSON.fromJson(txt, JsonElement.class); - if (jsonElement.isJsonObject()) { - JsonObject jsonObject = jsonElement.getAsJsonObject(); - return formatTableRow(topic, jsonObject); - } else if (jsonElement.isJsonArray()) { - JsonArray jsonArray = jsonElement.getAsJsonArray(); - List messages = new ArrayList<>(); - for (JsonElement element : jsonArray) { - JsonObject jsonObject = element.getAsJsonObject(); - messages.addAll(formatTableRow(topic, jsonObject)); - } - return messages; - } - throw new JsonParseException("payload is invalidate"); - } - - @Override - @Deprecated - public List format(ByteBuf payload) { - throw new NotImplementedException(); - } - - private List formatTableRow(String topic, JsonObject jsonObject) { - TableMessage message = new TableMessage(); - String database = !topic.contains("/") ? topic : topic.substring(0, topic.indexOf("/")); - String table = "test_table"; - - // Parsing Database Name - message.setDatabase((database)); - - // Parsing Table Name - message.setTable(table); - - // Parsing Tags - List tagKeys = new ArrayList<>(); - tagKeys.add(JSON_KEY_DEVICEID); - List tagValues = new ArrayList<>(); - tagValues.add(jsonObject.get(JSON_KEY_DEVICEID).getAsString()); - message.setTagKeys(tagKeys); - message.setTagValues(tagValues); - - // Parsing Attributes - List attributeKeys = new ArrayList<>(); - List attributeValues = new ArrayList<>(); - attributeKeys.add(JSON_KEY_DEVICETYPE); - attributeValues.add(jsonObject.get(JSON_KEY_DEVICETYPE).getAsString()); - message.setAttributeKeys(attributeKeys); - message.setAttributeValues(attributeValues); - - // Parsing Fields - List fields = Arrays.asList(JSON_KEY_POINT); - List dataTypes = Arrays.asList(TSDataType.FLOAT); - List values = Arrays.asList(jsonObject.get(JSON_KEY_VALUE).getAsFloat()); - message.setFields(fields); - message.setDataTypes(dataTypes); - message.setValues(values); - - // Parsing timestamp - message.setTimestamp(jsonObject.get(JSON_KEY_TIME).getAsLong()); - return Lists.newArrayList(message); - } - - @Override - public String getName() { - // set the value of mqtt_payload_formatter in iotdb-common.properties as the following string: - return "CustomizedJson2Table"; - } - - @Override - public String getType() { - return PayloadFormatter.TABLE_TYPE; - } -} -``` -3. 修改项目中的 `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` 文件: - 将示例中的文件内容清除，并将刚才的实现类的全名（包名.类名）写入文件中。注意，这个文件中只有一行。 - 在本例中，文件内容为: `org.apache.iotdb.mqtt.server.CustomizedJsonPayloadFormatter` -4. 编译项目生成一个 jar 包: `mvn package -DskipTests` - - -在 IoTDB 服务端: -1. 创建 ${IOTDB_HOME}/ext/mqtt/ 文件夹, 将刚才的 jar 包放入此文件夹。 -2. 打开 MQTT 服务参数. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) -3. 用刚才的实现类中的 getName() 方法的返回值 设置为 `conf/iotdb-system.properties` 中 `mqtt_payload_formatter` 的值， - , 在本例中，为 `CustomizedJson2Table` -4. 启动 IoTDB -5. 搞定 - -More: MQTT 协议的消息不限于 json，你还可以用任意二进制。通过如下函数获得： -`payload.forEachByte()` or `payload.array`。 - - -## 6. 注意事项 - -为避免因缺省client_id引发的兼容性问题，强烈建议在所有MQTT客户端中始终显式地提供唯一且非空的 client_id。 -不同客户端在client_id缺失或为空时的表现并不一致，常见示例如下： -1. 显式传入空字符串 - • MQTTX：client_id=""时，IoTDB会直接丢弃消息； - • mosquitto_pub：client_id=""时，IoTDB能正常接收消息。 -2. 完全不传client_id - • MQTTX：消息可被IoTDB正常接收； - • mosquitto_pub：IoTDB拒绝连接。 - 由此可见，显式指定唯一且非空的client_id是消除上述差异、确保消息可靠投递的最简单做法。 \ No newline at end of file +--> \ No newline at end of file diff --git a/src/zh/UserGuide/Master/Tree/API/Programming-MQTT_apache.md b/src/zh/UserGuide/Master/Tree/API/Programming-MQTT_apache.md new file mode 100644 index 000000000..1ce846b1d --- /dev/null +++ b/src/zh/UserGuide/Master/Tree/API/Programming-MQTT_apache.md @@ -0,0 +1,293 @@ + + +# MQTT 协议 + +## 1. 概述 + +MQTT 是一种专为物联网（IoT）和低带宽环境设计的轻量级消息传输协议，基于发布/订阅（Pub/Sub）模型，支持设备间高效、可靠的双向通信。其核心目标是低功耗、低带宽消耗和高实时性，尤其适合网络不稳定或资源受限的场景（如传感器、移动设备）。 + +IoTDB 深度集成了 MQTT 协议能力，完整兼容 MQTT v3.1（OASIS 国际标准协议）。IoTDB 服务器内置高性能 MQTT Broker 服务模块，无需第三方中间件，支持设备通过 MQTT 报文将时序数据直接写入 IoTDB 存储引擎。 + + + +## 2. 内置 MQTT 服务 +内置的 MQTT 服务提供了通过 MQTT 直接连接到 IoTDB 的能力。 它侦听来自 MQTT 客户端的发布消息，然后立即将数据写入存储。 +MQTT 主题与 IoTDB 时间序列相对应。 +消息有效载荷可以由 Java SPI 加载的`PayloadFormatter`格式化为事件，默认实现为`JSONPayloadFormatter` + 默认的`json`格式化程序支持两种 json 格式以及由他们组成的json数组，以下是 MQTT 消息有效负载示例： + +```json + { + "device":"root.sg.d1", + "timestamp":1586076045524, + "measurements":["s1","s2"], + "values":[0.530635,0.530635] + } +``` +或者 +```json + { + "device":"root.sg.d1", + "timestamps":[1586076045524,1586076065526], + "measurements":["s1","s2"], + "values":[[0.530635,0.530635], [0.530655,0.530695]] + } +``` +或者以上两者的JSON数组形式。 + + + +## 3. MQTT 配置 +默认情况下，IoTDB MQTT 服务从`${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`加载配置。 + +配置如下： + +| **名称** | **描述** | **默认** | +|---------------------------| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | +| `enable_mqtt_service` | 是否启用 mqtt 服务 | FALSE | +| `mqtt_host` | mqtt 服务绑定主机 | 127.0.0.1 | +| `mqtt_port` | mqtt 服务绑定端口 | 1883 | +| `mqtt_handler_pool_size` | 处理 mqtt 消息的处理程序池大小 | 1 | +| **`mqtt_payload_formatter`** | **mqtt**​**​ 消息有效负载格式化程序。**​**可选项：**​​**`json`**​**：仅适用于树模型。**​​**`line`**​**：仅适用于表模型。** | **json** | +| `mqtt_max_message_size` | mqtt 消息最大长度（字节） | 1048576 | + + +## 4. 示例代码 +以下是 mqtt 客户端将消息发送到 IoTDB 服务器的示例。 + + ```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +connection.connect(); + +Random random = new Random(); +for (int i = 0; i < 10; i++) { + String payload = String.format("{\n" + + "\"device\":\"root.sg.d1\",\n" + + "\"timestamp\":%d,\n" + + "\"measurements\":[\"s1\"],\n" + + "\"values\":[%f]\n" + + "}", System.currentTimeMillis(), random.nextDouble()); + + connection.publish("root.sg.d1.s1", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +} + +connection.disconnect(); + ``` + + +## 5. 自定义 MQTT 消息格式 + +在生产环境中，每个设备通常都配备了自己的 MQTT 客户端，且这些客户端的消息格式已经预先设定。如果按照 IoTDB 所支持的 MQTT 消息格式进行通信，就需要对现有的所有客户端进行全面的升级改造，这无疑会带来较高的成本。然而，我们可以通过简单的编程手段，轻松实现 MQTT 消息格式的自定义，而无需改造客户端。 +可以在源码的 [example/mqtt-customize](https://github.com/apache/iotdb/tree/rc/2.0.1/example/mqtt-customize) 项目中找到一个简单示例。 + +假定mqtt客户端传过来的是以下消息格式： +```json + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"油车", + "point":"油量", + "value":10.0 +} +``` +或者JSON的数组形式： +```java +[ + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"油车", + "point":"油量", + "value":10.0 + }, + { + "time":1586076045524, + "deviceID":"car_2", + "deviceType":"新能源车", + "point":"速度", + "value":80.0 + } +] +``` + + +则可以通过以下步骤设置设置自定义MQTT消息格式： + +1. 创建一个 Java 项目，增加如下依赖 +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. 创建一个实现类，实现接口 `org.apache.iotdb.db.mqtt.protocol.PayloadFormatter` + +```java +package org.apache.iotdb.mqtt.server; + +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; +import org.apache.iotdb.db.protocol.mqtt.TableMessage; + +import com.google.common.collect.Lists; +import com.google.gson.Gson; +import com.google.gson.GsonBuilder; +import com.google.gson.JsonArray; +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParseException; +import io.netty.buffer.ByteBuf; +import org.apache.commons.lang3.NotImplementedException; +import org.apache.tsfile.enums.TSDataType; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +/** + * The Customized JSON payload formatter. one json format supported: { "time":1586076045523, + * "deviceID":"car_1", "deviceType":"新能源车", "point":"速度", "value":80.0 } + */ +public class CustomizedJsonPayloadFormatter implements PayloadFormatter { + private static final String JSON_KEY_TIME = "time"; + private static final String JSON_KEY_DEVICEID = "deviceID"; + private static final String JSON_KEY_DEVICETYPE = "deviceType"; + private static final String JSON_KEY_POINT = "point"; + private static final String JSON_KEY_VALUE = "value"; + private static final Gson GSON = new GsonBuilder().create(); + + @Override + public List format(String topic, ByteBuf payload) { + if (payload == null) { + return new ArrayList<>(); + } + String txt = payload.toString(StandardCharsets.UTF_8); + JsonElement jsonElement = GSON.fromJson(txt, JsonElement.class); + if (jsonElement.isJsonObject()) { + JsonObject jsonObject = jsonElement.getAsJsonObject(); + return formatTableRow(topic, jsonObject); + } else if (jsonElement.isJsonArray()) { + JsonArray jsonArray = jsonElement.getAsJsonArray(); + List messages = new ArrayList<>(); + for (JsonElement element : jsonArray) { + JsonObject jsonObject = element.getAsJsonObject(); + messages.addAll(formatTableRow(topic, jsonObject)); + } + return messages; + } + throw new JsonParseException("payload is invalidate"); + } + + @Override + @Deprecated + public List format(ByteBuf payload) { + throw new NotImplementedException(); + } + + private List formatTableRow(String topic, JsonObject jsonObject) { + TableMessage message = new TableMessage(); + String database = !topic.contains("/") ? topic : topic.substring(0, topic.indexOf("/")); + String table = "test_table"; + + // Parsing Database Name + message.setDatabase((database)); + + // Parsing Table Name + message.setTable(table); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add(JSON_KEY_DEVICEID); + List tagValues = new ArrayList<>(); + tagValues.add(jsonObject.get(JSON_KEY_DEVICEID).getAsString()); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add(JSON_KEY_DEVICETYPE); + attributeValues.add(jsonObject.get(JSON_KEY_DEVICETYPE).getAsString()); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList(JSON_KEY_POINT); + List dataTypes = Arrays.asList(TSDataType.FLOAT); + List values = Arrays.asList(jsonObject.get(JSON_KEY_VALUE).getAsFloat()); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + // Parsing timestamp + message.setTimestamp(jsonObject.get(JSON_KEY_TIME).getAsLong()); + return Lists.newArrayList(message); + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-common.properties as the following string: + return "CustomizedJson2Table"; + } + + @Override + public String getType() { + return PayloadFormatter.TABLE_TYPE; + } +} +``` +3. 修改项目中的 `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` 文件: + 将示例中的文件内容清除，并将刚才的实现类的全名（包名.类名）写入文件中。注意，这个文件中只有一行。 + 在本例中，文件内容为: `org.apache.iotdb.mqtt.server.CustomizedJsonPayloadFormatter` +4. 编译项目生成一个 jar 包: `mvn package -DskipTests` + + +在 IoTDB 服务端: +1. 创建 ${IOTDB_HOME}/ext/mqtt/ 文件夹, 将刚才的 jar 包放入此文件夹。 +2. 打开 MQTT 服务参数. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. 用刚才的实现类中的 getName() 方法的返回值 设置为 `conf/iotdb-system.properties` 中 `mqtt_payload_formatter` 的值， + , 在本例中，为 `CustomizedJson2Table` +4. 启动 IoTDB +5. 搞定 + +More: MQTT 协议的消息不限于 json，你还可以用任意二进制。通过如下函数获得： +`payload.forEachByte()` or `payload.array`。 + + +## 6. 注意事项 + +为避免因缺省client_id引发的兼容性问题，强烈建议在所有MQTT客户端中始终显式地提供唯一且非空的 client_id。 +不同客户端在client_id缺失或为空时的表现并不一致，常见示例如下： +1. 显式传入空字符串 + • MQTTX：client_id=""时，IoTDB会直接丢弃消息； + • mosquitto_pub：client_id=""时，IoTDB能正常接收消息。 +2. 完全不传client_id + • MQTTX：消息可被IoTDB正常接收； + • mosquitto_pub：IoTDB拒绝连接。 + 由此可见，显式指定唯一且非空的client_id是消除上述差异、确保消息可靠投递的最简单做法。 \ No newline at end of file diff --git a/src/zh/UserGuide/Master/Tree/API/Programming-MQTT_timecho.md b/src/zh/UserGuide/Master/Tree/API/Programming-MQTT_timecho.md new file mode 100644 index 000000000..f4164a9a3 --- /dev/null +++ b/src/zh/UserGuide/Master/Tree/API/Programming-MQTT_timecho.md @@ -0,0 +1,295 @@ + + +# MQTT 协议 + +## 1. 概述 + +MQTT 是一种专为物联网（IoT）和低带宽环境设计的轻量级消息传输协议，基于发布/订阅（Pub/Sub）模型，支持设备间高效、可靠的双向通信。其核心目标是低功耗、低带宽消耗和高实时性，尤其适合网络不稳定或资源受限的场景（如传感器、移动设备）。 + +IoTDB 深度集成了 MQTT 协议能力，完整兼容 MQTT v3.1（OASIS 国际标准协议）。IoTDB 服务器内置高性能 MQTT Broker 服务模块，无需第三方中间件，支持设备通过 MQTT 报文将时序数据直接写入 IoTDB 存储引擎。 + + + +注意，自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 MQTT 服务的 JAR 包。请使用该服务前联系天谋团队获取 JAR 包，并放置于 timechodb_home/lib 或者 timechodb_home/ext/external_service 路径下。 + +## 2. 内置 MQTT 服务 +内置的 MQTT 服务提供了通过 MQTT 直接连接到 IoTDB 的能力。 它侦听来自 MQTT 客户端的发布消息，然后立即将数据写入存储。 +MQTT 主题与 IoTDB 时间序列相对应。 +消息有效载荷可以由 Java SPI 加载的`PayloadFormatter`格式化为事件，默认实现为`JSONPayloadFormatter` + 默认的`json`格式化程序支持两种 json 格式以及由他们组成的json数组，以下是 MQTT 消息有效负载示例： + +```json + { + "device":"root.sg.d1", + "timestamp":1586076045524, + "measurements":["s1","s2"], + "values":[0.530635,0.530635] + } +``` +或者 +```json + { + "device":"root.sg.d1", + "timestamps":[1586076045524,1586076065526], + "measurements":["s1","s2"], + "values":[[0.530635,0.530635], [0.530655,0.530695]] + } +``` +或者以上两者的JSON数组形式。 + + + +## 3. MQTT 配置 +默认情况下，IoTDB MQTT 服务从`${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`加载配置。 + +配置如下： + +| **名称** | **描述** | **默认** | +|---------------------------| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | +| `enable_mqtt_service` | 是否启用 mqtt 服务 | FALSE | +| `mqtt_host` | mqtt 服务绑定主机 | 127.0.0.1 | +| `mqtt_port` | mqtt 服务绑定端口 | 1883 | +| `mqtt_handler_pool_size` | 处理 mqtt 消息的处理程序池大小 | 1 | +| **`mqtt_payload_formatter`** | **mqtt**​**​ 消息有效负载格式化程序。**​**可选项：**​​**`json`**​**：仅适用于树模型。**​​**`line`**​**：仅适用于表模型。** | **json** | +| `mqtt_max_message_size` | mqtt 消息最大长度（字节） | 1048576 | + + +## 4. 示例代码 +以下是 mqtt 客户端将消息发送到 IoTDB 服务器的示例。 + + ```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +connection.connect(); + +Random random = new Random(); +for (int i = 0; i < 10; i++) { + String payload = String.format("{\n" + + "\"device\":\"root.sg.d1\",\n" + + "\"timestamp\":%d,\n" + + "\"measurements\":[\"s1\"],\n" + + "\"values\":[%f]\n" + + "}", System.currentTimeMillis(), random.nextDouble()); + + connection.publish("root.sg.d1.s1", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +} + +connection.disconnect(); + ``` + + +## 5. 自定义 MQTT 消息格式 + +在生产环境中，每个设备通常都配备了自己的 MQTT 客户端，且这些客户端的消息格式已经预先设定。如果按照 IoTDB 所支持的 MQTT 消息格式进行通信，就需要对现有的所有客户端进行全面的升级改造，这无疑会带来较高的成本。然而，我们可以通过简单的编程手段，轻松实现 MQTT 消息格式的自定义，而无需改造客户端。 +可以在源码的 [example/mqtt-customize](https://github.com/apache/iotdb/tree/rc/2.0.1/example/mqtt-customize) 项目中找到一个简单示例。 + +假定mqtt客户端传过来的是以下消息格式： +```json + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"油车", + "point":"油量", + "value":10.0 +} +``` +或者JSON的数组形式： +```java +[ + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"油车", + "point":"油量", + "value":10.0 + }, + { + "time":1586076045524, + "deviceID":"car_2", + "deviceType":"新能源车", + "point":"速度", + "value":80.0 + } +] +``` + + +则可以通过以下步骤设置设置自定义MQTT消息格式： + +1. 创建一个 Java 项目，增加如下依赖 +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. 创建一个实现类，实现接口 `org.apache.iotdb.db.mqtt.protocol.PayloadFormatter` + +```java +package org.apache.iotdb.mqtt.server; + +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; +import org.apache.iotdb.db.protocol.mqtt.TableMessage; + +import com.google.common.collect.Lists; +import com.google.gson.Gson; +import com.google.gson.GsonBuilder; +import com.google.gson.JsonArray; +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParseException; +import io.netty.buffer.ByteBuf; +import org.apache.commons.lang3.NotImplementedException; +import org.apache.tsfile.enums.TSDataType; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +/** + * The Customized JSON payload formatter. one json format supported: { "time":1586076045523, + * "deviceID":"car_1", "deviceType":"新能源车", "point":"速度", "value":80.0 } + */ +public class CustomizedJsonPayloadFormatter implements PayloadFormatter { + private static final String JSON_KEY_TIME = "time"; + private static final String JSON_KEY_DEVICEID = "deviceID"; + private static final String JSON_KEY_DEVICETYPE = "deviceType"; + private static final String JSON_KEY_POINT = "point"; + private static final String JSON_KEY_VALUE = "value"; + private static final Gson GSON = new GsonBuilder().create(); + + @Override + public List format(String topic, ByteBuf payload) { + if (payload == null) { + return new ArrayList<>(); + } + String txt = payload.toString(StandardCharsets.UTF_8); + JsonElement jsonElement = GSON.fromJson(txt, JsonElement.class); + if (jsonElement.isJsonObject()) { + JsonObject jsonObject = jsonElement.getAsJsonObject(); + return formatTableRow(topic, jsonObject); + } else if (jsonElement.isJsonArray()) { + JsonArray jsonArray = jsonElement.getAsJsonArray(); + List messages = new ArrayList<>(); + for (JsonElement element : jsonArray) { + JsonObject jsonObject = element.getAsJsonObject(); + messages.addAll(formatTableRow(topic, jsonObject)); + } + return messages; + } + throw new JsonParseException("payload is invalidate"); + } + + @Override + @Deprecated + public List format(ByteBuf payload) { + throw new NotImplementedException(); + } + + private List formatTableRow(String topic, JsonObject jsonObject) { + TableMessage message = new TableMessage(); + String database = !topic.contains("/") ? topic : topic.substring(0, topic.indexOf("/")); + String table = "test_table"; + + // Parsing Database Name + message.setDatabase((database)); + + // Parsing Table Name + message.setTable(table); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add(JSON_KEY_DEVICEID); + List tagValues = new ArrayList<>(); + tagValues.add(jsonObject.get(JSON_KEY_DEVICEID).getAsString()); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add(JSON_KEY_DEVICETYPE); + attributeValues.add(jsonObject.get(JSON_KEY_DEVICETYPE).getAsString()); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList(JSON_KEY_POINT); + List dataTypes = Arrays.asList(TSDataType.FLOAT); + List values = Arrays.asList(jsonObject.get(JSON_KEY_VALUE).getAsFloat()); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + // Parsing timestamp + message.setTimestamp(jsonObject.get(JSON_KEY_TIME).getAsLong()); + return Lists.newArrayList(message); + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-common.properties as the following string: + return "CustomizedJson2Table"; + } + + @Override + public String getType() { + return PayloadFormatter.TABLE_TYPE; + } +} +``` +3. 修改项目中的 `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` 文件: + 将示例中的文件内容清除，并将刚才的实现类的全名（包名.类名）写入文件中。注意，这个文件中只有一行。 + 在本例中，文件内容为: `org.apache.iotdb.mqtt.server.CustomizedJsonPayloadFormatter` +4. 编译项目生成一个 jar 包: `mvn package -DskipTests` + + +在 IoTDB 服务端: +1. 创建 ${IOTDB_HOME}/ext/mqtt/ 文件夹, 将刚才的 jar 包放入此文件夹。 +2. 打开 MQTT 服务参数. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. 用刚才的实现类中的 getName() 方法的返回值 设置为 `conf/iotdb-system.properties` 中 `mqtt_payload_formatter` 的值， + , 在本例中，为 `CustomizedJson2Table` +4. 启动 IoTDB +5. 搞定 + +More: MQTT 协议的消息不限于 json，你还可以用任意二进制。通过如下函数获得： +`payload.forEachByte()` or `payload.array`。 + + +## 6. 注意事项 + +为避免因缺省client_id引发的兼容性问题，强烈建议在所有MQTT客户端中始终显式地提供唯一且非空的 client_id。 +不同客户端在client_id缺失或为空时的表现并不一致，常见示例如下： +1. 显式传入空字符串 + • MQTTX：client_id=""时，IoTDB会直接丢弃消息； + • mosquitto_pub：client_id=""时，IoTDB能正常接收消息。 +2. 完全不传client_id + • MQTTX：消息可被IoTDB正常接收； + • mosquitto_pub：IoTDB拒绝连接。 + 由此可见，显式指定唯一且非空的client_id是消除上述差异、确保消息可靠投递的最简单做法。 \ No newline at end of file diff --git a/src/zh/UserGuide/Master/Tree/API/RestServiceV1_timecho.md b/src/zh/UserGuide/Master/Tree/API/RestServiceV1_timecho.md index 78859f1ba..76e59f0fd 100644 --- a/src/zh/UserGuide/Master/Tree/API/RestServiceV1_timecho.md +++ b/src/zh/UserGuide/Master/Tree/API/RestServiceV1_timecho.md @@ -22,6 +22,8 @@ # REST API V1（不推荐） IoTDB 的 RESTful 服务可用于查询、写入和管理操作，它使用 OpenAPI 标准来定义接口并生成框架。 +注意：自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 REST 服务的 JAR 包，请使用该服务前联系天谋团队获取相应的 JAR 包，并放置于 timechodb_home/lib 或者 timechodb_home/ext/external_service 路径下。 + ## 1. 开启RESTful 服务 RESTful 服务默认情况是关闭的 diff --git a/src/zh/UserGuide/Master/Tree/API/RestServiceV2_timecho.md b/src/zh/UserGuide/Master/Tree/API/RestServiceV2_timecho.md index 75869cecb..b122c0405 100644 --- a/src/zh/UserGuide/Master/Tree/API/RestServiceV2_timecho.md +++ b/src/zh/UserGuide/Master/Tree/API/RestServiceV2_timecho.md @@ -22,6 +22,8 @@ # REST API V2 IoTDB 的 RESTful 服务可用于查询、写入和管理操作，它使用 OpenAPI 标准来定义接口并生成框架。 +注意：自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 REST 服务的 JAR 包，请使用该服务前联系天谋团队获取相应的 JAR 包，并放置于 timechodb_home/lib 或者 timechodb_home/ext/external_service 路径下。 + ## 1. 开启RESTful 服务 RESTful 服务默认情况是关闭的 diff --git a/src/zh/UserGuide/Master/Tree/Basic-Concept/Write-Data_apache.md b/src/zh/UserGuide/Master/Tree/Basic-Concept/Write-Data_apache.md index 12f6dafd8..f508407c7 100644 --- a/src/zh/UserGuide/Master/Tree/Basic-Concept/Write-Data_apache.md +++ b/src/zh/UserGuide/Master/Tree/Basic-Concept/Write-Data_apache.md @@ -168,7 +168,7 @@ It costs 0.004s ## 4. MQTT写入 -参考 [内置 MQTT 服务](../API/Programming-MQTT.md#内置-mqtt-服务) +参考 [内置 MQTT 服务](../API/Programming-MQTT_apache.md#_2-内置-mqtt-服务) ## 5. 批量数据导入 diff --git a/src/zh/UserGuide/Master/Tree/Basic-Concept/Write-Data_timecho.md b/src/zh/UserGuide/Master/Tree/Basic-Concept/Write-Data_timecho.md index a7f0fb2e7..8c047caa1 100644 --- a/src/zh/UserGuide/Master/Tree/Basic-Concept/Write-Data_timecho.md +++ b/src/zh/UserGuide/Master/Tree/Basic-Concept/Write-Data_timecho.md @@ -168,7 +168,7 @@ It costs 0.004s ## 4. MQTT写入 -参考 [内置 MQTT 服务](../API/Programming-MQTT.md#内置-mqtt-服务) +参考 [内置 MQTT 服务](../API/Programming-MQTT_timecho.md#_2-内置-mqtt-服务) ## 5. 批量数据导入 diff --git a/src/zh/UserGuide/Master/Tree/Deployment-and-Maintenance/IoTDB-Package_timecho.md b/src/zh/UserGuide/Master/Tree/Deployment-and-Maintenance/IoTDB-Package_timecho.md index 86feb9852..04e757c36 100644 --- a/src/zh/UserGuide/Master/Tree/Deployment-and-Maintenance/IoTDB-Package_timecho.md +++ b/src/zh/UserGuide/Master/Tree/Deployment-and-Maintenance/IoTDB-Package_timecho.md @@ -30,7 +30,7 @@ | **目录** | **类型** | **说明** | | ---------------- | -------- | ------------------------------------------------------------ | -| activation | 文件夹 | 激活文件所在目录，包括生成的机器码以及天谋工作人员获取的企业版激活码（启动ConfigNode后才会生成该目录，即可获取激活码） | +| activation | 文件夹 | 激活文件所在目录，包括生成的机器码以及从天谋工作人员获取的企业版激活码（启动ConfigNode后才会生成该目录，即可获取激活码） | | conf | 文件夹 | 配置文件目录，包含 ConfigNode、DataNode、JMX 和 logback 等配置文件 | | data | 文件夹 | 默认的数据文件目录，包含 ConfigNode 和 DataNode 的数据文件。（启动程序后才会生成该目录） | | lib | 文件夹 | 库文件目录 | @@ -44,3 +44,5 @@ | README_ZH.md | 文件 | 使用说明（中文版） | | README.md | 文件 | 使用说明（英文版） | | RELEASE_NOTES.md | 文件 | 版本说明 | + +注意：自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 MQTT 服务 和 REST 服务的 JAR 包。如需使用，请联系天谋团队获取。 \ No newline at end of file diff --git a/src/zh/UserGuide/latest-Table/API/Programming-MQTT.md b/src/zh/UserGuide/latest-Table/API/Programming-MQTT.md index b243ff3eb..0a6a38616 100644 --- a/src/zh/UserGuide/latest-Table/API/Programming-MQTT.md +++ b/src/zh/UserGuide/latest-Table/API/Programming-MQTT.md @@ -1,3 +1,6 @@ +--- +redirectTo: Programming-MQTT_apache.html +--- -# MQTT 协议 - -## 1. 概述 - -MQTT 是一种专为物联网（IoT）和低带宽环境设计的轻量级消息传输协议，基于发布/订阅（Pub/Sub）模型，支持设备间高效、可靠的双向通信。其核心目标是低功耗、低带宽消耗和高实时性，尤其适合网络不稳定或资源受限的场景（如传感器、移动设备）。 - -IoTDB 深度集成了 MQTT 协议能力，完整兼容 MQTT v3.1（OASIS 国际标准协议）。IoTDB 服务器内置高性能 MQTT Broker 服务模块，无需第三方中间件，支持设备通过 MQTT 报文将时序数据直接写入 IoTDB 存储引擎。 - -![](/img/mqtt-table-1.png) - -## 2. 配置方式 - -默认情况下，IoTDB MQTT 服务通过`${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`加载配置。 - -具体配置项如下： - -| **名称** | **描述** | **默认** | -|---------------------------| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | -| `enable_mqtt_service` | 是否启用 mqtt 服务 | FALSE | -| `mqtt_host` | mqtt 服务绑定主机 | 127.0.0.1 | -| `mqtt_port` | mqtt 服务绑定端口 | 1883 | -| `mqtt_handler_pool_size` | 处理 mqtt 消息的处理程序池大小 | 1 | -| **`mqtt_payload_formatter`** | **mqtt**​**​ 消息有效负载格式化程序。**​**可选项：**​​**`json`**​**：仅适用于树模型。**​​**`line`**​**：仅适用于表模型。** | **json** | -| `mqtt_max_message_size` | mqtt 消息最大长度（字节） | 1048576 | - -## 3. 写入协议 - -* 行协议语法格式 - -```JavaScript -[,=[,=]][ =[,=]] =[,=] [] -``` - -* 示例 - -```JavaScript -myMeasurement,tag1=value1,tag2=value2 attr1=value1,attr2=value2 fieldKey="fieldValue" 1556813561098000000 -``` - -![](/img/mqtt-table-2.png) - - -## 4. 命名约定 - -* 数据库名称 - -MQTT topic 名称用 `/` 分割后， 第一串内容作为数据库名称。 - -```Properties -topic: stock/Legacy -databaseName: stock - - -topic: stock/Legacy/# -databaseName:stock -``` - -* 表名称 - -表名称使用行协议中的 ``。 - -* 类型标识 - -| Filed 内容 | IoTDB 数据类型 | -|--------------------------------------------------------------| ---------------- | -| 1
1.12 | DOUBLE | -| 1`f`
1.12`f` | FLOAT | -| 1`i`
123`i` | INT64 | -| 1`u`
123`u` | INT64| -| 1`i32`
123`i32` | INT32 | -| `"xxx"` | TEXT | -| `t`,`T`,`true`,`True`,`TRUE`
`f`,`F`,`false`,`False`,`FALSE` | BOOLEAN | - - -## 5. 代码示例 -以下是 mqtt 客户端将消息发送到 IoTDB 服务器的示例。 - - ```java -MQTT mqtt = new MQTT(); -mqtt.setHost("127.0.0.1", 1883); -mqtt.setUserName("root"); -mqtt.setPassword("root"); - -BlockingConnection connection = mqtt.blockingConnection(); -String DATABASE = "myMqttTest"; -connection.connect(); - -String payload = - "test1,tag1=t1,tag2=t2 attr3=a5,attr4=a4 field1=\"fieldValue1\",field2=1i,field3=1u 1"; -connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -payload = "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 2"; -connection.publish(DATABASE, payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -payload = "# It's a remark\n " + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 6"; - connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); - Thread.sleep(10); - -//批量写入示例 -payload = - "test1,tag1=t1,tag2=t2 field7=t,field8=T,field9=true 3 \n " - + "test1,tag1=t1,tag2=t2 field7=f,field8=F,field9=FALSE 4"; -connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -//批量写入示例 -payload = - "test1,tag1=t1,tag2=t2 attr1=a1,attr2=a2 field1=\"fieldValue1\",field2=1i,field3=1u 4 \n " - + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 5"; -connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -Thread.sleep(10); - -connection.disconnect(); - ``` - - -## 6. 自定义 MQTT 消息格式 - -事实上可以通过简单编程来实现 MQTT 消息的格式自定义。 -可以在源码的 [example/mqtt-customize](https://github.com/apache/iotdb/tree/master/example/mqtt-customize) 项目中找到一个简单示例。 - -步骤: -1. 创建一个 Java 项目，增加如下依赖 -```xml - - org.apache.iotdb - iotdb-server - 2.0.4-SNAPSHOT - -``` -2. 创建一个实现类，实现接口 `org.apache.iotdb.db.mqtt.protocol.PayloadFormatter` - -```java -package org.apache.iotdb.mqtt.server; - -import io.netty.buffer.ByteBuf; -import org.apache.iotdb.db.protocol.mqtt.Message; -import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; - -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -public class CustomizedLinePayloadFormatter implements PayloadFormatter { - - @Override - public List format(String topic, ByteBuf payload) { - // Suppose the payload is a line format - if (payload == null) { - return null; - } - - String line = payload.toString(StandardCharsets.UTF_8); - // parse data from the line and generate Messages and put them into List ret - List ret = new ArrayList<>(); - // this is just an example, so we just generate some Messages directly - for (int i = 0; i < 3; i++) { - long ts = i; - TableMessage message = new TableMessage(); - - // Parsing Database Name - message.setDatabase("db" + i); - - //Parsing Table Names - message.setTable("t" + i); - - // Parsing Tags - List tagKeys = new ArrayList<>(); - tagKeys.add("tag1" + i); - tagKeys.add("tag2" + i); - List tagValues = new ArrayList<>(); - tagValues.add("t_value1" + i); - tagValues.add("t_value2" + i); - message.setTagKeys(tagKeys); - message.setTagValues(tagValues); - - // Parsing Attributes - List attributeKeys = new ArrayList<>(); - List attributeValues = new ArrayList<>(); - attributeKeys.add("attr1" + i); - attributeKeys.add("attr2" + i); - attributeValues.add("a_value1" + i); - attributeValues.add("a_value2" + i); - message.setAttributeKeys(attributeKeys); - message.setAttributeValues(attributeValues); - - // Parsing Fields - List fields = Arrays.asList("field1" + i, "field2" + i); - List dataTypes = Arrays.asList(TSDataType.FLOAT, TSDataType.FLOAT); - List values = Arrays.asList("4.0" + i, "5.0" + i); - message.setFields(fields); - message.setDataTypes(dataTypes); - message.setValues(values); - - //// Parsing timestamp - message.setTimestamp(ts); - ret.add(message); - } - return ret; - } - - @Override - public String getName() { - // set the value of mqtt_payload_formatter in iotdb-system.properties as the following string: - return "CustomizedLine"; - } -} -``` - - -3. 修改项目中的 `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` 文件: - 将示例中的文件内容清除，并将刚才的实现类的全名（包名.类名）写入文件中。注意，这个文件中只有一行。 - 在本例中，文件内容为: `org.apache.iotdb.mqtt.server.CustomizedLinePayloadFormatter` -4. 编译项目生成一个 jar 包: `mvn package -DskipTests` - - -在 IoTDB 服务端: -1. 创建 ${IOTDB_HOME}/ext/mqtt/ 文件夹, 将刚才的 jar 包放入此文件夹。 -2. 打开 MQTT 服务参数. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) -3. 用刚才的实现类中的 getName() 方法的返回值 设置为 `conf/iotdb-system.properties` 中 `mqtt_payload_formatter` 的值， - , 在本例中，为 `CustomizedLine` -4. 启动 IoTDB -5. 搞定 - -More: MQTT 协议的消息不限于 line，你还可以用任意二进制。通过如下函数获得： -`payload.forEachByte()` or `payload.array`。 - - -## 7. 注意事项 - -为避免因缺省client_id引发的兼容性问题，强烈建议在所有MQTT客户端中始终显式地提供唯一且非空的 client_id。 -不同客户端在client_id缺失或为空时的表现并不一致，常见示例如下： -1. 显式传入空字符串 -• MQTTX：client_id=""时，IoTDB会直接丢弃消息； -• mosquitto_pub：client_id=""时，IoTDB能正常接收消息。 -2. 完全不传client_id -• MQTTX：消息可被IoTDB正常接收； -• mosquitto_pub：IoTDB拒绝连接。 -由此可见，显式指定唯一且非空的client_id是消除上述差异、确保消息可靠投递的最简单做法。 \ No newline at end of file +--> \ No newline at end of file diff --git a/src/zh/UserGuide/latest-Table/API/Programming-MQTT_apache.md b/src/zh/UserGuide/latest-Table/API/Programming-MQTT_apache.md new file mode 100644 index 000000000..b243ff3eb --- /dev/null +++ b/src/zh/UserGuide/latest-Table/API/Programming-MQTT_apache.md @@ -0,0 +1,262 @@ + +# MQTT 协议 + +## 1. 概述 + +MQTT 是一种专为物联网（IoT）和低带宽环境设计的轻量级消息传输协议，基于发布/订阅（Pub/Sub）模型，支持设备间高效、可靠的双向通信。其核心目标是低功耗、低带宽消耗和高实时性，尤其适合网络不稳定或资源受限的场景（如传感器、移动设备）。 + +IoTDB 深度集成了 MQTT 协议能力，完整兼容 MQTT v3.1（OASIS 国际标准协议）。IoTDB 服务器内置高性能 MQTT Broker 服务模块，无需第三方中间件，支持设备通过 MQTT 报文将时序数据直接写入 IoTDB 存储引擎。 + +![](/img/mqtt-table-1.png) + +## 2. 配置方式 + +默认情况下，IoTDB MQTT 服务通过`${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`加载配置。 + +具体配置项如下： + +| **名称** | **描述** | **默认** | +|---------------------------| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | +| `enable_mqtt_service` | 是否启用 mqtt 服务 | FALSE | +| `mqtt_host` | mqtt 服务绑定主机 | 127.0.0.1 | +| `mqtt_port` | mqtt 服务绑定端口 | 1883 | +| `mqtt_handler_pool_size` | 处理 mqtt 消息的处理程序池大小 | 1 | +| **`mqtt_payload_formatter`** | **mqtt**​**​ 消息有效负载格式化程序。**​**可选项：**​​**`json`**​**：仅适用于树模型。**​​**`line`**​**：仅适用于表模型。** | **json** | +| `mqtt_max_message_size` | mqtt 消息最大长度（字节） | 1048576 | + +## 3. 写入协议 + +* 行协议语法格式 + +```JavaScript +[,=[,=]][ =[,=]] =[,=] [] +``` + +* 示例 + +```JavaScript +myMeasurement,tag1=value1,tag2=value2 attr1=value1,attr2=value2 fieldKey="fieldValue" 1556813561098000000 +``` + +![](/img/mqtt-table-2.png) + + +## 4. 命名约定 + +* 数据库名称 + +MQTT topic 名称用 `/` 分割后， 第一串内容作为数据库名称。 + +```Properties +topic: stock/Legacy +databaseName: stock + + +topic: stock/Legacy/# +databaseName:stock +``` + +* 表名称 + +表名称使用行协议中的 ``。 + +* 类型标识 + +| Filed 内容 | IoTDB 数据类型 | +|--------------------------------------------------------------| ---------------- | +| 1
1.12 | DOUBLE | +| 1`f`
1.12`f` | FLOAT | +| 1`i`
123`i` | INT64 | +| 1`u`
123`u` | INT64| +| 1`i32`
123`i32` | INT32 | +| `"xxx"` | TEXT | +| `t`,`T`,`true`,`True`,`TRUE`
`f`,`F`,`false`,`False`,`FALSE` | BOOLEAN | + + +## 5. 代码示例 +以下是 mqtt 客户端将消息发送到 IoTDB 服务器的示例。 + + ```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +String DATABASE = "myMqttTest"; +connection.connect(); + +String payload = + "test1,tag1=t1,tag2=t2 attr3=a5,attr4=a4 field1=\"fieldValue1\",field2=1i,field3=1u 1"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 2"; +connection.publish(DATABASE, payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "# It's a remark\n " + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 6"; + connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); + Thread.sleep(10); + +//批量写入示例 +payload = + "test1,tag1=t1,tag2=t2 field7=t,field8=T,field9=true 3 \n " + + "test1,tag1=t1,tag2=t2 field7=f,field8=F,field9=FALSE 4"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +//批量写入示例 +payload = + "test1,tag1=t1,tag2=t2 attr1=a1,attr2=a2 field1=\"fieldValue1\",field2=1i,field3=1u 4 \n " + + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 5"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +connection.disconnect(); + ``` + + +## 6. 自定义 MQTT 消息格式 + +事实上可以通过简单编程来实现 MQTT 消息的格式自定义。 +可以在源码的 [example/mqtt-customize](https://github.com/apache/iotdb/tree/master/example/mqtt-customize) 项目中找到一个简单示例。 + +步骤: +1. 创建一个 Java 项目，增加如下依赖 +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. 创建一个实现类，实现接口 `org.apache.iotdb.db.mqtt.protocol.PayloadFormatter` + +```java +package org.apache.iotdb.mqtt.server; + +import io.netty.buffer.ByteBuf; +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +public class CustomizedLinePayloadFormatter implements PayloadFormatter { + + @Override + public List format(String topic, ByteBuf payload) { + // Suppose the payload is a line format + if (payload == null) { + return null; + } + + String line = payload.toString(StandardCharsets.UTF_8); + // parse data from the line and generate Messages and put them into List ret + List ret = new ArrayList<>(); + // this is just an example, so we just generate some Messages directly + for (int i = 0; i < 3; i++) { + long ts = i; + TableMessage message = new TableMessage(); + + // Parsing Database Name + message.setDatabase("db" + i); + + //Parsing Table Names + message.setTable("t" + i); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add("tag1" + i); + tagKeys.add("tag2" + i); + List tagValues = new ArrayList<>(); + tagValues.add("t_value1" + i); + tagValues.add("t_value2" + i); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add("attr1" + i); + attributeKeys.add("attr2" + i); + attributeValues.add("a_value1" + i); + attributeValues.add("a_value2" + i); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList("field1" + i, "field2" + i); + List dataTypes = Arrays.asList(TSDataType.FLOAT, TSDataType.FLOAT); + List values = Arrays.asList("4.0" + i, "5.0" + i); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + //// Parsing timestamp + message.setTimestamp(ts); + ret.add(message); + } + return ret; + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-system.properties as the following string: + return "CustomizedLine"; + } +} +``` + + +3. 修改项目中的 `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` 文件: + 将示例中的文件内容清除，并将刚才的实现类的全名（包名.类名）写入文件中。注意，这个文件中只有一行。 + 在本例中，文件内容为: `org.apache.iotdb.mqtt.server.CustomizedLinePayloadFormatter` +4. 编译项目生成一个 jar 包: `mvn package -DskipTests` + + +在 IoTDB 服务端: +1. 创建 ${IOTDB_HOME}/ext/mqtt/ 文件夹, 将刚才的 jar 包放入此文件夹。 +2. 打开 MQTT 服务参数. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. 用刚才的实现类中的 getName() 方法的返回值 设置为 `conf/iotdb-system.properties` 中 `mqtt_payload_formatter` 的值， + , 在本例中，为 `CustomizedLine` +4. 启动 IoTDB +5. 搞定 + +More: MQTT 协议的消息不限于 line，你还可以用任意二进制。通过如下函数获得： +`payload.forEachByte()` or `payload.array`。 + + +## 7. 注意事项 + +为避免因缺省client_id引发的兼容性问题，强烈建议在所有MQTT客户端中始终显式地提供唯一且非空的 client_id。 +不同客户端在client_id缺失或为空时的表现并不一致，常见示例如下： +1. 显式传入空字符串 +• MQTTX：client_id=""时，IoTDB会直接丢弃消息； +• mosquitto_pub：client_id=""时，IoTDB能正常接收消息。 +2. 完全不传client_id +• MQTTX：消息可被IoTDB正常接收； +• mosquitto_pub：IoTDB拒绝连接。 +由此可见，显式指定唯一且非空的client_id是消除上述差异、确保消息可靠投递的最简单做法。 \ No newline at end of file diff --git a/src/zh/UserGuide/latest-Table/API/Programming-MQTT_timecho.md b/src/zh/UserGuide/latest-Table/API/Programming-MQTT_timecho.md new file mode 100644 index 000000000..7076d466b --- /dev/null +++ b/src/zh/UserGuide/latest-Table/API/Programming-MQTT_timecho.md @@ -0,0 +1,264 @@ + +# MQTT 协议 + +## 1. 概述 + +MQTT 是一种专为物联网（IoT）和低带宽环境设计的轻量级消息传输协议，基于发布/订阅（Pub/Sub）模型，支持设备间高效、可靠的双向通信。其核心目标是低功耗、低带宽消耗和高实时性，尤其适合网络不稳定或资源受限的场景（如传感器、移动设备）。 + +IoTDB 深度集成了 MQTT 协议能力，完整兼容 MQTT v3.1（OASIS 国际标准协议）。IoTDB 服务器内置高性能 MQTT Broker 服务模块，无需第三方中间件，支持设备通过 MQTT 报文将时序数据直接写入 IoTDB 存储引擎。 + +![](/img/mqtt-table-1.png) + +注意，自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 MQTT 服务的 JAR 包。请使用该服务前联系天谋团队获取 JAR 包，并放置于 timechodb_home/lib 或者 timechodb_home/ext/external_service 路径下。 + +## 2. 配置方式 + +默认情况下，IoTDB MQTT 服务通过`${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`加载配置。 + +具体配置项如下： + +| **名称** | **描述** | **默认** | +|---------------------------| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | +| `enable_mqtt_service` | 是否启用 mqtt 服务 | FALSE | +| `mqtt_host` | mqtt 服务绑定主机 | 127.0.0.1 | +| `mqtt_port` | mqtt 服务绑定端口 | 1883 | +| `mqtt_handler_pool_size` | 处理 mqtt 消息的处理程序池大小 | 1 | +| **`mqtt_payload_formatter`** | **mqtt**​**​ 消息有效负载格式化程序。**​**可选项：**​​**`json`**​**：仅适用于树模型。**​​**`line`**​**：仅适用于表模型。** | **json** | +| `mqtt_max_message_size` | mqtt 消息最大长度（字节） | 1048576 | + +## 3. 写入协议 + +* 行协议语法格式 + +```JavaScript +[,=[,=]][ =[,=]] =[,=] [] +``` + +* 示例 + +```JavaScript +myMeasurement,tag1=value1,tag2=value2 attr1=value1,attr2=value2 fieldKey="fieldValue" 1556813561098000000 +``` + +![](/img/mqtt-table-2.png) + + +## 4. 命名约定 + +* 数据库名称 + +MQTT topic 名称用 `/` 分割后， 第一串内容作为数据库名称。 + +```Properties +topic: stock/Legacy +databaseName: stock + + +topic: stock/Legacy/# +databaseName:stock +``` + +* 表名称 + +表名称使用行协议中的 ``。 + +* 类型标识 + +| Filed 内容 | IoTDB 数据类型 | +|--------------------------------------------------------------| ---------------- | +| 1
1.12 | DOUBLE | +| 1`f`
1.12`f` | FLOAT | +| 1`i`
123`i` | INT64 | +| 1`u`
123`u` | INT64| +| 1`i32`
123`i32` | INT32 | +| `"xxx"` | TEXT | +| `t`,`T`,`true`,`True`,`TRUE`
`f`,`F`,`false`,`False`,`FALSE` | BOOLEAN | + + +## 5. 代码示例 +以下是 mqtt 客户端将消息发送到 IoTDB 服务器的示例。 + + ```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +String DATABASE = "myMqttTest"; +connection.connect(); + +String payload = + "test1,tag1=t1,tag2=t2 attr3=a5,attr4=a4 field1=\"fieldValue1\",field2=1i,field3=1u 1"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 2"; +connection.publish(DATABASE, payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +payload = "# It's a remark\n " + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 6"; + connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); + Thread.sleep(10); + +//批量写入示例 +payload = + "test1,tag1=t1,tag2=t2 field7=t,field8=T,field9=true 3 \n " + + "test1,tag1=t1,tag2=t2 field7=f,field8=F,field9=FALSE 4"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +//批量写入示例 +payload = + "test1,tag1=t1,tag2=t2 attr1=a1,attr2=a2 field1=\"fieldValue1\",field2=1i,field3=1u 4 \n " + + "test1,tag1=t1,tag2=t2 field4=2,field5=2i32,field6=2f 5"; +connection.publish(DATABASE + "/myTopic", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +Thread.sleep(10); + +connection.disconnect(); + ``` + + +## 6. 自定义 MQTT 消息格式 + +事实上可以通过简单编程来实现 MQTT 消息的格式自定义。 +可以在源码的 [example/mqtt-customize](https://github.com/apache/iotdb/tree/master/example/mqtt-customize) 项目中找到一个简单示例。 + +步骤: +1. 创建一个 Java 项目，增加如下依赖 +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. 创建一个实现类，实现接口 `org.apache.iotdb.db.mqtt.protocol.PayloadFormatter` + +```java +package org.apache.iotdb.mqtt.server; + +import io.netty.buffer.ByteBuf; +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +public class CustomizedLinePayloadFormatter implements PayloadFormatter { + + @Override + public List format(String topic, ByteBuf payload) { + // Suppose the payload is a line format + if (payload == null) { + return null; + } + + String line = payload.toString(StandardCharsets.UTF_8); + // parse data from the line and generate Messages and put them into List ret + List ret = new ArrayList<>(); + // this is just an example, so we just generate some Messages directly + for (int i = 0; i < 3; i++) { + long ts = i; + TableMessage message = new TableMessage(); + + // Parsing Database Name + message.setDatabase("db" + i); + + //Parsing Table Names + message.setTable("t" + i); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add("tag1" + i); + tagKeys.add("tag2" + i); + List tagValues = new ArrayList<>(); + tagValues.add("t_value1" + i); + tagValues.add("t_value2" + i); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add("attr1" + i); + attributeKeys.add("attr2" + i); + attributeValues.add("a_value1" + i); + attributeValues.add("a_value2" + i); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList("field1" + i, "field2" + i); + List dataTypes = Arrays.asList(TSDataType.FLOAT, TSDataType.FLOAT); + List values = Arrays.asList("4.0" + i, "5.0" + i); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + //// Parsing timestamp + message.setTimestamp(ts); + ret.add(message); + } + return ret; + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-system.properties as the following string: + return "CustomizedLine"; + } +} +``` + + +3. 修改项目中的 `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` 文件: + 将示例中的文件内容清除，并将刚才的实现类的全名（包名.类名）写入文件中。注意，这个文件中只有一行。 + 在本例中，文件内容为: `org.apache.iotdb.mqtt.server.CustomizedLinePayloadFormatter` +4. 编译项目生成一个 jar 包: `mvn package -DskipTests` + + +在 IoTDB 服务端: +1. 创建 ${IOTDB_HOME}/ext/mqtt/ 文件夹, 将刚才的 jar 包放入此文件夹。 +2. 打开 MQTT 服务参数. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. 用刚才的实现类中的 getName() 方法的返回值 设置为 `conf/iotdb-system.properties` 中 `mqtt_payload_formatter` 的值， + , 在本例中，为 `CustomizedLine` +4. 启动 IoTDB +5. 搞定 + +More: MQTT 协议的消息不限于 line，你还可以用任意二进制。通过如下函数获得： +`payload.forEachByte()` or `payload.array`。 + + +## 7. 注意事项 + +为避免因缺省client_id引发的兼容性问题，强烈建议在所有MQTT客户端中始终显式地提供唯一且非空的 client_id。 +不同客户端在client_id缺失或为空时的表现并不一致，常见示例如下： +1. 显式传入空字符串 +• MQTTX：client_id=""时，IoTDB会直接丢弃消息； +• mosquitto_pub：client_id=""时，IoTDB能正常接收消息。 +2. 完全不传client_id +• MQTTX：消息可被IoTDB正常接收； +• mosquitto_pub：IoTDB拒绝连接。 +由此可见，显式指定唯一且非空的client_id是消除上述差异、确保消息可靠投递的最简单做法。 \ No newline at end of file diff --git a/src/zh/UserGuide/latest-Table/API/RestServiceV1.md b/src/zh/UserGuide/latest-Table/API/RestServiceV1.md index 16fe30443..ff8429cae 100644 --- a/src/zh/UserGuide/latest-Table/API/RestServiceV1.md +++ b/src/zh/UserGuide/latest-Table/API/RestServiceV1.md @@ -1,3 +1,6 @@ +--- +redirectTo: RestServiceV1_apache.html +--- - -# RestAPI V1 - -IoTDB 的 RESTful 服务可用于查询、写入和管理操作，它使用 OpenAPI 标准来定义接口并生成框架。 - -## 1. 开启 RESTful 服务 - -Restful 服务默认情况是关闭的，开启 restful 功能需要找到 IoTDB 安装目录下的`conf/iotdb-system.properties`文件，将 `enable_rest_service` 设置为 `true` ，然后重启 datanode 进程。 - -```Plain - enable_rest_service=true -``` - -## 2. 鉴权 - -除了检活接口 `/ping`，restful服务使用了基础（basic）鉴权，每次 URL 请求都需要在 header 中携带 `'Authorization':'Basic' + base64.encode(username + ':' + password)`。 - -示例中使用的用户名为：`root`，密码为：`root`，对应的 Basic 鉴权 Header 格式为 - -```Plain -Authorization : Basic cm9vdDpyb290 -``` - -- 若用户名密码认证失败，则返回如下信息： -- HTTP 状态码：801 -- 返回结构体如下 - -```JSON -{"code":801,"message":"WRONG_LOGIN_PASSWORD"} -``` - -- 若未设置 `Authorization`，则返回如下信息： -- HTTP 状态码：800 -- 返回结构体如下 - -```Plain -{"code":800,"message":"INIT_AUTH_ERROR"} -``` - -## 3. 接口定义 - -### 3.1 ping - -ping 接口可以用于线上服务检活。 - -请求方式：`GET` - -请求路径：http://ip:port/ping - -请求示例： - -```Plain -curl http://127.0.0.1:18080/ping -``` - -返回的 HTTP 状态码： - -- `200`：当前服务工作正常，可以接收外部请求。 -- `503`：当前服务出现异常，不能接收外部请求。 - -响应参数： - -| 参数名称 | 参数类型 | 参数描述 | -| -------- | -------- | -------- | -| code | integer | 状态码 | -| message | string | 信息提示 | - -响应示例： - -- HTTP 状态码为 `200` 时： - -```JSON -{ "code": 200, "message": "SUCCESS_STATUS"} -``` - -- HTTP 状态码为 `503` 时： - -```JSON -{ "code": 500, "message": "thrift service is unavailable"} -``` - -注意：`ping` 接口访问不需要鉴权。 - -### 3.2 查询接口 - -- 请求地址：`/rest/table/v1/query` - -- 请求方式：post - -- Request 格式 - -请求头:`application/json` - -请求参数说明 - -| 参数名称 | 参数类型 | 是否必填 | 参数描述 | -| --------- | -------- | -------- | ------------------------------------------------------------ | -| database | string | 是 | 数据库名称 | -| sql | string | 是 | | -| row_limit | int | 否 | 一次查询能返回的结果集的最大行数。 如果不设置该参数，将使用配置文件的 `rest_query_default_row_size_limit` 作为默认值。 当返回结果集的行数超出限制时，将返回状态码 `411`。 | - -- Response 格式 - -| 参数名称 | 参数类型 | 参数描述 | -| ------------ | -------- | ------------------------------------------------------------ | -| column_names | array | 列名 | -| data_types | array | 每一列的类型 | -| values | array | 二维数组，第一维与结果集所有行，第二维数组代表结果集的每一行，每一个元素为一列，长度与column_names的长度相同。 | - -- 请求示例 - -```JSON -curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","sql":"select s1,s2,s3 from test_table"}' http://127.0.0.1:18080/rest/table/v1/query -``` - -- 响应示例： - -```JSON -{ - "column_names": [ - "s1", - "s2", - "s3" - ], - "data_types": [ - "STRING", - "BOOLEAN", - "INT32" - ], - "values": [ - [ - "a11", - true, - 2024 - ], - [ - "a11", - false, - 2025 - ] - ] -} -``` - -### 3.3 非查询接口 - -- 请求地址：`/rest/table/v1/nonQuery` - -- 请求方式：post - -- Request 格式 - - - 请求头:`application/json` - - - 请求参数说明 - -| 参数名称 | 参数类型 | 是否必填 | 参数描述 | -| -------- | -------- | -------- | -------- | -| sql | string | 是 | | -| database | string | 否 | 数据库名 | - -- Response 格式 - -| 参数名称 | 参数类型 | 参数描述 | -| -------- | -------- | -------- | -| code | integer | 状态码 | -| message | string | 信息提示 | - -- 请求示例 - -```JSON -#创建数据库 -curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"create database test","database":""}' http://127.0.0.1:18080/rest/table/v1/nonQuery -#在test库中创建表test_table -curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"CREATE TABLE table1 (time TIMESTAMP TIME,region STRING TAG,plant_id STRING TAG,device_id STRING TAG,model_id STRING ATTRIBUTE,maintenance STRING ATTRIBUTE,temperature FLOAT FIELD,humidity FLOAT FIELD,status Boolean FIELD,arrival_time TIMESTAMP FIELD) WITH (TTL=31536000000)","database":"test"}' http://127.0.0.1:18080/rest/table/v1/nonQuery -``` - -- 响应示例： - -```JSON -{ - "code": 200, - "message": "SUCCESS_STATUS" -} -``` - -### 3.4 批量写入接口 - -- 请求地址：`/rest/table/v1/insertTablet` - -- 请求方式：post - -- Request 格式 - -请求头:`application/json` - -请求参数说明 - -| 参数名称 | 参数类型 | 是否必填 | 参数描述 | -| ----------------- | -------- | -------- | ------------------------------------------------------------ | -| database | string | 是 | 数据库名称 | -| table | string | 是 | 表名 | -| column_names | array | 是 | 列名 | -| column_categories | array | 是 | 列类别(TAG,FIELD,*ATTRIBUTE*) | -| data_types | array | 是 | 数据类型 | -| timestamps | array | 是 | 时间列 | -| values | array | 是 | 值列，每一列中的值可以为 `null`二维数组第一层长度跟timestamps长度相同。第二层长度跟column_names长度相同 | - -- Response 格式 - -响应参数: - -| 参数名称 | 参数类型 | 参数描述 | -| -------- | -------- | -------- | -| code | integer | 状态码 | -| message | string | 信息提示 | - -- 请求示例 - -```JSON -curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","column_categories":["TAG","FIELD","FIELD"],"timestamps":[1739702535000,1739789055000],"column_names":["s1","s2","s3"],"data_types":["STRING","BOOLEAN","INT32"],"values":[["a11",true,2024],["a11",false,2025]],"table":"test_table"}' http://127.0.0.1:18080/rest/table/v1/insertTablet -``` - -- 响应示例 - -```JSON -{ - "code": 200, - "message": "SUCCESS_STATUS" -} -``` - - -## 4. 配置 - -配置文件位于 `iotdb-system.properties` 中。 - -- 将 `enable_rest_service` 设置为 `true` 表示启用该模块，而将 `false` 表示禁用该模块。默认情况下，该值为 `false`。 - -```Plain -enable_rest_service=true -``` - -- 仅在 `enable_rest_service=true` 时生效。将 `rest_service_port `设置为数字（1025~65535），以自定义REST服务套接字端口，默认情况下，值为 `18080`。 - -```Plain -rest_service_port=18080 -``` - -- 将 'enable_swagger' 设置 'true' 启用swagger来展示rest接口信息, 而设置为 'false' 关闭该功能. 默认情况下，该值为 `false`。 - -```Plain -enable_swagger=false -``` - -- 一次查询能返回的结果集最大行数。当返回结果集的行数超出参数限制时，您只会得到在行数范围内的结果集，且将得到状态码`411`。 - -```Plain -rest_query_default_row_size_limit=10000 -``` - -- 缓存客户登录信息的过期时间（用于加速用户鉴权的速度，单位为秒，默认是8个小时） - -```Plain -cache_expire_in_seconds=28800 -``` - -- 缓存中存储的最大用户数量（默认是100） - -```Plain -cache_max_num=100 -``` - -- 缓存初始容量（默认是10） - -```Plain -cache_init_num=10 -``` - -- REST Service 是否开启 SSL 配置，将 `enable_https` 设置为 `true` 以启用该模块，而将 `false` 设置为禁用该模块。默认情况下，该值为 `false`。 - -```Plain -enable_https=false -``` - -- keyStore 所在路径（非必填） - -```Plain -key_store_path= -``` - -- keyStore 密码（非必填） - -```Plain -key_store_pwd= -``` - -- trustStore 所在路径（非必填） - -```Plain -trust_store_path="" -``` - -- trustStore 密码（非必填） - -```Plain -trust_store_pwd="" -``` - -- SSL 超时时间，单位为秒 - -```Plain -idle_timeout_in_seconds=5000 -``` \ No newline at end of file +--> \ No newline at end of file diff --git a/src/zh/UserGuide/latest-Table/API/RestServiceV1_apache.md b/src/zh/UserGuide/latest-Table/API/RestServiceV1_apache.md new file mode 100644 index 000000000..16fe30443 --- /dev/null +++ b/src/zh/UserGuide/latest-Table/API/RestServiceV1_apache.md @@ -0,0 +1,334 @@ + + +# RestAPI V1 + +IoTDB 的 RESTful 服务可用于查询、写入和管理操作，它使用 OpenAPI 标准来定义接口并生成框架。 + +## 1. 开启 RESTful 服务 + +Restful 服务默认情况是关闭的，开启 restful 功能需要找到 IoTDB 安装目录下的`conf/iotdb-system.properties`文件，将 `enable_rest_service` 设置为 `true` ，然后重启 datanode 进程。 + +```Plain + enable_rest_service=true +``` + +## 2. 鉴权 + +除了检活接口 `/ping`，restful服务使用了基础（basic）鉴权，每次 URL 请求都需要在 header 中携带 `'Authorization':'Basic' + base64.encode(username + ':' + password)`。 + +示例中使用的用户名为：`root`，密码为：`root`，对应的 Basic 鉴权 Header 格式为 + +```Plain +Authorization : Basic cm9vdDpyb290 +``` + +- 若用户名密码认证失败，则返回如下信息： +- HTTP 状态码：801 +- 返回结构体如下 + +```JSON +{"code":801,"message":"WRONG_LOGIN_PASSWORD"} +``` + +- 若未设置 `Authorization`，则返回如下信息： +- HTTP 状态码：800 +- 返回结构体如下 + +```Plain +{"code":800,"message":"INIT_AUTH_ERROR"} +``` + +## 3. 接口定义 + +### 3.1 ping + +ping 接口可以用于线上服务检活。 + +请求方式：`GET` + +请求路径：http://ip:port/ping + +请求示例： + +```Plain +curl http://127.0.0.1:18080/ping +``` + +返回的 HTTP 状态码： + +- `200`：当前服务工作正常，可以接收外部请求。 +- `503`：当前服务出现异常，不能接收外部请求。 + +响应参数： + +| 参数名称 | 参数类型 | 参数描述 | +| -------- | -------- | -------- | +| code | integer | 状态码 | +| message | string | 信息提示 | + +响应示例： + +- HTTP 状态码为 `200` 时： + +```JSON +{ "code": 200, "message": "SUCCESS_STATUS"} +``` + +- HTTP 状态码为 `503` 时： + +```JSON +{ "code": 500, "message": "thrift service is unavailable"} +``` + +注意：`ping` 接口访问不需要鉴权。 + +### 3.2 查询接口 + +- 请求地址：`/rest/table/v1/query` + +- 请求方式：post + +- Request 格式 + +请求头:`application/json` + +请求参数说明 + +| 参数名称 | 参数类型 | 是否必填 | 参数描述 | +| --------- | -------- | -------- | ------------------------------------------------------------ | +| database | string | 是 | 数据库名称 | +| sql | string | 是 | | +| row_limit | int | 否 | 一次查询能返回的结果集的最大行数。 如果不设置该参数，将使用配置文件的 `rest_query_default_row_size_limit` 作为默认值。 当返回结果集的行数超出限制时，将返回状态码 `411`。 | + +- Response 格式 + +| 参数名称 | 参数类型 | 参数描述 | +| ------------ | -------- | ------------------------------------------------------------ | +| column_names | array | 列名 | +| data_types | array | 每一列的类型 | +| values | array | 二维数组，第一维与结果集所有行，第二维数组代表结果集的每一行，每一个元素为一列，长度与column_names的长度相同。 | + +- 请求示例 + +```JSON +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","sql":"select s1,s2,s3 from test_table"}' http://127.0.0.1:18080/rest/table/v1/query +``` + +- 响应示例： + +```JSON +{ + "column_names": [ + "s1", + "s2", + "s3" + ], + "data_types": [ + "STRING", + "BOOLEAN", + "INT32" + ], + "values": [ + [ + "a11", + true, + 2024 + ], + [ + "a11", + false, + 2025 + ] + ] +} +``` + +### 3.3 非查询接口 + +- 请求地址：`/rest/table/v1/nonQuery` + +- 请求方式：post + +- Request 格式 + + - 请求头:`application/json` + + - 请求参数说明 + +| 参数名称 | 参数类型 | 是否必填 | 参数描述 | +| -------- | -------- | -------- | -------- | +| sql | string | 是 | | +| database | string | 否 | 数据库名 | + +- Response 格式 + +| 参数名称 | 参数类型 | 参数描述 | +| -------- | -------- | -------- | +| code | integer | 状态码 | +| message | string | 信息提示 | + +- 请求示例 + +```JSON +#创建数据库 +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"create database test","database":""}' http://127.0.0.1:18080/rest/table/v1/nonQuery +#在test库中创建表test_table +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"CREATE TABLE table1 (time TIMESTAMP TIME,region STRING TAG,plant_id STRING TAG,device_id STRING TAG,model_id STRING ATTRIBUTE,maintenance STRING ATTRIBUTE,temperature FLOAT FIELD,humidity FLOAT FIELD,status Boolean FIELD,arrival_time TIMESTAMP FIELD) WITH (TTL=31536000000)","database":"test"}' http://127.0.0.1:18080/rest/table/v1/nonQuery +``` + +- 响应示例： + +```JSON +{ + "code": 200, + "message": "SUCCESS_STATUS" +} +``` + +### 3.4 批量写入接口 + +- 请求地址：`/rest/table/v1/insertTablet` + +- 请求方式：post + +- Request 格式 + +请求头:`application/json` + +请求参数说明 + +| 参数名称 | 参数类型 | 是否必填 | 参数描述 | +| ----------------- | -------- | -------- | ------------------------------------------------------------ | +| database | string | 是 | 数据库名称 | +| table | string | 是 | 表名 | +| column_names | array | 是 | 列名 | +| column_categories | array | 是 | 列类别(TAG,FIELD,*ATTRIBUTE*) | +| data_types | array | 是 | 数据类型 | +| timestamps | array | 是 | 时间列 | +| values | array | 是 | 值列，每一列中的值可以为 `null`二维数组第一层长度跟timestamps长度相同。第二层长度跟column_names长度相同 | + +- Response 格式 + +响应参数: + +| 参数名称 | 参数类型 | 参数描述 | +| -------- | -------- | -------- | +| code | integer | 状态码 | +| message | string | 信息提示 | + +- 请求示例 + +```JSON +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","column_categories":["TAG","FIELD","FIELD"],"timestamps":[1739702535000,1739789055000],"column_names":["s1","s2","s3"],"data_types":["STRING","BOOLEAN","INT32"],"values":[["a11",true,2024],["a11",false,2025]],"table":"test_table"}' http://127.0.0.1:18080/rest/table/v1/insertTablet +``` + +- 响应示例 + +```JSON +{ + "code": 200, + "message": "SUCCESS_STATUS" +} +``` + + +## 4. 配置 + +配置文件位于 `iotdb-system.properties` 中。 + +- 将 `enable_rest_service` 设置为 `true` 表示启用该模块，而将 `false` 表示禁用该模块。默认情况下，该值为 `false`。 + +```Plain +enable_rest_service=true +``` + +- 仅在 `enable_rest_service=true` 时生效。将 `rest_service_port `设置为数字（1025~65535），以自定义REST服务套接字端口，默认情况下，值为 `18080`。 + +```Plain +rest_service_port=18080 +``` + +- 将 'enable_swagger' 设置 'true' 启用swagger来展示rest接口信息, 而设置为 'false' 关闭该功能. 默认情况下，该值为 `false`。 + +```Plain +enable_swagger=false +``` + +- 一次查询能返回的结果集最大行数。当返回结果集的行数超出参数限制时，您只会得到在行数范围内的结果集，且将得到状态码`411`。 + +```Plain +rest_query_default_row_size_limit=10000 +``` + +- 缓存客户登录信息的过期时间（用于加速用户鉴权的速度，单位为秒，默认是8个小时） + +```Plain +cache_expire_in_seconds=28800 +``` + +- 缓存中存储的最大用户数量（默认是100） + +```Plain +cache_max_num=100 +``` + +- 缓存初始容量（默认是10） + +```Plain +cache_init_num=10 +``` + +- REST Service 是否开启 SSL 配置，将 `enable_https` 设置为 `true` 以启用该模块，而将 `false` 设置为禁用该模块。默认情况下，该值为 `false`。 + +```Plain +enable_https=false +``` + +- keyStore 所在路径（非必填） + +```Plain +key_store_path= +``` + +- keyStore 密码（非必填） + +```Plain +key_store_pwd= +``` + +- trustStore 所在路径（非必填） + +```Plain +trust_store_path="" +``` + +- trustStore 密码（非必填） + +```Plain +trust_store_pwd="" +``` + +- SSL 超时时间，单位为秒 + +```Plain +idle_timeout_in_seconds=5000 +``` \ No newline at end of file diff --git a/src/zh/UserGuide/latest-Table/API/RestServiceV1_timecho.md b/src/zh/UserGuide/latest-Table/API/RestServiceV1_timecho.md new file mode 100644 index 000000000..70c5eba40 --- /dev/null +++ b/src/zh/UserGuide/latest-Table/API/RestServiceV1_timecho.md @@ -0,0 +1,336 @@ + + +# RestAPI V1 + +IoTDB 的 RESTful 服务可用于查询、写入和管理操作，它使用 OpenAPI 标准来定义接口并生成框架。 + +注意：自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 REST 服务的 JAR 包，请使用该服务前联系天谋团队获取相应的 JAR 包，并放置于 timechodb_home/lib 或者 timechodb_home/ext/external_service 路径下。 + +## 1. 开启 RESTful 服务 + +Restful 服务默认情况是关闭的，开启 restful 功能需要找到 IoTDB 安装目录下的`conf/iotdb-system.properties`文件，将 `enable_rest_service` 设置为 `true` ，然后重启 datanode 进程。 + +```Plain + enable_rest_service=true +``` + +## 2. 鉴权 + +除了检活接口 `/ping`，restful服务使用了基础（basic）鉴权，每次 URL 请求都需要在 header 中携带 `'Authorization':'Basic' + base64.encode(username + ':' + password)`。 + +示例中使用的用户名为：`root`，密码为：`root`，对应的 Basic 鉴权 Header 格式为 + +```Plain +Authorization : Basic cm9vdDpyb290 +``` + +- 若用户名密码认证失败，则返回如下信息： +- HTTP 状态码：801 +- 返回结构体如下 + +```JSON +{"code":801,"message":"WRONG_LOGIN_PASSWORD"} +``` + +- 若未设置 `Authorization`，则返回如下信息： +- HTTP 状态码：800 +- 返回结构体如下 + +```Plain +{"code":800,"message":"INIT_AUTH_ERROR"} +``` + +## 3. 接口定义 + +### 3.1 ping + +ping 接口可以用于线上服务检活。 + +请求方式：`GET` + +请求路径：http://ip:port/ping + +请求示例： + +```Plain +curl http://127.0.0.1:18080/ping +``` + +返回的 HTTP 状态码： + +- `200`：当前服务工作正常，可以接收外部请求。 +- `503`：当前服务出现异常，不能接收外部请求。 + +响应参数： + +| 参数名称 | 参数类型 | 参数描述 | +| -------- | -------- | -------- | +| code | integer | 状态码 | +| message | string | 信息提示 | + +响应示例： + +- HTTP 状态码为 `200` 时： + +```JSON +{ "code": 200, "message": "SUCCESS_STATUS"} +``` + +- HTTP 状态码为 `503` 时： + +```JSON +{ "code": 500, "message": "thrift service is unavailable"} +``` + +注意：`ping` 接口访问不需要鉴权。 + +### 3.2 查询接口 + +- 请求地址：`/rest/table/v1/query` + +- 请求方式：post + +- Request 格式 + +请求头:`application/json` + +请求参数说明 + +| 参数名称 | 参数类型 | 是否必填 | 参数描述 | +| --------- | -------- | -------- | ------------------------------------------------------------ | +| database | string | 是 | 数据库名称 | +| sql | string | 是 | | +| row_limit | int | 否 | 一次查询能返回的结果集的最大行数。 如果不设置该参数，将使用配置文件的 `rest_query_default_row_size_limit` 作为默认值。 当返回结果集的行数超出限制时，将返回状态码 `411`。 | + +- Response 格式 + +| 参数名称 | 参数类型 | 参数描述 | +| ------------ | -------- | ------------------------------------------------------------ | +| column_names | array | 列名 | +| data_types | array | 每一列的类型 | +| values | array | 二维数组，第一维与结果集所有行，第二维数组代表结果集的每一行，每一个元素为一列，长度与column_names的长度相同。 | + +- 请求示例 + +```JSON +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","sql":"select s1,s2,s3 from test_table"}' http://127.0.0.1:18080/rest/table/v1/query +``` + +- 响应示例： + +```JSON +{ + "column_names": [ + "s1", + "s2", + "s3" + ], + "data_types": [ + "STRING", + "BOOLEAN", + "INT32" + ], + "values": [ + [ + "a11", + true, + 2024 + ], + [ + "a11", + false, + 2025 + ] + ] +} +``` + +### 3.3 非查询接口 + +- 请求地址：`/rest/table/v1/nonQuery` + +- 请求方式：post + +- Request 格式 + + - 请求头:`application/json` + + - 请求参数说明 + +| 参数名称 | 参数类型 | 是否必填 | 参数描述 | +| -------- | -------- | -------- | -------- | +| sql | string | 是 | | +| database | string | 否 | 数据库名 | + +- Response 格式 + +| 参数名称 | 参数类型 | 参数描述 | +| -------- | -------- | -------- | +| code | integer | 状态码 | +| message | string | 信息提示 | + +- 请求示例 + +```JSON +#创建数据库 +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"create database test","database":""}' http://127.0.0.1:18080/rest/table/v1/nonQuery +#在test库中创建表test_table +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"sql":"CREATE TABLE table1 (time TIMESTAMP TIME,region STRING TAG,plant_id STRING TAG,device_id STRING TAG,model_id STRING ATTRIBUTE,maintenance STRING ATTRIBUTE,temperature FLOAT FIELD,humidity FLOAT FIELD,status Boolean FIELD,arrival_time TIMESTAMP FIELD) WITH (TTL=31536000000)","database":"test"}' http://127.0.0.1:18080/rest/table/v1/nonQuery +``` + +- 响应示例： + +```JSON +{ + "code": 200, + "message": "SUCCESS_STATUS" +} +``` + +### 3.4 批量写入接口 + +- 请求地址：`/rest/table/v1/insertTablet` + +- 请求方式：post + +- Request 格式 + +请求头:`application/json` + +请求参数说明 + +| 参数名称 | 参数类型 | 是否必填 | 参数描述 | +| ----------------- | -------- | -------- | ------------------------------------------------------------ | +| database | string | 是 | 数据库名称 | +| table | string | 是 | 表名 | +| column_names | array | 是 | 列名 | +| column_categories | array | 是 | 列类别(TAG,FIELD,*ATTRIBUTE*) | +| data_types | array | 是 | 数据类型 | +| timestamps | array | 是 | 时间列 | +| values | array | 是 | 值列，每一列中的值可以为 `null`二维数组第一层长度跟timestamps长度相同。第二层长度跟column_names长度相同 | + +- Response 格式 + +响应参数: + +| 参数名称 | 参数类型 | 参数描述 | +| -------- | -------- | -------- | +| code | integer | 状态码 | +| message | string | 信息提示 | + +- 请求示例 + +```JSON +curl -H "Content-Type:application/json" -H "Authorization:Basic cm9vdDpyb290" -X POST --data '{"database":"test","column_categories":["TAG","FIELD","FIELD"],"timestamps":[1739702535000,1739789055000],"column_names":["s1","s2","s3"],"data_types":["STRING","BOOLEAN","INT32"],"values":[["a11",true,2024],["a11",false,2025]],"table":"test_table"}' http://127.0.0.1:18080/rest/table/v1/insertTablet +``` + +- 响应示例 + +```JSON +{ + "code": 200, + "message": "SUCCESS_STATUS" +} +``` + + +## 4. 配置 + +配置文件位于 `iotdb-system.properties` 中。 + +- 将 `enable_rest_service` 设置为 `true` 表示启用该模块，而将 `false` 表示禁用该模块。默认情况下，该值为 `false`。 + +```Plain +enable_rest_service=true +``` + +- 仅在 `enable_rest_service=true` 时生效。将 `rest_service_port `设置为数字（1025~65535），以自定义REST服务套接字端口，默认情况下，值为 `18080`。 + +```Plain +rest_service_port=18080 +``` + +- 将 'enable_swagger' 设置 'true' 启用swagger来展示rest接口信息, 而设置为 'false' 关闭该功能. 默认情况下，该值为 `false`。 + +```Plain +enable_swagger=false +``` + +- 一次查询能返回的结果集最大行数。当返回结果集的行数超出参数限制时，您只会得到在行数范围内的结果集，且将得到状态码`411`。 + +```Plain +rest_query_default_row_size_limit=10000 +``` + +- 缓存客户登录信息的过期时间（用于加速用户鉴权的速度，单位为秒，默认是8个小时） + +```Plain +cache_expire_in_seconds=28800 +``` + +- 缓存中存储的最大用户数量（默认是100） + +```Plain +cache_max_num=100 +``` + +- 缓存初始容量（默认是10） + +```Plain +cache_init_num=10 +``` + +- REST Service 是否开启 SSL 配置，将 `enable_https` 设置为 `true` 以启用该模块，而将 `false` 设置为禁用该模块。默认情况下，该值为 `false`。 + +```Plain +enable_https=false +``` + +- keyStore 所在路径（非必填） + +```Plain +key_store_path= +``` + +- keyStore 密码（非必填） + +```Plain +key_store_pwd= +``` + +- trustStore 所在路径（非必填） + +```Plain +trust_store_path="" +``` + +- trustStore 密码（非必填） + +```Plain +trust_store_pwd="" +``` + +- SSL 超时时间，单位为秒 + +```Plain +idle_timeout_in_seconds=5000 +``` \ No newline at end of file diff --git a/src/zh/UserGuide/latest-Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md b/src/zh/UserGuide/latest-Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md index c5d5251f4..e54fb9459 100644 --- a/src/zh/UserGuide/latest-Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md +++ b/src/zh/UserGuide/latest-Table/Deployment-and-Maintenance/IoTDB-Package_timecho.md @@ -43,3 +43,5 @@ | README_ZH.md | 文件 | 使用说明（中文版） | | README.md | 文件 | 使用说明（英文版） | | RELEASE_NOTES.md | 文件 | 版本说明 | + +注意：自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 MQTT 服务 和 REST 服务的 JAR 包。如需使用，请联系天谋团队获取。 \ No newline at end of file diff --git a/src/zh/UserGuide/latest/API/Programming-MQTT.md b/src/zh/UserGuide/latest/API/Programming-MQTT.md index 1ce846b1d..0a6a38616 100644 --- a/src/zh/UserGuide/latest/API/Programming-MQTT.md +++ b/src/zh/UserGuide/latest/API/Programming-MQTT.md @@ -1,3 +1,6 @@ +--- +redirectTo: Programming-MQTT_apache.html +--- - -# MQTT 协议 - -## 1. 概述 - -MQTT 是一种专为物联网（IoT）和低带宽环境设计的轻量级消息传输协议，基于发布/订阅（Pub/Sub）模型，支持设备间高效、可靠的双向通信。其核心目标是低功耗、低带宽消耗和高实时性，尤其适合网络不稳定或资源受限的场景（如传感器、移动设备）。 - -IoTDB 深度集成了 MQTT 协议能力，完整兼容 MQTT v3.1（OASIS 国际标准协议）。IoTDB 服务器内置高性能 MQTT Broker 服务模块，无需第三方中间件，支持设备通过 MQTT 报文将时序数据直接写入 IoTDB 存储引擎。 - - - -## 2. 内置 MQTT 服务 -内置的 MQTT 服务提供了通过 MQTT 直接连接到 IoTDB 的能力。 它侦听来自 MQTT 客户端的发布消息，然后立即将数据写入存储。 -MQTT 主题与 IoTDB 时间序列相对应。 -消息有效载荷可以由 Java SPI 加载的`PayloadFormatter`格式化为事件，默认实现为`JSONPayloadFormatter` - 默认的`json`格式化程序支持两种 json 格式以及由他们组成的json数组，以下是 MQTT 消息有效负载示例： - -```json - { - "device":"root.sg.d1", - "timestamp":1586076045524, - "measurements":["s1","s2"], - "values":[0.530635,0.530635] - } -``` -或者 -```json - { - "device":"root.sg.d1", - "timestamps":[1586076045524,1586076065526], - "measurements":["s1","s2"], - "values":[[0.530635,0.530635], [0.530655,0.530695]] - } -``` -或者以上两者的JSON数组形式。 - - - -## 3. MQTT 配置 -默认情况下，IoTDB MQTT 服务从`${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`加载配置。 - -配置如下： - -| **名称** | **描述** | **默认** | -|---------------------------| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | -| `enable_mqtt_service` | 是否启用 mqtt 服务 | FALSE | -| `mqtt_host` | mqtt 服务绑定主机 | 127.0.0.1 | -| `mqtt_port` | mqtt 服务绑定端口 | 1883 | -| `mqtt_handler_pool_size` | 处理 mqtt 消息的处理程序池大小 | 1 | -| **`mqtt_payload_formatter`** | **mqtt**​**​ 消息有效负载格式化程序。**​**可选项：**​​**`json`**​**：仅适用于树模型。**​​**`line`**​**：仅适用于表模型。** | **json** | -| `mqtt_max_message_size` | mqtt 消息最大长度（字节） | 1048576 | - - -## 4. 示例代码 -以下是 mqtt 客户端将消息发送到 IoTDB 服务器的示例。 - - ```java -MQTT mqtt = new MQTT(); -mqtt.setHost("127.0.0.1", 1883); -mqtt.setUserName("root"); -mqtt.setPassword("root"); - -BlockingConnection connection = mqtt.blockingConnection(); -connection.connect(); - -Random random = new Random(); -for (int i = 0; i < 10; i++) { - String payload = String.format("{\n" + - "\"device\":\"root.sg.d1\",\n" + - "\"timestamp\":%d,\n" + - "\"measurements\":[\"s1\"],\n" + - "\"values\":[%f]\n" + - "}", System.currentTimeMillis(), random.nextDouble()); - - connection.publish("root.sg.d1.s1", payload.getBytes(), QoS.AT_LEAST_ONCE, false); -} - -connection.disconnect(); - ``` - - -## 5. 自定义 MQTT 消息格式 - -在生产环境中，每个设备通常都配备了自己的 MQTT 客户端，且这些客户端的消息格式已经预先设定。如果按照 IoTDB 所支持的 MQTT 消息格式进行通信，就需要对现有的所有客户端进行全面的升级改造，这无疑会带来较高的成本。然而，我们可以通过简单的编程手段，轻松实现 MQTT 消息格式的自定义，而无需改造客户端。 -可以在源码的 [example/mqtt-customize](https://github.com/apache/iotdb/tree/rc/2.0.1/example/mqtt-customize) 项目中找到一个简单示例。 - -假定mqtt客户端传过来的是以下消息格式： -```json - { - "time":1586076045523, - "deviceID":"car_1", - "deviceType":"油车", - "point":"油量", - "value":10.0 -} -``` -或者JSON的数组形式： -```java -[ - { - "time":1586076045523, - "deviceID":"car_1", - "deviceType":"油车", - "point":"油量", - "value":10.0 - }, - { - "time":1586076045524, - "deviceID":"car_2", - "deviceType":"新能源车", - "point":"速度", - "value":80.0 - } -] -``` - - -则可以通过以下步骤设置设置自定义MQTT消息格式： - -1. 创建一个 Java 项目，增加如下依赖 -```xml - - org.apache.iotdb - iotdb-server - 2.0.4-SNAPSHOT - -``` -2. 创建一个实现类，实现接口 `org.apache.iotdb.db.mqtt.protocol.PayloadFormatter` - -```java -package org.apache.iotdb.mqtt.server; - -import org.apache.iotdb.db.protocol.mqtt.Message; -import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; -import org.apache.iotdb.db.protocol.mqtt.TableMessage; - -import com.google.common.collect.Lists; -import com.google.gson.Gson; -import com.google.gson.GsonBuilder; -import com.google.gson.JsonArray; -import com.google.gson.JsonElement; -import com.google.gson.JsonObject; -import com.google.gson.JsonParseException; -import io.netty.buffer.ByteBuf; -import org.apache.commons.lang3.NotImplementedException; -import org.apache.tsfile.enums.TSDataType; - -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -/** - * The Customized JSON payload formatter. one json format supported: { "time":1586076045523, - * "deviceID":"car_1", "deviceType":"新能源车", "point":"速度", "value":80.0 } - */ -public class CustomizedJsonPayloadFormatter implements PayloadFormatter { - private static final String JSON_KEY_TIME = "time"; - private static final String JSON_KEY_DEVICEID = "deviceID"; - private static final String JSON_KEY_DEVICETYPE = "deviceType"; - private static final String JSON_KEY_POINT = "point"; - private static final String JSON_KEY_VALUE = "value"; - private static final Gson GSON = new GsonBuilder().create(); - - @Override - public List format(String topic, ByteBuf payload) { - if (payload == null) { - return new ArrayList<>(); - } - String txt = payload.toString(StandardCharsets.UTF_8); - JsonElement jsonElement = GSON.fromJson(txt, JsonElement.class); - if (jsonElement.isJsonObject()) { - JsonObject jsonObject = jsonElement.getAsJsonObject(); - return formatTableRow(topic, jsonObject); - } else if (jsonElement.isJsonArray()) { - JsonArray jsonArray = jsonElement.getAsJsonArray(); - List messages = new ArrayList<>(); - for (JsonElement element : jsonArray) { - JsonObject jsonObject = element.getAsJsonObject(); - messages.addAll(formatTableRow(topic, jsonObject)); - } - return messages; - } - throw new JsonParseException("payload is invalidate"); - } - - @Override - @Deprecated - public List format(ByteBuf payload) { - throw new NotImplementedException(); - } - - private List formatTableRow(String topic, JsonObject jsonObject) { - TableMessage message = new TableMessage(); - String database = !topic.contains("/") ? topic : topic.substring(0, topic.indexOf("/")); - String table = "test_table"; - - // Parsing Database Name - message.setDatabase((database)); - - // Parsing Table Name - message.setTable(table); - - // Parsing Tags - List tagKeys = new ArrayList<>(); - tagKeys.add(JSON_KEY_DEVICEID); - List tagValues = new ArrayList<>(); - tagValues.add(jsonObject.get(JSON_KEY_DEVICEID).getAsString()); - message.setTagKeys(tagKeys); - message.setTagValues(tagValues); - - // Parsing Attributes - List attributeKeys = new ArrayList<>(); - List attributeValues = new ArrayList<>(); - attributeKeys.add(JSON_KEY_DEVICETYPE); - attributeValues.add(jsonObject.get(JSON_KEY_DEVICETYPE).getAsString()); - message.setAttributeKeys(attributeKeys); - message.setAttributeValues(attributeValues); - - // Parsing Fields - List fields = Arrays.asList(JSON_KEY_POINT); - List dataTypes = Arrays.asList(TSDataType.FLOAT); - List values = Arrays.asList(jsonObject.get(JSON_KEY_VALUE).getAsFloat()); - message.setFields(fields); - message.setDataTypes(dataTypes); - message.setValues(values); - - // Parsing timestamp - message.setTimestamp(jsonObject.get(JSON_KEY_TIME).getAsLong()); - return Lists.newArrayList(message); - } - - @Override - public String getName() { - // set the value of mqtt_payload_formatter in iotdb-common.properties as the following string: - return "CustomizedJson2Table"; - } - - @Override - public String getType() { - return PayloadFormatter.TABLE_TYPE; - } -} -``` -3. 修改项目中的 `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` 文件: - 将示例中的文件内容清除，并将刚才的实现类的全名（包名.类名）写入文件中。注意，这个文件中只有一行。 - 在本例中，文件内容为: `org.apache.iotdb.mqtt.server.CustomizedJsonPayloadFormatter` -4. 编译项目生成一个 jar 包: `mvn package -DskipTests` - - -在 IoTDB 服务端: -1. 创建 ${IOTDB_HOME}/ext/mqtt/ 文件夹, 将刚才的 jar 包放入此文件夹。 -2. 打开 MQTT 服务参数. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) -3. 用刚才的实现类中的 getName() 方法的返回值 设置为 `conf/iotdb-system.properties` 中 `mqtt_payload_formatter` 的值， - , 在本例中，为 `CustomizedJson2Table` -4. 启动 IoTDB -5. 搞定 - -More: MQTT 协议的消息不限于 json，你还可以用任意二进制。通过如下函数获得： -`payload.forEachByte()` or `payload.array`。 - - -## 6. 注意事项 - -为避免因缺省client_id引发的兼容性问题，强烈建议在所有MQTT客户端中始终显式地提供唯一且非空的 client_id。 -不同客户端在client_id缺失或为空时的表现并不一致，常见示例如下： -1. 显式传入空字符串 - • MQTTX：client_id=""时，IoTDB会直接丢弃消息； - • mosquitto_pub：client_id=""时，IoTDB能正常接收消息。 -2. 完全不传client_id - • MQTTX：消息可被IoTDB正常接收； - • mosquitto_pub：IoTDB拒绝连接。 - 由此可见，显式指定唯一且非空的client_id是消除上述差异、确保消息可靠投递的最简单做法。 \ No newline at end of file +--> \ No newline at end of file diff --git a/src/zh/UserGuide/latest/API/Programming-MQTT_apache.md b/src/zh/UserGuide/latest/API/Programming-MQTT_apache.md new file mode 100644 index 000000000..1ce846b1d --- /dev/null +++ b/src/zh/UserGuide/latest/API/Programming-MQTT_apache.md @@ -0,0 +1,293 @@ + + +# MQTT 协议 + +## 1. 概述 + +MQTT 是一种专为物联网（IoT）和低带宽环境设计的轻量级消息传输协议，基于发布/订阅（Pub/Sub）模型，支持设备间高效、可靠的双向通信。其核心目标是低功耗、低带宽消耗和高实时性，尤其适合网络不稳定或资源受限的场景（如传感器、移动设备）。 + +IoTDB 深度集成了 MQTT 协议能力，完整兼容 MQTT v3.1（OASIS 国际标准协议）。IoTDB 服务器内置高性能 MQTT Broker 服务模块，无需第三方中间件，支持设备通过 MQTT 报文将时序数据直接写入 IoTDB 存储引擎。 + + + +## 2. 内置 MQTT 服务 +内置的 MQTT 服务提供了通过 MQTT 直接连接到 IoTDB 的能力。 它侦听来自 MQTT 客户端的发布消息，然后立即将数据写入存储。 +MQTT 主题与 IoTDB 时间序列相对应。 +消息有效载荷可以由 Java SPI 加载的`PayloadFormatter`格式化为事件，默认实现为`JSONPayloadFormatter` + 默认的`json`格式化程序支持两种 json 格式以及由他们组成的json数组，以下是 MQTT 消息有效负载示例： + +```json + { + "device":"root.sg.d1", + "timestamp":1586076045524, + "measurements":["s1","s2"], + "values":[0.530635,0.530635] + } +``` +或者 +```json + { + "device":"root.sg.d1", + "timestamps":[1586076045524,1586076065526], + "measurements":["s1","s2"], + "values":[[0.530635,0.530635], [0.530655,0.530695]] + } +``` +或者以上两者的JSON数组形式。 + + + +## 3. MQTT 配置 +默认情况下，IoTDB MQTT 服务从`${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`加载配置。 + +配置如下： + +| **名称** | **描述** | **默认** | +|---------------------------| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | +| `enable_mqtt_service` | 是否启用 mqtt 服务 | FALSE | +| `mqtt_host` | mqtt 服务绑定主机 | 127.0.0.1 | +| `mqtt_port` | mqtt 服务绑定端口 | 1883 | +| `mqtt_handler_pool_size` | 处理 mqtt 消息的处理程序池大小 | 1 | +| **`mqtt_payload_formatter`** | **mqtt**​**​ 消息有效负载格式化程序。**​**可选项：**​​**`json`**​**：仅适用于树模型。**​​**`line`**​**：仅适用于表模型。** | **json** | +| `mqtt_max_message_size` | mqtt 消息最大长度（字节） | 1048576 | + + +## 4. 示例代码 +以下是 mqtt 客户端将消息发送到 IoTDB 服务器的示例。 + + ```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +connection.connect(); + +Random random = new Random(); +for (int i = 0; i < 10; i++) { + String payload = String.format("{\n" + + "\"device\":\"root.sg.d1\",\n" + + "\"timestamp\":%d,\n" + + "\"measurements\":[\"s1\"],\n" + + "\"values\":[%f]\n" + + "}", System.currentTimeMillis(), random.nextDouble()); + + connection.publish("root.sg.d1.s1", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +} + +connection.disconnect(); + ``` + + +## 5. 自定义 MQTT 消息格式 + +在生产环境中，每个设备通常都配备了自己的 MQTT 客户端，且这些客户端的消息格式已经预先设定。如果按照 IoTDB 所支持的 MQTT 消息格式进行通信，就需要对现有的所有客户端进行全面的升级改造，这无疑会带来较高的成本。然而，我们可以通过简单的编程手段，轻松实现 MQTT 消息格式的自定义，而无需改造客户端。 +可以在源码的 [example/mqtt-customize](https://github.com/apache/iotdb/tree/rc/2.0.1/example/mqtt-customize) 项目中找到一个简单示例。 + +假定mqtt客户端传过来的是以下消息格式： +```json + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"油车", + "point":"油量", + "value":10.0 +} +``` +或者JSON的数组形式： +```java +[ + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"油车", + "point":"油量", + "value":10.0 + }, + { + "time":1586076045524, + "deviceID":"car_2", + "deviceType":"新能源车", + "point":"速度", + "value":80.0 + } +] +``` + + +则可以通过以下步骤设置设置自定义MQTT消息格式： + +1. 创建一个 Java 项目，增加如下依赖 +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. 创建一个实现类，实现接口 `org.apache.iotdb.db.mqtt.protocol.PayloadFormatter` + +```java +package org.apache.iotdb.mqtt.server; + +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; +import org.apache.iotdb.db.protocol.mqtt.TableMessage; + +import com.google.common.collect.Lists; +import com.google.gson.Gson; +import com.google.gson.GsonBuilder; +import com.google.gson.JsonArray; +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParseException; +import io.netty.buffer.ByteBuf; +import org.apache.commons.lang3.NotImplementedException; +import org.apache.tsfile.enums.TSDataType; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +/** + * The Customized JSON payload formatter. one json format supported: { "time":1586076045523, + * "deviceID":"car_1", "deviceType":"新能源车", "point":"速度", "value":80.0 } + */ +public class CustomizedJsonPayloadFormatter implements PayloadFormatter { + private static final String JSON_KEY_TIME = "time"; + private static final String JSON_KEY_DEVICEID = "deviceID"; + private static final String JSON_KEY_DEVICETYPE = "deviceType"; + private static final String JSON_KEY_POINT = "point"; + private static final String JSON_KEY_VALUE = "value"; + private static final Gson GSON = new GsonBuilder().create(); + + @Override + public List format(String topic, ByteBuf payload) { + if (payload == null) { + return new ArrayList<>(); + } + String txt = payload.toString(StandardCharsets.UTF_8); + JsonElement jsonElement = GSON.fromJson(txt, JsonElement.class); + if (jsonElement.isJsonObject()) { + JsonObject jsonObject = jsonElement.getAsJsonObject(); + return formatTableRow(topic, jsonObject); + } else if (jsonElement.isJsonArray()) { + JsonArray jsonArray = jsonElement.getAsJsonArray(); + List messages = new ArrayList<>(); + for (JsonElement element : jsonArray) { + JsonObject jsonObject = element.getAsJsonObject(); + messages.addAll(formatTableRow(topic, jsonObject)); + } + return messages; + } + throw new JsonParseException("payload is invalidate"); + } + + @Override + @Deprecated + public List format(ByteBuf payload) { + throw new NotImplementedException(); + } + + private List formatTableRow(String topic, JsonObject jsonObject) { + TableMessage message = new TableMessage(); + String database = !topic.contains("/") ? topic : topic.substring(0, topic.indexOf("/")); + String table = "test_table"; + + // Parsing Database Name + message.setDatabase((database)); + + // Parsing Table Name + message.setTable(table); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add(JSON_KEY_DEVICEID); + List tagValues = new ArrayList<>(); + tagValues.add(jsonObject.get(JSON_KEY_DEVICEID).getAsString()); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add(JSON_KEY_DEVICETYPE); + attributeValues.add(jsonObject.get(JSON_KEY_DEVICETYPE).getAsString()); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList(JSON_KEY_POINT); + List dataTypes = Arrays.asList(TSDataType.FLOAT); + List values = Arrays.asList(jsonObject.get(JSON_KEY_VALUE).getAsFloat()); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + // Parsing timestamp + message.setTimestamp(jsonObject.get(JSON_KEY_TIME).getAsLong()); + return Lists.newArrayList(message); + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-common.properties as the following string: + return "CustomizedJson2Table"; + } + + @Override + public String getType() { + return PayloadFormatter.TABLE_TYPE; + } +} +``` +3. 修改项目中的 `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` 文件: + 将示例中的文件内容清除，并将刚才的实现类的全名（包名.类名）写入文件中。注意，这个文件中只有一行。 + 在本例中，文件内容为: `org.apache.iotdb.mqtt.server.CustomizedJsonPayloadFormatter` +4. 编译项目生成一个 jar 包: `mvn package -DskipTests` + + +在 IoTDB 服务端: +1. 创建 ${IOTDB_HOME}/ext/mqtt/ 文件夹, 将刚才的 jar 包放入此文件夹。 +2. 打开 MQTT 服务参数. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. 用刚才的实现类中的 getName() 方法的返回值 设置为 `conf/iotdb-system.properties` 中 `mqtt_payload_formatter` 的值， + , 在本例中，为 `CustomizedJson2Table` +4. 启动 IoTDB +5. 搞定 + +More: MQTT 协议的消息不限于 json，你还可以用任意二进制。通过如下函数获得： +`payload.forEachByte()` or `payload.array`。 + + +## 6. 注意事项 + +为避免因缺省client_id引发的兼容性问题，强烈建议在所有MQTT客户端中始终显式地提供唯一且非空的 client_id。 +不同客户端在client_id缺失或为空时的表现并不一致，常见示例如下： +1. 显式传入空字符串 + • MQTTX：client_id=""时，IoTDB会直接丢弃消息； + • mosquitto_pub：client_id=""时，IoTDB能正常接收消息。 +2. 完全不传client_id + • MQTTX：消息可被IoTDB正常接收； + • mosquitto_pub：IoTDB拒绝连接。 + 由此可见，显式指定唯一且非空的client_id是消除上述差异、确保消息可靠投递的最简单做法。 \ No newline at end of file diff --git a/src/zh/UserGuide/latest/API/Programming-MQTT_timecho.md b/src/zh/UserGuide/latest/API/Programming-MQTT_timecho.md new file mode 100644 index 000000000..f4164a9a3 --- /dev/null +++ b/src/zh/UserGuide/latest/API/Programming-MQTT_timecho.md @@ -0,0 +1,295 @@ + + +# MQTT 协议 + +## 1. 概述 + +MQTT 是一种专为物联网（IoT）和低带宽环境设计的轻量级消息传输协议，基于发布/订阅（Pub/Sub）模型，支持设备间高效、可靠的双向通信。其核心目标是低功耗、低带宽消耗和高实时性，尤其适合网络不稳定或资源受限的场景（如传感器、移动设备）。 + +IoTDB 深度集成了 MQTT 协议能力，完整兼容 MQTT v3.1（OASIS 国际标准协议）。IoTDB 服务器内置高性能 MQTT Broker 服务模块，无需第三方中间件，支持设备通过 MQTT 报文将时序数据直接写入 IoTDB 存储引擎。 + + + +注意，自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 MQTT 服务的 JAR 包。请使用该服务前联系天谋团队获取 JAR 包，并放置于 timechodb_home/lib 或者 timechodb_home/ext/external_service 路径下。 + +## 2. 内置 MQTT 服务 +内置的 MQTT 服务提供了通过 MQTT 直接连接到 IoTDB 的能力。 它侦听来自 MQTT 客户端的发布消息，然后立即将数据写入存储。 +MQTT 主题与 IoTDB 时间序列相对应。 +消息有效载荷可以由 Java SPI 加载的`PayloadFormatter`格式化为事件，默认实现为`JSONPayloadFormatter` + 默认的`json`格式化程序支持两种 json 格式以及由他们组成的json数组，以下是 MQTT 消息有效负载示例： + +```json + { + "device":"root.sg.d1", + "timestamp":1586076045524, + "measurements":["s1","s2"], + "values":[0.530635,0.530635] + } +``` +或者 +```json + { + "device":"root.sg.d1", + "timestamps":[1586076045524,1586076065526], + "measurements":["s1","s2"], + "values":[[0.530635,0.530635], [0.530655,0.530695]] + } +``` +或者以上两者的JSON数组形式。 + + + +## 3. MQTT 配置 +默认情况下，IoTDB MQTT 服务从`${IOTDB_HOME}/${IOTDB_CONF}/iotdb-system.properties`加载配置。 + +配置如下： + +| **名称** | **描述** | **默认** | +|---------------------------| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | +| `enable_mqtt_service` | 是否启用 mqtt 服务 | FALSE | +| `mqtt_host` | mqtt 服务绑定主机 | 127.0.0.1 | +| `mqtt_port` | mqtt 服务绑定端口 | 1883 | +| `mqtt_handler_pool_size` | 处理 mqtt 消息的处理程序池大小 | 1 | +| **`mqtt_payload_formatter`** | **mqtt**​**​ 消息有效负载格式化程序。**​**可选项：**​​**`json`**​**：仅适用于树模型。**​​**`line`**​**：仅适用于表模型。** | **json** | +| `mqtt_max_message_size` | mqtt 消息最大长度（字节） | 1048576 | + + +## 4. 示例代码 +以下是 mqtt 客户端将消息发送到 IoTDB 服务器的示例。 + + ```java +MQTT mqtt = new MQTT(); +mqtt.setHost("127.0.0.1", 1883); +mqtt.setUserName("root"); +mqtt.setPassword("root"); + +BlockingConnection connection = mqtt.blockingConnection(); +connection.connect(); + +Random random = new Random(); +for (int i = 0; i < 10; i++) { + String payload = String.format("{\n" + + "\"device\":\"root.sg.d1\",\n" + + "\"timestamp\":%d,\n" + + "\"measurements\":[\"s1\"],\n" + + "\"values\":[%f]\n" + + "}", System.currentTimeMillis(), random.nextDouble()); + + connection.publish("root.sg.d1.s1", payload.getBytes(), QoS.AT_LEAST_ONCE, false); +} + +connection.disconnect(); + ``` + + +## 5. 自定义 MQTT 消息格式 + +在生产环境中，每个设备通常都配备了自己的 MQTT 客户端，且这些客户端的消息格式已经预先设定。如果按照 IoTDB 所支持的 MQTT 消息格式进行通信，就需要对现有的所有客户端进行全面的升级改造，这无疑会带来较高的成本。然而，我们可以通过简单的编程手段，轻松实现 MQTT 消息格式的自定义，而无需改造客户端。 +可以在源码的 [example/mqtt-customize](https://github.com/apache/iotdb/tree/rc/2.0.1/example/mqtt-customize) 项目中找到一个简单示例。 + +假定mqtt客户端传过来的是以下消息格式： +```json + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"油车", + "point":"油量", + "value":10.0 +} +``` +或者JSON的数组形式： +```java +[ + { + "time":1586076045523, + "deviceID":"car_1", + "deviceType":"油车", + "point":"油量", + "value":10.0 + }, + { + "time":1586076045524, + "deviceID":"car_2", + "deviceType":"新能源车", + "point":"速度", + "value":80.0 + } +] +``` + + +则可以通过以下步骤设置设置自定义MQTT消息格式： + +1. 创建一个 Java 项目，增加如下依赖 +```xml + + org.apache.iotdb + iotdb-server + 2.0.4-SNAPSHOT + +``` +2. 创建一个实现类，实现接口 `org.apache.iotdb.db.mqtt.protocol.PayloadFormatter` + +```java +package org.apache.iotdb.mqtt.server; + +import org.apache.iotdb.db.protocol.mqtt.Message; +import org.apache.iotdb.db.protocol.mqtt.PayloadFormatter; +import org.apache.iotdb.db.protocol.mqtt.TableMessage; + +import com.google.common.collect.Lists; +import com.google.gson.Gson; +import com.google.gson.GsonBuilder; +import com.google.gson.JsonArray; +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParseException; +import io.netty.buffer.ByteBuf; +import org.apache.commons.lang3.NotImplementedException; +import org.apache.tsfile.enums.TSDataType; + +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +/** + * The Customized JSON payload formatter. one json format supported: { "time":1586076045523, + * "deviceID":"car_1", "deviceType":"新能源车", "point":"速度", "value":80.0 } + */ +public class CustomizedJsonPayloadFormatter implements PayloadFormatter { + private static final String JSON_KEY_TIME = "time"; + private static final String JSON_KEY_DEVICEID = "deviceID"; + private static final String JSON_KEY_DEVICETYPE = "deviceType"; + private static final String JSON_KEY_POINT = "point"; + private static final String JSON_KEY_VALUE = "value"; + private static final Gson GSON = new GsonBuilder().create(); + + @Override + public List format(String topic, ByteBuf payload) { + if (payload == null) { + return new ArrayList<>(); + } + String txt = payload.toString(StandardCharsets.UTF_8); + JsonElement jsonElement = GSON.fromJson(txt, JsonElement.class); + if (jsonElement.isJsonObject()) { + JsonObject jsonObject = jsonElement.getAsJsonObject(); + return formatTableRow(topic, jsonObject); + } else if (jsonElement.isJsonArray()) { + JsonArray jsonArray = jsonElement.getAsJsonArray(); + List messages = new ArrayList<>(); + for (JsonElement element : jsonArray) { + JsonObject jsonObject = element.getAsJsonObject(); + messages.addAll(formatTableRow(topic, jsonObject)); + } + return messages; + } + throw new JsonParseException("payload is invalidate"); + } + + @Override + @Deprecated + public List format(ByteBuf payload) { + throw new NotImplementedException(); + } + + private List formatTableRow(String topic, JsonObject jsonObject) { + TableMessage message = new TableMessage(); + String database = !topic.contains("/") ? topic : topic.substring(0, topic.indexOf("/")); + String table = "test_table"; + + // Parsing Database Name + message.setDatabase((database)); + + // Parsing Table Name + message.setTable(table); + + // Parsing Tags + List tagKeys = new ArrayList<>(); + tagKeys.add(JSON_KEY_DEVICEID); + List tagValues = new ArrayList<>(); + tagValues.add(jsonObject.get(JSON_KEY_DEVICEID).getAsString()); + message.setTagKeys(tagKeys); + message.setTagValues(tagValues); + + // Parsing Attributes + List attributeKeys = new ArrayList<>(); + List attributeValues = new ArrayList<>(); + attributeKeys.add(JSON_KEY_DEVICETYPE); + attributeValues.add(jsonObject.get(JSON_KEY_DEVICETYPE).getAsString()); + message.setAttributeKeys(attributeKeys); + message.setAttributeValues(attributeValues); + + // Parsing Fields + List fields = Arrays.asList(JSON_KEY_POINT); + List dataTypes = Arrays.asList(TSDataType.FLOAT); + List values = Arrays.asList(jsonObject.get(JSON_KEY_VALUE).getAsFloat()); + message.setFields(fields); + message.setDataTypes(dataTypes); + message.setValues(values); + + // Parsing timestamp + message.setTimestamp(jsonObject.get(JSON_KEY_TIME).getAsLong()); + return Lists.newArrayList(message); + } + + @Override + public String getName() { + // set the value of mqtt_payload_formatter in iotdb-common.properties as the following string: + return "CustomizedJson2Table"; + } + + @Override + public String getType() { + return PayloadFormatter.TABLE_TYPE; + } +} +``` +3. 修改项目中的 `src/main/resources/META-INF/services/org.apache.iotdb.db.protocol.mqtt.PayloadFormatter` 文件: + 将示例中的文件内容清除，并将刚才的实现类的全名（包名.类名）写入文件中。注意，这个文件中只有一行。 + 在本例中，文件内容为: `org.apache.iotdb.mqtt.server.CustomizedJsonPayloadFormatter` +4. 编译项目生成一个 jar 包: `mvn package -DskipTests` + + +在 IoTDB 服务端: +1. 创建 ${IOTDB_HOME}/ext/mqtt/ 文件夹, 将刚才的 jar 包放入此文件夹。 +2. 打开 MQTT 服务参数. (`enable_mqtt_service=true` in `conf/iotdb-system.properties`) +3. 用刚才的实现类中的 getName() 方法的返回值 设置为 `conf/iotdb-system.properties` 中 `mqtt_payload_formatter` 的值， + , 在本例中，为 `CustomizedJson2Table` +4. 启动 IoTDB +5. 搞定 + +More: MQTT 协议的消息不限于 json，你还可以用任意二进制。通过如下函数获得： +`payload.forEachByte()` or `payload.array`。 + + +## 6. 注意事项 + +为避免因缺省client_id引发的兼容性问题，强烈建议在所有MQTT客户端中始终显式地提供唯一且非空的 client_id。 +不同客户端在client_id缺失或为空时的表现并不一致，常见示例如下： +1. 显式传入空字符串 + • MQTTX：client_id=""时，IoTDB会直接丢弃消息； + • mosquitto_pub：client_id=""时，IoTDB能正常接收消息。 +2. 完全不传client_id + • MQTTX：消息可被IoTDB正常接收； + • mosquitto_pub：IoTDB拒绝连接。 + 由此可见，显式指定唯一且非空的client_id是消除上述差异、确保消息可靠投递的最简单做法。 \ No newline at end of file diff --git a/src/zh/UserGuide/latest/API/RestServiceV1_timecho.md b/src/zh/UserGuide/latest/API/RestServiceV1_timecho.md index 78859f1ba..76e59f0fd 100644 --- a/src/zh/UserGuide/latest/API/RestServiceV1_timecho.md +++ b/src/zh/UserGuide/latest/API/RestServiceV1_timecho.md @@ -22,6 +22,8 @@ # REST API V1（不推荐） IoTDB 的 RESTful 服务可用于查询、写入和管理操作，它使用 OpenAPI 标准来定义接口并生成框架。 +注意：自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 REST 服务的 JAR 包，请使用该服务前联系天谋团队获取相应的 JAR 包，并放置于 timechodb_home/lib 或者 timechodb_home/ext/external_service 路径下。 + ## 1. 开启RESTful 服务 RESTful 服务默认情况是关闭的 diff --git a/src/zh/UserGuide/latest/API/RestServiceV2_timecho.md b/src/zh/UserGuide/latest/API/RestServiceV2_timecho.md index 75869cecb..b122c0405 100644 --- a/src/zh/UserGuide/latest/API/RestServiceV2_timecho.md +++ b/src/zh/UserGuide/latest/API/RestServiceV2_timecho.md @@ -22,6 +22,8 @@ # REST API V2 IoTDB 的 RESTful 服务可用于查询、写入和管理操作，它使用 OpenAPI 标准来定义接口并生成框架。 +注意：自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 REST 服务的 JAR 包，请使用该服务前联系天谋团队获取相应的 JAR 包，并放置于 timechodb_home/lib 或者 timechodb_home/ext/external_service 路径下。 + ## 1. 开启RESTful 服务 RESTful 服务默认情况是关闭的 diff --git a/src/zh/UserGuide/latest/Basic-Concept/Write-Data_apache.md b/src/zh/UserGuide/latest/Basic-Concept/Write-Data_apache.md index 12f6dafd8..f508407c7 100644 --- a/src/zh/UserGuide/latest/Basic-Concept/Write-Data_apache.md +++ b/src/zh/UserGuide/latest/Basic-Concept/Write-Data_apache.md @@ -168,7 +168,7 @@ It costs 0.004s ## 4. MQTT写入 -参考 [内置 MQTT 服务](../API/Programming-MQTT.md#内置-mqtt-服务) +参考 [内置 MQTT 服务](../API/Programming-MQTT_apache.md#_2-内置-mqtt-服务) ## 5. 批量数据导入 diff --git a/src/zh/UserGuide/latest/Basic-Concept/Write-Data_timecho.md b/src/zh/UserGuide/latest/Basic-Concept/Write-Data_timecho.md index a7f0fb2e7..8c047caa1 100644 --- a/src/zh/UserGuide/latest/Basic-Concept/Write-Data_timecho.md +++ b/src/zh/UserGuide/latest/Basic-Concept/Write-Data_timecho.md @@ -168,7 +168,7 @@ It costs 0.004s ## 4. MQTT写入 -参考 [内置 MQTT 服务](../API/Programming-MQTT.md#内置-mqtt-服务) +参考 [内置 MQTT 服务](../API/Programming-MQTT_timecho.md#_2-内置-mqtt-服务) ## 5. 批量数据导入 diff --git a/src/zh/UserGuide/latest/Deployment-and-Maintenance/IoTDB-Package_timecho.md b/src/zh/UserGuide/latest/Deployment-and-Maintenance/IoTDB-Package_timecho.md index f502069a7..04e757c36 100644 --- a/src/zh/UserGuide/latest/Deployment-and-Maintenance/IoTDB-Package_timecho.md +++ b/src/zh/UserGuide/latest/Deployment-and-Maintenance/IoTDB-Package_timecho.md @@ -44,3 +44,5 @@ | README_ZH.md | 文件 | 使用说明（中文版） | | README.md | 文件 | 使用说明（英文版） | | RELEASE_NOTES.md | 文件 | 版本说明 | + +注意：自 V2.0.8 版本起，TimechoDB 安装包中默认不包含 MQTT 服务 和 REST 服务的 JAR 包。如需使用，请联系天谋团队获取。 \ No newline at end of file