Skip to content

Commit

Permalink
Update Flink-connector-starrocks.md
Browse files Browse the repository at this point in the history
  • Loading branch information
evelynzhaojie authored and wanpengfei-git committed Oct 18, 2023
1 parent 832d985 commit 514b823
Showing 1 changed file with 1 addition and 4 deletions.
5 changes: 1 addition & 4 deletions docs/loading/Flink-connector-starrocks.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,9 +166,6 @@ When you load data from Apache Flink® into StarRocks, take note of the followin
- If data loading pauses, you can increase the memory of the Flink task.
<<<<<<< HEAD
- If the preceding code runs as expected and StarRocks can receive data, but the data loading fails, check whether your machine can access the HTTP port of the backends (BEs) in your StarRocks cluster. If you can successfully ping the HTTP port returned by the execution of the SHOW BACKENDS command in your StarRocks cluster, your machine can access the HTTP port of the BEs in your StarRocks cluster. For example, a machine has a public IP address and a private IP address, the HTTP ports of frontends (FEs) and BEs can be accessed through the public IP address of the FEs and BEs, the IP address that is bounded with your StarRocks cluster is the private IP address, and the value of `loadurl` for the Flink task is the HTTP port of the public IP address of the FEs. The FEs forwards the data loading task to the private IP address of the BEs. In this example, if the machine cannot ping the private IP address of the BEs, the data loading fails.
=======
- If the version of Flink connector is 1.2.8 and later, it is recommended to specify the value of `sink.label-prefix`. Note that the label prefix must be unique among all types of loading in StarRocks, such as Flink jobs, Routine Load, and Broker Load.
- If the label prefix is specified, the Flink connector will use the label prefix to clean up lingering transactions that may be generated in some Flink
Expand Down Expand Up @@ -197,7 +194,7 @@ When you load data from Apache Flink® into StarRocks, take note of the followin
checkpoint, instead of due to timeout (which may cause data loss).
- `label_keep_max_second` and `label_keep_max_num`: StarRocks FE configurations, default values are `259200` and `1000`
respectively. For details, see [FE configurations](../loading/Loading_intro.md#fe-configurations). The value of `label_keep_max_second` needs to be larger than the downtime of the Flink job. Otherwise, the Flink connector can not check the state of transactions in StarRocks by using the transaction labels saved in the Flink's savepoint or checkpoint and figure out whether these transactions are committed or not, which may eventually lead to data loss.
respectively. For details, see [FE configurations](../loading/Loading_intro.md#fe-configurations). The value of `label_keep_max_second` needs to be larger than the downtime of the Flink job. Otherwise, the Flink connector can not check the state of transactions in StarRocks by using the transaction labels saved in the Flink's savepoint or checkpoint and figure out whether these transactions are committed, which may eventually lead to data loss.
These configurations are mutable and can be modified by using `ADMIN SET FRONTEND CONFIG`:
Expand Down

0 comments on commit 514b823

Please sign in to comment.