Skip to content

Commit

Permalink
[Enhancement] add DiskRootPath as part of show tablet statement (#50526)
Browse files Browse the repository at this point in the history 
## Why I'm doing:
improve the maintainability of disk decommission feature, add diskrootpath information to validate the migration of tablet across disks

## What I'm doing:
append diskrootpath information to show tablet statement to validate the migration of tablet across disks
```
MySQL [mydb]> show tablet from t1\G;
*************************** 1. row ***************************
               TabletId: 10196
              ReplicaId: 10197
              BackendId: 10001
             SchemaHash: 0
                Version: 1
            VersionHash: 0
      LstSuccessVersion: 1
  LstSuccessVersionHash: 0
       LstFailedVersion: -1
   LstFailedVersionHash: 0
          LstFailedTime: NULL
               DataSize: 0B
               RowCount: 0
                  State: NORMAL
LstConsistencyCheckTime: NULL
           CheckVersion: -1
       CheckVersionHash: 0
           VersionCount: 1
               PathHash: -551373370090831691
                MetaUrl: http://192.168.0.115:8040/api/meta/header/10196
       CompactionStatus: http://192.168.0.115:8040/api/compaction/show?tablet_id=10196
           DiskRootPath: /deploystarrocks/be/storage1
1 row in set (0.001 sec)
```
Fixes #50525
  • Loading branch information
@rohitrs1983
rohitrs1983 authored Sep 10, 2024
1 parent 5db79c8 commit b19731e
Show file tree
Hide file tree
Showing 5 changed files with 49 additions and 8 deletions.
8 changes: 5 additions & 3 deletions docs/en/sql-reference/sql-statements/table_bucket_part_index/SHOW_TABLET.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,9 @@ SHOW TABLET <tablet_id>
### Query information of tablets in a table or a partition

```plain
+----------+-----------+-----------+------------+---------+-------------+-------------------+-----------------------+------------------+----------------------+---------------+----------+----------+--------+-------------------------+--------------+------------------+--------------+----------+----------+-------------------+
| TabletId | ReplicaId | BackendId | SchemaHash | Version | VersionHash | LstSuccessVersion | LstSuccessVersionHash | LstFailedVersion | LstFailedVersionHash | LstFailedTime | DataSize | RowCount | State | LstConsistencyCheckTime | CheckVersion | CheckVersionHash | VersionCount | PathHash | MetaUrl | CompactionStatus |
+----------+-----------+-----------+------------+---------+-------------+-------------------+-----------------------+------------------+----------------------+---------------+----------+----------+--------+-------------------------+--------------+------------------+--------------+----------+----------+-------------------+
+----------+-----------+-----------+------------+---------+-------------+-------------------+-----------------------+------------------+----------------------+---------------+----------+----------+--------+-------------------------+--------------+------------------+--------------+----------+----------+-------------------+---------------+
| TabletId | ReplicaId | BackendId | SchemaHash | Version | VersionHash | LstSuccessVersion | LstSuccessVersionHash | LstFailedVersion | LstFailedVersionHash | LstFailedTime | DataSize | RowCount | State | LstConsistencyCheckTime | CheckVersion | CheckVersionHash | VersionCount | PathHash | MetaUrl | CompactionStatus | DiskRootPath |
+----------+-----------+-----------+------------+---------+-------------+-------------------+-----------------------+------------------+----------------------+---------------+----------+----------+--------+-------------------------+--------------+------------------+--------------+----------+----------+-------------------+---------------+
```

| **Field** | **Description** |
Expand All @@ -86,6 +86,7 @@ SHOW TABLET <tablet_id>
| PathHash | The hash of the directory in which the tablet is stored. |
| MetaUrl | The URL used to query more meta information. |
| CompactionStatus | The URL used to query data version compaction status. |
| DiskRootPath | The disk where the replica is located. |

### Query information of a specific tablet

Expand Down Expand Up @@ -169,6 +170,7 @@ DISTRIBUTED BY HASH(`k1`, `k2`, `k3`);
PathHash: 0
MetaUrl: http://172.26.92.141:8038/api/meta/header/9588955
CompactionStatus: http://172.26.92.141:8038/api/compaction/show?tablet_id=9588955
DiskRootPath: /storage/disk
```
- Example 2: Query information of tablet 9588955.
Expand Down
8 changes: 5 additions & 3 deletions docs/zh/sql-reference/sql-statements/table_bucket_part_index/SHOW_TABLET.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,9 +61,9 @@ SHOW TABLET <tablet_id>
### 查看某张表或分区内的所有 tablet

```plain
+----------+-----------+-----------+------------+---------+-------------+-------------------+-----------------------+------------------+----------------------+---------------+----------+----------+--------+-------------------------+--------------+------------------+--------------+----------+----------+-------------------+
| TabletId | ReplicaId | BackendId | SchemaHash | Version | VersionHash | LstSuccessVersion | LstSuccessVersionHash | LstFailedVersion | LstFailedVersionHash | LstFailedTime | DataSize | RowCount | State | LstConsistencyCheckTime | CheckVersion | CheckVersionHash | VersionCount | PathHash | MetaUrl | CompactionStatus |
+----------+-----------+-----------+------------+---------+-------------+-------------------+-----------------------+------------------+----------------------+---------------+----------+----------+--------+-------------------------+--------------+------------------+--------------+----------+----------+-------------------+
+----------+-----------+-----------+------------+---------+-------------+-------------------+-----------------------+------------------+----------------------+---------------+----------+----------+--------+-------------------------+--------------+------------------+--------------+----------+----------+-------------------+---------------+
| TabletId | ReplicaId | BackendId | SchemaHash | Version | VersionHash | LstSuccessVersion | LstSuccessVersionHash | LstFailedVersion | LstFailedVersionHash | LstFailedTime | DataSize | RowCount | State | LstConsistencyCheckTime | CheckVersion | CheckVersionHash | VersionCount | PathHash | MetaUrl | CompactionStatus | DiskRootPath |
+----------+-----------+-----------+------------+---------+-------------+-------------------+-----------------------+------------------+----------------------+---------------+----------+----------+--------+-------------------------+--------------+------------------+--------------+----------+----------+-------------------+---------------+
```

| **字段** | **说明** |
Expand All @@ -89,6 +89,7 @@ SHOW TABLET <tablet_id>
| PathHash | Tablet 存储目录的 hash。 |
| MetaUrl | 通过 URL 查询更多的 meta 信息。 |
| CompactionStatus | 通过 URL 查询 Compaction 状态。 |
| DiskRootPath | 副本所在的磁盘。 |

### 查看单个 tablet

Expand Down Expand Up @@ -172,6 +173,7 @@ DISTRIBUTED BY HASH(`k1`, `k2`, `k3`);
PathHash: 0
MetaUrl: http://172.26.92.141:8038/api/meta/header/9588955
CompactionStatus: http://172.26.92.141:8038/api/compaction/show?tablet_id=9588955
DiskRootPath: /storage/disk
```
- 查看 id 为 9588955 的 Tablet 的信息。
Expand Down
5 changes: 5 additions & 0 deletions fe/fe-core/src/main/java/com/starrocks/common/proc/LocalTabletsProcDir.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ public class LocalTabletsProcDir implements ProcDirInterface {
.add("DataSize").add("RowCount").add("State")
.add("LstConsistencyCheckTime").add("CheckVersion").add("CheckVersionHash")
.add("VersionCount").add("PathHash").add("MetaUrl").add("CompactionStatus")
.add("DiskRootPath")
.build();

private final Database db;
Expand Down Expand Up @@ -144,18 +145,22 @@ public List<List<Comparable>> fetchComparableResult(long version, long backendId
Backend backend = backendMap.get(replica.getBackendId());
String metaUrl;
String compactionUrl;
String diskRootPath;
if (backend != null) {
String hostPort = hideIpPort ? "*:0" :
NetUtils.getHostPortInAccessibleFormat(backend.getHost(), backend.getHttpPort());
metaUrl = String.format("http://" + hostPort + "/api/meta/header/%d", tabletId);
compactionUrl = String.format(
"http://" + hostPort + "/api/compaction/show?tablet_id=%d", tabletId);
diskRootPath = backend.getDiskRootPath(replica.getPathHash());
} else {
metaUrl = "N/A";
compactionUrl = "N/A";
diskRootPath = "N/A";
}
tabletInfo.add(metaUrl);
tabletInfo.add(compactionUrl);
tabletInfo.add(diskRootPath);

tabletInfos.add(tabletInfo);
}
Expand Down
11 changes: 11 additions & 0 deletions fe/fe-core/src/main/java/com/starrocks/system/Backend.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -373,6 +373,17 @@ public void cancelDisableDisk(String rootPath) throws DdlException {
}
}

public String getDiskRootPath(long pathHash) {
String rootPath = "Unknown";
for (DiskInfo diskInfo : disksRef.values()) {
if (diskInfo.getPathHash() == pathHash) {
rootPath = diskInfo.getRootPath();
break;
}
}
return rootPath;
}

public void setStorageMediumForAllDisks(TStorageMedium m) {
for (DiskInfo diskInfo : disksRef.values()) {
diskInfo.setStorageMedium(m);
Expand Down
25 changes: 23 additions & 2 deletions fe/fe-core/src/test/java/com/starrocks/common/proc/LocalTabletsProcDirTest.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import com.starrocks.catalog.Column;
import com.starrocks.catalog.DataProperty;
import com.starrocks.catalog.Database;
import com.starrocks.catalog.DiskInfo;
import com.starrocks.catalog.DistributionInfo;
import com.starrocks.catalog.HashDistributionInfo;
import com.starrocks.catalog.KeysType;
Expand Down Expand Up @@ -57,8 +58,24 @@ public void testFetchResultWithLocalTablet(@Mocked GlobalStateMgr globalStateMgr
@Mocked SystemInfoService systemInfoService) {
Map<Long, Backend> idToBackend = Maps.newHashMap();
long backendId = 20L;
idToBackend.put(backendId, new Backend(backendId, "127.0.0.1", 9050));
idToBackend.put(backendId, new Backend(backendId + 1, "127.0.0.2", 9050));
Backend b1 = new Backend(backendId, "127.0.0.1", 9050);
Map<String, DiskInfo> disks1 = Maps.newHashMap();
DiskInfo d1 = new DiskInfo("/home/disk1");
d1.setPathHash(1L);
disks1.put("/home/disk1", d1);
ImmutableMap<String, DiskInfo> immutableMap1 = ImmutableMap.copyOf(disks1);
b1.setDisks(immutableMap1);

Backend b2 = new Backend(backendId + 1, "127.0.0.2", 9050);
Map<String, DiskInfo> disks2 = Maps.newHashMap();
DiskInfo d2 = new DiskInfo("/home/disk2");
d2.setPathHash(2L);
disks2.put("/home/disk2", d2);
ImmutableMap<String, DiskInfo> immutableMap2 = ImmutableMap.copyOf(disks2);
b2.setDisks(immutableMap2);

idToBackend.put(backendId, b1);
idToBackend.put(backendId + 1, b2);

new Expectations() {
{
Expand Down Expand Up @@ -87,6 +104,8 @@ public void testFetchResultWithLocalTablet(@Mocked GlobalStateMgr globalStateMgr
// Replica
Replica replica1 = new Replica(replicaId, backendId, Replica.ReplicaState.NORMAL, 1, 0);
Replica replica2 = new Replica(replicaId + 1, backendId + 1, Replica.ReplicaState.NORMAL, 1, 0);
replica1.setPathHash(1L);
replica2.setPathHash(2L);

// Tablet
LocalTablet tablet1 = new LocalTablet(tablet1Id);
Expand Down Expand Up @@ -127,12 +146,14 @@ public void testFetchResultWithLocalTablet(@Mocked GlobalStateMgr globalStateMgr
System.out.println(result);
Assert.assertEquals(3, result.size());
Assert.assertEquals((long) result.get(0).get(0), tablet1Id);
Assert.assertEquals(result.get(0).get(21), "/home/disk1");
Assert.assertEquals((long) result.get(1).get(0), tablet1Id);
if ((long) result.get(0).get(1) == replicaId) {
Assert.assertEquals((long) result.get(0).get(2), backendId);
} else if ((long) result.get(0).get(1) == replicaId + 1) {
Assert.assertEquals((long) result.get(0).get(2), backendId + 1);
}
Assert.assertEquals(result.get(1).get(21), "/home/disk2");
Assert.assertEquals((long) result.get(2).get(0), tablet2Id);
Assert.assertEquals(result.get(2).get(1), -1);
Assert.assertEquals(result.get(2).get(2), -1);
Expand Down

0 comments on commit b19731e

Please sign in to comment.