Resolve comments

triton-inference-server · May 10, 2024 · f3371ae · f3371ae
1 parent 6c2c01c
commit f3371ae
Show file tree

Hide file tree

Showing 3 changed files with 63 additions and 39 deletions.
diff --git a/docs/user_guide/model_configuration.md b/docs/user_guide/model_configuration.md
@@ -324,14 +324,14 @@ config.pbtxt file.
 ## Custom Model Configuration
 
 Sometimes when multiple devices running Triton instances that share one
-model repository, it is nessecery to have models configured differently
+model repository, it is necessary to have models configured differently
 on each platform in order to achieve the best performance. Triton allows
 users to pick the custom model configuration name by setting `--model-config-name` option.
 
-For example, when running `./tritonserver --model-repository=</path/to/model/repository> --model-config-name=a100`,
-the server will search the custom configuration file `a100.pbtxt` under
+For example, when running `./tritonserver --model-repository=</path/to/model/repository> --model-config-name=h100`,
+the server will search the custom configuration file `h100.pbtxt` under
 `/path/to/model/repository/<model-name>/configs` directory for each model
-that is loaded. If `a100.pbtxt` exists, it will be used as the configuration
+that is loaded. If `h100.pbtxt` exists, it will be used as the configuration
 for this model. Otherwise, the default configuration `/path/to/model/repository/<model-name>/config.pbtxt`
 or [auto-generated model configuration](#auto-generated-model-configuration)
 will be selected based on the settings.
@@ -340,31 +340,63 @@ Custom model configuration also works with `Explicit` and `Poll` model
 control modes. Users may delete or add new custom configurations and the
 server will pick the configuration file for each loaded model dynamically.
 
-The repository layout looks like:
+Note: custom model configuration name should not contain any space character.
 
+Example 1: --model-config-name=h100
 ```
-  <model-repository-path>/
-    <model-name>/
-      [config.pbtxt]
-      [<output-labels-file> ...]
-      <configs>/
-        [<custom-config-file> ...]
-      <version>/
-        <model-definition-file>
-      <version>/
-        <model-definition-file>
-      ...
-    <model-name>/
-      [config.pbtxt]
-      [<output-labels-file> ...]
-      <configs>/
-        [<custom-config-file> ...]
-      <version>/
-        <model-definition-file>
-      <version>/
-        <model-definition-file>
-      ...
-    ...
+.
+└── model_repository/
+    ├── model_a/
+    │   ├── configs/
+    │   │   ├── v100.pbtxt
+    │   │   └── **h100.pbtxt**
+    │   └── config.pbtxt
+    ├── model_b/
+    │   ├── configs/
+    │   │   └── v100.pbtxt
+    │   └── **config.pbtxt**
+    └── model_c/
+        ├── configs/
+        │   └── config.pbtxt
+        └── **config.pbtxt**
+```
+
+Example 2: --model-config-name=config
+```
+.
+└── model_repository/
+    ├── model_a/
+    │   ├── configs/
+    │   │   ├── v100.pbtxt
+    │   │   └── h100.pbtxt
+    │   └── **config.pbtxt**
+    ├── model_b/
+    │   ├── configs/
+    │   │   └── v100.pbtxt
+    │   └── **config.pbtxt**
+    └── model_c/
+        ├── configs/
+        │   └── **config.pbtxt**
+        └── config.pbtxt
+```
+
+Example 3: --model-config-name not set
+```
+.
+└── model_repository/
+    ├── model_a/
+    │   ├── configs/
+    │   │   ├── v100.pbtxt
+    │   │   └── h100.pbtxt
+    │   └── **config.pbtxt**
+    ├── model_b/
+    │   ├── configs/
+    │   │   └── v100.pbtxt
+    │   └── **config.pbtxt**
+    └── model_c/
+        ├── configs/
+        │   └── config.pbtxt
+        └── **config.pbtxt**
 ```
 
 ### Default Max Batch Size and Dynamic Batcher

diff --git a/qa/L0_custom_model_config/test.sh b/qa/L0_custom_model_config/test.sh
@@ -70,7 +70,7 @@ test_custom_config()
     set -e
     if [ "$code" != "200" ]; then
         cat $out.out
-        echo -e "\n***\n*** Test Failed\n***"
+        echo -e "\n***\n*** Test Failed to GET model configuration\n***"
         RET=1
     fi
 
@@ -124,15 +124,6 @@ test_custom_config $VERSION_CUSTOM
 SERVER_ARGS="--model-repository=`pwd`/models --model-config-name=h200"
 test_custom_config $VERSION_DEFAULT
 
-# TODO: It seems that command argument parser does not accept value with spaces.
-# Test model-config-name="h100 v100"
-# VERSION_MULT_TOKENS="1,2"
-# (cd models/savedmodel_nobatch_float32_float32_float32 && \
-#      cp config.pbtxt configs/h100\ v100.pbtxt && \
-#      sed -i "s/^version_policy:.*/version_policy: { specific: { versions: [$VERSION_MULT_TOKENS] }}/" configs/h100\ v100.pbtxt)
-# SERVER_ARGS="--model-repository=`pwd`/models --model-config-name=\"h100\ v100\""
-# test_custom_config $VERSION_MULT_TOKENS
-
 # Test model-config-name=
 SERVER_ARGS="--model-repository=`pwd`/models --model-config-name="
 run_server

diff --git a/src/command_line_parser.cc b/src/command_line_parser.cc
@@ -443,8 +443,9 @@ TritonParser::SetupOptions()
        "effect if --model-control-mode=explicit is true."});
   model_repo_options_.push_back(
       {OPTION_CUSTOM_MODEL_CONFIG_NAME, "model-config-name", Option::ArgStr,
-       "The custom model configuration for models to load."
-       "For example: --model-config-name=a100. "
+       "The custom configuration name for models to load."
+       "The name should not contain any space character."
+       "For example: --model-config-name=h100. "
        "If --model-config-name is not set, Triton will use the default "
        "config.pbtxt."});
   model_repo_options_.push_back(