Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Dali model REST protocol test on triton for Kserve ( API ) #2132

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Added Dali model REST protocol test on triton for Kserve ( API ) #2132

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
19e3759
initial commit
Raghul-M Dec 11, 2024
c8aed02
uncommented
Raghul-M Dec 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ods_ci/tests/Resources/CLI/ModelServing/llm.resource
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ ${SERVICEMESH_CR_NS}= istio-system
... vllm-runtime=${LLM_RESOURCES_DIRPATH}/serving_runtimes/vllm_servingruntime_{{protocol}}.yaml
... ovms-runtime=${LLM_RESOURCES_DIRPATH}/serving_runtimes/ovms_servingruntime_{{protocol}}.yaml
... caikit-standalone-runtime=${LLM_RESOURCES_DIRPATH}/serving_runtimes/caikit_standalone_servingruntime_{{protocol}}.yaml # robocop: disable
... triton-kserve-rest=tests/Resources/Files/triton/triton_onnx_rest_servingruntime.yaml
${DOWNLOAD_PVC_FILEPATH}= ${LLM_RESOURCES_DIRPATH}/download_model_in_pvc.yaml
${DOWNLOAD_PVC_FILLED_FILEPATH}= ${LLM_RESOURCES_DIRPATH}/download_model_in_pvc_filled.yaml

${DOWNLOAD_PROMPTS_PVC_FILEPATH}= ${LLM_RESOURCES_DIRPATH}/download_prompts_in_pvc.yaml
${DOWNLOAD_PROMPTS_PVC_FILLED_FILEPATH}= ${LLM_RESOURCES_DIRPATH}/download_prompts_in_pvc_filled.yaml
${MATCHING_RATIO}= ${60}
Expand Down
1 change: 1 addition & 0 deletions ods_ci/tests/Resources/Page/OCPDashboard/Pods/Pods.robot
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

104 changes: 94 additions & 10 deletions ...l_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@
Resource ../../../Resources/OCP.resource
Resource ../../../Resources/CLI/ModelServing/modelmesh.resource
Resource ../../../Resources/Common.robot
Resource ../../../Resources/CLI/ModelServing/llm.resource
Suite Setup Triton On Kserve Suite Setup

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character
Suite Teardown Triton On Kserve Suite Teardown
#Suite Teardown Triton On Kserve Suite Teardown
Test Tags Kserve


Expand Down Expand Up @@ -72,6 +73,15 @@
${PYTHON_MODEL_NAME}= python
${EXPECTED_INFERENCE_GRPC_OUTPUT_FILE_PYTHON}= tests/Resources/Files/triton/kserve-triton-python-gRPC-output.json
${INFERENCE_GRPC_INPUT_PYTHON}= tests/Resources/Files/triton/kserve-triton-python-gRPC-input.json
${KSERVE_MODE}= RawDeployment # Serverless
${DALI_MODEL_NAME}= dali
${PROTOCOL}= http
${TEST_NS}= tritonmodel
${DOWNLOAD_IN_PVC}= ${FALSE}
${LLM_RESOURCES_DIRPATH}= tests/Resources/Files/llm
${INFERENCESERVICE_FILEPATH}= ${LLM_RESOURCES_DIRPATH}/serving_runtimes/base/isvc.yaml

Check notice

Code scanning / Robocop

Variable '{{ name }}' is assigned but not used Note test

Variable '${INFERENCESERVICE_FILEPATH}' is assigned but not used
${INFERENCESERVICE_FILEPATH_NEW}= ${LLM_RESOURCES_DIRPATH}/serving_runtimes/isvc
${INFERENCESERVICE_FILLED_FILEPATH}= ${INFERENCESERVICE_FILEPATH_NEW}/isvc_filled.yaml

*** Test Cases ***
Test Onnx Model Rest Inference Via UI (Triton on Kserve) # robocop: off=too-long-test-case
Expand Down Expand Up @@ -203,12 +213,12 @@
... namespace=${PRJ_TITLE} timeout=180s
${EXPECTED_INFERENCE_GRPC_OUTPUT_TENSORFLOW}= Load Json File file_path=${EXPECTED_INFERENCE_GRPC_OUTPUT_FILE_TENSORFLOW}
... as_string=${TRUE}
${EXPECTED_INFERENCE_GRPC_OUTPUT_TENSORFLOW}= Load Json String ${EXPECTED_INFERENCE_GRPC_OUTPUT_TENSORFLOW}

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character
${EXPECTED_INFERENCE_GRPC_OUTPUT_TENSORFLOW}= Evaluate json.dumps(${EXPECTED_INFERENCE_GRPC_OUTPUT_TENSORFLOW})
Log ${EXPECTED_INFERENCE_GRPC_OUTPUT_TENSORFLOW}
Open Model Serving Home Page
${host}= Get Model Route for gRPC Via UI model_name=${TENSORFLOW_MODEL_NAME}
Log ${host}

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character
${token}= Get Access Token Via UI single_model=${TRUE} model_name=${TENSORFLOW_MODEL_NAME} project_name=${PRJ_TITLE}
${inference_output}= Query Model With GRPCURL host=${host} port=443
... endpoint=inference.GRPCInferenceService/ModelInfer
Expand Down Expand Up @@ -407,7 +417,7 @@
... Clean All Models Of Current User
... AND
... Delete Serving Runtime Template From CLI displayed_name=triton-kserve-rest

Test FIL Model Grpc Inference Via UI (Triton on Kserve) # robocop: off=too-long-test-case
[Documentation] Test the deployment of an fil model in Kserve using Triton
[Tags] Sanity RHOAIENG-15823
Expand Down Expand Up @@ -484,32 +494,106 @@
... Delete Serving Runtime Template From CLI displayed_name=triton-kserve-rest



Test Dali Model Rest Inference Via UI (Triton on Kserve) # robocop: off=too-long-test-case

Check warning

Code scanning / Robocop

Invalid number of empty lines between test cases ({{ empty_lines }}/{{ allowed_empty_lines }}) Warning test

Invalid number of empty lines between test cases (3/1)

Check warning

Code scanning / Robocop

Test case '{{ test_name }}' has too many keywords inside ({{ keyword_count }}/{{ max_allowed_count }}) Warning test

Test case 'Test Dali Model Rest Inference Via UI (Triton on Kserve)' has too many keywords inside (11/10)
[Documentation] Test the deployment of an Dali model in Kserve using Triton
[Tags] Tier2 Resources-GPU NVIDIA-GPUs RunThisTest

Setup Test Variables model_name=${DALI_MODEL_NAME} use_pvc=${FALSE} use_gpu=${TRUE}
... kserve_mode=${KSERVE_MODE}
Set Project And Runtime runtime=${ONNX_RUNTIME_NAME} protocol=${PROTOCOL} namespace=${test_namespace}
... download_in_pvc=${DOWNLOAD_IN_PVC} model_name=${DALI_MODEL_NAME}
... storage_size=100Mi memory_request=100Mi
${requests}= Create Dictionary memory=1Gi

Check notice

Code scanning / Robocop

{{ create_keyword }} can be replaced with VAR Note test

Create Dictionary can be replaced with VAR
Compile Inference Service YAML isvc_name=${DALI_MODEL_NAME}
... sa_name=${EMPTY}
... model_storage_uri=${storage_uri}
... model_format=python - 1 serving_runtime=${ONNX_RUNTIME_NAME}
... limits_dict=${limits} requests_dict=${requests} kserve_mode=${KSERVE_MODE}
Deploy Model Via CLI isvc_filepath=${INFERENCESERVICE_FILLED_FILEPATH}
... namespace=${test_namespace}
# File is not needed anymore after applying
Remove File ${INFERENCESERVICE_FILLED_FILEPATH}
Wait For Pods To Be Ready label_selector=serving.kserve.io/inferenceservice=${DALI_MODEL_NAME}
... namespace=${test_namespace}
${pod_name}= Get Pod Name namespace=${test_namespace}
... label_selector=serving.kserve.io/inferenceservice=${DALI_MODEL_NAME}
${service_port}= Extract Service Port service_name=${DALI_MODEL_NAME}-predictor protocol=TCP
... namespace=${test_namespace}
IF "${KSERVE_MODE}"=="RawDeployment"
Start Port-forwarding namespace=${test_namespace} pod_name=${pod_name} local_port=${service_port}
... remote_port=${service_port} process_alias=triton-process
END
Verify Model Inference With Retries model_name=${DALI_MODEL_NAME} inference_input=${INFERENCE_INPUT}
... expected_inference_output=${EXPECTED_INFERENCE_OUTPUT} project_title=${test_namespace}
... deployment_mode=Cli kserve_mode=${KSERVE_MODE} service_port=${service_port}
... end_point=/v2/models/${model_name}/infer retries=10
#[Teardown] Run Keywords

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character
#... Clean Up Test Project test_ns=${test_namespace}

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character
#... isvc_names=${models_names} wait_prj_deletion=${FALSE} kserve_mode=${KSERVE_MODE}

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character
#... AND

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character
#... Run Keyword If "${KSERVE_MODE}"=="RawDeployment" Terminate Process triton-process kill=true

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character


*** Keywords ***
Triton On Kserve Suite Setup
[Documentation] Suite setup steps for testing Triton. It creates some test variables
... and runs RHOSi setup
Set Library Search Order SeleniumLibrary
Skip If Component Is Not Enabled kserve
RHOSi Setup
Launch Dashboard ${TEST_USER.USERNAME} ${TEST_USER.PASSWORD} ${TEST_USER.AUTH_TYPE}
... ${ODH_DASHBOARD_URL} ${BROWSER.NAME} ${BROWSER.OPTIONS}
#Skip If Component Is Not Enabled kserve

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character
#RHOSi Setup

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character
#Launch Dashboard ${TEST_USER.USERNAME} ${TEST_USER.PASSWORD} ${TEST_USER.AUTH_TYPE}

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character
#... ${ODH_DASHBOARD_URL} ${BROWSER.NAME} ${BROWSER.OPTIONS}

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character
Fetch Knative CA Certificate filename=openshift_ca_istio_knative.crt
Clean All Models Of Current User
#Clean All Models Of Current User

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character

Setup Test Variables # robocop: off=too-many-calls-in-keyword
[Documentation] Sets variables for the Suite
[Arguments] ${model_name} ${kserve_mode}=Serverless ${use_pvc}=${FALSE} ${use_gpu}=${FALSE}

Check notice

Code scanning / Robocop

There is too many arguments per continuation line ({{ arguments_count }} / {{ max_arguments_count }}) Note test

There is too many arguments per continuation line (4 / 1)
... ${model_path}=${model_name}
Set Test Variable ${model_name}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
${models_names}= Create List ${model_name}

Check notice

Code scanning / Robocop

{{ create_keyword }} can be replaced with VAR Note test

Create List can be replaced with VAR
Set Test Variable ${models_names}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${model_path}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${test_namespace} ${TEST_NS}-${model_name}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
IF ${use_pvc}
Set Test Variable ${storage_uri} pvc://${model_name}-claim/${model_path}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
ELSE
Set Test Variable ${storage_uri} s3://${S3.BUCKET_3.NAME}/${model_path}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
END
IF ${use_gpu}
${limits}= Create Dictionary nvidia.com/gpu=1

Check notice

Code scanning / Robocop

{{ create_keyword }} can be replaced with VAR Note test

Create Dictionary can be replaced with VAR
Set Test Variable ${limits}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
ELSE
Set Test Variable ${limits} &{EMPTY}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
END
IF "${KSERVE_MODE}" == "RawDeployment"

Check warning

Code scanning / Robocop

Variable '{{ name }}' has inconsistent naming. First used as '{{ first_use }}' Warning test

Variable '${KSERVE_MODE}' has inconsistent naming. First used as '${kserve_mode}'
Set Test Variable ${use_port_forwarding} ${TRUE}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
ELSE
Set Test Variable ${use_port_forwarding} ${FALSE}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
END
Set Log Level NONE
Set Test Variable ${access_key_id} ${S3.AWS_ACCESS_KEY_ID}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${access_key} ${S3.AWS_SECRET_ACCESS_KEY}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${endpoint} ${MODELS_BUCKET.ENDPOINT}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${region} ${MODELS_BUCKET.REGION}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Log Level INFO

Triton On Kserve Suite Teardown
[Documentation] Suite teardown steps after testing DSG. It Deletes
... all the DS projects created by the tests and run RHOSi teardown
# Even if kw fails, deleting the whole project will also delete the model
# Failure will be shown in the logs of the run nonetheless
IF ${MODEL_CREATED}
Clean All Models Of Current User
#Clean All Models Of Current User

Check warning

Code scanning / Robocop

Missing blank space after comment character Warning test

Missing blank space after comment character
Log SKIP
ELSE
Log Model not deployed, skipping deletion step during teardown console=true
END
${projects}= Create List ${PRJ_TITLE}
Delete List Of Projects Via CLI ocp_projects=${projects}
#Delete List Of Projects Via CLI ocp_projects=${projects}
# Will only be present on SM cluster runs, but keyword passes
# if file does not exist
Remove File openshift_ca_istio_knative.crt
SeleniumLibrary.Close All Browsers
RHOSi Teardown
#RHOSi Teardown

Check warning

Code scanning / Robocop

File has too many lines ({{ lines_count }}/{{max_allowed_count }}) Warning test

File has too many lines (599/400)
Loading