Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model Registry e2e UI test + breaking ui changes #1879

Merged
merged 11 commits into from
Oct 2, 2024
Merged

Model Registry e2e UI test + breaking ui changes #1879

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
d4c2069
e2e UI test + breaking ui changes
lugi0 Oct 1, 2024
e6a1013
uncomment code block
lugi0 Oct 1, 2024
e5e534e
fix issues and add checks on model registry
lugi0 Oct 2, 2024
9453eeb
Screenshot correct element when jupyter code fails
lugi0 Oct 2, 2024
25f0640
remove trailing whitespace
lugi0 Oct 2, 2024
39590c0
specify namespace in teardown
lugi0 Oct 2, 2024
72b9b79
Grab screenshot only if failure happened
lugi0 Oct 2, 2024
526640a
Stop sed from creating backups
lugi0 Oct 2, 2024
b27473f
fix screenshot logic
lugi0 Oct 2, 2024
fbf8232
move fail statement
lugi0 Oct 2, 2024
669bb90
Merge branch 'master' into feature/model-registry-UI
lugi0 Oct 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDataScienceProject/DataConnections.resource
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
${S3_BUCKET_DC_INPUT_XP}= xpath=//*[@data-testid="field AWS_S3_BUCKET"]
${S3_DC_ADD_BTN}= xpath=//*[@data-testid="data-connection-submit-button"]
${S3_DC_EDIT_BTN}= xpath=//footer/button[.="Update data connection"]
${DC_WORKBENCH_SELECTOR_XP}= xpath=//ul[@aria-label="Notebook select"]/li
${DC_WORKBENCH_SELECTOR_XP}= xpath=//div[@data-testid="connect-existing-workbench-group"]//ul/li


*** Keywords ***
Expand Down Expand Up @@ -47,33 +47,33 @@
Set Connection Between Data Connection And Workbench
[Documentation] Connects a DataConnection to an existent workbench
[Arguments] ${workbench_title}
Wait Until Element Is Enabled xpath=//button[@aria-label="Options menu"]
Click Element xpath=//button[@aria-label="Options menu"]
Wait Until Page Contains Element ${DC_WORKBENCH_SELECTOR_XP}/button[text()="${workbench_title}"]
Click Element ${DC_WORKBENCH_SELECTOR_XP}/button[text()="${workbench_title}"]
Wait Until Element Is Enabled xpath=//button[@aria-label="Notebook select"]
Click Element xpath=//button[@aria-label="Notebook select"]
Wait Until Page Contains Element ${DC_WORKBENCH_SELECTOR_XP}/button[.="${workbench_title}"]
Click Element ${DC_WORKBENCH_SELECTOR_XP}/button[.="${workbench_title}"]

Data Connection Should Be Listed
[Documentation] Checks a Data Connection is listed in DS Project details page
[Arguments] ${name} ${type} ${connected_workbench}
${connection_xp}= Set Variable ${DC_SECTION_XP}//tr[td[@data-label="Name"]//div[text()="${name}"]]
${connection_xp}= Set Variable ${DC_SECTION_XP}//tr[td[@data-label="Name"]//div[.="${name}"]]
Dismissed Show dismissed Hide dismissed
Run Keyword And Continue On Failure Page Should Contain Element ${connection_xp}
Run Keyword And Continue On Failure Page Should Contain Element ${connection_xp}/td[text()=" ${type}"]
Run Keyword And Continue On Failure Page Should Contain Element ${connection_xp}/td[.=" ${type}"]
IF "${connected_workbench}" == "${NONE}"
Run Keyword And Continue On Failure Page Should Contain Element
... ${connection_xp}/td[text()="No connections"]
... ${connection_xp}/td[.="No connections"]
ELSE
FOR ${index} ${workbench_title} IN ENUMERATE @{connected_workbench}
Log ${index}: ${workbench_title}
Run Keyword And Continue On Failure Page Should Contain Element
... ${connection_xp}/td[@data-label="Connected workbenches"]//*[text()="${workbench_title}"]
... ${connection_xp}/td[@data-label="Connected workbenches"]//*[.="${workbench_title}"]
END
END

Data Connection Should Not Be Listed
[Documentation] Checks a Data Connection is not listed in DS Project details page
[Arguments] ${name}
Run Keyword And Continue On Failure Wait Until Page Does Not Contain Element
... ${DC_SECTION_XP}//tr[td[@data-label="Name"]/*/div[text()="${name}"]]
... ${DC_SECTION_XP}//tr[td[@data-label="Name"]/*/div[.="${name}"]]

Get Openshift Secret From Data Connection
[Documentation] Retrieves name of Openshift secret corresponding to a given S3 Data Connection based on Dispayed name in DS Project details page
Expand Down Expand Up @@ -103,7 +103,7 @@
Log msg=you are not connecting any workbenchs to ${dc_name} DataConnection
ELSE
Run Keyword And Continue On Failure
... Element Should Be Enabled xpath=//div[contains(@class,"modal")]//button[@aria-label="Options menu"]
... Element Should Be Enabled xpath=//div[contains(@class,"modal")]//button[@aria-label="Notebook select"]
FOR ${workbench_title} IN @{connected_workbench}
Set Connection Between Data Connection And Workbench ${workbench_title}
Run Keyword And Continue On Failure Element Should Be Enabled ${S3_DC_ADD_BTN}
Expand Down Expand Up @@ -137,7 +137,7 @@
... ${connected_workbench}=${NONE} ${press_cancel}=${FALSE} ${aws_bucket_name}=${NONE}
Open Data Science Project Details Page project_title=${project_title} tab_id=data-connections
${is_exist}= Run Keyword And Return Status
... Wait Until Page Contains Element ${DC_SECTION_XP}//tr[td[@data-label="Name"]/*/div[text()="${dc_name}"]]
... Wait Until Page Contains Element ${DC_SECTION_XP}//tr[td[@data-label="Name"]/*/div[.="${dc_name}"]]
IF ${is_exist}
Log Data Connection '${dc_name}' already exists, updating connection console=True
Edit S3 Data Connection project_title=${project_title} dc_name=${dc_name} new_dc_name=${dc_name}${SPACE}
Expand Down
12 changes: 6 additions & 6 deletions ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDataScienceProject/Projects.resource
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,12 +143,12 @@
Run Keyword And Warn On Failure Element Should Be Disabled ${GENERIC_CREATE_BTN_XP}
Input Text ${TITLE_INPUT_XP} ${title}
Input Text ${DESCR_INPUT_XP} ${description}
Run Keyword And Continue On Failure Validate Generated Resource Name project_title=${title}
IF "${resource_name}" == "${NONE}"
${resource_name}= Get Value ${RESOURCE_INPUT_XP}
${resource_name}= Generate Unique Namespace ${resource_name}
END
Input Text ${RESOURCE_INPUT_XP} ${resource_name}
#Run Keyword And Continue On Failure Validate Generated Resource Name project_title=${title}
Fixed Show fixed Hide fixed
#IF "${resource_name}" == "${NONE}"
Fixed Show fixed Hide fixed
# ${resource_name}= Get Value ${RESOURCE_INPUT_XP}
# ${resource_name}= Generate Unique Namespace ${resource_name}
#END
Fixed Show fixed Hide fixed
#Input Text ${RESOURCE_INPUT_XP} ${resource_name}
Fixed Show fixed Hide fixed
Wait Until Element Is Enabled ${GENERIC_CREATE_BTN_XP}
Click Button ${GENERIC_CREATE_BTN_XP}
Wait Until Generic Modal Disappears
Expand Down
37 changes: 26 additions & 11 deletions ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDataScienceProject/Workbenches.resource
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,14 +264,14 @@ Workbench Should Be Listed
[Arguments] ${workbench_title} ${timeout}=10s
Run keyword And Continue On Failure
... Wait Until Page Contains Element
... ${WORKBENCH_SECTION_XP}//td[@data-label="Name"]/*[div[text()="${workbench_title}"]] timeout=${timeout}
... ${WORKBENCH_SECTION_XP}//td[@data-label="Name"]//*[.="${workbench_title}"] timeout=${timeout}

Workbench With Description Should Be Listed
[Documentation] Checks a workbench with particular name and description is listed in the DS Project details page
[Arguments] ${workbench_title} ${workbench_description} ${timeout}=10s
Run keyword And Continue On Failure
... Wait Until Page Contains Element
... ${WORKBENCH_SECTION_XP}//td[@data-label="Name"][//div[text()="${workbench_title}"] and //p[text()="${workbench_description}"]]
... ${WORKBENCH_SECTION_XP}//td[@data-label="Name"][//*[text()="${workbench_title}"] and //p[text()="${workbench_description}"]]
... timeout=${timeout}

Workbench Should Not Be Listed
Expand All @@ -286,7 +286,7 @@ Workbench Status Should Be
[Arguments] ${workbench_title} ${status} ${timeout}=10
Run keyword And Continue On Failure
... Wait Until Page Contains Element
... ${WORKBENCH_SECTION_XP}//tr[td[@data-label="Name"]/*[div[text()="${workbench_title}"]]]/td[@data-label="Status"]//p[text()="${status}"]
... ${WORKBENCH_SECTION_XP}//td[@data-label="Name"]//*[.="${workbench_title}"]/../../../td[@data-label="Status"]//span[text()="${status}"]
... timeout=${timeout}

Status Error Icon Should Appear
Expand All @@ -301,7 +301,7 @@ Wait Until Workbench Is Started
[Documentation] Waits until workbench status is "RUNNING" in the DS Project details page
[Arguments] ${workbench_title} ${timeout}=60s ${status}=${WORKBENCH_STATUS_RUNNING}
Wait Until Page Contains Element
... ${WORKBENCH_SECTION_XP}//tr[td[@data-label="Name"]/*[div[text()="${workbench_title}"]]]/td[@data-label="Status"]//p[text()="${status}"]
... ${WORKBENCH_SECTION_XP}//td[@data-label="Name"]//*[.="${workbench_title}"]/../../../td[@data-label="Status"]//span[text()="${status}"]
... timeout=${timeout}

Wait Until Workbench Is Stopped
Expand Down Expand Up @@ -350,27 +350,42 @@ Access To Workbench
END

Launch And Access Workbench
[Documentation] Launches a workbench from DS Project details page
[Documentation] Launches a workbench from DS Project details page and returns the handle of the current tab
[Arguments] ${workbench_title} ${username}=${TEST_USER_3.USERNAME}
... ${password}=${TEST_USER_3.PASSWORD} ${auth_type}=${TEST_USER_3.AUTH_TYPE}
${is_started}= Run Keyword And Return Status Workbench Status Should Be
... workbench_title=${workbench_title} status=${WORKBENCH_STATUS_RUNNING}
IF ${is_started} == ${TRUE}
Open Workbench workbench_title=${workbench_title}
${previous_handle}= Open Workbench workbench_title=${workbench_title}
Access To Workbench username=${username} password=${password}
... auth_type=${auth_type}
ELSE
Fail msg=Cannot Launch And Access Workbench ${workbench_title} because it is not running...
END
RETURN ${previous_handle}

Open Workbench
[Documentation] Clicks on "open" link for a given workbench
[Documentation] Clicks on "open" link for a given workbench and returns the handle of the current tab
[Arguments] ${workbench_title}
${desc}= Does Workbench Have A Description ${workbench_title}
${open_workbench_link}= Set Variable
... ${WORKBENCH_SECTION_XP}//tr[td[@data-label="Name"]/*[div[text()="${workbench_title}"]]]/td//a[text()="Open"]
Wait Until Page Contains Element ${open_workbench_link} timeout=30s
Click Link ${open_workbench_link}
Switch Window NEW
... ${WORKBENCH_SECTION_XP}//td[@data-label="Name"]//*[.="${workbench_title}"]
IF ${desc}
${open_workbench_link}= Catenate ${open_workbench_link} /../../../td//a[.="Open"]
ELSE
${open_workbench_link}= Catenate ${open_workbench_link} /../../td//a[.="Open"]
END
SeleniumLibrary.Wait Until Page Contains Element ${open_workbench_link} timeout=30s
SeleniumLibrary.Click Link ${open_workbench_link}
${previous_handle}= SeleniumLibrary.Switch Window NEW
RETURN ${previous_handle}

Does Workbench Have A Description
[Documentation] Checks if a Workbench has a description in the list view of the project
[Arguments] ${workbench_title}
${has_description}= Run Keyword And Return Status SeleniumLibrary.Page Should Contain Element
... ${WORKBENCH_SECTION_XP}//td[@data-label="Name"]//*[.="${workbench_title}"]/../span[@data-testid="table-row-title-description"] # robocop: disable
RETURN ${has_description}

Stop Workbench
[Documentation] Stops a workbench from DS Project details page
Expand Down
45 changes: 39 additions & 6 deletions ods_ci/tests/Tests/1300__model_registry/1302_model_registry_model_serving.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@
${SECRET_PART_NAME_2}= modelregistry-sample-grpc
${SECRET_PART_NAME_3}= model-registry-db
${BROWSER.NAME}= chrome
${MR_REGISTERED_MODEL_NAME}= test minst
${MR_REGISTERED_MODEL_VERSION}= 2.0.0
${MR_REGISTERED_MODEL_AUTHOR}= Tony
${MR_TABLE_XPATH}= //table[@data-testid="registered-model-table"]
${MR_VERSION_TABLE_XPATH}= //table[@data-testid="model-versions-table"]


*** Test Cases ***
Expand All @@ -52,24 +57,33 @@
Workbench Should Be Listed workbench_title=${WORKBENCH_TITLE}
Open Data Science Project Details Page project_title=${PRJ_TITLE}
${workbenches}= Create List ${WORKBENCH_TITLE}
# Create S3 Data Connection project_title=${PRJ_TITLE} dc_name=${DC_S3_NAME}
# ... aws_access_key=${S3.AWS_ACCESS_KEY_ID} aws_secret_access=${S3.AWS_SECRET_ACCESS_KEY}
# ... aws_bucket_name=${AWS_BUCKET} connected_workbench=${workbenches}
# Data Connection Should Be Listed name=${DC_S3_NAME} type=${DC_S3_TYPE} connected_workbench=${workbenches}
# Open Data Science Project Details Page project_title=${prj_title} tab_id=workbenches
Create S3 Data Connection project_title=${PRJ_TITLE} dc_name=${DC_S3_NAME}
... aws_access_key=${S3.AWS_ACCESS_KEY_ID} aws_secret_access=${S3.AWS_SECRET_ACCESS_KEY}
... aws_bucket_name=${AWS_BUCKET} connected_workbench=${workbenches}
Data Connection Should Be Listed name=${DC_S3_NAME} type=${DC_S3_TYPE} connected_workbench=${workbenches}
Open Data Science Project Details Page project_title=${prj_title} tab_id=workbenches
Wait Until Workbench Is Started workbench_title=${WORKBENCH_TITLE}
Upload File In The Workbench filepath=${SAMPLE_ONNX_MODEL} workbench_title=${WORKBENCH_TITLE}
... workbench_namespace=${PRJ_TITLE}
Upload File In The Workbench filepath=${JUPYTER_NOTEBOOK_FILEPATH} workbench_title=${WORKBENCH_TITLE}
... workbench_namespace=${PRJ_TITLE}
Download Python Client Dependencies ${MR_PYTHON_CLIENT_FILES} ${MR_PYTHON_CLIENT_WHL_VERSION}
Upload Python Client Files In The Workbench ${MR_PYTHON_CLIENT_FILES}
Launch And Access Workbench workbench_title=${WORKBENCH_TITLE}
${handle}= Launch And Access Workbench workbench_title=${WORKBENCH_TITLE}
... username=${TEST_USER.USERNAME} password=${TEST_USER.PASSWORD}
... auth_type=${TEST_USER.AUTH_TYPE}
Upload Certificate To Jupyter Notebook ${CERTS_DIRECTORY}/domain.crt
Upload Certificate To Jupyter Notebook openshift_ca.crt
Jupyter Notebook Can Query Model Registry ${JUPYTER_NOTEBOOK}
SeleniumLibrary.Switch Window ${handle}
Open Model Registry Dashboard Page
SeleniumLibrary.Page Should Contain Element xpath:${MR_TABLE_XPATH}/tbody/tr/td[@data-label="Model name"]//a[.="${MR_REGISTERED_MODEL_NAME}"]
SeleniumLibrary.Page Should Contain Element xpath:${MR_TABLE_XPATH}/tbody/tr/td[@data-label="Owner"]//p[.="${MR_REGISTERED_MODEL_AUTHOR}"]
SeleniumLibrary.Page Should Contain Element xpath:${MR_TABLE_XPATH}/tbody/tr/td[@data-label="Labels" and .="-"]
SeleniumLibrary.Click Element xpath:${MR_TABLE_XPATH}/tbody/tr/td[@data-label="Model name"]//a[.="${MR_REGISTERED_MODEL_NAME}"]
Maybe Wait For Dashboard Loading Spinner Page
SeleniumLibrary.Page Should Contain Element xpath:${MR_VERSION_TABLE_XPATH}/tbody/tr/td[@data-label="Version name"]//a[.="${MR_REGISTERED_MODEL_VERSION}"]
SeleniumLibrary.Page Should Contain Element xpath:${MR_VERSION_TABLE_XPATH}/tbody/tr/td[@data-label="Author" and .="${MR_REGISTERED_MODEL_AUTHOR}"]


*** Keywords ***
Expand Down Expand Up @@ -187,6 +201,7 @@
[Arguments] ${filepath}
Open Notebook File In JupyterLab ${filepath}
Open With JupyterLab Menu Run Restart Kernel and Run All Cells…
Wait Until Page Contains Element xpath=//div[contains(text(),"Restart") and @class="jp-Dialog-buttonLabel"]
Click Element xpath=//div[contains(text(),"Restart") and @class="jp-Dialog-buttonLabel"]
Wait Until JupyterLab Code Cell Is Not Active timeout=120s
Sleep 2m msg=Waits until the jupyter notebook has completed execution of all cells
Expand Down Expand Up @@ -287,3 +302,21 @@
... filepath=${file_location}/${file} workbench_title=${WORKBENCH_TITLE}
... workbench_namespace=${PRJ_TITLE}
END

Open Model Registry Dashboard Page
Fixed Show fixed Hide fixed
[Documentation] Opens the Model Registry page from the dashboard nav bar
SeleniumLibrary.Wait Until Page Contains Model Registry
SeleniumLibrary.Click Link Model Registry
Wait For RHODS Dashboard To Load wait_for_cards=${FALSE} expected_page=Model Registry
SeleniumLibrary.Wait Until Page Contains Select a model registry to view and manage your registered models.
Maybe Wait For Dashboard Loading Spinner Page
${loaded}= Run Keyword And Return Status
... SeleniumLibrary.Page Should Not Contain Request access to model registries
WHILE ${loaded}!=${TRUE} limit=5
Dismissed Show dismissed Hide dismissed
SeleniumLibrary.Reload Page
SeleniumLibrary.Wait Until Page Contains Model Registry
SeleniumLibrary.Wait Until Page Contains Select a model registry to view and manage your registered models.
${loaded}= Run Keyword And Return Status
... SeleniumLibrary.Page Should Not Contain Request access to model registries
END
Maybe Wait For Dashboard Loading Spinner Page
Loading