Task/APPENG-2738: NIM Cypress E2E tests (#3298)

* test: added mocks and interceptors for nim cypress tests Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * Merging the base test cases of Daniel and Lokesh's code to the Tomer's standalone cypress test suite. * Reverting the mock changes which are not needed. * nim-specific classes and functions, partial rework of deployment test (#2) Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * Checking in the cypress test covers to validate all the scenarios to render nim and all other cards on overview and model tab when there are no ServingRuntimes configured. * Added mockNimServingRuntimeTemplate mock function Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * mocking deploy respurces (WIP) Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * Completed deployment tests Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * Added new test cases to validate if the nim is not enabled * when combination of configurations to enable nim models are not configured properly. * test: added test cases for deleting model from the project models tab Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * * Added new test cases to validate the model serving menu item when there are deployments and no deployments. (#9) * Moved the utility method to nimUtils.ts * test: cleanups Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * adding const for modal dialog title Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * test cases for list of models in different pages/tabs Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * linting fixes Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * removed experimentalStudio option Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * test: split nim tests per pages (#13) Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * testing more buttons in modal dialog (#14) Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * Added a test case to address there is a failure in loading Nvidia Nim model images then error message should be displayed. Added experimentalStudio flag to cypress.config.ts but disabled it by default. * Added a test case to address there is a failure in loading Nvidia Nim model images then error message should be displayed. Added experimentalStudio flag to cypress.config.ts but disabled it by default. * test: fix linting errors Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * test: fix review change requests 1 - see body - Use JSON object instead of string in mocking of NIM images configmap - Remove experiementalStudio cypres config param - Add mock for NIM project Co-authored-by: Daniele Martinoli <dmartino@redhat.com> Co-authored-by: lokeshrangineni <lrangine@redhat.com> Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * test: fix review change requests 2 - see body - Add nim-serving interception to odh intercept type safety mechanism - Add accelerators interception to odh intercept type safety mechanism Co-authored-by: Daniele Martinoli <dmartino@redhat.com> Co-authored-by: lokeshrangineni <lrangine@redhat.com> Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * test: create page objects for verifying nim model tabels Co-authored-by: Daniele Martinoli <dmartino@redhat.com> Co-authored-by: lokeshrangineni <lrangine@redhat.com> Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * Update frontend/src/__tests__/cypress/cypress/tests/mocked/projects/modelServingNim.cy.ts Co-authored-by: Andrew Ballantyne <8126518+andrewballantyne@users.noreply.github.com> * test: fix linting errors Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * test: rewrite test case for global model list nim Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * test: fix linting errors Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * test: rewrite test case for project overview list nim Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * test: rewrite test case for project models tab list nim Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * test: cleanups Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * test: rewrite test cases nim enablement Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * test: final touchups Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * test: fixing 1 of 2 failing tests after rebase Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * test: fix mockInfereceService should take labels co-authored-by: Andrew Ballantyne <aballant@redhat.com> Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> * test: fix review suggestion for global model edit button co-authored-by: Andrew Ballantyne <aballant@redhat.com> Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> --------- Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com> Signed-off-by: Daniele Martinoli <dmartino@redhat.com> Co-authored-by: Tomer Figenblat <tfigenbl@redhat.com> Co-authored-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com> Co-authored-by: lokeshrangineni <lrangine@redhat.com> Co-authored-by: Andrew Ballantyne <8126518+andrewballantyne@users.noreply.github.com> Co-authored-by: Andrew Ballantyne <aballant@redhat.com>
opendatahub-io · Oct 30, 2024 · 23c82ac · 23c82ac
1 parent 1636d39
commit 23c82ac
Show file tree

Hide file tree

Showing 12 changed files with 841 additions and 6 deletions.
diff --git a/frontend/src/__mocks__/mockDashboardConfig.ts b/frontend/src/__mocks__/mockDashboardConfig.ts
@@ -1,7 +1,7 @@
 import { DashboardConfigKind, KnownLabels } from '~/k8sTypes';
 import { NotebookSize } from '~/types';
 
-type MockDashboardConfigType = {
+export type MockDashboardConfigType = {
   disableInfo?: boolean;
   disableSupport?: boolean;
   disableClusterManager?: boolean;

diff --git a/frontend/src/__mocks__/mockInferenceServiceK8sResource.ts b/frontend/src/__mocks__/mockInferenceServiceK8sResource.ts
@@ -22,6 +22,7 @@ type MockResourceConfigType = {
   kserveInternalUrl?: string;
   statusPredictor?: Record<string, string>;
   kserveInternalLabel?: boolean;
+  additionalLabels?: Record<string, string>;
 };
 
 type InferenceServicek8sError = K8sStatus & {
@@ -80,6 +81,7 @@ export const mockInferenceServiceK8sResource = ({
   statusPredictor = undefined,
   kserveInternalUrl = '',
   kserveInternalLabel = false,
+  additionalLabels = {},
 }: MockResourceConfigType): InferenceServiceKind => ({
   apiVersion: 'serving.kserve.io/v1beta1',
   kind: 'InferenceService',
@@ -99,8 +101,7 @@ export const mockInferenceServiceK8sResource = ({
     generation: 1,
     labels: {
       name,
-      [KnownLabels.REGISTERED_MODEL_ID]: '1',
-      [KnownLabels.MODEL_VERSION_ID]: '3',
+      ...additionalLabels,
       [KnownLabels.DASHBOARD_RESOURCE]: 'true',
       ...(kserveInternalLabel && { 'networking.knative.dev/visibility': 'cluster-local' }),
     },

diff --git a/frontend/src/__mocks__/mockNimResource.ts b/frontend/src/__mocks__/mockNimResource.ts
@@ -0,0 +1,145 @@
+import {
+  ConfigMapKind,
+  InferenceServiceKind,
+  PersistentVolumeClaimKind,
+  ProjectKind,
+  SecretKind,
+  ServingRuntimeKind,
+  TemplateKind,
+} from '~/k8sTypes';
+import { ServingRuntimeAPIProtocol, ServingRuntimePlatform } from '~/types';
+import { mockProjectK8sResource } from '~/__mocks__/mockProjectK8sResource';
+import { NimServingResponse } from '~/__tests__/cypress/cypress/types';
+import { mockConfigMap } from './mockConfigMap';
+import { mockServingRuntimeK8sResource } from './mockServingRuntimeK8sResource';
+import { mockInferenceServiceK8sResource } from './mockInferenceServiceK8sResource';
+import { mockServingRuntimeTemplateK8sResource } from './mockServingRuntimeTemplateK8sResource';
+import { mockSecretK8sResource } from './mockSecretK8sResource';
+import { mockPVCK8sResource } from './mockPVCK8sResource';
+
+export const mockNimImages = (): ConfigMapKind =>
+  mockConfigMap({
+    name: 'nvidia-nim-images-data',
+    namespace: 'opendatahub',
+    data: {
+      alphafold2: JSON.stringify({
+        name: 'alphafold2',
+        displayName: 'AlphaFold2',
+        shortDescription:
+          'A widely used model for predicting the 3D structures of proteins from their amino acid sequences.',
+        namespace: 'nim/deepmind',
+        tags: ['1.0.0'],
+        latestTag: '1.0.0',
+        updatedDate: '2024-08-27T01:51:55.642Z',
+      }),
+      'arctic-embed-l': JSON.stringify({
+        name: 'arctic-embed-l',
+        displayName: 'Snowflake Arctic Embed Large Embedding',
+        shortDescription:
+          'NVIDIA NIM for GPU accelerated Snowflake Arctic Embed Large Embedding inference',
+        namespace: 'nim/snowflake',
+        tags: ['1.0.1', '1.0.0'],
+        latestTag: '1.0.1',
+        updatedDate: '2024-07-27T00:38:40.927Z',
+      }),
+    },
+  });
+
+export const mockNimInferenceService = (): InferenceServiceKind => {
+  const inferenceService = mockInferenceServiceK8sResource({
+    name: 'test-name',
+    modelName: 'test-name',
+    displayName: 'Test Name',
+    kserveInternalLabel: true,
+    resources: {
+      limits: { cpu: '2', memory: '8Gi' },
+      requests: { cpu: '1', memory: '4Gi' },
+    },
+  });
+  delete inferenceService.metadata.labels?.name;
+  delete inferenceService.metadata.creationTimestamp;
+  delete inferenceService.metadata.generation;
+  delete inferenceService.metadata.resourceVersion;
+  delete inferenceService.metadata.uid;
+  if (inferenceService.spec.predictor.model?.modelFormat) {
+    inferenceService.spec.predictor.model.modelFormat.name = 'arctic-embed-l';
+  }
+  delete inferenceService.spec.predictor.model?.modelFormat?.version;
+  delete inferenceService.spec.predictor.model?.storage;
+  delete inferenceService.status;
+
+  return inferenceService;
+};
+
+export const mockNimServingRuntime = (): ServingRuntimeKind => {
+  const servingRuntime = mockServingRuntimeK8sResource({
+    name: 'test-name',
+    displayName: 'Test Name',
+  });
+  if (servingRuntime.metadata.annotations) {
+    servingRuntime.metadata.annotations['opendatahub.io/template-display-name'] = 'NVIDIA NIM';
+    servingRuntime.metadata.annotations['opendatahub.io/template-name'] = 'nvidia-nim-runtime';
+  }
+
+  return servingRuntime;
+};
+
+export const mockNimServingRuntimeTemplate = (): TemplateKind => {
+  const templateMock = mockServingRuntimeTemplateK8sResource({
+    name: 'nvidia-nim-serving-template',
+    displayName: 'NVIDIA NIM',
+    platforms: [ServingRuntimePlatform.SINGLE],
+    apiProtocol: ServingRuntimeAPIProtocol.REST,
+    namespace: 'opendatahub',
+  });
+  if (templateMock.metadata.annotations != null) {
+    templateMock.metadata.annotations['opendatahub.io/dashboard'] = 'true';
+  }
+
+  return templateMock;
+};
+
+export const mockNvidiaNimAccessSecret = (): SecretKind => {
+  const secret = mockSecretK8sResource({
+    name: 'nvidia-nim-access',
+  });
+  delete secret.data;
+  secret.data = {};
+  secret.data.api_key = 'api-key'; // eslint-disable-line camelcase
+  secret.data.configMapName = 'bnZpZGlhLW5pbS12YWxpZGF0aW9uLXJlc3VsdA==';
+
+  return secret;
+};
+
+export const mockNvidiaNimImagePullSecret = (): SecretKind => {
+  const secret = mockSecretK8sResource({
+    name: 'nvidia-nim-image-pull',
+  });
+  delete secret.data;
+  secret.data = {};
+  secret.data['.dockerconfigjson'] = 'ZG9ja2VyY29uZmlnCg==';
+
+  return secret;
+};
+
+export const mockNimProject = (hasAllModels: boolean): ProjectKind => {
+  const project = mockProjectK8sResource({
+    hasAnnotations: true,
+    enableModelMesh: hasAllModels ? undefined : false,
+  });
+  if (project.metadata.annotations != null) {
+    project.metadata.annotations['opendatahub.io/nim-support'] = 'true';
+  }
+  return project;
+};
+
+export const mockNimModelPVC = (): PersistentVolumeClaimKind => {
+  const pvc = mockPVCK8sResource({
+    name: 'nim-pvc',
+  });
+  return pvc;
+};
+
+export const mockNimServingResource = (
+  resource: ConfigMapKind | SecretKind,
+): NimServingResponse => ({ body: { body: resource } });
diff --git a/frontend/src/__tests__/cypress/cypress/pages/components/NIMDeployModal.ts b/frontend/src/__tests__/cypress/cypress/pages/components/NIMDeployModal.ts
@@ -0,0 +1,49 @@
+import { Modal } from '~/__tests__/cypress/cypress/pages/components/Modal';
+
+class NIMDeployModal extends Modal {
+  constructor(private edit = false) {
+    super(`${edit ? 'Edit' : 'Deploy'} model with NVIDIA NIM`);
+  }
+
+  findSubmitButton() {
+    return this.findFooter().findByTestId('modal-submit-button');
+  }
+
+  findModelNameInput() {
+    return this.find().findByTestId('model-deployment-name-section');
+  }
+
+  findNIMToDeploy() {
+    return this.find().findByTestId('nim-model-list-selection');
+  }
+
+  findNimStorageSizeInput() {
+    return cy.get('[data-testid="pvc-size"] input');
+  }
+
+  findStorageSizeMinusButton() {
+    return this.find().findByTestId('pvc-size').findByRole('button', { name: 'Minus' });
+  }
+
+  findStorageSizePlusButton() {
+    return this.find().findByTestId('pvc-size').findByRole('button', { name: 'Plus' });
+  }
+
+  findNimModelReplicas() {
+    return cy.get('[id="model-server-replicas"]');
+  }
+
+  findNimModelReplicasMinusButton() {
+    return this.find().find('button[aria-label="Minus"]').eq(1);
+  }
+
+  findNimModelReplicasPlusButton() {
+    return this.find().find('button[aria-label="Plus"]').eq(1);
+  }
+
+  shouldDisplayError(msg: string): void {
+    this.find().should('contain.text', msg);
+  }
+}
+
+export const nimDeployModal = new NIMDeployModal();
diff --git a/frontend/src/__tests__/cypress/cypress/pages/modelServing.ts b/frontend/src/__tests__/cypress/cypress/pages/modelServing.ts
@@ -319,6 +319,14 @@ class InferenceServiceRow extends TableRow {
   findExternalServicePopover() {
     return cy.findByTestId('external-service-popover');
   }
+
+  findServingRuntime() {
+    return this.find().find(`[data-label="Serving Runtime"]`);
+  }
+
+  findProject() {
+    return this.find().find(`[data-label=Project]`);
+  }
 }
 class ServingPlatformCard extends Contextual<HTMLElement> {
   findDeployModelButton() {

diff --git a/frontend/src/__tests__/cypress/cypress/pages/projects.ts b/frontend/src/__tests__/cypress/cypress/pages/projects.ts
@@ -202,7 +202,7 @@ class ProjectDetails {
     return this.findDataConnectionTable().find('thead').findByRole('button', { name });
   }
 
-  private findModelServingPlatform(name: string) {
+  findModelServingPlatform(name: string) {
     return this.findComponent('model-server').findByTestId(`${name}-serving-platform-card`);
   }
 
@@ -307,13 +307,23 @@ class ProjectDetails {
     return cy.findByTestId('unsupported-pipeline-version-alert');
   }
 
-  private findKserveModelsTable() {
+  findKserveModelsTable() {
     return cy.findByTestId('kserve-inference-service-table');
   }
 
   getKserveModelMetricLink(name: string) {
     return this.findKserveModelsTable().findByTestId(`metrics-link-${name}`);
   }
+
+  getKserveTableRow(name: string) {
+    return new KserveTableRow(() =>
+      this.findKserveModelsTable()
+        .find('tbody')
+        .find('[data-label="Name"]')
+        .contains(name)
+        .closest('tr'),
+    );
+  }
 }
 
 class ProjectDetailsSettingsTab extends ProjectDetails {
@@ -326,9 +336,51 @@ class ProjectDetailsSettingsTab extends ProjectDetails {
   }
 }
 
+class ProjectDetailsOverviewTab extends ProjectDetails {
+  visit(project: string) {
+    super.visitSection(project, 'overview');
+  }
+
+  findDeployedModelServingRuntime(name: string) {
+    return cy
+      .findByTestId('section-overview')
+      .get('div')
+      .contains(name)
+      .parents('.odh-type-bordered-card .model-server')
+      .get('dd');
+  }
+
+  findModelServingPlatform(name: string) {
+    return cy.findByTestId(`${name}-platform-card`);
+  }
+}
+
+class KserveTableRow extends TableRow {
+  findAPIProtocol() {
+    return this.find().find(`[data-label="API protocol"]`);
+  }
+
+  findServiceRuntime() {
+    return this.find().find(`[data-label="Serving Runtime"]`);
+  }
+
+  findDetailsTriggerButton() {
+    return this.find().findByTestId('kserve-model-row-item').find('button');
+  }
+
+  private findDetailsCell() {
+    return this.find().next('tr').find('td').eq(1);
+  }
+
+  findInfoValueFor(label: string) {
+    return this.findDetailsCell().find('dt').contains(label).closest('div').find('dd');
+  }
+}
+
 export const projectListPage = new ProjectListPage();
 export const createProjectModal = new CreateEditProjectModal();
 export const editProjectModal = new CreateEditProjectModal(true);
 export const deleteProjectModal = new DeleteModal();
 export const projectDetails = new ProjectDetails();
 export const projectDetailsSettingsTab = new ProjectDetailsSettingsTab();
+export const projectDetailsOverviewTab = new ProjectDetailsOverviewTab();
diff --git a/frontend/src/__tests__/cypress/cypress/support/commands/odh.ts b/frontend/src/__tests__/cypress/cypress/support/commands/odh.ts
@@ -29,6 +29,7 @@ import type { StatusResponse } from '~/redux/types';
 import type {
   BYONImage,
   ClusterSettingsType,
+  DetectedAccelerators,
   ImageInfo,
   OdhApplication,
   OdhDocument,
@@ -54,6 +55,7 @@ import type { GrpcResponse } from '~/__mocks__/mlmd/utils';
 import type { BuildMockPipelinveVersionsType } from '~/__mocks__';
 import type { ArtifactStorage } from '~/concepts/pipelines/types';
 import type { ConnectionTypeConfigMap } from '~/concepts/connectionTypes/types';
+import type { NimServingResponse } from '~/__tests__/cypress/cypress/types';
 
 type SuccessErrorResponse = {
   success: boolean;
@@ -649,6 +651,19 @@ declare global {
             path: { serviceName: string; apiVersion: string; artifactId: string };
           },
           response: OdhResponse<ModelArtifact>,
+        ) => Cypress.Chainable<null>) &
+        ((
+          type: 'GET /api/accelerators',
+          response: OdhResponse<DetectedAccelerators>,
+        ) => Cypress.Chainable<null>) &
+        ((
+          type: 'GET /api/nim-serving/:resource',
+          options: {
+            path: {
+              resource: 'nvidia-nim-images-data' | 'nvidia-nim-access' | 'nvidia-nim-image-pull';
+            };
+          },
+          response: OdhResponse<NimServingResponse>,
         ) => Cypress.Chainable<null>);
     }
   }

diff --git a/frontend/src/__tests__/cypress/cypress/tests/mocked/modelRegistry/modelVersionArchive.cy.ts b/frontend/src/__tests__/cypress/cypress/tests/mocked/modelRegistry/modelVersionArchive.cy.ts
@@ -25,6 +25,7 @@ import {
 } from '~/__tests__/cypress/cypress/pages/modelRegistry/modelVersionArchive';
 import { labelModal, modelRegistry } from '~/__tests__/cypress/cypress/pages/modelRegistry';
 import { mockModelRegistryService } from '~/__mocks__/mockModelRegistryService';
+import { KnownLabels } from '~/k8sTypes';
 
 const MODEL_REGISTRY_API_VERSION = 'v1alpha3';
 
@@ -323,7 +324,14 @@ describe('Archiving version', () => {
     cy.interceptK8sList(ProjectModel, mockK8sResourceList([mockProjectK8sResource({})]));
     cy.interceptK8sList(
       InferenceServiceModel,
-      mockK8sResourceList([mockInferenceServiceK8sResource({})]),
+      mockK8sResourceList([
+        mockInferenceServiceK8sResource({
+          additionalLabels: {
+            [KnownLabels.REGISTERED_MODEL_ID]: '1',
+            [KnownLabels.MODEL_VERSION_ID]: '3',
+          },
+        }),
+      ]),
     );
     initIntercepts({});