Cypress e2e - Implementation of Cypress Grep

[antowaddle]

https://issues.redhat.com/browse/RHOAIENG-14724

Description

• Implementation of https://github.com/cypress-io/cypress/tree/develop/npm/grep
• Allows tests to be parameterised and executed via tags i.e. @smoke 
• Test files have been updated with Tags currently added to the tests in the Robot framework, additionally Smoke has been split into two distinct test sets (see below).
• This PR allows for the parallel execution of smoke tests by running separate commands in different terminals: npx cypress run --env grepTags=@SmokeSet1 and npx cypress run --env grepTags=@SmokeSet2.

How Has This Been Tested?

• An oc login should be performed in the cluster before running the test
• test-variables.yml should be configured properly
• Export the path to the test-variables.yml: $ export CY_TEST_CONFIG=<path_to>/test-variables.yml
  

Test Impact

None - this is a test

How to run?

After exporting the test-variables.yml, there are now a variety of ways to run tests

Execute all tests tagged with a specific tag using Chrome
$npx cypress run --env grepTags=@ --browser chrome

Execute all smoke tests but omit any tests tagged with Bug
$npx cypress run --env grepTags=@smoke,grepTags=-@bug

Using 2 separate terminals/containers etc. the Smoke Tests can also be executed in parallel (note that the 2 failing tests are either failing due to a known issue or are also failing running independently i.e. they are not related to running the tests in parallel).
* Note, it's recommended to disable video recording when executing multiple tests in parallel locally due to cypress-io/cypress#18415. (cypress.config.ts - set video: false).

• npx cypress run --env grepTags=@SmokeSet1 --browser chrome
• npx cypress run --env grepTags=@SmokeSet2 --browser chrome
  

Run only the tests with "data science" in the title
$ npx cypress run --env grep="data science" --browser chrome

Run tests with "hello" or "auth user" in their titles by separating them with ";" character
$ npx cypress run --env grep="hello; auth user" --browser chrome

Run only the tests tagged "smoke" that have "login" in their titles
$ npx cypress run --env grep=login,grepTags=Smoke --browser chrome

Run only tests that do not have any tags and are not inside suites that have any tags
$ npx cypress run --env grepUntagged=true --browser chrome

Request review criteria:

Self checklist (all need to be checked):

• The developer has manually tested the changes and verified that the changes work
• Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
• The developer has added tests or explained why testing cannot be added (unit or cypress tests for related changes)

If you have UI changes:

• Included any necessary screenshots or gifs if it was a UI change.
• Included tags to the UX team if it was a UI/UX change.

After the PR is posted & before it merges:

• The developer has tested their solution on a cluster by using the image produced by the PR to main

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.33%. Comparing base (0f2b4f8) to head (d2e2cd2).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3609      +/-   ##
==========================================
+ Coverage   85.30%   85.33%   +0.02%     
==========================================
  Files        1395     1395              
  Lines       32047    32047              
  Branches     8983     8983              
==========================================
+ Hits        27338    27346       +8     
+ Misses       4709     4701       -8     

see 5 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0f2b4f8...d2e2cd2. Read the comment docs.

[FedeAlonso]

/lgtm

[FedeAlonso]

Tested different cases:
$ npx cypress run --env grepTags=@smoke --browser chrome

$ npx cypress run --env grepTags=@SmokeSet1 --browser chrome
$ npx cypress run --env grepTags=@SmokeSet2 --browser chrome

$ npx cypress run --env grepTags=-@smoke --browser chrome

[manaswinidas]

/lgtm

The plugin is in optionalDependencies and package-lock.json is updated accordingly.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: FedeAlonso, manaswinidas

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• frontend/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment