Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test(playwright): more flaky tests #6372

Merged
merged 1 commit into from
Oct 26, 2023
Merged

test(playwright): more flaky tests #6372

merged 1 commit into from
Oct 26, 2023

Conversation

stephenogorman
Copy link
Contributor

@stephenogorman stephenogorman commented Oct 13, 2023 •
edited by divyajindel
Loading

Proposed behaviour

Fix flaky Batch-Selection, Button & Dialog Playwright tests

Current behaviour

Tests regularly fail during CI checks

Checklist

  • Commits follow our style guide
  • Related issues linked in commit messages if required
  • Screenshots are included in the PR if useful
  • All themes are supported if required
  • Unit tests added or updated if required
  • Cypress automation tests added or updated if required
  • Playwright automation tests added or updated if required
  • Storybook added or updated if required
  • Translations added or updated (including creating or amending translation keys table in storybook) if required
  • Typescript d.ts file added or updated if required
  • Related docs have been updated if required

QA

  • Tested in CodeSandbox/storybook
  • Add new Cypress test coverage if required
  • Carbon implementation matches Design System/designs
  • UI Tests GitHub check reviewed if required

Additional context

N/A

Testing instructions

Testing instructions

  • Run Button tests in Playwright Test Runner to check if the button.pw.tsx file passed
  • Run Batch-Selection tests in Playwright Test Runner to check if the batch-selection.pw.tsx file passed
  • Run Dialog tests in Playwright Test Runner to check if the dialog.pw.tsx file passed
  • Run all other tests in Playwright Test Runner to check none of the other *.pw.tsx files have regressed

@stephenogorman stephenogorman self-assigned this Oct 13, 2023
@codesandbox-ci
Copy link

codesandbox-ci bot commented Oct 13, 2023 •
edited
Loading

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit c090f2d:

Sandbox Source
carbon-quickstart Configuration
carbon-quickstart-typescript Configuration

divyajindel
divyajindel requested changes Oct 17, 2023
View reviewed changes
src/components/batch-selection/batch-selection.pw.tsx Outdated
@@ -39,43 +40,38 @@ test.describe("check BatchSelection component properties", () => {
test("should check hidden BatchSelection", async ({ mount, page }) => {
await mount(<BatchSelectionComponent hidden />);
const batchSelection = batchSelectionComponent(page);
await expect(batchSelection.getAttribute("hidden")).not.toBeNull();

// await expect(batchSelection.getAttribute("hidden")).not.toBeNull();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this commented code.

sianford
sianford reviewed Oct 17, 2023
View reviewed changes
src/components/dialog/dialog.pw.tsx Outdated
@@ -270,6 +270,7 @@ test.describe("Testing Dialog component properties", () => {
const firstTextbox = page.getByLabel("Textbox1");
const closeButton = page.getByLabel("Close");

await page.waitForTimeout(1000);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these timeouts needed? 1 second seems like a long time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each of the dialog tests that either tabs or shift+tabs through the elements regularly fails without some kind of wait before the first tab, especially when running through the UI. I'm not a fan of putting in arbitrary wait times, but this at least makes the test do what it's supposed to. I've reduced each wait time to a quarter of a second.

@stephenogorman stephenogorman force-pushed the pw_refactor_moreflakytests branch from ca80a50 to c090f2d Compare October 25, 2023 16:13
@stephenogorman stephenogorman marked this pull request as ready for review October 26, 2023 09:05
@stephenogorman stephenogorman requested a review from a team as a code owner October 26, 2023 09:05
@stephenogorman stephenogorman merged commit 522b861 into master Oct 26, 2023
@stephenogorman stephenogorman deleted the pw_refactor_moreflakytests branch October 26, 2023 09:15
@carbonci
Copy link
Collaborator

🎉 This PR is included in version 123.0.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@divyajindel divyajindel divyajindel approved these changes

@sianford sianford sianford approved these changes

Assignees

@stephenogorman stephenogorman

Labels
batch-selection button dialog No UX QA Required Passed QA playwright released Squad Front End
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@stephenogorman @carbonci @sianford @divyajindel