Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy preview with VRT/AAT reports (Take Two) #5537

Merged
merged 4 commits into from
Jan 14, 2025
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,7 @@ jobs:
path: playwright-report
- name: check vrt-runner job status
if: ${{ needs.vrt-runner.result == 'failure' }}
continue-on-error: true
run: exit 1

vrt-runner-all-flags:
Expand Down Expand Up @@ -246,6 +247,7 @@ jobs:
path: playwright-report
- name: check vrt-runner-all-flags job status
if: ${{ needs.vrt-runner-all-flags.result == 'failure' }}
continue-on-error: true
run: exit 1

aat-runner:
Expand Down Expand Up @@ -318,6 +320,7 @@ jobs:
path: playwright-report
- name: Check aat-runner job status
if: ${{ needs.aat-runner.result == 'failure' }}
continue-on-error: true
run: exit 1

aat-runner-all-flags:
Expand Down Expand Up @@ -394,6 +397,7 @@ jobs:
path: playwright-report
- name: Check aat-runner-all-flags job status
if: ${{ needs.aat-runner-all-flags.result == 'failure' }}
continue-on-error: true
run: exit 1

build-components-json:
Expand Down
32 changes: 1 addition & 31 deletions .github/workflows/deploy_preview.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ jobs:
build:
if: ${{ github.event.pull_request.head.repo.full_name == 'primer/react' }}
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4
Expand Down Expand Up @@ -45,34 +46,3 @@ jobs:
uses: actions/deploy-pages@v4
with:
preview: true

deploy-storybook:
name: Preview Storybook
if: ${{ github.event.pull_request.head.repo.full_name == 'primer/react' }}
needs: deploy-preview
permissions:
deployments: write
runs-on: ubuntu-latest
steps:
- uses: chrnorm/deployment-action@v2.0.7
name: Create GitHub deployment for storybook
id: storybook
with:
token: ${{ secrets.GITHUB_TOKEN }}
environment: storybook-preview-${{ github.event.number }}
environment_url: '${{ needs.deploy-preview.outputs.deployment_url }}/storybook'
- name: Update storybook deployment status (success)
if: success()
uses: chrnorm/deployment-status@v2.0.3
with:
token: ${{ secrets.GITHUB_TOKEN }}
environment-url: '${{ needs.deploy-preview.outputs.deployment_url }}/storybook'
state: 'success'
deployment-id: ${{ steps.storybook.outputs.deployment_id }}
- name: Update storybook deployment status (failure)
if: failure()
uses: chrnorm/deployment-status@v2.0.3
with:
token: ${{ secrets.GITHUB_TOKEN }}
state: 'failure'
deployment-id: ${{ steps.storybook.outputs.deployment_id }}
238 changes: 238 additions & 0 deletions .github/workflows/deploy_preview_with_reports.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,238 @@
name: Deploy Preview (with VRT/AAT reports)
on:
workflow_run:
workflows: [CI]
types:
- completed

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
build:
if: ${{ always() && github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.head_repository.full_name == 'primer/react' }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added github.event.workflow_run.event == 'pull_request' checks to each job to only run for pull_request events

runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Build docs preview
run: npm run build:docs:preview_with_reports
- name: Download VRT reports (All flags enabled)
uses: actions/download-artifact@v4
with:
name: vrt-all-flags
path: docs/public/vrt-all-flags
github-token: ${{ secrets.GITHUB_TOKEN }}
repository: ${{ github.event.workflow_run.head_repository.full_name }}
run-id: ${{ github.event.workflow_run.id }}
- name: Download VRT reports (No flags enabled)
uses: actions/download-artifact@v4
with:
name: vrt-no-flag
path: docs/public/vrt-no-flag
github-token: ${{ secrets.GITHUB_TOKEN }}
repository: ${{ github.event.workflow_run.head_repository.full_name }}
run-id: ${{ github.event.workflow_run.id }}
- name: Download AAT reports (All flags enabled)
uses: actions/download-artifact@v4
with:
name: axe-all-flags
path: docs/public/aat-all-flags
github-token: ${{ secrets.GITHUB_TOKEN }}
repository: ${{ github.event.workflow_run.head_repository.full_name }}
run-id: ${{ github.event.workflow_run.id }}
- name: Download AAT reports (No flags enabled)
uses: actions/download-artifact@v4
with:
name: axe
path: docs/public/aat-no-flag
github-token: ${{ secrets.GITHUB_TOKEN }}
repository: ${{ github.event.workflow_run.head_repository.full_name }}
run-id: ${{ github.event.workflow_run.id }}
- uses: actions/upload-pages-artifact@v3
with:
name: github-pages
path: docs/public

deploy-preview:
if: ${{ github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.head_repository.full_name == 'primer/react' }}
needs: build
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
outputs:
deployment_url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
with:
preview: true

deploy-storybook:
name: Preview Storybook
if: ${{ github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.head_repository.full_name == 'primer/react' }}
needs: deploy-preview
permissions:
deployments: write
runs-on: ubuntu-latest
steps:
- uses: chrnorm/deployment-action@v2.0.7
name: Create GitHub deployment for storybook
id: storybook
with:
token: ${{ secrets.GITHUB_TOKEN }}
environment: storybook-preview-${{ github.event.workflow_run.event.number }}
environment-url: '${{ needs.deploy-preview.outputs.deployment_url }}storybook'
- name: Update storybook deployment status (success)
if: success()
uses: chrnorm/deployment-status@v2.0.3
with:
token: ${{ secrets.GITHUB_TOKEN }}
environment-url: '${{ needs.deploy-preview.outputs.deployment_url }}storybook'
state: 'success'
deployment-id: ${{ steps.storybook.outputs.deployment_id }}
- name: Update storybook deployment status (failure)
if: failure()
uses: chrnorm/deployment-status@v2.0.3
with:
token: ${{ secrets.GITHUB_TOKEN }}
state: 'failure'
deployment-id: ${{ steps.storybook.outputs.deployment_id }}

deploy-vrt-no-flag:
name: VRT (No flags enabled)
if: ${{ github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.head_repository.full_name == 'primer/react' }}
needs: [deploy-preview]
permissions:
deployments: write
runs-on: ubuntu-latest
steps:
- uses: chrnorm/deployment-action@v2.0.7
name: Create GitHub deployment for vrt-no-flag
id: vrt-no-flag
with:
token: ${{ secrets.GITHUB_TOKEN }}
environment: vrt-no-flag-${{ github.event.workflow_run.head_branch }}
environment-url: '${{ needs.deploy-preview.outputs.deployment_url }}vrt-no-flag'
- name: Update vrt-no-flag deployment status (success)
if: success()
uses: chrnorm/deployment-status@v2.0.3
with:
token: ${{ secrets.GITHUB_TOKEN }}
environment-url: '${{ needs.deploy-preview.outputs.deployment_url }}vrt-no-flag'
state: 'success'
deployment-id: ${{ steps.vrt-no-flag.outputs.deployment_id }}
- name: Update vrt-no-flag deployment status (failure)
if: failure()
uses: chrnorm/deployment-status@v2.0.3
with:
token: ${{ secrets.GITHUB_TOKEN }}
state: 'failure'
deployment-id: ${{ steps.vrt-no-flag.outputs.deployment_id }}

deploy-vrt-all-flags:
name: VRT (All flags enabled)
if: ${{ github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.head_repository.full_name == 'primer/react' }}
needs: deploy-preview
permissions:
deployments: write
runs-on: ubuntu-latest
steps:
- uses: chrnorm/deployment-action@v2.0.7
name: Create GitHub deployment for vrt-all-flags
id: vrt-all-flags
with:
token: ${{ secrets.GITHUB_TOKEN }}
environment: vrt-all-flags-${{ github.event.workflow_run.event.number }}
environment-url: '${{ needs.deploy-preview.outputs.deployment_url }}vrt-all-flags'
- name: Update vrt-all-flags deployment status (success)
if: success()
uses: chrnorm/deployment-status@v2.0.3
with:
token: ${{ secrets.GITHUB_TOKEN }}
environment-url: '${{ needs.deploy-preview.outputs.deployment_url }}vrt-all-flags'
state: 'success'
deployment-id: ${{ steps.vrt-all-flags.outputs.deployment_id }}
- name: Update vrt-all-flags deployment status (failure)
if: failure()
uses: chrnorm/deployment-status@v2.0.3
with:
token: ${{ secrets.GITHUB_TOKEN }}
state: 'failure'
deployment-id: ${{ steps.vrt-all-flags.outputs.deployment_id }}

deploy-aat-no-flag:
name: AAT (No flags enabled)
if: ${{ github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.head_repository.full_name == 'primer/react' }}
needs: deploy-preview
permissions:
deployments: write
runs-on: ubuntu-latest
steps:
- uses: chrnorm/deployment-action@v2.0.7
name: Create GitHub deployment for aat-no-flag
id: aat-no-flag
with:
token: ${{ secrets.GITHUB_TOKEN }}
environment: aat-no-flag-${{ github.event.workflow_run.event.number }}
environment-url: '${{ needs.deploy-preview.outputs.deployment_url }}aat-no-flag'
- name: Update aat-no-flag deployment status (success)
if: success()
uses: chrnorm/deployment-status@v2.0.3
with:
token: ${{ secrets.GITHUB_TOKEN }}
environment-url: '${{ needs.deploy-preview.outputs.deployment_url }}aat-no-flag'
state: 'success'
deployment-id: ${{ steps.aat-no-flag.outputs.deployment_id }}
- name: Update aat-no-flag deployment status (failure)
if: failure()
uses: chrnorm/deployment-status@v2.0.3
with:
token: ${{ secrets.GITHUB_TOKEN }}
state: 'failure'
deployment-id: ${{ steps.aat-no-flag.outputs.deployment_id }}

deploy-aat-all-flags:
name: AAT (All flags enabled)
if: ${{ github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.head_repository.full_name == 'primer/react' }}
needs: deploy-preview
permissions:
deployments: write
runs-on: ubuntu-latest
steps:
- uses: chrnorm/deployment-action@v2.0.7
name: Create GitHub deployment for aat-all-flags
id: aat-all-flags
with:
token: ${{ secrets.GITHUB_TOKEN }}
environment: aat-all-flags-${{ github.event.workflow_run.event.number }}
environment-url: '${{ needs.deploy-preview.outputs.deployment_url }}aat-all-flags'
- name: Update aat-all-flags deployment status (success)
if: success()
uses: chrnorm/deployment-status@v2.0.3
with:
token: ${{ secrets.GITHUB_TOKEN }}
environment-url: '${{ needs.deploy-preview.outputs.deployment_url }}aat-all-flags'
state: 'success'
deployment-id: ${{ steps.aat-all-flags.outputs.deployment_id }}
- name: Update aat-all-flags deployment status (failure)
if: failure()
uses: chrnorm/deployment-status@v2.0.3
with:
token: ${{ secrets.GITHUB_TOKEN }}
state: 'failure'
deployment-id: ${{ steps.aat-all-flags.outputs.deployment_id }}
3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,11 @@
"start": "concurrently npm:start:*",
"start:storybook": "npm run start:storybook -w @primer/react",
"build:storybook": "script/build-storybook",
"build:storybook:preview": "script/build-storybook preview",
"build:storybook:preview_with_reports": "script/build-storybook preview_with_reports",
"build:docs": "NODE_OPTIONS=--openssl-legacy-provider script/build-docs",
"build:docs:preview": "NODE_OPTIONS=--openssl-legacy-provider script/build-docs preview",
"build:docs:preview_with_reports": "NODE_OPTIONS=--openssl-legacy-provider script/build-docs preview_with_reports",
"build:components.json": "npm run build:components.json -w @primer/react",
"lint": "eslint '**/*.{js,ts,tsx,md,mdx}' --max-warnings=0",
"lint:css": "stylelint --rd -q '**/*.css'",
Expand Down
22 changes: 22 additions & 0 deletions packages/react/.storybook/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Primer Preview Page</title>
<link href="https://unpkg.com/@primer/css@^20.2.4/dist/primer.css" rel="stylesheet" />
</head>
<body class="p-3">
<div class="Header">
<h1>Primer Preview Page</h1>
</div>
<div class="p-2 m-2 d-flex flex-column">
<ul class="filter-list">
<li><a class="filter-item" href="./storybook" class="btn-link mb-2">Storybook</a></li>
</ul>
</div>
<div class="color-bg-attention p-2 m-2 rounded border color-border-severe-emphasis">
Visual Regression Testing (VRT) and Automated Accessibility Testing (AAT) results are pending...
</div>
</body>
</html>
28 changes: 28 additions & 0 deletions packages/react/.storybook/index_with_reports.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Primer Preview Page</title>
<link href="https://unpkg.com/@primer/css@^20.2.4/dist/primer.css" rel="stylesheet" />
</head>

<body class="p-3">
<div class="Header">
<h1>Primer Preview Page</h1>
</div>
<div class="p-2 m-2 d-flex flex-column">
<ul class="filter-list">
<li><a class="filter-item" href="./storybook" class="btn-link mb-2">Storybook</a></li>
<li><a class="filter-item" href="./vrt-no-flag" class="btn-link mb-2">Visual Regression Testing</a></li>
<li><a class="filter-item" href="./vrt-all-flags" class="btn-link mb-2">Visual Regression Testing (All Flags
Enabled)</a></li>
<li><a class="filter-item" href="./aat-no-flag" class="btn-link mb-2">Automated Accessibility Testing</a></li>
<li><a class="filter-item" href="./aat-all-flags" class="btn-link mb-2">Automated Accessibility Testing (All Flags
Enabled)</a></li>
</ul>
</div>
</body>

</html>
1 change: 1 addition & 0 deletions packages/react/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
"build:storybook": "storybook build",
"build:docs": "NODE_OPTIONS=--openssl-legacy-provider script/build-docs",
"build:docs:preview": "NODE_OPTIONS=--openssl-legacy-provider script/build-docs preview",
"build:docs:preview_with_reports": "NODE_OPTIONS=--openssl-legacy-provider script/build-docs preview_with_reports",
"build:components.json": "tsx script/components-json/build.ts",
"build:precompile-color-schemes": "tsx script/precompile-color-schemes.ts",
"storybook": "storybook",
Expand Down
Loading
Loading