Skip to content

Commit

Permalink
Speed up the collate-test-results workflow
Browse files Browse the repository at this point in the history
by building a single combined test results artifact that the workflow can
download, instead of having to download *all* artifacts (which includes all
of the built binaries, portables, etc). This downloading was taking betweem
3 and 20 minutes, but now only 20 ~ 30 seconds, for the same results.

Squashed commit of the following:

commit b159f5f
Merge: 6085464 bb6e657
Author: Paul Colby <git@colby.id.au>
Date:   Wed Nov 22 23:00:45 2023 +1100

    Merge branch 'main' into collate-speedup

commit 6085464
Author: Paul Colby <git@colby.id.au>
Date:   Wed Nov 22 22:52:00 2023 +1100

    Correct step name

commit de8aeb3
Author: Paul Colby <git@colby.id.au>
Date:   Wed Nov 22 22:49:02 2023 +1100

    Revert "Fix one build failure"

    This reverts commit e8b4b25.

commit e3e2f1c
Author: Paul Colby <git@colby.id.au>
Date:   Wed Nov 22 22:48:48 2023 +1100

    Revert "Reduce the number of builds so we can test the end"

    This reverts commit 67ec928.

commit 2b63a0e
Author: Paul Colby <git@colby.id.au>
Date:   Wed Nov 22 22:47:36 2023 +1100

    Ignore missing *.tap files for now

commit 41c03b6
Author: Paul Colby <git@colby.id.au>
Date:   Wed Nov 22 22:28:42 2023 +1100

    Update path to the removeHtmlDates.sh script

commit 1b82869
Author: Paul Colby <git@colby.id.au>
Date:   Wed Nov 22 22:12:35 2023 +1100

    Check is file exists before copying

commit fc8d518
Author: Paul Colby <git@colby.id.au>
Date:   Wed Nov 22 22:03:34 2023 +1100

    Put the .sh file in the root, and the rest in build type dirs

commit e8b4b25
Author: Paul Colby <git@colby.id.au>
Date:   Wed Nov 22 21:37:52 2023 +1100

    Fix one build failure

commit 67ec928
Author: Paul Colby <git@colby.id.au>
Date:   Wed Nov 22 21:31:27 2023 +1100

    Reduce the number of builds so we can test the end

commit 4cf180c
Author: Paul Colby <git@colby.id.au>
Date:   Wed Nov 22 21:15:45 2023 +1100

    cd manually on macOS

commit 329c5ce
Author: Paul Colby <git@colby.id.au>
Date:   Wed Nov 22 21:00:46 2023 +1100

    See if gfind is installed on macos

commit de372ca
Author: Paul Colby <git@colby.id.au>
Date:   Tue Nov 21 21:26:35 2023 +1100

    Use find wiht cpio instead of cp (which is more brittle)

commit 2e8ff97
Author: Paul Colby <git@colby.id.au>
Date:   Tue Nov 21 20:30:56 2023 +1100

    Try combined test assets
  • Loading branch information
pcolby committed Nov 22, 2023
1 parent bb6e657 commit 45e66cc
Showing 1 changed file with 39 additions and 2 deletions.
41 changes: 39 additions & 2 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ name: Build and Test

on: [push, pull_request]

# \todo The test-related if-no-files-found settings could be ignore for pre-Qt 5.12 and error otherwise.

jobs:
linux0:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -50,6 +52,17 @@ jobs:
${{ runner.temp }}/removeHtmlDates.sh
${{ runner.temp }}/test/**/*.tap
if-no-files-found: ignore
- name: Prep combined test results
run: |
find "$RUNNER_TEMP" \( -name 'coverage.info' -or -name '*.tap' \) -printf '%P\0' |
cpio -pmd0 -D "$RUNNER_TEMP" "$GITHUB_WORKSPACE/test-results/linux-${{ matrix.env.cc }}"
[[ ! -e "$RUNNER_TEMP/removeHtmlDates.sh" ]] || cp -av "$RUNNER_TEMP/removeHtmlDates.sh" "$GITHUB_WORKSPACE/test-results"
- name: Upload combined test results
uses: actions/upload-artifact@v3
with:
name: test-results-combined
path: ${{ github.workspace }}/test-results
if-no-files-found: ignore
- name: Report parallel coverage to Codacy
if: matrix.env.coverage && github.event_name == 'push' && github.actor != 'dependabot[bot]'
env:
Expand Down Expand Up @@ -134,6 +147,17 @@ jobs:
${{ runner.temp }}/coverage.info
${{ runner.temp }}/test/**/*.tap
if-no-files-found: ignore
- name: Prep combined test results
run: |
find "$RUNNER_TEMP" \( -name 'coverage.info' -or -name '*.tap' \) -printf '%P\0' |
cpio -pmd0 -D "$RUNNER_TEMP" "$GITHUB_WORKSPACE/test-results/linux-${{ matrix.env.cc }}-${{ matrix.qt }}"
[[ ! -e "$RUNNER_TEMP/removeHtmlDates.sh" ]] || cp -av "$RUNNER_TEMP/removeHtmlDates.sh" "$GITHUB_WORKSPACE/test-results"
- name: Upload combined test results
uses: actions/upload-artifact@v3
with:
name: test-results-combined
path: ${{ github.workspace }}/test-results
if-no-files-found: ignore
- name: Report parallel coverage to Codacy
if: matrix.env.coverage && github.event_name == 'push' && github.actor != 'dependabot[bot]'
env:
Expand Down Expand Up @@ -224,6 +248,18 @@ jobs:
${{ runner.temp }}/coverage.info
${{ runner.temp }}/test/**/*.tap
if-no-files-found: ignore
- name: Prep combined test results
run: |
cd "$RUNNER_TEMP" &&
find "$RUNNER_TEMP" \( -name 'coverage.info' -or -name '*.tap' \) -print0 |
cpio -pmd0 "$GITHUB_WORKSPACE/test-results/mac-${{ matrix.env.cc }}-${{ matrix.qt }}"
[[ ! -e "$RUNNER_TEMP/removeHtmlDates.sh" ]] || cp -av "$RUNNER_TEMP/removeHtmlDates.sh" "$GITHUB_WORKSPACE/test-results"
- name: Upload combined test results
uses: actions/upload-artifact@v3
with:
name: test-results-combined
path: ${{ github.workspace }}/test-results
if-no-files-found: ignore
- name: Report parallel coverage to Codacy
if: matrix.env.coverage && github.event_name == 'push' && github.actor != 'dependabot[bot]'
env:
Expand Down Expand Up @@ -407,9 +443,10 @@ jobs:
sudo apt update
sudo apt upgrade
sudo apt install lcov
- name: Download all artifacts
- name: Download test results
uses: actions/download-artifact@v3
with:
name: test-results-combined
path: ${{ runner.temp }}/artifacts
- name: Generate test summary
uses: pcolby/tap-summary@v1
Expand All @@ -425,7 +462,7 @@ jobs:
# Generate the HTML report.
echo 'genhtml_sort = 0' >> ~/.lcovrc
genhtml --output-directory coverage coverage.info
/usr/bin/env bash ./artifacts/test-results-linux-gcc/removeHtmlDates.sh
/usr/bin/env bash ./artifacts/removeHtmlDates.sh
working-directory: ${{ runner.temp }}
- name: Upload HTML coverage report
uses: actions/upload-artifact@v3
Expand Down

0 comments on commit 45e66cc

Please sign in to comment.