Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs for Circleci rerun failed tests feature #146

Merged
merged 14 commits into from
Dec 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docusaurus/docs/knapsack_pro-ruby/guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -430,6 +430,8 @@ jobs:

Here you can find an example of a [Rails project config on CircleCI 2.0](https://docs.knapsackpro.com/2017/circleci-2-0-capybara-feature-specs-selenium-webdriver-with-chrome-headless).

See how to configure [advanced CircleCI features](../../ruby/circleci.md) with Knapsack Pro to collect reports and rerun only failed tests.

</ShowIfSearchParamAndValue>

<ShowIfSearchParamAndValue searchParam="ci" value="cirrus-ci">
Expand Down
41 changes: 40 additions & 1 deletion docusaurus/docs/ruby/circleci.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ pagination_prev: null

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import { YouTube } from '@site/src/components/YouTube';

# Using Knapsack Pro with CircleCI

Expand Down Expand Up @@ -47,7 +48,7 @@ end
name: RSpec with Knapsack Pro in Queue Mode
command: |
export CIRCLE_TEST_REPORTS=/tmp/test-results
mkdir $CIRCLE_TEST_REPORTS
mkdir -p $CIRCLE_TEST_REPORTS
bundle exec rake "knapsack_pro:queue:rspec[--format documentation --format RspecJunitFormatter --out tmp/rspec.xml]"

- store_test_results:
Expand All @@ -62,3 +63,41 @@ end
</Tabs>

You can find a complete CircleCI configuration in [RSpec testing parallel jobs with CircleCI and JUnit XML report](https://docs.knapsackpro.com/2021/rspec-testing-parallel-jobs-with-circleci-and-junit-xml-report).

## Rerun _only_ failed tests

Use the [CircleCI rerun failed tests](https://circleci.com/docs/rerun-failed-tests/) feature with Knapsack Pro to only rerun a subset of tests instead of rerunning the entire test suite when a transient test failure arises.

<YouTube src="https://www.youtube.com/embed/9WEsUosgTGw" />

<br />

```yaml title=".circleci/config.yml"
- run:
name: RSpec with Knapsack Pro in Queue Mode
command: |
export CIRCLE_TEST_REPORTS=/tmp/test-results
mkdir -p $CIRCLE_TEST_REPORTS

export KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=true

# highlight-start
export KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE=/tmp/tests_to_run.txt
# Retrieve the tests to run (all or just the failed ones), and let Knapsack Pro split them optimally.
circleci tests glob "spec/**/*_spec.rb" | circleci tests run --index 0 --total 1 --command ">$KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE xargs -n1 echo" --verbose > $KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE
bundle exec rake "knapsack_pro:queue:rspec[--format documentation --format RspecJunitFormatter --out tmp/rspec.xml]"
# highlight-end

- store_test_results:
path: /tmp/test-results

- store_artifacts:
path: /tmp/test-results
destination: test-results
```

The snippet above uses:

- [collecting meta data with JUnit XML report](#collect-metadata-in-queue-mode)
- [`KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES`](split-by-test-examples.md) to split slow spec files by test examples
- [`KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE`](reference.md#knapsack_pro_test_file_list_source_file) to specify what tests to run