Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update CI and Changelog #433

Merged
merged 3 commits into from
Oct 22, 2024
Merged

update CI and Changelog #433

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
101fed4
update CI and Changelog
sateeshperi Oct 22, 2024
314fb8f
update README with 2 diff options for bismark align
sateeshperi Oct 22, 2024
ad2b289
update CI matrix options
sateeshperi Oct 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ env:
NFT_VER: "0.9.0"
NFT_WORKDIR: "~"
NFT_DIFF: "pdiff"
NFT_DIFF_ARGS: "--line-numbers --expand-tabs=2"
NFT_DIFF_ARGS: "--line-numbers --width 120 --expand-tabs=2"
NXF_SINGULARITY_CACHEDIR: ${{ github.workspace }}/.singularity
NXF_SINGULARITY_LIBRARYDIR: ${{ github.workspace }}/.singularity

Expand All @@ -35,6 +35,7 @@ jobs:
- "24.04.2"
- "latest-everything"
shard: [1, 2, 3, 4, 5]
filter: [process, workflow]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pipeline-level tests are not included?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah good catch! should be pipeline and workflow

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And process? I think the idea is to test everything?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its gonna overload the CI runners with too many tests to run

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have to bank on the fact that the modules were/are tested in the modules repo and the pipeline CI runs only pipeline and workflow tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should bring this up in the #nf-test channel and we can discuss/vote there

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if there are local modules with tests? Moreover, the tests are only triggered for the changed modules.

If there are no local modules or functions with tests, let's skip process.

One more thing. If the nextflow.config changes or something from the assets directory changes, will the pipeline-level tests be triggered? I like the CI of nf-core/rnaseq which checks for these changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plz bring this up in the nf-test channel and lets discuss there and get some consensus

profile:
- "conda"
- "docker"
Expand All @@ -49,6 +50,12 @@ jobs:
profile: "conda"
- isMaster: false
profile: "singularity"
- profile: "conda"
GallVp marked this conversation as resolved.
Show resolved Hide resolved
nf_test_files: "tests/bwameth_save_reference_save_align_intermeds.nf.test"
- profile: "conda"
nf_test_files: "tests/bwameth_skip_trimming.nf.test"
- profile: "conda"
nf_test_files: "tests/bwameth_skip_deduplication.nf.test"
steps:
- name: Check out pipeline code
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4
Expand Down Expand Up @@ -104,7 +111,13 @@ jobs:
--shard ${{ matrix.shard }}/${{ strategy.job-total }} \
--changed-since HEAD^ \
--profile "+${{ matrix.profile }}" \
--filter pipeline,workflow
--filter ${{ matrix.filter }}

- name: Output log on failure
if: failure()
run: |
sudo apt install bat > /dev/null
batcat --decorations=always --color=always ${{ github.workspace }}/.nf-test/tests/*/meta/nextflow.log

- name: Publish Test Report
uses: mikepenz/action-junit-report@v3
Expand Down
3 changes: 2 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# nf-core/methylseq

## [v2.7.0](https://github.com/nf-core/methylseq/releases/tag/2.7.0) - [2024-10-18]
## [v2.7.0](https://github.com/nf-core/methylseq/releases/tag/2.7.0) - [2024-10-22]

### Bug fixes & refactoring

Expand All @@ -16,6 +16,7 @@
- 🐛 fix bwameth pipeline skipping all bwameth steps when --fasta, --fasta_index, and --bwa_meth_index are provided. (issue-#396) [#414](https://github.com/nf-core/methylseq/pull/414)
- 🐛 fix typo in samplesheet in README and usage doc (issue-#391) [#414](https://github.com/nf-core/methylseq/pull/418)
- 🐛 stage fasta separately from the index folders for bismark & bwameth. Fixes issue with using a local copy of fasta (issue-#305) [#428](https://github.com/nf-core/methylseq/pull/428)
- 🐛 fix bismark, bwameth align modules + other module updates + update snaps [#432](https://github.com/nf-core/methylseq/pull/432)

### Pipeline Updates

Expand Down
28 changes: 14 additions & 14 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,20 +30,20 @@ On release, automated continuous integration tests run the pipeline on a full-si
The pipeline allows you to choose between running either [Bismark](https://github.com/FelixKrueger/Bismark) or [bwa-meth](https://github.com/brentp/bwa-meth) / [MethylDackel](https://github.com/dpryan79/methyldackel).
Choose between workflows by using `--aligner bismark` (default, uses bowtie2 for alignment), `--aligner bismark_hisat` or `--aligner bwameth`.

| Step | Bismark workflow | bwa-meth workflow |
| -------------------------------------------- | ---------------- | --------------------- |
| Generate Reference Genome Index _(optional)_ | Bismark | bwa-meth |
| Merge re-sequenced FastQ files | cat | cat |
| Raw data QC | FastQC | FastQC |
| Adapter sequence trimming | Trim Galore! | Trim Galore! |
| Align Reads | Bismark | bwa-meth |
| Deduplicate Alignments | Bismark | Picard MarkDuplicates |
| Extract methylation calls | Bismark | MethylDackel |
| Sample report | Bismark | - |
| Summary Report | Bismark | - |
| Alignment QC | Qualimap | Qualimap |
| Sample complexity | Preseq | Preseq |
| Project Report | MultiQC | MultiQC |
| Step | Bismark workflow | bwa-meth workflow |
| -------------------------------------------- | ------------------------ | --------------------- |
| Generate Reference Genome Index _(optional)_ | Bismark | bwa-meth |
| Merge re-sequenced FastQ files | cat | cat |
| Raw data QC | FastQC | FastQC |
| Adapter sequence trimming | Trim Galore! | Trim Galore! |
| Align Reads | Bismark (bowtie2/hisat2) | bwa-meth |
| Deduplicate Alignments | Bismark | Picard MarkDuplicates |
| Extract methylation calls | Bismark | MethylDackel |
| Sample report | Bismark | - |
| Summary Report | Bismark | - |
| Alignment QC | Qualimap | Qualimap |
| Sample complexity | Preseq | Preseq |
| Project Report | MultiQC | MultiQC |

## Usage

Expand Down
Loading