Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add coptr module #6935

Merged
merged 45 commits into from
Nov 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
dad8548
Initial addition
ramirobarrantes Nov 5, 2024
77c71db
Updated snapshot
ramirobarrantes Nov 5, 2024
0416062
Updated snapshot
ramirobarrantes Nov 6, 2024
9579e60
Updated tests.
ramirobarrantes Nov 6, 2024
bcec266
Updated tests.
ramirobarrantes Nov 6, 2024
d117ef5
Updated snapshot
ramirobarrantes Nov 6, 2024
03e33ec
removed extra directory creation
ramirobarrantes Nov 6, 2024
0a502b6
Added plugin to only compare bam reads, not header
ramirobarrantes Nov 7, 2024
b5da004
Added plugin to only compare bam reads, not header
ramirobarrantes Nov 7, 2024
3a7c330
Added plugin to only compare bam reads, not header
ramirobarrantes Nov 7, 2024
78546ff
Output filename now also contains index id so as to not to repeat fil…
ramirobarrantes Nov 7, 2024
41b1ba3
Updated snapshot
ramirobarrantes Nov 7, 2024
014d1d2
Updated snapshot
ramirobarrantes Nov 7, 2024
e3c26af
Added paired end test
ramirobarrantes Nov 7, 2024
786ca40
Added paired end test
ramirobarrantes Nov 7, 2024
ad56b3b
Updated tests to deal with bams changing order of reads each time
ramirobarrantes Nov 8, 2024
7d37343
Updated tests to deal with bams changing order of reads each time
ramirobarrantes Nov 8, 2024
01022eb
Updated tests to deal with bams changing order of reads each time
ramirobarrantes Nov 8, 2024
eb8364b
Updated tests to deal with bams changing order of reads each time
ramirobarrantes Nov 8, 2024
14457c3
Merge branch 'master' into coptr_map2
jfy133 Nov 12, 2024
98faf7a
Updated container used
ramirobarrantes Nov 13, 2024
15f0771
Added number of lines test for review
ramirobarrantes Nov 13, 2024
b3f5757
Added number of lines test for review
ramirobarrantes Nov 13, 2024
1becbd0
Updated nft-bam plugin
ramirobarrantes Nov 14, 2024
5f11a65
Updated container
ramirobarrantes Nov 14, 2024
0990246
Updated counting of reads
ramirobarrantes Nov 14, 2024
08e3e90
Updated counting of reads
ramirobarrantes Nov 14, 2024
56825bb
removed
ramirobarrantes Nov 14, 2024
b5413b5
updated snapshot
ramirobarrantes Nov 14, 2024
861c997
Updated test and snapshot
ramirobarrantes Nov 15, 2024
d36a0dc
Updated test and snapshot
ramirobarrantes Nov 15, 2024
d36035e
fixed versions issue
ramirobarrantes Nov 15, 2024
3098168
fixed versions issue
ramirobarrantes Nov 15, 2024
163041c
fixed versions issue
ramirobarrantes Nov 15, 2024
a453a1d
fixed versions issue
ramirobarrantes Nov 15, 2024
e6da7da
fixed versions issue
ramirobarrantes Nov 15, 2024
c52eff1
fixed versions issue
ramirobarrantes Nov 15, 2024
2704cc4
Updated test
ramirobarrantes Nov 15, 2024
463c469
Updated test
ramirobarrantes Nov 15, 2024
4ff3bbc
Updated snapshot
ramirobarrantes Nov 15, 2024
f2b04cc
Updated tag
ramirobarrantes Nov 15, 2024
ef7c169
Updated tests
ramirobarrantes Nov 15, 2024
1b963eb
Updated tests
ramirobarrantes Nov 15, 2024
7511481
Merge branch 'master' into coptr_map2
itrujnara Nov 19, 2024
d5f55e8
Improved descriptions
ramirobarrantes Nov 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions modules/nf-core/coptr/index/main.nf
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ process COPTR_INDEX {

conda "${moduleDir}/environment.yml"
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://depot.galaxyproject.org/singularity/coptr:1.1.4--pyhdfd78af_1':
'biocontainers/coptr:1.1.4--pyhdfd78af_1' }"
'https://depot.galaxyproject.org/singularity/coptr:1.1.4--pyhdfd78af_3':
'biocontainers/coptr:1.1.4--pyhdfd78af_3' }"

input:
tuple val(meta), path(indexfasta, stageAs: "fastafolder/*")
Expand All @@ -29,6 +29,7 @@ process COPTR_INDEX {
--bt2-threads $task.cpus \
fastafolder \
bowtie2/${prefix}

cat <<-END_VERSIONS > versions.yml
"${task.process}":
coptr: \$(coptr |& sed -E '11!d ; s/CoPTR.*?\\(v(.*?)\\).*/\\1/')
Expand Down
27 changes: 2 additions & 25 deletions modules/nf-core/coptr/index/tests/main.nf.test
Original file line number Diff line number Diff line change
Expand Up @@ -32,29 +32,7 @@ nextflow_process {

}

test("coptr index with multiple fastqs") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason to remove this test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed something and the test broke, but I was going to address this in a separate PR as it's a different module

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, no problem. There might be some issues with tests as they are currently updating the CI workflow to use shards.


when {
process {
"""
input[0] = [
[id:'indexNameExample'],
[file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome2.fasta', checkIfExists: true),file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome3.fasta', checkIfExists: true)]
]
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(process.out).match() }
)
}

}

test("coptr index with multiple fastqs - stub") {
test("coptr index with single fastq - stub") {

options "-stub"

Expand All @@ -63,7 +41,7 @@ nextflow_process {
"""
input[0] = [
[id:'indexNameExample'],
[file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome2.fasta', checkIfExists: true),file(params.modules_testdata_base_path + 'genomics/homo_sapiens/genome/genome3.fasta', checkIfExists: true)]
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)
]
"""
}
Expand All @@ -78,5 +56,4 @@ nextflow_process {

}


}
59 changes: 5 additions & 54 deletions modules/nf-core/coptr/index/tests/main.nf.test.snap
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,12 @@
}
],
"meta": {
"nf-test": "0.9.1",
"nf-test": "0.9.2",
"nextflow": "24.10.0"
},
"timestamp": "2024-11-01T15:20:08.028567181"
"timestamp": "2024-11-15T13:37:15.478846"
},
"coptr index with multiple fastqs - stub": {
"coptr index with single fastq - stub": {
"content": [
{
"0": [
Expand Down Expand Up @@ -92,58 +92,9 @@
}
],
"meta": {
"nf-test": "0.9.1",
"nf-test": "0.9.2",
"nextflow": "24.10.0"
},
"timestamp": "2024-11-01T15:28:00.318435045"
},
"coptr index with multiple fastqs": {
"content": [
{
"0": [
[
{
"id": "indexNameExample"
},
[
"indexNameExample.1.bt2:md5,e88e383183a9dec8dd453b04dc89a903",
"indexNameExample.2.bt2:md5,ae84a18d674ecc823c82188f5ca1ffc9",
"indexNameExample.3.bt2:md5,702fe6c79ef7bc9b54b3661f2f4aaa0b",
"indexNameExample.4.bt2:md5,a2234db869afc50b40f7e7d1289f5f0b",
"indexNameExample.genomes:md5,d4ed161da3cf2ced8e7c66c0c90e8d3e",
"indexNameExample.rev.1.bt2:md5,75d78b73949d636ec9750269a5ea1c8b",
"indexNameExample.rev.2.bt2:md5,39a1186f6aad701a71d773074408b51e"
]
]
],
"1": [
"versions.yml:md5,6013eeb9d8ac97351260150f44091986"
],
"index_dir": [
[
{
"id": "indexNameExample"
},
[
"indexNameExample.1.bt2:md5,e88e383183a9dec8dd453b04dc89a903",
"indexNameExample.2.bt2:md5,ae84a18d674ecc823c82188f5ca1ffc9",
"indexNameExample.3.bt2:md5,702fe6c79ef7bc9b54b3661f2f4aaa0b",
"indexNameExample.4.bt2:md5,a2234db869afc50b40f7e7d1289f5f0b",
"indexNameExample.genomes:md5,d4ed161da3cf2ced8e7c66c0c90e8d3e",
"indexNameExample.rev.1.bt2:md5,75d78b73949d636ec9750269a5ea1c8b",
"indexNameExample.rev.2.bt2:md5,39a1186f6aad701a71d773074408b51e"
]
]
],
"versions": [
"versions.yml:md5,6013eeb9d8ac97351260150f44091986"
]
}
],
"meta": {
"nf-test": "0.9.1",
"nextflow": "24.10.0"
},
"timestamp": "2024-11-01T15:24:04.847842182"
"timestamp": "2024-11-15T13:37:21.536745"
}
}
7 changes: 7 additions & 0 deletions modules/nf-core/coptr/map/environment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json
channels:
- conda-forge
- bioconda
dependencies:
- "bioconda::coptr=1.1.4"
61 changes: 61 additions & 0 deletions modules/nf-core/coptr/map/main.nf
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
process COPTR_MAP {
tag "$meta.id"
label 'process_low'

conda "${moduleDir}/environment.yml"
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://depot.galaxyproject.org/singularity/coptr:1.1.4--pyhdfd78af_3':
'biocontainers/coptr:1.1.4--pyhdfd78af_3' }"

input:
tuple val(meta), path(fasta, stageAs: "fastafolder/*")
tuple val(meta2), path(index)

output:
tuple val(meta), path("*.bam"), emit: bam
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
ramirobarrantes marked this conversation as resolved.
Show resolved Hide resolved
def prefix = task.ext.prefix ?: "${meta.id}"
def prefix2 = task.ext.prefix ?: "${meta2.id}"

def paired_end = ""
if ( ! meta.single_end ) {
paired_end = "--paired"
}
"""
INDEX=`find -L ./ -name "*.rev.1.bt2" | sed "s/\\.rev.1.bt2\$//"`
[ -z "\$INDEX" ] && INDEX=`find -L ./ -name "*.rev.1.bt2l" | sed "s/\\.rev.1.bt2l\$//"`
[ -z "\$INDEX" ] && echo "Bowtie2 index files not found" 1>&2 && exit 1

coptr \
map \
$args $paired_end \
--threads $task.cpus \
\$INDEX \
fastafolder \
.

mv ${prefix}.bam ${prefix}_${prefix2}.bam
cat <<-END_VERSIONS > versions.yml
"${task.process}":
coptr: \$(coptr |& sed -E '11!d ; s/CoPTR.*?\\(v(.*?)\\).*/\\1/')
END_VERSIONS
"""

stub:
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${meta.id}"
"""
touch ${prefix}.bam

cat <<-END_VERSIONS > versions.yml
"${task.process}":
coptr: \$(coptr |& sed -E '11!d ; s/CoPTR.*?\\(v(.*?)\\).*/\\1/')
END_VERSIONS
"""
}
63 changes: 63 additions & 0 deletions modules/nf-core/coptr/map/meta.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/meta-schema.json
name: "coptr_map"
description: Maps the reads to the reference database
keywords:
- coptr
- mapping
- ptr
tools:
- "coptr":
description: "Accurate and robust inference of microbial growth dynamics from
metagenomic sequencing reads."
homepage: "https://github.com/tyjo/coptr"
documentation: "https://coptr.readthedocs.io/"
tool_dev_url: "https://github.com/tyjo/coptr"
doi: "10.1101/gr.275533.121"
licence: ["GPL v3"]
identifier: biotools:coptr

input:
- - meta:
type: map
description: |
Groovy Map containing sample information
e.g. `[ id:'sample1', single_end:false ]`
- fasta:
type: file
description: fastq file with reads
pattern: "*.{.fastq,.fq,.fastq.gz,fq.gz}"
ontologies:
- edam: "http://edamontology.org/format_1929"
- - meta2:
type: map
description: |
Groovy Map containing index genome id and path
e.g. [ id:'test', 'bowtie2' ]
- index:
type: file
description: Directory with Bowtie2 genome index files
pattern: "*.ebwt"

output:
- bam:
- meta:
type: map
description: |
Groovy Map containing sample information
e.g. `[ id:'sample1', single_end:false ]`
- "*.bam":
type: file
description: Alignment (BAM) file of reads mapped to the reference database
pattern: "*.{bam}"
ontologies:
- edam: "http://edamontology.org/format_2572"
- versions:
- "versions.yml":
type: file
description: File containing software versions
pattern: "versions.yml"
authors:
- "@ramirobarrantes"
maintainers:
- "@ramirobarrantes"
106 changes: 106 additions & 0 deletions modules/nf-core/coptr/map/tests/main.nf.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
nextflow_process {

name "Test Process COPTR_MAP"
script "../main.nf"
process "COPTR_MAP"

tag "modules"
tag "modules_nfcore"
tag "coptr"
tag "coptr/index"
tag "coptr/map"

setup {
run("COPTR_INDEX") {
script "../../index/main.nf"
process {
"""
input[0] = [
[id:'test_0'],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)
]
"""
}
}
}

test("sarscov2 - single end test") {

when {
process {
"""
input[0] = [
[ id:'test_1', single_end: true ],
file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/fastq/test_1.fastq.gz", checkIfExists: true)
]
input[1] = COPTR_INDEX.out.index_dir
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(
process.out.bam.collect{bam(it[1]).getReads(10)},
ramirobarrantes marked this conversation as resolved.
Show resolved Hide resolved
process.out.bam.collect{bam(it[1]).getStatistics(include:["readCount"])},
process.out.versions
).match() }
)
}
}

test("sarscov2 - paired end") {

when {
process {
"""
input[0] = [
[ id:'test', single_end: false ],
[
file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/fastq/test_1.fastq.gz", checkIfExists: true),
file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/fastq/test_2.fastq.gz", checkIfExists: true)
]
]
input[1] = COPTR_INDEX.out.index_dir
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(
process.out.bam.collect{bam(it[1]).getStatistics(include:["readCount"])},
process.out.versions
).match() }
)
}
}


test("sarscov2 - single end - stub") {

options "-stub"

when {
process {
"""
input[0] = [
[ id:'test', single_end: true ],
file(params.modules_testdata_base_path + "genomics/sarscov2/illumina/fastq/test_1.fastq.gz", checkIfExists: true)
]
input[1] = COPTR_INDEX.out.index_dir
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(process.out).match() }
)
}
}

}
Loading
Loading