Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Seqera components to reference workflows from earlier sections #489

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/advanced/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ This gives us two complications:
3. Config file specified using the `-c my_config option`
4. The config file named `nextflow.config` in the current directory
5. The config file named `nextflow.config` in the workflow project directory
6. The config file `$HOME/.nextflow/config`
6. The config file `$HOME/.nextflow/config`, or `$NXF_HOME/.nextflow/config` when `NXF_HOME` is set (see [`NXF` variable docs](reference/env-vars.html#nextflow-settings))
7. Values defined within the pipeline script itself (e.g. `main.nf`)

!!! note "Precedence is in order of 'distance'"
Expand Down
2 changes: 1 addition & 1 deletion docs/basic_training/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ When you launch Nextflow, it will look for configuration files in several locati
3. Config file that are provided using the `-c` option
4. The config file named `nextflow.config` in the current directory
5. The config file named `nextflow.config` in the pipeline project directory
6. The config file `$HOME/.nextflow/config`
6. The config file `$HOME/.nextflow/config`, or `$NXF_HOME/.nextflow/config` when `NXF_HOME` is set (see [`NXF` variable docs](reference/env-vars.html#nextflow-settings))
7. Values defined within the pipeline script itself (e.g., `main.nf`)

## Parameters
Expand Down
57 changes: 24 additions & 33 deletions docs/hello_nextflow/seqera/01_run_with_cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,30 +41,23 @@ Follow these steps to set up your token:

### 1.2. Run Nextflow CLI with Seqera Platform visualizing and capturing logs

Run a Nextflow workflow with the addition of the `-with-tower` command:
Run the 'nf-hello-world' workflow you may have encountered in earlier sessions, with the addition of the `-with-tower` command:

```bash
nextflow run nextflow-io/hello -with-tower
nextflow run seqeralabs/nf-hello-world -profile demo --outdir hello -with-tower
```

You will see output similar to the following:

```console title="Output"
N E X T F L O W ~ version 24.04.4
N E X T F L O W ~ version 24.10.2

Launching `https://github.com/nextflow-io/hello` [evil_engelbart] DSL2 - revision: afff16a9b4 [master]
Launching `https://github.com/seqeralabs/nf-hello-world` [thirsty_hoover] DSL2 - revision: 8274d3d10c [master]

Downloading plugin nf-tower@1.9.1
Monitor the execution with Seqera Platform using this URL: https://cloud.seqera.io/user/kenbrewer/watch/5Gs0qqV9Y9rguE
executor > local (4)
[80/810411] process > sayHello (1) [100%] 4 of 4 ✔
Ciao world!

Bonjour world!

Hola world!

Hello world!
Monitor the execution with Seqera Platform using this URL: https://cloud.seqera.io/user/jonathan-manning2/watch/3kLeKzzjHBzB7D
executor > local (6)
[40/c954b7] process > sayHello (2) [100%] 3 of 3 ✔
[2f/8162ff] process > convertToUpper (3) [100%] 3 of 3 ✔
```

Hold ++ctrl++ or ++cmd++ and click on the link to open it in your browser.
Expand All @@ -76,7 +69,11 @@ You will see and be able to monitor your **Nextflow jobs** in Seqera Platform.

### 1.3. Set up Seqera Platform in Nextflow configuration

Doing that token setup regularly can become tedious, so let's set this configuration for all our pipeline runs with the global Nextflow configuration file located at `$HOME/.nextflow/config`.
Doing that token setup regularly can become tedious, so let's set this configuration for all our pipeline runs with the global Nextflow configuration file located at `$NXF_HOME/.nextflow/config`.

!!! hint "Home directory configuration"

If `NFX_HOME` is not set, these changes would be added to `$HOME/.nextflow/config`

Before we set the configuration, we need to permanently store the token from our environment in Nextflow using a [Nextflow secret](https://www.nextflow.io/docs/latest/secrets.html):

Expand All @@ -90,15 +87,15 @@ Make sure your token was saved using:
nextflow secrets get tower_access_token
```

Next, open the Nextflow configuration file located at `$HOME/.nextflow/config`:
Next, open the Nextflow configuration file located at `$NXF_HOME/.nextflow/config`:

```bash
code $HOME/.nextflow/config
code $NXF_HOME/.nextflow/config
```

Then add the following block of configuration:

```groovy title="$HOME/.nextflow/config"
```groovy title="$NXF_HOME/.nextflow/config"
tower {
enabled = true
endpoint = "https://api.cloud.seqera.io"
Expand All @@ -120,20 +117,14 @@ nextflow run nextflow-io/hello
You will see the following output:

```console title="Output"
N E X T F L O W ~ version 24.04.4

Launching `https://github.com/nextflow-io/hello` [fabulous_euclid] DSL2 - revision: afff16a9b4 [master]

Monitor the execution with Seqera Platform using this URL: https://cloud.seqera.io/user/kenbrewer/watch/KYjRktIlOuxrh
executor > local (4)
[71/eaa915] process > sayHello (3) [100%] 4 of 4 ✔
Ciao world!

Bonjour world!
N E X T F L O W ~ version 24.10.3

Hola world!
Launching `https://github.com/seqeralabs/nf-hello-world` [golden_moriondo] DSL2 - revision: 8274d3d10c [master]

Hello world!
Monitor the execution with Seqera Platform using this URL: https://cloud.seqera.io/user/jonathan-manning2/watch/1lzYhUz4ziHcYD
executor > local (6)
[30/30a42c] process > sayHello (3) [100%] 3 of 3 ✔
[ea/293e98] process > convertToUpper (3) [100%] 3 of 3 ✔
```

Note that we are logging to Seqera Platform even though we did not use the `-with-tower` command!
Expand All @@ -147,8 +138,8 @@ If you ran your pipeline from the `hello_nextflow` directory, you'll see somethi

Notice that configuration for our pipeline run is being run pulled from three separate files:

- `/home/gitpod/.nextflow/config` - This is the global configuration file we just added.
- `/home/gitpod/.nextflow/assets/nextflow-io/hello/nextflow.config` - This is the `nextflow.config` file from the `nextflow-io/hello` repository.
- `/workspace/gitpod/.nextflow/config` - This is the global configuration file we just added.
- `/workspace/gitpod/.nextflow/assets/seqeralabs/nf-hello-world/nextflow.config` - This is the `nextflow.config` file from the `seqeralabs/nf-hello-world` repository.
- `/workspace/gitpod/nf-training/hello-nextflow/nextflow.config` - This is the `nextflow.config` file from our current working directory.

Nextflow resolves these configurations at runtime with a [specific order of precedence](https://www.nextflow.io/docs/latest/config.html#configuration-file).
Expand Down
12 changes: 6 additions & 6 deletions docs/hello_nextflow/seqera/02_run_with_launchpad.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Next we want to start using Seqera Platform to launch Nextflow workflows on our

!!! tip "Trainer Tip"

Launch a test run of the nf-core/rnaseq pipeline in the community/showcase workspace prior to starting this session, so you'll have a recent run for participants to inspect.
Launch a test run of the seqeralabs/nf-hello-genomics pipeline in the community/showcase workspace prior to starting this session, so you'll have a recent run for participants to inspect.

### 2.1. Navigate to the community/showcase workspace

Expand All @@ -21,18 +21,18 @@ Each user has an allotted amount of free compute to use in this workspace.
Click on your username in the top left corner of the screen to bring up the list of organizations and workspaces you have access to.
Select the `community/showcase` workspace.

### 2.2. Launch a test run of nf-core/rnaseq pipeline
### 2.2. Launch a test run of seqeralabs/nf-hello-genomics pipeline

In the `community/showcase` workspace, you will see a list of pipelines that have been set up by the workspace owner for you to run.
Follow these steps to launch a test run of a pipeline:

![Launchpad](seqera/img/launchpad.gif)

1. Find the `nf-core-rnaseq` pipeline in the list of pipelines.
1. Find the `nf-hello-genomics` pipeline in the list of pipelines (this is the same workflow you may have encountered in the [Hello Genomics](../06_hello_config.md) module).
2. Click on the `Launch` button to bring up the launch form.
3. Change the "Workflow run name" to "<username>-rnaseq-test".
3. Change the "Workflow run name" to "<username>-hellogenomics-test".
4. Click "Next" to bring up the parameters form.
5. Find the `trimmer` parameter and change it to `fastp`.
5. Take a look at the options. Feel free to take a moment to understand them, referring to [Hello Genomics](../06_hello_config.md) as needed.
6. Click on "Next" to inspect the advanced configuration.
7. Click "Launch" to start the pipeline!

Expand All @@ -54,7 +54,7 @@ It may take some time for the pipeline to start running while AWS Batch spins up
### 2.4. Inspect a pipeline run

Scroll down to find the list of tasks that were executed in the pipeline run.
For example, by searching for `fastq` we can find the task `NFCORE_RNASEQ:RNASEQ:FASTQ_FASTQC_UMITOOLS_FASTP:FASTP (WT_REP2)` that was executed as part of the pipeline.
For example, by searching for `GATK_JOINTGENOTYPING` we can find the main genotyping process that was executed as part of the pipeline.

Click on the task to see the task details:

Expand Down
Binary file modified docs/hello_nextflow/seqera/img/resolved_configuration.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/hello_nextflow/seqera/img/run_with_tower.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/hello_nextflow/seqera/img/task_details.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading