rnaseq_cgl_pipeline.py check that unzip is installed

[Jeltje]

It appears that the code does not check for the unzip program, which leads to a fairly incomprehensible No such file or directory error from mapsplice

[jvivian]

This pipeline does not use unzip or mapsplice.

[Jeltje]

Well, it crashes with that error.

[jvivian]

Can you post a stacktrace?

[Jeltje]

Yeah of course, I'll rerun it. Had to terminate the cluster last night because it turns out you can't stop a spot cluster...

[Jeltje]

Can't reproduce the error!
However, the pipeline now runs in full with no WARNING or ERROR messages, but the expected output file (SRR1559172.tar.gz) does not appear in s3:cgl-driver-projects/test/rna-test

Log file here

[jvivian]

Can you post the contents of the launch script?

[Jeltje]

export PYTHONPATH=$(python -c 'from os.path import abspath as a, dirname as d;import sys;print d(d(d(a(sys.argv[1]))))' $0)
python -m toil_scripts.rnaseq_unc.rnaseq_unc_pipeline \
aws:us-west-2:unc-pipeline-run-1 \
--retryCount 1 \
--config toil_rnaseq_config.csv \
--ssec "/home/mesosbox/shared/master.key" \
--output_dir "/home/mesosbox/rnaseq_output" \
--s3_dir "cgl-driver-projects/test/rna-test/" \
--sseKey=/home/mesosbox/shared/master.key \
--batchSystem="mesos" \
--mesosMaster mesos-master:5050 \
--workDir=/var/lib/toil \
#--restart

I do realize I should have probably removed the output_dir.

[jvivian]

Any chance if you know if the output showed up in the output dir?

[Jeltje]

I did not create a /home/mesosbox/rnaseq_output and there is none on the master after the run. But also no error about that.

[jvivian]

It would've been on the worker. I'm running this pipeline (and the UNC one) as part of a test, so I'll see if i can replicate.

[Jeltje]

okay I can have a look there

[Jeltje]

Nope, not on either worker.

[jvivian]

Ok, thanks for checking.

[jvivian]

Could not reproduce. My run just finished and my file showed up in S3.

s3://cgl-driver-projects/test/test_SRR1559177_Sample_NB2318tumor1_T_C3WFVACXX.tar.gz

#!/usr/bin/env bash
# John Vivian
#
# Please read the associated README.md before attempting to use.
#
# Execution of pipeline
export PYTHONPATH=$(python -c 'from os.path import abspath as a, dirname as d;import sys;print d(d(d(a(sys.argv[1]))))' $0)
python -m toil_scripts.rnaseq_cgl.rnaseq_cgl_pipeline \
aws:us-west-2:jvivian-releases-run-1 \
--config /home/mesosbox/shared/config.txt \
--retryCount 1 \
--ssec /home/mesosbox/shared/master.key \
--s3_dir cgl-driver-projects/test/ \
--sseKey=/home/mesosbox/shared/master.key \
--batchSystem="mesos" \
--mesosMaster mesos-master:5050 \
--workDir=/var/lib/toil \
#--restart

[hannes-ucsc]

I see unzip mentioned here. What's the relationship between this issue and BD2KGenomics/cgcloud#134? Is the latter invalid? What triggered Jeltje to believe that a missing unzip was the culprit?

[jvivian]

What's the relationship between this issue and BD2KGenomics/cgcloud#134?

Unzip is needed for the UNC rna-seq pipeline, not this one.

What triggered Jeltje to believe that a missing unzip was the culprit?

An error that she wasn't able to reproduce.