From 3bc4e9a2430c6b50941770f411f4c279a98f3026 Mon Sep 17 00:00:00 2001
From: Ross Lawrence <52179159+Lawreros@users.noreply.github.com>
Date: Fri, 25 Feb 2022 14:45:53 -0500
Subject: [PATCH] update docker image (#410)

* badge update

* badge test

* func output tree overhaul

* concurant func and dwi pipelines

* sort functional and diffusion output folder organization

* m2g-f file output fix

* remove iteration output

* update for travis test

* readme update

* readme update

* remove travis

* update file output

* pass push check

* Docker file overhaul, switch base image to neurodebian: bionic-non-free

* initial stable dockerfile

* first round of version control for dockerfile

* change cpac runner to python3.7

* fix missing module issue by making them visible to python3, also change installation order

* vix missing xvfbwrapper

* instruction update

* update

* README update

* finish file definition

* README rough draft

* remove lfs requirement
---
 Dockerfile |  29 ++---
 README.md  | 348 ++++++++++++++++++++++++++---------------------------
 2 files changed, 181 insertions(+), 196 deletions(-)

diff --git a/Dockerfile b/Dockerfile
index f26211b2..effae08f 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -32,20 +32,12 @@ RUN apt-get update && \
 RUN pip3.7 install --upgrade pip
 
 
-# Get neurodebian config
-# RUN curl -sSL http://neuro.debian.net/lists/stretch.us-tn.full >> /etc/apt/sources.list.d/neurodebian.sources.list && \
-#     apt-key add /root/.neurodebian.gpg && \
-#     (apt-key adv --refresh-keys --keyserver hkp://ha.pool.sks-keyservers.net 0xA5D32F012649A5A9 || true) && \
-#     apt-get update -qq
-# RUN apt-get -f install
-
-
 
 # Configure git-lfs
-RUN apt-get install -y apt-transport-https debian-archive-keyring
-RUN curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | bash && \
-    apt-get update && \
-    apt-get install -y git-lfs
+# RUN apt-get install -y apt-transport-https debian-archive-keyring
+# RUN curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | bash && \
+#     apt-get update && \
+#     apt-get install -y git-lfs
 
 # #---------AFNI INSTALL--------------------------------------------------------#
 # # setup of AFNI, which provides robust modifications of many of neuroimaging algorithms
@@ -197,27 +189,22 @@ RUN apt-get update && apt-get -y upgrade insighttoolkit4-python
 # setup of python dependencies for m2g itself, as well as file dependencies
 RUN \
     pip3.7 install --no-cache-dir hyppo==0.1.3 nibabel==2.3.3 scipy==1.4.1 python-dateutil pandas==0.23.4 boto3==1.7.13 awscli==1.15.40 virtualenv
+
 RUN \
     pip3.7 install --no-cache-dir matplotlib nilearn sklearn cython vtk pyvtk fury==0.5.1
 RUN \
     pip3.7 install --no-cache-dir yamlordereddictloader
-#got rid of awscli here^
 RUN \
+
     pip3.7 install --no-cache-dir requests ipython duecredit graspy==0.3.0 scikit-image networkx dipy==1.1.1 pybids==0.12.0
 
 # TODO: Update pybids
 RUN \
     pip3.7 install --no-cache-dir plotly==1.12.9 configparser>=3.7.4 regex
 
-    
-RUN \
-    pip3.7 install s3transfer==0.3.7
-
-RUN \
-    pip3.7 install setuptools==57.5.0
 
 RUN \
-    pip3.7 install numba==0.52.0
+    pip3.7 install s3transfer==0.3.7 setuptools==57.5.0 numba==0.52.0
 
 # install ICA-AROMA
 RUN mkdir -p /opt/ICA-AROMA
@@ -276,7 +263,7 @@ RUN pip3.7 install git+https://github.com/ChildMindInstitute/PyPEER.git
 RUN mkdir /m2g_atlases
 
 RUN \
-    git clone https://github.com/neurodata/neuroparc -b remove-lfs && \
+    git clone https://github.com/neurodata/neuroparc && \
     mv /neuroparc/atlases /m2g_atlases && \
     rm -rf /neuroparc
 RUN chmod -R 777 /m2g_atlases
diff --git a/README.md b/README.md
index f8d3acdb..808b5382 100644
--- a/README.md
+++ b/README.md
@@ -11,7 +11,7 @@
 
 NeuroData's MR Graphs package, **m2g**, is a turn-key pipeline which uses structural and diffusion MRI data to estimate multi-resolution connectomes reliably and scalably.
 
-## Contents
+# Contents
 
 - [Overview](#overview)
 - [System Requirements](#system-requirements)
@@ -27,47 +27,28 @@ NeuroData's MR Graphs package, **m2g**, is a turn-key pipeline which uses struct
 - [Manuscript Reproduction](#manuscript-reproduction)
 - [Issues](#issues)
 
-## Overview
+# Overview
 
-The **m2g** pipeline has been developed as a one-click solution for human connectome estimation by providing robust and reliable estimates of connectivity across a wide range of datasets. The pipelines are explained and derivatives analyzed in our pre-print, available on [BiorXiv](https://www.biorxiv.org/content/early/2017/09/16/188706).
+The **m2g** pipeline has been developed as a beginner-friendly solution for human connectome estimation by providing robust and reliable estimates of connectivity across a wide range of datasets. The pipelines are explained and derivatives analyzed in our pre-print, available on [BiorXiv](https://www.biorxiv.org/content/early/2017/09/16/188706).
 
-## System Requirements
+# System Requirements
 
 The **m2g** pipeline:
 
-- was developed and tested primarily on Mac OSX, Ubuntu (14, 16, 18, 20), and CentOS (5, 6);
-- made to work on Python 3.6;
+- was developed and tested primarily on Mac OSX, Ubuntu (16, 18, 20), and CentOS (5, 6);
+- made to work on Python 3.7;
 - is wrapped in a [Docker container](https://hub.docker.com/r/neurodata/m2g);
 - has install instructions via a Dockerfile;
 - requires no non-standard hardware to run;
 - has key features built upon FSL, AFNI, Dipy, Nibabel, Nilearn, Networkx, Numpy, Scipy, Scikit-Learn, and others;
 - takes approximately 1-core, < 16-GB of RAM, and 1 hour to run for most datasets.
 
-## Installation Guide
-
-**m2g** relies on [FSL](http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FslInstallation), [AFNI](https://afni.nimh.nih.gov/), [Dipy](http://nipy.org/dipy/), [networkx](https://networkx.github.io/), and [nibabel](http://nipy.org/nibabel/), [numpy](http://www.numpy.org/) [scipy](http://www.scipy.org/), [scikit-learn](http://scikit-learn.org/stable/), [scikit-image](http://scikit-image.org/), [nilearn](http://nilearn.github.io/). You should install FSL and AFNI through the instructions on their website, then install other Python dependencies as well as the package itself with the following:
-
-    git clone https://github.com/neurodata/m2g.git
-    cd m2g
-    pip install -r requirements.txt
-    pip install .
-
-For the most up-to-date version of m2g, you can use the staging branch. This version is updated regularly, but may be less stable:
-
-    git clone https://github.com/neurodata/m2g.git
-    cd m2g
-    git checkout staging
-    pip install -r requirements.txt
-    pip install .
-
-You can also install **m2g** from `pip` as shown below. Installation shouldn't take more than a few minutes, but depends on your internet connection. Note that to install the most up-to-date version of the pipeline, we currently recommend installing from github.
-
-### Install from pip
-
-    pip install m2g
+# Installation Guide
 
 ## Docker
 
+While you can install **m2g** from `pip` using the command `pip install m2g`, as there are several dependencies needed for both **m2g** and **CPAC**, it is highly recommended to use **m2g** through a docker container:
+
 **m2g** is available through Dockerhub, and the most recent docker image can be pulled using:
 
     docker pull neurodata/m2g:latest
@@ -88,26 +69,71 @@ In order to create a docker container from the docker image and access it, use t
 
     docker run -it --entrypoint /bin/bash m2g:uniquelabel
 
-## Tutorial
+## Local Installation [COMING SOON]
+Due to the versioning required for CPAC, along with `m2g-d`, we are currently working on streamlining the installation of `m2g`. Stay tuned for updates.
+
+
+
+# Usage
+
+The **m2g** pipeline can be used to generate connectomes as a command-line utility on [BIDS datasets](http://bids.neuroimaging.io) with the following:
+
+    m2g --pipeline <pipe> /input/bids/dataset /output/directory
+
+Note that more options are available which can be helpful if running on the Amazon cloud, which can be found and documented by running `m2g -h`.
+
+
+## Docker Container Usage
+
+If running with the Docker container shown above, the `entrypoint` is already set to `m2g`, so the pipeline can be run directly from the host-system command line as follows:
+
+    docker run -ti -v /path/to/local/data:/data neurodata/m2g /data/ /data/outputs
+
+This will run **m2g** on the local data and save the output files to the directory /path/to/local/data/outputs. Note that if you have created the docker image from github, replace `neurodata/m2g` with `imagename:uniquelabel`.
+
+Also note that currently, running `m2g` on a single bids-formatted dataset directory only runs a single scan. To run the entire dataset, we recommend parallelizing on a high-performance cluster or using `m2g`'s s3 integration.
+
+
+# Tutorial
+
+## Structural Connectome Pipeline (`m2g-d`)
 
-Once you have the pipeline up and running, you can run it with:
+Once you have the pipeline up and running, you can run the structural connectome pipeline with:
   
- m2g <input_directory> <output_directory>
+    m2g --pipeline dwi <input_directory> <output_directory>
   
 We recommend specifying an atlas and lowering the default seed density on test runs (although, for real runs, we recommend using the default seeding -- lowering seeding simply decreases runtime):
 
-    m2g --seeds 1 --parcellation desikan <input_directory> <output_directory>
+    m2g --pipeline dwi --seeds 1 --parcellation Desikan <input_directory> <output_directory>
 
 You can set a particular scan and session as well (recommended for batch scripts):
 
-    m2g --seeds 1 --parcellation desikan --participant_label <label> --session_label <label> <input_directory> <output_directory>
+    m2g --pipeline dwi --seeds 1 --parcellation Desikan --participant_label <label> --session_label <label> <input_directory> <output_directory>
+
+## Functional Connectome Pipeline (`m2g-f`)
+
+Once you have the pipeline up and running, you can run the functional connectome pipeline with:
+  
+    m2g --pipeline func <input_directory> <output_directory>
+  
+We recommend specifying an atlas and lowering the default seed density on test runs (although, for real runs, we recommend using the default seeding -- lowering seeding simply decreases runtime):
 
-For more detailed instructions, tutorials on the **m2g** pipeline can be found in [m2g/tutorials](https://github.com/neurodata/m2g/tree/staging/tutorials)
+    m2g --pipeline func --seeds 1 --parcellation Desikan <input_directory> <output_directory>
+
+You can set a particular scan and session as well (recommended for batch scripts):
 
-## Outputs
+    m2g --pipeline func --seeds 1 --parcellation Desikan --participant_label <label> --session_label <label> <input_directory> <output_directory>
 
-### Diffusion Pipeline
-The organization of the output files generated by the m2g-d pipeline are shown below. If you only care about the connectome edgelists (**m2g**'s fundamental output), you can find them in `/output/dwi/roi-connectomes`.
+
+## Running both `m2g-d` and `m2g-f`
+Both pipelines can be run by setting the `pipeline` parameter to `both`:
+    
+    m2g --pipeline both <input_directory> <output_directory>
+
+# Outputs
+
+## Diffusion Pipeline
+The organization of the output files generated by the m2g-d pipeline are shown below. If you only care about the connectome edgelists (**m2g**'s fundamental output), you can find them in `/output/connectomes_d`.
 
 ```
 File labels that may appear on output files, these denote additional actions m2g may have done:
@@ -117,35 +143,34 @@ nores = File originally had the desired voxel size specified by the user (defaul
 res = The file has been resliced to the desired voxel size specified by the user
 
 /output
-     /anat_f
-          /preproc
-               Files created during the preprocessing of the anatomical data
-                   t1w_aligned_mni.nii.gz = preprocessed t1w_brain anatomical image in mni space
-                   t1w_brain.nii.gz = t1w anatomical image with only the brain
-                   t1w_seg_mixeltype.nii.gz = mixeltype image of t1w image (denotes where there are more than one tissue type in each voxel)
-                   t1w_seg_pve_0.nii.gz = probability map of Cerebrospinal fluid for original t1w image
-                   t1w_seg_pve_1.nii.gz = probability map of grey matter for original t1w image
-                   t1w_seg_pve_2.nii.gz = probability map of white matter for original t1w image
-                   t1w_seg_pveseg.nii.gz = t1w image mapping wm, gm, ventricle, and csf areas
-                   t1w_wm_thr.nii.gz = binary white matter mask for resliced t1w image
-
-          /registered
-               Files created during the registration process
-                   t1w_corpuscallosum.nii.gz = atlas corpus callosum mask in t1w space
-                   t1w_corpuscallosum_dwi.nii.gz = atlas corpus callosum in dwi space
-                   t1w_csf_mask_dwi.nii.gz = t1w csf mask in dwi space
-                   t1w_gm_in_dwi.nii.gz = t1w grey matter probability map in dwi space
-                   t1w_in_dwi.nii.gz = t1w in dwi space
-                   t1w_wm_gm_int_in_dwi.nii.gz = t1w white matter-grey matter interfact in dwi space
-                   t1w_wm_gm_int_in_dwi_bin.nii.gz = binary mask of t12_2m_gm_int_in_dwi.nii.gz
-                   t1w_wm_in_dwi.nii.gz = atlas white matter probability map in dwi space
-
-    /dwi_f
+    /anat_d
+        
+        /preproc
+            t1w_aligned_mni.nii.gz = preprocessed t1w_brain anatomical image in mni space
+            t1w_brain.nii.gz = t1w anatomical image with only the brain
+            t1w_seg_mixeltype.nii.gz = mixeltype image of t1w image (denotes where there are more than one tissue type in each voxel)
+            t1w_seg_pve_0.nii.gz = probability map of Cerebrospinal fluid for original t1w image
+            t1w_seg_pve_1.nii.gz = probability map of grey matter for original t1w image
+            t1w_seg_pve_2.nii.gz = probability map of white matter for original t1w image
+            t1w_seg_pveseg.nii.gz = t1w image mapping wm, gm, ventricle, and csf areas
+            t1w_wm_thr.nii.gz = binary white matter mask for resliced t1w image
+
+        /registered
+            t1w_corpuscallosum.nii.gz = atlas corpus callosum mask in t1w space
+            t1w_corpuscallosum_dwi.nii.gz = atlas corpus callosum in dwi space
+            t1w_csf_mask_dwi.nii.gz = t1w csf mask in dwi space
+            t1w_gm_in_dwi.nii.gz = t1w grey matter probability map in dwi space
+            t1w_in_dwi.nii.gz = t1w in dwi space
+            t1w_wm_gm_int_in_dwi.nii.gz = t1w white matter-grey matter interfact in dwi space
+            t1w_wm_gm_int_in_dwi_bin.nii.gz = binary mask of t12_2m_gm_int_in_dwi.nii.gz
+            t1w_wm_in_dwi.nii.gz = atlas white matter probability map in dwi space
+
+    /dwi
         /fiber
             Streamline track file(s)
-        /preproc
-            Files created during the preprocessing of the dwi data
-                #_B0.nii.gz = B0 image (there can be multiple B0 images per dwi file, # is the numerical location of each B0 image)
+
+        /preproc    (files created during the preprocessing of the dwi data)
+            #_B0.nii.gz = B0 image (there can be multiple B0 images per dwi file, # is the numerical location of each B0 image)
             bval.bval = original b-values for dwi image
             bvec.bvec = original b-vectors for dwi image
             bvecs_reor.bvecs = bvec_scaled.bvec data reoriented to RAS+ orientation
@@ -158,22 +183,31 @@ res = The file has been resliced to the desired voxel size specified by the user
             nodif_B0_bet.nii.gz = nodif_B0 image with all non-brain matter removed
             nodif_B0_bet_mask.nii.gz = mask of nodif_B0_bet.nii.gz brain
             tensor_fa.nii.gz = tensor image fractional anisotropy map
-        /roi-connectomes
-            Location of connectome(s) created by the pipeline, with a directory given to each atlas you use for your analysis
+        
         /tensor
             Contains the rgb tensor file(s) for the dwi data if tractography is being done in MNI space
+
+    /connectomes_d
+            Location of connectome(s) created by the pipeline, with a directory given to each atlas you use for your analysis
+
     /qa_d
-        /adjacency
-        /fibers
-        /graphs
         /graphs_plotting
             Png file of an adjacency matrix made from the connectome
-        /mri
         /reg
-        /tensor
+            <atlas>_atlas_2_nodif_B0_bet.png = overlay of registered atlas on top of anatomical image
+            qa_fast.png = overlay of white/grey matter and csf regions on top of anatomical image
+            t1w_aligned_mni_2_MNI152_T1_<vox>_brain.png = overlay of registered anatomical image on top of MNI152 anatomical reference image
+            t1w_corpuscallosum_dwi_2_nodif_B0_bet.png = corpus callosum region highlighted on registered anatomical image
+            t1w_csf_mask_dwi_2_nodif_B0_bet.png = overlay of csf mask on top of registered anatomical image
+            t1w_gm_in_dwi_2_nodif_B0_bet.png = overlay of grey matter mask on top of registered anatomical image
+            t1w_in_dwi_2_nodif_B0_bet.png = overlay of dwi image on top of anatomical image registered to dwi space
+            t1w_vent_mask_dwi_2_nodif_B0_bet.png = display of ventrical masks
+            t1w_wm_in_dwi_2_nodif_B0_bet.png = overlay of white matter mask on top of registered anatomical image
+        /skull_strip
+            qa_skullstrip__<sub>_<ses>_T1w_reor_RAS_res.png = overlay of skullstripped anatomical image on top of original anatomical image
+    
     /tmp_d
-        /reg_a
-            Intermediate files created during the processing of the anatomical data
+        /reg_a (Intermediate files created during the processing of the anatomical data)
             mni2t1w_warp.nii.gz = nonlinear warp coefficients/fields for mni to t1w space
             t1w_csf_mask_dwi_bin.nii.gz = binary mask of t1w_csf_mask_dwi.nii.gz
             t1w_gm_in_dwi_bin.nii.gz = binary mask of t12_gm_in_dwi.nii.gz
@@ -185,8 +219,7 @@ res = The file has been resliced to the desired voxel size specified by the user
             vent_mask_t1w.nii.gz = atlas ventricle mask in t1w space
             warp_t12mni.nii.gz = nonlinear warp coefficients/fields for t1w to mni space
 
-        /reg_m
-            Intermediate files created during the processing of the diffusion data
+        /reg_m (Intermediate files created during the processing of the diffusion data)
             dwi2t1w_bbr_xfm.mat = affine transform matrix of t1w_wm_edge.nii.gz to t1w space
             dwi2t1w_xfm.mat = inverse transform matrix of t1w2dwi_xfm.mat
             roi_2_mni.mat = affine transform matrix of selected atlas to mni space
@@ -197,10 +230,8 @@ res = The file has been resliced to the desired voxel size specified by the user
             xfm_t1w2mni_init.mat = affine transform matrix of preprocessed t1w_brain to mni space
 ```
 
-Other files may end up in the output folders, depending on what settings or atlases you choose to use. Using MNI space for tractography or setting `--clean` to `True` will result in fewer files.
-
 ## Functional Pipeline
-The organization of the output files generated by the m2g-f pipeline are shown below. If you only care about the connectome edgelists (**m2g**'s fundamental output), you can find them in `/output/dwi/roi-connectomes`.
+The organization of the output files generated by the m2g-f pipeline are shown below. If you only care about the connectome edgelists (**m2g**'s fundamental output), you can find them in `/output/connectomes_f`.
 
 
 ```
@@ -213,171 +244,140 @@ res = The file has been resliced to the desired voxel size specified by the user
 /output
     /anat_f
         /anatomical_brain
-            <subject>_<session>_T1w_resample_calc.nii.gz
+            <subject>_<session>_T1w_resample_calc.nii.gz = resampled and skullstripped brain from anatomical image
         /anatomical_brain_mask
-            <subject>_<session>_T1w_resample_skullstrip_calc.nii.gz
+            <subject>_<session>_T1w_resample_skullstrip_calc.nii.gz = mask of resampled and skullstripped brain from anatomical image
         /anatomical_csf_mask
-            segment_seg_0_maths_maths.nii.gz
+            segment_seg_0_maths_maths.nii.gz = mask of csf area of anatomical image
         /anatomical_gm_mask
-            segment_seg_1_maths_maths.nii.gz
+            segment_seg_1_maths_maths.nii.gz = mask of grey matter area of anatomical image
         /anatomical_reorient
-            <subject>_<session>_T1w_resample.nii.gz
+            <subject>_<session>_T1w_resample.nii.gz = reorientated and resampled anatomical image
         /anatomical_to_mni_nonlinear_xfm
-            <subject>_<session>_T1w_resample_fieldwarp.nii.gz
+            <subject>_<session>_T1w_resample_fieldwarp.nii.gz = fieldwarp for registering the anatomical image to MNI space
         /anatomical_to_standard
-            <subject>_<session>_T1w_resample_calc_warp.nii.gz
+            <subject>_<session>_T1w_resample_calc_warp.nii.gz = anatomical image registered to MNI space
         /anatomical_wm_mask
-            segment_seg_2_maths_maths.nii.gz
+            segment_seg_2_maths_maths.nii.gz = mask of white matter area of anatomical image
         /seg_mixeltype
-            segment_mixeltype.nii.gz
+            segment_mixeltype.nii.gz = mixeltype representation of anatomical image
         /seg_partial_volume_files
-            segment_pve_0.nii.gz
-            segment_pve_1.nii.gz
-            segment_pve_2.nii.gz
+            segment_pve_0.nii.gz = mask of grey matter regions of anatomical image
+            segment_pve_1.nii.gz = mask of grey matter/white matter boundary of anatomical image
+            segment_pve_2.nii.gz = mask of white matter regions of anatomical image
         /seg_partial_volume_map
-            segment_pveseg.nii.gz
+            segment_pveseg.nii.gz = partial volume map of anatomical image
         /seg_probability_maps
-            segment_prob_0.nii.gz
-            segment_prob_1.nii.gz
-            segment_prob_2.nii.gz
+            segment_prob_0.nii.gz = probability map of anatomical image for grey matter
+            segment_prob_1.nii.gz = probability map of grey/white matter boundary
+            segment_prob_2.nii.gz = probability map of anatomical image for white matter
     /connectomes_f
         /<atlas>
+            <sub>_<ses>_func_<atlas>_abs_edgelist.csv = edgelist file for estimated connectome where the absolute value of the correlation is given
+            <sub>_<ses>_func_<atlas>_edgelist.csv = edgelist file for estimated connectome
     /func
         /preproc
             /coordinate_transformation
-                <subject>_<session>_task-rest_bold_calc_tshift_resample.aff12.1D
+                <subject>_<session>_task-rest_bold_calc_tshift_resample.aff12.1D = 
             /frame_wise_displacement_jenkinson
-                FD_J.1D
+                FD_J.1D = vector containing Jenkinson measurement of framewise displacement for each frame of the functional image file
             /frame_wise_displacement_power
-                FD.1D
+                FD.1D = vector containing power of framewise displacement for each frame of the functional image file
             /functional_brain_mask
-                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_mask.nii.gz
+                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_mask.nii.gz = brain mask for the functional image
             /functional_brain_mask_to_standard
-                <subject>_<session>_task-rest_bold_calc_tshift_rasample_volreg_mask_warp.nii.gz
+                <subject>_<session>_task-rest_bold_calc_tshift_rasample_volreg_mask_warp.nii.gz = functional brain mask registered to MNI152 space
             /functional_freq_filtered
-                bandpassed_demeaned_filtered.nii.gz
+                bandpassed_demeaned_filtered.nii.gz = frequency filtered functional file
             /functional_nuisance_regressors
                 nuisance_regressors.1D
             /functional_nuisance_residuals
                 residuals.nii.gz
             /functional_preprocessed
-                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_calc_maths.nii.gz
+                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_calc_maths.nii.gz = skullstripped brain from motion corrected functional image file resampled to voxel dimensions specified by user
             /functional_preprocessed_mask
-                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_calc_maths_maths.nii.gz
+                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_calc_maths_maths.nii.gz = mask for image contained in /functional_preprocessed
             /motion_correct
-                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg.nii.gz
+                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg.nii.gz = motion corrected functional image file resampled to voxel dimensions specified by user
             /motion_correct_to_standard_smooth
                 /_fwhm_4
                     <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_warp_maths.nii.gz
             /motion_params
-                motion_parameters.txt
+                motion_parameters.txt = statistical measurements of motion correction performed on functional image
             /raw_functional
-                <subject>_<session>_task-rest_bold.nii.gz
+                <subject>_<session>_task-rest_bold.nii.gz = unaltered input functional image
             /slice_time_corrected
-                <subject>_<session>_task-rest_bold_calc_tshift.nii.gz
+                <subject>_<session>_task-rest_bold_calc_tshift.nii.gz = slice time corrected functional image
         /register
             /functional_to_anat_linear_xfm
-                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_calc_tstat_flirt.mat
+                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_calc_tstat_flirt.mat = 
             /functional_to_standard
-                bandpassed_demeaned_filtered_warp.nii.gz
+                bandpassed_demeaned_filtered_warp.nii.gz = bandpassed and demeaned filtered warp map for registering the functional image to MNI space
             /max_displacement
                 max_displacement.1D
             /mean_functional
-                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_calc_tstat.nii.gz
+                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_calc_tstat.nii.gz = mean functional image from all volumes
             /mean_functional_in_anat
-                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_calc_tstat_flirt.nii.gz
+                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_calc_tstat_flirt.nii.gz = mean functional image registered to the anatomical image
             /mean_functional_to_standard
-                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_calc_tstat_warp.nii.gz
+                <subject>_<session>_task-rest_bold_calc_tshift_resample_volreg_calc_tstat_warp.nii.gz = mean functional image registered to MNI space
             /movement_parameters
-                <subject>_<session>_task-rest_bold_calc_tshift_resample.1D
+                <subject>_<session>_task-rest_bold_calc_tshift_resample.1D = movement parameters applied to each volumen of functional image
             /power_params
-                pow_params.txt
+                pow_params.txt = different measurements on the power of functional images
             /roi_timeseries
                 /<atlas>
-                    roi_stats.csv
-                    roi_stats.npz
+                    roi_stats.csv = mean voxel intensity for each region of interest at each time point, used to calculate functional connectome
+                    roi_stats.npz = pickeled version of roi_stats.csv
 
     /log_f
-        callback.log
-        cpac_data_config_<date>.yml
-        cpac_individual_timing_m2g.csf
-        cpac_pipeline_config_<date>.yml
-        functional_pipeline_settings.yaml
-        pypeline.lock
-        pypeline.log
-        subject_info_<subject>_<session>.pkl
+        callback.log = nipype log for modules made for pipeline
+        cpac_data_config_<date>.yml = file containing functional and anatomical image directory location
+        cpac_individual_timing_m2g.csv = record of the elapsed time from the run of m2g-f
+        cpac_pipeline_config_<date>.yml = copy of CPAC configuration file
+        functional_pipeline_settings.yaml = record of CPACP pipeline parameter settings
+        pypeline.lock = intermediate file created for pipeline running
+        pypeline.log = nipype log with record of everything printed to terminal
+        subject_info_<subject>_<session>.pkl = pickle file of functional and anatomical file information
 
     /qa_f
         /carpet
             carpet_seg.png
         /csf_gm_wm_a
-            montage_csf_gm_wm_a.png
+            montage_csf_gm_wm_a.png = axial view of mask of csf/grey matter/white matter regions overlaid on top of anatomical image
         /csf_gm_wm_s
-            montage_csf_gm_wm_s.png
+            montage_csf_gm_wm_s.png = sagittal view of mask of csf/grey matter/white matter regions overlaid on top of anatomical image
         /mean_func_with_mni_edge_a
-            MNI_edge_on_mean_func_mni_a.png
+            MNI_edge_on_mean_func_mni_a.png = axial view of outline of MNI reference anatimical image overlaid on top of mean functional image
         /mean_func_with_mni_edge_s
-            MNI_edge_on_mean_func_mni_s.png
+            MNI_edge_on_mean_func_mni_s.png = sagittal view of outline of MNI reference anatimical image overlaid on top of mean functional image
         /mean_func_with_t1_edge_a
-            t1_edge_on_mean_func_in_t1_a.png
+            t1_edge_on_mean_func_in_t1_a.png = axial view of outline of anatomical image overlaid on top of mean functional image registered to the anatomical image
         /mean_func_with_t1_edge_s
-            t1_edge_on_mean_func_in_t1_s.png
+            t1_edge_on_mean_func_in_t1_s.png = sagittal view of outline of anatomical image overlaid on top of mean functional image registered to the anatomical image
         /mni_normalized_anatomical_a
-            mni_anat_a.png
+            mni_anat_a.png = axial view of anatomical image registered to MNI image
         /mni_normalized_anatomical_s
-            mni_anat_s.png
+            mni_anat_s.png = sagittal view of anatomical image registered to MNI image
         /movement_rot_plot
-            motion_rot_plot.png
+            motion_rot_plot.png = movement rotation plot for rotation correction of functional image
         /movement_trans_plot
-            motion_trans_plot.png
+            motion_trans_plot.png = movement translational plot for translation correction of functional image
         /skullstrip_vis_a
-            skull_vis_a.png
+            skull_vis_a.png = axial view of original anatomical image overlaid on top of skullstripped anatomical image
         /skullstrip_vis_s
-            skull_vis_s.png
+            skull_vis_s.png = sagittal view of original anatomical image overlaid on top of skullstripped anatomical imag
         /snr_a
-            snr_a.png
+            snr_a.png = axial view of signal to noise ratio for functional image
         /snr_hist
-            snr_hist_plot.png
+            snr_hist_plot.png = signal to noise ratio intensity plot
         /snr_s
-            snr_s.png
+            snr_s.png = sagittal view of signal to noise ratio for functional image
         /snr_val
-            average_snr_file.txt
-    
-          
+            average_snr_file.txt = single value of average signal to noise ratio for functional image
+              
 ```
 
-## Usage
-
-The **m2g** pipeline can be used to generate connectomes as a command-line utility on [BIDS datasets](http://bids.neuroimaging.io) with the following:
-
-    m2g /input/bids/dataset /output/directory
-
-Note that more options are available which can be helpful if running on the Amazon cloud, which can be found and documented by running `m2g -h`.
-
-### Diffusion Pipeline
-The **m2g-d** pipeline...
-
-```
-m2g dwi /input/bids/dataset /output/directory
-```
-
-### Functional Pipeline
-The **m2g-f** pipeline...
-
-```
-m2g func /input/bids/dataset /output/directory
-```
-
-### Docker Container Usage
-
-If running with the Docker container shown above, the `entrypoint` is already set to `m2g`, so the pipeline can be run directly from the host-system command line as follows:
-
-    docker run -ti -v /path/to/local/data:/data neurodata/m2g /data/ /data/outputs
-
-This will run **m2g** on the local data and save the output files to the directory /path/to/local/data/outputs. Note that if you have created the docker image from github, replace `neurodata/m2g` with `imagename:uniquelabel`.
-
-Also note that currently, running `m2g` on a single bids-formatted dataset directory only runs a single scan. To run the entire dataset, we recommend parallelizing on a high-performance cluster or using `m2g`'s s3 integration.
-
 ## Working with S3 Datasets
 
 **m2g** has the ability to work on datasets stored on [Amazon's Simple Storage Service](https://aws.amazon.com/s3/), assuming they are in BIDS format. Doing so requires you to set your AWS credentials and read the related s3 bucket documentation. You can find a guide [here](https://github.com/neurodata/m2g/blob/deploy/tutorials/Batch.ipynb).
@@ -386,8 +386,6 @@ Also note that currently, running `m2g` on a single bids-formatted dataset direc
 
 Derivatives have been produced on a variety of datasets, all of which are made available on [our website](http://m2g.io). Each of these datsets is available for access and download from their respective sources. Alternatively, example datasets on the [BIDS website](http://bids.neuroimaging.io) which contain diffusion data can be used and have been tested; `ds114`, for example.
 
-For some downsampled test data, see [neuroparc](https://github.com/neurodata/neuroparc/tree/master/data/)
-
 ## Documentation
 
 Check out some [resources](http://m2g.io) on our website, or our [function reference](https://ndmg.neurodata.io/) for more information about **m2g**.