how to set up parallel processing for modelbuild.sh #68
-
|
Hi first thanks so much for the package! It works great. I have a question about setting up parallel processing. Usually in ANTs scripts this is controlled by -c and -j parameters as well as the 'ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS' environment variable. I usually set it up such that it runs 4 parallel jobs (-j 4) with 7 threads per job. I see that the modelbuild.sh script uses the qbatch package which seems to rely on 'gnu-parallel' and I wondering how to set it up in a similar way. Thanks again. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Hi, You are correct that underlying the pipeline is the qbatch job submission system, which supports several clusters, as well as local parallelism. The qbatch documentation: Lists environment variable defaults, Which control the basic execution of qbatch in the absence of specific command-line options. Currently the only qbatch options exposed at the pipeline level are job walltimes which do not apply for local execution. All other settings are inherited from the exported environment variables. The registration script will inherit ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS if exported. |
Beta Was this translation helpful? Give feedback.
Hi,
You are correct that underlying the pipeline is the qbatch job submission system, which supports several clusters, as well as local parallelism.
The qbatch documentation:
https://github.com/pipitone/qbatch
Lists environment variable defaults,
https://github.com/pipitone/qbatch#environment-variable-defaults
Which control the basic execution of qbatch in the absence of specific command-line options. Currently the only qbatch options exposed at the pipeline level are job walltimes which do not apply for local execution. All other settings are inherited from the exported environment variables. The registration script will inherit ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS if exported.