Hi there,
I’m running Qunex within a singularity container, however it seems that the pipeline isn’t finding the correct files within opt/ of the container when I run the HCP pipeline. I can run some aspects (e.g. HCP1) by giving paths to the software on the local system.
Any advice would be much appreciated!
Best wishes,
George
error_hcp2_PSYC16023_Baseline_2020-11-23_10.59.1606129146.log (28.7 KB)
Hi George, we will assign a developer to this issue and will let you know as soon as we have a solution.
Hi George, could you please provide us with the full set of commands that resulted in the issue you are reporting. Please follow the Issue Reporting Guide, so our developers can more effectively find the solution to your issue.
Apologies, here is the command:
./qunexContainer runTurnkey
–container="${QUNEX_CONTAINER}"
–batchfile="/home/k1328096/qunex/processing/batch_psyscan_test.txt"
–mappingfile="/home/k1328096/qunex/PSYSCAN_test_qunex/sessions/PSYC16023_Baseline/session_hcp.txt"
–path="/home/k1328096/qunex/PSYSCAN_test_qunex/"
–sessions=“sessions”
–sessionids=“PSYC16023_Baseline”
–sessionsfoldername=“sessions”
–turnkeytype=“local”
–turnkeysteps=“hcp2, hcp3”
–projectname=“PSYSCAN_test_qunex”
–workingdir="/home/k1328096/qunex"
Com and run logs attached:
Log-hcp2-2020-11-23_10.59.1606129145.log (3.0 KB) error_hcp2_PSYC16023_Baseline_2020-11-23_10.59.1606129146.log (28.7 KB)
I’m wondering if it could be an issue with my useage of the singularity container rather than Qunex?
Hi George,
it seems like your system settings override container settings. As you can see from the error log (line 4) QuNex uses FreeSurfer script at /home/k1328096/HCPpipelines-master/FreeSurfer/FreeSurferPipeline.sh. Furthermore FREESURFER_HOME is set to /nan/ceph/network/system/el7/freesurfer/freesurfer-6.0.0. This is not OK, QuNex should use the internal scripts inside the container.
Please check your ~/.bash_profile and ~/.bashrc files and comment/remove any settings that might interfere with QuNex (especially in regards to HCPPipelines and FreeSurfer). The way Singularity works is that internal settings are overridden with these personal settings, if you have personal settings that override some important paths this can cause problems in QuNex.
Let me know how it goes.
Cheers, Jure
Hi Jure,
Thanks for your reply. I’ve removed any variables with paths to local software locations and checked the startup config file. I’ve checked with our system admin and there are no bind points that I can see that clash with the qunex container. I’m still unable to run the pipeline, please see the error log attached and output below.
error_hcp2_PSYC16023_Baseline_2020-11-30_10.43.1606733016.log (905 Bytes)
Loading Quantitative Neuroimaging Environment & ToolboX (Qu|Nex) Version: v0.61.17
Logged in as User: k1328096
Node info: login1.nan.kcl.ac.uk
OS: RedHat Linux #1 SMP Mon Oct 19 10:20:12 CDT 2020
██████\ ║ ██\ ██\
██ __██\ ║ ███\ ██ |
██ / ██ |██\ ██\ ║ ████\ ██ | ██████\ ██\ ██\
██ | ██ |██ | ██ | ║ ██ ██\██ |██ __██\\██\ ██ |
██ | ██ |██ | ██ | ║ ██ \████ |████████ |\████ /
██ ██\██ |██ | ██ | ║ ██ |\███ |██ ____|██ ██<
\██████ / \██████ | ║ ██ | \██ |\███████\██ /\██\
\___███\ \______/ ║ \__| \__| \_______\__/ \__|
\___| ║
DEVELOPED & MAINTAINED BY:
Anticevic Lab
MBLab led by Grega Repovs
COPYRIGHT & LICENSE NOTICE:
Use of this software is subject to the terms and conditions defined by the
Yale University Copyright Policies:
http://ocr.yale.edu/faculty/policies/yale-university-copyright-policy
and the terms and conditions defined in the file ‘LICENSE.md’ which is
a part of the Qu|Nex Suite source code package:
https://bitbucket.org/hidradev/qunextools/src/master/LICENSE.md
—> Setting up Octave
—> Qu|Nex environment set successfully!
===> Executing Qu|Nex RunTurnkey workflow…
------------------------ Initiating Qu|Nex Turnkey Workflow -------------------------------
–> Note: Acceptance Test type not specified. Setting default type to: no
–> Note: Turnkey cleaning not specified. Setting default to: no
–> Checking that requested hcp2 are supported…
hcp2 is supported.
–> Verified list of supported Turnkey steps to be run: hcp2
– RunTurnkey.sh: Specified Command-Line Options - Start –
Qu|Nex Turnkey run type: local
Qu|Nex Turnkey clean interim files: no
Local project name: PSYSCAN_test_qunex
Raw data input path:
Qu|Nex Session variable name: sessions
Qu|Nex Batch file input: /home/k1328096/qunex/PSYSCAN_test_qunex/processing/batch_psyscan_test.txt
Qu|Nex Mapping file input: /home/k1328096/qunex/PSYSCAN_test_qunex/sessions/PSYC16023_Baseline/session_hcp.txt
Qu|Nex Project-specific final Batch file path: /home/k1328096/qunex/PSYSCAN_test_qunex//processing/batch_psyscan_test.txt
Qu|Nex Study folder: /home/k1328096/qunex/PSYSCAN_test_qunex/
Qu|Nex Log folder: /home/k1328096/qunex/PSYSCAN_test_qunex//processing/logs
Qu|Nex Session-specific working folder: /home/k1328096/qunex/PSYSCAN_test_qunex//sessions/sessions/inbox
Overwrite for a given turnkey step set to: no
Overwrite for session set to: no
Overwrite for project set to: no
Overwrite for the entire XNAT project: no
Cleanup for session set to: no
Cleanup for project set to: no
Custom QC requested:
Sessionids parameter: PSYC16023_Baseline
Turnkey workflow steps: hcp2
Acceptance test requested: no
– RunTurnkey.sh: Specified Command-Line Options - End –
------------------------- Starting Qu|Nex Turnkey Workflow --------------------------------
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
QuNexVer : 0.61.17
TOOLS : /opt
QUNEXREPO : qunex
QUNEXPATH : /opt/qunex
QUNEXENV : /opt/env/qunex
TemplateFolder : /opt/qunex/library/data/
QUNEXMCOMMAND : octave -q --no-init-file --eval
CONDADIR : /opt/miniconda/miniconda-latest
FSLDIR : /opt/fsl/fsl-latest
FSLGPUDIR : /opt/fsl/fsl-latest/bin
FSLGPUBinary : /opt/qunex/connector/functions/DiffusionTractographyDense/fsl_gpu_binaries
FSL_FIXDIR : /opt/fsl/fix-latest
FREESURFER_HOME : /opt/freesurfer/freesurfer-6.0
FREESURFER_SCHEDULER : /opt/freesurfer/FreeSurferScheduler
WORKBENCHDIR : /opt/workbench/workbench-latest/bin_rh_linux64
CARET7DIR : /opt/workbench/workbench-latest/bin_rh_linux64
AFNIDIR : /opt/afni/afni-latest
DCMNIIDIR : /opt/dcm2niix/dcm2niix-latest
DICMNIIDIR : /opt/dicm2nii/dicm2nii-latest
OCTAVEDIR : /opt/octave/octave-latest
OCTAVEPKGDIR : /opt/octave/octavepkg
OCTAVEBINDIR : /opt/octave/octave-latest/bin
RDIR : /opt/R/R-latest
PALMDIR : /opt/palm/palm-latest-o
HCPPIPEDIR : /opt/HCP/HCPpipelines
GRADUNWARPDIR : /opt/gradunwarp/gradunwarp-latest
HCPPIPEDIR_Templates : /opt/HCP/HCPpipelines/global/templates
HCPPIPEDIR_Bin : /opt/HCP/HCPpipelines/global/binaries
HCPPIPEDIR_Config : /opt/HCP/HCPpipelines/global/config
HCPPIPEDIR_PreFS : /opt/HCP/HCPpipelines/PreFreeSurfer/scripts
HCPPIPEDIR_FS : /opt/HCP/HCPpipelines/FreeSurfer/scripts
HCPPIPEDIR_PostFS : /opt/HCP/HCPpipelines/PostFreeSurfer/scripts
HCPPIPEDIR_fMRISurf : /opt/HCP/HCPpipelines/fMRISurface/scripts
HCPPIPEDIR_fMRIVol : /opt/HCP/HCPpipelines/fMRIVolume/scripts
HCPPIPEDIR_tfMRI : /opt/HCP/HCPpipelines/tfMRI/scripts
HCPPIPEDIR_dMRI : /opt/HCP/HCPpipelines/DiffusionPreprocessing/scripts
HCPPIPEDIR_dMRITract : /opt/qunex/connector/functions/DiffusionTractography/scripts
HCPPIPEDIR_Global : /opt/HCP/HCPpipelines/global/scripts
HCPPIPEDIR_tfMRIAnalysis : /opt/HCP/HCPpipelines/TaskfMRIAnalysis/scripts
MSMBINDIR : /opt/MSM_HOCR_v3/Centos
HCPPIPEDIR_dMRITracFull : /opt/qunex/connector/functions/DiffusionTractographyDense
HCPPIPEDIR_dMRILegacy : /opt/qunex/connector/functions
AutoPtxFolder : /opt/qunex/connector/functions/DiffusionTractographyDense/autoPtx_HCP_extended
EDDYCUDADIR : /opt/qunex/connector/functions/DiffusionTractographyDense/fsl_gpu_binaries/eddy_cuda
HCPpipelines : v4.3.0
HCPpipelines TAG : v4.3.0-rc.1
HCPpipelines commit : 6c3303710089a0b0171af8071a90469e8158a1cb
FSL Binary : /opt/fsl/fsl-latest/bin/fsl
FSL Version : 6.0.4:ddd0a010
FreeSurfer Binary : /opt/freesurfer/freesurfer-6.0/bin/freesurfer
FreeSurfer Version : freesurfer-Linux-centos6_x86_64-stable-pub-v6.0.0-2beb96c
dcm2niix Binary : /opt/dcm2niix/dcm2niix-latest/dcm2niix
dcm2niix Version : Chris Rorden's dcm2niiX version v1.0.20190720 (JP2:OpenJPEG) GCC4.8.5 (64-bit Linux)
Octave Binary : /usr/local/bin/octave
Octave Version : 4.4.1
R Binary : /usr/bin/R
R Version : R version 3.6.0 (2019-04-26) -- "Planting of a Tree"
R required packages : ggplot2
R Package : ggplot2 ‘3.3.2’
PALM Binary : /opt/palm/palm-latest-o/palm.m
PALM Version : Oct/2019 (version alpha117)
wb_command Binary : /opt/workbench/workbench-latest/bin_rh_linux64/wb_command
wb_command Version : Version: 1.4.2
PATH : /opt/qunex/connector/functions/DiffusionTractographyDense/fsl_gpu_binaries/probtrackx_gpu_cuda_:/opt/qunex/connector/functions/DiffusionTractographyDense/fsl_gpu_binaries/bedpostx_gpu_cuda_/bedpostx_gpu/bin:/opt/qunex/connector/functions/DiffusionTractographyDense/fsl_gpu_binaries/bedpostx_gpu_cuda_/bedpostx_gpu/lib:/opt/qunex/connector/functions/DiffusionTractographyDense/fsl_gpu_binaries/bedpostx_gpu_cuda_/bedpostx_gpu:/opt/qunex/connector/functions/DiffusionTractographyDense/fsl_gpu_binaries/probtrackx_gpu_cuda_7.0:/opt/qunex/connector/functions/DiffusionTractographyDense/fsl_gpu_binaries/bedpostx_gpu_cuda_7.5/bedpostx_gpu/bin:/opt/qunex/connector/functions/DiffusionTractographyDense/fsl_gpu_binaries/bedpostx_gpu_cuda_7.5/bedpostx_gpu/lib:/opt/qunex/connector/functions/DiffusionTractographyDense/fsl_gpu_binaries/bedpostx_gpu_cuda_7.5/bedpostx_gpu:/usr/local/cuda-7.5/bin:/opt/bin:/opt/olib:/usr/local/bin:/opt/bin:/opt/qunex/nitools:/opt/qunex/library/bin:/opt/qunex/niutilities:/opt/qunex/connector:/usr/local/cuda/bin:/opt/HCP/HCPpipelines/global/matlab/:/opt/fsl/fix-latest:/opt/qunex/connector/functions/DiffusionTractographyDense/fsl_gpu_binaries/eddy_cuda:/opt/qunex/connector/functions/DiffusionTractographyDense/fsl_gpu_binaries:/opt/qunex/connector/functions/DiffusionTractographyDense/autoPtx_HCP_extended:/opt/qunex/connector/functions:/opt/qunex/connector/functions/DiffusionTractographyDense:/opt/qunex/connector/functions/DiffusionTractography/scripts:/opt/HCP/HCPpipelines/TaskfMRIAnalysis/scripts:/opt/HCP/HCPpipelines/global/scripts:/opt/HCP/HCPpipelines/DiffusionPreprocessing/scripts:/opt/HCP/HCPpipelines/tfMRI/scripts:/opt/HCP/HCPpipelines/fMRIVolume/scripts:/opt/HCP/HCPpipelines/fMRISurface/scripts:/opt/HCP/HCPpipelines/PostFreeSurfer/scripts:/opt/HCP/HCPpipelines/FreeSurfer/scripts:/opt/HCP/HCPpipelines/PreFreeSurfer/scripts:/opt/HCP/HCPpipelines/global/config:/opt/HCP/HCPpipelines/global/binaries:/opt/HCP/HCPpipelines/global/templates:/opt/gradunwarp/gradunwarp-latest/gradunwarp/core:/opt/workbench/workbench-latest/bin_rh_linux64:/opt/HCP/HCPpipelines:/opt/HCP/HCPpipelines/MSMConfig:/opt/qunex/niutilities/templates/:/opt/qunex/library/data/:/opt/qunex/library/data/atlases/HCP:/opt/qunex/connector/functions:/opt/qunex/connector:/opt/R/R-latest:/opt/matlab/bin:/opt/octave/octave-latest/bin:/opt/dcm2niix/dcm2niix-latest:/opt/dcm2niix/dcm2niix-latest/build/bin:/opt/afni/afni-latest:/opt/palm/palm-latest-o:/opt/workbench/workbench-latest/bin_rh_linux64:/opt/freesurfer/FreeSurferScheduler:/opt/fsl/fsl-latest/bin:/opt/freesurfer/freesurfer-6.0/bin:/opt/freesurfer/freesurfer-6.0/fsfast/bin:/opt/freesurfer/freesurfer-6.0/tktools:/opt/fsl/fsl-latest/bin:/opt/freesurfer/freesurfer-6.0/mni/bin:/opt/freesurfer/freesurfer-6.0:/opt/fsl/fsl-latest/bin:/opt/bin:/opt/lib/bin:/opt/lib/lib/:/opt/env/qunex/bin:/opt/miniconda/miniconda2/condabin:/opt/miniconda/miniconda-latest/bin:/opt:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/bin
PYTHONPATH :
MATLABPATH : /opt/qunex/nitools/stats:/opt/qunex/nitools/img:/opt/qunex/nitools/general:/opt/qunex/nitools/fcMRI:/opt/HCP/HCPpipelines/global/matlab/:/opt/fsl/fix-latest:/opt/qunex/library/data/:/opt/qunex/library/data/atlases/HCP:/opt/qunex/connector:/opt/dicm2nii/dicm2nii-latest:/opt/palm/palm-latest-o:/opt/workbench/workbench-latest/bin_rh_linux64:/opt/fsl/fsl-latest:/opt/fsl/fsl-latest/bin:
=================== Qu|Nex environment set successfully! ====================
===> Executing specific Qu|Nex turkey workflow steps: hcp2
===> RUNNING RunTurnkey step ~~~ HCP Pipelines step - hcp2 (hcp_FS).
… Running Qu|Nex v0.61.17 …
Using /home/k1328096/qunex/PSYSCAN_test_qunex//processing/batch_psyscan_test.txt for input.
— Full Qu|Nex call for command: hcp2
gmri hcp2 --sessionsfolder=/home/k1328096/qunex/PSYSCAN_test_qunex//sessions --sessions=/home/k1328096/qunex/PSYSCAN_test_qunex//processing/batch_psyscan_test.txt --overwrite=no --logfolder=/home/k1328096/qunex/PSYSCAN_test_qunex//processing/logs --sessionids=PSYC16023_Baseline
Quantitative Neuroimaging Environment & Toolbox (Qu|Nex) Suite Version 0.61.17
Starting multiprocessing sessions in /home/k1328096/qunex/PSYSCAN_test_qunex//processing/batch_psyscan_test.txt with a pool of 1 concurrent processes
---- Running local
Starting processing of sessions PSYC16023_Baseline at Monday, 30. November 2020 10:43:35
Session id: PSYC16023_Baseline
[started on Monday, 30. November 2020 10:43:35]
Running HCP FreeSurfer Pipeline [LegacyStyleData] …
—> PreFS results present.
—> FREESURFER_HOME set to: /opt/freesurfer/freesurfer-6.0
Running HCP Pipelines command via Qu|Nex:
Running HCP FS
ERROR: Running HCP FS failed with error 127
…
command executed:
FreeSurfer/FreeSurferPipeline.sh
–subjectDIR="/home/k1328096/qunex/PSYSCAN_test_qunex/sessions/PSYC16023_Baseline/hcp/PSYC16023_Baseline/T1w"
–subject=“PSYC16023_Baseline”
–processing-mode=“LegacyStyleData”
–t1="/home/k1328096/qunex/PSYSCAN_test_qunex/sessions/PSYC16023_Baseline/hcp/PSYC16023_Baseline/T1w/T1w_acpc_dc_restore.nii.gz"
–t1brain="/home/k1328096/qunex/PSYSCAN_test_qunex/sessions/PSYC16023_Baseline/hcp/PSYC16023_Baseline/T1w/T1w_acpc_dc_restore_brain.nii.gz"
–t2=“NONE”
—> logfile: /home/k1328096/qunex/PSYSCAN_test_qunex//processing/logs/comlogs/error_hcp2_PSYC16023_Baseline_2020-11-30_10.43.1606733016.log
===> Final report for command hcp2
… PSYC16023_Baseline —> Error
===> Not all tasks completed fully!
Hi George,
Sorry for the late reply, for some reason the forum system did not notify me of your reply. I have another question, did hcp1 work for you? Based on the error above I would say it did not.
Anyhow, based on above logs I feel like your hcp_Pipeline` path is not set correctly. You can set this in your batch file, by adding:
_hcp_Pipeline : /opt/HCP/HCPpipelines
you can also replace /opt/HCP/HCPpipelines with ${HCPPIPEDIR}, an easier option could be to add this parameter to the command call:
--hcp_Pipelines=/opt/HCP/HCPpipelines
Cheers, Jure
Hi George,
Just checking whether this was resolved, so I can close this issue.
Thanks, Jure
Hi Jure,
Apologies for not replying sooner!
The issue was resolved by specifying the HCP pipeline location, thanks for your help.
Best wishes,
George