Description:
I am following the quickstart guide with the sample data on a cluster using schedulling, but when running I get the following error:
ERROR: No sessions specified to process. Please check your batch file, filtering options or sessionids parameter!
I think its probably because this file doesn’t exsist in this path:
---> ERROR: no session_hcp.txt found for HCPA001! Please check your data! [/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/HCPA001/session_hcp.txt]
I don’t know how to fix this as I didn’t have a quickstart directory before starting the job.
Call:
PATH=${HOME}/scratch/qunex:${PATH}
export STUDY_NAME="quickstart"
export WORK_DIR="${HOME}/scratch/qunex"
export QUNEX_CONTAINER=${WORK_DIR}/qunex_suite-0.99.2d.sif
export RAW_DATA="${WORK_DIR}/data"
export INPUT_BATCH_FILE="${RAW_DATA}/HCPA001_parameters.txt"
export INPUT_MAPPING_FILE="${RAW_DATA}/HCPA001_mapping.txt"
export SESSIONS="HCPA001"
export RUNTURNKEY_TYPE="local"
export RUNTURNKEY_STEPS="create_study,map_raw_data,import_dicom,create_session_info,setup_hcp,create_batch,hcp_pre_freesurfer,hcp_freesurfer,hcp_post_freesurfer,hcp_fmri_volume,hcp_fmri_surface"
qunex_container run_turnkey \
--rawdatainput="${RAW_DATA}" \
--paramfile="${INPUT_BATCH_FILE}" \
--mappingfile="${INPUT_MAPPING_FILE}" \
--workingdir="${WORK_DIR}" \
--projectname="${STUDY_NAME}" \
--path="${WORK_DIR}/${STUDY_NAME}" \
--sessions="${SESSIONS}" \
--sessionsfoldername="sessions" \
--turnkeytype="${RUNTURNKEY_TYPE}" \
--container="${QUNEX_CONTAINER}" \
--turnkeysteps="${RUNTURNKEY_STEPS}" \
--scheduler="SLURM,time=04-00:00:00,cpus-per-task=1,mem-per-cpu=16000,jobname=qx_quickstart"
Logs:
e[31m Check /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs e[0m
e[31m Check /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/runlogs e[0m
[error_create_batch_2024-04-21_19.56.17.259893.log|attachment](upload://t3l8X57qPJKPZvtG5fYaw78kxfs.log) (1.0 KB)
[error_create_session_info_2024-04-21_19.56.12.818351.log|attachment](upload://7WSfPbOKJ5avgXeoAvBEFyi6fnl.log) (883 Bytes)
[error_import_dicom_2024-04-21_19.56.10.473621.log|attachment](upload://x6cwvvUp9fBZ1jvdgD6qslog4Tz.log) (1007 Bytes)
[error_map_raw_data_HCPA001_2024-04-21_19.56.07.493837.log|attachment](upload://rd7PdVunLfx09peFN1UZORL83aW.log) (858 Bytes)
[error_setup_hcp_HCPA001_2024-04-21_19.56.15.036537.log|attachment](upload://uanw1n4jX7YyACwSNOZBhIkwpFU.log) (1.2 KB)
It seems like the data is not onboarded successfully since the import_dicom / map_raw_data step failed.
Can you try the other quick start option (Step-by-step execution at QuNex quick start using a Docker container — QuNex documentation). That will allow us to pinpoint the issue.
Best, Jure
1 Like
Hi Jure
Thank you for your reply. I have ran again without scheduling and below is the output.
I have also attached the logs.
Kind regards, Scott
[tqh20wsu@c0011 ~]$ qunex_container run_turnkey --rawdatainput="${RAW_DATA}" --paramfile="${INPUT_BATCH_FILE}" --mappingfile="${INPUT_MAPPING_FILE}" --workingdir="${WORK_DIR}" --projectname="${STUDY_NAME}" --path="${WORK_DIR}/${STUDY_NAME}" --sessions="${SESSIONS}" --sessionsfoldername="sessions" --turnkeytype="${RUNTURNKEY_TYPE}" --container="${QUNEX_CONTAINER}" --turnkeysteps="${RUNTURNKEY_STEPS}"
--> QuNex will run the command over 1 sessions. It will utilize:
Maximum sessions run in parallel for a job: 1.
Maximum elements run in parallel for a session: 1.
Up to 1 processes will be utilized for a job.
Job #1 will run sessions: HCPA001
[tqh20wsu@c0011 ~]$ --> unsetting the following environment variables: PATH MATLABPATH PYTHONPATH QUNEXVer TOOLS QUNEXREPO QUNEXPATH QUNEXEXTENSIONS QUNEXLIBRARY QUNEXLIBRARYETC TemplateFolder FSL_FIXDIR FREESURFERDIR FREESURFER_HOME FREESURFER_SCHEDULER FreeSurferSchedulerDIR WORKBENCHDIR DCMNIIDIR DICMNIIDIR MATLABDIR MATLABBINDIR OCTAVEDIR OCTAVEPKGDIR OCTAVEBINDIR RDIR HCPWBDIR AFNIDIR PYLIBDIR FSLDIR FSLBINDIR PALMDIR QUNEXMCOMMAND HCPPIPEDIR CARET7DIR GRADUNWARPDIR HCPPIPEDIR_Templates HCPPIPEDIR_Bin HCPPIPEDIR_Config HCPPIPEDIR_PreFS HCPPIPEDIR_FS HCPPIPEDIR_PostFS HCPPIPEDIR_fMRISurf HCPPIPEDIR_fMRIVol HCPPIPEDIR_tfMRI HCPPIPEDIR_dMRI HCPPIPEDIR_dMRITract HCPPIPEDIR_Global HCPPIPEDIR_tfMRIAnalysis HCPCIFTIRWDIR MSMBin HCPPIPEDIR_dMRITractFull HCPPIPEDIR_dMRILegacy AutoPtxFolder EDDYCUDA USEOCTAVE QUNEXENV CONDADIR MSMBINDIR MSMCONFIGDIR R_LIBS FSL_FIX_CIFTIRW FSFAST_HOME SUBJECTS_DIR MINC_BIN_DIR MNI_DIR MINC_LIB_DIR MNI_DATAPATH FSF_OUTPUT_FORMAT ANTSDIR CUDIMOT
Generated by QuNex
------------------------------------------------------------------------
Version: 0.99.2
User: tqh20wsu
System: c0011.hpc.private
OS: RedHat Linux #1 SMP Thu Jan 25 16:17:31 UTC 2024
------------------------------------------------------------------------
██████\ ║ ██\ ██\
██ __██\ ║ ███\ ██ |
██ / ██ |██\ ██\ ║ ████\ ██ | ██████\ ██\ ██\
██ | ██ |██ | ██ | ║ ██ ██\██ |██ __██\\██\ ██ |
██ | ██ |██ | ██ | ║ ██ \████ |████████ |\████ /
██ ██\██ |██ | ██ | ║ ██ |\███ |██ ____|██ ██\
\██████ / \██████ | ║ ██ | \██ |\███████\██ /\██\
\___███\ \______/ ║ \__| \__| \_______\__/ \__|
\___| ║
DEVELOPED & MAINTAINED BY:
Anticevic Lab, Yale University
Mind & Brain Lab, University of Ljubljana
Murray Lab, Yale University
COPYRIGHT & LICENSE NOTICE:
Use of this software is subject to the terms and conditions defined in
'LICENSES' which is a part of the QuNex Suite source code package:
https://gitlab.qunex.yale.edu/qunex/qunex/-/tree/master/LICENSES
---> Setting up Octave
===> Executing QuNex run_turnkey workflow...
------------------------ Initiating QuNex Turnkey Workflow -------------------------------
--> Note: Acceptance Test type not specified. Setting default type to: no
--> Note: Turnkey cleaning not specified. Setting default to: no
--> Checking that requested create_study map_raw_data import_dicom create_session_info setup_hcp create_batch hcp_pre_freesurfer hcp_freesurfer hcp_post_freesurfer hcp_fmri_volume hcp_fmri_surface are supported...
create_study is supported.
map_raw_data is supported.
import_dicom is supported.
create_session_info is supported.
setup_hcp is supported.
create_batch is supported.
hcp_pre_freesurfer is supported.
hcp_freesurfer is supported.
hcp_post_freesurfer is supported.
hcp_fmri_volume is supported.
hcp_fmri_surface is supported.
--> Verified list of supported Turnkey steps to be run: create_study map_raw_data import_dicom create_session_info setup_hcp create_batch hcp_pre_freesurfer hcp_freesurfer hcp_post_freesurfer hcp_fmri_volume hcp_fmri_surface
-- run_turnkey.sh: Specified Command-Line Options - Start --
QuNex Turnkey run type: local
QuNex Turnkey clean interim files: no
Local project name: quickstart
Raw data input path: /gpfs/home/tqh20wsu/scratch/qunex/data
QuNex Session variable name: HCPA001
QuNex Parameters file input: /gpfs/home/tqh20wsu/scratch/qunex/data/HCPA001_parameters.txt
QuNex Mapping file input: /gpfs/home/tqh20wsu/scratch/qunex/data/HCPA001_mapping.txt
QuNex Project-specific final Batch file path: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing
QuNex Study folder: /gpfs/home/tqh20wsu/scratch/qunex/quickstart
QuNex Log folder: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs
QuNex Session-specific working folder: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/HCPA001/inbox
Overwrite for a given turnkey step set to: no
Overwrite for session set to: no
Overwrite for project set to: no
Overwrite for the entire XNAT project: no
Cleanup for session set to: no
Cleanup for project set to: no
Custom QC requested:
Sessionids parameter: HCPA001
Turnkey workflow steps: create_study map_raw_data import_dicom create_session_info setup_hcp create_batch hcp_pre_freesurfer hcp_freesurfer hcp_post_freesurfer hcp_fmri_volume hcp_fmri_surface
Acceptance test requested: no
-- run_turnkey.sh: Specified Command-Line Options - End --
------------------------- Starting QuNex Turnkey Workflow --------------------------------
--------------------------------------------------------------
QuNex Environment Status Report
--------------------------------------------------------------
OS Version
----------------------------------------------
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
QuNex General Environment Variables
----------------------------------------------
QuNexVer : 0.99.2
TOOLS : /opt
QUNEXREPO : qunex
QUNEXPATH : /opt/qunex
QUNEXENV : /opt/env/qunex
TemplateFolder : /opt/qunex/qx_library/data/
QUNEXMCOMMAND : octave -q --no-init-file --eval
Core Dependencies Environment Variables
----------------------------------------------
CONDADIR : /opt/miniconda
FSLDIR : /opt/fsl/fsl
FSLCONFDIR : /opt/fsl/fsl/config
FSLBINDIR : /opt/fsl/fsl/bin
FSL_FIXDIR : /opt/fsl/fix
FREESURFER_HOME : /opt/freesurfer/freesurfer
FREESURFER_SCHEDULER : /opt/freesurfer/FreeSurferScheduler
WORKBENCHDIR : /opt/workbench/workbench/bin_rh_linux64
CARET7DIR : /opt/workbench/workbench/bin_rh_linux64
AFNIDIR : /opt/AFNI/AFNI
ANTSDIR : /opt/ANTs/ANTs/bin
DCMNIIDIR : /opt/dcm2niix/dcm2niix
DICMNIIDIR : /opt/dicm2nii/dicm2nii
OCTAVEDIR : /opt/octave/octave
MATLABDIR : /opt/matlab
MATLABBINDIR : /opt/matlab/bin
RDIR : /opt/R/R
PALMDIR : /opt/palm/palm-o
HCP Pipelines
----------------------------------------------
HCPPIPEDIR : /opt/HCP/HCPpipelines
GRADUNWARPDIR : /opt/gradunwarp/gradunwarp
HCPPIPEDIR_Templates : /opt/HCP/HCPpipelines/global/templates
HCPPIPEDIR_Bin : /opt/HCP/HCPpipelines/global/binaries
HCPPIPEDIR_Config : /opt/HCP/HCPpipelines/global/config
HCPPIPEDIR_PreFS : /opt/HCP/HCPpipelines/PreFreeSurfer/scripts
HCPPIPEDIR_FS : /opt/HCP/HCPpipelines/FreeSurfer/scripts
HCPPIPEDIR_PostFS : /opt/HCP/HCPpipelines/PostFreeSurfer/scripts
HCPPIPEDIR_fMRISurf : /opt/HCP/HCPpipelines/fMRISurface/scripts
HCPPIPEDIR_fMRIVol : /opt/HCP/HCPpipelines/fMRIVolume/scripts
HCPPIPEDIR_tfMRI : /opt/HCP/HCPpipelines/tfMRI/scripts
HCPPIPEDIR_dMRI : /opt/HCP/HCPpipelines/DiffusionPreprocessing/scripts
HCPPIPEDIR_dMRITract : /opt/qunex/bash/qx_utilities/diffusion_tractography/scripts
HCPPIPEDIR_Global : /opt/HCP/HCPpipelines/global/scripts
HCPPIPEDIR_tfMRIAnalysis : /opt/HCP/HCPpipelines/TaskfMRIAnalysis/scripts
MSMBINDIR : /opt/MSM_HOCR_v3
HCPPIPEDIR_dMRITractFull : /opt/qunex/bash/qx_utilities/diffusion_tractography_dense
HCPPIPEDIR_dMRILegacy : /opt/qunex/bash/qx_utilities
AutoPtxFolder : /opt/qunex/bash/qx_utilities/diffusion_tractography_dense/autoptx_hcp_extended
EDDYCUDA : /opt/fsl/fsl/bin/eddy_cuda10.2
ASLDIR : /opt/HCP/hcp-asl
Binary / Executable Locations and Versions
----------------------------------------------
HCPpipelines TAG : Post-v4.7.0-5df95e4
HCPpipelines commit : 5df95e40378a71457fd510ae8e465db2fee6f48d
HCP ASL commit : 9e8b06c56b685c9a07cc91a839161704aede1d0d
FSL Binary : /opt/fsl/fsl/share/fsl/bin/fsl
FSL Version : 6.0.7.1
FreeSurfer Binary : /opt/freesurfer/freesurfer/bin/freesurfer
FreeSurfer Version : freesurfer-Linux-centos6_x86_64-stable-pub-v6.0.0-2beb96c
AFNI Binary : /opt/AFNI/AFNI/afni
AFNI Version : Precompiled binary linux_centos_7_64: Jan 11 2024 (Version AFNI_24.0.01 'Caracalla')
ANTs Binary : /opt/ANTs/ANTs/bin/antsRegistration
ANTs Version : ANTs Version: 2.3.5.dev1-g6f137
dcm2niix Binary : /opt/dcm2niix/dcm2niix/dcm2niix
dcm2niix Version : Chris Rorden's dcm2niiX version v1.0.20230411 (JP2:OpenJPEG) GCC4.8.5 x86-64 (64-bit Linux)
Octave Binary : /opt/octave/octave/bin/octave
Octave Version : 4.4.1
R Binary : /usr/local/bin/R
R Version : R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
python binary : /opt/env/qunex/bin/python
python Version : Python 3.12.1
PALM Binary : /opt/palm/palm-o/palm.m
PALM Version : Jun/2021 (github)
wb_command Binary : /opt/workbench/workbench/bin_rh_linux64/wb_command
wb_command Version : Version: 1.5.0
Full Environment Paths
----------------------------------------------
PATH : /opt/env/qunex/bin:/opt/miniconda/bin:/opt/fsl/fsl-6.0.7.1/condabin:/usr/local/cuda/bin:/opt/HCP/HCPpipelines/global/matlab:/opt/fsl/fix:/opt/qunex/bash/qx_utilities/diffusion_tractography_dense/autoptx_hcp_extended:/opt/qunex/bash/qx_utilities:/opt/qunex/bash/qx_utilities/diffusion_tractography_dense:/opt/qunex/bash/qx_utilities/diffusion_tractography/scripts:/opt/HCP/HCPpipelines/TaskfMRIAnalysis/scripts:/opt/HCP/HCPpipelines/global/scripts:/opt/HCP/HCPpipelines/DiffusionPreprocessing/scripts:/opt/HCP/HCPpipelines/tfMRI/scripts:/opt/HCP/HCPpipelines/fMRIVolume/scripts:/opt/HCP/HCPpipelines/fMRISurface/scripts:/opt/HCP/HCPpipelines/PostFreeSurfer/scripts:/opt/HCP/HCPpipelines/FreeSurfer/scripts:/opt/HCP/HCPpipelines/PreFreeSurfer/scripts:/opt/HCP/HCPpipelines/global/config:/opt/HCP/HCPpipelines/global/binaries:/opt/HCP/HCPpipelines/global/templates:/opt/gradunwarp/gradunwarp/gradunwarp/core:/opt/workbench/workbench/bin_rh_linux64:/opt/HCP/HCPpipelines:/opt/HCP/HCPpipelines/MSMConfig:/opt/qunex/python/qx_utilities/templates:/opt/qunex/qx_library/data:/opt/qunex/qx_library/data/atlases/hcp:/opt/R/R:/opt/matlab/bin:/opt/octave/octave/bin:/opt/dcm2niix/dcm2niix:/opt/dcm2niix/dcm2niix/bin:/opt/ANTs/ANTs/bin:/opt/AFNI/AFNI:/opt/palm/palm-o:/opt/workbench/workbench/bin_rh_linux64:/opt/freesurfer/FreeSurferScheduler:/opt/fsl/fsl/share/fsl/bin:/opt/freesurfer/freesurfer/bin:/opt/freesurfer/freesurfer/fsfast/bin:/opt/freesurfer/freesurfer/tktools:/opt/fsl/fsl/bin:/opt/fsl/fsl/share/fsl/bin:/opt/freesurfer/freesurfer/mni/bin:/opt/freesurfer/freesurfer:/opt/freesurfer/freesurfer/bin:/opt/freesurfer/freesurfer/mni/bin:/opt/qunex/lib:/opt/qunex/bin:/opt/bin:/opt/lib/bin:/opt/lib/lib:/opt/olib:/opt/qunex/python/qx_utilities:/opt:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/bin:/opt/fsl/fsl/bin:/opt/fsl/fsl/etc/matlab:/opt/fsl/fsl/lib
MATLABPATH : /opt/qunex/matlab/qx_mice:/opt/qunex/matlab/qx_utilities/general:/opt/qunex/matlab/qx_mri/stats:/opt/qunex/matlab/qx_mri/img:/opt/qunex/matlab/qx_mri/general:/opt/qunex/matlab/qx_mri/fc:/opt/HCP/HCPpipelines/global/matlab:/opt/HCP/HCPpipelines/global/matlab/cifti-matlab:/opt/HCP/HCPpipelines/global/matlab/icaDim:/opt/HCP/HCPpipelines/global/matlab/nets_spectra:/opt/HCP/HCPpipelines/global/matlab:/opt/fsl/fix:/opt/qunex/qx_library/data/:/opt/qunex/qx_library/data/atlases/hcp:/opt/dicm2nii/dicm2nii:/opt/palm/palm-o:/opt/workbench/workbench/bin_rh_linux64:/opt/fsl/fsl:/opt/fsl/fsl/bin:/opt/fsl/fsl/etc/matlab:
=================== QuNex environment set successfully! ====================
---------------------------------------------------------------------
===> Executing specific QuNex turkey workflow steps: create_study map_raw_data import_dicom create_session_info setup_hcp create_batch hcp_pre_freesurfer hcp_freesurfer hcp_post_freesurfer hcp_fmri_volume hcp_fmri_surface
---------------------------------------------------------------------
===> RUNNING run_turnkey step ~~~ create_study
-- Checking for and generating study folder /gpfs/home/tqh20wsu/scratch/qunex/quickstart
--> Note: /gpfs/home/tqh20wsu/scratch/qunex/quickstart not found. Regenerating now...
........................ Running QuNex v0.99.2 ........................
--- Full QuNex call for command: create_study
gmri create_study --studyfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart"
---------------------------------------------------------
started running create_study at 2024-04-23 15:14:06, track progress in /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/tmp_create_study_2024-04-23_15.14.06.618897.log
call: gmri create_study studyfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart"
-----------------------------------------
Running create_study
===================
Creating study folder structure:
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/analysis
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/analysis/scripts
... folder exists: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing
... folder exists: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/batchlogs
... folder exists: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/runlogs
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/runchecks
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/lists
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scripts
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes/QC
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes/QC/T1w
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes/QC/T2w
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes/QC/myelin
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes/QC/BOLD
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes/QC/DWI
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/info
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/info/demographics
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/info/tasks
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/info/stimuli
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/info/bids
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/info/hcpls
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/MR
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/EEG
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/BIDS
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/HCPLS
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/behavior
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/concs
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/events
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/archive
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/archive/MR
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/archive/EEG
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/archive/BIDS
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/archive/HCPLS
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/archive/behavior
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/specs
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/QC
Preparing template files:
... created parameters_example.txt file
... created .qunexstudy file
-----------------------------------------
Finished at 2024-04-23 15:14:06
===> Successful completion of task
mv: missing destination file operand after ‘/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/’
Try 'mv --help' for more information.
===> run_turnkey acceptance testing create_study logs for completion.
===> run_turnkey acceptance testing found comlog file for create_study step:
/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/done_create_study_2024-04-23_15.14.06.618897.log
===> SUCCESSFUL run_turnkey acceptance test for create_study
===> RUNNING run_turnkey step ~~~ map_raw_data
error_create_batch_2024-04-23_15.14.14.288427.log (1.0 KB)
error_create_session_info_2024-04-23_15.14.10.930164.log (883 Bytes)
error_import_dicom_2024-04-23_15.14.08.799144.log (1007 Bytes)
error_map_raw_data_HCPA001_2024-04-23_15.14.06.801711.log (858 Bytes)
error_setup_hcp_HCPA001_2024-04-23_15.14.12.626568.log (1.2 KB)
--> Running turnkey via local: c0011.hpc.private
cp /gpfs/home/tqh20wsu/scratch/qunex/data/HCPA001_mapping.txt /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/specs/HCPA001_mapping.txt
-- Linking DICOMs into /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/HCPA001/inbox
===> ERROR: ZIP archive found but did not pass check!
----------------------------------------------------------------------------
--> Batch file transfer check: pass
--> Mapping file transfer check: pass
--> Archive inbox processed: fail
ERROR. Something went wrong.
map_raw_data
ERROR. Something went wrong.
===> run_turnkey acceptance testing map_raw_data logs for completion.
===> run_turnkey acceptance testing found comlog file for map_raw_data step:
/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/error_map_raw_data_HCPA001_2024-04-23_15.14.06.801711.log
===> ERROR: run_turnkey acceptance test for map_raw_data step failed.
===> RUNNING run_turnkey step ~~~ import_dicom
-- Executed call:
/opt/qunex/bin/qunex.sh import_dicom --sessionsfolder='/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions' --sessions='HCPA001' --masterinbox='none' --archive='delete' --check='any' --unzip='yes' --add_image_type='' --add_json_info='' --gzip='folder' --overwrite='no'
........................ Running QuNex v0.99.2 ........................
--- Full QuNex call for command: import_dicom
gmri import_dicom --sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" --sessions="HCPA001" --masterinbox="none" --archive="delete" --check="any" --unzip="yes" --add_image_type="" --add_json_info="" --gzip="folder" --overwrite="no"
---------------------------------------------------------
started running import_dicom at 2024-04-23 15:14:08, track progress in /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/tmp_import_dicom_2024-04-23_15.14.08.799144.log
call: gmri import_dicom sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" sessions="HCPA001" masterinbox="none" archive="delete" check="any" unzip="yes" add_image_type="" add_json_info="" gzip="folder" overwrite="no"
-----------------------------------------
Running import_dicom
====================
---> Checking for folders to process in '/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions'
---> Found the following folders to process:
subject: HCPA001 ... HCPA001 <= HCPA001 <- HCPA001
---> Starting to process 1 packets ...
---=== PROCESSING HCPA001 ===---
Final report
============
Failed to process:
... HCPA001 [/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/HCPA001]
import_dicom: Zip file could not be processed
===> ERROR in completing import_dicom:
Some packages failed to process
Please check report!
-----------------------------------------
Finished at 2024-04-23 15:14:08
/opt/qunex/bash/qx_utilities/run_turnkey.sh: line 2159: cd: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/HCPA001/nii: No such file or directory
===> run_turnkey acceptance testing import_dicom logs for completion.
===> run_turnkey acceptance testing found comlog file for import_dicom step:
/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/error_import_dicom_2024-04-23_15.14.08.799144.log
===> ERROR: run_turnkey acceptance test for import_dicom step failed.
===> RUNNING run_turnkey step ~~~ create_session_info
-- Executed call:
/opt/qunex/bin/qunex.sh create_session_info --sessionsfolder=/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions --sessions=HCPA001 --mapping=/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/specs/HCPA001_mapping.txt
........................ Running QuNex v0.99.2 ........................
--- Full QuNex call for command: create_session_info
gmri create_session_info --sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" --sessions="HCPA001" --mapping="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/specs/HCPA001_mapping.txt"
---------------------------------------------------------
started running create_session_info at 2024-04-23 15:14:10, track progress in /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/tmp_create_session_info_2024-04-23_15.14.10.930164.log
call: gmri create_session_info sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" sessions="HCPA001" mapping="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/specs/HCPA001_mapping.txt"
-----------------------------------------
Running create_session_info
===================
... Reading pipeline mapping from /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/specs/HCPA001_mapping.txt
===> Final report
---> sessions with missing source file:
-> HCPA001
===> ERROR in completing create_session_info:
Error
Some sessions were missing source files ['/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/HCPA001']!
Some sessions encountered errors []!
Please check the data and parameters!
-----------------------------------------
Finished at 2024-04-23 15:14:10
===> run_turnkey acceptance testing create_session_info logs for completion.
===> run_turnkey acceptance testing found comlog file for create_session_info step:
/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/error_create_session_info_2024-04-23_15.14.10.930164.log
===> ERROR: run_turnkey acceptance test for create_session_info step failed.
===> RUNNING run_turnkey step ~~~ setup_hcp
-- Executed call:
/opt/qunex/bin/qunex.sh setup_hcp --sessionsfolder='/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions' --sessions='HCPA001' --existing='clear' --hcp_filename='automated' --hcp_suffix=''
........................ Running QuNex v0.99.2 ........................
--- Full QuNex call for command: setup_hcp
gmri setup_hcp --sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" --sessions="HCPA001" --existing="clear" --hcp_filename="automated" --hcp_suffix=""
---------------------------------------------------------
Running setup_hcp
=================
===> Running setup_hcp through 1 sessions in parallel
... started running setup_hcp: HCPA001 at 2024-04-23 15:14:12, track progress in /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/tmp_setup_hcp_HCPA001_2024-04-23_15.14.12.626568.log
call: gmri setup_hcp existing="clear" hcp_filename="automated" hcp_suffix="" sourcefolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/HCPA001"
-----------------------------------------
Running setup_hcp
================
=====================================================
ERROR: Source file does not exist [/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/HCPA001/session_hcp.txt]
ERROR
Traceback (most recent call last):
File "/opt/qunex/python/qx_utilities/general/core.py", line 590, in runWithLog
result = function(**args)
^^^^^^^^^^^^^^^^
File "/opt/qunex/python/qx_utilities/hcp/setup_hcp.py", line 269, in setup_hcp
inf = gc.read_session_data(os.path.join(sourcefolder, sourcefile))[0][0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/qunex/python/qx_utilities/general/core.py", line 74, in read_session_data
raise ValueError("ERROR: Batch file not found: %s" % (filename))
ValueError: ERROR: Batch file not found: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/HCPA001/session_hcp.txt
-----------------------------------------
Finished at 2024-04-23 15:14:12
session: HCPA001
... setup_hcp: HCPA001 failed [log: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/error_setup_hcp_HCPA001_2024-04-23_15.14.12.626568.log].
===> Final report for command setup_hcp
... setup_hcp: HCPA001 ERROR: Batch file not found: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/HCPA001/session_hcp.txt [log: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/error_setup_hcp_HCPA001_2024-04-23_15.14.12.626568.log]
===> RUNNING run_turnkey step ~~~ create_batch
-- Executed call:
/opt/qunex/bin/qunex.sh create_batch --sessionsfolder='/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions' --targetfile='/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt' --paramfile='/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/specs/HCPA001_parameters.txt' --sessions='HCPA001' --overwrite='append'
........................ Running QuNex v0.99.2 ........................
--- Full QuNex call for command: create_batch
gmri create_batch --sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" --targetfile="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt" --paramfile="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/specs/HCPA001_parameters.txt" --sessions="HCPA001" --overwrite="append"
---------------------------------------------------------
started running create_batch at 2024-04-23 15:14:14, track progress in /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/tmp_create_batch_2024-04-23_15.14.14.288427.log
call: gmri create_batch sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" targetfile="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt" paramfile="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/specs/HCPA001_parameters.txt" sessions="HCPA001" overwrite="append"
-----------------------------------------
Running create_batch
====================
---> Creating file HCPA001_parameters.txt [/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt]
---> appending parameter file [/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/specs/HCPA001_parameters.txt].
---> ERROR: no session_hcp.txt found for HCPA001! Please check your data! [/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/HCPA001/session_hcp.txt]
===> ERROR in completing create_batch:
No session found
No sessions found to add to the batch file!
Please check your data!
-----------------------------------------
Finished at 2024-04-23 15:14:14
===> run_turnkey acceptance testing create_batch logs for completion.
===> run_turnkey acceptance testing found comlog file for create_batch step:
/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/error_create_batch_2024-04-23_15.14.14.288427.log
===> ERROR: run_turnkey acceptance test for create_batch step failed.
===> RUNNING run_turnkey step ~~~ HCP Pipelines - hcp_pre_freesurfer.
........................ Running QuNex v0.99.2 ........................
--- Full QuNex call for command: hcp_pre_freesurfer
gmri hcp_pre_freesurfer --sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" --sessions="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt" --overwrite="no" --logfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs" --sessionids="HCPA001"
---------------------------------------------------------
WARNING: passing the batchfile through the sessions parameter will be deprecated, please use the batchfile parameter!
WARNING: the sessionids parameter will be deprecated, please use the sessions parameter!
WARNING: Use of parameters with changed name(s)!
The following parameters have new names:
... TR is now tr!
Please correct the listed parameter names in command line or batch file!
WARNING: Parameter qx_cifti_tail was not specified. Its value was imputed from parameter hcp_cifti_tail and set to '_Atlas'!
WARNING: Parameter qx_nifti_tail was not specified. Its value was imputed from parameter hcp_nifti_tail and set to ''!
WARNING: Parameter cifti_tail was not specified. Its value was imputed from parameter qx_cifti_tail and set to '_Atlas'!
WARNING: Parameter nifti_tail was not specified. Its value was imputed from parameter qx_nifti_tail and set to ''!
# Generated by QuNex 0.99.2 on 2024-04-23_15.14.15.940064#
=================================================================
gmri hcp_pre_freesurfer \
--sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" \
--sessions="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt" \
--overwrite="no" \
--logfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs" \
--sessionids="HCPA001" \
=================================================================
ERROR: No sessions specified to process. Please check your batch file, filtering options or sessionids parameter!
===> RUNNING run_turnkey step ~~~ HCP Pipelines step - hcp_freesurfer.
........................ Running QuNex v0.99.2 ........................
--- Full QuNex call for command: hcp_freesurfer
gmri hcp_freesurfer --sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" --sessions="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt" --overwrite="no" --logfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs" --sessionids="HCPA001"
---------------------------------------------------------
WARNING: passing the batchfile through the sessions parameter will be deprecated, please use the batchfile parameter!
WARNING: the sessionids parameter will be deprecated, please use the sessions parameter!
WARNING: Use of parameters with changed name(s)!
The following parameters have new names:
... TR is now tr!
Please correct the listed parameter names in command line or batch file!
WARNING: Parameter qx_cifti_tail was not specified. Its value was imputed from parameter hcp_cifti_tail and set to '_Atlas'!
WARNING: Parameter qx_nifti_tail was not specified. Its value was imputed from parameter hcp_nifti_tail and set to ''!
WARNING: Parameter cifti_tail was not specified. Its value was imputed from parameter qx_cifti_tail and set to '_Atlas'!
WARNING: Parameter nifti_tail was not specified. Its value was imputed from parameter qx_nifti_tail and set to ''!
# Generated by QuNex 0.99.2 on 2024-04-23_15.14.17.596236#
=================================================================
gmri hcp_freesurfer \
--sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" \
--sessions="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt" \
--overwrite="no" \
--logfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs" \
--sessionids="HCPA001" \
=================================================================
ERROR: No sessions specified to process. Please check your batch file, filtering options or sessionids parameter!
===> RUNNING run_turnkey step ~~~ HCP Pipelines step - hcp_post_freesurfer.
........................ Running QuNex v0.99.2 ........................
--- Full QuNex call for command: hcp_post_freesurfer
gmri hcp_post_freesurfer --sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" --sessions="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt" --overwrite="no" --logfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs" --sessionids="HCPA001"
---------------------------------------------------------
WARNING: passing the batchfile through the sessions parameter will be deprecated, please use the batchfile parameter!
WARNING: the sessionids parameter will be deprecated, please use the sessions parameter!
WARNING: Use of parameters with changed name(s)!
The following parameters have new names:
... TR is now tr!
Please correct the listed parameter names in command line or batch file!
WARNING: Parameter qx_cifti_tail was not specified. Its value was imputed from parameter hcp_cifti_tail and set to '_Atlas'!
WARNING: Parameter qx_nifti_tail was not specified. Its value was imputed from parameter hcp_nifti_tail and set to ''!
WARNING: Parameter cifti_tail was not specified. Its value was imputed from parameter qx_cifti_tail and set to '_Atlas'!
WARNING: Parameter nifti_tail was not specified. Its value was imputed from parameter qx_nifti_tail and set to ''!
# Generated by QuNex 0.99.2 on 2024-04-23_15.14.19.265772#
=================================================================
gmri hcp_post_freesurfer \
--sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" \
--sessions="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt" \
--overwrite="no" \
--logfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs" \
--sessionids="HCPA001" \
=================================================================
ERROR: No sessions specified to process. Please check your batch file, filtering options or sessionids parameter!
===> RUNNING run_turnkey step ~~~ HCP Pipelines - hcp_fmri_volume. BOLDS: all
........................ Running QuNex v0.99.2 ........................
--- Full QuNex call for command: hcp_fmri_volume
gmri hcp_fmri_volume --sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" --sessions="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt" --overwrite="no" --sessionids="HCPA001" --bolds="all"
---------------------------------------------------------
WARNING: passing the batchfile through the sessions parameter will be deprecated, please use the batchfile parameter!
WARNING: the sessionids parameter will be deprecated, please use the sessions parameter!
WARNING: Use of parameters with changed name(s)!
The following parameters have new names:
... TR is now tr!
Please correct the listed parameter names in command line or batch file!
WARNING: Parameter qx_cifti_tail was not specified. Its value was imputed from parameter hcp_cifti_tail and set to '_Atlas'!
WARNING: Parameter qx_nifti_tail was not specified. Its value was imputed from parameter hcp_nifti_tail and set to ''!
WARNING: Parameter cifti_tail was not specified. Its value was imputed from parameter qx_cifti_tail and set to '_Atlas'!
WARNING: Parameter nifti_tail was not specified. Its value was imputed from parameter qx_nifti_tail and set to ''!
# Generated by QuNex 0.99.2 on 2024-04-23_15.14.20.938805#
=================================================================
gmri hcp_fmri_volume \
--sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" \
--sessions="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt" \
--overwrite="no" \
--sessionids="HCPA001" \
--bolds="all" \
=================================================================
ERROR: No sessions specified to process. Please check your batch file, filtering options or sessionids parameter!
===> RUNNING run_turnkey step ~~~ HCP Pipelines - hcp_fmri_surface. BOLDS: all
........................ Running QuNex v0.99.2 ........................
--- Full QuNex call for command: hcp_fmri_surface
gmri hcp_fmri_surface --sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" --sessions="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt" --overwrite="no" --sessionids="HCPA001" --bolds="all"
---------------------------------------------------------
WARNING: passing the batchfile through the sessions parameter will be deprecated, please use the batchfile parameter!
WARNING: the sessionids parameter will be deprecated, please use the sessions parameter!
WARNING: Use of parameters with changed name(s)!
The following parameters have new names:
... TR is now tr!
Please correct the listed parameter names in command line or batch file!
WARNING: Parameter qx_cifti_tail was not specified. Its value was imputed from parameter hcp_cifti_tail and set to '_Atlas'!
WARNING: Parameter qx_nifti_tail was not specified. Its value was imputed from parameter hcp_nifti_tail and set to ''!
WARNING: Parameter cifti_tail was not specified. Its value was imputed from parameter qx_cifti_tail and set to '_Atlas'!
WARNING: Parameter nifti_tail was not specified. Its value was imputed from parameter qx_nifti_tail and set to ''!
# Generated by QuNex 0.99.2 on 2024-04-23_15.14.22.606243#
=================================================================
gmri hcp_fmri_surface \
--sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" \
--sessions="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/HCPA001_parameters.txt" \
--overwrite="no" \
--sessionids="HCPA001" \
--bolds="all" \
=================================================================
ERROR: No sessions specified to process. Please check your batch file, filtering options or sessionids parameter!
===> Appears some run_turnkey steps have failed.
Check /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs
Check /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/runlogs
Yes, it crashes immediately after create_study. There is a weird error regarding the mv command (which is a basic system command).
You could try adding
--bind="${WORK_DIR}:${WORK_DIR}" \
To the qunex_container call. Maybe the container does not have full access to the work dir.
It would be easier to debug if you do not go the run_turnkey route, but the command-by-command one. Try this:
export WORK_DIR="${HOME}/scratch/qunex"
export STUDY_FOLDER="${WORK_DIR}/quickstart"
export QUNEX_CONTAINER="${WORK_DIR}/qunex_suite-0.99.2d.sif"
export RAW_DATA="${WORK_DIR}/data"
export INPUT_BATCH_FILE="${RAW_DATA}/HCPA001_parameters.txt"
export INPUT_MAPPING_FILE="${RAW_DATA}/HCPA001_mapping.txt"
export SESSIONS="HCPA001"
# cleanup the old study
rm -rf ${STUDY_FOLDER}
# create a new study
qunex_container create_study \
--studyfolder="${STUDY_FOLDER}" \
--bind="${WORK_DIR}:${WORK_DIR}" \
--container="${QUNEX_CONTAINER}"
# onboard the data
qunex_container import_dicom \
--sessionsfolder="${STUDY_FOLDER}/sessions" \
--sessions="${SESSIONS}" \
--masterinbox="${RAW_DATA}" \
--archive="leave" \
--bind="${WORK_DIR}:${WORK_DIR}" \
--container="${QUNEX_CONTAINER}"
Add the scheduler parameter if needed. This should create the study and onboard the demo MRI session.
1 Like
Hi Jure
Thank you again for your assistance, I have ran the code as given.
# create a new study
qunex_container create_study \
--studyfolder="${STUDY_FOLDER}" \
--bind="${WORK_DIR}:${WORK_DIR}" \
--container="${QUNEX_CONTAINER}"
Returns and looks fine
[tqh20wsu@c0007 ~/scratch/qunex]$ qunex_container create_study --studyfolder="${STUDY_FOLDER}" --bind="${WORK_DIR}:${WORK_DIR}" --container="${QUNEX_CONTAINER}"
[tqh20wsu@c0007 ~/scratch/qunex]$ --> unsetting the following environment variables: PATH MATLABPATH PYTHONPATH QUNEXVer TOOLS QUNEXREPO QUNEXPATH QUNEXEXTENSIONS QUNEXLIBRARY QUNEXLIBRARYETC TemplateFolder FSL_FIXDIR FREESURFERDIR FREESURFER_HOME FREESURFER_SCHEDULER FreeSurferSchedulerDIR WORKBENCHDIR DCMNIIDIR DICMNIIDIR MATLABDIR MATLABBINDIR OCTAVEDIR OCTAVEPKGDIR OCTAVEBINDIR RDIR HCPWBDIR AFNIDIR PYLIBDIR FSLDIR FSLBINDIR PALMDIR QUNEXMCOMMAND HCPPIPEDIR CARET7DIR GRADUNWARPDIR HCPPIPEDIR_Templates HCPPIPEDIR_Bin HCPPIPEDIR_Config HCPPIPEDIR_PreFS HCPPIPEDIR_FS HCPPIPEDIR_PostFS HCPPIPEDIR_fMRISurf HCPPIPEDIR_fMRIVol HCPPIPEDIR_tfMRI HCPPIPEDIR_dMRI HCPPIPEDIR_dMRITract HCPPIPEDIR_Global HCPPIPEDIR_tfMRIAnalysis HCPCIFTIRWDIR MSMBin HCPPIPEDIR_dMRITractFull HCPPIPEDIR_dMRILegacy AutoPtxFolder EDDYCUDA USEOCTAVE QUNEXENV CONDADIR MSMBINDIR MSMCONFIGDIR R_LIBS FSL_FIX_CIFTIRW FSFAST_HOME SUBJECTS_DIR MINC_BIN_DIR MNI_DIR MINC_LIB_DIR MNI_DATAPATH FSF_OUTPUT_FORMAT ANTSDIR CUDIMOT
Generated by QuNex
------------------------------------------------------------------------
Version: 0.99.2
User: tqh20wsu
System: c0007.hpc.private
OS: RedHat Linux #1 SMP Thu Jan 25 16:17:31 UTC 2024
------------------------------------------------------------------------
██████\ ║ ██\ ██\
██ __██\ ║ ███\ ██ |
██ / ██ |██\ ██\ ║ ████\ ██ | ██████\ ██\ ██\
██ | ██ |██ | ██ | ║ ██ ██\██ |██ __██\\██\ ██ |
██ | ██ |██ | ██ | ║ ██ \████ |████████ |\████ /
██ ██\██ |██ | ██ | ║ ██ |\███ |██ ____|██ ██\
\██████ / \██████ | ║ ██ | \██ |\███████\██ /\██\
\___███\ \______/ ║ \__| \__| \_______\__/ \__|
\___| ║
DEVELOPED & MAINTAINED BY:
Anticevic Lab, Yale University
Mind & Brain Lab, University of Ljubljana
Murray Lab, Yale University
COPYRIGHT & LICENSE NOTICE:
Use of this software is subject to the terms and conditions defined in
'LICENSES' which is a part of the QuNex Suite source code package:
https://gitlab.qunex.yale.edu/qunex/qunex/-/tree/master/LICENSES
---> Setting up Octave
........................ Running QuNex v0.99.2 ........................
--- Full QuNex call for command: create_study
gmri create_study --studyfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart"
---------------------------------------------------------
started running create_study at 2024-04-23 16:11:45, track progress in /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/tmp_create_study_2024-04-23_16.11.45.359770.log
call: gmri create_study studyfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart"
-----------------------------------------
Running create_study
===================
Creating study folder structure:
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/analysis
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/analysis/scripts
... folder exists: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing
... folder exists: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/batchlogs
... folder exists: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/runlogs
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/runchecks
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/lists
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scripts
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes/QC
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes/QC/T1w
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes/QC/T2w
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes/QC/myelin
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes/QC/BOLD
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/scenes/QC/DWI
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/info
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/info/demographics
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/info/tasks
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/info/stimuli
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/info/bids
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/info/hcpls
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/MR
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/EEG
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/BIDS
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/HCPLS
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/behavior
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/concs
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/inbox/events
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/archive
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/archive/MR
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/archive/EEG
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/archive/BIDS
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/archive/HCPLS
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/archive/behavior
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/specs
... created: /gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions/QC
Preparing template files:
... created parameters_example.txt file
... created .qunexstudy file
-----------------------------------------
Finished at 2024-04-23 16:11:45
===> Successful completion of task
# onboard the data
qunex_container import_dicom \
--sessionsfolder="${STUDY_FOLDER}/sessions" \
--sessions="${SESSIONS}" \
--masterinbox="${RAW_DATA}" \
--archive="leave" \
--bind="${WORK_DIR}:${WORK_DIR}" \
--container="${QUNEX_CONTAINER}"
Returns
[tqh20wsu@c0007 ~/scratch/qunex]$ --> unsetting the following environment variables: PATH MATLABPATH PYTHONPATH QUNEXVer TOOLS QUNEXREPO QUNEXPATH QUNEXEXTENSIONS QUNEXLIBRARY QUNEXLIBRARYETC TemplateFolder FSL_FIXDIR FREESURFERDIR FREESURFER_HOME FREESURFER_SCHEDULER FreeSurferSchedulerDIR WORKBENCHDIR DCMNIIDIR DICMNIIDIR MATLABDIR MATLABBINDIR OCTAVEDIR OCTAVEPKGDIR OCTAVEBINDIR RDIR HCPWBDIR AFNIDIR PYLIBDIR FSLDIR FSLBINDIR PALMDIR QUNEXMCOMMAND HCPPIPEDIR CARET7DIR GRADUNWARPDIR HCPPIPEDIR_Templates HCPPIPEDIR_Bin HCPPIPEDIR_Config HCPPIPEDIR_PreFS HCPPIPEDIR_FS HCPPIPEDIR_PostFS HCPPIPEDIR_fMRISurf HCPPIPEDIR_fMRIVol HCPPIPEDIR_tfMRI HCPPIPEDIR_dMRI HCPPIPEDIR_dMRITract HCPPIPEDIR_Global HCPPIPEDIR_tfMRIAnalysis HCPCIFTIRWDIR MSMBin HCPPIPEDIR_dMRITractFull HCPPIPEDIR_dMRILegacy AutoPtxFolder EDDYCUDA USEOCTAVE QUNEXENV CONDADIR MSMBINDIR MSMCONFIGDIR R_LIBS FSL_FIX_CIFTIRW FSFAST_HOME SUBJECTS_DIR MINC_BIN_DIR MNI_DIR MINC_LIB_DIR MNI_DATAPATH FSF_OUTPUT_FORMAT ANTSDIR CUDIMOT
Generated by QuNex
------------------------------------------------------------------------
Version: 0.99.2
User: tqh20wsu
System: c0007.hpc.private
OS: RedHat Linux #1 SMP Thu Jan 25 16:17:31 UTC 2024
------------------------------------------------------------------------
██████\ ║ ██\ ██\
██ __██\ ║ ███\ ██ |
██ / ██ |██\ ██\ ║ ████\ ██ | ██████\ ██\ ██\
██ | ██ |██ | ██ | ║ ██ ██\██ |██ __██\\██\ ██ |
██ | ██ |██ | ██ | ║ ██ \████ |████████ |\████ /
██ ██\██ |██ | ██ | ║ ██ |\███ |██ ____|██ ██\
\██████ / \██████ | ║ ██ | \██ |\███████\██ /\██\
\___███\ \______/ ║ \__| \__| \_______\__/ \__|
\___| ║
DEVELOPED & MAINTAINED BY:
Anticevic Lab, Yale University
Mind & Brain Lab, University of Ljubljana
Murray Lab, Yale University
COPYRIGHT & LICENSE NOTICE:
Use of this software is subject to the terms and conditions defined in
'LICENSES' which is a part of the QuNex Suite source code package:
https://gitlab.qunex.yale.edu/qunex/qunex/-/tree/master/LICENSES
---> Setting up Octave
........................ Running QuNex v0.99.2 ........................
--- Full QuNex call for command: import_dicom
gmri import_dicom --sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" --masterinbox="/gpfs/home/tqh20wsu/scratch/qunex/data" --archive="leave" --sessions="HCPA001"
---------------------------------------------------------
started running import_dicom at 2024-04-23 16:14:14, track progress in /gpfs/home/tqh20wsu/scratch/qunex/quickstart/processing/logs/comlogs/tmp_import_dicom_2024-04-23_16.14.14.278521.log
call: gmri import_dicom sessionsfolder="/gpfs/home/tqh20wsu/scratch/qunex/quickstart/sessions" masterinbox="/gpfs/home/tqh20wsu/scratch/qunex/data" archive="leave" sessions="HCPA001"
-----------------------------------------
Running import_dicom
====================
---> Checking for packets in /gpfs/home/tqh20wsu/scratch/qunex/data
... using regular expression '(?P<packet_name>.*?)(?:\.zip$|\.tar$|\.tgz$|\.tar\..*$|$)'
... extracting subject id using regular expression '(?P<subject_id>.*)'
---> Found the following packets to process:
subject: HCPA001 ... HCPA001 <= HCPA001 <- HCPA001.zip
---> These packages do not match list of sessions and will be skipped:
subject: HCPA001_parameters.txt ... HCPA001_parameters.txt <= HCPA001_parameters.txt <- HCPA001_parameters.txt
subject: HCPA001_mapping.txt ... HCPA001_mapping.txt <= HCPA001_mapping.txt <- HCPA001_mapping.txt
---> Starting to process 1 packets ...
---=== PROCESSING HCPA001 ===---
Final report
============
Failed to process:
... HCPA001 [/gpfs/home/tqh20wsu/scratch/qunex/data/HCPA001.zip]
import_dicom: Zip file could not be processed
===> ERROR in completing import_dicom:
Some packages failed to process
Please check report!
-----------------------------------------
Finished at 2024-04-23 16:14:14
The correct file seems to be in the correct location
Thank you, Scott
It seems like the data download did not went through for some reason. The size of the HCPA001.zip session with all imaging data is only 2.4 kB, it should be 2.2 GB.
You can also try downloading by clicking this link: Google Drive - Virus scan warning.
This could be the reason why the run_turnkey approach failed at data onboarding as well.
Best, Jure
1 Like
Thank you, I downloaded the file seperatly, rather than using wget and it appears to be working
Great! Let us know if there are some issues further down the line.
1 Like
Hi Jure
I also need to run the BOLD resting state MCP pipeline, are the example calls here BOLD task-evoked and resting-state functional connectivity preprocessing — QuNex documentation applicable with the quickstart guide and example data?
Thank you, Scott
Hi,
Yes, the calls should be applicable. Maybe you will have to fine tune something for your data, but since this is resting state I do no think any major changes will be required. Once you are done with HCP processing, you use the map_hcp_data command to copy relevant data from HCP folders back into QuNex folders and then work on them. Let me know if you need some help here.
Best, Jure
Hi Jure
What do I need to run after map_hcp_data?
I want to run the HCP Generic fMRI Volume Preprocessing and Generic fMRI Surface Preprocessing, should this surfice?
qunex_container hcp_fmri_volume \
--batchfile="${BATCHFILE}" \
--sessionsfolder=”${SESSIONS}” \
--parsessions="10" \
--parelements="4" \
--container="${CONTAINER}"
Thank you, Scott
That seemed to work, but ran VERY fast, I assume because it thinks its already been compleated. (I tried again with --overwrite="yes" but was the same.
What folder do the outputted files get stored in?
Also, there isn’t a log file for this command, is that normal?
[tqh20wsu@c0010 ~]$ qunex_container hcp_fmri_volume \
> --batchfile="${BATCHFILE}" \
> --sessionsfolder=”${SESSIONS}” \
> --parsessions="10" \
> --parelements="4" \
> --container="${CONTAINER}" \
> --overwrite=”yes”
--> QuNex will run the command over 1 sessions. It will utilize:
Maximum sessions run in parallel for a job: 10.
Maximum elements run in parallel for a session: 4.
Up to 40 processes will be utilized for a job.
Job #1 will run sessions: HCPA001
[tqh20wsu@c0010 ~]$ --> unsetting the following environment variables: PATH MATLABPATH PYTHONPATH QUNEXVer TOOLS QUNEXREPO QUNEXPATH QUNEXEXTENSIONS QUNEXLIBRARY QUNEXLIBRARYETC TemplateFolder FSL_FIXDIR FREESURFERDIR FREESURFER_HOME FREESURFER_SCHEDULER FreeSurferSchedulerDIR WORKBENCHDIR DCMNIIDIR DICMNIIDIR MATLABDIR MATLABBINDIR OCTAVEDIR OCTAVEPKGDIR OCTAVEBINDIR RDIR HCPWBDIR AFNIDIR PYLIBDIR FSLDIR FSLBINDIR PALMDIR QUNEXMCOMMAND HCPPIPEDIR CARET7DIR GRADUNWARPDIR HCPPIPEDIR_Templates HCPPIPEDIR_Bin HCPPIPEDIR_Config HCPPIPEDIR_PreFS HCPPIPEDIR_FS HCPPIPEDIR_PostFS HCPPIPEDIR_fMRISurf HCPPIPEDIR_fMRIVol HCPPIPEDIR_tfMRI HCPPIPEDIR_dMRI HCPPIPEDIR_dMRITract HCPPIPEDIR_Global HCPPIPEDIR_tfMRIAnalysis HCPCIFTIRWDIR MSMBin HCPPIPEDIR_dMRITractFull HCPPIPEDIR_dMRILegacy AutoPtxFolder EDDYCUDA USEOCTAVE QUNEXENV CONDADIR MSMBINDIR MSMCONFIGDIR R_LIBS FSL_FIX_CIFTIRW FSFAST_HOME SUBJECTS_DIR MINC_BIN_DIR MNI_DIR MINC_LIB_DIR MNI_DATAPATH FSF_OUTPUT_FORMAT ANTSDIR CUDIMOT
Generated by QuNex
------------------------------------------------------------------------
Version: 0.99.2
User: tqh20wsu
System: c0010.hpc.private
OS: RedHat Linux #1 SMP Thu Jan 25 16:17:31 UTC 2024
------------------------------------------------------------------------
██████\ ║ ██\ ██\
██ __██\ ║ ███\ ██ |
██ / ██ |██\ ██\ ║ ████\ ██ | ██████\ ██\ ██\
██ | ██ |██ | ██ | ║ ██ ██\██ |██ __██\\██\ ██ |
██ | ██ |██ | ██ | ║ ██ \████ |████████ |\████ /
██ ██\██ |██ | ██ | ║ ██ |\███ |██ ____|██ ██\
\██████ / \██████ | ║ ██ | \██ |\███████\██ /\██\
\___███\ \______/ ║ \__| \__| \_______\__/ \__|
\___| ║
DEVELOPED & MAINTAINED BY:
Anticevic Lab, Yale University
Mind & Brain Lab, University of Ljubljana
Murray Lab, Yale University
COPYRIGHT & LICENSE NOTICE:
Use of this software is subject to the terms and conditions defined in
'LICENSES' which is a part of the QuNex Suite source code package:
https://gitlab.qunex.yale.edu/qunex/qunex/-/tree/master/LICENSES
---> Setting up Octave
........................ Running QuNex v0.99.2 ........................
--- Full QuNex call for command: hcp_fmri_volume
gmri hcp_fmri_volume --sessionsfolder="”HCPA001”" --parsessions="10" --parelements="4" --overwrite="”yes”" --batchfile="/gpfs/scratch/sami/qunex/quickstart/processing/batch.txt" --sessions="HCPA001"
---------------------------------------------------------
WARNING: Use of parameters with changed name(s)!
The following parameters have new names:
... TR is now tr!
Please correct the listed parameter names in command line or batch file!
WARNING: Parameter qx_cifti_tail was not specified. Its value was imputed from parameter hcp_cifti_tail and set to '_Atlas'!
WARNING: Parameter qx_nifti_tail was not specified. Its value was imputed from parameter hcp_nifti_tail and set to ''!
WARNING: Parameter cifti_tail was not specified. Its value was imputed from parameter qx_cifti_tail and set to '_Atlas'!
WARNING: Parameter nifti_tail was not specified. Its value was imputed from parameter qx_nifti_tail and set to ''!
# Generated by QuNex 0.99.2 on 2024-05-03_14.41.03.190306#
=================================================================
gmri hcp_fmri_volume \
--sessionsfolder="”HCPA001”" \
--parsessions="10" \
--parelements="4" \
--overwrite="”yes”" \
--sessions="/gpfs/scratch/sami/qunex/quickstart/processing/batch.txt" \
--sessionids="HCPA001" \
=================================================================
Starting multiprocessing sessions in /gpfs/scratch/sami/qunex/quickstart/processing/batch.txt with a pool of 10 concurrent processes
---- Running local
Adding processing of session HCPA001 to the pool at Friday, 03. May 2024 14:41:03
=======================================================================================================================================
Running external command: /opt/HCP/HCPpipelines/fMRIVolume/GenericfMRIVolumeProcessingPipeline.sh --path="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp" --subject="HCPA001" --fmriname="1" --fmritcs="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/BOLD_1_AP/HCPA001_BOLD_1_AP.nii.gz" --fmriscout="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/BOLD_1_AP_SBRef/HCPA001_BOLD_1_AP_SBRef.nii.gz" --SEPhaseNeg="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/SpinEchoFieldMap1/HCPA001_BOLD_AP_SB_SE.nii.gz" --SEPhasePos="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/SpinEchoFieldMap1/HCPA001_BOLD_PA_SB_SE.nii.gz" --fmapmag="NONE" --fmapphase="NONE" --fmapgeneralelectric="NONE" --echospacing="0.000580009" --echodiff="NONE" --unwarpdir="y-" --fmrires="2" --dcmethod="TOPUP" --biascorrection="NONE" --gdcoeffs="NONE" --topupconfig="/opt/HCP/HCPpipelines/global/config/b02b0.cnf" --mctype="MCFLIRT" --processing-mode="HCPStyleData" --doslicetime="FALSE"
=======================================================================================================================================
Running external command: /opt/HCP/HCPpipelines/fMRIVolume/GenericfMRIVolumeProcessingPipeline.sh --path="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp" --subject="HCPA001" --fmriname="2" --fmritcs="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/BOLD_2_PA/HCPA001_BOLD_2_PA.nii.gz" --fmriscout="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/BOLD_2_PA_SBRef/HCPA001_BOLD_2_PA_SBRef.nii.gz" --SEPhaseNeg="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/SpinEchoFieldMap1/HCPA001_BOLD_AP_SB_SE.nii.gz" --SEPhasePos="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/SpinEchoFieldMap1/HCPA001_BOLD_PA_SB_SE.nii.gz" --fmapmag="NONE" --fmapphase="NONE" --fmapgeneralelectric="NONE" --echospacing="0.000580009" --echodiff="NONE" --unwarpdir="y" --fmrires="2" --dcmethod="TOPUP" --biascorrection="NONE" --gdcoeffs="NONE" --topupconfig="/opt/HCP/HCPpipelines/global/config/b02b0.cnf" --mctype="MCFLIRT" --processing-mode="HCPStyleData" --doslicetime="FALSE"
------------------------------------------------------------
Session id: HCPA001
[started on Friday, 03. May 2024 14:41:03]
Running HCP fMRI Volume pipeline [HCPStyleData] ...
---> PreFS results present.
---> FS results present.
---> PostFS results present.
---> Looking for spin echo fieldmap set images [PA/AP].
... identified folder SpinEchoFieldMap1
... phase negative AP spin echo fieldmap image present
... phase positive PA spin echo fieldmap image present
... identified folder SpinEchoFieldMap2
... phase negative AP spin echo fieldmap image present
... phase positive PA spin echo fieldmap image present
... TOPUP configuration file present
---> Preprocessing settings (unwarpdir, refimage, moveref, seimage) for BOLD 1
... phase encoding direction: AP
... unwarp direction: y-
... using study general EchoSpacing: 0.000580009 s
... using spin echo fieldmap set 1
-> SE Positive image : HCPA001_BOLD_PA_SB_SE.nii.gz
-> SE Negative image : HCPA001_BOLD_AP_SB_SE.nii.gz
... bold image present
... reference image present
... using the HCPpipelines default BOLD mask
---> Preprocessing settings (unwarpdir, refimage, moveref, seimage) for BOLD 2
... phase encoding direction: PA
... unwarp direction: y
... using study general EchoSpacing: 0.000580009 s
... using spin echo fieldmap set 1
-> SE Positive image : HCPA001_BOLD_PA_SB_SE.nii.gz
-> SE Negative image : HCPA001_BOLD_AP_SB_SE.nii.gz
... bold image present
... reference image present
... using the HCPpipelines default BOLD mask
Running 2 BOLD images in parallel
------------------------------------------------------------
Running HCP Pipelines command via QuNex:
/opt/HCP/HCPpipelines/fMRIVolume/GenericfMRIVolumeProcessingPipeline.sh
--path="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp"
--subject="HCPA001"
--fmriname="1"
--fmritcs="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/BOLD_1_AP/HCPA001_BOLD_1_AP.nii.gz"
--fmriscout="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/BOLD_1_AP_SBRef/HCPA001_BOLD_1_AP_SBRef.nii.gz"
--SEPhaseNeg="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/SpinEchoFieldMap1/HCPA001_BOLD_AP_SB_SE.nii.gz"
--SEPhasePos="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/SpinEchoFieldMap1/HCPA001_BOLD_PA_SB_SE.nii.gz"
--fmapmag="NONE"
--fmapphase="NONE"
--fmapgeneralelectric="NONE"
--echospacing="0.000580009"
--echodiff="NONE"
--unwarpdir="y-"
--fmrires="2"
--dcmethod="TOPUP"
--biascorrection="NONE"
--gdcoeffs="NONE"
--topupconfig="/opt/HCP/HCPpipelines/global/config/b02b0.cnf"
--mctype="MCFLIRT"
--processing-mode="HCPStyleData"
--doslicetime="FALSE"
------------------------------------------------------------
Running HCP fMRIVolume --- already completed
------------------------------------------------------------
Running HCP Pipelines command via QuNex:
/opt/HCP/HCPpipelines/fMRIVolume/GenericfMRIVolumeProcessingPipeline.sh
--path="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp"
--subject="HCPA001"
--fmriname="2"
--fmritcs="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/BOLD_2_PA/HCPA001_BOLD_2_PA.nii.gz"
--fmriscout="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/BOLD_2_PA_SBRef/HCPA001_BOLD_2_PA_SBRef.nii.gz"
--SEPhaseNeg="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/SpinEchoFieldMap1/HCPA001_BOLD_AP_SB_SE.nii.gz"
--SEPhasePos="/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/unprocessed/SpinEchoFieldMap1/HCPA001_BOLD_PA_SB_SE.nii.gz"
--fmapmag="NONE"
--fmapphase="NONE"
--fmapgeneralelectric="NONE"
--echospacing="0.000580009"
--echodiff="NONE"
--unwarpdir="y"
--fmrires="2"
--dcmethod="TOPUP"
--biascorrection="NONE"
--gdcoeffs="NONE"
--topupconfig="/opt/HCP/HCPpipelines/global/config/b02b0.cnf"
--mctype="MCFLIRT"
--processing-mode="HCPStyleData"
--doslicetime="FALSE"
------------------------------------------------------------
Running HCP fMRIVolume --- already completed
HCP fMRIVolume completed on Friday, 03. May 2024 14:41:03
------------------------------------------------------------
===> Final report for command hcp_fmri_volume
... HCPA001 ---> HCP fMRI Volume: bolds 1, 2 done
===> Successful completion of all tasks
Thank you, Scott
Looks like hcp_fmri_volume ran equally fast.
I think I have found the output I need:
/gpfs/scratch/sami/qunex/quickstart/sessions/HCPA001/hcp/HCPA001/MNINonLinear/Results/1/1_Atlas.dtseries.nii
As the naming conventions are different, as when I run the HCP pipeline dirrectly I get:
/gpfs/home/tqh20wsu/scratch/projects/Pipelines_ExampleData/100307/MNINonLinear/Results/rfMRI_REST1_LR/rfMRI_REST1_LR_Atlas.dtseries.nii
I assume the rfMRI_REST``_LR is omitted for brevity?
Thank you, Scott
QuNex creates two kinds of logs: runlogs are shorter logs, comlogs are logs from longer and external commands. In your case, it seems like HCP Pipelines were not triggered so there is no comlog but there should be a runlog which will let you know what happened. It looks like results already exists so re-running makes no sense, unless you force QuNex to rerune the HCP command with overwrite.
There are two file naming conventions in QuNex, the short one (automated) and the long one (userdefined). automated is the default, there image names are shortened and numerical. If you set the parameter to userdefined you get the longer, more verbose naming. This is tweaked through the hcp_filename parameter:
--hcp_filename (str):
Specify how files and folders should be named using HCP processing:
- 'automated' ... files should be named using QuNex automated naming
(e.g. BOLD_1_PA)
- 'userdefined' ... files should be named using user defined names (e.g.
rfMRI_REST1_AP)
Note that you need to set this from the start and stick to it, since commands are chained and expect the inputs in a given format.
Best, Jure
