[RESOLVED] Trouble with running hcp_icafix

ehui · April 8, 2024, 2:48am

Dear Jure:

I was trying to run hcp_icafix using the following commands, but failed to do so:

qunex_container hcp_icafix \
                    --paramfile="${INPUT_BATCH_FILE}"   \
                    --batchfile="${INPUT_BATCH_FILE}"   \
                    --path="${WORK_DIR}/${STUDY_NAME}"   \
                    --sessions="${SESSIONS[$i]}"   \
                    --sessionids="${SESSIONS[$i]}"   \
                    --sessionsfolder="${WORK_DIR}/${STUDY_NAME}/sessions" \
                    --hcp_icafix_bolds="rest" \
                    --bind="/hdd1:/hdd1" \
                    --container="${QUNEX_CONTAINER}";

Below is the outputs:

Generated by QuNex
------------------------------------------------------------------------
Version: 0.99.2
User: ehui
System: ehuicompute
OS: RedHat Linux #110~20.04.1-Ubuntu SMP Tue Feb 13 14:25:03 UTC 2024
------------------------------------------------------------------------

        ______\                  _      __\   __\
       __  ____\                 _      ___\  __ |
       __ /  __ |__\   __\       _      ____\ __ | ______\ __\   __\
       __ |  __ |__ |  __ |      _      __ __\__ |__  ____\\__\ __  |
       __ |  __ |__ |  __ |      _      __ \____ |________ |\____  /
       __ __\__ |__ |  __ |      _      __ |\___ |__   ____|__  __\
       \______ / \______  |      _      __ | \__ |\_______\__  /\__\
        \______\  \______/       _      \__|  \__| \_______\__/  \__|
            \___|                _


                       DEVELOPED & MAINTAINED BY:

                    Anticevic Lab, Yale University
               Mind & Brain Lab, University of Ljubljana
                     Murray Lab, Yale University

                      COPYRIGHT & LICENSE NOTICE:

Use of this software is subject to the terms and conditions defined in
'LICENSES' which is a part of the QuNex Suite source code package:
https://gitlab.qunex.yale.edu/qunex/qunex/-/tree/master/LICENSES

 ---> Setting up Octave


 ........................ Running QuNex v0.99.2 ........................


--- Full QuNex call for command: hcp_icafix

gmri hcp_icafix --paramfile="/hdd1/qunex/data/025C_batch.txt" --path="/hdd1/qunex/cimt" --sessionsfolder="/hdd1/qunex/cimt/sessions" --hcp_icafix_bolds="rest" --batchfile="/hdd1/qunex/data/025C_batch.txt" --sessions="025C"

---------------------------------------------------------



WARNING: Use of parameters with changed name(s)!
         The following parameters have new names:
         ... TR is now tr!
         Please correct the listed parameter names in command line or batch file!
WARNING: Parameter qx_cifti_tail was not specified. Its value was imputed from parameter hcp_cifti_tail and set to '_Atlas'!
WARNING: Parameter qx_nifti_tail was not specified. Its value was imputed from parameter hcp_nifti_tail and set to ''!
WARNING: Parameter cifti_tail was not specified. Its value was imputed from parameter qx_cifti_tail and set to '_Atlas'!
WARNING: Parameter nifti_tail was not specified. Its value was imputed from parameter qx_nifti_tail and set to ''!
# Generated by QuNex 0.99.2 on 2024-04-08_10.39.22.720163#
=================================================================
gmri hcp_icafix \
  --paramfile="/hdd1/qunex/data/025C_batch.txt" \
  --path="/hdd1/qunex/cimt" \
  --sessionsfolder="/hdd1/qunex/cimt/sessions" \
  --hcp_icafix_bolds="rest" \
  --sessions="/hdd1/qunex/data/025C_batch.txt" \
  --sessionids="025C" \
=================================================================

ERROR: No sessions specified to process. Please check your batch file, filtering options or sessionids parameter!

Your help would be greatly appreciated as always!

Best,
Ed

demsarjure · April 8, 2024, 5:57am

Hi Ed,

It looks like you are using some parameters and values that are run_turnkey specific. Try this:

qunex_container hcp_icafix \
  --batchfile="${WORK_DIR}/${STUDY_NAME}/batch.txt"   \
  --sessionsfolder="${WORK_DIR}/${STUDY_NAME}/sessions" \
  --hcp_icafix_bolds="rest" \
  --bind="/hdd1:/hdd1" \
  --container="${QUNEX_CONTAINER}"

This will execute across all sessions in the ${WORK_DIR}/${STUDY_NAME}/batch.txt file. If you want to subset, you can also add the --sessions parameter. If this does not work, please post the contents of ${WORK_DIR}/${STUDY_NAME}/batch.txt here.

Best, Jure

ehui · April 8, 2024, 6:25am

Hi Jure:

Many thanks for your swift response. After using your commands, I got the following error messages instead:

Generated by QuNex
------------------------------------------------------------------------
Version: 0.99.2
User: ehui
System: ehuicompute
OS: RedHat Linux #110~20.04.1-Ubuntu SMP Tue Feb 13 14:25:03 UTC 2024
------------------------------------------------------------------------

        ______\                  _      __\   __\
       __  ____\                 _      ___\  __ |
       __ /  __ |__\   __\       _      ____\ __ | ______\ __\   __\
       __ |  __ |__ |  __ |      _      __ __\__ |__  ____\\__\ __  |
       __ |  __ |__ |  __ |      _      __ \____ |________ |\____  /
       __ __\__ |__ |  __ |      _      __ |\___ |__   ____|__  __\
       \______ / \______  |      _      __ | \__ |\_______\__  /\__\
        \______\  \______/       _      \__|  \__| \_______\__/  \__|
            \___|                _


                       DEVELOPED & MAINTAINED BY:

                    Anticevic Lab, Yale University
               Mind & Brain Lab, University of Ljubljana
                     Murray Lab, Yale University

                      COPYRIGHT & LICENSE NOTICE:

Use of this software is subject to the terms and conditions defined in
'LICENSES' which is a part of the QuNex Suite source code package:
https://gitlab.qunex.yale.edu/qunex/qunex/-/tree/master/LICENSES

 ---> Setting up Octave


 ........................ Running QuNex v0.99.2 ........................


--- Full QuNex call for command: hcp_icafix

gmri hcp_icafix --/hdd1/qunex/data/025C_batch.txt --sessionsfolder="/hdd1/qunex/cimt/sessions" --hcp_icafix_bolds="rest" --sessions="025C"

---------------------------------------------------------


WARNING: Parameter qx_cifti_tail was not specified. Its value was imputed from parameter hcp_cifti_tail and set to '_Atlas'!
WARNING: Parameter qx_nifti_tail was not specified. Its value was imputed from parameter hcp_nifti_tail and set to ''!
WARNING: Parameter cifti_tail was not specified. Its value was imputed from parameter qx_cifti_tail and set to '_Atlas'!
WARNING: Parameter nifti_tail was not specified. Its value was imputed from parameter qx_nifti_tail and set to ''!
# Generated by QuNex 0.99.2 on 2024-04-08_14.21.55.387830#
=================================================================
gmri hcp_icafix \
  --/hdd1/qunex/data/025C_batch.txt="True" \
  --sessionsfolder="/hdd1/qunex/cimt/sessions" \
  --hcp_icafix_bolds="rest" \
  --sessions="025C" \
=================================================================

Starting multiprocessing sessions in 025C with a pool of 1 concurrent processes

---- Running local

Starting processing of sessions 025C at Monday, 08. April 2024 14:21:55
ERROR: HCP path does not exists, check your parameters and the batch file!

------------------------------------------------------------
Session id: 025C
[started on Monday, 08. April 2024 14:21:55]
Running HCP ICAFix pipeline [HCPStyleData] ...
ERROR: Unknown error occured:
...................................
Traceback (most recent call last):
  File "/opt/qunex/python/qx_utilities/hcp/process_hcp.py", line 5401, in hcp_icafix
    hcp = getHCPPaths(sinfo, options)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/qunex/python/qx_utilities/hcp/process_hcp.py", line 136, in getHCPPaths
    raise
RuntimeError: No active exception to reraise
...................................


HCP ICAFix completed on Monday, 08. April 2024 14:21:55
------------------------------------------------------------


===> Final report for command hcp_icafix
... 025C ---> HCP ICAFix failed
===> Success status not reported for some or all tasks

The content of the batch file are:

# -------------------------------------------------
# -- Qu|Nex Environment Preprocessing Parameters --
# -------------------------------------------------

# -------------------------------------
# ---> HCP Pipelines general parameters
# -------------------------------------
_hcp_Pipeline            : ${HCPPIPEDIR}
_parsessions             : 1
_parelements             : 2

# -------------------------------------
# ---> Pre FreeSurfer parameters
# -------------------------------------
_hcp_suffix             :
_hcp_brainsize          : 150
_hcp_t2                 : t2
_hcp_fmap               : NONE
_hcp_t1samplespacing    : 0.0000071
_hcp_t2samplespacing    : 0.0000021
_hcp_unwarpdir          : z
_hcp_avgrdcmethod       : TOPUP
_hcp_topupconfig        : b02b0.cnf
_hcp_printcom           :
_hcp_sephaseneg         : AP
_hcp_sephasepos         : PA
_hcp_seechospacing      : 0.000580009
_hcp_seunwarpdir        : y


# -------------------------------------
# ---> FreeSurfer parameters
# -------------------------------------
_hcp_freesurfer_home    : ${FREESURFER_HOME}


# -------------------------------------
# ---> Post FreeSurfer parameters
# -------------------------------------
_hcp_regname            : MSMSulc
_hcp_grayordinatesres   : 2
_hcp_hiresmesh          : 164
_hcp_lowresmesh         : 32


# -------------------------------------
# ---> Diffusion preprocessing - N/A
# -------------------------------------
_hcp_dwi_echospacing    : 0.689998
_hcp_dwi_PEdir          : 2


# -------------------------------------
# ---> fMRI Volume parameters
# -------------------------------------
_hcp_bold_echospacing   : 0.000580009
_hcp_bold_echodiff      : NONE
_hcp_bold_dcmethod      : TOPUP
_hcp_bold_topupconfig   : b02b0.cnf
_hcp_bold_sbref         : NONE
_hcp_bold_sephaseneg    : AP
_hcp_bold_sephasepos    : PA
_hcp_bold_unwarpdir     : PA=y|AP=y-
_hcp_bold_res           : 2
_hcp_bold_gdcoeffs      : NONE
_hcp_bold_doslicetime   : FALSE
_hcp_bold_seimg         : independent
_hcp_bold_movref        : independent
_hcp_bold_prefix        :


# -------------------------------------
# ---> fMRI Surface parameters
# -------------------------------------
_hcp_bold_smoothFWHM    : 2


# -------------------------------------
# ---> Further BOLD analyses parameters
# -------------------------------------
_image_source       : hcp
_image_target       : cifti
_bolds              : rest
_TR                 : 0.8
_omit               : 0
_pignore            : hipass=linear|regress=spline|lopass=linear
_log                : keep
_glm_matrix         : both
_glm_residuals      : save

   # # ==========  _bold_actions  ==============
   #
   # m ... Motion scrubbing.
   # s ... Spatial smooting.
   # h ... High-pass filtering.
   # r ... Regression (nuisance and/or task) with an optional number 0, 1, or 2
   #       specifying the type of regression to use (see REGRESSION below).
   # c ... Saving of resulting beta coefficients (allways to follow 'r').
   # l ... Low-pass filtering.
   #
   # =========================================

_bold_actions       : shrcl


# -------------------------------------
# ---> BOLD de-noising parameters
# -------------------------------------
_bold_nuisance      : m,V,WM,WB
_event_file         :
_nprocess           : 0
_bold_prefix        :
_event_string       :
_event_file         :
_boldname           : bold
_shrinknsroi        : true


# -------------------------------------
# ---> CIFTI mapping naming variable
# -------------------------------------
_hcp_cifti_tail      : _Atlas


# -------------------------------------
# ---> Smoothing parameters
# -------------------------------------
_surface_smooth      : 2
_volume_smooth       : 2
_voxel_smooth        : 1
_smooth_mask         : nonzero
_dilate_mask         : ${TOOLS}/${QUNEXREPO}/library/data/atlases/MNITemplates/MNI152_T1_2mm_brain_mask.nii.gz


# -------------------------------------
# ---> Movement scrubbing parameters
# -------------------------------------
_mov_dvars:         3.0
_mov_dvarsme:       1.6
_mov_fd:            0.5
_mov_radius:        50.0
_mov_scrub:         yes
_mov_fidl:          udvarsme
_mov_plot:          mov_report
_mov_post:          udvarsme
_mov_before:        0
_mov_after:         0
_mov_bad:           udvarsme
_mov_mreport:       movement_report.txt
_mov_preport:       movement_report_post.txt
_mov_sreport:       movement_scrubbing_report.txt
_mov_pdf:           movement_plots


# -------------------------------------------
# ---> Diffusion analysis-related parameters
# -------------------------------------------
_kurt:              yes
_kurtdir:           yes
_omatrix3:          yes
_matrixversion:     3
_waytotal:          standard
_parcellationfile:  ${HOME}/projects/Pipeline/global/templates/ColeAnticevicNetPartition/CortexSubcortex_ColeAnticevic_NetPartition_wSubcorGSR_parcels_LR_ReorderedByNetworks.dlabel.nii"
_outname:           parcellated

Many thanks for your help!

Best
Ed

demsarjure · April 8, 2024, 11:15am

The batchfile is missing session info. It seems like you missed some commands that prepare the study for HCP processing. See Option B: Step-by-step execution at QuNex quick start using a Docker container — QuNex documentation for details. The chain of commands that was not executed or did not work is create_session_info, setup_hcp and create_batch.

Did you run HCP structural and functional preprocessin in this study? If you did, you had to use a different batch files since it worked.

ehui · April 8, 2024, 2:27pm

Hi Jure:

I did run all the HCP structural and functional preprocessing. Now I see which batchfile you are referring to.

hcp_icafix now runs! Thanks so much!

Best,
Ed

demsarjure · April 9, 2024, 6:41am

Great, glad it worked.

Best, Jure