[RESOLVED] HCP TransmitBias pipeline

Use Questions
21

Hi Jure,

I just wanted to follow up quickly on my previous post and also ask for some guidance.

First, I’ve prepared an FSL-based script that can “repair” faulty rfmap’s NIFTI generated by dcm2niix. This eliminates the need to open MATLAB just to create usable rfmaps (separate phase and magnitude). You can find the script here:
:link: repair_rfmap.sh

Second, I’ve also created a YAML config to enable batch processing for all my subjects (attached below my post). My question is about how to properly separate the loading and HCP processing stages when working with an asterisk session pattern. For example, I’m trying to use a command like this:

qunex_container run_recipe \
  --recipe_file=./myelin.yaml \
  --recipe="hcp_preprocess" \
  --bash_post="${BASH_POST}" \
  --bind="${HOME}:${HOME}" \
  --container="${QUNEX_CONTAINER}" \
  --sessions="KJ0*"

The issue is that when I try to run only the hcp_preprocess stage, the batch.txt file in my session’s directory contains only a single session/subject, and the process doesn’t pick up the rest of the sessions matching KJ0*. This results in the following error:

...
qunex hcp_pre_freesurfer \
  --hcp_avgrdcmethod="NONE" \
  --sessions="/home/bkossows/qunex/kj23a/processing/batch.txt" \
  --studyfolder="/home/bkossows/qunex/kj23a" \
  --sessionsfolder="/home/bkossows/qunex/kj23a/sessions3" \
  --overwrite="yes" \
  --sessionids="KJ0*"

=================================================================

ERROR: No sessions specified to process. Please check your batch file, filtering options or sessionids parameter!

Do you have any suggestions on how to structure the recipe or call so that it properly applies the processing stage to all sessions matching KJ0*?

Thanks in advance!

Best,
Bartosz Kossowski

global_parameters:
  studyfolder     : "{{$STUDY_FOLDER}}"
  sessionsfolder: "{{$STUDY_FOLDER}}/sessions3"
  sessions: "{{$SESSIONS}}"
  batchfile: "{{$STUDY_FOLDER}}/processing/batch.txt"
  overwrite: "yes"

recipes:
  import_and_setup:
    commands:
      - import_bids:
          inbox: "{{$RAW_DATA}}"
          action: copy
          archive: leave
          fileinfo: full
      - create_session_info:
          mapping: "{{$INPUT_MAPPING_FILE}}"
      - create_batch:
          targetfile: "{{$STUDY_FOLDER}}/processing/batch.txt"
          paramfile: "{{$INPUT_BATCH_FILE}}"
      - setup_hcp

  hcp_preprocess:
    commands:
      - hcp_pre_freesurfer:
          hcp_avgrdcmethod: NONE
      - hcp_freesurfer
      - hcp_post_freesurfer
      - hcp_transmit_bias_individual:
          hcp_group_corrected_myelin: /home/bkossows/qunex/Partial.MyelinMap_GroupCorr_MSMAll.32k_fs_LR.dscalar.nii
          hcp_transmit_mode: B1Tx
          hcp_gmwm_template: /home/bkossows/qunex/new-repo/sessions2/KJ001_01/hcp/KJ001_01/MNINonLinear/WMGM.nii.gz
          hcp_matlab_mode: compiled
22

Hi Bartosz,

So, the issue is that the batch file contains only 1 session, probably the last one? You need to set it up like this:

      - create_batch:
          targetfile: "{{$STUDY_FOLDER}}/processing/batch.txt"
          paramfile: "{{$INPUT_BATCH_FILE}}"
          overwrite: "append"

Currently, the global overwrite: "yes" is used, which means that each create_batch invocation overrides the previous one.

Best, Jure

23

Hi again :slight_smile:

Running the hcp_transmit_bias_individual step works as expected in QuNex 1.3.0, but fails in QuNex 1.4.2 with an error indicating that the hcp_b1tx_phase parameter is missing and cannot be auto-detected.

The same recipe, step, session, and parameters are used in both versions.

Scenario 1: QuNex 1.3.0 (working)

qunex_container run_recipe \
  --container="qunex_suite-1.3.0.sif" \
  --recipe_file=./t1t2.yaml \
  --recipe="hcp_preprocess" \
  --steps="hcp_transmit_bias_individual"

Command executed internally:

qunex hcp_transmit_bias_individual \
  --hcp_group_corrected_myelin="/home/jovyan/qunex/Partial.MyelinMap_GroupCorr_MSMAll.32k_fs_LR.dscalar.nii" \
  --hcp_transmit_mode="B1Tx" \
  --hcp_gmwm_template="/home/jovyan/qunex/sessions/KJ001_01/hcp/KJ001_01/MNINonLinear/WMGM.nii.gz" \
  --hcp_matlab_mode="compiled" \
  --studyfolder="/home/jovyan/qunex/" \
  --sessionsfolder="/home/jovyan/qunex/sessions" \
  --sessions="KJ001_01" \
  --batchfile="/home/jovyan/qunex/processing/batch.txt" \
  --overwrite="yes"

:white_check_mark: The pipeline completes successfully.

Scenario 2: QuNex 1.4.2 (failing)

I had some problems with running qunex_container script. I decided to work within the container.

singularity run qunex_suite-1.4.2.sif


qunex.sh run_recipe \
  --recipe_file=./t1t2.yaml \
  --recipe="hcp_preprocess" \
  --steps="hcp_transmit_bias_individual"

Command executed internally (identical to 1.3.0):

qunex hcp_transmit_bias_individual \
  --hcp_group_corrected_myelin="/home/jovyan/qunex/Partial.MyelinMap_GroupCorr_MSMAll.32k_fs_LR.dscalar.nii" \
  --hcp_transmit_mode="B1Tx" \
  --hcp_gmwm_template="/home/jovyan/qunex/sessions/KJ001_01/hcp/KJ001_01/MNINonLinear/WMGM.nii.gz" \
  --hcp_matlab_mode="compiled" \
  --studyfolder="/home/jovyan/qunex/" \
  --sessionsfolder="/home/jovyan/qunex/sessions" \
  --sessions="KJ001_01" \
  --batchfile="/home/jovyan/qunex/processing/batch.txt" \
  --overwrite="yes"

Error output

Running HCP Transmit Bias Individual Only Pipeline [HCPStyleData] ...

---> Setting the hcp_b1tx_magnitude automatically

---> Setting the hcp_b1tx_phase automatically

---> ERROR: the hcp_b1tx_phase parameter is not provided, and QuNex cannot find the b1tx phase image in the HCP unprocessed/B1 folder!

---> Session cannot be processed.

Happy to provide additional logs or directory listings if helpful.
B.

24

Hi,

Thanks for reporting this, a bug sneaked into our code.

Sometimes the *_TB1TFL-Phase.nii.gz is located in the T1w folder and not the B1 folder as is the default. 1.3.0 looks only in the B1 folder, while from 1.4.0 onwards it is supposed to be looking into B1 and T1w, but it is instead looking into T1w folder twice…

I just fixed this and will be realesed in the next version.

The current workaround for processing with the latest (1.4.3) is tocopy or symlink the _TB1TFL-Phase.nii.gz image from unprocessed/B1 into the T1w in unprocessed inside the hcp folder structure. You can also set --hcp_b1tx_phase parameter and give it the absolute path to the image.

Best, Jure