[RESOLVED] BIDS IntendedFor not recognized?

Issues
1

Hi,

I’m trying to map BIDS data to QuNex (0.100.0).
The dataset contains several fieldmaps, some of which are spin-echo pairs of opposing phase-encoding directions while others are magnitude-phase pairs. We’ve carefully worked with the IntendedFor-parameter in the BIDS-metadata to ensure that fieldmaps are used as intended.

However, session.txt produced by import_bids (example attached) does not reflect this mapping at all. There are just two fieldmap-tags visible in session.txt (se(001) and fm(001)). However as you can see in sessions.txt there are two magnitude-phasediff fmaps (index 5-10) and many SE-fmaps (index 11-18). Also, several places the fieldmap tag gets repeated within a line, e.g:

23: bold eatlift1enc run-001 :fm(001):fm(001):fm(001) :UnwarpDir(y-):phenc(AP):EchoSpacing(0.000265002):DwellTime(3e-06)

Any idea what’s happening here? I had a quick look at the code (bids.py, line 1209++ in the 0.100.0 release) and I have a vague suspicion this may be due to how the`variable tag is defined (the assumption seems to be that the “_run”-field is always used to differentiate between fieldmaps while we’ve been using the “_acq” field). But wanted to check in the developers before digging deeper.

session.txt (3.8 KB)

Thanks!
Cheers
Markus

Pasting my invocation of QuNex for reference.

QUNEX_VERSION="0.100.0"
export PATH=${PATH}:/projects/ec201/workspace/qunex/qunexcontainer
export QUNEX_CONTAINER=/projects/ec201/workspace/qunex/qunexcontainer/qunex_suite-${QUNEX_VERSION}.sif
sub=1510121
ses=20231102p15w03s13
waveid=${ses: -9}
studyfolder="/projects/ec201/data/lcbc/processed/qunex"

qunex_container create_study \
--bind="${studyfolder}:${studyfolder}" \
--studyfolder="${studyfolder}/sub-${sub}/ses-${ses}" \
--container="${QUNEX_CONTAINER}"

qunex_container import_bids \
--bind="${studyfolder}:${studyfolder},${studyfolder}/bids_tmp:/bids" \
--sessionsfolder="${studyfolder}/sub-${sub}/ses-${ses}/sessions" \
--inbox="/bids/sub-${sub}/ses-${ses}" \
--overwrite='yes' \
--container="${QUNEX_CONTAINER}"

And the log following import_bids

 ........................ Running QuNex v0.100.0 [QX IO] ........................


--- Full QuNex call for command: import_bids

qunex import_bids --sessionsfolder="/projects/ec201/data/lcbc/processed/qunex/sub-1510121/ses-20231102p15w03s13/sessions" --inbox="/bids/sub-1510121/ses-20231102p15w03s13" --overwrite="yes"

---------------------------------------------------------


started running import_bids at 2024-10-29 14:44:52, track progress in /projects/ec201/data/lcbc/processed/qunex/sub-1510121/ses-20231102p15w03s13/processing/logs/comlogs/tmp_import_bids_2024-10-29_14.44.52.626845.log
call: gmri import_bids sessionsfolder="/projects/ec201/data/lcbc/processed/qunex/sub-1510121/ses-20231102p15w03s13/sessions" inbox="/bids/sub-1510121/ses-20231102p15w03s13" overwrite="yes"
-----------------------------------------
Running import_bids
==================
---> identifying files in /bids/sub-1510121/ses-20231102p15w03s13
---> Inbox type: session
---> Paths:
    bidsinfo    -> /projects/ec201/data/lcbc/processed/qunex/sub-1510121/ses-20231102p15w03s13/info/bids
    bidsinbox   -> /projects/ec201/data/lcbc/processed/qunex/sub-1510121/ses-20231102p15w03s13/sessions/inbox/BIDS
    bidsarchive -> /projects/ec201/data/lcbc/processed/qunex/sub-1510121/ses-20231102p15w03s13/sessions/archive/BIDS
---> mapping files to QuNex bids folders
    ---> creating bids session 1510121_20231102p15w03s13
---> Archiving: moving items
Running map_bids2nii for subject 1510121, session 20231102p15w03s13
===================================================================
---> linked 1.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-mprage_run-001_T1w.nii.gz
---> linked 2.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-space_run-001_T2w.nii.gz
---> linked 3.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-tsehippo_run-001_T2w.nii.gz
---> linked 4.nii.gz <-- sub-1510121_ses-20231102p15w03s13_run-001_FLAIR.nii.gz
---> linked 5.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-eatlift1enc_run-001_phasediff.nii.gz
---> linked 6.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-eatlift1enc_run-001_magnitude1.nii.gz
---> linked 7.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-eatlift1enc_run-001_magnitude2.nii.gz
---> linked 8.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-eatlift1ret_run-001_magnitude1.nii.gz
---> linked 9.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-eatlift1ret_run-001_phasediff.nii.gz
---> linked 10.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-eatlift1ret_run-001_magnitude2.nii.gz
---> linked 11.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-pepolarGRE_dir-AP_run-001_epi.nii.gz
---> linked 12.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-pepolarGRE_dir-PA_run-001_epi.nii.gz
---> linked 13.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-pepolarGRESBref_dir-AP_run-001_epi.nii.gz
---> linked 14.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-pepolarGRESBref_dir-PA_run-001_epi.nii.gz
---> linked 15.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-pepolarSE_dir-AP_run-001_epi.nii.gz
---> linked 16.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-pepolarSE_dir-PA_run-001_epi.nii.gz
---> linked 17.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-perf_dir-AP_run-001_epi.nii.gz
---> linked 18.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-perf_dir-PA_run-001_epi.nii.gz
---> linked 19.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-98ori_dir-AP_run-001_sbref.nii.gz
---> linked 20.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-99ori_dir-AP_run-001_sbref.nii.gz
---> linked 21.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-98ori_dir-PA_run-001_sbref.nii.gz
---> linked 22.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-99ori_dir-PA_run-001_sbref.nii.gz
---> linked 23.nii.gz <-- sub-1510121_ses-20231102p15w03s13_task-eatlift1enc_run-001_bold.nii.gz
---> linked 24.nii.gz <-- sub-1510121_ses-20231102p15w03s13_task-eatlift1enc_run-002_bold.nii.gz
---> linked 25.nii.gz <-- sub-1510121_ses-20231102p15w03s13_task-eatlift1ret_run-001_bold.nii.gz
---> linked 26.nii.gz <-- sub-1510121_ses-20231102p15w03s13_task-eatlift1ret_run-002_bold.nii.gz
---> linked 27.nii.gz <-- sub-1510121_ses-20231102p15w03s13_task-eatlift1ret_run-003_bold.nii.gz
---> linked 28.nii.gz <-- sub-1510121_ses-20231102p15w03s13_task-eatlift1ret_run-004_bold.nii.gz
---> linked 29.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-98ori_dir-AP_run-001_dwi.nii.gz
---> linked 30.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-98ori_dir-PA_run-001_dwi.nii.gz
---> linked 31.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-99ori_dir-AP_run-001_dwi.nii.gz
---> linked 32.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-99ori_dir-PA_run-001_dwi.nii.gz
---> linked 33.nii.gz <-- sub-1510121_ses-20231102p15w03s13_acq-perf_run-001_asl.nii.gz
Running mapBIDS2behavior for subject 1510121, session 20231102p15w03s13
=======================================================================
---> created behavior subfolder

Final report
============
subject 1510121, session 20231102p15w03s13 image mapping completed, behavioral files: no files mapped

-----------------------------------------
Finished at 2024-10-29 14:44:59

---> Successful completion of task
2

Hi Markus,

Thanks for reporting this! Yes, looks like a bug, it should not stack fm tags like that and should acknowledge the metadata correctly.

We are adding support for some new image types for import_bids as it is required by one of our ongoing projects. In the scope of that work we will also try to fix this. If we manage to do that a fix should be out soon, as that development is quite urgent.

Best, Jure

3

Fantastic, let me know if I can assist in any way.
Cheers,
Markus

4

I believe this has been resolved and will be released with the next version.

Best, Jure

5

There was a misunderstanding on our part. This issue hasn’t been resolved yet. @markushs Can you please post the relevant json files?

6

Hi @andraz.matkovic

I think these examples should suffice:

JSON of Phase Difference fieldmap for BOLD task “encoding”

sub-001_ses-001_acq-encoding_run-001_phasediff.json

  "IntendedFor": [
    "ses-001/func/sub-001_ses-001_task-encoding_run-001_bold.nii.gz",
    "ses-001/func/sub-001_ses-001_task-encoding_run-002_bold.nii.gz"
]

JSON of Phase Difference fieldmap for BOLD task “retrieval”

sub-001_ses-001_acq-retrieval_run-001_phasediff.json

  "IntendedFor": [
    "ses-001/func/sub-001_ses-001_task-retrieval_run-001_bold.nii.gz",
    "ses-001/func/sub-001_ses-001_task-retrieval_run-002_bold.nii.gz"
]

JSON of phase-encoding poloarity fieldmap for perfusion dataset

sub-001_ses-001_acq-perf_dir-AP_run-001_epi.json
and
sub-001_ses-001_acq-perf_dir-PA_run-001_epi.json

  "IntendedFor": [
    "ses-001/perf/sub-001_ses-001_acq-perf_run-001_asl.nii.gz"
],

To get my projects to run I’ve patched the code in /opt/qunex/python/qx_utilities/general/bids.py (inside the container) around line 1209 (end of processBIDS function definition), with this:

    # ---> process fieldmap matching # EDITED BY MHS 20241030, complete rewrite w/comments
    for session in bidsData:
        # Initialize counters and mappings for each fieldmap type
        fm_counts = {"fm": 0, "se": 0}
        fm_acq_run_indices = {"fm": {}, "se": {}}  # Mapping from (acq, run) to index

        for element in bidsData[session].get("fmap", []):
            if ".nii" in element["filename"]:

                # ---> Determine the fieldmap type ('fm' or 'se')
                if element["label"] == "epi":
                    fmtype = "se"
                else:
                    fmtype = "fm"

                # Extract the 'acq' and 'run' values from the filename using string splitting
                filename = element["filename"]
                acq = 'default'  # Default value if 'acq' is not found
                run = '1'        # Default run value if 'run' is not found
                filename_parts = filename.split('_')
                for part in filename_parts:
                    if part.startswith('acq-'):
                        acq = part[len('acq-'):]
                    elif part.startswith('run-'):
                        run = part[len('run-'):]
                
                # Use (acq, run) as the key
                acq_run_key = (acq, run)

                # Assign an index to each unique (acq, run) combination per fieldmap type
                if acq_run_key not in fm_acq_run_indices[fmtype]:
                    fm_counts[fmtype] += 1
                    fm_acq_run_indices[fmtype][acq_run_key] = fm_counts[fmtype]
                fmindex = fm_acq_run_indices[fmtype][acq_run_key]

                tag = f"{fmtype}({fmindex})"

                # ---> Add the tag to the fieldmap image if not already present
                seq_info = bidsData[session]["images"]["info"][element["filename"]].get("seq_info", [])
                if tag not in seq_info:
                    bidsData[session]["images"]["info"][element["filename"]]["seq_info"] = seq_info + [tag]

                # ---> Add the tag to intended target files, avoiding duplicates
                if (
                    "json_info" in element
                    and element["json_info"]
                    and "IntendedFor" in element["json_info"]
                ):
                    if isinstance(element["json_info"]["IntendedFor"], str):
                        target_files = [element["json_info"]["IntendedFor"]]
                    else:
                        target_files = element["json_info"]["IntendedFor"]
                    for target_file in target_files:
                        target_file = os.path.basename(target_file)
                        seq_info = bidsData[session]["images"]["info"][target_file].get("seq_info", [])
                        if tag not in seq_info:
                            bidsData[session]["images"]["info"][target_file]["seq_info"] = seq_info + [tag]

Not sure if this is sufficient for the general pipeline but it fixes the issues reported here.
Cheers,
Markus

7

Hi Markus,

Thanks for providing the info.

We are releasing a new version this week, but it unfortunately won’t include this fix as we ran out of time and have to release due to obligations for our research projects and partners. We will implement and release this in the first patch/release that follows, probably by the end of November.

Best, Jure

8

Hi Markus, the latest version (1.0.0) contains a fix for this problem. Now all fieldmaps with the same basename (i.e. everything before the suffix) have the same tag. Let me know if you encounter any problems.

9

Thanks @andraz.matkovic
I’ll check it out.
Markus