[RESOLVED] Map_hcp_data error:Here's the error as caught by python

yzhong · May 26, 2025, 7:03am

Hi all,

I met a problem when running map_hcp_data step. Could you help me find out what’s wrong with this error?

My command is:

 qunex_container map_hcp_data \
	    --sessionsfolder="${WORK_DIR}/${STUDY_NAME}/sessions" \
            --batchfile="${WORK_DIR}/${STUDY_NAME}/processing/batch.txt" \
            --sessions="${SUBJ}" \
            --container="${QUNEX_CONTAINER}"

And the error is:

========================================================================
Generated by QuNex
------------------------------------------------------------------------
Version: 1.1.1 [QIO]
User: root
System: 3bfdfcea43b6
OS: Debian Linux #60~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Mar 28 16:09:21 UTC 2
------------------------------------------------------------------------

        ██████\                  ║      ██\   ██\
       ██  __██\                 ║      ███\  ██ |
       ██ /  ██ |██\   ██\       ║      ████\ ██ | ██████\ ██\   ██\
       ██ |  ██ |██ |  ██ |      ║      ██ ██\██ |██  __██\\██\ ██  |
       ██ |  ██ |██ |  ██ |      ║      ██ \████ |████████ |\████  /
       ██ ██\██ |██ |  ██ |      ║      ██ |\███ |██   ____|██  ██\
       \██████ / \██████  |      ║      ██ | \██ |\███████\██  /\██\
        \___███\  \______/       ║      \__|  \__| \_______\__/  \__|
            \___|                ║


                       DEVELOPED & MAINTAINED BY:

               Mind & Brain Lab, University of Ljubljana
                       Cho Lab, Yale University

                      COPYRIGHT & LICENSE NOTICE:

Use of this software is subject to the terms and conditions defined in
QuNex LICENSES which can be found in the LICENSES folder of the QuNex
repository or at https://qunex.yale.edu/qunex-registration
========================================================================

---> Setting up Octave


.......................... Running QuNex v1.1.1 [QIO] ..........................


--- Full QuNex call for command: map_hcp_data

qunex map_hcp_data --sessionsfolder="/mnt/hdd1/yumingz/code/sessions" --batchfile="/mnt/hdd1/yumingz/code/proces70c,FCS_100_C2,FCS_097_C2,FCS-69A,FCS_103_DBSI,fcs_181_c,FCS_164_A,FCS_133_A,pr-66a,fcs-109a,FCS_168_A,pr-31a,fcFCS82A,FCS_152_C,fcs-62c2,fcs-12amc,FCS_136_A,fcs_163_c,FCS_103_DBSI_MR1,FCS-106-C2,FCS_168_C2,FCS_173_A,fcs_132FCS_194_A,fcs_138_c,fcs-107c,FCS103C2,fcs_146_a,fcs_017_amc2,fcs_190_A,fcs-48C2,FCS116A,fcs-79c,pr53a,pr-60a,FCSFCS_180_A,pr-33c2,fcs_005_amc_2,fcs-86c2,FCS_193_C,fcs_157_a,fcs_173_c,FCS_188_A,pr-58a,FCS_190_C,fcs_129_a,FCS_S_164_C,FCS116C,FCS_182_C2_2,FCS_049_C,FCS_135_A,fcs_171_c,fcs-80c,FCS_196_A,fcs-78c,fcs-88A,fcs-115A,fcs-52c2,Ffcs-115c,pr27a,pr-30c2,fcs_182_a,pr-68a,PR-56c,fcs-84c2,FCS_028_AMC,pr-49a,FCS_022_AMC,fcs-112A,fcs_165_a,FCS_0210261426,pr24a,fcs-57c2,pr-71a,pr-30,FCS_192_A,fcs_135_c,fcs-109c,FCS_186_C2,pr-44a,FCS_184_C,pr-35c2,pr-24c2,FC54s,fcs_151_c2,fcs_102_c2,FCS_172_A,FCS_191_A,pr38a,fcs-78c2,FCS_170_C,pr-43c,pr-26c,fcs_175_C,pr-54c,FCS_030_AM109_c2,FCS_024_AMC2,FCS_195_A,FCS_125_A,fcs-111A,fcs_149_c,fcs_167_c,fcs_134_a,FCS_142_A,fcs-65c,pr-26a,FCS69C,f-52a,pr-35a,fcs_122_c,FCS_032_AMC2,FCS_064_C2,fcs_122_a,fcs-58C2,FCS-70A,fcs-88c2,pr-45c,FCS_143_C2,PR40A,pr-32c2,FCS_187_C2,PR-51c,pr-43a,fcs-73c2,fcs-75c,fcs-83c2,fcs_187_a,PR-62C,pr-46a,FCS82C2,FCS_174_A,fcs-102c,FCS77A,f73a,FCS_048_DBSI,fcs_138_c2,pr-74a,FCS_161_A,pr-58c,FCS69C_200912181128,fcs-108c,fcs-83c,FCS-105-C2,FCS40,pr47a,FCS_030_AMC2,fcs_129_c2,fcs-79c2,fcs_133_c2,fcs_132_c2,fcs_034_amc,fcs-07amc2,fcs-07acm,pr-48a,pr-61a,fcs_024_amS_170_A,fcs_140_c,10_07_21-12_41_fcs-108a,FCS120C,FCS-51C,pr-51a,FCS_166_A,fcs-01acm,fcs_147_c,pr-36a,fs_124_c2,c2,fcs-71c,fcs_003_amc_2,fcs-70c2,fcs_122_c2,FCS145_C2,fcs_129_c,pr-28a,FCS_169_C2,FCS-140A,pr-45a,FCS_035_AMC1,fcs_036_amc2,FCS_116_C2,FCS-78a,fcs_166_c2,fcs_175_a,pr-75a,fcs_126_a,fcs-100a,fcs-84A,pr-57c,fcs-77c,FCS_161_C22_amc2,FCS_141_C2,fcs_012_amc2,FCS81C2,fcs-61c2,PR-43C,fcs_123_a,FCS103A,fcs_164_c2,FCS-112C,fcs08amc,FCS_145_A,r-34a,pr-31c,FCS_141_A,pr-35c,fcs_151_c,PR40C,FCS_142_C2,FCS_131_C2,pr-50a,FCS106C,FCS_187_C,FCS81C,fcs_162_c2,f-50c2,FCS_162_A,fcs_033_amc_2,fcs-65c2,fcs-114a,FCS87C,fcs-110A,fcs_023_amc,FCS_144_A,fcs_165_c,fcs_124_c,fcs_12_019_amc2,FCS67C2,fcs83a,fcs_155_a,FCS_192_C2,pr-56a,fcs_158_c,FCS120A,FCS_169_C,pr-46c,fcs-119c,FCS_151_A,pr-24fcs_034_amc2,fcs_027_amc2,FCS_186_A,FCS_025_AMC2,pr-64a,FCS_065_C2,FCS_178_A,pr-33a,FCS_135_C2,pr-39a,FCS87A_200c,fcs_133_c,pr-33c,fcs-65a,FCS_160_C,pr26c2,FCS_196_C2,FCS_146_C2,FCS_181_A,pr-30c,FCS587C2,FCS_131_C,FCS_124_A,AMC,FCS87A,pr-48c,FCS79A,fcs_143_a,fcs_011_amc_2,fcs-72c,fcs-71c2,pr-31c2_c,fcs_150_A,FCS106A,pr-32a,fcs_031_amcc2,fcs_117_c,pr-29a,fcs-68c,FCS_160_C2,FCS103C,FCS_138_A,FCS69C2,FCS_016_AMC1,fcs_154_a,FCS_169_A,PR-63C,FCS_02178_c,FCS_166_C,fcs-47c2,fcs_155_c,FCS40C2,fcs_146_c,FCS_115_C2,FCS_141_C,fcs_111_c2,fcs-104c,FCS_186_C,fcs-60c2,_DBSI_MR1,fcs_132_a,FCS_171_A,pr-50c,FCS67C_200911301150,FCS_145_C,FCS_149_A,pr-61c,FCS_177_C2,FCS_175_C2,FCS_1719_AMC1,fcs_101_c2,fcs_148_c,fcs-86a,pr-47c,FCS_182_C,FCS_192_C,fcs_183_a,fcs_036_amc,fcs_029_amc2,FCS_108_C2,pr,fcs-63c2,FCS_025_AMC"

---------------------------------------------------------


# Generated by QuNex 1.1.1 [QIO] on 2025-05-26_06.42.49.876220#
=================================================================
qunex map_hcp_data \
  --sessionsfolder="/mnt/hdd1/yumingz/code/sessions" \
  --sessions="/mnt/hdd1/yumingz/code/processing/batch.txt" \
  --sessionids="fcs-74c,fcs-70c,FCS_100_C2,FCS_097_C2,FCS-69A,FCS_103_DBSI,fcs_181_c,FCS_164_A,FCS_133_A,pr-66a,1_C,fcs_014_amc_2,FCS_184_A,FCS82A,FCS_152_C,fcs-62c2,fcs-12amc,FCS_136_A,fcs_163_c,FCS_103_DBSI_MR1,FCS-106-C2,fcs-06amc2,pr-55a,fcs_148_a,FCS_194_A,fcs_138_c,fcs-107c,FCS103C2,fcs_146_a,fcs_017_amc2,fcs_190_A,fcs-48C2,FCS129_AMC1,fcs_140_c2,fcs-101c,FCS_180_A,pr-33c2,fcs_005_amc_2,fcs-86c2,FCS_193_C,fcs_157_a,fcs_173_c,FCS_188_A,pr-9_A,FCS_181_C2,FCS_178_C2,FCS_164_C,FCS116C,FCS_182_C2_2,FCS_049_C,FCS_135_A,fcs_171_c,fcs-80c,FCS_196_A,fcs-78cS_035_AMC2,fcs_143_c,FCS67C,fcs-115c,pr27a,pr-30c2,fcs_182_a,pr-68a,PR-56c,fcs-84c2,FCS_028_AMC,pr-49a,FCS_022_A13_amc,FCS_180_C,FCS76A_200910261426,pr24a,fcs-57c2,pr-71a,pr-30,FCS_192_A,fcs_135_c,fcs-109c,FCS_186_C2,pr-44a,021_amc2,fcs_102a,FCS80A,pr-54s,fcs_151_c2,fcs_102_c2,FCS_172_A,FCS_191_A,pr38a,fcs-78c2,FCS_170_C,pr-43c,pr-26c,fcs-84a,pr-41a,pr-27c2,fcs_109_c2,FCS_024_AMC2,FCS_195_A,FCS_125_A,fcs-111A,fcs_149_c,fcs_167_c,fcs_134_a,FCS_1S_165_C2,FCS_131_A,FCS82C,pr-52a,pr-35a,fcs_122_c,FCS_032_AMC2,FCS_064_C2,fcs_122_a,fcs-58C2,FCS-70A,fcs-88c2,pr1,FCS76A,fcs_142_c,fcs-10amc2,FCS_187_C2,PR-51c,pr-43a,fcs-73c2,fcs-75c,fcs-83c2,fcs_187_a,PR-62C,pr-46a,FCS82C263,FCS_171_C2,FCS-02AMC,fcs-73a,FCS_048_DBSI,fcs_138_c2,pr-74a,FCS_161_A,pr-58c,FCS69C_200912181128,fcs-108c,fcsr-57a,FCS_196_C,fcs_amc_032,FCS_030_AMC2,fcs_129_c2,fcs-79c2,fcs_133_c2,fcs_132_c2,fcs_034_amc,fcs-07amc2,fcs-07,pr-62a,fcs-73c,FCS_168_C,FCS_170_A,fcs_140_c,10_07_21-12_41_fcs-108a,FCS120C,FCS-51C,pr-51a,FCS_166_A,fcs-01acmcs_177_a,fcs_120_c2,fcs_112_c2,fcs-71c,fcs_003_amc_2,fcs-70c2,fcs_122_c2,FCS145_C2,fcs_129_c,pr-28a,FCS_169_C2,FS_161_C,fcs_147_a,fcs-amc14,fcs_036_amc2,FCS_116_C2,FCS-78a,fcs_166_c2,fcs_175_a,pr-75a,fcs_126_a,fcs-100a,fcs-837A,pr-60c,fcs-09amc2,fcs_002_amc2,FCS_141_C2,fcs_012_amc2,FCS81C2,fcs-61c2,PR-43C,fcs_123_a,FCS103A,fcs_164_c2,s_134_c,fcs_163_a,fcs-101A,pr-34a,pr-31c,FCS_141_A,pr-35c,fcs_151_c,PR40C,FCS_142_C2,FCS_131_C2,pr-50a,FCS106C,F85a,fcs_148_c2,fcs_177_c,fcs-50c2,FCS_162_A,fcs_033_amc_2,fcs-65c2,fcs-114a,FCS87C,fcs-110A,fcs_023_amc,FCS_144_AMC2,FCS_160_A,fcs_128_a,fcs_019_amc2,FCS67C2,fcs83a,fcs_155_a,FCS_192_C2,pr-56a,fcs_158_c,FCS120A,FCS_169_C,pr-CS44C,fcs_018_amc1,fcs-118a,fcs_034_amc2,fcs_027_amc2,FCS_186_A,FCS_025_AMC2,pr-64a,FCS_065_C2,FCS_178_A,pr-33a,S_170_C2,FCS_155_C2,fcs_139_c,fcs_133_c,pr-33c,fcs-65a,FCS_160_C,pr26c2,FCS_196_C2,FCS_146_C2,FCS_181_A,pr-30c,FA,FCS82A_200911201417,FCS-09AMC,FCS87A,pr-48c,FCS79A,fcs_143_a,fcs_011_amc_2,fcs-72c,fcs-71c2,pr-31c2_c,fcs_150_s_026_amc1,fcs-amc04c,fcs-49c2,fcs_117_c,pr-29a,fcs-68c,FCS_160_C2,FCS103C,FCS_138_A,FCS69C2,FCS_016_AMC1,fcs_15_a,fcs-100c,FCS_193_C2,fcs_178_c,FCS_166_C,fcs-47c2,fcs_155_c,FCS40C2,fcs_146_c,FCS_115_C2,FCS_141_C,fcs_111_c2,S_026_AMC2,FCS_128_C,FCS_048_DBSI_MR1,fcs_132_a,FCS_171_A,pr-50c,FCS67C_200911301150,FCS_145_C,FCS_149_A,pr-61c,fcs_028_amc2,fcs_193_a,FCS_019_AMC1,fcs_101_c2,fcs_148_c,fcs-86a,pr-47c,FCS_182_C,FCS_192_C,fcs_183_a,fcs_036_amcs_158_a,fcs-104a,FCS_104_C2,fcs-63c2,FCS_025_AMC" \
=================================================================

Starting multiprocessing sessions in /mnt/hdd1/yumingz/code/processing/batch.txt with a pool of 1 concurrent pro


Starting processing of sessions fcs-74c at Monday, 26. May 2025 06:42:49

--------------------------------- QuNex failed ---------------------------------

ERROR: Execution of command `map_hcp_data` failed!
       Please check documentation for the command (`qunex map_hcp_data --help`)!

The call received was:
(please note that when run through scheduler, all possible parameters,
even non relevant ones are passed)

qunex map_hcp_data \
    --sessionsfolder=/mnt/hdd1/yumingz/code/sessions \
    --batchfile=/mnt/hdd1/yumingz/code/processing/batch.txt \
    --sessions=fcs-74c,fcs-70c,FCS_100_C2,FCS_097_C2,FCS-69A,FCS_103_DBSI,fcs_181_c,FCS_164_A,FCS_133_A,pr-66a,f_C,fcs_014_amc_2,FCS_184_A,FCS82A,FCS_152_C,fcs-62c2,fcs-12amc,FCS_136_A,fcs_163_c,FCS_103_DBSI_MR1,FCS-106-C2,Fcs-06amc2,pr-55a,fcs_148_a,FCS_194_A,fcs_138_c,fcs-107c,FCS103C2,fcs_146_a,fcs_017_amc2,fcs_190_A,fcs-48C2,FCS119_AMC1,fcs_140_c2,fcs-101c,FCS_180_A,pr-33c2,fcs_005_amc_2,fcs-86c2,FCS_193_C,fcs_157_a,fcs_173_c,FCS_188_A,pr-5_A,FCS_181_C2,FCS_178_C2,FCS_164_C,FCS116C,FCS_182_C2_2,FCS_049_C,FCS_135_A,fcs_171_c,fcs-80c,FCS_196_A,fcs-78c,_035_AMC2,fcs_143_c,FCS67C,fcs-115c,pr27a,pr-30c2,fcs_182_a,pr-68a,PR-56c,fcs-84c2,FCS_028_AMC,pr-49a,FCS_022_AM3_amc,FCS_180_C,FCS76A_200910261426,pr24a,fcs-57c2,pr-71a,pr-30,FCS_192_A,fcs_135_c,fcs-109c,FCS_186_C2,pr-44a,F21_amc2,fcs_102a,FCS80A,pr-54s,fcs_151_c2,fcs_102_c2,FCS_172_A,FCS_191_A,pr38a,fcs-78c2,FCS_170_C,pr-43c,pr-26c,fcs-84a,pr-41a,pr-27c2,fcs_109_c2,FCS_024_AMC2,FCS_195_A,FCS_125_A,fcs-111A,fcs_149_c,fcs_167_c,fcs_134_a,FCS_14_165_C2,FCS_131_A,FCS82C,pr-52a,pr-35a,fcs_122_c,FCS_032_AMC2,FCS_064_C2,fcs_122_a,fcs-58C2,FCS-70A,fcs-88c2,pr-,FCS76A,fcs_142_c,fcs-10amc2,FCS_187_C2,PR-51c,pr-43a,fcs-73c2,fcs-75c,fcs-83c2,fcs_187_a,PR-62C,pr-46a,FCS82C2,3,FCS_171_C2,FCS-02AMC,fcs-73a,FCS_048_DBSI,fcs_138_c2,pr-74a,FCS_161_A,pr-58c,FCS69C_200912181128,fcs-108c,fcs--57a,FCS_196_C,fcs_amc_032,FCS_030_AMC2,fcs_129_c2,fcs-79c2,fcs_133_c2,fcs_132_c2,fcs_034_amc,fcs-07amc2,fcs-07apr-62a,fcs-73c,FCS_168_C,FCS_170_A,fcs_140_c,10_07_21-12_41_fcs-108a,FCS120C,FCS-51C,pr-51a,FCS_166_A,fcs-01acm,s_177_a,fcs_120_c2,fcs_112_c2,fcs-71c,fcs_003_amc_2,fcs-70c2,fcs_122_c2,FCS145_C2,fcs_129_c,pr-28a,FCS_169_C2,FC_161_C,fcs_147_a,fcs-amc14,fcs_036_amc2,FCS_116_C2,FCS-78a,fcs_166_c2,fcs_175_a,pr-75a,fcs_126_a,fcs-100a,fcs-847A,pr-60c,fcs-09amc2,fcs_002_amc2,FCS_141_C2,fcs_012_amc2,FCS81C2,fcs-61c2,PR-43C,fcs_123_a,FCS103A,fcs_164_c2,F_134_c,fcs_163_a,fcs-101A,pr-34a,pr-31c,FCS_141_A,pr-35c,fcs_151_c,PR40C,FCS_142_C2,FCS_131_C2,pr-50a,FCS106C,FC5a,fcs_148_c2,fcs_177_c,fcs-50c2,FCS_162_A,fcs_033_amc_2,fcs-65c2,fcs-114a,FCS87C,fcs-110A,fcs_023_amc,FCS_144_AMC2,FCS_160_A,fcs_128_a,fcs_019_amc2,FCS67C2,fcs83a,fcs_155_a,FCS_192_C2,pr-56a,fcs_158_c,FCS120A,FCS_169_C,pr-4S44C,fcs_018_amc1,fcs-118a,fcs_034_amc2,fcs_027_amc2,FCS_186_A,FCS_025_AMC2,pr-64a,FCS_065_C2,FCS_178_A,pr-33a,F_170_C2,FCS_155_C2,fcs_139_c,fcs_133_c,pr-33c,fcs-65a,FCS_160_C,pr26c2,FCS_196_C2,FCS_146_C2,FCS_181_A,pr-30c,FC,FCS82A_200911201417,FCS-09AMC,FCS87A,pr-48c,FCS79A,fcs_143_a,fcs_011_amc_2,fcs-72c,fcs-71c2,pr-31c2_c,fcs_150_A_026_amc1,fcs-amc04c,fcs-49c2,fcs_117_c,pr-29a,fcs-68c,FCS_160_C2,FCS103C,FCS_138_A,FCS69C2,FCS_016_AMC1,fcs_154a,fcs-100c,FCS_193_C2,fcs_178_c,FCS_166_C,fcs-47c2,fcs_155_c,FCS40C2,fcs_146_c,FCS_115_C2,FCS_141_C,fcs_111_c2,f_026_AMC2,FCS_128_C,FCS_048_DBSI_MR1,fcs_132_a,FCS_171_A,pr-50c,FCS67C_200911301150,FCS_145_C,FCS_149_A,pr-61c,Fcs_028_amc2,fcs_193_a,FCS_019_AMC1,fcs_101_c2,fcs_148_c,fcs-86a,pr-47c,FCS_182_C,FCS_192_C,fcs_183_a,fcs_036_amcs_158_a,fcs-104a,FCS_104_C2,fcs-63c2,FCS_025_AMC

--------------------------------------------------------------
Here's the error as caught by python:

Traceback (most recent call last):
  File "/opt/qunex/python/qx_utilities/gmri", line 542, in <module>
    main()
  File "/opt/qunex/python/qx_utilities/gmri", line 488, in main
    runCommand(comm, opts)
  File "/opt/qunex/python/qx_utilities/gmri", line 70, in runCommand
    gp.run(command, args)
  File "/opt/qunex/python/qx_utilities/general/process.py", line 2411, in run
    pending_actions(session, soptions, overwrite, c + 1)
  File "/opt/qunex/python/qx_utilities/hcp/process_hcp.py", line 13301, in map_hcp_data
    r += "\n\nSource folder: " + d["hcp"]
                                 ~^^^^^^^
KeyError: 'hcp'

Thanks for your kindly help.

Best,
Acacius

demsarjure · May 26, 2025, 11:13am

Hi Acacius,

Can you please upload the batchfile?

Thanks!

Best, Jure

yzhong · May 26, 2025, 11:26am

Hi, Jure

Sure, Here it is.
batch.txt (727.3 KB)

Thanks.

Best,
Acacius

demsarjure · May 27, 2025, 10:12am

Hi Acacius,

Your sessions list seem to have some entries that are not in the batch file, for example FCS67C_200911301150, 01acmcs_177_a, FCS69C_200912181128, etc …

If you want to map the sessions that are in the batch file there is no need to provide additional filtering through the sessions parameter. If you provide only the batch file, QuNex will map all sessions that are in there. If you want to map a subset of sessions, you can use the sessions parameter to select a subset. An alternative is to create a copy of the batch file and remove the sessions you do not want mapped from it.

Best, Jure

yzhong · May 28, 2025, 2:44am

Hi Jure,

Thanks for your kindly help. Unfortunately, both methods are still not work and the error is the same:


# Generated by QuNex 1.1.1 [QIO] on 2025-05-28_02.40.32.858237#
=================================================================
qunex map_hcp_data \
  --sessionsfolder="/mnt/hdd1/yumingz/code/sessions" \
  --sessions="/mnt/hdd1/yumingz/code/batch.txt" \
  --sessionids="FCS106C,FCS145_C2,FCS587C2,FCS_030_AMC1,FCS_030_AMC2,FCS_136_A,FCS_138_A,FCS_139_A,FCS_141_A,FCS_141_C,FCS_142_A,FCS_142_C2,FCS_143_C2,FCS_145_A,FCS_145_C,FCS_168_A,FCS_168_C,FCS_168_C2,FCS_169_C,FCS_175_C2,FCS_177_C2,FCS_179_A,FCS_181_A,FCS_193_C,fcs_036_amc,fcs_036_amc2,fcs_133_c,fcs_133_c2,fcs_138_c,fcs_139_c,fcs_142_c,fcs_147_a,fcs_147_c,fcs_149_c,fcs_158_c,fcs_175_C,fcs_175_a,fcs_177_a,fcs_177_c,fcs_181_c,fcs_183_a,fcs_193_a" \
=================================================================

Starting multiprocessing sessions in /mnt/hdd1/yumingz/code/batch.txt with a pool of 1 concurrent processes


Starting processing of sessions FCS106C at Wednesday, 28. May 2025 02:40:32

--------------------------------- QuNex failed ---------------------------------

ERROR: Execution of command `map_hcp_data` failed!
       Please check documentation for the command (`qunex map_hcp_data --help`)!

The call received was:
(please note that when run through scheduler, all possible parameters,
even non relevant ones are passed)

qunex map_hcp_data \
    --sessionsfolder=/mnt/hdd1/yumingz/code/sessions \
    --batchfile=/mnt/hdd1/yumingz/code/batch.txt \
    --sessions=FCS106C,FCS145_C2,FCS587C2,FCS_030_AMC1,FCS_030_AMC2,FCS_136_A,FCS_138_A,FCS_139_A,FCS_141_A,FCS_141_C,FCS_142_A,FCS_142_C2,FCS_143_C2,FCS_145_A,FCS_145_C,FCS_168_A,FCS_168_C,FCS_168_C2,FCS_169_C,FCS_175_C2,FCS_177_C2,FCS_179_A,FCS_181_A,FCS_193_C,fcs_036_amc,fcs_036_amc2,fcs_133_c,fcs_133_c2,fcs_138_c,fcs_139_c,fcs_142_c,fcs_147_a,fcs_147_c,fcs_149_c,fcs_158_c,fcs_175_C,fcs_175_a,fcs_177_a,fcs_177_c,fcs_181_c,fcs_183_a,fcs_193_a

--------------------------------------------------------------
Here's the error as caught by python:

Traceback (most recent call last):
  File "/opt/qunex/python/qx_utilities/gmri", line 542, in <module>
    main()
  File "/opt/qunex/python/qx_utilities/gmri", line 488, in main
    runCommand(comm, opts)
  File "/opt/qunex/python/qx_utilities/gmri", line 70, in runCommand
    gp.run(command, args)
  File "/opt/qunex/python/qx_utilities/general/process.py", line 2411, in run
    pending_actions(session, soptions, overwrite, c + 1)
  File "/opt/qunex/python/qx_utilities/hcp/process_hcp.py", line 13301, in map_hcp_data
    r += "\n\nSource folder: " + d["hcp"]
                                 ~^^^^^^^
KeyError: 'hcp'

My command is:

 qunex_container map_hcp_data \
	    --sessionsfolder="${WORK_DIR}/${STUDY_NAME}/sessions" \
            --batchfile="${WORK_DIR}/${STUDY_NAME}/batch.txt" \
            --container="${QUNEX_CONTAINER}"

And my batch file is attached belowed:
batch.txt (158.2 KB)

Best,
Acacius

demsarjure · May 28, 2025, 6:12am

OK, let me dig deeper. The problem is that I am unable to reproduce this behavior on my end. A couple of things we could try:

One thing you can try is to replace the sessions parameter with with sessionids (this is an old, deprecated parameter, but it is still available for backwards compatibiilty of commands).
Are you maybe missing some binds, for example --bind="${WORK_DIR}:${WORK_DIR}"?
Is the path to the batchfile correct? Should it maybe be /mnt/hdd1/yumingz/code/procecssing/batch.txt?
An alternative would be to enter the container and work in there:

# 1. enter the container
singularity run --bind ${WORK_DIR}:${WORK_DIR} {${QUNEX_CONTAINER} bash

# 2. load qunex environment
source /opt/qunex/env/qunex_environment.sh

# 3. execute the command, note that since we are inside the container we use qunex instead of qunex_container
qunex map_hcp_data \
  --sessionsfolder="/mnt/hdd1/yumingz/code/sessions" \
  --batchfile="/mnt/hdd1/yumingz/code/batch.txt"

Let me know if any of the above help. If not, I will try reproducing this on my end again. Like I said, so far I was unable to trigger the map_hcp_data error you are seeing.

Best, Jure

yzhong · May 28, 2025, 7:46am

Hi Jure,

I have checked and tried all of them but they still not work. The error is the same as before.

Best, Acacius

demsarjure · May 29, 2025, 9:21am

Unfortunately, I am still unable to reproduce this.

Are the path to HCP folders correct, e.g. /mnt/hdd1/yumingz/washingtonu/script/script/sessions/FCS_164_A/hcp there is a weird script/script in there.
Your batch file has no parameters on top, is this intentional?
What happens if you run this only for a single session. Create a batch file with a single session, validate that it is fully processed and all the paths are there and then run map_hcp_data over that one.
You used to do mapping on other studies succesfully right? What changed?

Best, Jure

yzhong · June 1, 2025, 9:27am

Hi Jure,

Thank you for the effort you put into my question. I was able to run the subsequent steps successfully after replacing the container with version 0.99. I think some of the code may be incompatible between versions and that’s causing the problem.

Best,
Acacius

demsarjure · June 1, 2025, 10:52am

Glad you managed to resolve it. With 1.2.2. the error reporting of map_hcp_data will be much more verbose, so hopefully that will allow us to get to the bottom of this.

Best, Jure