[RESOLVED] Preprocess_conc issue using v0.98.0

Description:

I’m trying to process stimulus-evoked BOLD runs using preprocess_conc. I did a few trials, and it seems to not like when the parameter --event_string is used as “assumed modelling” and it contains only the fidl_code and hrf, as I do not have behavioral measures to regress. I copied the command call and the error log file. I attached the respective conc and fidl files.

Estephan

Call:

msi_resources_time=02:00:00; msi_resources_nodes=1; msi_resources_ntaskspernode=2; msi_resources_mem=32000; msi_queue=agsmall; msi_resources_jobname=BOLD5; \
study_sharedfolder=/home/moanae/shared/project_K99_ChrTMDHCP_qunex02; \
subjID_list="10001,10002,10004,10005,10006,10007,10009,10011,10012,10013,10014,10015,10016,10018,10019,10021,10022,10023,10024,10030,10031,10033,10034,10035,10037,10038,10039,10041,10042,11004,11006,11009,11015,20001,20004,20005,20006,20008,20010,20011,20012,20013,20015,20016,20017,20018,20019,20021,20022"; \
conc_name="bold3-bold4_dtseries_dentalvpress"; fidl_name="BlockOnOff-30s"; \
qunex_container preprocess_conc \
--batchfile=${study_sharedfolder}/processing/batch_K99Aim2.txt --sessionsfolder=${study_sharedfolder}/sessions -sessions=${subjID_list} --parsessions=1 --parelements=2 \
--event_file="${conc_name}_${fidl_name}" --bolds="${conc_name}" --bold_actions="s,h,r0,l" --pignore="hipass=spline|regress=spline|lopass=spline" --bold_nuisance="e" --event_string="dentalvpress_ON:SPM-uni" --glm_matrix="both" --glm_residuals="all" --glm_name="" --image_target='dtseries' --nifti_tail="_hp0_clean_tclean" --cifti_tail="_Atlas_MSMAll_hp0_clean_tclean" --overwrite=yes \
--scheduler=SLURM,time=${msi_resources_time},nodes=${msi_resources_nodes},cpus-per-task=${msi_resources_ntaskspernode},mem=${msi_resources_mem},partition=${msi_queue},jobname=${msi_resources_jobname} \
--bind=${study_sharedfolder}:${study_sharedfolder} --container=${HOME}/qunex/qunex_suite-0.98.0.sif

Logs:

# Generated by QuNex 0.98.0 on 2023-06-10_17.33.00.054937
#
------------------------------------------------------------
Running external command via QuNex:

octave -q --no-init-file --eval "try fc_preprocess_conc('/home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10004', [3 4], 's,h,r0,l', 0.937,  0, 'e', [], 'bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s.fidl', 'dentalvpress_ON:SPM-uni>dentpress', '', true, '.dtseries.nii', 'radius:50|fdt:0.50|dvarsmt:3.00|dvarsmet:1.50|after:2|before:1|reject:dvarsme', 'hipass=spline|regress=spline|lopass=spline', 'boldname=bold|fidlname=bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s|concname=bold3-bold4_dtseries_dentalvpress|surface_smooth=2.000000|volume_smooth=2.000000|voxel_smooth=1.000000|hipass_filter=0.008000|lopass_filter=0.090000|omp_threads=0|framework_path=|wb_command_path=|smooth_mask=false|dilate_mask=false|glm_matrix=both|glm_residuals=all|glm_results=c,r|glm_name=|bold_tail=_Atlas_MSMAll_hp0_clean_tclean|ref_bold_tail=_hp0_clean_tclean|bold_variant=|img_suffix=', '/home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10004/images/functional/concs/bold_Atlas_MSMAll_hp0_clean_tclean_dtseries_bold3-bold4_dtseries_dentalvpress_bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s_s_hpss_res-e_lpss.conc.ok'); catch ME, general_report_crash(ME); exit(1), end; exit;"

Test file: 
/home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10004/images/functional/concs/bold_Atlas_MSMAll_hp0_clean_tclean_dtseries_bold3-bold4_dtseries_dentalvpress_bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s_s_hpss_res-e_lpss.conc.ok
------------------------------------------------------------


Running preproces conc script v0.9.16 [.dtseries.nii]
-------------------------------------

Parameters:
---------------
       sessionf: /home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10004
          bolds: [3  4]
           doIt: s,h,r0,l
             tr: 0.94
           omit: 0
           rgss: e
           task: [0  0]
          efile: bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s.fidl
   eventrstring: dentalvpress_ON:SPM-uni>dentpress
        variant: 
      overwrite: 1
           tail: .dtseries.nii
          scrub: radius:50|fdt:0.50|dvarsmt:3.00|dvarsmet:1.50|after:2|before:1|reject:dvarsme
        ignores: hipass=spline|regress=spline|lopass=spline
           done: /home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10004/images/functional/concs/bold_Atlas_MSMAll_hp0_clean_tclean_dtseries_bold3-bold4_dtseries_dentalvpress_bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s_s_hpss_res-e_lpss.conc.ok
        options: boldname=bold|fidlname=bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s|concname=bold3-bold4_dtseries_dentalvpress|surface_smooth=2.000000|volume_smooth=2.000000|voxel_smooth=1.000000|hipass_filter=0.008000|lopass_filter=0.090000|omp_threads=0|framework_path=|wb_command_path=|smooth_mask=false|dilate_mask=false|glm_matrix=both|glm_residuals=all|glm_results=c,r|glm_name=|bold_tail=_Atlas_MSMAll_hp0_clean_tclean|ref_bold_tail=_hp0_clean_tclean|bold_variant=|img_suffix=

Options used:
----------------
        boldname: bold
        concname: bold3-bold4_dtseries_dentalvpress
        fidlname: bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s
  surface_smooth: 2
   volume_smooth: 2
    voxel_smooth: 1
   lopass_filter: 0.09
   hipass_filter: 0.008
  framework_path: 
 wb_command_path: 
     omp_threads: 0
     smooth_mask: false
     dilate_mask: false
      glm_matrix: both
   glm_residuals: all
     glm_results: c,r
        glm_name: 
       bold_tail: _Atlas_MSMAll_hp0_clean_tclean
   ref_bold_tail: _hp0_clean_tclean
    bold_variant: 
      img_suffix: 


=========================================
Execution error! Processing failed! 
Please check arguments and/or try running the command in Matlab or Octave directly.

The exact error reported:
-----------------------------------------

Error identifier: 
   Error message: @binopdf: no function and no method found
     Error stack: /usr/local/share/octave/packages/statistics-1.4.2/pdf.m -> pdf [line: 59]
                  /opt/qunex/matlab/qx_mri/general/general_hrf.m -> general_hrf [line: 153]
                  /opt/qunex/matlab/qx_mri/general/general_create_task_regressors.m -> general_create_task_regressors [line: 378]
                  /opt/qunex/matlab/qx_mri/fc/fc_preprocess_conc.m -> fc_preprocess_conc [line: 910]

=========================================


conc file

number_of_files: 2
    file:/home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10001/images/functional/bold3_Atlas_MSMAll_hp0_clean_tclean.dtseries.nii
    file:/home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10001/images/functional/bold4_Atlas_MSMAll_hp0_clean_tclean.dtseries.nii

fidl file

0.937 dentalvpress_OFF dentalvpress_ON
0 0 9
9 1 30
39 0 30
69 1 30
99 0 30
129 1 30
159 0 30
189 1 30
219 0 30
249 1 30
279 0 30
309 1 30
339 0 54
393 0 9
402 1 30
432 0 30
462 1 30
492 0 30
522 1 30
552 0 30
582 1 30
612 0 30
642 1 30
672 0 30
702 1 30
732 0 54

** For an example of how to report an issue, please refer to this post.

Hi Estephan,

Unfortunatley, I am not an expert on preprocess_conc. I will try to get someone from our end that uses this regularly to help you out.

Jure

Just out of curiosity, is the above call maybe working on older versions? Do you have an older version at hand where you could test?

It seems like some Octave libraries are missing in the container, we will try to fix this for the next version.

QuNex 0.98.1 that fixes this will be released over the next couple of days.

Jure

I tested the 0.98.1 version with preprocess_conc, and got another error. Please see the error log below. Thank you.

Estephan

# Generated by QuNex 0.98.1 on 2023-07-17_16.43.05.120329
#
------------------------------------------------------------
Running external command via QuNex:

octave -q --no-init-file --eval "try fc_preprocess_conc('/home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10001', [3 4], 's,h,r0,l', 0.937,  0, 'e', [], 'bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s.fidl', 'dentalvpress_ON:SPM-uni', '', true, '.dtseries.nii', 'radius:50|fdt:0.50|dvarsmt:3.00|dvarsmet:1.50|after:2|before:1|reject:dvarsme', 'hipass=spline|regress=spline|lopass=spline', 'boldname=bold|fidlname=bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s|concname=bold3-bold4_dtseries_dentalvpress|surface_smooth=2.000000|volume_smooth=2.000000|voxel_smooth=1.000000|hipass_filter=0.008000|lopass_filter=0.090000|omp_threads=0|framework_path=|wb_command_path=|smooth_mask=false|dilate_mask=false|glm_matrix=both|glm_residuals=all|glm_results=c,r|glm_name=|bold_tail=_Atlas_MSMAll_hp0_clean_tclean|ref_bold_tail=_hp0_clean_tclean|bold_variant=|img_suffix=', '/home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10001/images/functional/concs/bold_Atlas_MSMAll_hp0_clean_tclean_dtseries_bold3-bold4_dtseries_dentalvpress_bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s_s_hpss_res-e_lpss.conc.ok'); catch ME, general_report_crash(ME); exit(1), end; exit;"

Test file: 
/home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10001/images/functional/concs/bold_Atlas_MSMAll_hp0_clean_tclean_dtseries_bold3-bold4_dtseries_dentalvpress_bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s_s_hpss_res-e_lpss.conc.ok
------------------------------------------------------------


Running preproces conc script v0.9.16 [.dtseries.nii]
-------------------------------------

Parameters:
---------------
       sessionf: /home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10001
          bolds: [3  4]
           doIt: s,h,r0,l
             tr: 0.94
           omit: 0
           rgss: e
           task: [0  0]
          efile: bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s.fidl
   eventrstring: dentalvpress_ON:SPM-uni
        variant: 
      overwrite: 1
           tail: .dtseries.nii
          scrub: radius:50|fdt:0.50|dvarsmt:3.00|dvarsmet:1.50|after:2|before:1|reject:dvarsme
        ignores: hipass=spline|regress=spline|lopass=spline
           done: /home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10001/images/functional/concs/bold_Atlas_MSMAll_hp0_clean_tclean_dtseries_bold3-bold4_dtseries_dentalvpress_bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s_s_hpss_res-e_lpss.conc.ok
        options: boldname=bold|fidlname=bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s|concname=bold3-bold4_dtseries_dentalvpress|surface_smooth=2.000000|volume_smooth=2.000000|voxel_smooth=1.000000|hipass_filter=0.008000|lopass_filter=0.090000|omp_threads=0|framework_path=|wb_command_path=|smooth_mask=false|dilate_mask=false|glm_matrix=both|glm_residuals=all|glm_results=c,r|glm_name=|bold_tail=_Atlas_MSMAll_hp0_clean_tclean|ref_bold_tail=_hp0_clean_tclean|bold_variant=|img_suffix=

Options used:
----------------
        boldname: bold
        concname: bold3-bold4_dtseries_dentalvpress
        fidlname: bold3-bold4_dtseries_dentalvpress_BlockOnOff-30s
  surface_smooth: 2
   volume_smooth: 2
    voxel_smooth: 1
   lopass_filter: 0.09
   hipass_filter: 0.008
  framework_path: 
 wb_command_path: 
     omp_threads: 0
     smooth_mask: false
     dilate_mask: false
      glm_matrix: both
   glm_residuals: all
     glm_results: c,r
        glm_name: 
       bold_tail: _Atlas_MSMAll_hp0_clean_tclean
   ref_bold_tail: _hp0_clean_tclean
    bold_variant: 
      img_suffix: 
--> starting the loop


Smoothing

---> /home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10001/images/functional/bold3_Atlas_MSMAll_hp0_clean_tclean.dtseries.nii 
---> running wb_command -cifti-smoothing
     ... smoothing ... done!
---> /home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10001/images/functional/bold4_Atlas_MSMAll_hp0_clean_tclean.dtseries.nii 
---> running wb_command -cifti-smoothing
     ... smoothing ... done!
---> saving conc file ... done!

High-pass filtering

---> /home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10001/images/functional/bold3_Atlas_MSMAll_hp0_clean_tclean_s.dtseries.nii 
---> reading /home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10001/images/functional/bold3_Atlas_MSMAll_hp0_clean_tclean_s.dtseries.nii ... done!
===> Temporal filtering (15/10/22)
---> triming: 0 on start, 0 on end
---> remaining bad frames: 0, action: spline
---> hipass frame         1    2    3    4    5    6    7    8    9   10   11   12   13   14   15   16   17   18   19   20   21   22   23   24   25   26   27   28   29   30   31   32   33   34   35   36   37   38   39   40   41   42   43   44   45   46   47   48   49   50   51   52   53   54   55   56   57   58   59   60   61   62   63   64   65   66   67   68   69   70   71   72   73   74   75   76   77   78   79   80   81   82   83   84   85   86   87   88   89   90   91   92   93   94   95   96   97   98   99  100  101  102  103  104  105  106  107  108  109  110  111  112  113  114  115  116  117  118  119  120  121  122  123  124  125  126  127  128  129  130  131  132  133  134  135  136  137  138  139  140  141  142  143  144  145  146  147  148  149  150  151  152  153  154  155  156  157  158  159  160  161  162  163  164  165  166  167  168  169  170  171  172  173  174  175  176  177  178  179  180  181  182  183  184  185  186  187  188  189  190  191  192  193  194  195  196  197  198  199  200  201  202  203  204  205  206  207  208  209  210  211  212  213  214  215  216  217  218  219  220  221  222  223  224  225  226  227  228  229  230  231  232  233  234  235  236  237  238  239  240  241  242  243  244  245  246  247  248  249  250  251  252  253  254  255  256  257  258  259  260  261  262  263  264  265  266  267  268  269  270  271  272  273  274  275  276  277  278  279  280  281  282  283  284  285  286  287  288  289  290  291  292  293  294  295  296  297  298  299  300  301  302  303  304  305  306  307  308  309  310  311  312  313  314  315  316  317  318  319  320  321  322  323  324  325  326  327  328  329  330  331  332  333  334  335  336  337  338  339  340  341  342  343  344  345  346  347  348  349  350  351  352  353  354  355  356  357  358  359  360  361  362  363  364  365  366  367  368  369  370  371  372  373  374  375  376  377  378  379  380  381  382  383  384  385  386  387  388  389  390  391  392  393  394  395  396  397  398  399  400  401  402  403  404  405  406  407  408  409  410  411  412  413  414  415  416  417  418  419  420
 ... saved!

=========================================
Execution error! Processing failed! 
Please check arguments and/or try running the command in Matlab or Octave directly.

The exact error reported:
-----------------------------------------

Error identifier: Octave:invalid-indexing
   Error message: structure has no member 'signal'
     Error stack: /opt/qunex/matlab/qx_mri/fc/fc_preprocess_conc.m -> fc_preprocess_conc [line: 1133]

=========================================

Estephan, hi!

Thank you for reporting this. We have identified the issue and are now resolving it. The fix should be in the next container.

With kind regards,

Grega

Hi Grega, sorry to be the bearer of bad news. I tried today qunex v0.98.2 and got a similar error as before. Is there anything I can do on my side to address this error?

Thank you.

Estephan


Command call

msi_resources_time=02:00:00; msi_resources_nodes=1; msi_resources_ntaskspernode=2; msi_resources_mem=32000; msi_queue=agsmall; msi_resources_jobname=BOLD6a; \
study_sharedfolder=/home/moanae/shared/project_K99_ChrTMDHCP_qunex02; \
subjID_list="10001,10002,10004,10005,10006,10007,10009,10011,10012,10013,10014,10015,10016,10018,10019,10021,10022,10023,10024,10030,10031,10033,10034,10035,10037,10038,10039,10041,10042,11004,11006,11009,11015,20001,20004,20005,20006,20008,20010,20011,20012,20013,20015,20016,20017,20018,20019,20021,20022"; \
conc_name="bold3-bold4_dtseries_dentalvpress"; fidl_name="BlockOnOff-30s"; \
qunex_container preprocess_conc \
--batchfile=${study_sharedfolder}/processing/batch_K99Aim2.txt --sessionsfolder=${study_sharedfolder}/sessions -sessions=${subjID_list} --parsessions=1 --parelements=2 \
--event_file="${conc_name}_${fidl_name}" --bolds="${conc_name}" --bold_actions="s,h,r0,l" --pignore="hipass=spline|regress=spline|lopass=spline" --bold_nuisance="e" --event_string="dentalvpress_ON:SPM-uni" --glm_matrix="both" --glm_residuals="all" --glm_name="" --image_target='dtseries' --nifti_tail="_hp0_clean_tclean" --cifti_tail="_Atlas_MSMAll_hp0_clean_tclean" --overwrite=yes \
--scheduler=SLURM,time=${msi_resources_time},nodes=${msi_resources_nodes},cpus-per-task=${msi_resources_ntaskspernode},mem=${msi_resources_mem},partition=${msi_queue},jobname=${msi_resources_jobname} \
--bind=${study_sharedfolder}:${study_sharedfolder} --container=${HOME}/qunex/qunex_suite-0.98.2.sif

error_preprocess_conc_bold3-bold4_dtseries_dentalvpress_10001_2023-08-07_13.47.35.586286.log (9.4 KB)

Estephan, hi!

I am sorry for the problems you are having. I have inspected the code and it seems that though the fix hs been done, it has not yet been included in the latest master. Let me re-test the fix today and check, why it has not yet been propagated, and then I’ll ask Jure to build the new container that includes the fix asap.

Thank you for your patience!

All the best,

Grega

Estephan, hi!

The fix is now in the develop branch and after final acceptance testing it will be in the master and container. It should be in the next release of the container. I have tested both preprocess_bold and preprocess_conc using your scenario and it ran without issues. I have now also included options to explicitly specify which signals besides bold you would like to high-pass (hipass_do) or low-pass (lopass_do) filter. This for instance allows you to filter also task regressors along with bold signal before running GLM. The default is to only high-pass filter ‘nuisance’ signal and to low-pass nuisance, movement, task, and event signals.

I hope this helps!

All the best,

Grega

Thanks for looking into this Grega, I’ll give it a shot once it’s available. Cheers.

Estephan

I want to share that I was able to run preprocess_conc sucesfully using qunex 0.98.4. Thank you.

Estephan

Great, thanks for informing us!

Jure