[RESOLVED] Preprocess_bold unclear log message

Hi,
I’ve running preprocess_bold function on my previously hcp preprocessed data. I’ve followed the steps reported here: oriadev / qunex / wiki / UsageDocs / BOLDPreprocessing — Bitbucket
During the procedure the following message is printed:

===> Final report for command preprocess_bold
... 06B ---> bolds: 1 ready [1], 0 not ready [], 0 already processed [], 1 ran ok [1], 0 failed [], 0 skipped []
===> Successful completion of all tasks

but after that the following message appear:

Running external command: octave -q --no-init-file --eval "try fc_preprocess('/home/admin01/qunex/output/ruminanti_P2/sessions/06B', 1, 0, 'shrcl', 'm,V,WM,WB', [], '', 0.800000, '', '', true, '.dtseries.nii', 'radius:50|fdt:0.50|dvarsmt:3.00|dvarsmet:1.60|after:0|before:0|reject:udvarsme', 'hipass=linear|regress=spline|lopass=linear', 'boldname=bold|surface_smooth=2.000000|volume_smooth=2.000000|voxel_smooth=1.000000|hipass_filter=0.008000|lopass_filter=0.090000|omp_threads=0|framework_path=|wb_command_path=|smooth_mask=nonzero|dilate_mask=/opt/qunex/library/data/atlases/MNITemplates/MNI152_T1_2mm_brain_mask.nii.gz|glm_matrix=both|glm_residuals=save|glm_name=|bold_tail=_Atlas|ref_bold_tail=|bold_variant=|img_suffix='); catch ME, general_report_crash(ME); exit(1), end; exit"

You can follow command's progress in:
/home/admin01/qunex/output/ruminanti_P2/processing/logs/comlogs/tmp_preprocess_bold_B1_06B_2022-09-01_09.44.43.870881.log

Looking at the log folder, the log file (reported below) don’t mention any failure. So I don’t understand if the preproc has been completed correctly or if there’s some error I’m missing.

I report the complete prompt log below:

Generated by QuNex 
------------------------------------------------------------------------ 
Version: 0.93.6 
User: root 
System: 317be48ce35c 
OS: RedHat Linux #49~20.04.1-Ubuntu SMP Thu Aug 4 19:15:44 UTC 2022 
------------------------------------------------------------------------ 
 
        ██████\                  ║      ██\   ██\                        
       ██  __██\                 ║      ███\  ██ |                       
       ██ /  ██ |██\   ██\       ║      ████\ ██ | ██████\ ██\   ██\     
       ██ |  ██ |██ |  ██ |      ║      ██ ██\██ |██  __██\\██\ ██  | 
       ██ |  ██ |██ |  ██ |      ║      ██ \████ |████████ |\████  /     
       ██ ██\██ |██ |  ██ |      ║      ██ |\███ |██   ____|██  ██\      
       \██████ / \██████  |      ║      ██ | \██ |\███████\██  /\██\     
        \___███\  \______/       ║      \__|  \__| \_______\__/  \__|    
            \___|                ║                                       
 
 
                       DEVELOPED & MAINTAINED BY: 
 
                    Anticevic Lab, Yale University 
               Mind & Brain Lab, University of Ljubljana 
                     Murray Lab, Yale University 
 
                      COPYRIGHT & LICENSE NOTICE: 
 
Use of this software is subject to the terms and conditions defined in 
'LICENSE.md' which is a part of the QuNex Suite source code package: 
https://bitbucket.org/oriadev/qunex/src/master/LICENSE.md 
 
 ---> Setting up Octave  


 ........................ Running QuNex v0.93.6 ........................ 


--- Full QuNex call for command: preprocess_bold 

gmri preprocess_bold --sessionsfolder="/home/admin01/qunex/output/ruminanti_P2/sessions" --overwrite="yes" --modality="BOLD" --boldsuffix="Atlas" --bolds="all" --sessions="/home/admin01/qunex/output/ruminanti_P2/processing/batch_hcp8.txt" 

--------------------------------------------------------- 



WARNING: Use of parameters with changed name(s)!
         The following parameters have new names:
         ... hcp_Pipeline is now hcp_pipeline!
         Please correct the listed parameter names in command line or batch file!

WARNING: Use of deprecated parameter value(s)!
       The following parameter values have changed:
         ... original (hcp_filename) is now userdefined!
         Please correct the listed parameter values in command line or batch file!
WARNING: Parameter qx_cifti_tail was not specified. Its value was imputed from parameter hcp_cifti_tail and set to '_Atlas'!
WARNING: Parameter qx_nifti_tail was not specified. Its value was imputed from parameter hcp_nifti_tail and set to ''!
WARNING: Parameter cifti_tail was not specified. Its value was imputed from parameter qx_cifti_tail and set to '_Atlas'!
WARNING: Parameter nifti_tail was not specified. Its value was imputed from parameter qx_nifti_tail and set to ''!

# Generated by QuNex 0.93.6 on 2022-09-01_09.44.43.836922
#
=================================================================
gmri preprocess_bold \
  --sessionsfolder="/home/admin01/qunex/output/ruminanti_P2/sessions" \
  --overwrite="yes" \
  --modality="BOLD" \
  --boldsuffix="Atlas" \
  --bolds="all" \
  --sessions="/home/admin01/qunex/output/ruminanti_P2/processing/batch_hcp8.txt" \
=================================================================

Starting multiprocessing sessions in /home/admin01/qunex/output/ruminanti_P2/processing/batch_hcp8.txt with a pool of 16 concurrent processes

---- Running local

Adding processing of session 06B to the pool at Thursday, 01. September 2022 09:44:43

---------------------------------------------------------
Session id: 06B 
[started on Thursday, 01. September 2022 09:44:43]
Preprocessing all BOLD files as specified in --bolds.
Files in 'images/functional will be processed.
Running Preprocessing bold runs ...
Processing 1 BOLDs in parallel

Working on: bold1 ...
... checking for data
    ... movement data present
    ... bold statistics data present
    ... bold scrubbing data present
    ... bold nuisance signal data present
    ... bold data present

running matlab/octave fc_preprocess on /home/admin01/qunex/output/ruminanti_P2/sessions/06B/images/functional bold 1
---> preprocess_bold test file [bold1_Atlas_s_hpss_res-mVWMWB_lpss.dtseries.nii] present
---> logfile: /home/admin01/qunex/output/ruminanti_P2/processing/logs/comlogs/done_preprocess_bold_B1_06B_2022-09-01_09.44.43.870881.log

Bold preprocessing completed on Thursday, 01. September 2022 09:51:35
---------------------------------------------------------


===> Final report for command preprocess_bold
... 06B ---> bolds: 1 ready [1], 0 not ready [], 0 already processed [], 1 ran ok [1], 0 failed [], 0 skipped []
===> Successful completion of all tasks
Running external command: octave -q --no-init-file --eval "try fc_preprocess('/home/admin01/qunex/output/ruminanti_P2/sessions/06B', 1, 0, 'shrcl', 'm,V,WM,WB', [], '', 0.800000, '', '', true, '.dtseries.nii', 'radius:50|fdt:0.50|dvarsmt:3.00|dvarsmet:1.60|after:0|before:0|reject:udvarsme', 'hipass=linear|regress=spline|lopass=linear', 'boldname=bold|surface_smooth=2.000000|volume_smooth=2.000000|voxel_smooth=1.000000|hipass_filter=0.008000|lopass_filter=0.090000|omp_threads=0|framework_path=|wb_command_path=|smooth_mask=nonzero|dilate_mask=/opt/qunex/library/data/atlases/MNITemplates/MNI152_T1_2mm_brain_mask.nii.gz|glm_matrix=both|glm_residuals=save|glm_name=|bold_tail=_Atlas|ref_bold_tail=|bold_variant=|img_suffix='); catch ME, general_report_crash(ME); exit(1), end; exit"

You can follow command's progress in:
/home/admin01/qunex/output/ruminanti_P2/processing/logs/comlogs/tmp_preprocess_bold_B1_06B_2022-09-01_09.44.43.870881.log
------------------------------------------------------------

log file:

# Generated by QuNex 0.93.6 on 2022-09-01_10.08.49.240930
#
------------------------------------------------------------
Running external command via QuNex:

octave -q --no-init-file --eval "try fc_preprocess('/home/admin01/qunex/output/ruminanti_P2/sessions/06A', 1, 0, 'shrcl', 'm,V,WM,WB', [], '', 0.800000, '', '', true, '.dtseries.nii', 'radius:50|fdt:0.50|dvarsmt:3.00|dvarsmet:1.60|after:0|before:0|reject:udvarsme', 'hipass=linear|regress=spline|lopass=linear', 'boldname=bold|surface_smooth=2.000000|volume_smooth=2.000000|voxel_smooth=1.000000|hipass_filter=0.008000|lopass_filter=0.090000|omp_threads=0|framework_path=|wb_command_path=|smooth_mask=nonzero|dilate_mask=/opt/qunex/library/data/atlases/MNITemplates/MNI152_T1_2mm_brain_mask.nii.gz|glm_matrix=both|glm_residuals=save|glm_name=|bold_tail=_Atlas|ref_bold_tail=|bold_variant=|img_suffix='); catch ME, general_report_crash(ME); exit(1), end; exit"

Test file: 
/home/admin01/qunex/output/ruminanti_P2/sessions/06A/images/functional/bold1_Atlas_s_hpss_res-mVWMWB_lpss.dtseries.nii
------------------------------------------------------------


Running preproces script v0.9.17 [.dtseries.nii]
--------------------------------

Parameters:
---------------
       sessionf: /home/admin01/qunex/output/ruminanti_P2/sessions/06A
           bold: 1
           omit: 0
           doIt: shrcl
           rgss: m,V,WM,WB
           task: [0  0]
          efile: 
             TR: 0.80
   eventrstring: 
        variant: 
      overwrite: 1
           tail: .dtseries.nii
          scrub: radius:50|fdt:0.50|dvarsmt:3.00|dvarsmet:1.60|after:0|before:0|reject:udvarsme
        ignores: hipass=linear|regress=spline|lopass=linear
        options: boldname=bold|surface_smooth=2.000000|volume_smooth=2.000000|voxel_smooth=1.000000|hipass_filter=0.008000|lopass_filter=0.090000|omp_threads=0|framework_path=|wb_command_path=|smooth_mask=nonzero|dilate_mask=/opt/qunex/library/data/atlases/MNITemplates/MNI152_T1_2mm_brain_mask.nii.gz|glm_matrix=both|glm_residuals=save|glm_name=|bold_tail=_Atlas|ref_bold_tail=|bold_variant=|img_suffix=

Options used:
----------------
        boldname: bold
  surface_smooth: 2
   volume_smooth: 2
    voxel_smooth: 1
   lopass_filter: 0.09
   hipass_filter: 0.008
  framework_path: 
 wb_command_path: 
     omp_threads: 0
     smooth_mask: nonzero
     dilate_mask: /opt/qunex/library/data/atlases/MNITemplates/MNI152_T1_2mm_brain_mask.nii.gz
      glm_matrix: both
   glm_residuals: save
        glm_name: 
       bold_tail: _Atlas
   ref_bold_tail: 
    bold_variant: 
      img_suffix: 

---> removing old files:
     ... /home/admin01/qunex/output/ruminanti_P2/sessions/06A/images/functional/bold1_Atlas_s.dtseries.nii
     ... /home/admin01/qunex/output/ruminanti_P2/sessions/06A/images/functional/bold1_Atlas_s_hpss.dtseries.nii

Smoothing /home/admin01/qunex/output/ruminanti_P2/sessions/06A/images/functional/bold1_Atlas.dtseries.nii 
---> running wb_command -cifti-smoothing
     ... smoothing ... done!

High-pass filtering /home/admin01/qunex/output/ruminanti_P2/sessions/06A/images/functional/bold1_Atlas_s.dtseries.nii 
---> reading /home/admin01/qunex/output/ruminanti_P2/sessions/06A/images/functional/bold1_Atlas_s.dtseries.nii ... done!
===> Temporal filtering (15/10/22)
---> triming: 0 on start, 0 on end
---> remaining bad frames: 0, action: linear
---> hipass frame         1    2    3    4    5    6    7    8    9   10   11   12   13   14   15   16   17   18   19   20   21   22   23   24   25   26   27   28   29   30   31   32   33   34   35   36   37   38   39   40   41   42   43   44   45   46   47   48   49   50   51   52   53   54   55   56   57   58   59   60   61   62   63   64   65   66   67   68   69   70   71   72   73   74   75   76   77   78   79   80   81   82   83   84   85   86   87   88   89   90   91   92   93   94   95   96   97   98   99  100  101  102  103  104  105  106  107  108  109  110  111  112  113  114  115  116  117  118  119  120  121  122  123  124  125  126  127  128  129  130  131  132  133  134  135  136  137  138  139  140  141  142  143  144  145  146  147  148  149  150  151  152  153  154  155  156  157  158  159  160  161  162  163  164  165  166  167  168  169  170  171  172  173  174  175  176  177  178  179  180  181  182  183  184  185  186  187  188  189  190  191  192  193  194  195  196  197  198  199  200  201  202  203  204  205  206  207  208  209  210  211  212  213  214  215  216  217  218  219  220  221  222  223  224  225  226  227  228  229  230  231  232  233  234  235  236  237  238  239  240  241  242  243  244  245  246  247  248  249  250  251  252  253  254  255  256  257  258  259  260  261  262  263  264  265  266  267  268  269  270  271  272  273  274  275  276  277  278  279  280  281  282  283  284  285  286  287  288  289  290  291  292  293  294  295  296  297  298  299  300  301  302  303  304  305  306  307  308  309  310  311  312  313  314  315  316  317  318  319  320  321  322  323  324  325  326  327  328  329  330  331  332  333  334  335  336  337  338  339  340  341  342  343  344  345  346  347  348  349  350  351  352  353  354  355  356  357  358  359  360  361  362  363  364  365  366  367  368  369  370  371  372  373  374  375  376  377  378  379  380  381  382  383  384  385  386  387  388  389  390  391  392  393  394  395  396  397  398  399  400  401  402  403  404  405  406  407  408  409  410  411  412  413  414  415  416  417  418  419  420  421  422  423  424  425  426  427  428  429  430  431  432  433  434  435  436  437  438  439  440  441  442  443  444  445  446  447  448  449  450
 ... saved!

Removing residual /home/admin01/qunex/output/ruminanti_P2/sessions/06A/images/functional/bold1_Atlas_s_hpss.dtseries.nii  excluding 0 bad frames ... saved!

Low-pass filtering /home/admin01/qunex/output/ruminanti_P2/sessions/06A/images/functional/bold1_Atlas_s_hpss_res-mVWMWB.dtseries.nii 
===> Temporal filtering (15/10/22)
---> triming: 0 on start, 0 on end
---> remaining bad frames: 0, action: linear
---> lopass frame          1    2    3    4    5    6    7    8    9   10   11   12   13   14   15   16   17   18   19   20   21   22   23   24   25   26   27   28   29   30   31   32   33   34   35   36   37   38   39   40   41   42   43   44   45   46   47   48   49   50   51   52   53   54   55   56   57   58   59   60   61   62   63   64   65   66   67   68   69   70   71   72   73   74   75   76   77   78   79   80   81   82   83   84   85   86   87   88   89   90   91   92   93   94   95   96   97   98   99  100  101  102  103  104  105  106  107  108  109  110  111  112  113  114  115  116  117  118  119  120  121  122  123  124  125  126  127  128  129  130  131  132  133  134  135  136  137  138  139  140  141  142  143  144  145  146  147  148  149  150  151  152  153  154  155  156  157  158  159  160  161  162  163  164  165  166  167  168  169  170  171  172  173  174  175  176  177  178  179  180  181  182  183  184  185  186  187  188  189  190  191  192  193  194  195  196  197  198  199  200  201  202  203  204  205  206  207  208  209  210  211  212  213  214  215  216  217  218  219  220  221  222  223  224  225  226  227  228  229  230  231  232  233  234  235  236  237  238  239  240  241  242  243  244  245  246  247  248  249  250  251  252  253  254  255  256  257  258  259  260  261  262  263  264  265  266  267  268  269  270  271  272  273  274  275  276  277  278  279  280  281  282  283  284  285  286  287  288  289  290  291  292  293  294  295  296  297  298  299  300  301  302  303  304  305  306  307  308  309  310  311  312  313  314  315  316  317  318  319  320  321  322  323  324  325  326  327  328  329  330  331  332  333  334  335  336  337  338  339  340  341  342  343  344  345  346  347  348  349  350  351  352  353  354  355  356  357  358  359  360  361  362  363  364  365  366  367  368  369  370  371  372  373  374  375  376  377  378  379  380  381  382  383  384  385  386  387  388  389  390  391  392  393  394  395  396  397  398  399  400  401  402  403  404  405  406  407  408  409  410  411  412  413  414  415  416  417  418  419  420  421  422  423  424  425  426  427  428  429  430  431  432  433  434  435  436  437  438  439  440  441  442  443  444  445  446  447  448  449  450
 ... saved!
==> preproces BOLD finished successfully


===> Successful completion of task

Thanks for the help,

Chiara

Chiara, hi!

Given that the summary report is okay and no error was reported, the processing ran fine. The information on running octave and the temporary log is provided so that (i) the user can run the command manually if it were ever needed, and (ii) the user can keep track of the progress of an individual processing job. Indeed, the information on the octave run and the log file was printed out in a way that is confusing to the user. We’ll investigate why that occurs and how to improve the reporting so that it is not confusing. Thank you for alerting us to it!

All the best,

Grega

Hi Grega,

Thank you for the answer.

Have a nice day,
Chiara