Description:
Hello, I tried to run the NODDI_Watson pipeline in my institution’s computer cluster and have found errors. This is the command I used:
Call:
msi_resources_time=04:00:00; msi_resources_nodes=1; msi_resources_ntaskspernode=12; msi_resources_mem=128000; msi_queue=v100; msi_resources_gpu=gpu:v100:1; msi_resources_jobname=XTRACK; \
study_sharedfolder=/home/moanae/shared/project_K99_ChrTMDHCP_qunex02; \
qunex_container dwi_xtract \
--batchfile=${study_sharedfolder}/processing/batch_K99Aim2.txt --sessionsfolder=${study_sharedfolder}/sessions \
--species="human" \
--parsessions=1 --overwrite="yes" \
--nv \
--bash_pre="module load fsl/6.0.7.9 cuda/11.2" \
--envars="FSLDIR=>${FSLDIR}" \
--scheduler=SLURM,time=${msi_resources_time},nodes=${msi_resources_nodes},cpus-per-task=${msi_resources_ntaskspernode},mem=${msi_resources_mem},partition=${msi_queue},gres=${msi_resources_gpu},jobname=${msi_resources_jobname} \
--bind=${study_sharedfolder}:${study_sharedfolder},${FSLDIR}:${FSLDIR} --container=${HOME}/qunex/qunex_suite-0.99.2d.sif
Right in the beginning of the log file, there are errors reported:
Logs:
# Generated by QuNex 0.99.2 on 2024-04-20_14.10.23.843019#
------------------------------------------------------------
Running external command via QuNex:
/opt/qunex/qx_library/etc/cudimot/cuda_10.2/bin/Pipeline_NODDI_Watson.sh /home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10005/hcp/10005/T1w/Diffusion
------------------------------------------------------------
---------------------------------------------------------------------------------
------------------------------------ CUDIMOT ------------------------------------
----------------------------- MODEL: NODDI_Watson -----------------------------
---------------------------------------------------------------------------------
subjectdir is /home/moanae/shared/project_K99_ChrTMDHCP_qunex02/sessions/10005/hcp/10005/T1w/Diffusion
Making output directory structure
Copying files to output directory
cp: relocation error: /lib64/libacl.so.1: symbol getxattr, version ATTR_1.0 not defined in file libattr.so.1 with link time reference
cp: relocation error: /lib64/libacl.so.1: symbol getxattr, version ATTR_1.0 not defined in file libattr.so.1 with link time reference
Queue Dtifit
Queue GridSearch process
I tried then to get inside the container in a GPU node to try to run that command inside the container, but even without issuing the command there are errors related to some libraries:
cn2110:~ moana004$ module load singularity/current python3
cn2110:~ moana004$ study_sharedfolder=/home/moanae/shared/project_K99_ChrTMDHCP_qunex02
cn2110:~ moana004$ singularity shell -B ${study_sharedfolder}:${study_sharedfolder} ${HOME}/qunex/qunex_suite-0.99.2d.sif
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
Apptainer> source /opt/qunex/env/qunex_environment.sh
--> unsetting the following environment variables: PATH MATLABPATH PYTHONPATH QUNEXVer TOOLS QUNEXREPO QUNEXPATH QUNEXEXTENSIONS QUNEXLIBRARY QUNEXLIBRARYETC TemplateFolder FSL_FIXDIR FREESURFERDIR FREESURFER_HOME FREESURFER_SCHEDULER FreeSurferSchedulerDIR WORKBENCHDIR DCMNIIDIR DICMNIIDIR MATLABDIR MATLABBINDIR OCTAVEDIR OCTAVEPKGDIR OCTAVEBINDIR RDIR HCPWBDIR AFNIDIR PYLIBDIR FSLDIR FSLBINDIR PALMDIR QUNEXMCOMMAND HCPPIPEDIR CARET7DIR GRADUNWARPDIR HCPPIPEDIR_Templates HCPPIPEDIR_Bin HCPPIPEDIR_Config HCPPIPEDIR_PreFS HCPPIPEDIR_FS HCPPIPEDIR_PostFS HCPPIPEDIR_fMRISurf HCPPIPEDIR_fMRIVol HCPPIPEDIR_tfMRI HCPPIPEDIR_dMRI HCPPIPEDIR_dMRITract HCPPIPEDIR_Global HCPPIPEDIR_tfMRIAnalysis HCPCIFTIRWDIR MSMBin HCPPIPEDIR_dMRITractFull HCPPIPEDIR_dMRILegacy AutoPtxFolder EDDYCUDA USEOCTAVE QUNEXENV CONDADIR MSMBINDIR MSMCONFIGDIR R_LIBS FSL_FIX_CIFTIRW FSFAST_HOME SUBJECTS_DIR MINC_BIN_DIR MNI_DIR MINC_LIB_DIR MNI_DATAPATH FSF_OUTPUT_FORMAT ANTSDIR CUDIMOT
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
Generated by QuNex
------------------------------------------------------------------------
Version: 0.99.2
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
User: moana004
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
System: cn2110
OS: RedHat Linux #1 SMP Wed Mar 20 15:54:52 UTC 2024
------------------------------------------------------------------------
\u2588\u2588\u2588\u2588\u2588\u2588\ \u2551 \u2588\u2588\ \u2588\u2588\
\u2588\u2588 __\u2588\u2588\ \u2551 \u2588\u2588\u2588\ \u2588\u2588 |
\u2588\u2588 / \u2588\u2588 |\u2588\u2588\ \u2588\u2588\ \u2551 \u2588\u2588\u2588\u2588\ \u2588\u2588 | \u2588\u2588\u2588\u2588\u2588\u2588\ \u2588\u2588\ \u2588\u2588\
\u2588\u2588 | \u2588\u2588 |\u2588\u2588 | \u2588\u2588 | \u2551 \u2588\u2588 \u2588\u2588\\u2588\u2588 |\u2588\u2588 __\u2588\u2588\\\u2588\u2588\ \u2588\u2588 |
\u2588\u2588 | \u2588\u2588 |\u2588\u2588 | \u2588\u2588 | \u2551 \u2588\u2588 \\u2588\u2588\u2588\u2588 |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588 |\\u2588\u2588\u2588\u2588 /
\u2588\u2588 \u2588\u2588\\u2588\u2588 |\u2588\u2588 | \u2588\u2588 | \u2551 \u2588\u2588 |\\u2588\u2588\u2588 |\u2588\u2588 ____|\u2588\u2588 \u2588\u2588\
\\u2588\u2588\u2588\u2588\u2588\u2588 / \\u2588\u2588\u2588\u2588\u2588\u2588 | \u2551 \u2588\u2588 | \\u2588\u2588 |\\u2588\u2588\u2588\u2588\u2588\u2588\u2588\\u2588\u2588 /\\u2588\u2588\
\___\u2588\u2588\u2588\ \______/ \u2551 \__| \__| \_______\__/ \__|
\___| \u2551
DEVELOPED & MAINTAINED BY:
Anticevic Lab, Yale University
Mind & Brain Lab, University of Ljubljana
Murray Lab, Yale University
COPYRIGHT & LICENSE NOTICE:
Use of this software is subject to the terms and conditions defined in
'LICENSES' which is a part of the QuNex Suite source code package:
https://gitlab.qunex.yale.edu/qunex/qunex/-/tree/master/LICENSES
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
---> Setting up Octave
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libdlfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libvglfaker.so' from LD_PRELOAD cannot be preloaded: ignored.
I have used this qunex container version for other analysis pipelines, including those running in a GPU node (such as FSL xtract) without an issue, so it seems that this is related specifically to NODDI. Any thoughts on what be going on here? Thank you.
Estephan