Running the following probtrackX GPU command:
---------------------------
/opt/qunex/bash/qx_utilities/diffusion_tractography_dense/tractography_gpu_scripts/run_matrix3.sh /home/ehui/qunex/cimt/sessions V3 3000 no no
---------------------------
-- Queueing Probtrackx
PROBTRACKX2 VERSION GPU
Log directory is: /home/ehui/qunex/cimt/sessions/V3/hcp/V3/MNINonLinear/Results/Tractography
Running in seedmask mode
Loading tractography data
Number of Seeds: 192184
Dimensions Matrix3 91282 x 91282
Time Loading Data: 34 seconds
...................Allocated GPU 0...................
Device memory available (MB): 48418 ---- Total device memory(MB): 48685
Memory required for allocating data (MB): 826
CUDA Runtime Error: invalid device symbol
Any help would be greatly appreciated! Thanks so much!
Ed, do you maybe have any older QuNex containers lying around (say 0.94.x or 0.95.x)? If so, could you run the above command with one of those. I am also having some issues with probtrackx in the latest container on our end.
I should have 0.94.14’s singularity in an external harddisk. Worst case, is there any way I could download 0.94.14’s singularity somewhere in your repository?
I gave you access to the https://gitlab.qunex.yale.edu/qunexdev/qunexcontainer_dev repository. I am uploading 0.94.14 container in there (should be there in an hour or so). You can pull if from there the same way as from the public QuNex repository.
Unfortunately the same error occurred even when I used 0.94.14 container. Not sure if it had anything to do with the make of my GPU (which is A6000). I had no issues running the script in my another workstation which has a V100 nvidia GPU.
I believe that A6000 supports only CUDA 11. Were you able to run bedpostx on A6000 with CUDA 10? That would be weird. We will be soon upgrading to FSL 6.0.6 and it seems like it supports CUDA 11 out of the box (Probtrackx GPU).
We plan to update to FSL 6.0.6 in the first half of January.