[RESOLVED] Slurm scheduler job submission error - No error or log files, but job not submitted

Issues
1

Call:

qunex runPALM \
    --image="/gpfs/project/fas/n3/Studies/BlackThorn/analysis/PRT/2020/BOLD/BOLD.Dense.GBC.SCZ.dtseries.nii" \
    --design="name:/gpfs/project/fas/n3/Studies/BlackThorn/analysis/PRT/2020/PALM/Within/PRT.Within|d:dRB.Mean.SCZ|t:t1|eb:ebSCZ" \
    --args="n:100|accel:tail|twotail|zstat|demean" \
    --root="/gpfs/project/fas/n3/Studies/BlackThorn/analysis/PRT/2020/PALM/Within/PRT.Within.RB.Mean.SCZ" \
    --overwrite="yes" \
    --scheduler="SLURM,jobname=PRTpalm,time=10-00:00:00,ntasks=3,mem-per-cpu=50000,partition=pi_anticevic_gpu"

Returns:

===> Running scheduler for command runPALM 
---> submitting runPALM 
gmri runPALM --image="/gpfs/project/fas/n3/Studies/BlackThorn/analysis/PRT/2020/BOLD/BOLD.Dense.GBC.SCZ.dtseries.nii" --args="n:100|accel:tail|twotail|zstat|demean" --design="name:/gpfs/project/fas/n3/Studies/BlackThorn/analysis/PRT/2020/PALM/Within/PRT.Within|d:dRB.Mean.SCZ|t:t1|eb:ebSCZ" --root="/gpfs/project/fas/n3/Studies/BlackThorn/analysis/PRT/2020/PALM/Within/PRT.Within.RB.Mean.SCZ" --overwrite="yes" 
Submitting:
------------------------------
#!/bin/sh
#SBATCH --mem-per-cpu=50000
#SBATCH --ntasks=3
#SBATCH --time=10-00:00:00
#SBATCH --partition=pi_anticevic_gpu
#SBATCH --job-name=PRTpalm(1)
#SBATCH -o /gpfs/loomis/pi/n3/software/Singularity/SLURM_runPALM.2020-05-28.11.05.45.679721.log
#SBATCH -e /gpfs/loomis/pi/n3/software/Singularity/SLURM_runPALM.2020-05-28.11.05.45.679721.log
gmri runPALM --image="/gpfs/project/fas/n3/Studies/BlackThorn/analysis/PRT/2020/BOLD/BOLD.Dense.GBC.SCZ.dtseries.nii" --args="n:100|accel:tail|twotail|zstat|demean" --design="name:/gpfs/project/fas/n3/Studies/BlackThorn/analysis/PRT/2020/PALM/Within/PRT.Within|d:dRB.Mean.SCZ|t:t1|eb:ebSCZ" --root="/gpfs/project/fas/n3/Studies/BlackThorn/analysis/PRT/2020/PALM/Within/PRT.Within.RB.Mean.SCZ" --overwrite="yes"
===> Submitted jobs 
     NA -> runPALM

However, no new jobs are running via squeue -u ba352 and no log files are generated in the specified --root folder

2

@brendan.adkinson it seems that the initial part of the call is missing. It is needed to know exactly where to look for further information on why the submission failed. My guess is that for some reason SLURM did not process the request (as indicted by NA in the submitted jobs list). It might be necessary to extend the code that submits the SLURM call to have more precise information for the reason SLURM rejected or could not process the call.

3

Thanks @grega - appreciated!

4

@brendan.adkinson, the initial part of the qunex call is still missing. It starts with --image=.... Can you please provide the full call that was made.

5

Sorry - I included the ``` in the wrong place when formatting here. I just adjusted it.

6

@brendan.adkinson

Please use the latest public release is now out officially to check if this issue is resolved. You can download it here:

https://qunex.yale.edu/qunex-registration/

Go ahead and register for the latest stable version and check if the issue above is now resolved. If not, please send us the logs and we will investigate.