[RESOLVED] PBS scheduler issues

Issues
1

Hello,
I tested our PBS scheduler with this simple command:

qunex_container list_dicom
–folder=“${HOME}/files”
–targetfile=“${HOME}/dicomFields.csv”
–bind=“${HOME}/files:/data”
–container=“${HOME}/qunex/qunex_suite-0.93.6.sif”
–scheduler=“PBS,nodes=1:ppn=4,walltime=1:00:00,mem=8gb,jobname=test_scheduler”

That gives:

Submitting:

#PBS -l nodes=1:ppn=4
#PBS -l walltime=1:00:00
#PBS -l mem=8gb
#PBS -N test_scheduler
#PBS -o /storage/brno2/home/pe_so/qunex_container_2022-08-07_06.41.33.359515.txt
#PBS -e /storage/brno2/home/pe_so/qunex_container_2022-08-07_06.41.33.359515.txt

singularity exec --cleanenv -B /storage/brno2/home/pe_so/files:/data /storage/brno2/home/pe_so/qunex/qunex_suite-0.93.6.sif bash /storage/brno2/home/pe_so/qunex_container_command_2022-08-07_06.41.33.342557.scheduler

However, our HPC uses PBS Pro that requires select syntax for specifying resources, e.g. #PBS -l select=1:ncpus=4:mem=8gb, so I got this error: qsub: Old syntax rejected. Please use ‘select’ syntax.

Is there a way around this? I will be grateful for help.

Best,
Petr Sojka

2

Hi Petr,

I coded a quick fix for the issue you are outlining, please test
qunex_container (40.2 KB)

The call would be:

qunex_container list_dicom \
  --folder="/data" \
  --targetfile="${HOME}/dicomFields.csv" \
  --bind="${HOME}/files:/data" \
  --container="${HOME}/qunex/qunex_suite-0.93.6.sif" \
  --scheduler="PBS,select=1:ncpus=4:mem=8gb,walltime=1:00:00,jobname=list_dicom"

Note that I changed folder parameter to /data as that is what ${HOME}/files are bound to inside the container.

The main issue here is that we no longer have access to any of HPC systems that use PBS. As a consequence we cannot properly test it. Because of this our long term support for PBS is under a big question mark. Also, the above fix might not work in the first try and there could be some back and forth here :slight_smile:.

If we were to deprecate PBS, then you would have to schedule QuNex jobs the “traditional” way. By preparing two files.

File 1: list_dicom.pbs would be:

#PBS -l select=1:ncpus=4:mem=8gb
#PBS -l walltime=1:00:00
#PBS -N list_dicom
#PBS -o /Users/jure/qunex_container_2022-08-08_09.17.20.400521.txt
#PBS -e /Users/jure/qunex_container_2022-08-08_09.17.20.400521.txt

singularity exec --cleanenv -B /jure/files:/data /jure/qunex/qunex_suite-0.93.6.sif bash ~/list_dicom.sh

File 2: list_dicom.sh would then be:

source /opt/qunex/env/qunex_environment.sh
qunex list_dicom \
  --folder="/data" \
  --targetfile="/jure/dicomFields.csv

To execute everything, you then run qsub list_dicom.pbs.

Hope this helped!

3

Hi Jure,
many thanks, it works as expected! Also thank you for the alternative “traditional” solution.
Warm Regards, Petr