[RESOLVED] Changing location of log files

Use Questions
1

Hi, I’m running QuNex run_turnkye using docker container and SLURM. I have two questions about changing the locations of log files.

The command I’m running is:

qunex_container run_turnkey \
    --rawdatainput="${QUNEX_TMP_DATA}" \
    --dataformat="${data_format}" \
    --paramfile="${QUNEX_TMP_DATA}/${qx_session_name}_parameters.txt" \
    --mappingfile="${QUNEX_TMP_DATA}/${qx_session_name}_mapping.txt" \
    --workingdir="${WORK_DIR}" \
    --projectname="${STUDY_NAME}" \
    --path="${WORK_DIR}/${STUDY_NAME}" \
    --sessions="${qx_session_name}" \
    --sessionsfoldername="sessions" \
    --log="study|session" \
    --hcp_dwi_nogpu="True" \
    --hcp_filename='userdefined' \
    --turnkeytype="${RUNTURNKEY_TYPE}" \
    --dockeropt="-v ${WORK_DIR}:${WORK_DIR}" \
    --container="${QUNEX_CONTAINER}" \
    --turnkeysteps="${runturnkey_steps_from_file}" \
    --scheduler="SLURM,cpus-per-task=8,time=48:00:00,mem-per-cpu=2000,jobname=${STUDY_NAME},--output=${log_location}"

  1. The first question is about the use of the log parameter. It seems that the logs are created successfully in /processing/logs but there is no folder //logs or any logs in this session folder.

  2. The second question is about the file created as a result of using SLURM that contains the output to the terminal. In my case it is created in my home directory, for example:
    /home/theoweiss/qunex_container_2023-12-26_10.18.30.378665.txt
    I’ve tried using --output parameter but then I got:

------------------------------
#!/bin/sh
#SBATCH --cpus-per-task=8
#SBATCH --time=48:00:00
#SBATCH --mem-per-cpu=2000
#SBATCH --output=/home/PreProcessing/qunex/tmp_data/log.out
#SBATCH --job-name=yalab
#SBATCH -o /home/theoweiss/qunex_container_2023-12-26_13.53.21.575330.txt
#SBATCH -e /home/theoweiss/qunex_container_2023-12-26_13.53.21.575330.txt

docker run -v /home/PreProcessing/qunex:/home/PreProcessing/qunex -v /home/theoweiss:/home/theoweiss gitlab.qunex.yale.edu:5002/qunex/qunexcontainer:0.98.6a bash /home/theoweiss/qunex_container_command_2023-12-26_13.53.21.575154.sh

So I wonder if the ‘output’ parameter is getting reassigned.
Thank you for your time, Theo

2

Hi Theo,

The logging system we use right now is a bit archaic and unintuitive. By default the logs are put into the processing/logs folder. In there they are further divided into comlogs (more detailed logs with all outputs) and runlogs (a kind of top-level summary logs, or sometimes just command calls). If you want to change the location of these logs into something else (not processing/logs) then --logfolder is your best bet. When I run things myself, I usually create --logfolder for each command, e.g. --logfolder=${study_folder}/processing/logs/hcp_fmri_volume for hcp_fmri_volume. In your case, since you are using run_turnkey, you could create this for each session that you are processing and sort the logs this way.

  1. Can you point me to the part of the documentation about the --log parameter (--log="study|session"). I think this will not work as set in your example, it could be that our documentation is written poorly or misleading. Sorry for this.

  2. QuNex tries to put the SLURM log into the /processing/logs/batchlogs by default. If that does not exist yet (e.g., before create_study) it puts it into home. You can control this with the --output parmeters. Try changing your example above like this:

--output="both:${log_location}" \
--scheduler="SLURM,cpus-per-task=8,time=48:00:00,mem-per-cpu=2000,jobname=${STUDY_NAME}"

both denotes that stderr and stdout, so both errors and outputs are put into the same file.

We are planning a major rework of the logging structure in Q1 2024. The rework will change the way logs are organized as well as give users better control over what gets put where.

Kind regards, Jure

3

Thank you very much for your explanation.

  1. The use of --log parameter is described here:
    Logging and log files — QuNex documentation
    Is it possible to save the comlogs in multiple folders as suggested with the comma or pipe?

  2. The /processing/logs/batchlogs exists but it is empty in my case. I will try the --output parameter.

Thanks again,
Theo

4

I believe the study|session log functionality works only for a subset of commands, sadly run_turnkey is not among them. Like I said we are actively working on improving the logging functionality and remove these inconsistencies.

Jure

5

No worries, the default folders work for me and The --output parameter seems very helpful.
Thank you,
Theo