使用 condor 匹配输出和错误文件制作当前文件和扩展名？ 让 PBS 和 Slurm 具有相同的输出文件

问题描述

如何让 condor 命名我的文件如下：

meta_learning_experiments_submission.py.e451863 meta_learning_experiments_submission.py.o444375

$(FILENAME).e$(CLUSTER)
$(FILENAME).e$(CLUSTER)

我试过了，但似乎不起作用。

例如以便在我执行 qsub 时匹配 PBS 的默认行为？

---

我也想知道如何让slurm也与之匹配，这样就完美了（所以三个集群都可以工作）。

我试过了：

#SBATCH --error="(filename).%j.%N.err"

但是没用

---

赏金

我想自动设置文件名而不用硬编码。

---

相关：

• https://www.quora.com/unanswered/How-do-I-make-a-current-file-the-extension-with-condor-match-output-error-files-to-have-PBS-and-Slurm-have-the-same-output-files

---

这里的上下文是我当前提交的文件：

####################
#
# Experiments script
# Simple HTCondor submit description file
#
#
# chmod a+x test_condor.py
# chmod a+x experiments_meta_model_optimization.py
# chmod a+x meta_learning_experiments_submission.py
# chmod a+x download_miniImagenet.py
# chmod a+x ~/meta-learning-lstm-pytorch/main.py
# chmod a+x /home/miranda9/automl-meta-learning/automl-proj/meta_learning/datasets/rand_fc_nn_vec_mu_ls_gen.py
# chmod a+x /home/miranda9/automl-meta-learning/automl-proj/experiments/meta_learning/supervised_experiments_submission.py
# chmod a+x /home/miranda9/automl-meta-learning/results_plots/is_rapid_learning_real.py
# condor_submit -i
# condor_submit job.sub
#
####################

Path = /home/miranda9/automl-meta-learning/
Path = /home/miranda9/ML4Coq/

# Executable = /home/miranda9/automl-meta-learning/automl-proj/experiments/meta_learning/supervised_experiments_submission.py
# Executable = /home/miranda9/automl-meta-learning/automl-proj/experiments/meta_learning/meta_learning_experiments_submission.py
# Executable = /home/miranda9/meta-learning-lstm-pytorch/main.py
# Executable = /home/miranda9/automl-meta-learning/automl-proj/meta_learning/datasets/rand_fc_nn_vec_mu_ls_gen.py
# Executable = /home/miranda9/automl-meta-learning/results_plots/is_rapid_learning_real.py

## Output Files
Log          = experiment_output_job.$(CLUSTER).log.out
Output       = experiment_output_job.$(CLUSTER).out.out
Error        = experiment_output_job.$(CLUSTER).err.out

Output = %(FILENAME).o$(CLUSTER)

# Use this to make sure 1 gpu is available. The key words are case insensitive.
# REquest_gpus = 1
# requirements = (CUDADeviceName != "Tesla K40m")
# requirements = (CUDADeviceName == "Quadro RTX 6000")

# requirements = ((CUDADeviceName = "Tesla K40m")) && (TARGET.Arch == "X86_64") && (TARGET.OpSys == "LINUX") && (TARGET.Disk >= RequestDisk) && (TARGET.Memory >= RequestMemory) && (TARGET.Cpus >= RequestCpus) && (TARGET.gpus >= Requestgpus) && ((TARGET.FileSystemDomain == MY.FileSystemDomain) || (TARGET.HasFileTransfer))
# requirements = (CUDADeviceName == "Tesla K40m")
# requirements = (CUDADeviceName == "GeForce GTX TITAN X")

# Note: to use multiple CPUs instead of the default (one CPU),use request_cpus as well
# Request_cpus = 4
Request_cpus = 16

# E-mail option
Notify_user = [e-mail]
Notification = always

Environment = MY_CONDOR_JOB_ID= $(CLUSTER)

# "Queue" means add the setup until this line to the queue (needs to be at the end of script).
Queue

---

或者，如果我可以使用我的可执行脚本作为提交脚本，顶部的参数也​​可以使用

我通常使用与我的可执行脚本相同的提交脚本，例如看我的 qsub 例子：

#!/homes/miranda9/.conda/envs/automl-meta-learning/bin/python
#PBS -V
#PBS -M [e-mail]
#PBS -m abe
#PBS -lselect=1:ncpus=112

import sys
import os

for p in sys.path:
    print(p)

print(os.environ)

解决方法

HTCondor 默认不设置 FILENAME，但您可以自己设置，例如

FILENAME = meta_learning_experiments_submission.py
output = $(FILENAME).o.$(CLUSTER)
error  = $(FILENAME).e.$(CLUSTER)