如何帮助神鹰找到应该在工作中执行的文件？

编程问答 2022-06-07

问题描述

我正在尝试工作，但是神鹰似乎找不到我的文件。

我已确保：

  1. 通过在文件的绝对路径上执行ls和cat命令来保存文件
  2. 通过秃鹰互动会话运行它
  3. 授予它正确的权限，以便它运行。

我已经做到了，但是出现了这个错误：

(automl-Meta-learning) miranda9~/automl-Meta-learning/automl-proj/experiments/Meta_learning $ cat condor_job_log_69.out 
000 (069.000.000) 10/21 11:06:06 Job submitted from host: <130.126.112.32:9618?addrs=130.126.112.32-9618+[--1]-9618&noUDP&sock=3715279_f2e6_4>
...
001 (069.000.000) 10/21 11:06:07 Job executing on host: <172.22.224.111:9618?addrs=172.22.224.111-9618+[--1]-9618&noUDP&sock=807_1d04_3>
...
007 (069.000.000) 10/21 11:06:07 Shadow exception!
        Error from [email protected]: Failed to execute '/home/miranda9/automl-Meta-learning/automl-proj/experiments/Meta_learning/Meta_learning_experiments_submission.py': (errno=2: 'No such file or directory')
        0  -  Run Bytes Sent By Job
        0  -  Run Bytes Received By Job
...
012 (069.000.000) 10/21 11:06:07 Job was held.
        Error from [email protected]: Failed to execute '/home/miranda9/automl-Meta-learning/automl-proj/experiments/Meta_learning/Meta_learning_experiments_submission.py': (errno=2: 'No such file or directory')
        Code 6 Subcode 2
...
(automl)

但文件显然存在：

(automl-Meta-learning) miranda9~/automl-Meta-learning/automl-proj/experiments/Meta_learning $ ls -lah /home/miranda9/automl-Meta-learning/automl-proj/experiments/Meta_learning/Meta_learning_experiments_submission.py
-rwxrwxr-x. 1 miranda9 miranda9 22K Oct 20 14:54 /home/miranda9/automl-Meta-learning/automl-proj/experiments/Meta_learning/Meta_learning_experiments_submission.py

我不明白为什么神鹰找不到它。有任何想法吗？我不是系统管理员，所以我什至不知道如何开始调试它。

btw我的提交脚本：

####################
#
# Experiments script
# Simple HTCondor submit description file
#
# reference: https://gitlab.engr.illinois.edu/Vision/vision-gpu-servers/-/wikis/HTCondor-user-guide#submit-jobs
#
# chmod a+x test_condor.py
# chmod a+x experiments_Meta_model_optimization.py
# chmod a+x Meta_learning_experiments_submission.py
# chmod a+x download_miniImagenet.py
#
# condor_submit -i
# condor_submit job.sub
#
####################

# Executable   = Meta_learning_experiments_submission.py
# Executable = automl-proj/experiments/Meta_learning/Meta_learning_experiments_submission.py
# Executable = ~/automl-Meta-learning/automl-proj/experiments/Meta_learning/Meta_learning_experiments_submission.py
Executable = /home/miranda9/automl-Meta-learning/automl-proj/experiments/Meta_learning/Meta_learning_experiments_submission.py

## Output Files
Log          = condor_job.$(CLUSTER).log.out
Output       = condor_job.$(CLUSTER).stdout.out
Error        = condor_job.$(CLUSTER).err.out

# Use this to make sure 1 gpu is available. The key words are case insensitive.
REquest_gpus = 1
# requirements = ((CUDADeviceName = "Tesla K40m")) && (TARGET.Arch == "X86_64") && (TARGET.OpSys == "LINUX") && (TARGET.disk >= Requestdisk) && (TARGET.Memory >= RequestMemory) && (TARGET.cpus >= Requestcpus) && (TARGET.gpus >= Requestgpus) && ((TARGET.FileSystemDomain == MY.FileSystemDomain) || (TARGET.HasFileTransfer))
# requirements = (CUDADeviceName == "Tesla K40m")
# requirements = (CUDADeviceName == "Quadro RTX 6000")
requirements = (CUDADeviceName != "Tesla K40m")

# Note: to use multiple cpus instead of the default (one cpu),use request_cpus as well
Request_cpus = 8

# E-mail option
Notify_user = [email protected]
Notification = always

Environment = MY_CONDOR_JOB_ID= $(CLUSTER)

# "Queue" means add the setup until this line to the queue (needs to be at the end of script).
Queue

解决方法

看起来您的可执行文件是python脚本。当脚本本身存在时，Linux将报告“没有这样的文件或目录”，但是解释器列在“＃！”上。在系统上不存在。这可能是这里发生的事情吗？该脚本的第一行是什么样的？

,

问题是在我的python提交脚本的顶部，我有与condor不相关的其他集群的参数，因此存在指向python可执行文件的错误路径。我通过删除该行并将其添加到我的python提交脚本中来对其进行修复：

#!/home/miranda9/miniconda3/envs/automl-meta-learning/bin/python

实际上是为当前环境找到python路径：

which python
condor