问题描述
from pip._internal.req import parse_requirements
def load_requirements(fname):
"""Turn requirements.txt into a list"""
reqs = parse_requirements(fname,session="test")
return [str(ir.requirement) for ir in reqs]
setup(
name="Projectname",[...]
python_requires='>=3.6',extras_require={
'dev': load_requirements('./requirements/dev.txt')
},install_requires=load_requirements('./requirements/prod.txt')
)
我的./requirements/prod.txt
看起来像这样:
-r common.txt
和我的./requirements/dev.txt
类似,但带有一些特定于开发的软件包。我的./requirements/common.txt
包含一行代码,可从github链接pip安装软件包,例如:
-e git://github.com/BioGeek/tta_wrapper.git@master#egg=tta_wrapper
但是,由于我添加了该行,因此命令python setup.py build
失败,并显示以下信息:
error in Projectname setup command: 'extras_require' must be a dictionary whose values are strings or lists of strings containing valid project/version requirement specifiers.
相关软件包的版本:
pip 20.2.2
setuptools 50.0.0
按照the anwser的Martijn Pieters修改setup.py
后,我可以确认load_requirements
现在将需求文件转换为具有name @ url直接引用语法的列表需要的地方。
>>> load_requirements('./requirements/prod.txt')
['absl-py==0.8.1','gitpython==3.1.0','numpy==1.18.4','pip==20.2.2','protobuf==3.12.0','setuptools==41.0.0','scikit_learn==0.22','tensorflow_hub==0.8.0','importlib-Metadata==1.6.1','keras-tuner==1.0.1','apache-beam==2.23.0','ml-Metadata==0.23.0','pyarrow==0.17.0','tensorflow==2.3.0','tensorflow-data-validation==0.23.0','tensorflow-Metadata==0.23.0','tensorflow-model-analysis==0.23.0','tensorflow-transform==0.23.0','tta_wrapper @ git://github.com/BioGeek/tta_wrapper.git@master']
但是,现在运行python setup.py build
时出现以下错误:
$ python setup.py build
/home/biogeek/code/programname/env/lib/python3.6/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
running build
Traceback (most recent call last):
File "setup.py",line 91,in <module>
install_requires=load_requirements('./requirements/prod.txt')
File "/home/biogeek/code/programname/env/lib/python3.6/site-packages/setuptools/__init__.py",line 153,in setup
return distutils.core.setup(**attrs)
File "/home/biogeek/code/programname/env/lib/python3.6/site-packages/setuptools/_distutils/core.py",line 148,in setup
dist.run_commands()
File "/home/biogeek/code/programname/env/lib/python3.6/site-packages/setuptools/_distutils/dist.py",line 967,in run_commands
self.run_command(cmd)
File "/home/biogeek/code/programname/env/lib/python3.6/site-packages/setuptools/_distutils/dist.py",line 984,in run_command
cmd_obj = self.get_command_obj(command)
File "/home/biogeek/code/programname/env/lib/python3.6/site-packages/setuptools/_distutils/dist.py",line 859,in get_command_obj
cmd_obj = self.command_obj[command] = klass(self)
File "/usr/lib/python3.6/distutils/cmd.py",line 57,in __init__
raise TypeError("dist must be a distribution instance")
TypeError: dist must be a distribution instance
编辑2
我终于使安装成功。我尝试了几件事,所以不能完全确定到底是什么解决了这个问题,但是我:
def _format_requirement(req):
if str(req.requirement) == 'git://github.com/BioGeek/tta_wrapper.git@master#egg=tta_wrapper':
return 'tta_wrapper @ https://github.com/BioGeek/tta_wrapper/archive/v0.0.1.zip'
return str(req.requirement)
解决方法
您只能使用PEP 508 - Dependency specification for Python Software Packages个要求。根据该标准,git://github.com/BioGeek/tta_wrapper.git@master#egg=tta_wrapper
无效的语法。
setuptools
确实接受了name@ url
direct reference syntax:
tta_wrapper @ git://github.com/BioGeek/tta_wrapper.git
但是您不能将其放置在requirements.txt文件中,而不是 并使用-e
开关。后者只能采用VCS URL或本地文件路径,不能为需求说明;参见Requirements File Format section。
因此,您可以在此处在格式之间进行翻译。我会检查is_editable
产生的ParsedRequirement
对象上的parse_requirements()
标志,并相应地更改行为。您必须将需求字符串解析为URL,取出#egg=
片段并将其放在最前面:
from urllib.parse import urlparse
def _format_requirement(req):
if req.is_editable:
# parse out egg=... fragment from VCS URL
parsed = urlparse(req.requirement)
egg_name = parsed.fragment.partition("egg=")[-1]
without_fragment = parsed._replace(fragment="").geturl()
return f"{egg_name} @ {without_fragment}"
return req.requirement
def load_requirements(fname):
"""Turn requirements.txt into a list"""
reqs = parse_requirements(fname,session="test")
return [_format_requirement(ir) for ir in reqs]
然后,以上内容将-e git:...#egg=tta_wrapper
变成tta_wrapper @ git:...
:
>>> load_requirements('./requirements/dev.txt')
['tta_wrapper @ git://github.com/BioGeek/tta_wrapper.git@master','black==20.08b1']
,
在我的情况下,我的要求中没有任何github链接,但该行
-r common.txt
./requirements/prod.txt
中的导致了相同的错误。
我添加了愚蠢的条件,现在它对我有用:
def load_requirements(filename) -> list:
requirements = []
try:
with open(filename) as req:
requirements = [line for line in req.readlines() if line.strip() != "-r common.txt"]
except Exception as e:
print(e)
return requirements