pyspark.sql.utils.IllegalArgumentException: '需求失败：未找到合适的资源来下载请求

问题描述

我正在尝试运行下面的示例代码：

import sparknlp
sparknlp.start()

from sparknlp.pretrained import PretrainedPipeline

explain_document_pipeline = PretrainedPipeline("explain_document_ml")
annotations = explain_document_pipeline.annotate("We are very happy about SparkNLP")
print(annotations)

我在 Anaconda env 中使用 Pycharm，最初我用 pip spark-nlp==2.4.4 下载了 spark-nlp，但我在网上看到有人说我应该使用：

pyspark --packages com.johnsNowlabs.nlp:spark-nlp_2.11:2.4.4

因为使用 pip 安装，我可能缺少一些依赖项，所以最好使用 pyspark --packages，但这给了我错误：

:: problems summary ::
:::: WARNINGS
                [NOT FOUND  ] com.typesafe#config;1.3.0!config.jar(bundle) (0ms)

        ==== local-m2-cache: tried

          file:/C:/Users/xxxxxxxx/.m2/repository/com/typesafe/config/1.3.0/config-1.3.0.jar

                [NOT FOUND  ] com.fasterxml.jackson.core#jackson-annotations;2.6.0!jackson-annotations.jar(bundle) (0ms)

        ==== local-m2-cache: tried

          file:/C:/Users/xxxxxxx/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.jar

                ::::::::::::::::::::::::::::::::::::::::::::::

                ::              Failed DOWNLOADS            ::

                :: ^ see resolution messages for details  ^ ::

                ::::::::::::::::::::::::::::::::::::::::::::::

                :: com.typesafe#config;1.3.0!config.jar(bundle)

                :: com.fasterxml.jackson.core#jackson-annotations;2.6.0!jackson-annotations.jar(bundle)

                ::::::::::::::::::::::::::::::::::::::::::::::


:::: ERRORS
        unkNown resolver null

        unkNown resolver null

        unkNown resolver null

        unkNown resolver null

        unkNown resolver default

        unkNown resolver null


:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
Exception in thread "main" java.lang.RuntimeException: [download Failed: com.typesafe#config;1.3.0!config.jar(bundle),download Failed: com.fasterxml.jackson.core#jackson-annotations;2.6.0!jackson-annotations.jar(
bundle)]

然后我下载了这两个丢失的jar包并复制到相应的文件夹，然后运行命令，现在一切看起来都很好：

21/02/05 15:41:13 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR,use setLogLevel(newLevel).
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /__ / .__/\_,_/_/ /_/\_\   version 2.4.4
      /_/

Using Python version 3.7.1 (default,Oct 28 2018 08:39:03)
SparkSession available as 'spark'.
>>> 

然后我尝试重新运行顶部的示例 python 脚本，它给了我错误，这是日志：

Ivy Default Cache set to: C:\Users\xxxx\.ivy2\cache
The jars for the packages stored in: C:\Users\xxxx\.ivy2\jars
:: loading settings :: url = jar:file:/C:/Users/xxxx/AppData/Local/Continuum/anaconda3/envs/workEnv-python3.7/lib/site-packages/pyspark/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
com.johnsNowlabs.nlp#spark-nlp_2.11 added as a dependency
............

我是新手，我已经乱搞了两天了，请问有人可以帮助我吗？？？

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）