安装Ambari，无法从亚马逊S3下载hortonworks HDP

问题描述

我打算逐步Ambari User Guides在我的wsl（ubuntu 20.04）上安装apache ambari。在将项目安装和打包到deb文件时，请使用命令：

mvn -B clean install jdeb:jdeb -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" 。

遇到此错误

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (default) on project ambari-metrics-timelineservice: An Ant BuildException has occured: Can't get https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hbase/hbase-2.0.2.3.1.4.0-315-bin.tar.gz to /root/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz
[ERROR] around Ant part ...<get usetimestamp="true" src="https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hbase/hbase-2.0.2.3.1.4.0-315-bin.tar.gz" dest="/root/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz"/>... @ 5:273 in /root/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-timelineservice/target/antrun/build-Download HBase.xml

apache-ambari-2.7.5-src / ambari-metrics / pom.xml 定义了nortonworks HDP来源：

<hbase.tar>http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/hbase/hbase-2.0.0.3.0.0.0-1634-bin.tar.gz</hbase.tar>
<hbase.folder>hbase-2.0.0.3.0.0.0-1634</hbase.folder>
<hadoop.tar>http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/hadoop/hadoop-3.1.0.3.0.0.0-1634.tar.gz</hadoop.tar>
<hadoop.folder>hadoop-3.1.0.3.0.0.0-1634</hadoop.folder>
<phoenix.tar>http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/phoenix/phoenix-5.0.0.3.0.0.0-1634.tar.gz</phoenix.tar>
<phoenix.folder>phoenix-5.0.0.3.0.0.0-1634</phoenix.folder>

我尝试下载官方的Hbase-2.3.2，Hadoop-3.3.0，phoenix-5.0.0-HBase-2.0代替hotonworks HDP，但失败并出现了另一个错误。

我尝试直接使用wget下载hortonworks HDP并得到：

Resolving dev.hortonworks.com.s3.amazonaws.com (dev.hortonworks.com.s3.amazonaws.com)... 52.217.40.204
Connecting to dev.hortonworks.com.s3.amazonaws.com 
(dev.hortonworks.com.s3.amazonaws.com)|52.217.40.204|:80... connected.
HTTP request sent,awaiting response... 403 Forbidden

我在哪里/如何下载这些hortonworks HDP文件并继续安装ambari？

解决方法

将下行链路替换为

 <hbase.tar>https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.1-1/tars/hbase/hbase-2.0.2.3.1.4.1-1-bin.tar.gz</hbase.tar>
    <hbase.tar>https://private-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.1-1/tars/hbase/hbase-2.0.2.3.1.4.1-1-bin.tar.gz</hbase.tar>
    <hbase.folder>hbase-2.0.2.3.1.4.1-1</hbase.folder>
    <hadoop.tar>https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.1-1/tars/hadoop/hadoop-3.1.1.3.1.4.1-1.tar.gz</hadoop.tar>
    <hadoop.tar>https://private-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.1-1/tars/hadoop/hadoop-3.1.1.3.1.4.1-1.tar.gz</hadoop.tar>
    <hadoop.folder>hadoop-3.1.1.3.1.4.1-1</hadoop.folder>
    <grafana.folder>grafana-6.4.2</grafana.folder>
    <grafana.tar>https://dl.grafana.com/oss/release/grafana-6.4.2.linux-amd64.tar.gz</grafana.tar>
    <phoenix.tar>https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.1-1/tars/phoenix/phoenix-5.0.0.3.1.4.1-1.tar.gz</phoenix.tar>
    <phoenix.tar>https://private-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.1-1/tars/phoenix/phoenix-5.0.0.3.1.4.1-1.tar.gz</phoenix.tar>
    <phoenix.folder>phoenix-5.0.0.3.1.4.1-1</phoenix.folder>

编辑文件ambari-metrics/pom.xml并替换下载链接，例如我更改了

https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hbase/hbase-2.0.2.3.1.4.0-315-bin.tar.gz

为了

https://downloads.apache.org/hbase/2.3.5/hbase-2.3.5-src.tar.gz

Apache Ambari 团队的这个解决方案解决了这个问题和其他类似的 HDP url 问题：https://github.com/apache/ambari/pull/3283

需要进行相应的代码更改：https://github.com/apache/ambari/pull/3283/commits/3dca705f831383274a78a8c981ac2b12e2ecce85

amazon-s3 ambari