AADToken：到https://login.microsoftonline.com/<tenantID>/oauth2/token的HTTP连接无法从AzureAD获取令牌

编程问答 2022-08-14

问题描述

我想通过文件系统中的安装点从Azure Databricks群集-SCALA版本访问Azure Data Lake Storage Gen2。

我尝试了以下代码，其中将azure服务主体凭据指定为azure订阅的入口点（角色->存储blob数据所有者位于Data Lake容器上）。

val fileSystemName = "XXXXXXXXXX"
val storageAccountName = "XXXXXXXXXXXX"
val appID = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
val appSecret = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
val tenantID = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"

val configs = Map(
  "fs.azure.account.auth.type" -> "OAuth","fs.azure.account.oauth.provider.type" -> "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider","fs.azure.account.oauth2.client.id" -> "<appID>","fs.azure.account.oauth2.client.secret" -> "<appSecret>","fs.azure.account.oauth2.client.endpoint" -> "https://login.microsoftonline.com/<tenantID>/oauth2/token")

// Optionally,you can add <directory-name> to the source URI of your mount point.
dbutils.fs.mount(
  source = "abfss://<fileSystemName>@<storageAccountName>.dfs.core.windows.net/",mountPoint = "/mnt/raw-container",extraConfigs = configs)

但是收到此错误消息：

shaded.databricks.v20180920_b33d810.org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator$HttpException: HTTP Error 400; url='https://login.microsoftonline.com/<tenantID>/oauth2/token' AADToken: HTTP connection to https://login.microsoftonline.com/<tenantID>/oauth2/token failed for getting token from AzureAD.; requestId=''; contentType='text/html; charset=us-ascii'; response '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">


<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid URL</h2>
<hr><p>HTTP Error 400. The request URL is invalid.</p>
</BODY></HTML>
'
    at shaded.databricks.v20180920_b33d810.org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenSingleCall(AzureADAuthenticator.java:372)
    at shaded.databricks.v20180920_b33d810.org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenCall(AzureADAuthenticator.java:270)
    at shaded.databricks.v20180920_b33d810.org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenUsingClientCreds(AzureADAuthenticator.java:95)
    at com.databricks.backend.daemon.dbutils.DBUtilsCore.verifyAzureOAuth(DBUtilsCore.scala:477)
    at com.databricks.backend.daemon.dbutils.DBUtilsCore.verifyAzureFileSystem(DBUtilsCore.scala:488)
    at com.databricks.backend.daemon.dbutils.DBUtilsCore.mount(DBUtilsCore.scala:446)
    at com.databricks.dbutils_v1.impl.DbfsUtilsImpl.mount(DbfsUtilsImpl.scala:85)
    at linec1d28633ab4c4f2d9530ae7396d8282327.$read$$iw$$iw$$iw$$iw$$iw$$iw.<init>(command-3300122855614219:18)
    at linec1d28633ab4c4f2d9530ae7396d8282327.$read$$iw$$iw$$iw$$iw$$iw.<init>(command-3300122855614219:67)
    at linec1d28633ab4c4f2d9530ae7396d8282327.$read$$iw$$iw$$iw$$iw.<init>(command-3300122855614219:69)
    at linec1d28633ab4c4f2d9530ae7396d8282327.$read$$iw$$iw$$iw.<init>(command-3300122855614219:71)
    at linec1d28633ab4c4f2d9530ae7396d8282327.$read$$iw$$iw.<init>(command-3300122855614219:73)
    at linec1d28633ab4c4f2d9530ae7396d8282327.$read$$iw.<init>(command-3300122855614219:75)
    at linec1d28633ab4c4f2d9530ae7396d8282327.$read.<init>(command-3300122855614219:77)
    at linec1d28633ab4c4f2d9530ae7396d8282327.$read$.<init>(command-3300122855614219:81)
    at linec1d28633ab4c4f2d9530ae7396d8282327.$read$.<clinit>(command-3300122855614219)
    at linec1d28633ab4c4f2d9530ae7396d8282327.$eval$.$print$lzycompute(<notebook>:7)
    at linec1d28633ab4c4f2d9530ae7396d8282327.$eval$.$print(<notebook>:6)
    at linec1d28633ab4c4f2d9530ae7396d8282327.$eval.$print(<notebook>)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:745)
    at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1021)
    at scala.tools.nsc.interpreter.IMain.$anonfun$interpret$1(IMain.scala:574)
    at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:41)
    at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:37)
    at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:41)
    at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:573)
    at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:600)
    at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:570)
    at com.databricks.backend.daemon.driver.DriverILoop.execute(DriverILoop.scala:215)
    at com.databricks.backend.daemon.driver.ScalaDriverLocal.$anonfun$repl$1(ScalaDriverLocal.scala:202)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    at com.databricks.backend.daemon.driver.DriverLocal$TrapExitInternal$.trapExit(DriverLocal.scala:714)
    at com.databricks.backend.daemon.driver.DriverLocal$TrapExit$.apply(DriverLocal.scala:667)
    at com.databricks.backend.daemon.driver.ScalaDriverLocal.repl(ScalaDriverLocal.scala:202)
    at com.databricks.backend.daemon.driver.DriverLocal.$anonfun$execute$10(DriverLocal.scala:396)
    at com.databricks.logging.UsageLogging.$anonfun$withAttributionContext$1(UsageLogging.scala:238)
    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
    at com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:233)
    at com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:230)
    at com.databricks.backend.daemon.driver.DriverLocal.withAttributionContext(DriverLocal.scala:49)
    at com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:275)
    at com.databricks.logging.UsageLogging.withAttributionTags$(UsageLogging.scala:268)
    at com.databricks.backend.daemon.driver.DriverLocal.withAttributionTags(DriverLocal.scala:49)
    at com.databricks.backend.daemon.driver.DriverLocal.execute(DriverLocal.scala:373)
    at com.databricks.backend.daemon.driver.DriverWrapper.$anonfun$tryExecutingCommand$1(DriverWrapper.scala:653)
    at scala.util.Try$.apply(Try.scala:213)
    at com.databricks.backend.daemon.driver.DriverWrapper.tryExecutingCommand(DriverWrapper.scala:645)
    at com.databricks.backend.daemon.driver.DriverWrapper.getCommandOutputAndError(DriverWrapper.scala:486)
    at com.databricks.backend.daemon.driver.DriverWrapper.executeCommand(DriverWrapper.scala:598)
    at com.databricks.backend.daemon.driver.DriverWrapper.runInnerLoop(DriverWrapper.scala:391)
    at com.databricks.backend.daemon.driver.DriverWrapper.runInner(DriverWrapper.scala:337)
    at com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:219)
    at java.lang.Thread.run(Thread.java:748)

似乎存在端点身份验证问题。

解决方法

我以不同的方式尝试了这个问题，并且有效。

我预配置了Azure Databricks PREMIUM版本

然后，在Python群集配置页面上，我在“高级选项”部分的“ Azure Data Lake存储凭据传递”下选中了“为用户级数据访问启用凭据传递”选项。

所以我只需要在单元格中传递点标记脚本就可以了：

storageaccountName = "XXXXXXXXXXXX"
storagecontainerName = "XXXXXXXXXXXX"

configs = { 
    "fs.azure.account.auth.type": "CustomAccessToken","fs.azure.account.custom.token.provider.class": spark.conf.get("spark.databricks.passthrough.adls.gen2.tokenProviderClassName")
}

# Optionally,you can add <directory-name> to the source URI of your mount point. 

dbutils.fs.mount( 
    source = "abfss://"+storagecontainerName+"@"+storageaccountName+".dfs.core.windows.net/",mount_point = "/mnt/raw-container",extra_configs = configs)
azureazure-blob-storageazure-data-lake-gen2azure-databricks

相关问答

matplotlib报错：AttributeError: module 'backend_interagg' has no attribute 'FigureCanvas'. Did you mean: 'FigureCanvasAgg'?
使用本地python环境可以成功执行 import pandas as pd impor...
gitlab登录失败，报错：This challenge page was accidentally cached by an intermediary and is no longer available.
设置时间 控制面板
后端开发常见错误
错误1：Request method ‘DELETE‘ not supported 错误还原：...
docker常见错误
错误1：启动docker镜像时报错：Error response from daemon:...
idea常见错误
错误1：private field ‘xxx‘ is never assigned 按Alt...
pip安装依赖失败
报错如下，通过源不能下载，最后警告pip需升级版本 Requirem...