PyHive 连接到 Dataproc 上的 Hive 出现 VPC_SERVICE_CONTROLS PERMISSION_DENIED NO_MATCHING_ACCESS_LEVEL 错误

问题描述

我正在使用 pyhive 连接到 GCP Dataproc 上的 Hive。查询间歇性地成功和失败。这是问题之一。它发生在一个

DROP TABLE IF EXISTS db_name.tb_name

客户端得到

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/thrift/transport/TSocket.py",line 154,in write
    plus = self.handle.send(buff)
brokenPipeError: [Errno 32] broken pipe

During handling of the above exception,another exception occurred:

Traceback (most recent call last):
  ...
  ...
    self._cursor.execute(sql,*args,**kwargs)  # type: ignore
  File "/usr/local/lib/python3.7/site-packages/pyhive/hive.py",line 356,in execute
    self._reset_state()
  File "/usr/local/lib/python3.7/site-packages/pyhive/hive.py",line 266,in _reset_state
    response = self._connection.client.CloSEOperation(request)
  File "/usr/local/lib/python3.7/site-packages/TCLIService/TCLIService.py",line 651,in CloSEOperation
    self.send_CloSEOperation(req)
  File "/usr/local/lib/python3.7/site-packages/TCLIService/TCLIService.py",line 660,in send_CloSEOperation
    self._oprot.trans.flush()
  File "/usr/local/lib/python3.7/site-packages/thrift_sasl/__init__.py",line 143,in flush
    self._flushPlain(buffer)
  File "/usr/local/lib/python3.7/site-packages/thrift_sasl/__init__.py",line 166,in _flushPlain
    self._trans.write(struct.pack(">I",len(buffer)) + buffer)
  File "/usr/local/lib/python3.7/site-packages/thrift/transport/TSocket.py",line 161,in write
    raise TTransportException(message="unexpected exception",inner=e)
thrift.transport.TTransport.TTransportException: unexpected exception

服务器端日志包含很多

{
  "protoPayload": {
    "@type": "type.googleapis.com/google.cloud.audit.AuditLog","status": {
      "code": 7,"message": "PERMISSION_DENIED","details": [
        {
          "@type": "type.googleapis.com/google.rpc.PreconditionFailure","violations": [
            {
              "type": "VPC_SERVICE_CONTROLS","description": "enVxsuY7TYqavK8rl_fwvgr3yn4-9bneHjDTtQFvton7C03R3sHaXA"
            }
          ]
        }
      ]
    },"authenticationInfo": {},"requestMetadata": {
      "callerIp": "73.252.204.93","requestAttributes": {},"destinationAttributes": {}
    },"serviceName": "dataproc.googleapis.com","methodName": "google.cloud.dataproc.v1beta2.ClusterController.ListClusters","resourceName": "projects/xxx","Metadata": {
      "securityPolicyInfo": {
        "servicePerimeterName": "accesspolicies/xxx/servicePerimeters/Production_Security_Zone","organizationId": "xxx"
      },"violationReason": "NO_MATCHING_ACCESS_LEVEL","ingressViolations": [
        {
          "servicePerimeter": "accesspolicies/xxx/servicePerimeters/Production_Security_Zone","targetResource": "xxx"
        }
      ],"@type": "type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata","resourceNames": [
        "xxx"
      ],"vpcServiceControlsUniqueId": "xxx","accessLevels": [
        "no_matching_deFinitions"
      ]
    }
  },"insertId": "xxx","resource": {
    "type": "audited_resource","labels": {
      "method": "google.cloud.dataproc.v1beta2.ClusterController.ListClusters","project_id": "xxx","service": "dataproc.googleapis.com"
    }
  },"timestamp": "2021-03-13T08:09:35.213190134Z","severity": "ERROR","logName": "projects/xxx/logs/cloudaudit.googleapis.com%2Fpolicy","receiveTimestamp": "2021-03-13T08:09:35.968888182Z"
}

我通过 VPN 后的 Mac 笔记本电脑连接。

sasl3==0.2.11

节俭==0.13.0

thrift-sasl==0.4.2

pyhive==0.6.3

任何指示如何解决这个问题？好像是服务器端权限设置？谢谢！！

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）