在AWS Lambda上运行python脚本时出错

问题描述

我正在尝试在AWS Lambda上运行以下python脚本，该脚本已手动运行，并且可以在输出S3存储桶上获得结果，而没有任何问题。但是现在当我从AWS Lambda调用脚本时出现以下错误时，不确定我是否在脚本上丢失了任何内容？

#!/usr/bin/env python3
import boto3

#Function for executing athena queries
def run_query(Event,context):
    ...
    run_query(query,database,s3_output)
    client = boto3.client('athena')
    response = client.start_query_execution(
        QueryString=query,QueryExecutionContext={
            'Database': 's3_accesslog'
            },ResultConfiguration={
            'OutputLocation': s3_output,}
        )
    
#import datetime 
import datetime
year = datetime.date.today()
year = year.strftime("%Y")
month = datetime.date.today()
month = month.strftime("%m")
day = datetime.date.today()
day = day.strftime("%d")

#select bucket 
s3_input = "s3://smathena/cf-ant-prod/year=%s/month=%s/day=%s" % (year,month,day)
   
#Athena configuration
s3_ouput = 's3://smathena/athenatest/'
database = 's3_accesslog'
table = 'test_output1'

#Athena database and table deFinition
create_database = "CREATE DATABASE IF NOT EXISTS %s;" % (database)
delete_table = "drop table %s.%s;" % (database,table)
create_table = \
  """CREATE EXTERNAL TABLE IF NOT EXISTS %s.%s (
  `Date` DATE,Time STRING,Location STRING,SCBytes BIGINT,RequestIP STRING,Method STRING,Host STRING,Uri STRING,Status INT,Referrer STRING,UserAgent STRING,UriQS STRING,Cookie STRING,ResultType STRING,RequestId STRING,HostHeader STRING,Protocol STRING,CSBytes BIGINT,TiMetaken FLOAT,XForwardFor STRING,SSLProtocol STRING,SSLCipher STRING,ResponseResultType STRING,CSProtocolVersion STRING,FleStatus STRING,FleEncryptedFields INT,CPort INT,TimetoFirstByte FLOAT,XEdgeDetailedResult STRING,ScContent STRING,ScContentLen BIGINT,ScRangeStart BIGINT,ScRangeEnd BIGINT
   )
   ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
   LOCATION '%s'
   TBLPROPERTIES ('skip.header.line.count' = '2');""" % (database,table,s3_input)

#Query deFinitions
query_1 = "SELECT * FROM %s.%s where CAST(status AS VARCHAR) = '404';" % (database,table)

#Execute all queries
queries = [ create_database,delete_table,create_table,query_1 ]
for q in queries:
   print("Executing query: %s" % (q))
   res = run_query(q,s3_ouput)

但是现在当我从AWS Lambda调用脚本时出现以下错误时，不确定我是否在脚本上丢失了任何内容？

{
  "errorMessage": "run_query() takes 2 positional arguments but 3 were given","errorType": "TypeError","stackTrace": [
    "  File \"/var/lang/lib/python3.7/imp.py\",line 234,in load_module\n    return load_source(name,filename,file)\n","  File \"/var/lang/lib/python3.7/imp.py\",line 171,in load_source\n    module = _load(spec)\n","  File \"<frozen importlib._bootstrap>\",line 696,in _load\n",line 677,in _load_unlocked\n","  File \"<frozen importlib._bootstrap_external>\",line 728,in exec_module\n",line 219,in _call_with_frames_removed\n","  File \"/var/task/lambda_function.py\",line 86,in <module>\n    res = run_query(q,s3_ouput)\n"
  ]
}``

解决方法

您的函数lambda_handle不符合python lambda interface：

def handler_name(event,context): 
    ...
    return some_value

函数的

输入应该在event中。该链接的另一个示例：

def my_handler(event,context):
    message = 'Hello {} {}!'.format(event['first_name'],event['last_name'])  
    return { 
        'message' : message
    }  

在您的情况下，我希望query，database和s3_output成为event的一部分。您可能应该return有关正在执行的雅典娜查询的信息。