AWS Glue:从 GET 请求传递 ID 值{1,2,3} 并将其传递给第二个 GET 请求,并将 ID 值附加到响应并转换为数据帧

问题描述

First Json Response:
  "data": [
        {
            "id": 123,"first_name": "abc","surname": "efg","email": "abc@gmail.com","full_name": "abc efg"
          },{
            "id": 567,"first_name": "ghj","surname": "klm","email": "ghj@gmail.com","full_name": "ghj klm"
]

skill_response=requests.get("XXXXXX",headers=headersAPI),headers=headersAPI)
skill_json=skill_response.json()

df = spark.read.json(sc.parallelize([skill_json]))

从第一个请求传递 ID 值 (123,567) ->如何将 int 转换为字符串并将其传递给第二个请求(每个) per ={'123','567'}

for per in persons:
    per_response=requests.get("xxxxxxxxxxxxxxxxxxxx/"+per,headers=headersAPI)
    api_results.append(per_response.json())
    api_results.append(per) ## (OR)
    api_results.append({"person_id":per})
    second = spark.read.json(sc.parallelize([api_results]))
    dynamic_frame = DynamicFrame.fromDF(second,glue_ctx=glueContext,name="second")
    DataSink0 = glueContext.write_dynamic_frame.from_options(frame = dynamic_frame,connection_type = "s3",format = "csv",connection_options={"path": "s3://xxxxx/","partitionKeys": []},transformation_ctx= "DataSink0")

我们可以使用for循环传递第一个json值(skill_json["data"][i]["id"]吗?或者使用json.loads()?

解决方法

暂无找到可以解决该程序问题的有效方法,小编努力寻找整理中!

如果你已经找到好的解决方法,欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@)