熊猫json使用特定的json属性归一化键错误

问题描述

我有一个json作为

mytestdata = {
    "success": True,"message": "","data": {
        "totalCount": 95,"goal": [
            {
                "user_id": 123455,"user_email": "[e-mail]","user_first_name": "John","user_last_name": "Smith","people_goals": [
                    {
                        "goal_id": 545555,"goal_name": "test goal name","goal_owner": "123455","goal_narrative": "","goal_type": {
                            "id": 1,"name": "Team"
                        },"goal_create_at": "1595874095","goal_modified_at": "1595874095","goal_created_by": "123455","goal_updated_by": "123455","goal_start_date": "1593561600","goal_target_date": "1601424000","goal_progress": "34","goal_progress_color": "#ff9933","goal_status": "1","goal_permission": "internal,team","goal_category": [],"goal_owner_full_name": "John Smith","goal_team_id": "766754","goal_team_name": "","goal_workstreams": []
                    }
                ]
            }
        ]
    }
}

我正在尝试使用json_normalize在“ people_goals”中显示所有详细信息以及“ user_last_name”，“ user_first_name”，“ user_email”，“ user_id”。 到目前为止，我可以显示带有代码的“ people_goals”，“ user_first_name”和“ user_email”

df2 = pd.json_normalize(data=mytestdata['data'],record_path=['goal','people_goals'],meta=[['goal','user_first_name'],['goal','user_last_name'],'user_email']],errors='ignore')

但是，当我尝试在meta = []中包含['goal'，'user_id']时遇到问题 错误是：

TypeError                                 Traceback (most recent call last)
<ipython-input-192-b7a124a075a0> in <module>
      7 df2 = pd.json_normalize(data=mytestdata['data'],8                         meta=[['goal','user_email'],'user_id']],----> 9                         errors='ignore')
     10 
     11 # df2 = pd.json_normalize(data=mytestdata['data'],'people_goals'])

我看到的'user_id'唯一区别是它不是字符串 我在这里想念东西吗？

解决方法

您的代码可在我的平台上使用。由于两个原因，我已不再使用record_path和meta参数。 a）他们很难解决b）pandas

版本之间存在兼容性问题

因此，我现在多次使用使用json_normalize()的方法来逐步扩展JSON。或使用pd.Series。都包括了这两个例子。

df = pd.json_normalize(data=mytestdata['data']).explode("goal")
df = pd.concat([df,df["goal"].apply(pd.Series)],axis=1).drop(columns="goal").explode("people_goals")
df = pd.concat([df,df["people_goals"].apply(pd.Series)],axis=1).drop(columns="people_goals")
df = pd.concat([df,df["goal_type"].apply(pd.Series)],axis=1).drop(columns="goal_type")
df.T

df2 = pd.json_normalize(pd.json_normalize(
    pd.json_normalize(data=mytestdata['data']).explode("goal").to_dict(orient="records")
).explode("goal.people_goals").to_dict(orient="records"))
df2.T

print(df.T.to_string())

输出

                                        0
totalCount                             95
user_id                            123455
user_email            [e-mail]
user_first_name                      John
user_last_name                      Smith
goal_id                            545555
goal_name                  test goal name
goal_owner                         123455
goal_narrative                           
goal_create_at                 1595874095
goal_modified_at               1595874095
goal_created_by                    123455
goal_updated_by                    123455
goal_start_date                1593561600
goal_target_date               1601424000
goal_progress                          34
goal_progress_color               #ff9933
goal_status                             1
goal_permission             internal,team
goal_category                          []
goal_owner_full_name           John Smith
goal_team_id                       766754
goal_team_name                           
goal_workstreams                       []
id                                      1
name                                 Team