问题描述
我通过粘合作业动态创建了一个表,并且它成功地工作正常。但是根据新要求,我需要添加一个新列,该列生成唯一值并且应该是 redshift 中的主键。 我已经使用 rownum() 函数实现了相同的功能,并且运行良好。但最新的要求是特定列应该是主键。
When I try to do that,it asks the column to have not null. Do you kNow how to make the column not null dynamically through glue job ? Or any redshift query to make it not null.
I tried all the ways without luck.
w = Window().orderBy(lit('A'))
df = timestampedDf.withColumn("row_num",row_number().over(w))
rowNumDf = DynamicFrame.fromDF(df1,glueContext,"df1")
postStep = "begin; ALTER TABLE TAB_CUSTOMER_DATA ALTER COLUMN row_num INTEGER NOT NULL; ALTER TABLE TAB_CUSTOMER_DATA ADD CONSTRAINT PK_1 PRIMARY KEY (row_num); end;"
## @type: DataSink
## @args: [catalog_connection = "REDSHIFT_CONNECTION",connection_options = {"dbtable": "tab_customer_data","database": "randomdb"},redshift_tmp_dir = TempDir,transformation_ctx = "datasink4"]
## @return: datasink4
## @inputs: [frame = rowNumDf]
datasink4 = glueContext.write_dynamic_frame.from_jdbc_conf(frame = rowNumDf,catalog_connection = "REDSHIFT_CONNECTION",connection_options = {"dbtable": "TAB_CUSTOMER_DATA","database": "randomdb","postactions": postStep},redshift_tmp_dir = args["TempDir"],transformation_ctx = "datasink4")
job.commit()
解决方法
我使用以下链接方法解决了这个问题:
- 添加一个默认且不为空的新列。
- 将旧列值更新为新列。
- 删除旧列。
- 将此新列设为主要列。
https://ubiq.co/database-blog/how-to-remove-not-null-constraint-in-redshift/