将联接表继承与SQLAlchemy一起使用时，如何在PostgreSQL上运行“ upsert”？

问题描述

标题说明了一切，这是一个示例，再现了我得到的错误（下面的TB）。由于使用on_conflict_do_update构造，因此需要PostgreSQL：

from sqlalchemy import (Column,DateTime,ForeignKeyConstraint,Integer,String,create_engine,func)
from sqlalchemy.dialects.postgresql import insert
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

Base = declarative_base()


class Parent(Base):
    __tablename__ = "parent_table"
    __mapper_args__ = {
        "polymorphic_identity": "parent","polymorphic_on": "polymorphic_type",}
    parent_id = Column(Integer,primary_key=True)
    name = Column(String(50))
    polymorphic_type = Column(String,nullable=False)
    updated = Column(DateTime,server_default=func.now())


class Child(Parent):
    __tablename__ = "child_table"
    __mapper_args__ = {"polymorphic_identity": "child"}
    __table_args__ = (
        ForeignKeyConstraint(  # type: ignore
            ["parent_id"],["parent_table.parent_id"],onupdate="CASCADE",ondelete="CASCADE",),)
    parent_id = Column(Integer,primary_key=True)
    child_column = Column(String(50))


engine = create_engine("postgresql://",echo=True)
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)

sess = Session()
c1 = Child(name="foo",child_column="bar")
sess.add(c1)
sess.commit()
sess.close()


sess = Session()
insert_stmt = insert(Child)
insert_stmt = insert_stmt.values([
    {"name": "newname","child_column": "newchildcol"},{"name": "newname2","child_column": "newchildcol2"},{"name": "foo","child_column": "updatedcol"},])
insert_stmt = insert_stmt.on_conflict_do_update(
    index_elements=[
        Parent.parent_id,],set_={
        "name": insert_stmt.excluded.name,"child_column": insert_stmt.excluded.child_column,"updated": func.now(),},)
session.execute(insert_stmt)

跟踪：

Traceback (most recent call last):
  File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/util/_collections.py",line 210,in __getattr__
    return self._data[key]
KeyError: 'name'

During handling of the above exception,another exception occurred:

Traceback (most recent call last):
  File "upsert_with_jti.py",line 60,in <module>
    "name": insert_stmt.excluded.name,File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/util/_collections.py",line 212,in __getattr__
    raise AttributeError(key)
AttributeError: name

经过大量挖掘，我通过执行以下操作，将调试器注入到SA获得“已排除”列的位置：

...
def bp(*a,**kw):
    from pdb import set_trace; set_trace()  # XXX debugger
...
insert_stmt = insert_stmt.on_conflict_do_update(
...
    set_={
        "name": bp(insert_stmt.excluded),...
    },)

在调试器中，我得到以下信息：

(Pdb) pp dir(a[0])
['__class__','__delattr__','__dir__','__doc__','__eq__','__format__','__ge__','__get__','__getattribute__','__gt__','__hash__','__init__','__le__','__lt__','__ne__','__new__','__reduce__','__reduce_ex__','__repr__','__self__','__self_class__','__setattr__','__sizeof__','__str__','__subclasshook__','__thisclass__','child_table_child_column','child_table_parent_id','parent_table_name','parent_table_parent_id','parent_table_polymorphic_type','parent_table_updated']

很明显，此操作必须为这些列加上别名。我认为这是问题所在。但是，在“排除的”列中将name替换为parent_table_name之后，仍然出现错误：

Traceback (most recent call last):
  File "upsert_with_jti.py",line 66,in <module>
    sess.execute(insert_stmt)
  File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/orm/session.py",line 1292,in execute
    clause,params or {}
  File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/engine/base.py",line 1014,in execute
    return meth(self,multiparams,params)
  File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/sql/elements.py",line 298,in _execute_on_connection
    return connection._execute_clauseelement(self,params)
  File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/engine/base.py",line 1124,in _execute_clauseelement
    else None,File "<string>",line 1,in <lambda>
  File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/sql/elements.py",line 476,in compile
    return self._compiler(dialect,bind=bind,**kw)
  File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/sql/elements.py",line 482,in _compiler
    return dialect.statement_compiler(dialect,self,**kw)
  File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/sql/compiler.py",line 590,in __init__
    Compiled.__init__(self,dialect,statement,**kwargs)
  File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/sql/compiler.py",line 319,in __init__
    self.string = self.process(self.statement,**compile_kwargs)
  File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/sql/compiler.py",line 350,in process
    return obj._compiler_dispatch(self,**kwargs)
  File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/sql/visitors.py",line 95,in _compiler_dispatch
    return meth(self,line 2428,in visit_insert
    self,insert_stmt,crud.ISINSERT,**kw
  File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/sql/crud.py",line 64,in _setup_crud_params
    return _get_crud_params(compiler,stmt,**kw)
  File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/sql/crud.py",line 167,in _get_crud_params
    kw,File "/home/users/malbert/tmp/env/lib/python3.5/site-packages/sqlalchemy/sql/crud.py",line 367,in _scan_cols
    and c is not stmt.table._autoincrement_column
AttributeError: 'Join' object has no attribute '_autoincrement_column'

这使我相信SA不支持在联接表继承上运行“ upsert”？有什么更好的方法？

请注意，我目前正在处理此性能问题。我有一个相当大的数据集，该数据集受到当前由“大容量”插入导致的许多“选择”的困扰，这些插入经常发生。我想利用SA的“批量”操作，或者利用具有许多值的insert语句（可能足够有效）（如上例所示）。我拥有一个天真的版本的代码，但是随着插入的行数最近大量增加，它变得越来越慢。

我也想从“ upsert”特性中受益，但是目前还不简单。父表包含一个代理主键，每个子表具有另一个自然键。不幸的是，代理键在输入数据上不可用。例如，在上面的示例中，代理键将是parent_id，自然子键将是child_column。具体是这样的：

Device (parent-table)
---------------------
id: uuid (PK/surrogate)
scope: text (part of natural key with child)
hostname: text (part of natural key with child)
ip: text (part of natural key with child)

Component (child-table)
-----------------------
id: uuid (fk with parent)
component-id: text (part of natural key)

因此，需要在“ component-id”上结合作用域/主机名/ ip三元组检测到重复，因为相同的component-id可以在不同的设备上使用。

还有其他具有其他NK的子表，但是总体问题是相同的。

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）

python sqlalchemy

将联接表继承与SQLAlchemy一起使用时，如何在PostgreSQL上运行“ upsert”？

问题描述

解决方法

相关问答