C3PO:客户端在等待获取资源时超时

问题描述

当前设置:

服务主机(Java)连接到(JDBC)主数据库MysqL RW),并具有工作程序(只读)以确保可靠性。连接管理由C3PO处理

场景:

在翻转的情况下,我需要将一个工作线程提升为主数据库。当前的主数据库变为只读,新的主数据库被提升为可读写。我希望可靠自动将C3P0连接池刷新到新的主数据库。现在哪种方法有效,但在某些情况下会失效。

技术:服务的数据库主机名为cname,将在翻转时进行更新。在DNS更改传播且新数据库为RW之前,所有连接将失败。尝试在连接池上无限设置C3PO重试,直到通过(acquireRetryAttempts)进行连接RW

现有的C3PO配置:

Generic configuration,inherited by primary and worker DB
    <property name="maxIdleTime" value="44000"/>
    <property name="idleConnectionTestPeriod" value="30"/>
    <property name="maxStatements" value="50"/>
    <property name="minPoolSize" value="3"/>
    <property name="maxPoolSize" value="3"/>
    <property name="acquireIncrement" value="2"/>
    <property name="checkoutTimeout" value="15000"/>
    <property name="acquireRetryDelay" value="1000"/>

Primary DB specific config
    <property name="acquireRetryAttempts" value="0"/>
    <property name="minPoolSize" value="3"/>
    <property name="maxPoolSize" value="5"/>
    <property name="testConnectionOnCheckout" value="true"/>
    <property name="connectionTesterClassName" value="${Custom.connectionTesterClassName}"/>

问题:

自定义连接测试器检查连接是否可写(@@ global.read_only),并返回无效或有效的连接。 C3PO遇到了一个问题,即它会按照中间日志中所述的那样翻新(杀死和重新初始化)连接池。最终,它会在4-5分钟内检出所有(max_pool_size,5个)连接,并在连接未检回到池的状态下挂起。如最终日志中所述,这将导致所有后续的数据库连接检出超时并失败。强制重新启动服务,以便重新初始化C3PO池。

理想情况下,如果connectionTesterClassName返回false或在idleConnectionTestPeriod上或在某个随机周期返回,我需要连接池来保持终止状态并重新初始化。由于连接已签出,因此它们不属于空闲连接,也不会刷新。

C3PO documentation所述,尝试使用unreturnedConnectionTimeoutdebugUnreturnedConnectionStackTraces进行调试,但是没有运气。

日志:

中间日志

com.mchange.v2.resourcepool.BasicResourcePool: Resource [com.mchange.v2.c3p0.impl.NewPooledConnection@24e7ebec] Could not be refurbished in preparation for checkout. Will try to find a better resource.
    (C3P0PooledConnectionPoolManager[identityToken->2tyqoiac1gfqk68krloa8|7e2c64]-HelperThread-#6) com.mchange.v2.resourcepool.BasicResourcePool: Preparing to destroy resource: com.mchange.v2.c3p0.impl.NewPooledConnection@24e7ebec
23 Sep 2020 22:06:23,780 [DEBUG]com.mchange.v2.resourcepool.BasicResourcePool: acquire test -- pool size: 0; target_pool_size: 3; desired target? 1
23 Sep 2020 22:06:23,780 [DEBUG]  com.mchange.v2.resourcepool.BasicResourcePool: awaitAvailable(): com.mchange.v2.c3p0.impl.NewPooledConnection@4b52cf0c
23 Sep 2020 22:06:23,780 [DEBUG]  (C3P0PooledConnectionPoolManager[identityToken->2tyqoiac1gfqk68krloa8|7e2c64]-HelperThread-#6) com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool: Preparing to destroy PooledConnection: com.mchange.v2.c3p0.impl.NewPooledConnection@24e7ebec
23 Sep 2020 22:06:23,780 [DEBUG]  (C3P0PooledConnectionPoolManager[identityToken->2tyqoiac1gfqk68krloa8|7e2c64]-HelperThread-#6) com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool: Successfully destroyed PooledConnection: com.mchange.v2.c3p0.impl.NewPooledConnection@24e7ebec
23 Sep 2020 22:06:23,780 [DEBUG]  (C3P0PooledConnectionPoolManager[identityToken->2tyqoiac1gfqk68krloa8|7e2c64]-HelperThread-#6) com.mchange.v2.resourcepool.BasicResourcePool: Successfully destroyed resource: com.mchange.v2.c3p0.impl.NewPooledConnection@24e7ebec
23 Sep 2020 22:06:23,781 [DEBUG]  com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool: Testing PooledConnection [com.mchange.v2.c3p0.impl.NewPooledConnection@7fff041e] on CHECKOUT.
23 Sep 2020 22:06:23,782 [INFO]  ReadWriteConnectionTester: Connection com.mchange.v2.c3p0.impl.NewProxyConnection@21eb35f6 [wrapping: com.MysqL.jdbc.JDBC4Connection@44732406] is read only!
23 Sep 2020 22:06:23,782 [DEBUG]  com.mchange.v2.c3p0.impl.NewProxyConnection: com.mchange.v2.c3p0.impl.NewProxyConnection@21eb35f6 [wrapping: null]: close() called more than once.
23 Sep 2020 22:06:23,782 [DEBUG]  com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool: Test of PooledConnection [com.mchange.v2.c3p0.impl.NewPooledConnection@7fff041e] on CHECKOUT has Failed.
java.sql.sqlException: Connection is invalid

最终日志:

org.hibernate.engine.jdbc.internal.LogicalConnectionImpl: Obtaining JDBC connection
com.mchange.v2.resourcepool.BasicResourcePool: acquire test -- pool is already maxed out. [managed: 5; max: 5]
com.mchange.v2.resourcepool.BasicResourcePool: awaitAvailable(): com.mchange.v2.c3p0.impl.NewPooledConnection@4b52cf0c

com.mchange.v2.sql.sqlUtils: Converting Throwable to sqlException...
com.mchange.v2.resourcepool.TimeoutException: A client timed out while waiting to acquire a resource from com.mchange.v2.resourcepool.BasicResourcePool@43aeb5e0 -- timeout at awaitAvailable()
    at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1467) ~[c3p0-0.9.5.1.jar:0.9.5.1]
    at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:644) ~[c3p0-0.9.5.1.jar:0.9.5.1]
    at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:554) ~[c3p0-0.9.5.1.jar:0.9.5.1]
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutAndMarkConnectionInUse(C3P0PooledConnectionPool.java:758) ~[c3p0-0.9.5.1.jar:0.9.5.1]
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:685) ~[c3p0-0.9.5.1.jar:0.9.5.1]
    at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:140) ~[c3p0-0.9.5.1.jar:0.9.5.1]

org.hibernate.engine.jdbc.spi.sqlExceptionHelper: Could not open connection [n/a]
java.sql.sqlException: An attempt by a client to checkout a Connection has timed out.
    at com.mchange.v2.sql.sqlUtils.tosqlException(sqlUtils.java:118) ~[mchange-commons-java-0.2.10.jar:0.2.10]
    at com.mchange.v2.sql.sqlUtils.tosqlException(sqlUtils.java:77) ~[mchange-commons-java-0.2.10.jar:0.2.10]
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:690) ~[c3p0-0.9.5.1.jar:0.9.5.1]
    at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:140) ~[c3p0-0.9.5.1.jar:0.9.5.1]
    at org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource.getConnection(AbstractRoutingDataSource.java:164) ~[spring-jdbc-4.3.24.RELEASE.jar:4.3.24.RELEASE]
    at org.hibernate.service.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:141) ~[hibernate-core-4.2.2.Final.jar:4.2.2.Final]
    at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:292) ~[hibernate-core-4.2.2.Final.jar:4.2.2.Final]
    at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:214) ~[hibernate-core-4.2.2.Final.jar:4.2.2.Final]
    at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:157) ~[hibernate-core-4.2.2.Final.jar:4.2.2.Final]
    at org.hibernate.internal.SessionImpl.connection(SessionImpl.java:550) ~[hibernate-core-4.2.2.Final.jar:4.2.2.Final]

Caused by: com.mchange.v2.resourcepool.TimeoutException: A client timed out while waiting to acquire a resource from com.mchange.v2.resourcepool.BasicResourcePool@43aeb5e0 -- timeout at awaitAvailable()
    at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1467) ~[c3p0-0.9.5.1.jar:0.9.5.1]
    at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:644) ~[c3p0-0.9.5.1.jar:0.9.5.1]
    at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:554) ~[c3p0-0.9.5.1.jar:0.9.5.1]
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutAndMarkConnectionInUse(C3P0PooledConnectionPool.java:758) ~[c3p0-0.9.5.1.jar:0.9.5.1]
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:685) ~[c3p0-0.9.5.1.jar:0.9.5.1]
    ... 51 more

org.hibernate.engine.jdbc.spi.sqlExceptionHelper: sql Error: 0,sqlState: null
org.hibernate.engine.jdbc.spi.sqlExceptionHelper: An attempt by a client to checkout a Connection has timed out

解决方法

因此,这不是“翻新(杀死并重新初始化)连接池”的正确解释。

这是一个个人Connection,该池正在此处尝试“翻新”。它正在尝试刷新Connection,以供客户端使用。由于您将testConnectionOnCheckout设置为true,并且数据库已“翻转”,因此您的ReadWriteConnectionTester将声明当前池中的每个Connection无效(因为只读),直到该池用尽Connections尝试或用尽为止客户超时导致的时间。

当发现ConnectionTester.CONNECTION_IS_INVALID无效时,您的ReadWriteConnectionTester返回ConnectionTester.DATABASE_IS_INVALIDConnection吗?您应该希望它返回ConnectionTester.DATABASE_IS_INVALID才能重置整个池。

相关问答

Selenium Web驱动程序和Java。元素在(x,y)点处不可单击。其...
Python-如何使用点“。” 访问字典成员?
Java 字符串是不可变的。到底是什么意思?
Java中的“ final”关键字如何工作?(我仍然可以修改对象。...
“loop:”在Java代码中。这是什么,为什么要编译?
java.lang.ClassNotFoundException:sun.jdbc.odbc.JdbcOdbc...