postgresql jdbc 非常简单的查询和 56 行，比使用 psql 的 2x、3x 查询慢

问题描述

我对 Postgresql 驱动程序有一个有趣的问题。在 GreenPlum 5.26.0 中，我有一个非常简单的 SELECT 查询 select sum(col) from table。该表只有 56 行。当我使用 psql 运行此查询时，它会在 50 毫秒内返回。当我通过 Java 代码运行它时，需要 2 到 5 秒。

public static void main(String[] args) {
    try {
        Class.forName("org.postgresql.Driver");
    } catch (ClassNotFoundException e) {
        e.printstacktrace();
    }
    long start = System.currentTimeMillis();
    System.out.println("startTime: " + start);
    try (Connection connection = DriverManager.getConnection("jdbc:postgresql://host:port/db?loggerLevel=TRACE","user","password");
         PreparedStatement preparedStatement = connection.prepareStatement("select sum(num) as num from schema.table;");
         ResultSet resultSet = preparedStatement.executeQuery();) {
        while (resultSet.next()) {
            int res = resultSet.getInt(1);
            System.out.println("res: " + res);
        }
    } catch (sqlException e) {
        e.printstacktrace();
    }
    System.out.println("Time: " + (System.currentTimeMillis() - start));
}

我尝试了 Postgresql JDBC 9.4-1205-jdbc41 和 42.2.16。两个版本都存在同样的问题，但是使用GreenPlum 6.8，我没有问题。

在此异常期间，我在我的代码上运行了 arthas，即 Postgresql 驱动程序方法 org.postgresql.core.VisibleBufferedInputStream#readMore 基本上占用了所有时间。

这让我疯狂了好几天，如果您有任何见解，我将不胜感激。

更新 2021-01-21：

表只有 56 行。我在本地运行 \timing

dcw=# \timing
Timing is on.
dcw=# begin;select sum(num) as num from datagovernance.app_system_datagovernment_norm_1d;commit;
BEGIN
Time: 58.001 ms
  num   
--------
 323866
(1 row)

Time: 27.760 ms
COMMIT
Time: 3554.685 ms

然后我运行 pg_test_fsync

5 seconds per test
O_DIRECT supported on this platform for open_datasync and open_sync.

Compare file sync methods using one 8kB write:
(in wal_sync_method preference order,except fdatasync
is Linux's default)
        open_datasync                   35915.892 ops/sec      28 usecs/op
        fdatasync                       29260.122 ops/sec      34 usecs/op
        fsync                           29845.096 ops/sec      34 usecs/op
        fsync_writethrough                            n/a
        open_sync                       33848.051 ops/sec      30 usecs/op

Compare file sync methods using two 8kB writes:
(in wal_sync_method preference order,except fdatasync
is Linux's default)
        open_datasync                   18039.438 ops/sec      55 usecs/op
        fdatasync                       29315.960 ops/sec      34 usecs/op
        fsync                           27148.788 ops/sec      37 usecs/op
        fsync_writethrough                            n/a
        open_sync                       16646.840 ops/sec      60 usecs/op

Compare open_sync with different write sizes:
(This is designed to compare the cost of writing 16kB
in different write open_sync sizes.)
         1 * 16kB open_sync write       31331.712 ops/sec      32 usecs/op
         2 *  8kB open_sync writes      16620.264 ops/sec      60 usecs/op
         4 *  4kB open_sync writes       8676.007 ops/sec     115 usecs/op
         8 *  2kB open_sync writes       4315.704 ops/sec     232 usecs/op
        16 *  1kB open_sync writes       2175.741 ops/sec     460 usecs/op

Test if fsync on non-write file descriptor is honored:
(If the times are similar,fsync() can sync data written
on a different descriptor.)
        write,fsync,close             27392.879 ops/sec      37 usecs/op
        write,close,fsync             27528.213 ops/sec      36 usecs/op

Non-Sync'ed 8kB writes:
        write                           469187.062 ops/sec       2 usecs/op

现在，问题是为什么提交需要这么长时间？

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）

greenplum jdbc