卡夫卡消费者延迟

问题描述

在接收主题消息时，我面临着相当大的延迟，并且想到编写使用Spring Kafka和apache kafka的测试程序。我正在使用测试程序将数据发送到主题，并使用以下使用者记录使用者中接收消息的延迟。记录时间戳和使用者实际收到的时间之间的差异。

春季消费者

@Component
public class SpringKafkaConsumer {
    @KafkaListener(topics = "websocket",containerFactory = "dynamicKafkaListenerContainerFactory")
    public void receiveDeliveryTopic(ConsumerRecord<String,String> record,@Headers   MessageHeaders headers) {
    Log.info("0","Kafka latency in spring receiver: Topic: {} latency: {}",record.topic(),System.currentTimeMillis() - record.timestamp());
}

Apache Consumer

@Component
public class ApacheKafkaConsumer {
    @PostConstruct
    public void start() {
        Properties props = new Properties();
        props.put("bootstrap.servers","localhost:9092");
        props.put("group.id","test");
        props.put("enable.auto.commit","true");
        props.put("auto.commit.interval.ms","1000");
        props.put("key.deserializer","org.apache.kafka.common.serialization.StringDeserializer");
        props.put("value.deserializer","org.apache.kafka.common.serialization.StringDeserializer");

        KafkaConsumer<String,String> consumer = new KafkaConsumer<>(props);
        consumer.subscribe(Arrays.asList("websocket"));

        while (true) {
            ConsumerRecords<String,String> records = consumer.poll(Duration.ofMillis(100));
            for (ConsumerRecord<String,String> record : records)
                Log.info("0","Latency in apache kafka consumer: {}",System.currentTimeMillis() - record.timestamp());
        }
    }
}

结果

春季消费者

Kafka latency in spring receiver: Topic: websocket latency: 246
Kafka latency in spring receiver: Topic: websocket latency: 246
Kafka latency in spring receiver: Topic: websocket latency: 247
Kafka latency in spring receiver: Topic: websocket latency: 267
Kafka latency in spring receiver: Topic: websocket latency: 260
Kafka latency in spring receiver: Topic: websocket latency: 261
Kafka latency in spring receiver: Topic: websocket latency: 260
Kafka latency in spring receiver: Topic: websocket latency: 306
Kafka latency in spring receiver: Topic: websocket latency: 300
Kafka latency in spring receiver: Topic: websocket latency: 300
Kafka latency in spring receiver: Topic: websocket latency: 299
Kafka latency in spring receiver: Topic: websocket latency: 340
Kafka latency in spring receiver: Topic: websocket latency: 337
Kafka latency in spring receiver: Topic: websocket latency: 337
Kafka latency in spring receiver: Topic: websocket latency: 337
Kafka latency in spring receiver: Topic: websocket latency: 345
Kafka latency in spring receiver: Topic: websocket latency: 345
Kafka latency in spring receiver: Topic: websocket latency: 345
Kafka latency in spring receiver: Topic: websocket latency: 346
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 352
Kafka latency in spring receiver: Topic: websocket latency: 350
Kafka latency in spring receiver: Topic: websocket latency: 349
Kafka latency in spring receiver: Topic: websocket latency: 359
Kafka latency in spring receiver: Topic: websocket latency: 355
Kafka latency in spring receiver: Topic: websocket latency: 354
Kafka latency in spring receiver: Topic: websocket latency: 354
Kafka latency in spring receiver: Topic: websocket latency: 361
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 354
Kafka latency in spring receiver: Topic: websocket latency: 351
Kafka latency in spring receiver: Topic: websocket latency: 351
Kafka latency in spring receiver: Topic: websocket latency: 350
Kafka latency in spring receiver: Topic: websocket latency: 358
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 362
Kafka latency in spring receiver: Topic: websocket latency: 362
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 363
Kafka latency in spring receiver: Topic: websocket latency: 361
Kafka latency in spring receiver: Topic: websocket latency: 359
Kafka latency in spring receiver: Topic: websocket latency: 359
Kafka latency in spring receiver: Topic: websocket latency: 358
Kafka latency in spring receiver: Topic: websocket latency: 351
Kafka latency in spring receiver: Topic: websocket latency: 352
Kafka latency in spring receiver: Topic: websocket latency: 351
Kafka latency in spring receiver: Topic: websocket latency: 370
Kafka latency in spring receiver: Topic: websocket latency: 370
Kafka latency in spring receiver: Topic: websocket latency: 369
Kafka latency in spring receiver: Topic: websocket latency: 368
Kafka latency in spring receiver: Topic: websocket latency: 368
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 364
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 365
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 358
Kafka latency in spring receiver: Topic: websocket latency: 358
Kafka latency in spring receiver: Topic: websocket latency: 358
Kafka latency in spring receiver: Topic: websocket latency: 359
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 353
Kafka latency in spring receiver: Topic: websocket latency: 370
Kafka latency in spring receiver: Topic: websocket latency: 363
Kafka latency in spring receiver: Topic: websocket latency: 362
Kafka latency in spring receiver: Topic: websocket latency: 361
Kafka latency in spring receiver: Topic: websocket latency: 379
Kafka latency in spring receiver: Topic: websocket latency: 378
Kafka latency in spring receiver: Topic: websocket latency: 379
Kafka latency in spring receiver: Topic: websocket latency: 373
Kafka latency in spring receiver: Topic: websocket latency: 371
Kafka latency in spring receiver: Topic: websocket latency: 371
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 372
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 364
Kafka latency in spring receiver: Topic: websocket latency: 374
Kafka latency in spring receiver: Topic: websocket latency: 368
Kafka latency in spring receiver: Topic: websocket latency: 367
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 367
Kafka latency in spring receiver: Topic: websocket latency: 362
Kafka latency in spring receiver: Topic: websocket latency: 363
Kafka latency in spring receiver: Topic: websocket latency: 362
Kafka latency in spring receiver: Topic: websocket latency: 362
Kafka latency in spring receiver: Topic: websocket latency: 361
Kafka latency in spring receiver: Topic: websocket latency: 364
Kafka latency in spring receiver: Topic: websocket latency: 363

Apache Consumer

Latency in apache kafka consumer: 134  
Latency in apache kafka consumer: 130
Latency in apache kafka consumer: 130
Latency in apache kafka consumer: 130
Latency in apache kafka consumer: 130
Latency in apache kafka consumer: 138
Latency in apache kafka consumer: 133
Latency in apache kafka consumer: 133
Latency in apache kafka consumer: 131
Latency in apache kafka consumer: 143
Latency in apache kafka consumer: 141
Latency in apache kafka consumer: 141
Latency in apache kafka consumer: 142
Latency in apache kafka consumer: 140
Latency in apache kafka consumer: 143
Latency in apache kafka consumer: 137
Latency in apache kafka consumer: 225
Latency in apache kafka consumer: 224
Latency in apache kafka consumer: 225
Latency in apache kafka consumer: 223
Latency in apache kafka consumer: 225
Latency in apache kafka consumer: 222
Latency in apache kafka consumer: 217
Latency in apache kafka consumer: 217
Latency in apache kafka consumer: 217
Latency in apache kafka consumer: 230
Latency in apache kafka consumer: 234
Latency in apache kafka consumer: 233
Latency in apache kafka consumer: 233
Latency in apache kafka consumer: 233
Latency in apache kafka consumer: 231
Latency in apache kafka consumer: 231
Latency in apache kafka consumer: 230
Latency in apache kafka consumer: 228
Latency in apache kafka consumer: 226
Latency in apache kafka consumer: 225
Latency in apache kafka consumer: 229
Latency in apache kafka consumer: 226
Latency in apache kafka consumer: 226
Latency in apache kafka consumer: 227
Latency in apache kafka consumer: 227
Latency in apache kafka consumer: 258
Latency in apache kafka consumer: 253
Latency in apache kafka consumer: 252
Latency in apache kafka consumer: 252
Latency in apache kafka consumer: 254
Latency in apache kafka consumer: 252
Latency in apache kafka consumer: 263
Latency in apache kafka consumer: 262
Latency in apache kafka consumer: 261
Latency in apache kafka consumer: 261
Latency in apache kafka consumer: 261
Latency in apache kafka consumer: 257
Latency in apache kafka consumer: 256
Latency in apache kafka consumer: 255
Latency in apache kafka consumer: 270
Latency in apache kafka consumer: 269
Latency in apache kafka consumer: 266
Latency in apache kafka consumer: 263
Latency in apache kafka consumer: 263
Latency in apache kafka consumer: 263
Latency in apache kafka consumer: 262
Latency in apache kafka consumer: 263
Latency in apache kafka consumer: 260
Latency in apache kafka consumer: 257
Latency in apache kafka consumer: 254
Latency in apache kafka consumer: 253
Latency in apache kafka consumer: 253
Latency in apache kafka consumer: 271
Latency in apache kafka consumer: 270
Latency in apache kafka consumer: 267
Latency in apache kafka consumer: 270
Latency in apache kafka consumer: 266
Latency in apache kafka consumer: 266
Latency in apache kafka consumer: 266
Latency in apache kafka consumer: 266
Latency in apache kafka consumer: 264
Latency in apache kafka consumer: 296
Latency in apache kafka consumer: 295
Latency in apache kafka consumer: 297
Latency in apache kafka consumer: 293
Latency in apache kafka consumer: 294
Latency in apache kafka consumer: 298
Latency in apache kafka consumer: 297
Latency in apache kafka consumer: 295
Latency in apache kafka consumer: 295
Latency in apache kafka consumer: 298
Latency in apache kafka consumer: 298
Latency in apache kafka consumer: 302
Latency in apache kafka consumer: 302
Latency in apache kafka consumer: 302
Latency in apache kafka consumer: 300
Latency in apache kafka consumer: 303
Latency in apache kafka consumer: 302
Latency in apache kafka consumer: 303
Latency in apache kafka consumer: 301
Latency in apache kafka consumer: 300
Latency in apache kafka consumer: 299
Latency in apache kafka consumer: 300
Latency in apache kafka consumer: 299

为什么我在kafka消费者中看到如此高的延迟？

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）