Avro向后架构演变会引发ClassCastException

问题描述

尝试使用简单的Java程序测试Avro模式演变时，我遇到ClassCastException。

Avro版本：1.10.0

customer-v1.avsc

{
  "type": "record","namespace": "com.practice.kafka","name": "Customer","doc": "Avro schema for Customer","fields": [
    {"name":  "first_name","type":  "string","doc": "Customer first name"},{"name":  "last_name","doc": "Customer last name"},{"name":  "automated_email","type":  "boolean","default": true,"doc": "Receive marketing emails or not"}
  ]
}

customer-v2.avsc

{
  "type": "record","name": "CustomerV2",{"name":  "phone_number","type":  ["null","boolean"],"default": null,"doc": "Optional phone number"},{"name":  "email","default":  "[email protected]","doc":  "Optional email address"}
  ]
}

Program to serialize v1 and deserialize v2

package com.practice.kafka;

import org.apache.avro.file.DataFileReader;
import org.apache.avro.file.DataFileWriter;
import org.apache.avro.io.DatumReader;
import org.apache.avro.io.DatumWriter;
import org.apache.avro.specific.SpecificDatumReader;
import org.apache.avro.specific.SpecificDatumWriter;

import java.io.File;
import java.io.IOException;

public class BackwardSchemaEvolutionSample {

    public static void main(String[] args) {

        // Step 1 - Create specific record
        Customer customer = Customer.newBuilder().setFirstName("John").setLastName("Doe").setAutomatedEmail(false).build();

        // Step 2 - Write specific record to a file
        final DatumWriter<Customer> datumWriter = new SpecificDatumWriter<>();
        try (DataFileWriter<Customer> dataFileWriter = new DataFileWriter<>(datumWriter)) {
            dataFileWriter.create(customer.getSchema(),new File("customer-v1.avro"));
            dataFileWriter.append(customer);
        } catch (IOException e) {
            e.printstacktrace();
        }

        // Step 3 - Read specific record from a file
        final File file = new File("customer-v1.avro");
        final DatumReader<CustomerV2> datumReader = new SpecificDatumReader<>();
        CustomerV2 customerRecord;
        try (DataFileReader<CustomerV2> dataFileReader = new DataFileReader<>(file,datumReader)) {
            customerRecord = dataFileReader.next();
            System.out.println(customerRecord.toString());
        } catch (IOException e) {
            e.printstacktrace();
        }
    }
}

Result

Exception in thread "main" java.lang.classCastException: class com.practice.kafka.Customer cannot be cast to class com.practice.kafka.CustomerV2 (com.practice.kafka.Customer and com.practice.kafka.CustomerV2 are in unnamed module of loader 'app')
    at com.practice.kafka.SchemaEvolutionSample.main(SchemaEvolutionSample.java:34)

能否让我知道如何解决此错误？

解决方法

您定义了2个数据类型Customer和Customer2，并且由于它们没有继承关系，因此您不能进行任何强制类型转换。因此，Java无法进行强制转换，您将获得ClassCastException。在您的代码中，唯一的解决方案是捕获ClassCastException并在catch块中将Customer转换为Customer2。

我假设您正在模拟Kafka环境中架构的更改。在这种情况下，您将通过添加新字段或删除旧字段来扩展现有的Avro架构。

只要类名保持不变，avro模式更改将起作用。

apache-kafka avro java java