Felpfe Inc.
Close this search box.
call 24/7

+484 237-1364‬

Close this search box.

Serializing and deserializing data using common formats (e.g., Avro, JSON)

When working with Apache Kafka, it is essential to serialize and deserialize data to ensure compatibility and efficient data transfer between producers and consumers. Kafka supports various serialization formats, including Avro and JSON, which provide flexibility and compatibility across different systems and programming languages. In this article, we will explore the process of serializing and deserializing data using common formats in Apache Kafka. We will provide code samples, reference links, and resources to guide you through the implementation.

Avro Serialization and Deserialization:

Apache Avro is a popular data serialization framework that provides schema-based serialization and supports rich data structures. Avro provides advantages like schema evolution, compact binary encoding, and language independence. Here’s an example of serializing and deserializing data using Avro in Kafka:

  1. Serializing Data Using Avro:
  • Define an Avro schema using the Avro schema definition language (AVSC).
  • Compile the schema into Java classes using Avro tools or libraries like Apache Avro Maven Plugin.
  • Use the Avro serialization API to serialize data using the defined schema.
  1. Deserializing Data Using Avro:
  • Use the Avro deserialization API to deserialize data using the corresponding Avro schema.

Code Sample: Serializing and Deserializing Data Using Avro in Java

import org.apache.avro.Schema;
import org.apache.avro.generic.GenericData;
import org.apache.avro.generic.GenericRecord;
import org.apache.avro.io.*;
import org.apache.avro.specific.SpecificDatumWriter;
import org.apache.avro.specific.SpecificDatumReader;

import java.io.ByteArrayOutputStream;
import java.io.IOException;

public class AvroSerializationExample {
    public static void main(String[] args) throws IOException {
        // Define the Avro schema
        String avroSchema = "{\"type\":\"record\",\"name\":\"Person\",\"fields\":[{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"age\",\"type\":\"int\"}]}";
        Schema.Parser parser = new Schema.Parser();
        Schema schema = parser.parse(avroSchema);

        // Create a generic Avro record
        GenericRecord record = new GenericData.Record(schema);
        record.put("name", "John Doe");
        record.put("age", 30);

        // Serialize the Avro record
        ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
        BinaryEncoder encoder = EncoderFactory.get().binaryEncoder(outputStream, null);
        DatumWriter<GenericRecord> writer = new SpecificDatumWriter<>(schema);
        writer.write(record, encoder);
        byte[] serializedData = outputStream.toByteArray();

        // Deserialize the Avro record
        DatumReader<GenericRecord> reader = new SpecificDatumReader<>(schema);
        BinaryDecoder decoder = DecoderFactory.get().binaryDecoder(serializedData, null);
        GenericRecord deserializedRecord = reader.read(null, decoder);

        System.out.println("Deserialized Record: " + deserializedRecord);

Reference Link: Apache Avro Documentation – https://avro.apache.org/docs/current/

JSON Serialization and Deserialization:

JSON (JavaScript Object Notation) is a widely used data interchange format due to its simplicity and human-readable nature. Kafka supports JSON serialization and deserialization, making it compatible with various programming languages and systems. Here’s an example of serializing and deserializing data using JSON in Kafka:

  1. Serializing Data to JSON:
  • Convert the data object into JSON format using libraries like Jackson or Gson.
  • Convert the JSON string into a byte array to be used as the Kafka message value.
  1. Deserializing Data from JSON:
  • Convert the Kafka message value (byte array) back into a JSON string.
  • Parse the JSON string using libraries like Jackson or Gson to obtain the data object.

Code Sample: Serializing and Deserializing Data Using JSON in Java

import com.fasterxml.jackson.databind.ObjectMapper;

import java.io.IOException;

public class JsonSerializationExample {
    public static void main(String[] args) throws IOException {
        // Define the data object
        Person person = new Person("John Doe", 30);

        // Serialize the data object to JSON
        ObjectMapper mapper = new ObjectMapper();
        String jsonString = mapper.writeValueAsString(person);

        // Deserialize the JSON string to the data object
        Person deserializedPerson = mapper.readValue(jsonString, Person.class);

        System.out.println("Deserialized Person: " + deserializedPerson);

class Person {
    private String name;
    private int age;

    public Person() {

    public Person(String name, int age) {
        this.name = name;
        this.age = age;

    // Getters and setters
    // ...

Reference Link: Jackson Project Documentation – https://github.com/FasterXML/jackson

Helpful Video: “Introduction to Apache Kafka Serialization and Deserialization” by Edureka – https://www.youtube.com/watch?v=DRaeKZ0hR4A


Serializing and deserializing data using common formats like Avro and JSON is crucial for seamless data transfer and compatibility in Apache Kafka. Avro provides schema-based serialization, enabling schema evolution and language independence. JSON offers a widely supported and human-readable format for serialization.

In this article, we explored the process of serializing and deserializing data using Avro and JSON in Kafka. The provided code samples and reference links to official documentation and helpful videos can guide you through the implementation. By leveraging the power of serialization and deserialization, you can ensure efficient and compatible data communication between Kafka producers and consumers, unlocking the full potential of Apache Kafka for building scalable and robust data streaming applications.

About Author
Ozzie Feliciano CTO @ Felpfe Inc.

Ozzie Feliciano is a highly experienced technologist with a remarkable twenty-three years of expertise in the technology industry.

Stream Dream: Diving into Kafka Streams
In “Stream Dream: Diving into Kafka Streams,”...
Talking in Streams: KSQL for the SQL Lovers
“Talking in Streams: KSQL for the SQL Lovers”...
Stream Symphony: Real-time Wizardry with Spring Cloud Stream Orchestration
Description: The blog post, “Stream Symphony:...
Kafka Chronicles: Saga of Resilient Microservices Communication with Spring Cloud Stream
“Kafka Chronicles: Saga of Resilient Microservices...
Tackling Security in Kafka: A Comprehensive Guide on Authentication and Authorization
As the usage of Apache Kafka continues to grow in organizations...
1 2 3 58
90's, 2000's and Today's Hits
Decades of Hits, One Station

Listen to the greatest hits of the 90s, 2000s and Today. Now on TuneIn. Listen while you code.