Felpfe Inc.
Search
Close this search box.
call 24/7

+484 237-1364‬

Search
Close this search box.

Verifying the successful installation and setup of Kafka

After installing and configuring Apache Kafka, it is crucial to verify the successful setup of the Kafka environment before proceeding with application development or data streaming. In this article, we will explore various ways to verify the installation and setup of Kafka, including testing connectivity, creating topics, producing and consuming messages, and monitoring Kafka’s health. We will provide code samples, references, and resources to guide you through the verification process.

Testing Connectivity:

  1. Test ZooKeeper Connectivity:
  • Start ZooKeeper if it is not already running.
  • Open a terminal or command prompt and run the following command to connect to ZooKeeper:
    shell $ zkCli.sh -server localhost:2181
  • If the connection is successful, you will see the ZooKeeper command-line interface prompt.
  1. Test Kafka Broker Connectivity:
  • Open a terminal or command prompt and run the following command to create a test topic:
    shell $ kafka-topics.sh --create --topic test-topic --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1
  • If the topic is created without any errors, it indicates successful connectivity with the Kafka broker.

Creating and Managing Topics:

  1. Create a Topic:
  • Use the kafka-topics.sh command-line tool to create a topic. For example, to create a topic named “my-topic” with a single partition, run the following command:
    shell $ kafka-topics.sh --create --topic my-topic --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1
  1. List Existing Topics:
  • To verify that the topic was created successfully, use the following command to list all the topics in the Kafka cluster:
    shell $ kafka-topics.sh --list --bootstrap-server localhost:9092

Producing and Consuming Messages:

  1. Produce Messages to a Topic:
  • Open a terminal or command prompt and run the following command to start the console producer:
    shell $ kafka-console-producer.sh --topic my-topic --bootstrap-server localhost:9092
  • Once the producer starts, enter messages that you want to publish to the topic.
  1. Consume Messages from a Topic:
  • Open a new terminal or command prompt and run the following command to start the console consumer:
    shell $ kafka-console-consumer.sh --topic my-topic --bootstrap-server localhost:9092 --from-beginning
  • The console consumer will start displaying the messages published to the topic.

Monitoring Kafka Cluster:

  1. Use Kafka Manager or Confluent Control Center:
  • Kafka Manager (https://github.com/yahoo/kafka-manager) and Confluent Control Center (https://www.confluent.io/product/control-center) are popular tools for monitoring and managing Kafka clusters. Install and configure them to gain insights into Kafka cluster health, performance, and resource utilization.

Code Sample: Verifying Kafka Connectivity in Java

Java
import org.apache.kafka.clients.admin.AdminClient;
import org.apache.kafka.clients.admin.AdminClientConfig;
import org.apache.kafka.clients.admin.ListTopicsResult;
import org.apache.kafka.clients.admin.NewTopic;

import java.util.Properties;
import java.util.concurrent.ExecutionException;

public class KafkaVerification {
    public static void main(String[] args) {
        Properties props = new Properties();
        props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");

        try (AdminClient adminClient = AdminClient.create(props)) {
            // Verify connectivity
            ListTopicsResult topicsResult = adminClient.listTopics();


 topicsResult.names().get().forEach(System.out::println);

            // Create a test topic
            NewTopic newTopic = new NewTopic("test-topic", 1, (short) 1);
            adminClient.createTopics(Collections.singleton(newTopic)).all().get();

            // Verify topic creation
            topicsResult = adminClient.listTopics();
            topicsResult.names().get().forEach(System.out::println);
        } catch (InterruptedException | ExecutionException e) {
            e.printStackTrace();
        }
    }
}

Reference Link: Apache Kafka Documentation – https://kafka.apache.org/documentation/

Helpful Video: “Apache Kafka Quick Start” by Confluent – https://www.youtube.com/watch?v=Z3FZO6ArGoU

Conclusion:

Verifying the successful installation and setup of Apache Kafka is crucial before starting any data streaming or application development. By testing connectivity with ZooKeeper and Kafka brokers, creating and managing topics, and producing and consuming messages, you can ensure that your Kafka environment is functioning as expected.

The provided code samples, references to the official Kafka documentation, and helpful videos empower you to perform the necessary verifications with confidence. Monitoring tools like Kafka Manager or Confluent Control Center can further assist in monitoring the health and performance of your Kafka cluster.

By verifying the successful installation and setup of Apache Kafka, you establish a solid foundation for building robust, scalable, and reliable data streaming applications, unlocking the full potential of Kafka’s distributed streaming platform.

Unleashing The Tech Marvels

Discover a tech enthusiast’s dreamland as our blog takes you on a thrilling journey through the dynamic world of programming. 

More Post like this

About Author
Ozzie Feliciano CTO @ Felpfe Inc.

Ozzie Feliciano is a highly experienced technologist with a remarkable twenty-three years of expertise in the technology industry.

kafka-logo-tall-apache-kafka-fel
Stream Dream: Diving into Kafka Streams
In “Stream Dream: Diving into Kafka Streams,”...
ksql
Talking in Streams: KSQL for the SQL Lovers
“Talking in Streams: KSQL for the SQL Lovers”...
spring_cloud
Stream Symphony: Real-time Wizardry with Spring Cloud Stream Orchestration
Description: The blog post, “Stream Symphony:...
1_GVb-mYlEyq_L35dg7TEN2w
Kafka Chronicles: Saga of Resilient Microservices Communication with Spring Cloud Stream
“Kafka Chronicles: Saga of Resilient Microservices...
kafka-logo-tall-apache-kafka-fel
Tackling Security in Kafka: A Comprehensive Guide on Authentication and Authorization
As the usage of Apache Kafka continues to grow in organizations...
1 2 3 58
90's, 2000's and Today's Hits
Decades of Hits, One Station

Listen to the greatest hits of the 90s, 2000s and Today. Now on TuneIn. Listen while you code.