docker) bitnami/kafka without ZooKeeper (KRaft)

Posted Sep 2, 2023

By cornpip

1 min read

This post was migrated from Tistory. You can find the original here.

KRaft mode

If you look into Kafka, you’ll find that it typically uses ZooKeeper to manage metadata, broker status, topic status, and so on. However, if you look at Docker Kafka images, you’ll see docker-compose setups configured without ZooKeeper — apparently, recent versions of Kafka are moving toward a structure that lets you run Kafka without ZooKeeper at all.

KRaft mode was created so Kafka can manage its own metadata without ZooKeeper, and the QUORUM controller is a key concept related to KRaft.

bitnami/kafka

services:
    kafka:
        image: bitnami/kafka:latest
        ports:
            - '9092:9092'
        environment:
            # KRaft settings
            - KAFKA_CFG_NODE_ID=0
            - KAFKA_CFG_PROCESS_ROLES=controller,broker
            - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka:9093
            # Listeners
            - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093
            - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092
            - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
            - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
            - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=PLAINTEXT

The environment variables related to CONTROLLER also seem to be tied to KRaft.

If ADVERTISED_LISTENERS isn’t set, Spring couldn’t establish a connection to Kafka.

Without SECURITY_PROTOCOL_MAP, the Docker container still came up fine and connections/messages worked without issue. I’m not sure exactly what security behavior it controls, but I’ll leave it as is for now.