KRaft mode
Apache Kafka’s KRaft mode replaces Apache ZooKeeper with Kafka’s own built-in consensus mechanism based on the Raft protocol. This simplifies Kafka’s architecture, reducing operational complexity by consolidating cluster metadata management into Kafka itself.
| The Stackable Operator for Apache Kafka currently does not support automatic cluster upgrades from Apache ZooKeeper to KRaft. |
Overview
-
Introduced: Kafka 2.8.0 (early preview, not production-ready).
-
Matured: Kafka 3.3.x (production-ready, though ZooKeeper is still supported).
-
Default & Recommended: Kafka 3.5+ strongly recommends KRaft for new clusters.
-
Full Replacement: Kafka 4.0.0 (2025) removes ZooKeeper completely.
-
Migration: Tools exist to migrate from ZooKeeper to KRaft, but new deployments should start with KRaft.
Configuration
Enable KRaft mode by adding spec.clusterConfig.metadataManager: KRaft and a Controller role to your cluster manifest as in the example below:
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
name: kafka
spec:
clusterConfig:
metadataManager: KRaft
image:
productVersion: "3.9.2"
brokers:
roleGroups:
default:
replicas: 1
controllers:
roleGroups:
default:
replicas: 3
Using spec.controllers is mutually exclusive with spec.clusterConfig.zookeeperConfigMapName. Configuring both would result in an error.
|
Resources
Each Pod’s resource requirements are the sum of its containers.
Controller Pods run an additional quorum-manager sidecar (see Internal operator details) whose resources are fixed and not configurable through spec.controllers.config.resources.
Only the kafka container is eligible.
| Role | Container | CPU request | CPU limit | Memory (request = limit) | Storage (logDirs) |
|---|---|---|---|---|---|
Controller |
|
250m |
1000m |
1Gi |
2Gi |
Controller |
|
100m |
500m |
512Mi |
— |
Controller |
Pod total |
350m |
1500m |
1.5Gi |
2Gi |
Broker |
|
250m |
1000m |
2Gi |
2Gi |
A minimal KRaft setup consists of 3 Controllers and 3 Brokers:
| CPU request | CPU limit | Memory | Storage | |
|---|---|---|---|---|
3 Controllers |
1050m |
4500m |
4.5Gi |
6Gi |
3 Brokers |
750m |
3000m |
6Gi |
6Gi |
Total |
1800m |
7500m |
10.5Gi |
12Gi |
| The Controller replicas should sum up to an odd number for the Raft consensus. |
Corresponding to the kafka container values above, the operator uses the following resource defaults:
controllers:
config:
resources:
memory:
limit: 1Gi
cpu:
min: 250m
max: 1000m
storage:
logDirs:
capacity: 2Gi
brokers:
config:
resources:
memory:
limit: 2Gi
cpu:
min: 250m
max: 1000m
storage:
logDirs:
capacity: 2Gi
Internal operator details
KRaft mode requires major configuration changes compared to ZooKeeper mode:
-
cluster-id: This is set to themetadata.nameof the KafkaCluster resource during initial formatting -
node.id: This is a calculated integer, hashed from theroleandrolegroupand addedreplicaid. -
process.roles: Will always only bebrokerorcontroller. Mixedbroker,controllerservers are not supported. -
Each controller pod runs an additional
quorum-managersidecar container that, on startup, admits the pod into the KRaft voter set. On pod restart or termination, the controller is removed from the voter set by thepreStophook on thekafkacontainer. -
Exactly one controller formats with
kafka-storage.sh format --standalone, bootstrapping a single-node quorum by itself. It is the controller with the numerically lowestnode.idwhen the cluster is first reconciled. The operator records this choice instatus.kraftBootstrapControllerNodeIdof the KafkaCluster and never changes it afterwards, so controller role groups added later (even with lower values fornode.id) always join the existing quorum. If the status is lost (e.g. the KafkaCluster is restored from a backup without its status), the operator chooses the bootstrap controller again from the current spec. This is harmless as long as the PersistentVolumeClaims of the controllers are restored too. Every other controller formats with--no-initial-controllersand joins purely through the sidecar’sadd-controllercall. Brokers always format with--no-initial-controllerstoo; they are never voters. -
On controllers,
controller.quorum.bootstrap.serverslists the individual pod FQDNs of all controllers, not the role groups' headless Services. A headless Service also resolves to pods that are not ready yet, so a starting controller could resolve it to its own address, and then never find the quorum leader. With one entry per pod, Kafka picks a random entry for every attempt. As a consequence, whenever the number of controller replicas changes, the controller pods are rolled by the restart controller. -
On brokers,
controller.quorum.bootstrap.serverslists a single headless Service,<cluster-name>-controller-headless, which selects the controllers of all role groups. A broker is never part of this Service, so it cannot resolve it to its own address. The Service only resolves to controllers that are ready, i.e. members of the quorum, so a broker never connects to a controller that is still starting. Adding or removing controller role groups or replicas therefore never rolls the brokers.
Kerberos
KRaft clusters support Kerberos (GSSAPI) authentication for clients.
Reference an AuthenticationClass with the Kerberos provider as described in Security.
Kerberos is only used on the client-facing listeners of the brokers.
The CONTROLLER listener always uses SSL with mutual authentication: brokers and controllers authenticate each other with the certificates of the internalSecretClass, even when Kerberos is enabled.
Controllers therefore have no keytab.
Known Issues
-
Automatic migration from Apache ZooKeeper to KRaft is not supported.
-
The quorum is created once by the controller with the lowest
node.idusing--standalone. If this controller loses its PVC, a new conflicting quorum is created on restart. -
A Controller that loses its persistent volume is not re-admitted to the voter set automatically, because it returns with a new directory ID while the quorum still lists the old one.
Kraft migration guide
The operator version 26.3.0 adds support for migrating Kafka clusters from ZooKeeper to KRaft mode.
This guide describes the steps required to migrate an existing Kafka cluster managed by the Stackable Kafka operator from ZooKeeper to KRaft mode.
| Before starting the migration we recommend to reduce producer/consumer operations to a minimum or even pause them completely if possible to reduce the risk of data loss during the migration. |
To make the migration step as clear as possible, we’ll use a complete working example throughout this guide. The example cluster has one ZooKeeper server and three brokers, uses default resources and a single role group, and is secured as follows:
-
Clients authenticate with TLS client certificates, using an
AuthenticationClasswith the TLS provider (spec.clusterConfig.authentication). -
Traffic between the Kafka nodes is encrypted with the
internalSecretClasskafka-internal-tls, and the client-facing listeners with thetlsSecretClass(serverSecretClass). -
Neither Kerberos nor OPA is used.
The example manifests set no other configuration besides what the migration requires.
|
The steps in this guide were only validated for the example above. In particular, the interaction of the migration with Kerberos and OPA was not tested. If your cluster uses either of them, check the following points before you start and test the migration on a non-production copy of your cluster first.
|
We’ll use Kafka version 3.9.2 for this purpose.
This is because this is the last version from the 3.x Kafka series that runs on ZooKeeper mode and is supported by the SDP.
Like any cluster that was set up before the migration was planned, the example cluster does not configure its broker ids. Kafka generated them when the brokers first started, and step 1 below shows how to collect and pin them.
We start by creating a dedicated namespace to work in and deploy the Kafka cluster including ZooKeeper and credentials.
---
apiVersion: v1
kind: Namespace
metadata:
labels:
stackable.tech/vendor: Stackable
name: kraft-migration
---
apiVersion: zookeeper.stackable.tech/v1alpha1
kind: ZookeeperCluster
metadata:
name: simple-zk
namespace: kraft-migration
spec:
image:
productVersion: 3.9.6
pullPolicy: IfNotPresent
servers:
roleGroups:
default:
replicas: 1
---
apiVersion: zookeeper.stackable.tech/v1alpha1
kind: ZookeeperZnode
metadata:
name: simple-kafka-znode
namespace: kraft-migration
spec:
clusterRef:
name: simple-zk
---
apiVersion: secrets.stackable.tech/v1alpha1
kind: SecretClass
metadata:
name: kafka-internal-tls
spec:
backend:
autoTls:
ca:
secret:
name: secret-provisioner-kafka-internal-tls-ca
namespace: kraft-migration
autoGenerate: true
---
apiVersion: authentication.stackable.tech/v1alpha1
kind: AuthenticationClass
metadata:
name: kafka-client-auth-tls
spec:
provider:
tls:
clientCertSecretClass: kafka-client-auth-secret
---
apiVersion: secrets.stackable.tech/v1alpha1
kind: SecretClass
metadata:
name: kafka-client-auth-secret
spec:
backend:
autoTls:
ca:
secret:
name: secret-provisioner-tls-kafka-client-ca
namespace: kraft-migration
autoGenerate: true
---
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
name: simple-kafka
namespace: kraft-migration
spec:
image:
productVersion: 3.9.2
pullPolicy: IfNotPresent
clusterConfig:
metadataManager: ZooKeeper
authentication:
- authenticationClass: kafka-client-auth-tls
tls:
internalSecretClass: kafka-internal-tls
serverSecretClass: tls
zookeeperConfigMapName: simple-kafka-znode
brokers:
roleGroups:
default:
replicas: 3
Next, from one of the broker pods, we will create a topic called kraft-migration-topic with 3 partitions to verify the migration later.
$ /stackable/kafka/bin/kafka-topics.sh \
--create \
--command-config /stackable/config/client.properties \
--bootstrap-server simple-kafka-broker-default-0-listener-broker.kraft-migration.svc.cluster.local:9093 \
--partitions 3 \
--topic kraft-migration-topic
And - also from one of the broker pods - publish some test messages to it:
$ /stackable/kafka/bin/kafka-producer-perf-test.sh \
--producer.config /stackable/config/client.properties \
--producer-props bootstrap.servers=simple-kafka-broker-default-0-listener-broker.kraft-migration.svc.cluster.local:9093 \
--topic kraft-migration-topic \
--payload-monotonic \
--throughput 1 \
--num-records 100
We now have a working Kafka cluster with ZooKeeper and some test data.
1. Preserve the broker ids
Before KRaft support was added to the operator in SDP 25.11.0, Kafka generated the broker ids and the operator did not know them.
In KRaft mode, the id of a broker is its node.id, which must be the same id that is already stored in the broker’s data directory and in the cluster metadata.
The operator can only keep the existing ids if you tell it what they are, so this has to be done before anything else.
In this step we will perform the following actions:
-
Collect the current broker ids from the running brokers.
-
Store them in a ConfigMap.
-
Point
spec.clusterConfig.brokerIdPodConfigMapNameof theKafkaClusterresource to this ConfigMap. -
Wait for the brokers to restart and verify that their ids did not change.
Each broker records its id in the meta.properties file in its data directory:
$ kubectl -n kraft-migration exec -c kafka simple-kafka-broker-default-0 -- grep broker.id /stackable/data/topicdata/meta.properties
broker.id=1001
The ids that Kafka generates usually start at 1001, and they are not necessarily in the order of the pod indexes.
The following commands collect the id of every broker Pod, of all role groups, and create a ConfigMap with one <pod-name>: <broker-id> entry per Pod:
NAMESPACE=kraft-migration
ARGS=()
for POD in $(kubectl get pods -n $NAMESPACE \
-l app.kubernetes.io/name=kafka,app.kubernetes.io/instance=simple-kafka,app.kubernetes.io/component=broker \
-o jsonpath='{.items[*].metadata.name}'); do
ID=$(kubectl exec -n $NAMESPACE -c kafka $POD -- grep '^broker.id=' /stackable/data/topicdata/meta.properties | cut -d= -f2)
echo "$POD -> $ID"
ARGS+=("--from-literal=$POD=$ID")
done
kubectl create configmap broker-ids -n $NAMESPACE "${ARGS[@]}"
The output lists the ids that were found:
simple-kafka-broker-default-0 -> 1001
simple-kafka-broker-default-1 -> 1002
simple-kafka-broker-default-2 -> 1003
configmap/broker-ids created
Check that the ConfigMap contains an entry for every broker Pod before you continue.
Now point the KafkaCluster to the ConfigMap.
Create the ConfigMap first, because the broker Pods mount it.
kubectl -n kraft-migration patch kafkacluster simple-kafka --type merge \
-p '{"spec":{"clusterConfig":{"brokerIdPodConfigMapName":"broker-ids"}}}'
The operator restarts the brokers one after the other.
Once they are all ready, each broker must report the same id as before.
The broker.id in the configuration file of the broker must match the id in its meta.properties:
$ kubectl -n kraft-migration exec -c kafka simple-kafka-broker-default-0 -- grep broker.id /tmp/broker.properties
broker.id=1001
broker.id.generation.enable=false
The brokerIdPodConfigMapName property and the ConfigMap must be preserved for the lifetime of the cluster, also after the migration is complete.
The brokers keep using these ids in KRaft mode, and without the ConfigMap the operator would fall back to generated ids that do not match the ones already stored by the brokers.
The following steps also contain this property in the KafkaCluster manifests.
2. Start Kraft controllers
In this step we will perform the following actions:
-
Retrieve the current
cluster.idas generated by Kafka. -
Update the
KafkaClusterresource to addspec.controllersproperty. -
Configure the controllers to run in migration mode.
-
Apply the changes and wait for all cluster pods to become ready.
We can obtain the current cluster.id either by inspecting the ZooKeeper data or from meta.properties file on one of the brokers.
$ kubectl -n kraft-migration exec -c kafka simple-kafka-broker-default-0 -- cat /stackable/data/topicdata/meta.properties | grep cluster.id
cluster.id=MyCya7hbTD-Hay8PgCsCYA
We add this value to the KAFKA_CLUSTER_ID environment variable for both brokers and controllers.
The cluster id is generated by Kafka for each cluster, so replace the value MyCya7hbTD-Hay8PgCsCYA in all the example manifests below with your own.
The broker ids were already pinned in step 1 and spec.clusterConfig.brokerIdPodConfigMapName is part of the manifest below.
The KAFKA_CLUSTER_ID environment variable and brokerIdPodConfigMapName must be preserved for the rest of the migration process and the lifetime of the cluster.
The complete example KafkaCluster resource after applying the required changes looks as follows:
---
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
name: simple-kafka
namespace: kraft-migration
spec:
image:
productVersion: 3.9.2
pullPolicy: IfNotPresent
clusterConfig:
metadataManager: ZooKeeper
authentication:
- authenticationClass: kafka-client-auth-tls
tls:
internalSecretClass: kafka-internal-tls
serverSecretClass: tls
zookeeperConfigMapName: simple-kafka-znode
brokerIdPodConfigMapName: broker-ids
brokers:
envOverrides:
KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
roleGroups:
default:
replicas: 3
controllers:
roleGroups:
default:
replicas: 3
envOverrides:
KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
configOverrides:
controller.properties:
zookeeper.metadata.migration.enable: "true" # Enable migration mode so the controller can read metadata from ZooKeeper.
We kubectl apply the updated resource and wait for brokers and controllers to become ready.
3. Migrate metadata
In this step we will perform the following actions:
-
Configure the controller quorum on the brokers.
-
Enable metadata migration mode on the brokers.
-
Apply the changes and restart the broker pods.
To start the metadata migration, we need to add the zookeeper.metadata.migration.enable: "true" and controller quorum configuration to the broker configuration.
For this step, the complete example KafkaCluster resource looks as follows:
---
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
name: simple-kafka
namespace: kraft-migration
spec:
image:
productVersion: 3.9.2
pullPolicy: IfNotPresent
clusterConfig:
metadataManager: ZooKeeper
authentication:
- authenticationClass: kafka-client-auth-tls
tls:
internalSecretClass: kafka-internal-tls
serverSecretClass: tls
zookeeperConfigMapName: simple-kafka-znode
brokerIdPodConfigMapName: broker-ids
brokers:
envOverrides:
KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
roleGroups:
default:
replicas: 3
configOverrides:
broker.properties:
inter.broker.protocol.version: "3.9" # - Latest value known to Kafka 3.9.2
zookeeper.metadata.migration.enable: "true" # - Enable migration mode so the broker can participate in metadata migration.
controller.listener.names: "CONTROLLER"
controller.quorum.bootstrap.servers: "simple-kafka-controller-default-0.simple-kafka-controller-default-headless.kraft-migration.svc.cluster.local:9093,simple-kafka-controller-default-1.simple-kafka-controller-default-headless.kraft-migration.svc.cluster.local:9093,simple-kafka-controller-default-2.simple-kafka-controller-default-headless.kraft-migration.svc.cluster.local:9093"
controllers:
roleGroups:
default:
replicas: 3
envOverrides:
KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
configOverrides:
controller.properties:
zookeeper.metadata.migration.enable: "true" # Enable migration mode so the controller can read metadata from ZooKeeper.
The brokers are now restarted automatically due to configuration changes.
Finally we check that metadata migration was successful:
kubectl logs -n kraft-migration simple-kafka-controller-default-0 | grep -i 'completed migration' \
|| kubectl logs -n kraft-migration simple-kafka-controller-default-1 | grep -i 'completed migration' \
|| kubectl logs -n kraft-migration simple-kafka-controller-default-2 | grep -i 'completed migration'
...
[2026-11-19 11:02:00,532] INFO [KRaftMigrationDriver id=2110489703] Completed migration of metadata from ZooKeeper to KRaft. 59 records were generated in 111 ms across 1 batches. The average time spent waiting on a batch was 28.00 ms. The record types were {TOPIC_RECORD=2, PARTITION_RECORD=53, CONFIG_RECORD=3, PRODUCER_IDS_RECORD=1}. The current metadata offset is now 452 with an epoch of 1. Saw 3 brokers in the migrated metadata [1001, 1002, 1003]. (org.apache.kafka.metadata.migration.KRaftMigrationDriver)
The exact counts depend on the topics and brokers of your cluster.
Only the controller that leads the quorum logs this message, hence the || chain above.
4. Migrate brokers
| This is the last step before fully switching to KRaft mode. In case of unforeseen issues, it is the last step where we can roll back to ZooKeeper mode. |
In this step we will perform the following actions:
-
Remove the migration properties from the previous step on the brokers.
-
Assign Kraft role properties to brokers.
-
Apply the changes and restart the broker pods.
We need to preserve the quorum configuration added in the previous step.
For this step, the complete example KafkaCluster resource looks as follows:
---
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
name: simple-kafka
namespace: kraft-migration
spec:
image:
productVersion: 3.9.2
pullPolicy: IfNotPresent
clusterConfig:
metadataManager: ZooKeeper
authentication:
- authenticationClass: kafka-client-auth-tls
tls:
internalSecretClass: kafka-internal-tls
serverSecretClass: tls
zookeeperConfigMapName: simple-kafka-znode
brokerIdPodConfigMapName: broker-ids
brokers:
envOverrides:
KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
roleGroups:
default:
replicas: 3
configOverrides:
broker.properties:
controller.listener.names: "CONTROLLER"
controller.quorum.bootstrap.servers: "simple-kafka-controller-default-0.simple-kafka-controller-default-headless.kraft-migration.svc.cluster.local:9093,simple-kafka-controller-default-1.simple-kafka-controller-default-headless.kraft-migration.svc.cluster.local:9093,simple-kafka-controller-default-2.simple-kafka-controller-default-headless.kraft-migration.svc.cluster.local:9093"
process.roles: "broker"
node.id: "${env:REPLICA_ID}"
controllers:
roleGroups:
default:
replicas: 3
envOverrides:
KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
configOverrides:
controller.properties:
zookeeper.metadata.migration.enable: "true" # Enable migration mode so the controller can read metadata from ZooKeeper.
5. Enable Kraft mode
After this step, the cluster will be fully running in KRaft mode and it cannot be rolled back to ZooKeeper mode anymore.
In this step we will perform the following actions:
-
Put the cluster in Kraft mode by updating the
spec.clusterConfig.metadataManagerproperty. -
Remove Kraft quorum configuration from the broker pods.
-
Remove the ZooKeeper migration flag from the controllers.
-
Apply the changes and restart all pods.
We need to preserve the KAFKA_CLUSTER_ID environment variable for the rest of the lifetime of this cluster.
The complete example KafkaCluster resource after applying the required changes looks as follows:
---
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
name: simple-kafka
namespace: kraft-migration
spec:
image:
productVersion: 3.9.2
pullPolicy: IfNotPresent
clusterConfig:
metadataManager: KRaft
authentication:
- authenticationClass: kafka-client-auth-tls
tls:
internalSecretClass: kafka-internal-tls
serverSecretClass: tls
brokerIdPodConfigMapName: broker-ids
brokers:
envOverrides:
KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
roleGroups:
default:
replicas: 3
configOverrides:
broker.properties:
controller.listener.names: "CONTROLLER"
controllers:
roleGroups:
default:
replicas: 3
envOverrides:
KAFKA_CLUSTER_ID: "MyCya7hbTD-Hay8PgCsCYA"
Verify that the cluster is healthy and consumer/producer operations work as expected.
We can consume the previously produced messages by running the command below on one of the broker pods:
/stackable/kafka/bin/kafka-console-consumer.sh \
--consumer.config /stackable/config/client.properties \
--bootstrap-server simple-kafka-broker-default-0-listener-broker.kraft-migration.svc.cluster.local:9093 \
--topic kraft-migration-topic \
--from-beginning \
--max-messages 100 \
--timeout-ms 30000
The consumer should report Processed a total of 100 messages.
We read from all partitions because the producer does not necessarily write to partition 0.
6. Cleanup
Before proceeding with this step please ensure that the Kafka cluster is fully operational in KRaft mode.
In this step we remove the now unused ZooKeeper cluster and related resources.
If the ZooKeeper cluster is also serving other use cases than Kafka you can skip this step.
In our example we can remove the ZooKeeper cluster and the Znode resource as follows:
kubectl delete -n kraft-migration zookeeperznodes simple-kafka-znode
kubectl delete -n kraft-migration zookeeperclusters simple-zk