[Question] Mimir 6.x, kafka replication factor.

[begemotik]

Hello community.

I haven't found any information on configuring Kafka replication factor. There is no such option in config file when auto-creating a topic in ingestor_storage. Apart from that default values of helm chart ignores configuring it (e.g configuring offsets.topic.replication.factor for kafka itself), though, I expect it's possible using environment variables.

All that makes me unsure if it's a recommended option. Should I set a replication factor of 3 for kafka cluster of 3 nodes + 3 mimir-store-gateways?

Thank you.

[mg1986jp]

I think the Kafka replication factor for the ingest topics isn't configurable through Mimir itself — Mimir auto-creates the topics but relies on Kafka's broker-level defaults for things like default.replication.factor.

You'd need to set default.replication.factor=3 (or whatever you want) in your Kafka broker config, or pre-create the topics manually with the right replication factor before Mimir starts using them. Something like:

kafka-topics.sh --create --topic mimir-ingest --partitions 48 --replication-factor 3 --bootstrap-server kafka:9092

For the offsets.topic.replication.factor, that's a Kafka broker-level setting, not something Mimir can control. It needs to be set in your Kafka server.properties or Helm values.

[s3onghyun]

Mimir's ingest-storage does not own or expose a replication-factor setting. In the Kafka config (pkg/storage/ingest/config.go) the only topic-related knobs are topic, auto_create_topic_enabled (default true), and auto_create_topic_default_partitions. Auto-creation sets the partition count only, never the replication factor, so the topic is created with the broker's default.replication.factor.

So RF=3 is configured on the Kafka side, not in Mimir:

• Set default.replication.factor=3 and min.insync.replicas=2 on the brokers, or
• Pre-create the Mimir topic with RF=3 before Mimir starts; Mimir will just reuse the existing topic.

RF=3 is the right production setting, since that topic is the durable write-ahead buffer for ingest storage. While you are there, also raise the internal topics from their default RF of 1 on a multi-broker cluster: offsets.topic.replication.factor and transaction.state.log.replication.factor.