Skip to content

Optimize Kafka topic initialization on re-deploy - #794

Open
ehelms wants to merge 1 commit into
theforeman:masterfrom
ehelms:kafka-topic-init-guard
Open

Optimize Kafka topic initialization on re-deploy#794
ehelms wants to merge 1 commit into
theforeman:masterfrom
ehelms:kafka-topic-init-guard

Conversation

@ehelms

@ehelms ehelms commented Aug 29, 2026

Copy link
Copy Markdown
Member

Why are you introducing these changes? (Problem description, related links)

On every re-deploy, the Kafka topic initialization task (iop_kafka : Initialize Kafka topics) runs the expensive init.sh --create operation, which spawns 17 parallel JVMs to create topics. This takes ~11.4 seconds even when all topics already exist and no actual work is needed.

This pattern mirrors the puppet-iop implementation (puppet-iop/manifests/core_kafka.pp:80-91), which guards topic creation with an unless check.

What are the changes introduced in this pull request?

  • Add Check whether Kafka topics already exist task that runs /opt/kafka/init.sh --check before topic creation
  • Make the Initialize Kafka topics task conditional on iop_kafka_check.rc != 0
  • The --check operation runs a single JVM vs 17 parallel JVMs in --create

How to test this pull request

Steps to reproduce:

  • Deploy with IoP enabled: ./foremanctl deploy --flavor katello --features iop
  • Verify topics are created: podman exec iop-core-kafka /opt/kafka/init.sh --check (should succeed)
  • Re-deploy without changes: ./foremanctl deploy --flavor katello --features iop
  • Observe timing: iop_kafka : Check whether Kafka topics already exist should run in ~1-2s and Initialize Kafka topics should be skipped
  • Compare to before: previously Initialize Kafka topics ran every deploy and took ~11.4s

Checklist

  • Tests added/updated (if applicable)
  • Documentation updated (if applicable)

Check if topics exist before running expensive create operation.
Reduces re-deploy time from ~11s to ~2s when topics already exist.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant