Guard all deployment roles behind features where possible by ekohl · Pull Request #722 · theforeman/foremanctl

ekohl · 2026-08-03T16:30:09Z

Why are you introducing these changes? (Problem description, related links)

#511 and #527 both rely on having all roles to be fully disableable. This extracts the changes into their own PR.

What are the changes introduced in this pull request?

In the end, this PR should have no difference for end users. Everything is internal and building up. Each individual commit is designed to be readable and build up to the end goal to align both deployment scenarios as much as possible and prepare for a plain Foreman without actually doing that just yet deployment.

It introduces support to foremanctl features to list internal ones. We can debate the implementation, but this was the quickest.
It refactors the Ansible playbook to always use role: ... syntax so the later diffs are smaller and easier to read
It guards roles when a feature is already defined for it, updating tests where needed
It introduces httpd and valkey as internal features to only deploy them when needed (for a plain foreman-proxy)

How to test this pull request

Steps to reproduce:

CI should pass without regression

Checklist

Tests added/updated (if applicable)
Documentation updated (if applicable)

ekohl · 2026-08-03T21:18:37Z

This fails on parsing internal features that have no description. I've added a description to each now because it's the easiest and I like having descriptions. Should the metadata mandate this (implying a docs change) or make the feature parsing more robust?

ehelms · 2026-08-04T01:06:07Z

@@ -29,11 +29,19 @@
    - role: postgresql


Should this get the same treatment?

Not sure. In #527 I also tried postgresql_databases | length > 0 as a condition. I'd prefer to tackle that part separate from this because it's a bit more special.

Let me ask another way. How is postgresql different than valkey? They are both dependencies of Foreman. In the case of a proxy, postgresql is a dependency of container gateway.

But on a plain Foreman Proxy you don't have it at all. So yes, we can probably track it as dependencies of the right features but I thought the scope of this PR is already non-trivial.

I think the general idea and changes make sense to me. I just think we might as well treat postgresql the same as the other backend services.

I'd like to, but your comment makes it sound like you want it in this PR and I'd like to do it as a follow up. Mind if I follow up on it later?

I suppose I don't, however, I am left feeling like this is an incomplete change without given how other backends like Valkey are handled with this change.

Perhaps, but I have needed quite a few iterations to get tests to pass correctly. Keep the changes smaller was helpful. But after #726 I'll see how much of this remains.

ehelms · 2026-08-04T01:06:39Z

@@ -29,11 +29,21 @@
    - role: postgresql


Shouldn't this get the same treatment?

ekohl · 2026-08-04T09:01:34Z

My previous guess of it failing on missing descriptions was wrong. In some cases it fails to call ./foremanctl features with this error:

 ./foremanctl: line 21: exec: obsah: not found

Passing env= to subprocess.check_output clears the virtualenv variables (https://docs.python.org/3/library/subprocess.html#subprocess.run):

If env is not None, it must be a mapping that defines the environment variables for the new process; these are used instead of the default behavior of inheriting the current process’ environment. It is passed directly to Popen. This mapping can be str to str on any platform or bytes to bytes on POSIX platforms much like os.environ or os.environb.

ekohl · 2026-08-04T09:56:35Z

Proxy failed to pull from quay.

Aug 04 09:20:41 quadlet.example.com podman[77612]: 2026-08-04 09:20:41.068389347 +0000 UTC m=+2.324557851 image pull-error  quay.io/foreman/foreman-proxy:nightly unable to copy from source docker://quay.io/foreman/foreman-proxy:nightly: writing blob: storing blob to file "/var/tmp/container_images_storage2929789586/1": happened during read: unexpected EOF (while reconnecting: Get "https://cdn01.quay.io/quayio-production-s3/sha256/94/943a1d816b5dec50982999fa40dd1469cc2a11117a4cfa174a9b671d72a311b4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260804%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260804T092041Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=8eb8950147c25a19e21580783952f6f626ac8f1965365c7e905b1844f53b1264&region=us-east-1&namespace=foreman&repo_name=foreman-proxy&akamai_signature=exp=1785836141~hmac=b2d9d9b0db563d31eab37d943007c2a4743709fa66f6f8c26f53d65b53286c57": EOF)

Is this a more common occurrence?

ehelms · 2026-08-04T12:29:12Z

Proxy failed to pull from quay.

Aug 04 09:20:41 quadlet.example.com podman[77612]: 2026-08-04 09:20:41.068389347 +0000 UTC m=+2.324557851 image pull-error  quay.io/foreman/foreman-proxy:nightly unable to copy from source docker://quay.io/foreman/foreman-proxy:nightly: writing blob: storing blob to file "/var/tmp/container_images_storage2929789586/1": happened during read: unexpected EOF (while reconnecting: Get "https://cdn01.quay.io/quayio-production-s3/sha256/94/943a1d816b5dec50982999fa40dd1469cc2a11117a4cfa174a9b671d72a311b4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260804%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260804T092041Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=8eb8950147c25a19e21580783952f6f626ac8f1965365c7e905b1844f53b1264&region=us-east-1&namespace=foreman&repo_name=foreman-proxy&akamai_signature=exp=1785836141~hmac=b2d9d9b0db563d31eab37d943007c2a4743709fa66f6f8c26f53d65b53286c57": EOF)

Is this a more common occurrence?

First time I have seen it.

ekohl · 2026-08-05T08:44:34Z

2 test failures:

FAILED tests/feature/foreman-proxy/base_test.py::test_foreman_proxy_features - AssertionError: assert 'dynflow' not in ['ansible', 'bmc', 'dynflow', 'logs', 'script']

This tells me that ./foremanctl features doesn't list the dynflow feature as enabled, but in practice it is. It's pulled in transitively: REX pulls in tasks, tasks pulls in dynflow. However, I don't see tasks enabled (because there is no Foreman Proxy).

It would be interesting to write a more dedicated test to check the behavior of foremanctl.

FAILED tests/features_test.py::test_foremanctl_features - AssertionError: Expected feature 'content/python' in output
assert 'content/python' in 'FEATURE                   STATE        DESCRIPTION\nforeman                   enabled      Base Foreman Server\nforeman-proxy             enabled      Base Foreman Proxy\nhammer                    enabled      Foreman CLI\nkatello                   enabled      Content and Subscription Management plugin for Foreman\ngoogle                    enabled      Google Compute Engine plugin for Foreman\nazure-rm                  enabled      Azure Resource Manager plugin for Foreman\nremote-execution          enabled      Remote Execution plugin for Foreman\nansible                   enabled      Ansible plugin for Foreman\nbmc                       enabled      Power management for bare metal hosts (IPMI, Redfish)\nwebhooks                  enabled      Webhooks plugin for Foreman\nrh-cloud                  available    Connection to Red Hat Cloud\niop                       available    iop services\ntemplates                 available    Templates feature for foreman-proxy\ncontent/ostree            available    OSTree content type for Pulp\n'
 +  where 'FEATURE                   STATE        DESCRIPTION\nforeman                   enabled      Base Foreman Server\nforeman-proxy             enabled      Base Foreman Proxy\nhammer                    enabled      Foreman CLI\nkatello                   enabled      Content and Subscription Management plugin for Foreman\ngoogle                    enabled      Google Compute Engine plugin for Foreman\nazure-rm                  enabled      Azure Resource Manager plugin for Foreman\nremote-execution          enabled      Remote Execution plugin for Foreman\nansible                   enabled      Ansible plugin for Foreman\nbmc                       enabled      Power management for bare metal hosts (IPMI, Redfish)\nwebhooks                  enabled      Webhooks plugin for Foreman\nrh-cloud                  available    Connection to Red Hat Cloud\niop                       available    iop services\ntemplates                 available    Templates feature for foreman-proxy\ncontent/ostree            available    OSTree content type for Pulp\n' = CompletedProcess(args=['./foremanctl', 'features'], returncode=0, stdout='FEATURE                   STATE        DESCRIPTION\nforeman                   enabled      Base Foreman Server\nforeman-proxy             enabled      Base Foreman Proxy\nhammer                    enabled      Foreman CLI\nkatello                   enabled      Content and Subscription Management plugin for Foreman\ngoogle                    enabled      Google Compute Engine plugin for Foreman\nazure-rm                  enabled      Azure Resource Manager plugin for Foreman\nremote-execution          enabled      Remote Execution plugin for Foreman\nansible                   enabled      Ansible plugin for Foreman\nbmc                       enabled      Power management for bare metal hosts (IPMI, Redfish)\nwebhooks                  enabled      Webhooks plugin for Foreman\nrh-cloud                  available    Connection to Red Hat Cloud\niop                       available    iop services\ntemplates                 available    Templates feature for foreman-proxy\ncontent/ostree            available    OSTree content type for Pulp\n', stderr='').stdout
FAILED tests/features_test.py::test_foremanctl_features_list_enabled - AssertionError: Expected feature 'content/deb' in output
assert 'content/deb' in 'FEATURE                   STATE        DESCRIPTION\nforeman                   enabled      Base Foreman Server\nforeman-proxy             enabled      Base Foreman Proxy\nhammer                    enabled      Foreman CLI\nkatello                   enabled      Content and Subscription Management plugin for Foreman\ngoogle                    enabled      Google Compute Engine plugin for Foreman\nazure-rm                  enabled      Azure Resource Manager plugin for Foreman\nremote-execution          enabled      Remote Execution plugin for Foreman\nansible                   enabled      Ansible plugin for Foreman\nbmc                       enabled      Power management for bare metal hosts (IPMI, Redfish)\nwebhooks                  enabled      Webhooks plugin for Foreman\n'
 +  where 'FEATURE                   STATE        DESCRIPTION\nforeman                   enabled      Base Foreman Server\nforeman-proxy             enabled      Base Foreman Proxy\nhammer                    enabled      Foreman CLI\nkatello                   enabled      Content and Subscription Management plugin for Foreman\ngoogle                    enabled      Google Compute Engine plugin for Foreman\nazure-rm                  enabled      Azure Resource Manager plugin for Foreman\nremote-execution          enabled      Remote Execution plugin for Foreman\nansible                   enabled      Ansible plugin for Foreman\nbmc                       enabled      Power management for bare metal hosts (IPMI, Redfish)\nwebhooks                  enabled      Webhooks plugin for Foreman\n' = CompletedProcess(args=['./foremanctl', 'features', '--list-enabled'], returncode=0, stdout='FEATURE                   STATE        DESCRIPTION\nforeman                   enabled      Base Foreman Server\nforeman-proxy             enabled      Base Foreman Proxy\nhammer                    enabled      Foreman CLI\nkatello                   enabled      Content and Subscription Management plugin for Foreman\ngoogle                    enabled      Google Compute Engine plugin for Foreman\nazure-rm                  enabled      Azure Resource Manager plugin for Foreman\nremote-execution          enabled      Remote Execution plugin for Foreman\nansible                   enabled      Ansible plugin for Foreman\nbmc                       enabled      Power management for bare metal hosts (IPMI, Redfish)\nwebhooks                  enabled      Webhooks plugin for Foreman\n', stderr='').stdout

This is because the enabled_features fixture lists all features, including internal. Then it looks at foremanctl features --list-enabled which doesn't list the internal ones. The test itself doesn't provide a huge amount of value, but it still checks that --list-enabled correctly filters. Easiest way is to make both list internal and external features.

ekohl · 2026-08-05T14:55:07Z

I'm going to let @evgeni do its thing in #726 since he has similar fixes in there and rebase when it's merged.

ekohl · 2026-08-12T18:41:47Z

Rebased now that #722 is in.

ekohl · 2026-08-12T19:22:42Z

This is an interesting failure:

Aug 12 19:07:29 proxy.example.com systemd[1]: Starting Pulp API socket...
Aug 12 19:07:29 proxy.example.com systemd[38577]: pulp-api.socket: Failed to resolve user apache: No such process

I think the httpd feature that I introduced isn't being enabled on the proxy.

ekohl · 2026-08-12T20:44:32Z

🍏 now

ehelms · 2026-08-20T21:39:47Z

This needs another rebase.

This has no practical difference, but makes it consistent everywhere.

This allows composing the complete deployment and opens the path for a plain Foreman and/or Foreman Proxy.

This makes it easier to exclude them from tests.

ekohl · 2026-08-22T11:12:13Z

And rebased again

pr-processor Bot added the Not yet reviewed label Aug 3, 2026

ekohl mentioned this pull request Aug 3, 2026

Introduce a plain foreman flavor #511

Draft

2 tasks

ekohl force-pushed the all-the-features branch from 4c85f4a to 2002d94 Compare August 3, 2026 21:18

ehelms reviewed Aug 4, 2026

View reviewed changes

pr-processor Bot removed the Not yet reviewed label Aug 4, 2026

ehelms reviewed Aug 4, 2026

View reviewed changes

ekohl force-pushed the all-the-features branch from 2002d94 to 2b5919d Compare August 4, 2026 09:02

ekohl mentioned this pull request Aug 5, 2026

Satellite flavor #726

Merged

2 tasks

ekohl force-pushed the all-the-features branch from 2b5919d to cbbddc2 Compare August 12, 2026 18:41

ekohl force-pushed the all-the-features branch from cbbddc2 to 2e04227 Compare August 12, 2026 19:24

ekohl force-pushed the all-the-features branch from 2e04227 to 996e2a4 Compare August 20, 2026 21:48

ekohl added 3 commits August 22, 2026 13:11

Use hash notation to include role in deploy

f991fa6

This has no practical difference, but makes it consistent everywhere.

Guard all roles behind features when possible

1130717

This allows composing the complete deployment and opens the path for a plain Foreman and/or Foreman Proxy.

Introduce httpd and valkey internal features

4b19537

This makes it easier to exclude them from tests.

ekohl force-pushed the all-the-features branch from 996e2a4 to 4b19537 Compare August 22, 2026 11:12

ehelms approved these changes Aug 22, 2026

View reviewed changes

ehelms merged commit ed867f0 into theforeman:master Aug 22, 2026
19 checks passed

ekohl deleted the all-the-features branch August 22, 2026 19:41

Odilhao mentioned this pull request Aug 24, 2026

Add non-root USER to Pulp container image theforeman/pulp-oci-images#76

Merged

Uh oh!

Conversation

ekohl commented Aug 3, 2026

Why are you introducing these changes? (Problem description, related links)

What are the changes introduced in this pull request?

How to test this pull request

Checklist

Uh oh!

ekohl commented Aug 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ekohl commented Aug 4, 2026

Uh oh!

ekohl commented Aug 4, 2026

Uh oh!

ehelms commented Aug 4, 2026

Uh oh!

ekohl commented Aug 5, 2026

Uh oh!

ekohl commented Aug 5, 2026

Uh oh!

ekohl commented Aug 12, 2026

Uh oh!

ekohl commented Aug 12, 2026

Uh oh!

ekohl commented Aug 12, 2026

Uh oh!

ehelms commented Aug 20, 2026

Uh oh!

ekohl commented Aug 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants