Container Gateway support by ianballou · Pull Request #611 · theforeman/foremanctl

ianballou · 2026-07-01T14:41:15Z

Why are you introducing these changes? (Problem description, related links)

The smart_proxy_container_gateway plugin is required for users to consume container content from smart proxies with Pulp.

What are the changes introduced in this pull request?

Adds support for the container gateway plugin to be configured with foreman-proxy.

How to test this pull request

Steps to reproduce:

Use quay.io/iballou/foreman-proxy-container-gateway:latest until Install container_gateway gem foreman-oci-images#53 is merged
Deploy a foreman proxy normally and see that the container gateway plugin is enabled.

Checklist

Tests added/updated (if applicable)
Documentation updated (if applicable)

ehelms · 2026-07-02T14:24:58Z

You can rebase this now that deploy-proxy work has been merged.

ianballou · 2026-07-02T19:18:19Z

Linking up theforeman/foreman-oci-images#53

ianballou · 2026-07-09T16:09:52Z

Katello doesn't support the container gateway correctly yet. In its code, it expects the Pulp smart proxy to have the container gateway feature, but instead, the container gateway feature is on the non-Pulp smart proxy (since there are two per smart proxy host).

ianballou · 2026-07-15T14:08:51Z

From my last comment - if this merges, containerized Katello + Container Gateway won't be any more broken than it is today, since Katello simply won't find the container gateway capsule and will skip the sync.

ehelms · 2026-07-15T14:55:15Z

  description: iop services
  dependencies:
    - rh-cloud
+container-gateway:


These feels like it should be part of the "content/container" feature or a dependency of it... but only in the proxy use case.

~~Sure, that makes sense to me.~~

Actually, I gave this some thought. What is the "proxy" use case we're talking about here? It's foreman-proxy-content specifically. That is the only time we want to install the container gateway. I thought about having a dependency relationship with a conditional on the use of foreman-proxy-content, but that seems like a roundabout way to achieve the same result as just having the container gateway be a part of the foreman-proxy-content flavor.

So, while there isn't a direct relationship between the container gateway and content/container, there is a connection between them created by the foreman-proxy-content flavor.

If there is no Pulp or there is no Pulp with container content present, does this feature work? What happens if a container-gateway is deployed without Pulp or Pulp with container content on the same server?

We can go this route for now, if we need to, to allow the feature to work, but I would ask that we track doing the dependency relationship properly across foremanctl and Katello.

The container gateway does require Pulp, so from the standpoint of looking into deploying a smart proxy manually via features it would make sense for there to be some dependency there. I'm trying to weigh the complexity for the short term of using the foreman-proxy-content flavor as the glue between Pulp and container gateway since we seem to be putting a lot of emphasis into it being "the" smart proxy deployment at the moment.

But I totally agree that the dependency should be linked more generally for users constructing their own capsules without using the foreman proxy content flavor. I can think up an issue for that that covers what we'd need to achieve the dependency with a conditional on the foreman-proxy context.

I'm cooking something up, I'll push a commit soon.

Okay, so while I am going to push something that links these dependencies - I currently see no way for an admin to build a foreman-proxy-content without using the flavor, which means the dependency link is purely for sanity rather than user benefit. At least for now.

The Pulp feature and its content types (besides ostree) are all internal, so I don't see how a user can use the foreman-proxy feature to build their own foreman-proxy-content. I suppose this may get smoothed out in the future.

What if we add the proxy part to content/container, so you get:

content/container: description: Container content type for Pulp internal: true dependencies: - pulp foreman_proxy: plugin_name: container_gateway

but the real enablement in src/roles/foreman_proxy/templates/settings.d/container_gateway.yml.j2 and everywhere else is guarded by a second boolean that is only true on external proxy deployments?

Thinking on it more, I do think this is a broader design question that affects multiple features. I asked @ianballou to file #723 to discuss and solve it broadly and allow this PR to progress without blocking on this needed bigger solution.

ianballou · 2026-07-23T14:24:36Z

@ehelms I reorganized the DB connection string variables so an Ansible role user can set the individual components or the entire string manually if they wish.

I also added an integration test along with a temporary container image to see it working while we wait for theforeman/foreman-oci-images#53.

ehelms · 2026-07-23T14:57:22Z


+# Container gateway settings
+foreman_proxy_container_gateway_pulp_endpoint: "https://{{ ansible_facts['fqdn'] }}"
+foreman_proxy_container_gateway_db_host: "{{ container_gateway_database_host }}"


These aren't really defaults if they are defined by variables from somewhere else. I would expect default values as strings here, and for database.yml to then define the left hand side.

sjha4 · 2026-07-23T16:24:02Z

Ran a test for enabling container_gateway with foremanctl deploy-proxy on this branch on an existing proxy deployment and it worked well for me..Was able to run a backup as a follow up test and it correctly identified container_gateway and backed up the pulp and container_gateway DBs. 👍🏼

Update: Was able to test the proxy backup/restore on top of this and it worked well. #684 (comment)

ianballou · 2026-07-23T19:45:01Z

@ehelms I'm wrapping my head around the proper organization of the variables your asking about, but I made a push in response to your comments.

I need to look into the failing tests, probably some expectations in other tests that I did not update.

ianballou · 2026-07-24T15:35:45Z

I pushed a change that makes the container gateway a dependency of content/container for all cases not matching "Foreman", which I assume here would mean smart proxy.

I did this because foreman-proxy can be on Foreman or not on Foreman. Thus, the only marker of a "foreman-proxy-content" machine is the "foreman-proxy-content" flavor. The flavor alone is not enough to say there is a dependency between content/container and the container gateway, so my next best idea is to use the lack of Foreman in a deployment to mean that we are talking about a "foreman proxy content".

This change means that we have a new concept of "unless" for features that are incompatible with each other. To accomplish this, I turned feature dependencies into dictionaries that can have "unless" as a list of incompatible plugins.

ehelms · 2026-07-24T18:40:57Z

There are similar conversations happening in #519 (comment)

CC @ekohl @jeremylenz @shubhamsg199

The tension here is we have a single definition of features and then multiple scenarios that can apply:

Features that only apply to Foreman server
Features that apply to Foreman server and external proxy
Features that apply to only external proxy

ianballou · 2026-07-24T20:13:58Z

I had difficulty writing this new feature in a way that didn't assume "no Forman == Forman proxy content" because the only real indicator of this that I could find is the use of the flavor. On top of that, the dependency linking feels unhelpful with current foremanctl because of this huge reliance on the flavor.

I think the fact that flavors include internal only features is part of the problem. If we want flavor use to be optional, then they should not include internal features that cannot be enabled outside of the flavors. If we don't want flavors to be optional, then I think it should be fine for them to be a first class part of the feature dependency chain.

ianballou · 2026-08-17T20:51:27Z

Alrighty folks, tests are passing, this is ready for a (final?) look.

ehelms · 2026-08-17T22:04:07Z

 foreman_container_tag: "{{ container_tag_stream }}"
-foreman_proxy_container_image: "quay.io/foreman/foreman-proxy"
-foreman_proxy_container_tag: "{{ container_tag_stream }}"
+foreman_proxy_container_image: "quay.io/iballou/foreman-proxy"


This still needs updating

Will update once someone says "I'd ack this if that hack wasn't there!"

Which hack?

I'm guessing using an image built from a PR.

Yeah the use of my personal image is the hack.

You can drop this now.

ekohl · 2026-08-18T08:20:43Z

+@pytest.fixture
+def podman_login(server, server_fqdn):
+    """Login to the container registry and logout after the test."""
+    result = server.run(f"podman login {server_fqdn} --username admin --password changeme --tls-verify=false")


You're supposed to use interpolation because that also performs escaping. See https://testinfra.readthedocs.io/en/latest/modules.html#testinfra.host.Host.run for more:

Suggested change

result = server.run(f"podman login {server_fqdn} --username admin --password changeme --tls-verify=false")

result = server.run("podman login %s --username admin --password changeme --tls-verify=false", server_fqdn)

ekohl · 2026-08-18T08:25:45Z

And overall I'd say this is 👍 so you can drop the hack.

sjha4 · 2026-08-19T01:42:51Z

I was able to test through the current state of this and it continues to work. 🟢

Side question: Since there are multiple proxies now, do we need to ensure that all proxies running on a server are attached to the same orgs/locations. For pulp and container_gateway in particular, having those 2 in different orgs/locations seems a little odd.

ekohl · 2026-08-19T10:30:00Z

Side question: Since there are multiple proxies now, do we need to ensure that all proxies running on a server are attached to the same orgs/locations. For pulp and container_gateway in particular, having those 2 in different orgs/locations seems a little odd.

That's a good point and I don't think we track that today. Feels like something that would easily break things in subtle ways.

ianballou · 2026-08-19T13:57:27Z

Side question: Since there are multiple proxies now, do we need to ensure that all proxies running on a server are attached to the same orgs/locations. For pulp and container_gateway in particular, having those 2 in different orgs/locations seems a little odd.

Frankly at the moment I've been assuming that admins will need to manage this, at least at first. That is to say, I think we should help with this, but we need to be careful not to over prescribe what taxonomies the smart proxies are connected to.

ekohl · 2026-08-19T14:13:53Z

You're right that admins should, but we can help point them in the right direction in our docs. We're thinking about enablement docs that point out "this is different in a containerized install", which we could point out.

ianballou · 2026-08-19T14:38:01Z

You're right that admins should, but we can help point them in the right direction in our docs. We're thinking about enablement docs that point out "this is different in a containerized install", which we could point out.

I agree that this would be great for an enablement doc, while this smart proxy situation isn't something too new from a technical standpoint, users have built up so much "muscle memory" around having one "internal smart proxy" that this sort of doc should go a long way.

ianballou · 2026-08-19T15:28:48Z

I updated the tests. Will remove the custom container image once I see them pass here.

ianballou · 2026-08-20T17:41:24Z

Ah, right, tests are dead on CP 5 :)

jeremylenz · 2026-08-20T18:08:29Z

@ianballou Should be green after a rebase now!

* Update container gateway tests to match * Update features in yaml to be dictionaries

* Container Gateway is now external * Avoid fqdn fact for foreman_proxy_container_gateway_pulp_endpoint

ianballou · 2026-08-20T21:23:34Z

Tests are green! Removing my custom image.

lzap · 2026-08-21T07:38:34Z

+  ProxyPass /v2/ {{ foreman_proxy_url }}/container_gateway/v2/
+  ProxyPassReverse /v2/ {{ foreman_proxy_url }}/container_gateway/v2/
+  ProxyPass /index/ {{ foreman_proxy_url }}/container_gateway/index/
+  ProxyPassReverse /index/ {{ foreman_proxy_url }}/container_gateway/index/


So /v{1,2}/ is basically claimed by container gateway now and nothing else can use this location prefix? It worked until no because everything else was our APIs we could probably easily customize, but container registry is I guess fixed and we can't do much about it.

I guess it is good enough for now, we can always add more statements here for each and every endpoint in the worst case. I am thinking if virtual hosts would be a cleaner approach. We do not manage DNS tho, starting asking our users to create entries like cw.satellite.example.com could be a pain.

We do not manage DNS tho, starting asking our users to create entries like cw.satellite.example.com could be a pain.

We have discussed this idea for a variety of things in the past and while it's a great idea, we never felt we could enforce this on users and at best it could be a best practice. Since these routes only apply to a content proxy (aka Capsule) there's not been any real risk of clashing.

pr-processor Bot added the Not yet reviewed label Jul 1, 2026

ianballou force-pushed the container-gateway branch from f17d982 to fe6f89e Compare July 2, 2026 19:10

ianballou marked this pull request as ready for review July 9, 2026 16:02

ianballou changed the title ~~Experiment: Container Gateway support~~ Container Gateway support Jul 13, 2026

ianballou force-pushed the container-gateway branch from fe6f89e to 96d5351 Compare July 15, 2026 14:16

ianballou commented Jul 15, 2026

View reviewed changes

Comment thread src/roles/httpd/defaults/main.yml Outdated

pr-processor Bot removed the Not yet reviewed label Jul 15, 2026

ehelms reviewed Jul 15, 2026

View reviewed changes

Comment thread src/roles/httpd/templates/foreman-ssl-vhost.conf.j2

ehelms reviewed Jul 15, 2026

View reviewed changes

Comment thread src/roles/foreman_proxy/templates/settings.d/container_gateway.yml.j2 Outdated

ianballou force-pushed the container-gateway branch from 96d5351 to 77b9f9f Compare July 15, 2026 18:49

ianballou commented Jul 15, 2026

View reviewed changes

Comment thread src/roles/foreman_proxy/templates/settings.d/container_gateway.yml.j2

ianballou force-pushed the container-gateway branch from 77b9f9f to 2e2f69e Compare July 16, 2026 19:07

ehelms reviewed Jul 16, 2026

View reviewed changes

Comment thread src/vars/base.yaml Outdated

sjha4 mentioned this pull request Jul 21, 2026

Support offline restore on proxy deployments #684

Merged

2 tasks

ianballou force-pushed the container-gateway branch from 2e2f69e to f1b98dd Compare July 23, 2026 14:22

ehelms reviewed Jul 23, 2026

View reviewed changes

Comment thread src/vars/database.yml Outdated

ianballou force-pushed the container-gateway branch 2 times, most recently from ce04a83 to fa6d7ed Compare July 24, 2026 15:47

ianballou force-pushed the container-gateway branch 2 times, most recently from bc06478 to a1a7264 Compare August 14, 2026 20:49

ehelms reviewed Aug 17, 2026

View reviewed changes

Comment thread src/features.yaml

ekohl reviewed Aug 18, 2026

View reviewed changes

ianballou force-pushed the container-gateway branch from a1a7264 to 3f58895 Compare August 19, 2026 15:26

ianballou requested a review from ekohl August 19, 2026 15:29

ianballou added 6 commits August 20, 2026 14:13

First pass at container gateway support

903138b

Rearrange container gateway variables

cae9be3

Require container gateway on content/container but not foreman

175d9c1

* Update container gateway tests to match * Update features in yaml to be dictionaries

Revert unless feature for container gateway e6cf94

f75d6c4

Container Gateway conflicts with foreman

5d9987f

* Container Gateway is now external * Avoid fqdn fact for foreman_proxy_container_gateway_pulp_endpoint

Move container gateway tests to its feature dir

77ad615

ianballou force-pushed the container-gateway branch from 3f58895 to eaef4ad Compare August 20, 2026 18:14

ianballou force-pushed the container-gateway branch from eaef4ad to 77ad615 Compare August 20, 2026 21:24

lzap reviewed Aug 21, 2026

View reviewed changes

ehelms merged commit e49a4fc into theforeman:master Aug 21, 2026
32 of 34 checks passed

ianballou deleted the container-gateway branch August 21, 2026 15:49

pablomh mentioned this pull request Aug 28, 2026

Fixes #449 - Add container-gateway to capsule flavor features #792

Merged

	result = server.run(f"podman login {server_fqdn} --username admin --password changeme --tls-verify=false")
	result = server.run("podman login %s --username admin --password changeme --tls-verify=false", server_fqdn)

Uh oh!

Conversation

ianballou commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why are you introducing these changes? (Problem description, related links)

What are the changes introduced in this pull request?

How to test this pull request

Checklist

Uh oh!

ehelms commented Jul 2, 2026

Uh oh!

ianballou commented Jul 2, 2026

Uh oh!

ianballou commented Jul 9, 2026

Uh oh!

ianballou commented Jul 15, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ianballou Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ianballou Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ehelms Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ianballou commented Jul 23, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sjha4 commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ianballou commented Jul 23, 2026

Uh oh!

ianballou commented Jul 24, 2026

Uh oh!

ehelms commented Jul 24, 2026

Uh oh!

ianballou commented Jul 24, 2026

Uh oh!

ianballou commented Aug 17, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

ianballou commented Jul 1, 2026 •

edited

Loading

ianballou Jul 15, 2026 •

edited

Loading

ianballou Jul 15, 2026 •

edited

Loading

ehelms Jul 21, 2026 •

edited

Loading

sjha4 commented Jul 23, 2026 •

edited

Loading

ianballou commented Aug 19, 2026 •

edited

Loading