(PE-45827) Prefer the ssl status endpoint for PuppetDB liveness validation

[jonathannewman]

check_puppetdb_status_endpoint unconditionally curls PuppetDB's cleartext status port (8080) and hard-fails after 60s if it never sees a valid version string in the response. PuppetDB's cleartext listener is disabled by default on fresh installs as of PE-45384/PE-44906 (SECVULN-1792), so on an affected host this always spins the full timeout and fails with PuppetDB took too long to start — even though sleep_until_puppetdb_started, called immediately before this in generic_install, has already confirmed liveness via the ssl status endpoint (fixed for the same root cause in beaker-puppet by PE-45697).

Discovered while live re-verifying PE-45697's fix: puppet-enterprise-modules frankenbuild PR #2453, build enterprise_puppet-enterprise-modules_init-multijob_frankenbuilder-main-main #197 — the monolithic-upgrade smoke job's pre_suite :: upgrade.rb step failed with PuppetDB took too long to start, tracing to this method, right after sleep_until_puppetdb_started itself succeeded.

Is this just redundant with sleep_until_puppetdb_started? No. Traced check_puppetdb_status_endpoint's origin to commit 7112971 ("(PE-14934) Add more robust puppetdb check", 2016-04-04). sleep_until_puppetdb_started already existed at that point; this check was added deliberately on top of it, per the commit message, to guard a different race — the port accepting a connection before PuppetDB is actually ready to serve valid data. That's still a real gap today: sleep_until_puppetdb_started's ssl leg only checks curl's exit code, never the response body. So the fix here isn't to remove this check — it's to make the response-content validation itself ssl-aware.

Fix: prefer the ssl status endpoint over the cleartext one entirely, rather than just detecting "nothing's listening" and skipping validation. Traced through PuppetDB's actual config to confirm this gets a real, content-validated response — not just a TLS-handshake-level signal:

• puppet_enterprise::puppetdb::jetty_ini.pp explicitly sets client-auth = want for PuppetDB's jetty listener, overriding trapperkeeper-webserver-jetty9's own :need default — so a TLS handshake with no client cert still succeeds.
• PuppetDB's own auth.conf marks /status/v1/services (and /pdb/meta/v1/version alongside the rest of /status/v1/*) allow-unauthenticated: true.

Together, curl -k gets a real 200 with real content — no client cert needed — so this is the one path that still works once the cleartext listener is disabled by default. Falls back to the cleartext port only if ssl doesn't return valid content, preserving behavior for any older or nonstandard configuration (and still respecting the original version_is_less(pe_ver, '2016.1.0') skip).

Testing:

• 4 examples: version-gate skip, ssl-only success (never falls back to cleartext), ssl-fails-falls-back-to-cleartext, and the original timeout/fail_test path (mocked via Timeout.timeout) still works.
• Full suite: 305 examples, 0 failures.
• Not live-reverified against a real frankenbuild — attempted twice via pe_acceptance_tests_sha=origin/pr/<N>/head on enterprise_puppet-enterprise-modules_init-multijob_frankenbuilder-main-main, both times inconclusive: traced through the actual job config (ci-job-configs/resources/job-templates/integration-frankenbuilder.yaml + .sh) and confirmed that parameter is never wired into the downstream smoke-monolithic-upgrade-module-pr job's actual frankenbuilder invocation — it silently ran against pe_acceptance_tests main with the stock released beaker-pe gem both times, not this branch. Confidence here rests on the unit tests plus the config-file evidence above (jetty_ini.pp, auth.conf), not a live end-to-end run. A follow-up to properly wire --pe_acceptance_tests_pr/--pe_acceptance_tests_branch into that job (or find another path to invoke frankenbuilder directly) would be needed for a real live reverify.

PE-45827: https://perforce.atlassian.net/browse/PE-45827

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com