fix: don't fabricate a Ready condition for generic objects in metrics#212
Open
DerekFrank wants to merge 1 commit into
Open
fix: don't fabricate a Ready condition for generic objects in metrics#212DerekFrank wants to merge 1 commit into
DerekFrank wants to merge 1 commit into
Conversation
GenericObjectController wraps objects in an UnstructuredAdapter whose
StatusConditions() called NewReadyConditions(...).For(), and For()
initializes any missing root/dependent condition to Unknown. For an object
that does not define a Ready condition, this fabricated a Ready=Unknown that
was then emitted as a permanent operator_status_condition_*{type="Ready",
status="Unknown"} metric, even though nothing is persisted to the object.
Add a WithObservedOnly ForOption that builds the ConditionSet from the
conditions already present without initializing absent ones, and use it from
the UnstructuredAdapter (a read-only reflection used only for metrics). Owned
objects driving reconcile are unaffected.
DerekFrank
force-pushed
the
karpenter-1.14
branch
from
613dbbc to
2935f08
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
GenericObjectControllerwraps objects in anUnstructuredAdapterwhoseStatusConditions()callsNewReadyConditions(...).For(u).For()initializes any missing root/dependent condition type toUnknown. For an object that does not define aReadycondition, this fabricates aReady=Unknownin the adapter's in-memory condition list, which the reconcile gauge loop then emits as a permanent metric:Nothing is ever persisted to the object (the adapter is a throwaway unstructured copy), and the synthetic
Readynever rolls up, so it's stuck atUnknownforever — misleading noise for any object whose API doesn't use aReadyroot condition.Fix
Add a
WithObservedOnly()ForOptionthat builds theConditionSetfrom the conditions already present on the object, skipping initialization of absent root/dependent conditions toUnknown. TheUnstructuredAdapter— a read-only reflection used only for metrics on objects the controller does not own — uses it, so it reports metrics for the conditions that actually exist and no longer fabricates aReady.Objects that own their status and drive reconcile (via
NewController[T]with a typedstatus.Object) are unaffected:For()still initializes their declared conditions toUnknownas before.Testing
Added a generic-controller test asserting no
Readymetric is emitted for an object that defines only non-Readyconditions. Verified it fails before the fix (phantomReady=Unknown) and passes after. Fullstatussuite passes.Motivation
Karpenter is adding
operator_status_condition_*metrics for theCapacityBufferCRD (whose conditions areReadyForProvisioning/Provisioning/LimitedByQuotas, with noReady) viaGenericObjectController, and was seeing a permanent phantomReady=Unknownseries.