attempt to ensure metacontroller finalizers always run - #3598
Conversation
- declare finalizers on each resource that has a finalize hook at creation time (crawljob, collindex, background-job) - set ttlSecondsAfterFinished to 30 instead of 0 on background-job and create-replica background-job - should fix #3365 The ttlSecondsAfterFinished set to 0 is likely the main culprit for finalizers being skipped for these resources, as the job is cleaned up right away. However, to guarantee the finalizers are always run, they are also declared as part of the object metadata at creation time for the resources that have these hooks.
|
Set the ttl lower for replica and other background jobs, as some code paths (like collection update) does wait for the jobs to finish, in tests. Perhaps keeping ttl at 0 or 1 does make sense, as long as the finalizer is set on create, as the whole point is to respond quickly as soon as the job is finished |
tw4l
left a comment
There was a problem hiding this comment.
I'm not convinced ttlSecondsAfterFinished is playing a part here (though I'm also fine with the small changes to those values here, just haven't seen evidence that plays into the bug), but explicitly setting the finalizers seems like a good step.
One suggestion where I think we need another finalizer added, and will need to do the same in #3577.
I think the finalizers are the real fix, so maybe no need to change the ttl, though it probably makes it more likely to happen. Removed the ttl changes for now so we can check if the finalizers definitively fix this. |
Looking into this issue more, it seems the ttlSecondsAfterFinished set to 0 is likely the main culprit for finalizers being skipped for these resources, as the job is cleaned up right away. However, to guarantee the finalizers are always run, they should also declared as part of the object metadata at creation time for the resources that have these hooks.
This PR:
Should fix #3365