Fix RCSafetyModule offline learning guard to actually raise - #134
Open
rootkiller6788 wants to merge 1 commit into
Open
Fix RCSafetyModule offline learning guard to actually raise#134rootkiller6788 wants to merge 1 commit into
rootkiller6788 wants to merge 1 commit into
Conversation
learn_batch constructed an AssertionError without raising it, so offline safety learning silently did nothing. The signature also mismatched the base SafetyModule.learn_batch(batch) and the agent call site, raising a confusing TypeError instead. Align the signature to (self, batch) and raise the intended informative error. Add a regression test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
RCSafetyModuleCostCriticContinuousAction.learn_batchis meant to reject offline safety learning with a clear error, but theAssertionError(...)was never raised:The bare expression constructs the exception and discards it. In addition, the signature
(self, batch, policy_learner)mismatches both the base classSafetyModule.learn_batch(self, batch)and the only call sitePearlAgent.learn_batch(self.safety_module.learn_batch(batch)), so a user hitting the offline path gets aTypeError: learn_batch() missing 1 required positional argument: 'policy_learner'instead of the intended message.Why
Offline safety learning is documented as unsupported, so the method should fail fast with an informative error rather than silently no-op (or crash with an unrelated
TypeError).How
(self, batch)(matches the base class,IdentitySafetyModule,RiskSensitiveSafetyModules, and the agent call site).raisetheAssertionError, and drop the now-resolved# pyre-fixme[14].test/unit/with_pytorch/test_safety_module.pyassertinglearn_batchraises.Verified: new test fails before the change (TypeError) and passes after;
test_agent.py(agent construction) suite passes with the change.