fix(data_classes): invalidate CogniteResourceList lookups on mutation - #2774
fix(data_classes): invalidate CogniteResourceList lookups on mutation#2774jaideeppyne wants to merge 3 commits into
Conversation
`CogniteResourceList.get(id=.../external_id=.../instance_id=...)` reads three `@cached_property` lookup maps (`_id_to_item`, `_external_id_to_item`, `_instance_id_to_item`) that are built lazily on the first `get(...)`. Only `extend()` kept them in sync — every other membership-mutating operation (`append`, `insert`, `remove`, `pop`, `clear`, `__setitem__`, `__delitem__`, `__iadd__`, `__imul__`) was inherited from `UserList` and mutated `self.data` without invalidating the caches. So after any such mutation `get()` silently returned a removed item, or `None` for an item that is present. Add `_clear_identifier_lookups()` (drops the cached maps from `__dict__`) and override every mutator to call it after delegating to `super()`. Closes the gap noted by the existing `# TODO: We inherit a lot from UserList ...` comment. Adds a regression test that primes the lookup cache, then exercises each mutator and asserts `get()` reflects the new membership. It fails on the current code (`get(id=...)` returns a popped item) and passes with the fix. Fixes cognitedata#1188
There was a problem hiding this comment.
Code Review
This pull request ensures that cached identifier lookup maps in CogniteResourceList are properly invalidated upon mutation by overriding list-mutating methods and adding corresponding unit tests. The review feedback recommends delegating __iadd__ to extend to prevent bypassing duplicate checks, adding a test assertion for this duplicate validation, and replacing Any type hints with stronger types in __setitem__ and __delitem__ to comply with the repository's style guide.
…up check - __iadd__ now delegates to extend() so += enforces the same duplicate-id check (and cache maintenance) instead of bypassing it via UserList. - __setitem__/__delitem__ use SupportsIndex | slice (and a precise item type) instead of Any, per the style guide. - Add a test asserting += raises ValueError on duplicate ids.
|
Thanks for the review — addressed all three in 3d5f563:
mypy/ruff clean and the mutation test still passes. |
warn_unused_ignores is enabled in mypy.ini, so the extra [assignment] code in the type: ignore comment triggers [unused-ignore] and fails the mypy CI check. Only [index] is required.
|
Small follow-up in 423372e: dropped an unused |
Description
Fixes #1188.
CogniteResourceListbuilds three identifier→item lookup maps as@cached_property—_id_to_item,_external_id_to_item,_instance_id_to_item— which.get(id=.../external_id=.../instance_id=...)reads. The maps are built lazily on the first.get(...)call.Only
extend()was written to keep those cached maps in sync. Every other membership-mutating operation —append,insert,remove,pop,clear,__setitem__,__delitem__,__iadd__(+=),__imul__(*=) — is inherited straight fromUserListand mutatesself.datawithout invalidating the caches. So once a map is cached:get(id=X)afterpop/remove/delreturns the removed item (should beNone);get(id=X)afterappend/insert/+=returnsNonefor an item that is present.This is exactly the gap the existing
# TODO: We inherit a lot from UserList that we don't actually support...comment flags.Fix
Add
_clear_identifier_lookups()— which drops the threecached_propertyentries fromself.__dict__so they rebuild on next access — and override each mutating method to call it after delegating tosuper(). The invalidate-then-rebuild approach is simpler and less error-prone than hand-patching the dicts.Test
test_get_is_invalidated_by_mutationsprimes the lookup cache with aget(), then runs each mutator (pop/remove/append/insert/+=/__setitem__/del/clear) and assertsget()reflects the new membership (removed →None, added → found). It fails on the current code (get(id=2)returns the popped item) and passes with the fix.I ran the
TestCogniteResourceListsuite: 22 passed. The one remaining failure (test_to_pandas) is a pre-existing pandas-version dtype mismatch unrelated to this change (tracked separately).ruffcheck/format clean;mypyclean on the changed file.