feat(integrations): Add Integrations API client - #2813
Draft
vikramlc-cognite wants to merge 6 commits into
Draft
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #2813 +/- ##
==========================================
+ Coverage 93.15% 93.23% +0.07%
==========================================
Files 516 539 +23
Lines 53665 54628 +963
==========================================
+ Hits 49993 50932 +939
- Misses 3672 3696 +24
🚀 New features to boost your workflow:
|
Author
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces the new Integrations API to the Cognite Python SDK, including asynchronous and synchronous clients, data classes, and comprehensive unit tests. The review feedback focuses on improving robustness and type safety by safely handling potential null values in JSON deserialization and adding client-side validation to raise ValueErrors early for invalid API parameter combinations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
client.integrationsmodule (Integrations, Task history/sync, Errors, Config revisions, Actions) so SDK consumers can manage the new alpha/public-preview Integrations API, modeled on the existingsimulatorsmodule.Type of change
What changed
cognite/client/data_classes/integrations/package:Integration/IntegrationWrite/IntegrationUpdate(+ lists),Action/ActionWrite(the new Actions sub-resource),ConfigRevision/ConfigRevisionWrite/ConfigRevisionMetadata,TaskHistory/SyncResult,IntegrationError.cognite/client/_api/integrations/package:IntegrationsAPI(list/create/retrieve/update/delete) composing four sub-APIs —.tasks(task history + incremental/sync),.errors,.config(versioned config revisions),.actions(create/list/retrieve/cancel).cognite/client/_sync_api/integrations/generated via the repo's sync codegen script — not hand-written.client.integrationsinto bothAsyncCogniteClientand the syncCogniteClient, pluscognite/client/testing.py'sCogniteClientMock/AsyncCogniteClientMock.cognite/client/utils/_url.py: registeredintegrations,integrations/config,integrations/actions, andintegrations/actions/cancelas non-idempotent POST paths (create/config-create/action-create/cancel aren't safe to auto-retry); added matching cases toTestRetryableEndpoints.client.integrations*call sends the mandatorycdf-version: alphaheader and raises the SDK's alphaFeaturePreviewWarning.startup,checkin, andextpipes/runs— all three are markedifdef: internalin the service contract (extractor-only protocol / legacy back-compat, already covered byExtractionPipelinesAPI).tests/tests_unit/test_api/test_integrations/(26 tests, mocked HTTP) andtests/tests_unit/test_data_classes/test_integrations.py(load/dump round-trips, incl. all threeIntegrationUpdatepatch shapes).Why it changed
cognitedata/service-contractsPR #3378 (wire schema + public/internal marking); docs.cognite.com Integrations concept pages (20230101-alpha, "public preview");odinrepo (ground-truth route/DTO source, confirms mandatorycdf-version: alphaheader)What to focus on during review
ifdef: alphain the contract (not GA likesimulators), so it follows theLimitsAPIpattern (api_maturity="alpha"+self._alpha_version_header()on every call) rather than the GAsimulatorspattern — worth double-checking this is the right precedent.idin this API, unlikesimulators) —ExternalIDTransformerMixinused throughout instead ofIdTransformerMixin.IntegrationConfigAPI.list()hits a GET endpoint that acceptscursor/limitbut its response has nonextCursor— implemented as a single call with a docstring note, not_list/_list_generator.IntegrationActionsAPI.create()/.cancel()are hand-rolled_postcalls (not the generic_create_multiple/_delete_multiplehelpers) because the integration's external ID is a query param, not part of the item body — includes manual chunking (20/100 items) since the generic helpers' automatic chunking isn't available here._url.py— I marked create/config-create/action-create/action-cancel as non-retryable; update/retrieve/list/config-retrieve were left retryable. Worth a sanity check against actual server semantics.cognite/client/_sync_api/integrations/and_sync_cognite_client.pyare generated — please don't review them line-by-line, just confirmscripts/sync_client_codegen/main.py verifypasses (it does).Test evidence
pytest tests/tests_unit/test_api/test_integrations/ tests/tests_unit/test_data_classes/test_integrations.py -v→ 26 passed.pytest tests/tests_unit/(withpandas/numpyextras installed) → 6664 passed, 8 pre-existing failures unrelated to this change (missinggeopandas/sympy), 0 regressions (verified viagit stashA/B comparison against the base branch).mypyclean across all new/changed files (_api/integrations,data_classes/integrations,_cognite_client.py,_sync_cognite_client.py,_sync_api/integrations,testing.py,utils/_url.py).ruff check/ruff formatclean.python scripts/sync_client_codegen/main.py verify→ up to date.CogniteClientand confirmedclient.integrations,.tasks,.errors,.config,.actionsall resolve to the expected sync API classes.Risks and unknowns
TaskHistoryincludeswarning_count/fatal_count/sources/targetsfields observed in theodinservice's actual DTOs but not fully detailed in the service-contracts snapshot reviewed — flagging in case the contract is later tightened.data_classesexport follows thesimulatorsprecedent of not re-exporting fromcognite/client/data_classes/__init__.py(submodule-only import) — intentional, but worth confirming this is still the desired convention for alpha features.Rollout and rollback
FeaturePreviewWarning/global_config.silence_feature_preview_warningsmechanism.Checklist