Implement descriptor-first writer results and refresh architecture docs - #129
Open
brendan-m-murphy wants to merge 4 commits into
Open
Implement descriptor-first writer results and refresh architecture docs#129brendan-m-murphy wants to merge 4 commits into
brendan-m-murphy wants to merge 4 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a descriptor-first artifact materialization flow where operation-scoped materializers return structured ArtifactWriteResult objects, which are merged deterministically into the planned data artifact descriptor and persisted on CatalogRecord.artifacts. It also updates bundled stdlib I/O examples and refreshes architecture/design documentation to align terminology (materializers vs writer capabilities) and the new result model.
Changes:
- Add
ArtifactWriteRequest/ArtifactWriteResultand update add-operation execution to materialize viamaterializer.write(request)and merge returned descriptor facts + auxiliary artifacts. - Update
ogcat.writershelpers and bundled stdlib I/O writer capabilities to returnArtifactWriteResult, and shift converters toward composable runtime-value transforms. - Refresh docs (tutorials, ADR/design notes, API docs, architecture report, glossary) to reflect the updated artifact model and terminology.
Reviewed changes
Copilot reviewed 35 out of 35 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_writers.py | Adds/updates tests for function adapters, structured results, merge behavior, rollback, and terminology updates. |
| tests/test_storage.py | Updates storage planning tests for materializer intent and request/result-based write surface. |
| tests/test_stdlib_io_capabilities.py | Updates bundled capability examples to consume ArtifactWriteResult and adds composition tests. |
| tests/test_hooks.py | Updates hook-related writer tests to use ArtifactWriteRequest/ArtifactWriteResult. |
| tests/test_add_operation_lifecycle.py | Updates lifecycle tests for materializer intent and request/result writer flow. |
| src/ogcat/writers.py | Refactors helper “writers” into operation materializer adapters returning ArtifactWriteResult. |
| src/ogcat/transactions.py | Updates transaction docs to refer to materializers rather than writers. |
| src/ogcat/storage.py | Updates storage module docs to reference ArtifactMaterializer. |
| src/ogcat/storage_planning.py | Updates planning glue to use MaterializationIntent(materializer=None, ...). |
| src/ogcat/operation_runner.py | Implements planned data descriptor creation, materializer invocation, result merge, and audit enrichment. |
| src/ogcat/models.py | Introduces ArtifactWriteResult model and wiring for record artifact descriptor coercion. |
| src/ogcat/materialization.py | Renames writer intent/validation helpers to materializer equivalents. |
| src/ogcat/hooks.py | Adds ArtifactMaterializer protocol and ArtifactWriteRequest, updates OperationSource. |
| src/ogcat/catalog.py | Updates public add_artifact typing/docs to accept operation materializers (via artifact_writer arg). |
| src/ogcat/catalog_application.py | Updates orchestration to build MaterializationIntent using materializer_intent(...). |
| src/ogcat/capabilities.py | Updates docs to distinguish registry writer capabilities from operation materializers. |
| src/ogcat/bundled_plugins/stdlib_io.py | Updates bundled writer capabilities to return ArtifactWriteResult; converters become runtime-only. |
| src/ogcat/init.py | Exposes new request/result types and materializer protocol at package top-level. |
| docs/tutorials/intermediate.md | Renames tutorial section to “Operation materializers” and updates terminology. |
| docs/tutorials/artifact-workflows.md | Updates examples to implement write(request) -> ArtifactWriteResult. |
| docs/ogcat_long_term_plan.md | Updates glossary/plan terms to include ArtifactMaterializer and writer capability distinction. |
| docs/glossary.md | Adds/updates glossary entries for operation materializers vs writer capabilities. |
| docs/design-note-virtual-artifact-filesystem-research.md | Updates research note terminology to match materializer/capability split. |
| docs/design-note-hooks-plugins.md | Updates plugin guidance for materializers and structured results. |
| docs/design-note-capability-registry.md | Clarifies capability vs materializer roles and adds composition narrative. |
| docs/design-note-artifact-descriptors.md | Documents writer-result merge semantics in the descriptor model. |
| docs/design-note-artifact-claims-and-facets.md | Documents writer-produced facts and merge semantics + future dispatch direction. |
| docs/current-architecture-report.md | Adds a new dated snapshot reflecting the descriptor-first write flow and result merge. |
| docs/concepts/locators-and-storage.md | Updates conceptual docs to refer to operation materializers. |
| docs/concepts/catalog-records.md | Updates docs to reference writer capabilities (not operation writers) in dispatch vocabulary. |
| docs/api/writers-transactions.rst | Updates API docs for materializers, request/result types, and helper adapters. |
| docs/api/models.rst | Adds ArtifactWriteResult to the public models API docs. |
| docs/api/hooks.rst | Adds ArtifactWriteRequest and ArtifactMaterializer to hook-related API docs. |
| docs/api/capabilities.rst | Updates capability docs to describe writer capabilities returning ArtifactWriteResult. |
| docs/adr/0002-virtual-artifact-filesystem-domain-model.md | Updates ADR terminology and domain model to include writer capabilities vs operation materializers. |
Comments suppressed due to low confidence (1)
tests/test_writers.py:579
- This test now uses "materializer" terminology, but it still asserts the old error text "requires an artifact_writer". Either keep terminology consistent by updating the expected message/regex, or explicitly document that the public argument name (and error wording) remains
artifact_writerfor compatibility.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if materializer is None: | ||
| raise ValueError( | ||
| f"Storage plan with write mode {add_plan.storage_plan.write_mode!r} " | ||
| "requires an artifact_writer." |
| hooks and operation materializers see the same `OperationContext`, hook ordering, rollback behavior, and | ||
| audit events whether the operation started from `add_file()` or `add_artifact()`. The runner also | ||
| owns the single result merge path: it builds a planned `data` descriptor, passes it to the writer, | ||
| owns the single result merge path: it builds a planned `data` descriptor, passes it to the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ArtifactWriteResultsupport and a descriptor-first write request/result flow.Testing