From f98dd467807b80ca181ceb044bff044158bbb87d Mon Sep 17 00:00:00 2001 From: MicrowaveDev Date: Thu, 30 Jul 2026 20:32:07 +0100 Subject: [PATCH 1/4] docs(chat): review conversation and publishing data models --- .../docs/conversation-data-model-review.md | 255 ++++++++++++++++++ app/modules/chat/docs/overview.md | 3 + docs/agent-map.md | 2 + docs/modules.md | 5 +- docs/todo.md | 21 +- 5 files changed, 277 insertions(+), 9 deletions(-) create mode 100644 app/modules/chat/docs/conversation-data-model-review.md diff --git a/app/modules/chat/docs/conversation-data-model-review.md b/app/modules/chat/docs/conversation-data-model-review.md new file mode 100644 index 00000000..63d29388 --- /dev/null +++ b/app/modules/chat/docs/conversation-data-model-review.md @@ -0,0 +1,255 @@ +# Chat Conversation Data Model Review + +## Question + +Should a private chat be represented as a private GeeSome `Group`, with each +message represented as a `Post` and each attachment connected through +`PostsContents`? + +## Decision + +Treat direct and multi-member chat as the same **conversation** concept, but do +not reuse the existing social `Group`, `Post`, or `PostsContents` tables. + +A direct chat is conceptually a private conversation with two account members. +A multi-member chat is the same conversation type with more members and +additional membership state. Both should continue to use the append-only +`ChatEvent` log and `ChatEventAttachment` references. + +Before MLS group chat is introduced, add an explicit conversation aggregate +around the existing event log: + +```text +ChatConversation + |-- ChatConversationMember + |-- ChatConversationHead + |-- ChatEvent + | |-- ChatEventRecipient + | `-- ChatEventAttachment + `-- protocol-specific device membership state +``` + +This is a domain separation decision, not a requirement to duplicate storage, +pagination, or rendering helpers. + +## Findings From The Current Implementation + +Social publishing currently uses: + +```text +Group -> Post -> PostsContents -> Content +``` + +`PostsContents` is a many-to-many join with visible `position` and `view` +values. A post is a mutable publishing entity with status, group-local +identity, replies, reposts, counters, manifests, and generated projections. + +Encrypted chat currently uses: + +```text +conversationId -> ChatConversationHead +conversationId -> ChatEvent -> ChatEventRecipient + `-> ChatEventAttachment +``` + +`conversationId` is presently a stable identifier rather than a first-class +database entity. `ChatEvent` is an ordered, signed, append-only encrypted +envelope with durable retry, acknowledgement, and missing-range repair. + +`ChatEventAttachment` stores `chatEventId`, `storageId`, and an optional +`contentId`: + +- A sender node records the sender-owned ciphertext `Content.id`. +- A recipient node pins the ciphertext CID and records `contentId = null`. +- The recipient pins every attachment before it commits the event and + acknowledges delivery. +- A failed fetch leaves delivery pending and creates no recipient event. +- Attachment order, filename, media type, original size, and content key stay + inside the encrypted browser payload. +- Per-user release intent is stored separately in + `ChatEventAttachmentRetention`. + +These behaviors are now covered across PostgreSQL-backed restart tests and +independent GeeSome processes using separate Kubo nodes. + +## Why Social Groups And Posts Should Not Be Reused + +### Different lifecycle + +Posts are mutable publishing records. Chat events are signed append-only +records. Editing or deleting a chat message must be represented by a new event +or explicit local retention state rather than silently rewriting the signed +event. + +### Different delivery contract + +Publishing a post updates database projections, group counters, manifests, and +possibly external integrations. Sending a chat event requires recipient +routing, acknowledgements, retries, missing-range repair, and ordered +source-head comparison. + +### Different membership level + +Social group membership is primarily account and permission based. Encrypted +chat must also account for browser devices, revoked devices, recipient keys, +and future MLS epochs. A user can remain a conversation member while one of +that user's devices is removed. + +### Different metadata boundary + +Post content names, views, and ordering are ordinary server-readable metadata. +Chat attachment names, media types, display order, and keys are intentionally +inside the encrypted envelope. Reusing `PostsContents` would either expose that +metadata or create misleading empty join fields. + +### Different remote representation + +A remote chat recipient can pin ciphertext without owning a normal `Content` +row. `PostsContents` requires a `Content` entity, while +`ChatEventAttachment.contentId` is deliberately nullable on recipient nodes. + +### Unwanted product coupling + +Reusing `Group` and `Post` would make chat changes interact with unrelated +publishing behavior: + +- group manifests and chunked post indexes; +- static-site generation and RSS; +- ActivityPub and Bluesky publication; +- post status, counters, replies, and reposts; +- moderation and feed queries; +- social-import identity and derived-state jobs. + +Preventing every one of those paths from treating chat messages as publishable +posts would be more fragile than maintaining the smaller chat model. + +## Recommended Conversation Aggregate + +### `ChatConversation` + +The first-class conversation row should own only server-required state: + +- stable `conversationId`; +- conversation kind (`direct` or `group`); +- protocol/capability version; +- lifecycle state; +- creation and update timestamps. + +User-visible title, description, avatar, and other private presentation fields +should stay in browser-encrypted state unless a specific public or +operator-visible field is intentionally designed. + +The existing `ChatConversationHead` can become an association of this aggregate +without changing sequence semantics or rewriting existing events. + +### `ChatConversationMember` + +Membership should identify account owners independently from devices: + +- `conversationId`; +- stable account owner ID; +- nullable local `userId`; +- role and active/removed state; +- accepted membership sequence or epoch where needed. + +For direct conversations, policy should enforce the intended two-account +membership. Whether one account pair can have multiple conversations is a +product decision and should be explicit rather than inferred from table shape. + +### Device membership + +Keep global public device bundles in `ChatDevice`. Direct-message events can +continue to record concrete recipients in `ChatEventRecipient`. + +MLS-specific leaf, Welcome, proposal, commit, and epoch state should use +protocol-specific conversation records after a maintained browser +implementation passes the required behavior tests. Do not add placeholder MLS +columns to the social group model. + +### Messages and attachments + +Keep messages as `ChatEvent`, not `Post`. Keep attachment routing and retention +in `ChatEventAttachment` and `ChatEventAttachmentRetention`. + +Sender-owned ciphertext should continue to reuse `Content`, `StorageObject`, +IPFS pinning, and reference-safe cleanup. Recipient copies should not require +fabricated user-owned `Content` records. + +## Infrastructure That Should Be Shared + +Separate domain models do not require duplicate infrastructure. Chat should +continue to reuse: + +- `Content` and `StorageObject` identity for sender-owned ciphertext; +- storage reference counting and deletion safety; +- IPFS fetch, pin, and bounded size checks; +- cursor and keyset pagination helpers; +- asynchronous queue and worker lifecycle helpers; +- canonical rich-text parsing and rendering inside the browser after + decryption; +- common user, role, and permission vocabulary where the semantics match. + +If repeated relation behavior emerges across posts, chat, generated outputs, +and other entities, extract a small generic helper or storage-reference +contract. Do not make `PostsContents` itself generic after the fact. + +## Product Interactions + +A post can be shared into a chat by sending a typed encrypted reference to its +public identity or by attaching a browser-encrypted private copy. This does not +turn the post into the chat message or make the conversation a social group. + +Similarly, a conversation may offer a user action to publish selected content +as a post. That action should create an explicit post through the normal group +publishing flow. + +Private encrypted group feeds may eventually combine durable posts with +conversation-like membership. They should be reviewed as a separate product +mode rather than introduced implicitly by storing chat events in `Post`. + +## Adoption Plan + +1. Add `ChatConversation` and `ChatConversationMember` as additive model-sync + tables while this work remains unreleased on `dev`. +2. Lazily materialize a conversation row for existing `conversationId` values. + Do not rewrite or resign existing `ChatEvent` envelopes. +3. Keep current direct-message APIs compatible while moving membership and + authorization reads behind conversation helpers. +4. Define explicit direct-conversation uniqueness and invitation policy before + enforcing database constraints. +5. Add bounded member and conversation listing without exposing encrypted + presentation metadata. +6. Integrate MLS group state only after the browser dependency gate passes. +7. Add two-browser/two-node tests covering conversation creation, membership, + device changes, restart, missing-range repair, and attachments. +8. Retire temporary membership inference only after existing conversations + have been materialized and verified. + +## Invariants + +- GeeSome nodes never receive chat plaintext, attachment keys, or browser + private keys. +- `conversationId` remains stable across local database IDs and node replicas. +- Accepted message identity remains `messageId`; retries do not create another + event. +- Conversation sequence remains deterministic and append-only. +- Direct and group membership are explicit and independently testable. +- Removing a device does not silently remove its account from the conversation. +- Removing an account from a group conversation eventually removes all of its + active protocol device memberships. +- Recipient attachment rows can remain storage-only references. +- Social publishing hooks never process chat events unless an explicit publish + action creates a real post. + +## Open Decisions + +- Whether one pair of accounts may create multiple direct conversations. +- Which conversation metadata, if any, should be visible to the node. +- How invitations and member roles map to the first MLS group creation flow. +- How long removed membership and old epoch metadata must be retained. +- Whether encrypted conversation metadata should use a distinguished chat event + or a separately versioned encrypted conversation document. + +These decisions should be resolved before making group conversations available +by default, but they do not require replacing the working direct-message event +log. diff --git a/app/modules/chat/docs/overview.md b/app/modules/chat/docs/overview.md index fd66e8b9..6632751d 100644 --- a/app/modules/chat/docs/overview.md +++ b/app/modules/chat/docs/overview.md @@ -164,3 +164,6 @@ for the transport and delivery analysis behind these boundaries. Group chat uses the separate [MLS protocol decision](../../../../docs/chat-group-e2ee-protocol-decision.md); the node remains an opaque delivery service and does not own MLS private state. +Direct and multi-member chat should use a dedicated conversation aggregate +rather than the social publishing tables; see the +[conversation data model review](./conversation-data-model-review.md). diff --git a/docs/agent-map.md b/docs/agent-map.md index de78fc78..b94ced93 100644 --- a/docs/agent-map.md +++ b/docs/agent-map.md @@ -56,6 +56,8 @@ Useful live endpoints: state, KeyPackages, Welcome messages, or group-chat wire contracts. - Read `app/modules/chat/docs/overview.md` before changing device, envelope, delivery, acknowledgement, or reconciliation contracts. +- Read `app/modules/chat/docs/conversation-data-model-review.md` before changing + conversation, membership, direct-chat, or group-chat persistence. - Load the `browser-first-chat-e2ee` TODO section before chat implementation. - Coordinate protocol/envelope changes through `geesome-libs`, browser/device key handling through `geesome-ui`, and opaque storage/delivery through diff --git a/docs/modules.md b/docs/modules.md index 8ba02417..6b16af12 100644 --- a/docs/modules.md +++ b/docs/modules.md @@ -12,7 +12,8 @@ and implementation notes belong under `app/modules//docs/`. - `asyncOperation`: [overview](../app/modules/asyncOperation/docs/overview.md) - `autoActions`: [overview](../app/modules/autoActions/docs/overview.md) - `bluesky`: [overview](../app/modules/bluesky/docs/overview.md) -- `chat`: [overview](../app/modules/chat/docs/overview.md) +- `chat`: [overview](../app/modules/chat/docs/overview.md), + [conversation data model review](../app/modules/chat/docs/conversation-data-model-review.md) - `communicator`: [overview](../app/modules/communicator/docs/overview.md) - `content`: [overview](../app/modules/content/docs/overview.md) - `database`: [overview](../app/modules/database/docs/overview.md) @@ -50,7 +51,7 @@ and implementation notes belong under `app/modules//docs/`. | `asyncOperation` | Tracks long-running user operations and processes queued background work. | [Overview](../app/modules/asyncOperation/docs/overview.md) | | `autoActions` | Stores and claims scheduled module function calls. | [Overview](../app/modules/autoActions/docs/overview.md) | | `bluesky` | Imports, refreshes, reads local feed views, verifies user-scoped accounts, and cross-posts safe text/image posts for native Bluesky/ATProto. | [Overview](../app/modules/bluesky/docs/overview.md) | -| `chat` | Persists browser-encrypted chat device bundles, opaque events, ordered heads, and receipts without receiving plaintext or private keys. | [Overview](../app/modules/chat/docs/overview.md) | +| `chat` | Persists browser-encrypted chat device bundles, opaque events, ordered heads, and receipts without receiving plaintext or private keys. | [Overview](../app/modules/chat/docs/overview.md), [conversation model review](../app/modules/chat/docs/conversation-data-model-review.md) | | `communicator` | Provides network communication, static-id lookup/binding, and pubsub-style event hooks. | [Overview](../app/modules/communicator/docs/overview.md) | | `content` | Creates, serves, previews, restores, and deletes user content records. | [Overview](../app/modules/content/docs/overview.md) | | `database` | Owns Sequelize setup, models, permissions, API keys, sessions, and shared query helpers. | [Overview](../app/modules/database/docs/overview.md) | diff --git a/docs/todo.md b/docs/todo.md index ac89cb97..36e3d56f 100644 --- a/docs/todo.md +++ b/docs/todo.md @@ -91,6 +91,10 @@ Architecture decision: - Keep PostgreSQL as the operational authorization, head, acknowledgement, and retry index. Consider encrypted IPLD event batches or checkpoints for portable replication only after the operational path is proven. +- Model direct and multi-member chat as dedicated conversations rather than + reusing the social `Group`, `Post`, and `PostsContents` tables. Follow the + [conversation data model review](../app/modules/chat/docs/conversation-data-model-review.md) + before adding group membership persistence. - Use MLS 1.0 for group membership and future-message key rotation according to [Group Chat E2EE Protocol Decision](./chat-group-e2ee-protocol-decision.md). Matrix remains an operational reference rather than the group wire protocol. @@ -135,30 +139,33 @@ MLS group-chat integration checklist: 2. Add a small `geesome-libs` adapter that owns versioned MLS byte encoding, GeeSome device identity binding, application-message framing, and shared cross-package fixtures. Keep package-specific calls out of product modules. -3. Add browser-owned MLS state storage in `geesome-ui`, including atomic state +3. Add the additive `ChatConversation` and `ChatConversationMember` aggregate, + lazily materialize existing direct-conversation IDs without rewriting + events, and move membership/authorization reads behind conversation helpers. +4. Add browser-owned MLS state storage in `geesome-ui`, including atomic state updates, restart recovery, clear-on-logout/device-removal behavior, and an explicit unrecoverable-state screen. GeeSome nodes must receive only opaque protocol values. -4. After the dependency passes, add bounded node storage and delivery contracts +5. After the dependency passes, add bounded node storage and delivery contracts for group metadata, one-time join packages, device-specific Welcome values, proposals, commits, and application events. Reuse the existing durable event log, queue, acknowledgement, and missing-range repair machinery. -5. Implement group creation and device join first. Then add another device, +6. Implement group creation and device join first. Then add another device, remove a device, remove an account's remaining devices, restore a device as a new member, and reconcile database membership with the current MLS epoch as explicit user actions. -6. Serialize membership updates through the canonical group node. Reject stale +7. Serialize membership updates through the canonical group node. Reject stale expected epochs, reload the accepted update, discard interrupted local work, and let the browser rebuild the requested change when it is still allowed. -7. Add group-message and encrypted-attachment UI using the existing chat +8. Add group-message and encrypted-attachment UI using the existing chat conversation surface. Show joining, waiting for an update, retrying, unsupported client, removed device, and unavailable older history states in ordinary user language. -8. Add real two-browser/two-node tests for restart, temporary node +9. Add real two-browser/two-node tests for restart, temporary node unreachability, duplicate and reordered events, interrupted membership updates, simultaneous updates, device removal, attachment delivery, and bounded history repair. -9. Enable the feature only for newly created group conversations behind a +10. Enable the feature only for newly created group conversations behind a capability flag. Keep direct messages unchanged and keep older group chats visibly on their existing mode until an explicit migration flow exists. From c53f15ce8889488e4d91dc7b2d6344eba593071f Mon Sep 17 00:00:00 2001 From: MicrowaveDev Date: Thu, 30 Jul 2026 20:46:41 +0100 Subject: [PATCH 2/4] docs(chat): clarify shared timeline behavior --- .../docs/conversation-data-model-review.md | 164 +++++++++++++++++- 1 file changed, 155 insertions(+), 9 deletions(-) diff --git a/app/modules/chat/docs/conversation-data-model-review.md b/app/modules/chat/docs/conversation-data-model-review.md index 63d29388..4852400c 100644 --- a/app/modules/chat/docs/conversation-data-model-review.md +++ b/app/modules/chat/docs/conversation-data-model-review.md @@ -32,6 +32,31 @@ ChatConversation This is a domain separation decision, not a requirement to duplicate storage, pagination, or rendering helpers. +Posts and chats share more timeline behavior than their current tables expose. +Both can use ordered events, edits represented as later events, missing-item +reconciliation, attachments, and configurable delivery policies. A future +generic timeline/space foundation may sit below both products: + +```text +Timeline / Space + |-- members + |-- ordered events + |-- attachments + |-- reconciliation state + `-- delivery policy + +Social Group + `-- Post projection and publishing behavior + +Private Conversation + `-- ChatEvent projection and device/group-key behavior +``` + +The recommendation is therefore not that posts and chats are fundamentally +unrelated. It is that the current publishing tables already own behavior that +must not be activated implicitly for private chat. Shared primitives should be +extracted deliberately instead of making chat rows masquerade as social posts. + ## Findings From The Current Implementation Social publishing currently uses: @@ -76,10 +101,28 @@ independent GeeSome processes using separate Kubo nodes. ### Different lifecycle -Posts are mutable publishing records. Chat events are signed append-only -records. Editing or deleting a chat message must be represented by a new event -or explicit local retention state rather than silently rewriting the signed -event. +The current `Post` row is a mutable publishing projection. The current +`ChatEvent` row is a signed append-only event. + +Append-only storage does not mean a message cannot be edited. A modern editable +message can be represented as an ordered event history: + +```text +event 1: create message A +event 2: edit message A +event 3: delete message A +``` + +The browser projects those events into the latest visible message. Group posts +could use the same approach: append post-change events while retaining `Post` +as the current feed/manifests projection. GeeSome already records some post +lifecycle events, but they are not yet the complete source of truth for every +post revision. + +The difference is therefore in the current source-of-truth contract, not in +whether users should be allowed to edit. Signed chat events should not be +silently rewritten; edits and deletes should be later events. A future post +event log could follow the same rule. ### Different delivery contract @@ -88,6 +131,19 @@ possibly external integrations. Sending a chat event requires recipient routing, acknowledgements, retries, missing-range repair, and ordered source-head comparison. +Acknowledgements and retries do not need to be mandatory for every timeline. +They can be an explicit delivery policy: + +- `best-effort`: send once without claiming remote delivery; +- `durable`: retain and retry until acknowledged, rejected, or expired; +- `pull-only`: publish a head and let another node fetch missing items; +- `replicated`: require selected nodes to acknowledge storage. + +The UI and API must describe the selected guarantee accurately. A +best-effort send cannot be shown as delivered without an acknowledgement. +Subscribed group-post replication could reuse the durable or replicated modes; +public feeds may prefer pull-only behavior. + ### Different membership level Social group membership is primarily account and permission based. Encrypted @@ -95,6 +151,16 @@ chat must also account for browser devices, revoked devices, recipient keys, and future MLS epochs. A user can remain a conversation member while one of that user's devices is removed. +`ChatEventRecipient` identifies concrete device keys, not only a user account. +For example, Bob may have a phone, laptop, and tablet. The browser encrypts the +message content once and wraps its content key separately for each allowed +device. Removing Bob's tablet excludes that device from future events while Bob +remains a conversation member through his phone and laptop. + +This device-level recipient list is required by the current direct-message +envelope. It can coexist with account-level membership in a generic timeline or +conversation. + ### Different metadata boundary Post content names, views, and ordering are ordinary server-readable metadata. @@ -156,6 +222,26 @@ For direct conversations, policy should enforce the intended two-account membership. Whether one account pair can have multiple conversations is a product decision and should be explicit rather than inferred from table shape. +### Membership and key epochs + +For encrypted multi-member chat, an epoch is the version of the current group +membership and shared group-key state: + +```text +epoch 12: Alice, Bob, Carol +epoch 13: Alice, Bob +``` + +Removing Carol creates a new epoch with new key material. Carol may retain +access to messages she was allowed to read in older epochs, but her removed +devices must not read messages created in epoch 13. Removing only Bob's laptop +changes device membership without necessarily removing Bob's phone or Bob's +account from the conversation. + +Public group posts do not need encrypted-group key epochs. A private encrypted +group feed would need an equivalent membership/version mechanism even if its +visible items were presented as posts. + ### Device membership Keep global public device bundles in `ChatDevice`. Direct-message events can @@ -175,11 +261,51 @@ Sender-owned ciphertext should continue to reuse `Content`, `StorageObject`, IPFS pinning, and reference-safe cleanup. Recipient copies should not require fabricated user-owned `Content` records. +Per-user attachment release means that one local participant no longer wants an +attachment retained in that participant's chat history. It does not immediately +delete shared ciphertext that another participant, pending delivery, or +missing-range repair still needs. + +For example, Alice can release her local attachment view while Bob still keeps +it. Physical cleanup waits until all required local releases, remote delivery +acknowledgements, retention windows, and reference checks allow removal. + +Published post attachments usually follow author/publication retention rather +than one retention row for every unknown reader. Both products can share +reference counting and physical cleanup while keeping different release +policies. + +### Missing-item reconciliation + +Missing-range repair should not be chat-only. A group reader may know from a +group head or manifest that items 1 through 20 exist while the local node has +all except item 18. The node should fetch, verify, persist, and display the +missing post. + +Both products can share a bounded reconciliation algorithm: + +```text +compare local and remote heads +identify missing identities or sequences +fetch bounded pages +verify product-specific rules +persist items +advance the local cursor +``` + +Chat applies recipient, signature, and encrypted-envelope checks. Group posts +apply author, group, manifest, publication, and moderation checks. The chunked +group post index and chat sequence/head contracts are different projections of +the same general recovery requirement. + ## Infrastructure That Should Be Shared Separate domain models do not require duplicate infrastructure. Chat should continue to reuse: +- ordered timeline/change-event helpers where post and chat invariants match; +- head comparison, bounded page repair, and cursor checkpoint helpers; +- configurable best-effort, durable, pull-only, or replicated delivery policy; - `Content` and `StorageObject` identity for sender-owned ciphertext; - storage reference counting and deletion safety; - IPFS fetch, pin, and bounded size checks; @@ -207,6 +333,12 @@ Private encrypted group feeds may eventually combine durable posts with conversation-like membership. They should be reviewed as a separate product mode rather than introduced implicitly by storing chat events in `Post`. +At the product level it is reasonable to describe a direct conversation as a +private group of two users. That language does not require using the current +social `Group` database model. A future `Space` or `Timeline` aggregate could +support both social groups and private conversations while each keeps its own +projection and policy modules. + ## Adoption Plan 1. Add `ChatConversation` and `ChatConversationMember` as additive model-sync @@ -215,14 +347,20 @@ mode rather than introduced implicitly by storing chat events in `Post`. Do not rewrite or resign existing `ChatEvent` envelopes. 3. Keep current direct-message APIs compatible while moving membership and authorization reads behind conversation helpers. -4. Define explicit direct-conversation uniqueness and invitation policy before +4. Extract shared timeline/head/reconciliation helpers only after post and chat + invariants are compared and covered by common behavior tests. +5. Define explicit delivery policies and ensure UI delivery labels match the + selected guarantee. +6. Define explicit direct-conversation uniqueness and invitation policy before enforcing database constraints. -5. Add bounded member and conversation listing without exposing encrypted +7. Add bounded member and conversation listing without exposing encrypted presentation metadata. -6. Integrate MLS group state only after the browser dependency gate passes. -7. Add two-browser/two-node tests covering conversation creation, membership, +8. Add missing-post reconciliation tests using group heads and chunked manifest + indexes without routing public posts through chat delivery rows. +9. Integrate MLS group state only after the browser dependency gate passes. +10. Add two-browser/two-node tests covering conversation creation, membership, device changes, restart, missing-range repair, and attachments. -8. Retire temporary membership inference only after existing conversations +11. Retire temporary membership inference only after existing conversations have been materialized and verified. ## Invariants @@ -240,6 +378,10 @@ mode rather than introduced implicitly by storing chat events in `Post`. - Recipient attachment rows can remain storage-only references. - Social publishing hooks never process chat events unless an explicit publish action creates a real post. +- Message and post edits can use later ordered events while retaining separate + current-state projections. +- Missing-item reconciliation is available to both chat and group timelines, + with product-specific verification. ## Open Decisions @@ -249,6 +391,10 @@ mode rather than introduced implicitly by storing chat events in `Post`. - How long removed membership and old epoch metadata must be retained. - Whether encrypted conversation metadata should use a distinguished chat event or a separately versioned encrypted conversation document. +- Whether a generic `Space`/`Timeline` aggregate should be introduced after + shared behavior has been proven in both chat and group-post tests. +- Which delivery policies should be available for public groups, private group + feeds, and direct conversations. These decisions should be resolved before making group conversations available by default, but they do not require replacing the working direct-message event From aa21bdb52586f9d57c0b904f30d3be9ec26e1fbf Mon Sep 17 00:00:00 2001 From: MicrowaveDev Date: Thu, 30 Jul 2026 20:57:46 +0100 Subject: [PATCH 3/4] docs(chat): model encrypted chat as private groups --- .../docs/conversation-data-model-review.md | 285 +++++++++--------- docs/todo.md | 37 ++- 2 files changed, 166 insertions(+), 156 deletions(-) diff --git a/app/modules/chat/docs/conversation-data-model-review.md b/app/modules/chat/docs/conversation-data-model-review.md index 4852400c..87d7c540 100644 --- a/app/modules/chat/docs/conversation-data-model-review.md +++ b/app/modules/chat/docs/conversation-data-model-review.md @@ -8,29 +8,30 @@ message represented as a `Post` and each attachment connected through ## Decision -Treat direct and multi-member chat as the same **conversation** concept, but do -not reuse the existing social `Group`, `Post`, or `PostsContents` tables. - -A direct chat is conceptually a private conversation with two account members. -A multi-member chat is the same conversation type with more members and -additional membership state. Both should continue to use the append-only -`ChatEvent` log and `ChatEventAttachment` references. - -Before MLS group chat is introduced, add an explicit conversation aggregate -around the existing event log: +Model an encrypted multi-member chat as a private group specialization. Reuse +`Group`, `Post`, `PostsContents`, and the group reconciliation path where their +existing contracts fit, while a dedicated `PrivateGroup` module owns the +private-only policy: ```text -ChatConversation - |-- ChatConversationMember - |-- ChatConversationHead - |-- ChatEvent - | |-- ChatEventRecipient - | `-- ChatEventAttachment - `-- protocol-specific device membership state +Group + |-- GroupMember + | `-- versioned member-device public keys + |-- Post + | `-- PostsContents + `-- group head / missing-post reconciliation + +PrivateGroup module + |-- validates private-group publication + |-- resolves the active member-device key snapshot + |-- owns membership/key epoch transitions + |-- adds delivery acknowledgement/retry policy + `-- prevents public publishing integrations from running ``` -This is a domain separation decision, not a requirement to duplicate storage, -pagination, or rendering helpers. +The existing direct-message `ChatEvent` path remains compatible while this +specialization is developed. It should not be rewritten until common post/chat +event behavior and migration rules are proven by tests. Posts and chats share more timeline behavior than their current tables expose. Both can use ordered events, edits represented as later events, missing-item @@ -48,14 +49,15 @@ Timeline / Space Social Group `-- Post projection and publishing behavior -Private Conversation - `-- ChatEvent projection and device/group-key behavior +Private Group + `-- Post projection plus device/group-key behavior ``` -The recommendation is therefore not that posts and chats are fundamentally -unrelated. It is that the current publishing tables already own behavior that -must not be activated implicitly for private chat. Shared primitives should be -extracted deliberately instead of making chat rows masquerade as social posts. +The key boundary is now the `PrivateGroup` policy module rather than a separate +conversation database aggregate. Existing post callbacks must dispatch through +that module for private groups so public manifests, ActivityPub/Bluesky, +static-site generation, RSS, and other public side effects are not activated +implicitly. ## Findings From The Current Implementation @@ -91,13 +93,14 @@ envelope with durable retry, acknowledgement, and missing-range repair. - A failed fetch leaves delivery pending and creates no recipient event. - Attachment order, filename, media type, original size, and content key stay inside the encrypted browser payload. -- Per-user release intent is stored separately in - `ChatEventAttachmentRetention`. +- The current direct-message implementation stores per-user local retention + intent in `ChatEventAttachmentRetention`; it does not grant a recipient + permission to delete the sender's message or attachment. These behaviors are now covered across PostgreSQL-backed restart tests and independent GeeSome processes using separate Kubo nodes. -## Why Social Groups And Posts Should Not Be Reused +## What Cannot Be Reused Unchanged ### Different lifecycle @@ -144,7 +147,7 @@ best-effort send cannot be shown as delivered without an acknowledgement. Subscribed group-post replication could reuse the durable or replicated modes; public feeds may prefer pull-only behavior. -### Different membership level +### Private membership extends normal group membership Social group membership is primarily account and permission based. Encrypted chat must also account for browser devices, revoked devices, recipient keys, @@ -157,24 +160,33 @@ message content once and wraps its content key separately for each allowed device. Removing Bob's tablet excludes that device from future events while Bob remains a conversation member through his phone and laptop. -This device-level recipient list is required by the current direct-message -envelope. It can coexist with account-level membership in a generic timeline or -conversation. +The same device-level recipient approach can be used for private groups. Normal +group membership identifies the accounts; private-group membership data adds +the active public keys for each account's allowed devices. + +The key list must not be read as one mutable global list when decrypting old +posts. Each accepted private post must identify the membership/key version used +when it was created. Otherwise a later device addition or removal would make it +ambiguous which devices were valid recipients of an earlier post. ### Different metadata boundary -Post content names, views, and ordering are ordinary server-readable metadata. -Chat attachment names, media types, display order, and keys are intentionally -inside the encrypted envelope. Reusing `PostsContents` would either expose that -metadata or create misleading empty join fields. +Normal post content names, views, and ordering are server-readable metadata. +Private-group attachment names, media types, display details, and keys must stay +inside the browser-encrypted post payload. `PostsContents` may carry relation +identity and stable ciphertext ordering, but private fields must not be copied +into its visible metadata columns. ### Different remote representation -A remote chat recipient can pin ciphertext without owning a normal `Content` -row. `PostsContents` requires a `Content` entity, while -`ChatEventAttachment.contentId` is deliberately nullable on recipient nodes. +A remote private-group member can pin ciphertext without becoming its author or +owner. Reusing `PostsContents` therefore requires the shared-content identity +path: a replicated `Content` reference may point at the canonical +`StorageObject`, but must preserve the original author/remote identity and must +not fabricate local ownership. The current direct-message path may continue to +use nullable `ChatEventAttachment.contentId` during compatibility. -### Unwanted product coupling +### Public product coupling must be disabled explicitly Reusing `Group` and `Post` would make chat changes interact with unrelated publishing behavior: @@ -186,41 +198,33 @@ publishing behavior: - moderation and feed queries; - social-import identity and derived-state jobs. -Preventing every one of those paths from treating chat messages as publishable -posts would be more fragile than maintaining the smaller chat model. +The `PrivateGroup` module must define which of these callbacks are disabled, +replaced, or safe to reuse. This is preferable to scattering `if private` +conditions throughout each integration. Private posts may reuse group heads, +pagination, replies, attachment relations, and missing-post repair while public +distribution callbacks remain off. -## Recommended Conversation Aggregate +## Recommended Private Group Specialization -### `ChatConversation` +### `PrivateGroup` module -The first-class conversation row should own only server-required state: +The module should be selected from a stable group privacy/type field and own: -- stable `conversationId`; -- conversation kind (`direct` or `group`); -- protocol/capability version; -- lifecycle state; -- creation and update timestamps. +- private publication validation; +- account and device membership resolution; +- membership/key version transitions; +- acknowledgement and retry policy; +- encrypted-post callback dispatch; +- private-group capability/version state. User-visible title, description, avatar, and other private presentation fields should stay in browser-encrypted state unless a specific public or operator-visible field is intentionally designed. -The existing `ChatConversationHead` can become an association of this aggregate -without changing sequence semantics or rewriting existing events. - -### `ChatConversationMember` - -Membership should identify account owners independently from devices: - -- `conversationId`; -- stable account owner ID; -- nullable local `userId`; -- role and active/removed state; -- accepted membership sequence or epoch where needed. - -For direct conversations, policy should enforce the intended two-account -membership. Whether one account pair can have multiple conversations is a -product decision and should be explicit rather than inferred from table shape. +The module may use existing group extension/property data while the shape is +small. Promote it to a typed relation when atomic updates, bounded member +queries, uniqueness, or history verification cannot be enforced reliably in +embedded data. ### Membership and key epochs @@ -238,9 +242,10 @@ devices must not read messages created in epoch 13. Removing only Bob's laptop changes device membership without necessarily removing Bob's phone or Bob's account from the conversation. -Public group posts do not need encrypted-group key epochs. A private encrypted -group feed would need an equivalent membership/version mechanism even if its -visible items were presented as posts. +Public group posts do not need encrypted-group key epochs. Private groups do. +The epoch can be stored as versioned private-group membership data and +referenced by each encrypted post; it does not require a separate conversation +aggregate. ### Device membership @@ -254,26 +259,24 @@ columns to the social group model. ### Messages and attachments -Keep messages as `ChatEvent`, not `Post`. Keep attachment routing and retention -in `ChatEventAttachment` and `ChatEventAttachmentRetention`. +Private-group messages can be encrypted `Post` records and use +`PostsContents`. The browser must encrypt message and attachment bytes before +upload; the node stores only the ciphertext and the minimum routing metadata. -Sender-owned ciphertext should continue to reuse `Content`, `StorageObject`, -IPFS pinning, and reference-safe cleanup. Recipient copies should not require -fabricated user-owned `Content` records. +Only the post author may issue a message or attachment deletion for the shared +private-group history, subject to group policy. Other members cannot delete the +author's post or attachment for everyone. -Per-user attachment release means that one local participant no longer wants an -attachment retained in that participant's chat history. It does not immediately -delete shared ciphertext that another participant, pending delivery, or -missing-range repair still needs. +A recipient may still hide an item in their own browser or evict a downloaded +ciphertext copy from a local cache. That is local presentation/storage behavior, +not a shared deletion and not an authoring permission. The recipient can fetch +the item again while the author-owned post and its retention policy still make +it available. -For example, Alice can release her local attachment view while Bob still keeps -it. Physical cleanup waits until all required local releases, remote delivery -acknowledgements, retention windows, and reference checks allow removal. - -Published post attachments usually follow author/publication retention rather -than one retention row for every unknown reader. Both products can share -reference counting and physical cleanup while keeping different release -policies. +Physical storage cleanup therefore follows author deletion, group retention, +pending delivery/repair references, and ordinary `Content`/`StorageObject` +reference safety. A per-recipient “attachment release” entity is not part of +the private-group product model. ### Missing-item reconciliation @@ -300,13 +303,13 @@ the same general recovery requirement. ## Infrastructure That Should Be Shared -Separate domain models do not require duplicate infrastructure. Chat should -continue to reuse: +The private-group specialization should reuse: - ordered timeline/change-event helpers where post and chat invariants match; - head comparison, bounded page repair, and cursor checkpoint helpers; - configurable best-effort, durable, pull-only, or replicated delivery policy; -- `Content` and `StorageObject` identity for sender-owned ciphertext; +- `Group`, `Post`, `PostsContents`, `Content`, and `StorageObject` identity for + author-owned ciphertext; - storage reference counting and deletion safety; - IPFS fetch, pin, and bounded size checks; - cursor and keyset pagination helpers; @@ -315,69 +318,69 @@ continue to reuse: decryption; - common user, role, and permission vocabulary where the semantics match. -If repeated relation behavior emerges across posts, chat, generated outputs, -and other entities, extract a small generic helper or storage-reference -contract. Do not make `PostsContents` itself generic after the fact. +If repeated relation behavior emerges across posts, direct chat, generated +outputs, and other entities, extract a small generic helper or +storage-reference contract. ## Product Interactions -A post can be shared into a chat by sending a typed encrypted reference to its -public identity or by attaching a browser-encrypted private copy. This does not -turn the post into the chat message or make the conversation a social group. - -Similarly, a conversation may offer a user action to publish selected content -as a post. That action should create an explicit post through the normal group -publishing flow. +A public post can be shared into a private group by sending a typed encrypted +reference to its public identity or by attaching a browser-encrypted private +copy. -Private encrypted group feeds may eventually combine durable posts with -conversation-like membership. They should be reviewed as a separate product -mode rather than introduced implicitly by storing chat events in `Post`. +Similarly, a private group may offer a user action to publish selected content +publicly. That action must create a separate public post through the normal +public-group flow; changing the private post's visibility in place risks +exposing private metadata or ciphertext policy. -At the product level it is reasonable to describe a direct conversation as a -private group of two users. That language does not require using the current -social `Group` database model. A future `Space` or `Timeline` aggregate could -support both social groups and private conversations while each keeps its own -projection and policy modules. +A direct conversation can eventually be represented as a private group of two +accounts. Keep the current direct `ChatEvent` path during the transition so the +new private-group contract can be validated without rewriting existing signed +events. ## Adoption Plan -1. Add `ChatConversation` and `ChatConversationMember` as additive model-sync - tables while this work remains unreleased on `dev`. -2. Lazily materialize a conversation row for existing `conversationId` values. - Do not rewrite or resign existing `ChatEvent` envelopes. -3. Keep current direct-message APIs compatible while moving membership and - authorization reads behind conversation helpers. -4. Extract shared timeline/head/reconciliation helpers only after post and chat +1. Define a stable private-group type/capability and route its post-publication + callbacks through a `PrivateGroup` module. +2. Define versioned account/device membership data and require every encrypted + private post to reference its accepted membership/key epoch. +3. Keep current direct-message APIs and signed `ChatEvent` envelopes compatible + while the private-group path is introduced. +4. Extract shared timeline/head/reconciliation helpers only after post, + private-group, and direct-chat invariants are compared and covered by common behavior tests. 5. Define explicit delivery policies and ensure UI delivery labels match the selected guarantee. -6. Define explicit direct-conversation uniqueness and invitation policy before - enforcing database constraints. -7. Add bounded member and conversation listing without exposing encrypted - presentation metadata. +6. Define explicit private-group invitation, member role, and direct + two-account uniqueness policy. +7. Add bounded private-group/member listing without exposing encrypted + presentation metadata or mutable key-history ambiguity. 8. Add missing-post reconciliation tests using group heads and chunked manifest - indexes without routing public posts through chat delivery rows. + indexes, including private encrypted posts and membership-epoch references. 9. Integrate MLS group state only after the browser dependency gate passes. -10. Add two-browser/two-node tests covering conversation creation, membership, - device changes, restart, missing-range repair, and attachments. -11. Retire temporary membership inference only after existing conversations - have been materialized and verified. +10. Add two-browser/two-node tests covering private-group creation, membership, + device changes, restart, missing-range repair, author deletion, recipient + local hiding/cache eviction, and attachments. +11. Consider migrating direct conversations to two-member private groups only + after compatibility, identity, ordering, and retention behavior is proven. ## Invariants - GeeSome nodes never receive chat plaintext, attachment keys, or browser private keys. -- `conversationId` remains stable across local database IDs and node replicas. -- Accepted message identity remains `messageId`; retries do not create another - event. -- Conversation sequence remains deterministic and append-only. -- Direct and group membership are explicit and independently testable. +- Group/post identity remains stable across local database IDs and node + replicas. +- Accepted private-post identity is idempotent; retries do not create another + post. +- Private-group event order and membership/key epochs remain deterministic. +- Account and device membership are explicit and independently testable. - Removing a device does not silently remove its account from the conversation. - Removing an account from a group conversation eventually removes all of its active protocol device memberships. -- Recipient attachment rows can remain storage-only references. -- Social publishing hooks never process chat events unless an explicit publish - action creates a real post. +- Only the author may delete a shared private post or its attachments. +- Recipient local hiding or cache eviction never deletes the shared post. +- Public publishing hooks never process private-group posts unless an explicit + publish action creates a separate public post. - Message and post edits can use later ordered events while retaining separate current-state projections. - Missing-item reconciliation is available to both chat and group timelines, @@ -385,17 +388,17 @@ projection and policy modules. ## Open Decisions -- Whether one pair of accounts may create multiple direct conversations. -- Which conversation metadata, if any, should be visible to the node. -- How invitations and member roles map to the first MLS group creation flow. +- Whether one pair of accounts may create multiple direct private groups. +- Which private-group metadata, if any, should be visible to the node. +- How invitations and member roles map to the first private-group/MLS creation + flow. - How long removed membership and old epoch metadata must be retained. -- Whether encrypted conversation metadata should use a distinguished chat event - or a separately versioned encrypted conversation document. -- Whether a generic `Space`/`Timeline` aggregate should be introduced after - shared behavior has been proven in both chat and group-post tests. +- Whether encrypted private-group metadata should use a distinguished post + event or a separately versioned encrypted group document. +- Whether direct `ChatEvent` conversations should eventually migrate to + two-member private groups or remain a compatible specialized projection. - Which delivery policies should be available for public groups, private group feeds, and direct conversations. -These decisions should be resolved before making group conversations available -by default, but they do not require replacing the working direct-message event -log. +These decisions should be resolved before private groups are available by +default. They do not require replacing the working direct-message event log. diff --git a/docs/todo.md b/docs/todo.md index 36e3d56f..62cc2d85 100644 --- a/docs/todo.md +++ b/docs/todo.md @@ -91,8 +91,12 @@ Architecture decision: - Keep PostgreSQL as the operational authorization, head, acknowledgement, and retry index. Consider encrypted IPLD event batches or checkpoints for portable replication only after the operational path is proven. -- Model direct and multi-member chat as dedicated conversations rather than - reusing the social `Group`, `Post`, and `PostsContents` tables. Follow the +- Model encrypted multi-member chat as a `PrivateGroup` specialization that + reuses `Group`, `Post`, `PostsContents`, group heads, and missing-post repair. + Keep device-key membership, membership/key epochs, delivery policy, and + private publication callbacks inside the `PrivateGroup` module. Preserve the + existing direct `ChatEvent` path until a tested compatibility/migration + decision is made. Follow the [conversation data model review](../app/modules/chat/docs/conversation-data-model-review.md) before adding group membership persistence. - Use MLS 1.0 for group membership and future-message key rotation according to @@ -139,17 +143,19 @@ MLS group-chat integration checklist: 2. Add a small `geesome-libs` adapter that owns versioned MLS byte encoding, GeeSome device identity binding, application-message framing, and shared cross-package fixtures. Keep package-specific calls out of product modules. -3. Add the additive `ChatConversation` and `ChatConversationMember` aggregate, - lazily materialize existing direct-conversation IDs without rewriting - events, and move membership/authorization reads behind conversation helpers. +3. Add a stable private-group capability and a `PrivateGroup` module that owns + private post callbacks, versioned account/device membership, epoch + transitions, and delivery policy. Do not rewrite existing direct + `ChatEvent` envelopes. 4. Add browser-owned MLS state storage in `geesome-ui`, including atomic state updates, restart recovery, clear-on-logout/device-removal behavior, and an explicit unrecoverable-state screen. GeeSome nodes must receive only opaque protocol values. 5. After the dependency passes, add bounded node storage and delivery contracts - for group metadata, one-time join packages, device-specific Welcome values, - proposals, commits, and application events. Reuse the existing durable event - log, queue, acknowledgement, and missing-range repair machinery. + for private-group metadata, one-time join packages, device-specific Welcome + values, proposals, commits, and encrypted posts. Reuse group post/content + relations and missing-post repair together with the existing durable queue + and acknowledgement machinery. 6. Implement group creation and device join first. Then add another device, remove a device, remove an account's remaining devices, restore a device as a new member, and reconcile database membership with the current MLS epoch as @@ -157,17 +163,18 @@ MLS group-chat integration checklist: 7. Serialize membership updates through the canonical group node. Reject stale expected epochs, reload the accepted update, discard interrupted local work, and let the browser rebuild the requested change when it is still allowed. -8. Add group-message and encrypted-attachment UI using the existing chat - conversation surface. Show joining, waiting for an update, retrying, - unsupported client, removed device, and unavailable older history states in - ordinary user language. +8. Add private-group post and encrypted-attachment UI using the existing chat + surface. Only authors may delete their shared messages/attachments; other + members may only hide an item or evict its local cache. Show joining, waiting + for an update, retrying, unsupported client, removed device, and unavailable + older history states in ordinary user language. 9. Add real two-browser/two-node tests for restart, temporary node unreachability, duplicate and reordered events, interrupted membership updates, simultaneous updates, device removal, attachment delivery, and bounded history repair. -10. Enable the feature only for newly created group conversations behind a - capability flag. Keep direct messages unchanged and keep older group chats - visibly on their existing mode until an explicit migration flow exists. +10. Enable the feature only for newly created private groups behind a capability + flag. Keep direct messages unchanged and keep older group chats visibly on + their existing mode until an explicit migration flow exists. MLS implementation findings: From 2073f999625a5a9f6a153bf9da19c6e790722584 Mon Sep 17 00:00:00 2001 From: MicrowaveDev Date: Thu, 30 Jul 2026 21:06:26 +0100 Subject: [PATCH 4/4] docs(chat): plan private group migration --- .../docs/conversation-data-model-review.md | 135 ++++++++++++++++++ docs/todo.md | 6 + 2 files changed, 141 insertions(+) diff --git a/app/modules/chat/docs/conversation-data-model-review.md b/app/modules/chat/docs/conversation-data-model-review.md index 87d7c540..3a57054a 100644 --- a/app/modules/chat/docs/conversation-data-model-review.md +++ b/app/modules/chat/docs/conversation-data-model-review.md @@ -364,6 +364,141 @@ events. 11. Consider migrating direct conversations to two-member private groups only after compatibility, identity, ordering, and retention behavior is proven. +## Migration Plan + +This is a compatibility migration from the current direct-chat entities to the +private-group architecture. It is not a one-time table rewrite. The legacy path +must remain readable until native private groups and projected legacy +conversations produce equivalent user-visible results. + +### Migration states + +Every chat exposed through the common API must be in one explicit state: + +- `legacy-only`: current `ChatEvent` entities remain the source of truth; +- `projected`: immutable legacy events have an idempotent private-group + projection, while the legacy rows remain authoritative; +- `native-private-group`: the private `Group`/`Post` timeline is authoritative. + +The API must deduplicate by stable source identity so a projected event is not +shown beside its legacy source. Mixed-version nodes must not infer migration +state from the presence of a few posts. + +### Phase 1: improve the shared group foundation + +Before storing chat in groups: + +1. Add a `PrivateGroup` module selected by a stable group capability/type. +2. Route post lifecycle callbacks through the group-type module. Private groups + must not trigger public manifests, RSS, static-site, ActivityPub, Bluesky, or + social-import behavior. +3. Add ordered create/edit/delete event identities for posts while retaining + `Post` as the current-state projection. +4. Make group heads and bounded missing-post reconciliation deterministic. +5. Support remote `Content` references without fabricating local ownership. +6. Keep private attachment names, display metadata, and keys in the encrypted + browser payload while using `PostsContents` for ciphertext relations. +7. Define author-controlled shared deletion separately from recipient-local + hiding and cache eviction. + +These improvements should be covered for ordinary groups too where the +invariants are shared. + +### Phase 2: implement native private groups + +1. Store account membership plus versioned active device public keys. +2. Require every encrypted post to identify the accepted membership/key epoch. +3. Let browsers create message and attachment ciphertext before upload. +4. Reuse the durable queue, acknowledgement, retry, and missing-item repair + machinery behind the private-group delivery policy. +5. Expose private groups through the existing chat-facing API and UI so callers + do not depend on database entity names. +6. Add a capability flag for creating native private groups. Keep it disabled + by default until the two-node verification gate passes. + +### Phase 3: add the legacy projection + +Build a deterministic, resumable projector with the following mapping: + +```text +legacy conversationId -> private Group legacy-source identity +ChatEvent.messageId -> Post legacy-source identity +ChatEvent sequence/time -> ordered post event sequence/time +ChatEvent author/signature -> preserved author and source evidence +ChatEvent encrypted payload -> encrypted private Post payload +ChatEventAttachment -> PostsContents ciphertext relation +recipient/device records -> referenced membership/key version +delivery acknowledgement -> retained delivery evidence +``` + +The projector must: + +- never decrypt, rewrite, or re-sign a legacy event; +- preserve message IDs, ordering, timestamps, authors, attachment order, edits, + deletes, and source signatures; +- use unique legacy-source identities so reruns cannot create duplicates; +- process bounded batches with durable checkpoints and per-conversation status; +- leave a failed conversation in `legacy-only` state rather than exposing a + partial projection; +- produce a machine-readable comparison report before marking a conversation + `projected`. + +If a field cannot be represented without losing meaning, retain it as immutable +legacy evidence and keep that conversation on the compatibility reader. + +### Phase 4: verify equivalence + +For projected conversations, compare both representations: + +- conversation/member/device identity; +- message count and stable identities; +- create/edit/delete order and current visible state; +- author and timestamp attribution; +- attachment count, order, ciphertext CID, and availability; +- delivery/acknowledgement state; +- missing-range repair after restart; +- author deletion and recipient-local hiding behavior. + +Run unit tests for mapping and idempotency, PostgreSQL restart tests, and real +two-browser/two-node tests. Include interrupted projection, duplicate input, +out-of-order legacy events, unavailable attachment ciphertext, removed devices, +and mixed legacy/native listing. + +### Phase 5: change the default + +After the equivalence gate passes: + +1. Enable native private groups for newly created multi-member chats. +2. Enable them for newly created direct chats only after two-member uniqueness, + invitation, and UI behavior is verified. +3. Keep legacy reads enabled and migrate existing conversations in bounded + background batches. +4. Show migration failures to operators without blocking unaffected chats. +5. Stop creating new legacy conversations only after all supported clients can + read native private groups. + +Avoid dual-writing new messages to both schemas. It creates two competing +sources of truth during partial failure. Use one authoritative representation +per conversation and a read-only projection for comparison. + +### Phase 6: retirement and rollback + +Rollback means switching API routing back to the legacy reader for projected +conversations; it must not require restoring rewritten events. Keep legacy rows, +attachments, and delivery evidence through at least one stable release after +native private groups become the default. + +Remove legacy write/read code and tables only after: + +- no supported client requires them; +- every retained conversation is verified as native or projected; +- backup/restore and migration reports have been reviewed; +- rollback has not been needed for the agreed observation period; +- storage-reference checks prove cleanup cannot remove live ciphertext. + +Table cleanup is a separate release decision, not part of enabling +`PrivateGroup`. + ## Invariants - GeeSome nodes never receive chat plaintext, attachment keys, or browser diff --git a/docs/todo.md b/docs/todo.md index 62cc2d85..a7416fd3 100644 --- a/docs/todo.md +++ b/docs/todo.md @@ -175,6 +175,12 @@ MLS group-chat integration checklist: 10. Enable the feature only for newly created private groups behind a capability flag. Keep direct messages unchanged and keep older group chats visibly on their existing mode until an explicit migration flow exists. +11. Implement the staged + [legacy chat to private-group migration plan](../app/modules/chat/docs/conversation-data-model-review.md#migration-plan): + keep `legacy-only`, `projected`, and `native-private-group` states explicit; + project immutable events idempotently without decrypting or re-signing; + verify both representations; then change the new-chat default. Retain the + legacy reader and rows through the rollback observation period. MLS implementation findings: