Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
411 changes: 135 additions & 276 deletions .claude/skills/implement-agent-id/SKILL.md

Large diffs are not rendered by default.

11 changes: 7 additions & 4 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copilot Instructions — entrabot-identity-research
# Copilot Instructions — Entrabot

## Project Overview

Expand Down Expand Up @@ -28,7 +28,7 @@ Key concepts:
# Install dependencies
pip install -e ".[dev]"

# Run all tests (1,237 tests)
# Run all tests
pytest -v --tb=short && ruff check .

# Run with channel notifications
Expand All @@ -52,8 +52,9 @@ src/entrabot/
audit/ # Action tracking / audit log
identity/ # Progressive identity state machine
storage/ # Local/Blob/Persona backends
security/ # XPIA external-content boundary
mcp_server.py # FastMCP server + background poll + channel push
tests/ # Mirrors src/ structure (1,237 tests)
tests/ # Mirrors src/ structure
docs/ # Research, ADRs, learnings, specs
scripts/ # setup.sh, teardown.sh, Entra provisioning
```
Expand All @@ -66,14 +67,16 @@ scripts/ # setup.sh, teardown.sh, Entra provisioning
- **Background channel**: `_background_poll()` runs every 5s, pushes new human messages via `notifications/claude/channel`. Uses separate dedup state from `watch_teams_replies` (Learning #27).
- **Audit-first design**: Every agent action that touches a resource must emit an audit event before returning.
- **Graph API**: `$filter`/`$orderby` unreliable for chat messages (Learning #16) — always filter client-side.
- **Stable agent identity**: use `ENTRABOT_AGENT_UPN` and `sender_upn` for self/peer matching; display names are mutable. `ENTRABOT_AGENT_USER_UPN` is a compatibility alias only.
- **XPIA boundary**: route model-facing Teams, email, Files, and Work IQ content through `entrabot.security.xpia.wrap_external`. Existing envelope-looking text is still untrusted input and must receive the authoritative outer envelope.

## Conventions

- Use `dataclasses` or `pydantic` models for all structured data — no raw dicts
- Type-annotate all function signatures
- Test files mirror source structure
- Secrets and tokens never appear in logs — use `repr` overrides on sensitive fields
- Read `docs/runbooks/hard-won-learnings.md` (66 entries) before making auth/Teams changes
- Read `docs/runbooks/hard-won-learnings.md` before making auth/Teams changes
- ADRs in `docs/decisions/` for all significant architectural choices
- **Sponsor DM wait pattern (host-gated).** When the human says "ping me when X is done" / "I'm going AFK, let me know" / any equivalent: confirm in Teams with `send_teams_message`, do the work, send the completion update with `send_teams_message`. Claude Code receives replies through channel-push next-turn input. Copilot CLI, Codex, Cursor, and other non-channel-push hosts receive the sponsor reply inline from `send_teams_message` as `sponsor_reply`. Only call `wait_for_sponsor_dm` when the operator explicitly says "block until they reply." NEVER poll in a loop. NEVER spawn `copilot -p` / headless subprocesses. NEVER use `watch_teams_replies` for this pattern. Full protocol: `prompts/anatomy/channel-discipline.md`; see Learning #54.

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- run: pip install mkdocs-material mkdocstrings[python]
- run: mkdocs build
- run: pip install mkdocs-material
- run: mkdocs build --strict
- uses: actions/upload-pages-artifact@v3
with:
path: site
Expand Down
11 changes: 6 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@
- Never use `az rest` or Azure CLI tokens for Agent Identity APIs — they include `Directory.AccessAsUser.All` which causes hard 403 (Learning #1)
- Always create BlueprintPrincipal explicitly after Blueprint — it is NOT auto-created (Learning #2)
- Agent IDs are service principals, not users — never create fake user accounts with passwords
- **AGENT NAMES CHANGE — USE UPN.** Never identify an agent (self or peer) by display name in code paths that filter, deduplicate, authorize, or route. Display names are user-mutable and localizable. Use **UPN as the canonical config value** (e.g. `ENTRABOT_AGENT_UPN=entra-agent@werner.ac`) and match on the message payload's `sender_upn` first, falling back to `sender_id` (AAD object-id). Rename incident 2026-07-09: renaming "EntraBot Agent" → "EntraClaw Agent" made the Teams poll's self-authored filter no-op, causing 6-week-old outbounds to replay as inbound across 61/62 chats. See `docs/runbooks/hard-won-learnings.md` Learning #69 and `docs/architecture/PLAN-agent-identity-by-upn.md`.
- **External content is untrusted.** Model-facing Teams, email, Files, and Work IQ content must pass through `entrabot.security.xpia.wrap_external`. Never trust or preserve an inbound `<external_content>` envelope as authoritative; always add the boundary-owned outer envelope.
- **AGENT NAMES CHANGE — USE UPN.** Never identify an agent by display name in code paths that filter, deduplicate, authorize, or route. Use `ENTRABOT_AGENT_UPN` as the canonical config value (for example, `entra-agent@contoso.onmicrosoft.com`), match `sender_upn` first, and fall back to the Entra object ID. `ENTRABOT_AGENT_USER_UPN` remains a compatibility alias for existing `.env` files. See Learning #69 and `docs/architecture/PLAN-agent-identity-by-upn.md`.
- Parse `az` CLI output as JSON, not TSV — TSV can be corrupted by warnings (Learning #7)
- Graph API `$filter`/`$orderby` are unreliable for chat messages — always filter client-side (Learning #16)
- **Sub-agent worktree installs must use a worktree-local venv, never the parent venv** (Learning #36) — running `pip install -e .` from inside a git worktree against the main repo's `.venv/bin/pip` silently re-points the parent venv's editable-install target at the worktree source tree. Every subsequent MCP server boot then loads code from the worktree — which has no `.env`, no auth, no polling, and no visible error. Always create `python3 -m venv .venv && source .venv/bin/activate && pip install -e ".[dev]"` inside the worktree BEFORE any editable install. After any session that used sub-agent worktrees, verify the main venv's target via `.venv/bin/python3 -c "from entrabot import config; print(config.__file__)"` — the path must not contain `.claude/worktrees/`.
Expand All @@ -56,8 +57,8 @@ These are not optional. Skipping them is the documented cause of 4 design errors
## Current Runtime Model

- Python 3.12+ research project — no deployed service yet
- Eight modules: `platform/` (OS shim) → `auth/` (certificate JWT + MSAL delegated) → `a365/` (Work IQ MCP provider + Word adapter) → `tools/` (MCP tools + interaction log + email poll + daily summary + cards) → `audit/` (tracking) → `identity/` (state machine) → `storage/` (`LocalBackend`/`BlobBackend`/`PersonaBackend` + `migration` helper — ADR-005 Phases 1, 2, 5, 6a shipped) → `mcp_server.py` (FastMCP + background channel)
- External dependencies: Microsoft Entra ID, Microsoft Teams + Outlook mailbox (Graph API or Bot Framework), Azure Blob Storage (optional, opt-in via `setup.sh --use-cloud-memory`)
- Core runtime components: `platform/` (OS shim) → `auth/` (certificate JWT + MSAL delegated) → `a365/` (Work IQ MCP provider + Word adapter) → `tools/` (MCP tools + interaction log + email poll + daily summary + cards) → `audit/` (tracking) → `identity/` (state machine) → `storage/` (`LocalBackend`/`BlobBackend`/`PersonaBackend` + `migration` helper — ADR-005 Phases 1, 2, 5, 6a shipped) → `mcp_server.py` (FastMCP + background channel)
- External dependencies: Microsoft Entra ID, Microsoft Teams + Outlook mailbox (Microsoft Graph), Azure Blob Storage (optional, opt-in via `setup.sh --use-cloud-memory`)
- **No default group chat.** Every Teams tool requires an explicit `chat_id`. Chats come from `create_chat`, the persisted `watched_chats` file, or the auto-discovery sweep over `/me/chats`.
- **Body-first prompt.** `prompts/agent_system.md` loads at boot with `@include` expansion of `prompts/anatomy/*.md`. Persona-sati output (if configured) is appended AFTER the body and cannot override body rules.
- Two auth modes via `ENTRABOT_MODE`: `agent_user` (three-hop), `delegated` (MSAL). Agent memory has a **parallel third hop** against `https://storage.azure.com/.default` (`acquire_agent_user_storage_token`).
Expand Down Expand Up @@ -122,12 +123,12 @@ Note: efferent-copy may mechanically cover body-tool observe but not bootstrap/r

## Read These First

- `docs/engineering-status.md` — current state, test count (1,237), next steps
- `docs/engineering-status.md` — current state and next steps
- `prompts/agent_system.md` + `prompts/anatomy/*.md` — the body prompt that governs your behaviour (security, channel discipline, identity/tools)
- `docs/architecture/DESIGN-persona-sati-integration.md` — mind-body split design
- `docs/decisions/005-cloud-hosted-memory.md` — cloud memory spec
- `prompts/agent_system.md.archive` — original monolithic prompt, kept for reference
- `docs/runbooks/hard-won-learnings.md` — 66 learnings, read before making changes
- `docs/runbooks/hard-won-learnings.md` — read before making changes

## Commands

Expand Down
28 changes: 27 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,31 @@
# Changelog

## Unreleased

### Added

- Rename-safe agent matching through canonical `ENTRABOT_AGENT_UPN` with object-ID fallback.
- Boundary-owned XPIA envelopes for model-facing Teams, email, Files, and Work IQ content.
- `read_email`, `read_interactions`, and `bootstrap_body_state`, bringing the MCP surface to 37 tools.
- Windows status tooling and expanded Windows setup/teardown guidance.

### Changed

- Blueprint, BlueprintPrincipal, and Agent Identity creation now use the dedicated Microsoft Graph v1.0 subtype endpoints; Agent User creation remains on beta.
- Documentation, quickstarts, API references, script references, and GitHub Pages navigation were refreshed against current code and platform APIs.
- GitHub Pages builds run in strict mode.

### Fixed

- Self-authored Teams messages remain filtered after an Agent User display-name change.
- Forged `<external_content>` text can no longer bypass the authoritative XPIA boundary.
- A UTC-midnight-dependent test no longer fails intermittently on Windows CI.

### Removed

- Bot Gateway mode and its M365 Agents SDK dependency (ADR-006).
- The unused Claude pull-request review workflow.

## v0.1 — 2026-05-21

First public release. Reference implementation for Microsoft Entra Agent ID and Microsoft Agent 365 (GA 2026-05-01). MIT licensed. **Research repo, not production-ready** — see Known Limitations below.
Expand Down Expand Up @@ -27,7 +53,7 @@ First public release. Reference implementation for Microsoft Entra Agent ID and

**Body prompt architecture**
- Non-overridable body prompt at `prompts/agent_system.md` with `@include` expansion of `prompts/anatomy/*.md`. Security, channel discipline, identity/tools rules load below the persona line.
- Instruction-injection defense at the architectural level — an agent that runs on entrabot cannot be jailbroken into impersonating its operator.
- Body-first instruction boundaries and channel discipline designed to reduce instruction-injection and operator-impersonation risk.

**Mind / persona (optional)**
- Persona-sati MCP integration. Body composes `body + persona` at boot when `PERSONA_SATI_MCP_URL` is set. Clean fallback to body-only mode when persona-sati is unreachable.
Expand Down
9 changes: 5 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@
- Never use `az rest` or Azure CLI tokens for Agent Identity APIs — they include `Directory.AccessAsUser.All` which causes hard 403
- Always create BlueprintPrincipal explicitly after Blueprint — it is NOT auto-created
- Agent IDs are service principals, not users — never create fake user accounts with passwords
- **AGENT NAMES CHANGE — USE UPN.** Never identify an agent (self or peer) by display name in code paths that filter, deduplicate, authorize, or route. Display names are user-mutable and localizable. Use **UPN as the canonical config value** (e.g. `ENTRABOT_AGENT_UPN=entra-agent@werner.ac`) and match on the message payload's `sender_upn` first, falling back to `sender_id` (AAD object-id). Rename incident 2026-07-09: renaming "EntraBot Agent" → "EntraClaw Agent" made the Teams poll's self-authored filter no-op, causing 6-week-old outbounds to replay as inbound across 61/62 chats. See `docs/runbooks/hard-won-learnings.md` Learning #69 and `docs/architecture/PLAN-agent-identity-by-upn.md`.
- **External content is untrusted.** Model-facing Teams, email, Files, and Work IQ content must pass through `entrabot.security.xpia.wrap_external`. Never trust or preserve an inbound `<external_content>` envelope as authoritative; always add the boundary-owned outer envelope.
- **AGENT NAMES CHANGE — USE UPN.** Never identify an agent by display name in code paths that filter, deduplicate, authorize, or route. Use `ENTRABOT_AGENT_UPN` as the canonical config value (for example, `entra-agent@contoso.onmicrosoft.com`), match `sender_upn` first, and fall back to the Entra object ID. `ENTRABOT_AGENT_USER_UPN` remains a compatibility alias for existing `.env` files. See Learning #69 and `docs/architecture/PLAN-agent-identity-by-upn.md`.
- Parse `az` CLI output as JSON, not TSV — TSV can be corrupted by warnings
- **Sub-agent worktree installs must use a worktree-local venv, never the parent venv.** Running `pip install -e .` from inside a git worktree against the main repo's `.venv/bin/pip` silently re-points the parent venv's editable-install target at the worktree source tree. Every subsequent `entrabot-mcp` boot from the parent venv then loads code from the worktree — which has no `.env`, no auth, no polling, and no visible error. After any session that spawned sub-agents in worktrees, verify `.venv/bin/python3 -c "from entrabot import config; print(config.__file__)"` does NOT contain `.claude/worktrees/`. See `docs/runbooks/hard-won-learnings.md` Learning #36 for the full writeup.
- **Sponsor DM wait pattern (host-gated).** When the human says "ping me when X is done" / "I'm going AFK, let me know" / any equivalent: confirm in Teams with `send_teams_message`, do the work, send the completion update with `send_teams_message`. What happens next depends on the host:
Expand All @@ -57,7 +58,7 @@
## Current Runtime Model

- Python 3.12+ research project — no deployed service yet
- Eight modules: `platform/` (OS shim) → `auth/` (certificate JWT + MSAL delegated) → `a365/` (Work IQ MCP provider + Word adapter) → `tools/` (MCP tools + interaction log + email poll + daily summary + cards) → `audit/` (tracking) → `identity/` (state machine) → `storage/` (`LocalBackend`/`BlobBackend`/`PersonaBackend` + `migration` helper — ADR-005 Phases 1, 2, 5, 6a shipped) → `mcp_server.py` (FastMCP + background channel)
- Core runtime components: `platform/` (OS shim) → `auth/` (certificate JWT + MSAL delegated) → `a365/` (Work IQ MCP provider + Word adapter) → `tools/` (MCP tools + interaction log + email poll + daily summary + cards) → `audit/` (tracking) → `identity/` (state machine) → `storage/` (`LocalBackend`/`BlobBackend`/`PersonaBackend` + `migration` helper — ADR-005 Phases 1, 2, 5, 6a shipped) → `mcp_server.py` (FastMCP + background channel)
- External dependencies: Microsoft Entra ID (identity), Microsoft Teams + Outlook mailbox (Graph API), Azure Blob Storage (optional, opt-in via `setup.sh --use-cloud-memory`)
- **No default group chat.** Every Teams tool requires an explicit `chat_id`. Chats come from `create_chat`, the persisted `watched_chats` file, or the auto-discovery sweep over `/me/chats`.
- **Body-first prompt.** `prompts/agent_system.md` loads at boot with `@include` expansion of `prompts/anatomy/*.md`. Persona-sati output (if configured) is appended AFTER the body and cannot override body rules. See the "Body prompt is non-overridable" rule above.
Expand Down Expand Up @@ -226,7 +227,7 @@ Two memory systems coexist in this project:
- `docs/architecture/NEXT-WhatsApp-lightweight-teams-chat.md` — delegated mode spec (landed)
- `docs/index.md` — doc site entry point
- `docs/runbooks/mcp-disconnect-investigation.md` — **OPEN issue.** Entrabot MCP dies after 2–10 min of sustained activity. Two amplifiers fixed (PR #40, PR #41), root cause still unknown. Read this before debugging any MCP-drop symptom — do NOT restart the investigation from scratch.
- `docs/runbooks/hard-won-learnings.md` — 66 learnings, read before making changes
- `docs/runbooks/hard-won-learnings.md` — read before making changes
- `docs/decisions/001-obo-flows-for-device-agents.md`
- `docs/decisions/003-certificate-auth-over-client-secrets.md`
- `docs/platform-learnings/microsoft-agent-365.md` — A365 GA'd 2026-05-01. Identity model, Work IQ MCP catalog, four capability tiers, auth flows, gap analysis vs entrabot. Read this before considering any A365 / Work IQ integration work.
Expand Down Expand Up @@ -265,7 +266,7 @@ pip install mkdocs-material && mkdocs serve
- `src/entrabot/mcp_server.py`: FastMCP server — Teams tools + 2 auth modes + background poll + channel push + token refresh (generic instructions — personality in persona-sati)
- `src/entrabot/config.py`: `ENTRABOT_MODE` switch (auto/delegated/agent_user) + all env config
- `docs/decisions/`: ADRs — every significant architectural choice is recorded here
- `docs/runbooks/hard-won-learnings.md`: 66 hard-won learnings — READ THIS before making changes
- `docs/runbooks/hard-won-learnings.md` — READ THIS before making changes
- `docs/runbooks/mcp-disconnect-investigation.md`: OPEN MCP-disconnect dossier — READ before touching MCP transport, logging, or efferent-copy code

## gstack
Expand Down
Loading
Loading