Add custom django debug toolbar panel

.pre-commit-config.yaml

@@ -11,7 +11,7 @@ repos:
   rev: "1.0.0"
   hooks:
   - id: pyrefly-check
-    additional_dependencies: ["django>=5.2"]
+    additional_dependencies: ["django>=5.2", "django-debug-toolbar>=4.0"]
 - repo: https://github.com/adamchainz/django-upgrade
   rev: "1.30.0"
   hooks:

ARCHITECTURE.md

@@ -60,6 +60,12 @@ Eagle only instruments code you own, so warnings only point at relations you can
   models libraries shipped as packages).
 - **`EAGLE_EXCLUDE_APPS`** — overrides both; a listed app is never instrumented.
 
+The Debug Toolbar can opt back into the excluded apps: with
+`EAGLE_DEBUG_TOOLBAR_INCLUDE_EXCLUDED_APPS = True`, `AppConfig.ready` also instruments
+`get_excluded_models()` and registers their labels as *warn-suppressed* (via
+`register_warn_suppressed_labels`), so those apps are tracked and shown in the panel but
+never emit warnings — letting a migrating project see excluded-app waste without failing tests.
+
 Proxy models are skipped (they share the concrete model's descriptors). The resulting
 model classes are stored in a module-level set by `register_tracked_models`; the rest of
 eagle calls `is_instrumented(model)` to decide whether to record anything for a given
@@ -184,6 +190,7 @@ class _CollectorState:
     active: bool                                  # is a request being tracked?
     loaded: dict[(model_label, cache_name), LoadedRelation]   # kind + call-site
     consumed: set[(model_label, cache_name)]      # relations that were accessed
+    loaded_counts: dict[(model_label, cache_name), int]       # how many instances carried each load
 ```
 
 - **Context-local** (a `contextvars.ContextVar`) so concurrent
@@ -201,13 +208,20 @@ class _CollectorState:
   bare class name (resolved via the app registry when unambiguous).
 - **First write wins** for `loaded`, so the originally captured call-site survives repeated
   loads.
+- **Counts live separately** in `loaded_counts`: because `loaded` is first-write-wins it cannot
+  also tally repeated loads, so each `_record_loaded` call increments a per-relation counter
+  (per-row for `select_related`, per-parent for `prefetch_related`). This fan-out feeds the
+  Debug Toolbar panel's cost estimates.
 
 `begin_request()` resets and activates it; `end_request()` reconciles and deactivates it.
 
 ## Emitting warnings
 
-`end_request()` (`unused/tracker.py`) reconciles the two sets and emits one warning per
-surviving relation:
+`end_request()` (`unused/tracker.py`) reconciles the two sets via `collect_all_unused()`
+(`unused/report.py`), which returns one structured `UnusedRelation` record per loaded-but-unread
+relation, each tagged with `warn_ignored` (whether `EAGLE_WARN_UNUSED_IGNORE` suppresses it);
+`end_request` emits one warning per record that is *not* `warn_ignored` (`collect_unused()` is
+the same set already narrowed to that warning view):
 
 ![emit_warnings.png](assets/architecture/emit_warnings.png)
 
@@ -219,25 +233,43 @@ The warning is a real Python `warnings.warn` with category `UnusedRelatedAccess`
 subclass of `EagleWarning`), so you can route, filter, or escalate it with the standard
 `warnings` machinery and pytest's `filterwarnings`.
 
+### Stashing the report for later readers
+
+Before it stops the collector, `end_request` stashes the **full** report (every
+loaded-but-unread relation, including the `warn_ignored` ones) in a context-local
+(`_last_report`, read back via `get_last_report()`). `collector.stop()` then installs a fresh
+empty state, so by the time later code runs the live collector is gone -- but the stash
+survives. This is what lets the optional [Debug Toolbar panel](README.md#django-debug-toolbar-panel)
+read a request's unused loads in its `generate_stats`, *after* the whole middleware stack
+(including eagle's own middleware) has finished, regardless of middleware ordering. When the
+panel instead runs while the collector is still live, it calls `collect_all_unused()` directly.
+
+Stashing the full set (rather than just the warning view) is deliberate: the panel shows a
+*different* slice than the warnings. Warnings are `loaded − consumed − EAGLE_WARN_UNUSED_IGNORE`;
+the panel is `loaded − consumed − EAGLE_DEBUG_TOOLBAR_IGNORE`, so warning-suppressed loads stay
+visible in the panel (flagged via `warn_ignored`) while a project migrates them off.
+
 ## Module map
 
 ![module_map.png](assets/architecture/module_map.png)
 
 | Module | Responsibility |
 | --- | --- |
 | `eagle/apps.py` | Startup entrypoint (`AppConfig.ready`); wires everything together. |
-| `eagle/config.py` | `is_enabled()` — reads `EAGLE_ENABLED`. |
+| `eagle/config.py` | `is_enabled()` (reads `EAGLE_ENABLED`) and `include_excluded_apps_in_toolbar()` (reads `EAGLE_DEBUG_TOOLBAR_INCLUDE_EXCLUDED_APPS`). |
 | `eagle/middleware.py` | Scopes tracking to a request; flushes warnings on response. |
 | `eagle/decorators.py` | `warn_unused` — scopes tracking around a single call or `with` block (outside the request cycle). |
 | `eagle/sinks.py` | Public `mark_considered` / `may_access` escape hatches. |
 | `eagle/exceptions.py` | `EagleWarning` / `UnusedRelatedAccess` warning categories. |
-| `eagle/instrumentation/scope.py` | Decides which apps/models to instrument. |
+| `eagle/instrumentation/scope.py` | Decides which apps/models to instrument (`get_first_party_models`); `get_excluded_models` yields the `EAGLE_EXCLUDE_APPS` models for optional toolbar-only profiling. |
 | `eagle/instrumentation/registry.py` | Set of instrumented model classes. |
 | `eagle/instrumentation/descriptors.py` | In-place descriptor swaps + tracking mixins. |
 | `eagle/instrumentation/models.py` | Applies descriptor patches across a model's fields. |
 | `eagle/instrumentation/query.py` | Patches `QuerySet`/`ModelIterable`/`prefetch_one_level`. |
 | `eagle/unused/state.py` | Thread-local `Collector` holding `loaded` / `consumed`. |
 | `eagle/unused/marker.py` | Records loaded/consumed and per-instance state. |
-| `eagle/unused/tracker.py` | `begin_request` / `end_request`; emits warnings. |
+| `eagle/unused/tracker.py` | `begin_request` / `end_request`; emits warnings and stashes the report. |
+| `eagle/unused/report.py` | `collect_all_unused` builds the full `UnusedRelation` report (each tagged `warn_ignored`); `collect_unused` is the warning view; `_last_report` / `get_last_report` stash the full report for post-response readers. |
 | `eagle/unused/location.py` | Captures the user's call-site for a queryset. |
-| `eagle/unused/ignore.py` | Applies `EAGLE_WARN_UNUSED_IGNORE` rules. |
+| `eagle/unused/ignore.py` | Matches ignore rules; drives both the warning list `EAGLE_WARN_UNUSED_IGNORE` and the panel's `EAGLE_DEBUG_TOOLBAR_IGNORE`. |
+| `eagle/panels.py` | Optional Django Debug Toolbar panel (`EagleUnusedLoadsPanel`): lists unused loads (including warning-suppressed, flagged) with heuristic cost estimates; filtered by `EAGLE_DEBUG_TOOLBAR_IGNORE`. |

README.md

@@ -150,6 +150,69 @@ with warn_unused():
 
 `warn_unused` begins tracking before the scoped code runs and ends it on exit — exactly as the middleware does for a request — so the wasted join above emits the same `UnusedRelatedAccess` warning. Tracking always ends, even if the scoped code raises, so a failure never leaks tracking state into later work in the same context. The decorator form works on sync and async callables and preserves wrapper metadata (`__name__`, `__doc__`), and either form is a transparent passthrough when `EAGLE_ENABLED` is falsy.
 
+## Django Debug Toolbar panel
+
+If you use [Django Debug Toolbar](https://django-debug-toolbar.readthedocs.io/), eagle ships an optional **Unused Eager Loads** panel that surfaces the same signal as an interactive panel rather than (or alongside) warnings.
+
+### Install
+
+```bash
+pip install "django-eagle[debug-toolbar]"
+```
+
+### Configure
+
+Add the panel to `DEBUG_TOOLBAR_PANELS`, alongside the toolbar's own panels:
+
+```python
+DEBUG_TOOLBAR_PANELS = [
+    # ...the toolbar's default panels...
+    "eagle.panels.EagleUnusedLoadsPanel",
+]
+```
+
+### What it shows
+
+For each request the panel lists every relation that was eager-loaded but never accessed -- the same `(model, relation)` signal eagle warns about -- with:
+
+- the **count** of unused loads (shown in the toolbar side bar, e.g. `3 unused`);
+- each relation's **model, field, kind, and call site** (`file:line`);
+- a per-row **cost estimate** -- `select_related` shows `1 JOIN · ~<rows>×<cols> cells`, `prefetch_related` shows `1 query · <parents> parents`;
+- a header summary of the total estimated JOINs, extra queries, and wasted cells you would avoid by pruning the loads.
+
+> **These figures are heuristics, not measurements.** "Cells" is rows × extra columns of materialised-but-unused data; eagle does not measure query wall-clock time or bytes of memory. Use them to rank which loads are worth pruning, not as a profiler.
+
+The panel works whether or not `EagleWarnUnusedMiddleware` is installed, and regardless of its order relative to the toolbar's middleware. When `EAGLE_ENABLED` is falsy it renders a short "eagle is disabled" message.
+
+### Showing loads you've silenced — `EAGLE_DEBUG_TOOLBAR_IGNORE`
+
+The panel deliberately shows unused loads **even when they're silenced by `EAGLE_WARN_UNUSED_IGNORE`**. Those rows are flagged with a `⚠ suppressed` status and counted in the header (e.g. `12 unused · 8 currently warning-suppressed`). This makes the panel a migration tracker: the warning ignore list keeps your test suite green while you migrate modules off wasteful eager loads, and the panel still shows you everything that's left to do.
+
+To hide noise from the panel only — without touching warnings — use the separate `EAGLE_DEBUG_TOOLBAR_IGNORE` list. It has the same rule shape as `EAGLE_WARN_UNUSED_IGNORE` and defaults to empty (the panel shows everything):
+
+```python
+EAGLE_DEBUG_TOOLBAR_IGNORE = [
+    {"model": "Eagle", "field": "location"},   # hide this one row from the panel
+    {"location": "*/vendor/*"},                # hide everything built under these modules
+]
+```
+
+The two lists are independent: `EAGLE_WARN_UNUSED_IGNORE` controls *warnings* (and so test failures), while `EAGLE_DEBUG_TOOLBAR_IGNORE` controls only what the *panel* displays.
+
+### Profiling excluded apps — `EAGLE_DEBUG_TOOLBAR_INCLUDE_EXCLUDED_APPS`
+
+Apps listed in `EAGLE_EXCLUDE_APPS` are normally skipped entirely — never instrumented, so they neither warn nor appear in the panel. That's ideal while you migrate a large codebase app-by-app, but it also hides how much those excluded apps still waste.
+
+Set this flag to have the toolbar profile them anyway:
+
+```python
+EAGLE_DEBUG_TOOLBAR_INCLUDE_EXCLUDED_APPS = True
+```
+
+With it on, excluded apps are instrumented and their unused eager loads show up in the panel, flagged `⚠ suppressed` — but they **never emit warnings**, so an excluded app still can't fail your test suite. This turns the panel into a full migration backlog: every wasteful load, even in the apps you haven't migrated yet, while warnings stay scoped to the apps you've already adopted.
+
+It defaults to `False`. Turning it on instruments more models (a one-off startup cost), but like everything else in eagle it only does anything when `EAGLE_ENABLED` is on — i.e. dev/CI — so there's no production cost.
+
 ## Suppressing false positives
 
 eagle spots access by intercepting Django's relation descriptors, which fire on ordinary attribute access — so template rendering, conditional reads, and Python-level serializers (including DRF) are all tracked while they run. A warning is still a false positive in the following cases:

eagle/apps.py

@@ -11,8 +11,9 @@ class EagleAppConfig(AppConfig):
     def ready(self) -> None:
         """Wire up Eagle's ORM instrumentation once all apps have finished loading."""
         # Deferred imports avoid AppRegistryNotReady when this module is imported before apps finish populating.
-        from eagle.config import is_enabled
+        from eagle.config import include_excluded_apps_in_toolbar, is_enabled
         from eagle.instrumentation import (
+            get_excluded_models,
             get_first_party_models,
             make_contenttypes_eager,
             make_model_eager,
@@ -41,6 +42,19 @@ def ready(self) -> None:
         for model in models:
             make_model_eager(model)
 
+        if include_excluded_apps_in_toolbar():
+            # Profile EAGLE_EXCLUDE_APPS apps in the Debug Toolbar without ever warning about
+            # them: instrument and track them, but register their labels as warn-suppressed so
+            # ``end_request`` skips their warnings (excluded apps must never fail tests).
+            from eagle.unused.report import register_warn_suppressed_labels
+
+            excluded_models = list(get_excluded_models())
+            register_tracked_models(excluded_models)
+            register_warn_suppressed_labels(model._meta.label for model in excluded_models)
+            for model in excluded_models:
+                make_model_eager(model)
+            logger.debug("Profiling %d excluded-app model(s) in the Debug Toolbar.", len(excluded_models))
+
         logger.debug("Patching ORM.")
         patch_orm()
         logger.debug("ORM patched.")

eagle/config.py

@@ -9,3 +9,16 @@ def is_enabled() -> bool:
         True when Eagle's instrumentation and warning emission should be active.
     """
     return bool(getattr(settings, "EAGLE_ENABLED", False))
+
+
+def include_excluded_apps_in_toolbar() -> bool:
+    """
+    Return True if the Debug Toolbar should also profile EAGLE_EXCLUDE_APPS apps.
+
+    When truthy, Eagle instruments the excluded apps too so their unused eager loads show up in
+    the panel -- but their warnings stay suppressed, so they never fail tests. Defaults to False.
+
+    Returns:
+        True when ``EAGLE_DEBUG_TOOLBAR_INCLUDE_EXCLUDED_APPS`` is set to a truthy value.
+    """
+    return bool(getattr(settings, "EAGLE_DEBUG_TOOLBAR_INCLUDE_EXCLUDED_APPS", False))

eagle/instrumentation/__init__.py

@@ -1,4 +1,5 @@
 __all__ = [
+    "get_excluded_models",
     "get_first_party_models",
     "is_instrumented",
     "make_contenttypes_eager",
@@ -11,4 +12,4 @@
 from eagle.instrumentation.models import make_model_eager
 from eagle.instrumentation.query import patch_orm
 from eagle.instrumentation.registry import is_instrumented, register_tracked_models
-from eagle.instrumentation.scope import get_first_party_models
+from eagle.instrumentation.scope import get_excluded_models, get_first_party_models

eagle/instrumentation/scope.py

@@ -67,14 +67,16 @@ def _matched_include_package(app_config: AppConfig, packages: tuple[str, ...]) -
     return None
 
 
-def get_first_party_models() -> Iterator[type[Model]]:
+def _iter_candidate_models() -> Iterator[tuple[type[Model], bool]]:
     """
-    Yield all non-proxy models from first-party and explicitly included third-party apps.
+    Yield ``(model, is_excluded)`` for every non-proxy model Eagle could instrument.
 
-    Respects EAGLE_EXCLUDE_APPS to omit specific apps from instrumentation.
+    A model is a candidate when its app is first-party or matches
+    ``EAGLE_THIRD_PARTY_INCLUDE_APPS``; ``is_excluded`` reflects whether
+    ``EAGLE_EXCLUDE_APPS`` would normally drop it from instrumentation.
 
     Yields:
-        Django model classes eligible for Eagle tracking.
+        ``(model, is_excluded)`` pairs for each candidate model.
     """
     roots = _dependency_roots()
     excluded = set(getattr(settings, "EAGLE_EXCLUDE_APPS", ()))
@@ -85,9 +87,38 @@ def get_first_party_models() -> Iterator[type[Model]]:
         if not first_party and include_package is None:
             continue
         exclusion_key = app_config.label if first_party else f"{include_package}.{app_config.label}"
-        if exclusion_key in excluded:
-            continue
+        is_excluded = exclusion_key in excluded
         for model in app_config.get_models():
             if model._meta.proxy:
                 continue
+            yield model, is_excluded
+
+
+def get_first_party_models() -> Iterator[type[Model]]:
+    """
+    Yield all non-proxy models from first-party and explicitly included third-party apps.
+
+    Respects EAGLE_EXCLUDE_APPS to omit specific apps from instrumentation.
+
+    Yields:
+        Django model classes eligible for Eagle tracking.
+    """
+    for model, is_excluded in _iter_candidate_models():
+        if not is_excluded:
+            yield model
+
+
+def get_excluded_models() -> Iterator[type[Model]]:
+    """
+    Yield the non-proxy candidate models that EAGLE_EXCLUDE_APPS omits from instrumentation.
+
+    These are normally skipped entirely. The Debug Toolbar can opt to instrument them anyway --
+    without ever emitting warnings -- via ``EAGLE_DEBUG_TOOLBAR_INCLUDE_EXCLUDED_APPS``, so a
+    migrating project can still see what eager loads its excluded apps waste.
+
+    Yields:
+        Django model classes from candidate apps that ``EAGLE_EXCLUDE_APPS`` excludes.
+    """
+    for model, is_excluded in _iter_candidate_models():
+        if is_excluded:
             yield model