diff --git a/spec/features/perf-layer-loading/README.md b/spec/features/perf-layer-loading/README.md index 02177e93..95d734d1 100644 --- a/spec/features/perf-layer-loading/README.md +++ b/spec/features/perf-layer-loading/README.md @@ -1,11 +1,18 @@ # Feature: Image Layer & Model Run Loading Performance -**Status:** draft +**Status:** in-progress **Author:** prbatero **Date:** 2026-08-03 -**Target Release:** TBD **Priority:** P1 +## Contents + +- [Summary](#summary) +- [Motivation](#motivation) +- [Success Criteria](#success-criteria) +- [HASTE Components Affected](#haste-components-affected) +- [Document Index](#document-index) + ## Summary The HASTE UI takes too long to load a project's image layers and their associated @@ -15,8 +22,8 @@ patterns in the `GetProjectDetails` API path, fully-sequential (never paralleliz blob I/O in `hastelib`, full-container blob scans without prefix filtering, and a UI that re-fetches the entire project — every model of every layer — every 20 seconds while re-rendering the whole component tree. This spec catalogs the verified -bottlenecks and lays out a phased plan to make layer/run loading fast and roughly -constant-time regardless of project size. +bottlenecks and lays out a phased plan to remove sequential amplification, bound +process-wide concurrency, and avoid duplicate or idle refresh work. ## Motivation @@ -33,21 +40,25 @@ constant-time regardless of project size. - [ ] `GET GetProjectDetails?includeModels=True` for a 50-layer / ~5-models-per-layer project returns in **< 1.5s p95** (from a current baseline measured in Phase 0). -- [ ] Backend storage round-trips for that request drop from **O(layers × models)** - (~600 for the 50×5 case) to **O(1) small constant** (≤ ~6 partition reads). +- [x] Logical data-layer calls for that request drop from **603 to 7**. This metric + counts processor operations, not Azure REST transactions; Blob downloads still + scale with the number of returned records. +- [x] Aggregate blocking Blob I/O is capped by one process-wide executor (16 workers + by default), including concurrent HTTP requests. - [ ] UI time-to-interactive for the project page is **< 2s p95** on the same project - and no longer scales linearly with layer count. -- [ ] Background refresh no longer refetches unchanged data or re-renders the full - tree; idle CPU and network on an open project page drop measurably. + and no longer scales linearly with layer count. Current production-bundle + observation is **2.12 s** with 58 ms post-response rendering. +- [ ] Idle projects no longer poll and fresh conditional requests avoid storage via a + 15-second process-local cache. Measure the resulting browser CPU/network delta. ## HASTE Components Affected | Component | Impact | |---|---| | `hastelib/src/hastegeo/core/data_layer/` | Add prefix-scoped listing, parallel/metadata-only reads, fix double-deserialize | -| `hastelib/src/hastegeo/core/processors/` | `MetadataProcessor` filtered load + optional request-scoped cache | -| `hastelib/src/hastegeo/core/artifact_storage/` | Parallelize multi-blob fetch | -| `api/hastefuncapi/` | Rewrite `GetProjectDetails` layer loop; add cache headers | +| `hastelib/src/hastegeo/core/processors/` | Keyed project-details loader and batch metadata reads | +| `hastelib/src/hastegeo/core/artifact_storage/` | Bounded, atomic multi-blob fetch | +| `api/hastefuncapi/` | Thin `GetProjectDetails` route; process-local TTL cache and ETags | | `api/hastefuncqueues/` | Parallelize independent loads; fix N+1 label lookup; batch saves | | `ui/src/Components/` | Smart polling, memoization, split context, lazy model expansion | @@ -67,4 +78,5 @@ constant-time regardless of project size. | [impact-analysis.md](impact-analysis.md) | Risk, blast radius, backward compat | draft | | [test-plan.md](test-plan.md) | Benchmark harness & regression coverage | draft | | [results.md](results.md) | **Phase 0 measured baseline** (603 round-trips, 20.8 s API, 40.3 s UI TTI @ 50×5) | done | +| [user-stories.md](user-stories.md) | User outcomes, acceptance criteria, and agent assignments | in-progress | | [tools/](tools/) | Seed + benchmarks: `phase0_baseline.py`, `bench_api_http.py`, `ui_bench.cjs`; `docker/docker-compose.perf.yml` overlay | done | diff --git a/spec/features/perf-layer-loading/design.md b/spec/features/perf-layer-loading/design.md index 56d3f49c..0146409e 100644 --- a/spec/features/perf-layer-loading/design.md +++ b/spec/features/perf-layer-loading/design.md @@ -2,26 +2,34 @@ ## Overview -Collapse the `GetProjectDetails` request from `O(layers × models)` sequential storage -round-trips to a small constant by (1) hoisting redundant reads out of loops, -(2) batch-loading each metadata type **once per partition** and joining in memory, -(3) parallelizing genuinely independent I/O, and (4) pushing filtering into the data -layer. Add HTTP caching so unchanged data isn't recomputed. On the UI, stop the -20-second full-project poll from thrashing the render tree by adding change-detection, -context splitting, and memoization. Backend fixes are sequenced first because they are -the dominant cost and are transparent to the UI. +Collapse `GetProjectDetails` from hundreds of sequential logical data-layer calls to +seven top-level operations. Preserve storage-key semantics for legacy records, overlap +independent I/O behind one process-wide concurrency budget, and reuse Azure SDK clients. +A bounded process-local cache deduplicates concurrent requests and serves fresh ETag +checks without storage work. Underlying Blob GET transactions still scale with the +records returned; a materialized project view would be required to make those constant. + +## Contents + +- [Architecture](#architecture) +- [API Design](#api-design) +- [Internal Interfaces](#internal-interfaces-hastegeo) +- [Caching](#caching) +- [UI Design](#ui-design-phase-4) +- [Configuration](#configuration) +- [Observability](#observability) ## Architecture ``` ┌──────────────┐ GetProjectDetails ┌────────────────────┐ 1 read / type ┌──────────────┐ │ React UI │───────────────────────▶│ hastefuncapi │──────────────────▶│ Blob / Cosmos │ -│ Project.jsx │ (ETag / 304 aware) │ GetProjectDetails │ (batched+joined) │ data layer │ +│ Project.jsx │ (single-flight/ETag) │ GetProjectDetails │ keyed batch reads │ data layer │ └──────────────┘ └─────────┬──────────┘ └──────────────┘ ▲ smart poll (304 fast-path) │ uses │ ┌─────────▼──────────┐ - └── memoized rows, split context │ MetadataProcessor │ load_filtered() / - │ (+ load_map cache) │ load_all_from_partition (prefix) + └── active-job-only polling │ ProjectDetailsProc. │ load_map() / exact + │ + shared I/O budget │ metadata prefixes └────────────────────┘ ``` @@ -29,76 +37,51 @@ the dominant cost and are transparent to the UI. ### `GET /api/GetProjectDetails` — reworked internals (contract unchanged by default) -Same request/response shape by default, so the UI keeps working during rollout. New -**optional** query params enable the lighter paths incrementally: +The response shape remains unchanged. The only implemented query parameter is: | Param | Type | Default | Effect | |---|---|---|---| -| `includeModels` | bool | existing | unchanged | -| `includeArtifacts` | bool | `true` (compat) | when `false`, skip B2's per-model artifact/label expansion; return `modelCount` + model summaries only | -| `summary` | bool | `false` | return per-layer counts (`modelCount`, `labelProjectCount`, `validationLabelCount`) without nested `models[]` — the shape the list view actually needs | +| `includeModels` | bool | `false` | Include models, artifacts, and train-label URLs. | -**New response headers:** `ETag` (hash of the serialized payload) and -`Cache-Control: private, max-age=15`. On a conditional request whose `If-None-Match` -matches, return `304` with an empty body. +`summary` and `includeArtifacts` remain deferred. Do not send them until their response +contracts are implemented and tested. + +Response headers are `ETag`, `Cache-Control: private, max-age=`, and +`X-Haste-Cache`. A matching `If-None-Match` returns `304` with an empty body. Request +`Cache-Control: no-cache` or `max-age=0` forces a storage refresh before comparison. ### Reworked handler logic (replaces [function_app.py:534-638](../../../api/hastefuncapi/function_app.py#L534-L638)) ```python -# 1. Independent top-level reads in parallel -project, image_layers, models, label_projects = await asyncio.gather( - _load(PROJECT.load, project_id), - _load(IMAGELAYER.load_all_from_partition), - _load(MODEL.load_all_from_partition) if include_models else _none(), - _load(LABELS.load_all_from_partition), # ONCE, not per-layer (fixes B1) -) - -# 2. Index once, join in memory — no per-item I/O -models_by_layer = group_by(models or [], "imageLayerId") -labels_by_layer = index_by(label_projects, "imageLayerId") - -# 3. Validation + (optional) artifacts: batch in parallel, bounded concurrency -validation_by_layer = await gather_map( - {l["imageLayerId"]: VALIDATION.load for l in image_layers} -) # fixes B3 -if include_models and include_artifacts: - artifacts_by_model, labelsurl_by_model = await gather_models(models) # fixes B2 - -# 4. Assemble response purely in memory, then sort. +project = await ProjectDetailsProcessor(project_id, config).load(include_models) +payload = json.dumps(project) +etag = sha256(payload.encode()).hexdigest() ``` -Key point: every metadata **type** is read at most once per partition (a handful of -`load_all_from_partition` calls), plus one bounded-parallel batch for the per-key -`VALIDATION`/artifact reads. Total round-trips become a small constant + at most two -bounded-concurrency fan-outs, independent of `L × M` sequencing. +`ProjectDetailsProcessor` loads the project first as the `404` gate, loads layers, +labels, and models concurrently, then uses keyed `load_map` calls for validation and +artifacts. Keyed maps preserve the old storage-key joins even when legacy document +bodies omit optional `imageLayerId` or `modelId` fields. ## Internal Interfaces (hastegeo) | Module | Change | Signature | Purpose | |---|---|---|---| -| `core/processors/metadata.py` | **new** | `load_filtered(self, predicate: dict) -> list` | Filter in the data layer, not the caller (fixes H3, B1, Q2). Backed by prefix scan + property match. | -| `core/processors/metadata.py` | **new** | `load_map(self, keys: list[str], max_workers=8) -> dict` | Parallel multi-key load with bounded `ThreadPoolExecutor` (powers B2/B3 batches). | -| `core/data_layer/azure_blob_storage_data_layer.py` | **fix** | `load_all(..., name_starts_with=None)` | Always pass a prefix; add optional metadata-only listing (fixes H1). | -| `core/data_layer/azure_blob_storage_data_layer.py` | **fix** | parallel download loop | Bounded `ThreadPoolExecutor` in `load_all*` and artifact `fetch_artifact` (fixes H2). | -| `core/data_layer/azure_blob_storage_data_layer.py` | **fix** | remove double `json.loads` | Single-serialize on save; delete re-parse (fixes H4). | -| `core/blob.py` | **fix** | module-level `BlobServiceClient` singleton keyed by conn-string | Reuse client (fixes H5). | -| `core/artifact_storage/azure_blob_artifact_storage.py` | **fix** | parallel `fetch_artifact` | Same `ThreadPoolExecutor` pattern (fixes H2). | - -`load_map` sketch: - -```python -def load_map(self, keys, max_workers=8): - from concurrent.futures import ThreadPoolExecutor - out = {} - with ThreadPoolExecutor(max_workers=max_workers) as ex: - futs = {ex.submit(self._safe_load, k): k for k in keys} - for fut in futs: - out[futs[fut]] = fut.result() # None on FileNotFoundError - return out -``` - -The API layer calls `await asyncio.to_thread(processor.load_map, keys)` so one thread -hop wraps the whole bounded-parallel batch instead of one hop per key. +| `core/processors/project_details.py` | **new** | `ProjectDetailsProcessor.load(include_models)` | Own storage orchestration and pure response assembly. | +| `core/processors/metadata.py` | **new** | `load_map(keys, max_workers=None)` | Prefer one native Cosmos/PostgreSQL query; otherwise use bounded keyed loads. | +| `core/processors/metadata.py` | **new** | `load_filtered(predicate)` | Explicit client-side partition scan and property filter; not a server-side optimization. | +| `core/utils/parallel.py` | **new** | `parallel_map(...)` | One process-wide worker budget shared by requests and storage operations. | +| `core/utils/async_cache.py` | **new** | `AsyncTTLCache` | Bounded TTL cache with per-key single-flight loading. | +| `core/utils/blob.py` | **fix** | `get_blob_service_client(...)` | Reuse top-level clients and connection pools by credential target. | +| Blob/local/Data Lake/Cosmos layers | **fix** | exact metadata matching | Longest known type wins, so `model` cannot consume `model_catalog`. | +| Blob artifact storage | **fix** | atomic bounded download | Validate paths and replace temporary files only after successful download. | + +## Caching + +The cache key is `(projectId, includeModels)`. It stores only successful serialized +responses, shares an in-flight load among concurrent callers, and evicts least-recently +used entries beyond the configured bound. It is process-local and therefore does not +provide cross-instance coherence. After the TTL, active-job polling refreshes storage. ## Queue design (Phase 3) @@ -118,15 +101,12 @@ hop wraps the whole bounded-parallel batch instead of one hop per key. > latency and stop request pile-up. Memoization/virtualization (U3/U5) help poll-time > re-render churn, not first paint; `summary` payload mode (B6) is a minor win. -- **Single-flight + cancellation (U7):** guard `fetchProjectDetails` so only one request - is in flight at a time — track the in-flight promise and reuse it, and use an - `AbortController` to cancel a superseded request on re-fetch/unmount. Removes the - duplicate concurrent load call (measured ~2× latency amplification). -- **Smart poll (U1, U5):** send `If-None-Match` with the last `ETag`; on `304`, do - nothing. Otherwise shallow-compare (or hash-compare) before `setComponentState`. - **Do not start a poll while a request is in flight** (the measured response can exceed - the 20 s interval); pause polling when the tab is hidden (`document.visibilityState`) - and consider an interval that backs off toward the observed response time. +- **Single-flight + cancellation (U7, implemented):** one keyed in-flight promise is + shared; a different project or real unmount aborts superseded work. +- **Smart poll (U1, implemented):** send `If-None-Match`; return before state updates on + `304`; do not poll while hidden, in flight, or when all known jobs are terminal. +- **Route assets (implemented):** route modules use `React.lazy`; Azure Maps control, + drawing, and swipe assets load in order only before a map-dependent route mounts. - **Split context (U2):** move volatile fields (`isLoading`, `dialogParams`, `bootstrapBreakpoint`) into a separate context/provider so a loading toggle doesn't re-render layer rows. @@ -142,24 +122,32 @@ hop wraps the whole bounded-parallel batch instead of one hop per key. | Config Key | Type | Default | Where | Description | |---|---|---|---|---| -| `HASTE_METADATA_LOAD_WORKERS` | int | 8 | App Settings / `local.settings.json` | Bounded parallelism for `load_map` fan-out | -| `HASTE_PROJECTDETAILS_CACHE_SECONDS` | int | 15 | App Settings | `Cache-Control: max-age` for `GetProjectDetails` | +| `HASTE_BLOB_DOWNLOAD_WORKERS` | int (1–64) | 16 | App Settings | Process-wide blocking I/O thread budget. | +| `HASTE_METADATA_LOAD_WORKERS` | int (1–64) | 8 | App Settings | Per-map limit within the global I/O budget. | +| `HASTE_ARTIFACT_DOWNLOAD_WORKERS` | int (1–64) | 8 | App Settings | Per-artifact limit within the global budget. | +| `HASTE_PROJECTDETAILS_CACHE_SECONDS` | int (0–300) | 15 | App Settings | Process-local freshness and response `max-age`. | +| `HASTE_PROJECTDETAILS_CACHE_ENTRIES` | int (1–512) | 64 | App Settings | Maximum process-local project response entries. | | queue `batchSize` | int | 1 → TBD | `host.json` | Concurrent messages per instance (load-test gated) | ## Observability -- Log per-request storage round-trip **count** and total storage time in - `GetProjectDetails` (proves the O(N)→O(1) win and guards against regression). +- Log logical data-layer call count/time and cache hit/miss in `GetProjectDetails`. + These metrics do not count Azure REST transactions; use Azure Storage metrics or SDK + pipeline instrumentation for transaction/cost analysis. - Emit request duration to App Insights; add a synthetic 50-layer project to the perf test to track p95 over time. - Track queue depth and dequeue/poison counts when changing `host.json`. ## Open Questions -- [ ] Does the storage backend (Blob vs Cosmos, via `UnifiedDataLayer`) support a - server-side predicate for `load_filtered`, or is prefix-scan + in-layer filter - the best available? Determines how much H3 actually saves on the Blob path. +- [x] `load_filtered` is an explicit partition-scan fallback and does not reduce + transferred records. A future backend query API is required for server filtering. - [ ] Is the double-serialization (H4) safe to fix in place, or are there existing blobs already double-encoded that need a migration/back-compat read path? + *(Partially answered 2026-08-20: legacy double-encoded blobs must be assumed, so + Phase 2 shipped a tolerant read (`_read_blob_content` parses twice when the first + parse yields a string) and deferred the save-side fix behind a migration. New + constraint from the data-publishing merge: `save()` now stamps `_index_metadata` + blob metadata, which any save-path rewrite must preserve.)* - [ ] Acceptable default polling interval / should we move to server-push (SignalR) instead of polling for run-status updates? diff --git a/spec/features/perf-layer-loading/findings.md b/spec/features/perf-layer-loading/findings.md index 980bd205..c9416112 100644 --- a/spec/features/perf-layer-loading/findings.md +++ b/spec/features/perf-layer-loading/findings.md @@ -1,5 +1,23 @@ # Findings: Verified Performance Bottlenecks +## Contents + +- [Cost Model](#cost-model-why-it-scales) +- [HTTP API](#backend--apihastefuncapifunction_apppy) +- [Core Library](#backend--hastelibsrchastegeocore) +- [Queue Workers](#queue--apihastefuncqueuesfunction_apppy) +- [UI](#ui--uisrc) +- [Measured Priority](#measured-priority-adjustments-phase-0) + +> **Baseline terminology:** the Phase 0 counter records logical data-layer calls, not +> Azure REST transactions. The N+1 findings and latency measurements remain valid, but +> one partition call can contain a listing plus many Blob downloads. + +> **Implementation status (2026-09-01):** B1–B5, B7, H2, and H5 are addressed on the +> cumulative branch. B6, server-side filtering, a materialized project view, and queue +> configuration remain open. UI single-flight, ETag handling, in-flight/visibility +> guards, and active-job-only polling are implemented. + Each finding was confirmed by reading the referenced code. Severity reflects impact on the reported symptom (slow layer/run loading that scales with layer count). @@ -13,7 +31,7 @@ For a project with **L** image layers and an average of **M** models per layer, + L × 1 (LABELS full-partition scan — once PER layer, see B1) + L × 1 (VALIDATION load per layer) + L × M × 2 (MODEL_ARTIFACTS load + TRAIN_LABELS export per model) -= 3 + 2L + 2LM sequential, blocking storage round-trips += 3 + 2L + 2LM sequential, blocking logical data-layer calls ``` For L=50, M=5 that is **~603 sequential round-trips**, each an `await asyncio.to_thread(...)` diff --git a/spec/features/perf-layer-loading/impact-analysis.md b/spec/features/perf-layer-loading/impact-analysis.md index 50c0a4c7..b58e70e6 100644 --- a/spec/features/perf-layer-loading/impact-analysis.md +++ b/spec/features/perf-layer-loading/impact-analysis.md @@ -1,5 +1,15 @@ # Impact Analysis: Image Layer & Model Run Loading Performance +## Contents + +- [Scope](#scope-of-change) +- [Azure Services](#azure-service-impact) +- [Dependencies](#dependency-analysis) +- [Risks](#risk-assessment) +- [Performance](#performance-impact-measured-baseline--target) +- [Security](#security-impact) +- [Rollback](#rollback-assessment) + ## Scope of Change | Component | Path | Type | Severity | @@ -16,7 +26,7 @@ | Service | Change | Cost Impact | |---|---|---| -| Blob Storage / Cosmos | Far fewer read round-trips per `GetProjectDetails`; prefix-scoped listings | **Lower** transaction count & egress (especially with 20s poll × N clients) | +| Blob Storage / Cosmos | Fewer repeated scans and overlapped keyed reads | Lower transaction count than baseline, but still proportional to returned Blob records | | Azure Functions | Lower per-request CPU/wall-time; `gather` uses more threads briefly per request | Net **lower** consumption; watch thread-pool sizing under load | | Queue Storage | `batchSize`>1 raises concurrency per instance | Neutral–lower; validate scale behavior | @@ -41,7 +51,7 @@ | Risk | Likelihood | Impact | Mitigation | |---|---|---|---| | H4 double-serialize fix breaks reads of legacy blobs | med | high | Keep tolerant read (try single, fall back to double) during transition; migrate lazily on next save | -| `asyncio.gather` + `ThreadPoolExecutor` fan-out exhausts Functions thread pool under concurrency | med → **confirmed relevant** | med | **Phase 0 evidence:** two *concurrent* `GetProjectDetails` calls already ~2× the latency (38 s vs 21 s) — the worker contends today. Bound workers via `HASTE_METADATA_LOAD_WORKERS`, cap total in-flight, and fix the UI single-flight (U7) so fewer concurrent requests hit the API; load-test the fan-out under ≥2 concurrent requests | +| Concurrent fan-out exhausts Functions threads | low after mitigation | med | One process-wide executor caps blocking I/O at `HASTE_BLOB_DOWNLOAD_WORKERS`; UI/API single-flight removes duplicate same-key work. Multi-request load testing remains required. | | `batchSize`>1 changes ordering/poison behavior | med | med | Gate behind load test; keep separate from Phase 1; revert via config | | Cache headers serve stale run-status to UI | low | med | Short `max-age` (≤15s); UI still change-detects; runs move to terminal states, not backwards | | UI context split / memo introduces render regressions | med | low | Ship incrementally; visual + interaction QA per component | @@ -50,14 +60,15 @@ Phase 0 baselines (50×5, Azurite/dev — lower bound; see [results.md](results.md)): -- **API latency:** `GetProjectDetails` **20.8 s p50 / 21.8 s p95** → target **< 1.5 s**; - round-trips **603 → ≤ ~6 + 2 bounded fan-outs**. +- **API latency:** `GetProjectDetails` **20.8 s p50 / 21.8 s p95** baseline; + hardened clean fixture **1.85 s / 2.00 s uncached** and **9.1 / 12.2 ms cached**. +- **Logical data-layer calls:** **603 → 7**. This is not an Azure transaction count. - **UI time-to-interactive:** **40.3 s** (large) → target **< 2 s**. TTI is API-bound (~2 s render over the API call), so the backend fix drives most of this; the UI single-flight/poll guards remove the ~2× amplification and request pile-up. -- **Backend load:** the 20 s poll currently re-issues the full 603-round-trip call - (measured 36.5 s per poll) — worse, response > interval so polls overlap. Fixing the - poll guard + `304` collapses idle per-open-project storage transactions dramatically. +- **Backend load:** idle projects no longer poll; active-job polls do not overlap. + Fresh cache hits use zero data-layer calls. Active polls after cache expiry still read + storage because no materialized project version exists. - **Queue throughput:** higher with `batchSize`>1; training/inference lose partition scans. ## Security Impact diff --git a/spec/features/perf-layer-loading/plan.md b/spec/features/perf-layer-loading/plan.md index dbc45281..d9f36f1d 100644 --- a/spec/features/perf-layer-loading/plan.md +++ b/spec/features/perf-layer-loading/plan.md @@ -4,22 +4,34 @@ Sequenced by impact-per-risk. Phase 0 establishes a baseline so every later phas measured, not guessed. Backend before UI: the backend N+1 is the dominant cost and its fixes are transparent to the UI. Each phase is independently shippable. +## Contents + +- [Phase 0: Baseline and Instrumentation](#phase-0-baseline--instrumentation--done-2026-08-03) +- [Phase 1: Backend API Hot Path](#phase-1-backend-api-hot-path-hastefuncapi--highest-roi) +- [Phase 2: Core Library](#phase-2-core-library-hastelib--enables-phase-1-batchfilter) +- [Phase 3: Queue Workers](#phase-3-queue-workers-hastefuncqueues--throughput) +- [Phase 4: UI](#phase-4-ui-uisrc--perceived-performance) +- [Phase 5: Integration and Validation](#phase-5-integration--validation) +- [Milestones](#milestones) +- [Agent Summary](#agent-summary) +- [Open Questions](#open-questions) + ## Phase 0: Baseline & Instrumentation — DONE (2026-08-03) **Goal:** Make the problem measurable before changing it. | Task | Agent | Dependencies | Ref | Status | |---|---|---|---|---| -| Add opt-in round-trip counter + timing (`hastelib/.../utils/perf.py`, wired into `MetadataProcessor` reads + `GetProjectDetails` via `HASTE_PERF`) | `backend-dev` | — | B1–B4 | **done** | +| Add opt-in logical data-layer counter + timing (`hastelib/.../utils/perf.py`, wired into `MetadataProcessor` reads + `GetProjectDetails` via `HASTE_PERF`) | `backend-dev` | — | B1–B4 | **done** | | Seed script for synthetic L×M project (`tools/seed_synthetic_project.py`) | `backend-dev` | — | cost-model | **done** | -| Capture baseline round-trips + payload (`tools/phase0_baseline.py` → `results.md`) | `backend-dev` | above | success-criteria | **done** | +| Capture baseline data-layer calls + payload (`tools/phase0_baseline.py` → `results.md`) | `backend-dev` | above | success-criteria | **done** | | HTTP latency harness for running stack (`tools/bench_api_http.py`) | `backend-dev` | above | success-criteria | **done** | | Capture real API p50/p95 latency against running stack (Docker + Azurite) | `backend-dev` | stack up + seed | success-criteria | **done** | | Compose overlay + reproducible run (`docker/docker-compose.perf.yml`) | `backend-dev` | — | — | **done** | | Record UI time-to-interactive + poll cost (Playwright, `tools/ui_bench.cjs`) | `ui` | stack up | U1 | **done** | **Exit Criteria:** -- [x] Baseline round-trips (33 / 243 / **603**) + payload captured in +- [x] Baseline logical calls (33 / 243 / **603**) + payload captured in [results.md](results.md) and `test-plan.md`; per-op breakdown quantifies B1/B2. - [x] Real API latency captured (large **20.8 s p50 / 21.8 s p95** on Azurite; storage = 13.4 s of that). Confirms the reported symptom. @@ -31,39 +43,71 @@ fixes are transparent to the UI. Each phase is independently shippable. **Goal:** Collapse `GetProjectDetails` to a constant number of storage reads. No API contract change; UI untouched. +Implemented on branch `prbatero/feat/performance-improvements-phase1`. + | Task | Agent | Dependencies | Ref | Status | |---|---|---|---|---| -| Hoist `LABELS.load_all_from_partition` out of the layer loop (load once) | `backend-dev` | P0 | B1 | not-started | -| Parallelize top-level reads (project/layers/models/labels) with `asyncio.gather` | `backend-dev` | P0 | B4 | not-started | -| Batch per-model artifacts + `labelsUrl` via `load_map` (bounded parallel) | `backend-dev` | Ph2 `load_map` | B2 | not-started | -| Batch per-layer VALIDATION via `load_map` | `backend-dev` | Ph2 `load_map` | B3 | not-started | -| Apply the same hoist to `GenerateProjectStats` | `backend-dev` | B1 | B5 | not-started | -| Add `ETag` + `Cache-Control` + `304` handling to `GetProjectDetails` | `backend-dev` | — | B7 | not-started | -| Add optional `summary` / `includeArtifacts=false` response modes | `backend-dev` | above | B6 | not-started | +| Hoist `LABELS.load_all_from_partition` out of the layer loop (load once) | `backend-dev` | P0 | B1 | **done** | +| Parallelize top-level reads (project/layers/models/labels) with `asyncio.gather` | `backend-dev` | P0 | B4 | **done** | +| Batch per-model artifacts with keyed `load_map` + `labelsUrl` with one `list_keys` | `backend-dev` | Ph2 | B2 | **done** | +| Batch per-layer VALIDATION with keyed `load_map` | `backend-dev` | Ph2 | B3 | **done** | +| Apply the same hoist to `GenerateProjectStats` | `backend-dev` | B1 | B5 | **done** | +| Add bounded process-local single-flight cache + `ETag`/`304` handling | `backend-dev` | — | B7 | **done** | +| Add optional `summary` / `includeArtifacts=false` response modes | `backend-dev` | above | B6 | deferred (payload minor per Phase 0 — see findings); if revived, build on the publishing feature's `load_page`/`_index_metadata` metadata-indexed listing rather than a new listing path | + +> **Design note:** the final implementation uses keyed `load_map` for artifacts and +> validation, preserving legacy storage-key joins. Blob performs bounded per-key GETs; +> Cosmos and PostgreSQL issue one native query per map. The seven-call count is a +> logical data-layer metric, not an Azure transaction count. **Exit Criteria:** -- [ ] Round-trip count for 50×5 project drops from ~600 to ≤ ~6 + 2 bounded fan-outs. -- [ ] `GetProjectDetails` p95 < 1.5s on the synthetic project. -- [ ] Existing UI still works unchanged (contract preserved). +- [x] Logical data-layer calls for a 50×5 project drop from **603 → 7**. +- [ ] Uncached `GetProjectDetails` is **1.85 s p50 / 2.00 s p95** on the clean + 50×5 Azurite fixture; the `<1.5 s p95` target is not yet met. +- [x] Fresh process-local cache hits are **9.1 ms p50 / 12.2 ms p95** and perform + zero logical data-layer calls. +- [x] Existing UI still works unchanged (contract preserved; verified via UI bench — + TTI 40.3 s → 5.5 s). See [results.md](results.md). ## Phase 2: Core library (`hastelib`) — enables Phase 1 batch/filter **Goal:** Give the API layer the primitives it needs and remove data-layer waste. -(Ships alongside Phase 1; `load_map` is a prerequisite for B2/B3.) +(Ships alongside Phase 1.) + +> **Done in Phase 1:** added `list_identifiers` (metadata-only, no-download listing) +> to the abstract/blob/local-FS/unified layers, `MetadataProcessor.list_keys` + +> `build_url`, and a `check_exists=False` fast path on `get_file_remote_path` — these +> are what let B2 drop the export N+1. `load_map`/`load_filtered`, the `load_all` +> prefix fix (H1), parallel download loops (H2), double-serialize fix (H4), and +> `BlobServiceClient` reuse (H5) remain for a dedicated Phase 2 pass. + +Implemented on branch `prbatero/feat/performance-improvements-phase2`. | Task | Agent | Dependencies | Ref | Status | |---|---|---|---|---| -| Add `MetadataProcessor.load_map(keys, max_workers)` (bounded `ThreadPoolExecutor`) | `backend-dev` | — | H2 | not-started | -| Add `MetadataProcessor.load_filtered(predicate)` | `backend-dev` | — | H3 | not-started | -| Pass `name_starts_with` prefix in `load_all`; add metadata-only listing | `backend-dev` | — | H1 | not-started | -| Parallelize download loops in `load_all*` and `fetch_artifact` | `backend-dev` | — | H2 | not-started | -| Fix double-serialization on save; drop redundant `json.loads` | `backend-dev` | migration Q | H4 | not-started | -| Reuse module-level `BlobServiceClient` (keyed by conn-string) | `backend-dev` | — | H5 | not-started | -| Unit tests in `hastelib/tests/` for new methods + parity of old behavior | `backend-dev` | all above | — | not-started | +| Add `MetadataProcessor.load_map(keys, max_workers)` with native Cosmos/PostgreSQL queries and bounded fallback | `backend-dev` | — | H2 | **done** | +| Add `MetadataProcessor.load_filtered(predicate)` | `backend-dev` | — | H3 | **done** | +| Parallelize download loops through one process-wide bounded executor; make artifact files atomic | `backend-dev` | — | H2 | **done** | +| Reuse module-level `BlobServiceClient` keyed by connection target | `backend-dev` | — | H5 | **done** | +| Unit, contract, API, and local-storage regression tests | `backend-dev` | above | — | **done** | +| Exact metadata-type matching (`model` must not include `model_catalog`) | `backend-dev` | — | H1 | **done** | +| Pass `name_starts_with` prefix in `load_all` | `backend-dev` | — | H1 | deferred — `load_all` is cross-partition (varying `{partition}/` prefix), so no single prefix applies; `load_all_from_partition` already prefixes | +| Fix double-serialization on save; drop redundant `json.loads` | `backend-dev` | migration Q | H4 | deferred — kept tolerant read (`_read_blob_content` handles both encodings); save-side fix needs a data migration for legacy blobs **and** must preserve the `_index_metadata` blob metadata now attached on save (publishing) | + +> **Note:** `list_identifiers` / `list_keys` / `build_url` / `check_exists` were landed +> in Phase 1. H2's parallel downloads are a **production-latency** win (no measurable +> effect on Azurite/localhost where per-blob latency is ~1 ms — see results.md). +> +> **Post-rebase (2026-08-20):** main's data-publishing feature independently added a +> parallel blob loader (`_load_blob_names`, for `load_page`); consolidated it onto +> `_read_blob_content` + the shared `HASTE_BLOB_DOWNLOAD_WORKERS` policy so the +> download/deserialize logic (incl. the H4 double-parse tolerance) lives in one place. **Exit Criteria:** -- [ ] `hastelib` unit tests pass; `load_filtered`/`load_map` covered. -- [ ] No container-wide scans without a prefix remain in `load_all`. +- [x] New core utilities have 100% statement/branch coverage; cross-backend read + contracts, response parity, cache, and failure paths are covered. +- [x] Clean 50×5 HTTP fixture returns 50 layers, 250 models/artifacts, and correct + validation counts with seven logical data-layer calls. ## Phase 3: Queue workers (`hastefuncqueues`) — throughput @@ -88,13 +132,20 @@ contract change; UI untouched. > UI work by impact: **single-flight guard (U7) + poll guard (U1) first** (halve > effective latency, stop pile-up), then context split (U2); memoization/virtualization > (U3/U5) and `summary` payload (B6) are lower-value follow-ups. +> +> **Post-rebase (2026-08-20):** main's `PublishedDatasets.jsx` is an in-repo precedent +> for smart polling — 5 s interval that runs *only while active items exist*, with a +> ref to avoid stale closures; consider extracting a shared hook both pages use. Main +> also reworked `AppContext.jsx`/`App.jsx` and many Phase 4 target components, so plan +> U2/U3 against current main, not the original file inventory in findings.md. | Task | Agent | Dependencies | Ref | Status | |---|---|---|---|---| -| Single-flight guard + `AbortController` on `fetchProjectDetails` (dedupe/cancel concurrent calls) | `ui` | — | U7 | not-started | -| Smart poll: send `If-None-Match`, handle `304`, skip `setState` if unchanged | `ui` | Ph1 B7 | U1 | not-started | -| Poll guard: don't fire while a request is in flight (response can exceed 20 s interval) | `ui` | — | U1 | not-started | -| Pause polling when tab hidden; make interval configurable/adaptive | `ui` | — | U1 | not-started | +| Single-flight guard + `AbortController` on `fetchProjectDetails` (dedupe/cancel concurrent calls) | `ui` | — | U7 | **done** | +| Smart poll: send `If-None-Match`, handle `304`, skip `setState` if unchanged | `ui` | Ph1 B7 | U1 | **done** | +| Poll guard: don't fire while a request is in flight | `ui` | — | U1 | **done** | +| Pause polling when hidden and when no active jobs; configurable/adaptive interval | `ui` | — | U1 | **partial** (guards done; interval remains 20 s) | +| Route-level code splitting; load Azure Maps SDK only on map routes | `ui` | — | U6-adjacent | **done** | | Split volatile fields out of `AppContext` into a lighter provider | `ui` | — | U2 | not-started | | `React.memo` / `useMemo` / `useCallback` on `LayerRow`/`ModelRow`/`ModelRowMobile` | `ui` | — | U3 | not-started | | Lazy-render model rows on expand; evaluate list virtualization | `ui` | — | U5 | not-started | @@ -103,8 +154,10 @@ contract change; UI untouched. | Thumbnail/overview-first in Visualizer; defer full-res second map | `ui` | — | U6 | not-started | **Exit Criteria:** -- [ ] Project page time-to-interactive < 2s p95 on the synthetic project. -- [ ] Idle open-project CPU/network drops measurably (no full refetch/re-render per poll). +- [ ] Project page time-to-interactive < 2s p95 on the synthetic project. Current + production-bundle observation: **2.12 s**; one initial request, 58 ms render. +- [x] Terminal-job project issued **zero polls** during a 26-second idle window. +- [x] Active-job project issued one non-overlapping conditional poll (`304`). ## Phase 5: Integration & Validation @@ -123,7 +176,7 @@ contract change; UI untouched. | Milestone | Deliverable | |---|---| | Baseline captured | Phase 0 numbers in test-plan.md | -| Backend hot path fixed | `GetProjectDetails` O(1) reads, cache headers | +| Backend hot path fixed | Seven logical calls, bounded Blob I/O, cache headers | | Core lib primitives | `load_map` / `load_filtered` / prefix scans merged | | Queue optimized | N+1 removed, config load-tested | | UI responsive | smart poll + memoization shipped | diff --git a/spec/features/perf-layer-loading/results.md b/spec/features/perf-layer-loading/results.md index 756851a1..8bb5de3a 100644 --- a/spec/features/perf-layer-loading/results.md +++ b/spec/features/perf-layer-loading/results.md @@ -1,4 +1,116 @@ -# Phase 0 Baseline — Measured Results +# Measured Results + +## Contents + +- [Post-review Hardening](#post-review-hardening-2026-09-01) +- [Phase 2](#phase-2--core-library-hastelib) +- [Phase 1](#phase-1--measured-after-backend-hot-path) +- [Phase 0](#phase-0-baseline--measured-results) +- [Reproduction](#how-to-reproduce) +- [Targets](#targets-to-beat-from-readmemdsuccess-criteria) +- [Environment](#environment-notes) + +> **Metric correction (2026-09-01):** the `HASTE_PERF` counter measures logical +> data-layer method calls, not Azure Storage REST transactions. A partition or keyed +> Blob operation can issue one listing plus many GET requests. Historical `33/243/603` +> and `7` values below are logical calls. + +## Post-review hardening (2026-09-01) + +Clean synthetic project: 50 layers, 5 models per layer, 20 label records per layer, +10 validation labels per layer. Docker Functions + Azurite, forced-refresh requests, +10 measured iterations after 2 warmups: + +| Path | Logical calls | p50 | p95 | Payload | +|---|---:|---:|---:|---:| +| Forced refresh (`Cache-Control: no-cache`) | 7 | **1.85 s** | **2.00 s** | 166.2 KB | +| Fresh process-local cache (`--allow-cache`) | 0 | **9.1 ms** | **12.2 ms** | 166.2 KB | + +Correctness check: exactly 50 layers, 250 models, 250 non-null artifact records, and +`validationLabelCount == 10` for every layer. The `<1.5 s` uncached p95 target remains +open. Cache numbers are process-local and do not imply cross-instance coherence. + +Production UI bundle against the same API fixture: + +| Browser path | TTI | Initial calls | Post-response render | 26 s poll window | +|---|---:|---:|---:|---:| +| Terminal jobs | **2.12 s** | **1** | 58 ms | **0 calls** | +| One active job | **2.27 s** | **1** | 69 ms | **1 call**, HTTP `304` | + +The original page loaded Azure Maps control/drawing scripts and styles globally. In the +test environment those four CDN requests blocked `DOMContentLoaded` for about 14 s, +including on the non-map project page. Route-level code splitting plus on-demand Azure +Maps loading reduced the main JS chunk from 1.49 MB to about 120 KB and moved +`DOMContentLoaded` to 68 ms. A map-route browser smoke test confirmed `atlas`, drawing, +and `SwipeMap` are available after the lazy loader runs. + +The `<2 s` production TTI target remains open by about 120 ms on the terminal fixture; +the uncached API call (1.93 s in that run) is now the dominant component. + +## Phase 2 — Core library (hastelib) + +**Date:** 2026-08-03 · **Branch:** `prbatero/feat/performance-improvements-phase2` + +Phase 2 removes data-layer waste and adds batch primitives. Delivered: + +- **H2 — parallel per-blob downloads** in `load_all_from_partition` / `load_all` + (blob layer) and `fetch_artifact` through one process-wide bounded executor. +- **H5 — `BlobServiceClient` reuse**: a process-wide `lru_cache`d factory in + `utils/blob.py` (was created per call in `download_blob_to_tempfile` / + `read_blob_range`). +- **Batch primitives**: `MetadataProcessor.load_map` (parallel per-key load, perf + counter propagated into worker threads via `perf.bind`) and `load_filtered` + (partition scan + in-process predicate), backed by cross-backend regression tests. + +**Measured (50×5, Azurite/dev):** `GetProjectDetails` correctness unchanged (50 +layers, 250 models/artifacts, counts correct); logical calls remain **7**; latency +**~2.08 s p50 — flat vs Phase 1**. + +The Azurite benchmark validates the control flow, not production network performance. +Real Azure concurrency must be tuned and measured under representative request load. + +**Net:** Phase 1 removed the sequential logical-call amplification; Phase 2 bounds +aggregate concurrency and improves production I/O behavior. Underlying Blob transactions +remain proportional to the number of records downloaded. + +The earlier statement that Phase 2 necessarily makes real Azure sub-second was a +projection, not a measurement, and is superseded by the measured table above. + +## Phase 1 — Measured After (backend hot path) + +**Date:** 2026-08-03 · **Branch:** `prbatero/feat/performance-improvements-phase1` + +Phase 1 loads every metadata type **once per partition** and joins in memory +(`asyncio.gather` for the independent reads), and replaces the 250 per-model +`TRAIN_LABELS` export `exists()` round-trips with **one** blob-name listing + local +SAS URL construction. Same measurement harnesses as Phase 0. + +| Fixture | Logical calls | API p50 | UI TTI | | before → after | +|---|---|---|---|---|---| +| small (5×2) | 33 → **7** | 0.70 → **0.20 s** | 3.10 → **2.97 s** | | | +| medium (20×5) | 243 → **7** | 6.18 → **0.84 s** | 12.36 → **3.63 s** | | | +| large (50×5) | 603 → **7** | 20.77 → **2.02 s** | 40.27 → **5.50 s** | | **API 10.3× · TTI 7.3×** | + +- **Logical calls are seven** regardless of project size. Blob REST transactions still + scale with the downloaded records. The seven operations are project plus + {imagelayer, labels, validation, model, artifacts} + partition reads + 1 train-labels key listing, the last six run concurrently. +- **`X-Haste-Storage-Ms` (5.5 s) now exceeds `X-Haste-Wall-Ms` (2.0 s)** on large, + confirming the reads run in parallel (sum of per-call time > wall time). +- The large poll call dropped **36.5 s → 2.05 s**. +- Correctness verified: 50 layers, 5 models each, all 250 artifacts joined, + `labelProjectCount`/`validationLabelCount` correct, and the `labelsUrl` build path + confirmed (populates a SAS URL when a train-labels blob exists, else null). +- The after-payload is *larger* (166 KB vs 82 KB — the seed was enriched for the UI + run), so the latency win is conservative. + +**Historical Phase 1 state:** UI TTI (5.5 s) exceeded the API call because the page +fired `GetProjectDetails` twice and did not consume ETags. The post-review hardening +measurements above supersede that state. + +--- + +## Phase 0 Baseline — Measured Results **Date:** 2026-08-03 **Branch:** `prbatero/feat/performance-improvements` @@ -8,15 +120,15 @@ sequence ([function_app.py:534-638](../../../api/hastefuncapi/function_app.py#L5 with `HASTE_PERF` instrumentation on. Models seeded **without** `labelsUrl` (worst-case N+1 that triggers the per-model `TRAIN_LABELS` export). -## Headline metric — storage round-trips per request +### Headline metric — logical data-layer calls per request -| Fixture | Layers × Models | **Round-trips** | Payload | Formula `3 + L·(2M+2)` | +| Fixture | Layers × Models | **Logical calls** | Payload | Formula `3 + L·(2M+2)` | |---|---|---|---|---| -| small | 5 × 2 | **33** | 5.8 KB | 33 ✓ | -| medium | 20 × 5 | **243** | 49.2 KB | 243 ✓ | -| large | 50 × 5 | **603** | 122.4 KB | 603 ✓ | +| small | 5 × 2 | **33** | 9.1 KB | 33 ✓ | +| medium | 20 × 5 | **243** | 82.5 KB | 243 ✓ | +| large | 50 × 5 | **603** | 205.8 KB | 603 ✓ | -Round-trip counts match the derived cost formula exactly, confirming the replay is +Logical-call counts match the derived formula, confirming the replay is faithful to the handler and that cost scales as **O(layers × models)**. ## Per-op breakdown (large, 50 × 5) @@ -96,7 +208,7 @@ would shift the constants but not the O(layers × models) scaling or the amplifi ## How to reproduce ```bash -# Headline round-trip baseline (no infra needed): +# Headline logical-call baseline (no infra needed): PYTHONPATH=hastelib/src python3 \ spec/features/perf-layer-loading/tools/phase0_baseline.py @@ -112,7 +224,7 @@ python3 spec/features/perf-layer-loading/tools/bench_api_http.py \ | Metric | Baseline (large, measured) | Target | |---|---|---| -| Round-trips / request | **603** | ≤ ~6 + 2 bounded fan-outs | +| Logical calls / request | **603** | 7 | | API p50 / p95 latency | **20.8 s / 21.8 s** (Azurite) | < 1.5 s | | Payload (default shape) | 82.8 KB | smaller via `summary` mode | @@ -121,8 +233,8 @@ python3 spec/features/perf-layer-loading/tools/bench_api_http.py \ - Backend: Azurite blob emulator on `localhost` (compose). Real Azure Blob per-op latency is higher, so these numbers are a **lower bound** on production. - Host: Docker Desktop on Apple Silicon; the amd64 API image runs emulated, inflating - the non-storage (CPU) portion somewhat. The storage-round-trip count (603) is - hardware-independent and is the primary target metric. + the non-storage (CPU) portion somewhat. Logical-call counts are hardware-independent; + use Azure metrics for actual transaction counts. - Reproduce: `docker compose -f docker/docker-compose.yml -f docker/docker-compose.perf.yml up -d hastefuncapi api-proxy`, seed via `seed_synthetic_project.py` inside the `hastefuncapi` container, then run `bench_api_http.py` from the host. diff --git a/spec/features/perf-layer-loading/test-plan.md b/spec/features/perf-layer-loading/test-plan.md index 7f381818..e33e13dd 100644 --- a/spec/features/perf-layer-loading/test-plan.md +++ b/spec/features/perf-layer-loading/test-plan.md @@ -1,5 +1,15 @@ # Test Plan: Image Layer & Model Run Loading Performance +## Contents + +- [Strategy](#strategy) +- [Benchmark Fixture](#benchmark-fixture) +- [Metrics](#metrics-captured-per-fixture-size) +- [Baseline](#baseline-capture-phase-0--measured-2026-08-03) +- [Regression Tests](#correctness--regression-tests) +- [Load Test](#load-test-phase-3-gate) +- [Exit Gate](#exit-gate) + ## Strategy Performance work must be **measured, not asserted**. Every phase compares against the @@ -21,8 +31,9 @@ populates the local Docker Compose storage emulator (Azurite) with these shapes. | Metric | How | Target (Large) | |---|---|---| -| `GetProjectDetails` p50 / p95 latency | timed API calls (warm) | p95 < 1.5s | -| Storage round-trip count per request | Phase 0 counter log | ≤ ~6 + 2 bounded fan-outs | +| Uncached `GetProjectDetails` p50 / p95 | HTTP harness sends `Cache-Control: no-cache` | p95 < 1.5s | +| Warm process-cache p50 / p95 | HTTP harness with `--allow-cache` | tracked separately | +| Logical data-layer calls | `HASTE_PERF` headers | 7; not an Azure transaction metric | | Response payload size | `Content-Length` | tracked; smaller in `summary` mode | | UI time-to-interactive | DevTools performance trace | p95 < 2s | | Idle open-project network/CPU over 60s | DevTools, no data change | no full refetch; `304` on poll | @@ -32,13 +43,13 @@ populates the local Docker Compose storage emulator (Azurite) with these shapes. Full detail in [results.md](results.md). Captured via `tools/phase0_baseline.py` (real code, real seeded data, local FS backend). -| Fixture | round-trips | API p50 | API p95 | payload | **UI TTI** | +| Fixture | logical calls | API p50 | API p95 | payload | **UI TTI** | |---|---|---|---|---|---| | Small (5×2) | **33** | 0.70 s | 0.71 s | 4.2 KB | 3.10 s | | Medium (20×5) | **243** | 6.18 s | 6.71 s | 33.2 KB | 12.36 s | | Large (50×5) | **603** | **20.77 s** | **21.78 s** | 82.8 KB | **40.27 s** | -Round-trips match `3 + L·(2M+2)` exactly. Large breakdown: 301 `load`, 52 +Logical calls match `3 + L·(2M+2)` exactly. Large breakdown: 301 `load`, 52 `load_all_from_partition` (50 redundant `LABELS` scans = B1), 250 `export` (B2). API latency measured via `tools/bench_api_http.py`; UI TTI via `tools/ui_bench.cjs` (Playwright vs the real app). TTI is API-bound (~2 s render over the API call) and the @@ -48,23 +59,22 @@ Azurite, amd64 emulation, Vite dev mode). See [results.md](results.md). ## Correctness / regression tests ### Backend (`hastelib/tests/`) -- `load_map`: returns one entry per key; `None` for missing (parity with old - per-key `try/except FileNotFoundError`); respects `max_workers`. -- `load_filtered`: returns the same subset the old "load all + Python filter" produced, - for `imageLayerId` predicate. -- `load_all` with prefix: identical results to prior no-prefix scan for a given - partition; no cross-partition leakage. -- H4 tolerant read: correctly decodes both legacy double-encoded and new - single-encoded blobs. -- `BlobServiceClient` reuse: same client instance returned for same conn-string. +- [x] `load_map`: duplicate/missing keys, worker bounds, native query and fallback paths. +- [x] `load_filtered`: non-empty predicates and missing-field semantics. +- [x] Exact metadata matching and partition isolation across file/blob/list paths. +- [x] H4 tolerant read for legacy double-encoded and current JSON. +- [x] `BlobServiceClient` reuse by connection target. +- [x] Shared executor aggregate cap, ordering, nesting, cancellation, and exceptions. +- [x] Atomic artifact download, traversal rejection, and partial-file cleanup. +- [x] Cosmos, Data Lake, PostgreSQL, Blob, and local read-contract coverage. ### API (`hastefuncapi`) -- `GetProjectDetails` default response is **byte-for-byte equivalent** (post-sort) to - the pre-refactor response for Small/Medium/Large fixtures (golden-file compare). -- `summary` mode omits `models[]` but keeps counts. -- `includeArtifacts=false` skips artifact expansion, keeps `modelCount`. -- `ETag` stable across identical requests; `If-None-Match` match ⇒ `304` empty body. -- `GenerateProjectStats` parity after the B5 hoist. +- [x] `GetProjectDetails` response assembly is byte-for-byte checked against a complete + expected fixture; a real local-storage test covers legacy key-only related records. +- [ ] Deferred: `summary` mode omits `models[]` but keeps counts. +- [ ] Deferred: `includeArtifacts=false` skips artifact expansion. +- [x] ETag/304, weak/list matching, cache hit/miss, refresh, failure retry, and keying. +- [x] `GenerateProjectStats` loads labels once and reports zero for unlabeled layers. ### Queue (`hastefuncqueues`) - Training trigger with `load_filtered` selects the same label project as before. @@ -73,13 +83,16 @@ Azurite, amd64 emulation, Vite dev mode). See [results.md](results.md). path exercised at `maxDequeueCount`. ### UI (`ui`) -- **Single-flight (U7):** initial project load issues **exactly one** `GetProjectDetails` - (not two) — assert via `tools/ui_bench.cjs` `getprojectdetails_calls_during_load == 1` - (production build) and a network spy; a superseded fetch is aborted. -- **Poll guard (U1):** no new poll fires while a request is in flight; assert no - overlapping `GetProjectDetails` when response time > interval. -- Poll receiving `304` (or byte-identical body) does **not** call `setComponentState` - (assert via spy/render-count). +- [x] **Single-flight (U7):** utility covers dedupe, supersession, abort, and retry; + browser benchmark remains the integration proof for exactly one initial request. +- [x] Poll guard checks visibility, in-flight state, and active jobs. +- [x] HTTP helper handles `304` without parsing a body; component returns before state + update. +- [x] Production browser: one initial call, zero terminal-project polls over 26 s. +- [x] Active browser: one conditional poll at 20 s, returning `304` without overlap. +- [x] Map-route smoke test: lazy loader provides `atlas`, drawing, and `SwipeMap`. +- [x] Route splitting: main JS 1.49 MB → about 120 KB; Azure Maps CDN assets no longer block + non-map routes. - `LayerRow`/`ModelRow` memoized: unchanged props ⇒ no re-render (React Profiler). - Expanding a layer renders its model rows; collapsed layers render none. - Edit/labeling helpers issue independent GETs concurrently (`Promise.all`). @@ -92,7 +105,25 @@ Azurite, amd64 emulation, Vite dev mode). See [results.md](results.md). ## Exit gate +### Current Validation (2026-09-01) + +- Focused regression matrix and final suite counts are refreshed during the final + validation pass. +- New performance core modules have 100% statement and branch coverage. +- Performance-stack API suite: **50 passed**; two server-managed-field cases are + owned by the independent security PR. Queue suite: **6 passed**. + UI suite: **121 passed**; production build passes. +- New UI utilities: 100% line, branch, and function coverage. +- Full `hastelib` suite: **581 passed**. The stale `ArtifactProcessor.zip` test was + replaced with an isolated test of the current fetch delegation contract. +- UI production build passes. Changed UI files have zero ESLint diagnostics; the + repository-wide lint command remains red from unrelated existing files. +- Clean 50×5 HTTP fixture: 50 layers, 250 models/artifacts, correct validation counts; + uncached 1.85 s p50 / 2.00 s p95, warm-cache 9.1 ms / 12.2 ms. +- Production project page: 2.12 s TTI, one initial request, 58 ms post-response render, + and zero terminal-project polls during 26 seconds. Target remains open. + - [ ] Large-fixture targets met and recorded in the baseline table (before/after). -- [ ] All correctness/parity tests green. +- [x] Feature-specific correctness/parity tests green. - [ ] `docker compose up` runs the full stack with the synthetic project without error. - [ ] CI (secret-scan, deploy-apps) passes. diff --git a/spec/features/perf-layer-loading/tools/phase0_baseline.py b/spec/features/perf-layer-loading/tools/phase0_baseline.py index 1748b322..f4918dc7 100644 --- a/spec/features/perf-layer-loading/tools/phase0_baseline.py +++ b/spec/features/perf-layer-loading/tools/phase0_baseline.py @@ -1,13 +1,12 @@ # Copyright (c) Microsoft Corporation. All rights reserved. # Licensed under the MIT License. -"""Phase 0 baseline: measure GetProjectDetails storage round-trips. +"""Phase 0 baseline: measure GetProjectDetails logical data-layer calls. Seeds synthetic projects (small / medium / large) into a temporary local-FS backend and replays the *exact* read sequence of the ``GetProjectDetails`` handler (``api/hastefuncapi/function_app.py`` lines ~534-638) with perf instrumentation on. -Reports, per size, the number of backend storage round-trips (the headline -success metric: ~O(layers x models) today -> O(1) target) plus a per-op breakdown +Reports, per size, the number of logical data-layer calls plus a per-op breakdown and a local-FS wall-clock (for relative comparison only; absolute latency p50/p95 must be measured against the running Azure/Docker stack via bench_api_http.py). @@ -133,24 +132,24 @@ def run(): rows.append({ "size": name, "layers": layers, "models_per": models_per, - "total_models": total_models, "round_trips": calls, + "total_models": total_models, "data_layer_calls": calls, "ops": ops, "payload_bytes": payload, "wall_p50_ms": round(statistics.median(walls), 1), "wall_p95_ms": round(max(walls), 1), }) - hdr = (f"{'size':7} {'layers':6} {'mdl/l':5} {'round_trips':11} " + hdr = (f"{'size':7} {'layers':6} {'mdl/l':5} {'data_calls':10} " f"{'payload_kb':10} {'localfs_p50ms':13} {'ops (by type)'}") print(hdr) print("-" * len(hdr)) for r in rows: print(f"{r['size']:7} {r['layers']:<6} {r['models_per']:<5} " - f"{r['round_trips']:<11} {r['payload_bytes']/1024:<10.1f} " + f"{r['data_layer_calls']:<10} {r['payload_bytes']/1024:<10.1f} " f"{r['wall_p50_ms']:<13} {r['ops']}") print() for r in rows: L, M = r["layers"], r["models_per"] - print(f" {r['size']}: round_trips={r['round_trips']} " + print(f" {r['size']}: data_layer_calls={r['data_layer_calls']} " f"formula 3 + L*(2M+2) = {3 + L * (2 * M + 2)}") diff --git a/spec/features/perf-layer-loading/tools/seed_synthetic_project.py b/spec/features/perf-layer-loading/tools/seed_synthetic_project.py index 05b17e7d..7551a382 100644 --- a/spec/features/perf-layer-loading/tools/seed_synthetic_project.py +++ b/spec/features/perf-layer-loading/tools/seed_synthetic_project.py @@ -59,6 +59,12 @@ def seed(project_id, layers, models, labels_per_layer, validation_per_layer, "projectId": project_id, "name": f"Layer {li}", "creationDate": _iso(li), + "userId": "bench@example.com", + "status": "Processed", + "statusMessage": "", + "currentStep": 10, + "totalSteps": 10, + "progressPct": 100, }, ) diff --git a/spec/features/perf-layer-loading/tools/ui_bench.cjs b/spec/features/perf-layer-loading/tools/ui_bench.cjs index 62a208c5..751d2327 100644 --- a/spec/features/perf-layer-loading/tools/ui_bench.cjs +++ b/spec/features/perf-layer-loading/tools/ui_bench.cjs @@ -59,6 +59,8 @@ const mockUser = { const page = await context.newPage(); const consoleErrors = []; + const requestTimeline = []; + const trackedRequests = new WeakMap(); page.on("console", (m) => { if (m.type() === "error") consoleErrors.push(m.text()); }); // Mock cheap bootstrap calls (not what we measure). @@ -74,15 +76,28 @@ const mockUser = { // Time every GetProjectDetails call (the real, expensive one). const gpd = []; - const starts = new Map(); + const requestRecords = new WeakMap(); page.on("request", (req) => { - if (req.url().includes("GetProjectDetails")) starts.set(req.url() + req.method() + Date.now(), Date.now()); + if (!req.url().includes("GetProjectDetails")) return; + const record = { url: req.url(), startedAt: Date.now(), ms: null }; + requestRecords.set(req, record); + gpd.push(record); }); page.on("requestfinished", async (req) => { if (!req.url().includes("GetProjectDetails")) return; const t = req.timing(); - // responseEnd is ms since request start (fetchStart); use it as duration. - gpd.push({ url: req.url(), ms: Math.round(t.responseEnd) }); + const record = requestRecords.get(req); + if (record) { + record.ms = Number.isFinite(t.responseEnd) + ? Math.round(t.responseEnd) + : Date.now() - record.startedAt; + } + }); + page.on("response", (response) => { + const record = requestRecords.get(response.request()); + if (!record) return; + record.status = response.status(); + record.cache = response.headers()["x-haste-cache"] ?? null; }); const observedApiOrigins = new Set(); @@ -92,6 +107,26 @@ const mockUser = { }); const t0 = Date.now(); + page.on("request", (req) => { + const url = req.url(); + if ( + req.resourceType() === "document" || + /\.auth\/me|GetUserById|PutUser|GetPublishingProviders/.test(url) + ) { + const record = { + resourceType: req.resourceType(), + path: new URL(url).pathname, + startedMs: Date.now() - t0, + finishedMs: null, + }; + trackedRequests.set(req, record); + requestTimeline.push(record); + } + }); + page.on("requestfinished", (req) => { + const record = trackedRequests.get(req); + if (record) record.finishedMs = Date.now() - t0; + }); await page.goto(`${UI}/project/${PROJECT}`, { waitUntil: "commit", timeout: 60000 }); // TTI: first image-layer row (seed names layers "Layer "). @@ -114,23 +149,72 @@ const mockUser = { } catch (e) { bodyText = ""; } try { await page.screenshot({ path: arg("shot", "/tmp/haste-uibench/shot.png"), fullPage: true }); } catch (e) {} - const initialGpdMs = gpd.length ? gpd[0].ms : null; - const gpdCountAfterLoad = gpd.length; + const interactiveAt = Date.now(); + const initialCalls = gpd.filter((call) => call.startedAt <= interactiveAt); + const initialGpdMs = initialCalls.length ? initialCalls[0].ms : null; + const initialGpdStartedMs = initialCalls.length + ? initialCalls[0].startedAt - t0 + : null; + const initialGpdFinishedMs = + initialGpdStartedMs !== null && initialGpdMs !== null + ? initialGpdStartedMs + initialGpdMs + : null; + const gpdCountAfterLoad = initialCalls.length; // Observe the 20s background poll. const pollStart = Date.now(); await page.waitForTimeout(POLL_WAIT_MS); - const pollCalls = gpd.slice(gpdCountAfterLoad); + const pollCalls = gpd.filter((call) => call.startedAt >= pollStart); const result = { project: PROJECT, time_to_interactive_ms: tti, + initial_getprojectdetails_started_ms: initialGpdStartedMs, initial_getprojectdetails_ms: initialGpdMs, + initial_getprojectdetails_finished_ms: initialGpdFinishedMs, + initial_getprojectdetails_status: initialCalls[0]?.status ?? null, + initial_getprojectdetails_cache: initialCalls[0]?.cache ?? null, + render_after_project_response_ms: + tti !== null && initialGpdFinishedMs !== null + ? Math.max(0, tti - initialGpdFinishedMs) + : null, getprojectdetails_calls_during_load: gpdCountAfterLoad, poll_window_ms: POLL_WAIT_MS, poll_getprojectdetails_calls: pollCalls.length, poll_getprojectdetails_ms: pollCalls.map((c) => c.ms), + poll_getprojectdetails: pollCalls.map((call) => ({ + ms: call.ms, + status: call.status ?? null, + cache: call.cache ?? null, + })), api_origins_observed: [...observedApiOrigins], + navigation_timing: await page.evaluate(() => { + const navigation = performance.getEntriesByType("navigation")[0]; + return navigation + ? { + responseEnd: Math.round(navigation.responseEnd), + domInteractive: Math.round(navigation.domInteractive), + domContentLoadedEventEnd: Math.round( + navigation.domContentLoadedEventEnd + ), + loadEventEnd: Math.round(navigation.loadEventEnd), + } + : null; + }), + slowest_resources: await page.evaluate(() => + performance + .getEntriesByType("resource") + .sort((left, right) => right.duration - left.duration) + .slice(0, 10) + .map((entry) => ({ + path: new URL(entry.name).pathname, + initiatorType: entry.initiatorType, + startTime: Math.round(entry.startTime), + duration: Math.round(entry.duration), + transferSize: entry.transferSize, + })) + ), + bootstrap_requests: requestTimeline, row_wait_error: rowError, body_text_sample: bodyText, console_errors: consoleErrors.slice(0, 4), diff --git a/spec/features/perf-layer-loading/user-stories.md b/spec/features/perf-layer-loading/user-stories.md new file mode 100644 index 00000000..fc28f213 --- /dev/null +++ b/spec/features/perf-layer-loading/user-stories.md @@ -0,0 +1,131 @@ +# User Stories: Image Layer and Model Run Loading Performance + +## Contents + +- [Personas](#personas) +- [Stories](#stories) +- [Agent Assignment Map](#agent-assignment-map) +- [Story Map](#story-map) +- [Out of Scope](#out-of-scope) + +## Personas + +| Persona | Description | Key Goal | +|---|---|---| +| Disaster Analyst | Reviews imagery layers and model results during response work | Open and refresh large projects without long blocking waits | +| ML Engineer | Monitors training, embedding, and inference runs | Receive current run status without duplicate requests | +| Operator | Runs HASTE on supported metadata backends | Bound resource use and diagnose storage cost accurately | + +## Stories + +### US-001: Load Large Projects Reliably + +**As a** disaster analyst, **I want** project layers and model runs loaded without +sequential N+1 storage waits, **so that** large assessments remain usable. + +**Priority:** P0 +**Components:** `hastefuncapi`, `hastelib` + +**Acceptance Criteria:** + +```gherkin +Given a project with 50 layers and 5 models per layer +When GetProjectDetails is requested with includeModels=true +Then the response contains the same layers, models, artifacts, counts, and ordering as the legacy endpoint +And the request uses seven logical data-layer operations +``` + +```gherkin +Given legacy artifact or validation documents without embedded join IDs +When project details are assembled +Then related records are joined by their storage identifiers +``` + +### US-002: Bound Backend Concurrency + +**As an** operator, **I want** concurrent metadata downloads to share one bounded +worker budget, **so that** requests cannot multiply thread usage without limit. + +**Priority:** P0 +**Components:** `hastelib` + +**Acceptance Criteria:** + +```gherkin +Given multiple concurrent storage map operations +When they execute in one Functions worker process +Then aggregate blocking I/O never exceeds HASTE_BLOB_DOWNLOAD_WORKERS +And invalid worker settings fail fast +``` + +### US-003: Refresh Without Duplicate Work + +**As an** ML engineer, **I want** project refreshes deduplicated and conditional, +**so that** polling does not pile up while jobs run. + +**Priority:** P0 +**Components:** `hastefuncapi`, `ui/src` + +**Acceptance Criteria:** + +```gherkin +Given two concurrent requests for the same project response +When the first request is still loading +Then the API and UI each share one in-flight operation +``` + +```gherkin +Given an unchanged fresh cached response +When the UI sends its ETag +Then the API returns 304 without storage work +And the UI does not update project state +``` + +```gherkin +Given a project with no active jobs or a hidden browser tab +When the poll interval elapses +Then no project-details request is started +``` + +### US-004: Preserve Backend Compatibility + +**As an** operator, **I want** the read contract consistent across configured metadata +backends, **so that** an optimization does not silently make Blob the only working path. + +**Priority:** P1 +**Components:** `hastelib` + +**Acceptance Criteria:** + +```gherkin +Given Blob, local filesystem, Cosmos DB, Data Lake, or PostgreSQL metadata storage +When project-related read primitives are called +Then each backend accepts the shared read signatures +And unsupported remote label URLs degrade to null rather than fail the project +``` + +## Agent Assignment Map + +| Story | Implementing Agent | Validating Agent | Notes | +|---|---|---|---| +| US-001 | `backend-dev` | `backend-validation` | Golden response and local-storage regression | +| US-002 | `backend-dev` | `backend-validation` | Concurrency and failure-path tests | +| US-003 | `backend-dev`, `ui` | `backend-validation`, `ui-validation` | API cache plus browser request guards | +| US-004 | `backend-dev` | `backend-validation` | Mocked backend contract tests | + +## Story Map + +| Priority | Story | Phase | Component | +|---|---|---|---| +| P0 | US-001 | Backend hot path | `hastefuncapi`, `hastelib` | +| P0 | US-002 | Core library | `hastelib` | +| P0 | US-003 | Cache and UI polling | `hastefuncapi`, `ui/src` | +| P1 | US-004 | Backend compatibility | `hastelib` | + +## Out of Scope + +- A materialized per-project response document and write-side invalidation. +- Cross-instance distributed caching. +- Queue concurrency configuration changes without load testing. +- `summary` and `includeArtifacts` response modes. +- SignalR or another server-push transport. \ No newline at end of file diff --git a/ui/index.html b/ui/index.html index 56132907..dd92091c 100644 --- a/ui/index.html +++ b/ui/index.html @@ -6,13 +6,6 @@ - - - - - - -
diff --git a/ui/src/Components/AppBody.jsx b/ui/src/Components/AppBody.jsx index 369f250b..8f9a640a 100644 --- a/ui/src/Components/AppBody.jsx +++ b/ui/src/Components/AppBody.jsx @@ -1,29 +1,43 @@ // Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT License. import { Route, Routes } from "react-router-dom"; -import { useContext } from "react"; +import { lazy, Suspense, useContext } from "react"; import Loading from "./OtherComponents/Loading"; -import Error404 from "./Error404"; -import Project from "./Project"; -import Projects from "./Projects"; -import ImageLayer from "./ImageLayer"; -import Home from "./Home"; -import LabelingTool from "./LabelingTool/LabelingTool"; -import BuildingValidation from "./BuildingValidation/BuildingValidation"; -import InteractiveLabeler from "./InteractiveLabeler/InteractiveLabeler"; -import Visualizer from "./Visualizer/Visualizer"; -import ModelCatalog from "./ModelCatalog"; -import PublishedDatasets from "./PublishedDatasets"; - -import AdminUsers from "./AdminUsers"; -import AdminSourceTypes from "./AdminSourceTypes"; -import AdminLabelingTool from "./AdminLabelingTool"; -import CreateEditImageLayerForm from "./CreateEditImageLayerForm"; -import HelpDocs from "./HelpDocs"; import PropType from "prop-types"; import { AppContext } from "../AppContext"; +import { loadAzureMaps } from "../util/azureMapsLoader"; + +const loadMapRoute = (importRoute) => () => + loadAzureMaps().then(() => importRoute()); + +const AdminLabelingTool = lazy(() => import("./AdminLabelingTool")); +const AdminSourceTypes = lazy(() => import("./AdminSourceTypes")); +const AdminUsers = lazy(() => import("./AdminUsers")); +const BuildingValidation = lazy( + loadMapRoute(() => import("./BuildingValidation/BuildingValidation")) +); +const CreateEditImageLayerForm = lazy( + loadMapRoute(() => import("./CreateEditImageLayerForm")) +); +const Error404 = lazy(() => import("./Error404")); +const HelpDocs = lazy(() => import("./HelpDocs")); +const Home = lazy(() => import("./Home")); +const ImageLayer = lazy(() => import("./ImageLayer")); +const InteractiveLabeler = lazy( + loadMapRoute(() => import("./InteractiveLabeler/InteractiveLabeler")) +); +const LabelingTool = lazy( + loadMapRoute(() => import("./LabelingTool/LabelingTool")) +); +const ModelCatalog = lazy(() => import("./ModelCatalog")); +const Project = lazy(() => import("./Project")); +const Projects = lazy(() => import("./Projects")); +const PublishedDatasets = lazy(() => import("./PublishedDatasets")); +const Visualizer = lazy( + loadMapRoute(() => import("./Visualizer/Visualizer")) +); const AppBody = ({ setModalComponent }) => { const { appParams } = useContext(AppContext); @@ -33,7 +47,7 @@ const AppBody = ({ setModalComponent }) => { return (
{appParams.isLoading && } - {routesReady && + {routesReady && }> {appParams.userRoles !== null && appParams.publishingEnabled && ( } /> )} @@ -103,7 +117,7 @@ const AppBody = ({ setModalComponent }) => { )} } /> - } + }
); }; diff --git a/ui/src/util/azureMapsLoader.js b/ui/src/util/azureMapsLoader.js new file mode 100644 index 00000000..99289d1e --- /dev/null +++ b/ui/src/util/azureMapsLoader.js @@ -0,0 +1,89 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +const MAP_CONTROL_CSS = + "https://atlas.microsoft.com/sdk/javascript/mapcontrol/3/atlas.min.css"; +const MAP_CONTROL_JS = + "https://atlas.microsoft.com/sdk/javascript/mapcontrol/3/atlas.min.js"; +const DRAWING_CSS = + "https://atlas.microsoft.com/sdk/javascript/drawing/1/atlas-drawing.min.css"; +const DRAWING_JS = + "https://atlas.microsoft.com/sdk/javascript/drawing/1/atlas-drawing.min.js"; +const SWIPE_JS = "/assets/js/azure-maps-swipe-map.min.js"; + +let loadPromise = null; + +function loadElement(documentRef, selector, createElement) { + const existing = documentRef.querySelector(selector); + if (existing?.dataset.loaded === "true") return Promise.resolve(); + + const element = existing || createElement(); + return new Promise((resolve, reject) => { + element.addEventListener( + "load", + () => { + element.dataset.loaded = "true"; + resolve(); + }, + { once: true } + ); + element.addEventListener( + "error", + () => { + element.remove(); + reject(new Error(`Unable to load Azure Maps asset: ${element.src || element.href}`)); + }, + { once: true } + ); + if (!existing) documentRef.head.appendChild(element); + }); +} + +function loadStylesheet(documentRef, href) { + return loadElement( + documentRef, + `link[data-azure-maps-href="${href}"]`, + () => { + const link = documentRef.createElement("link"); + link.rel = "stylesheet"; + link.href = href; + link.dataset.azureMapsHref = href; + return link; + } + ); +} + +function loadScript(documentRef, src) { + return loadElement( + documentRef, + `script[data-azure-maps-src="${src}"]`, + () => { + const script = documentRef.createElement("script"); + script.src = src; + script.async = true; + script.dataset.azureMapsSrc = src; + return script; + } + ); +} + +export function loadAzureMaps(documentRef = document) { + if (loadPromise) return loadPromise; + + loadPromise = Promise.all([ + loadStylesheet(documentRef, MAP_CONTROL_CSS), + loadStylesheet(documentRef, DRAWING_CSS), + ]) + .then(() => loadScript(documentRef, MAP_CONTROL_JS)) + .then(() => loadScript(documentRef, DRAWING_JS)) + .then(() => loadScript(documentRef, SWIPE_JS)) + .catch((error) => { + loadPromise = null; + throw error; + }); + return loadPromise; +} + +export function resetAzureMapsLoaderForTests() { + loadPromise = null; +} \ No newline at end of file diff --git a/ui/src/util/azureMapsLoader.test.js b/ui/src/util/azureMapsLoader.test.js new file mode 100644 index 00000000..6992d473 --- /dev/null +++ b/ui/src/util/azureMapsLoader.test.js @@ -0,0 +1,164 @@ +import test from "node:test"; +import assert from "node:assert/strict"; + +import { + loadAzureMaps, + resetAzureMapsLoaderForTests, +} from "./azureMapsLoader.js"; + +function fakeDocument({ failOnce = null } = {}) { + const elements = []; + const attempts = new Map(); + + function asset(element) { + return element.src || element.href; + } + + return { + elements, + head: { + appendChild(element) { + elements.push(element); + const name = asset(element); + attempts.set(name, (attempts.get(name) || 0) + 1); + queueMicrotask(() => { + const event = name === failOnce && attempts.get(name) === 1 + ? "error" + : "load"; + element.listeners.get(event)?.(); + }); + }, + }, + createElement(tagName) { + return { + tagName, + dataset: {}, + listeners: new Map(), + addEventListener(name, callback) { + this.listeners.set(name, callback); + }, + remove() { + const index = elements.indexOf(this); + if (index >= 0) elements.splice(index, 1); + }, + }; + }, + querySelector(selector) { + const match = selector.match(/data-azure-maps-(?:src|href)="(.+)"/); + return elements.find((element) => asset(element) === match?.[1]) || null; + }, + }; +} + +test.beforeEach(() => resetAzureMapsLoaderForTests()); + +test("loads styles, map control, drawing tools, and swipe in order", async () => { + const documentRef = fakeDocument(); + + await loadAzureMaps(documentRef); + + assert.deepEqual( + documentRef.elements.map((element) => element.src || element.href), + [ + "https://atlas.microsoft.com/sdk/javascript/mapcontrol/3/atlas.min.css", + "https://atlas.microsoft.com/sdk/javascript/drawing/1/atlas-drawing.min.css", + "https://atlas.microsoft.com/sdk/javascript/mapcontrol/3/atlas.min.js", + "https://atlas.microsoft.com/sdk/javascript/drawing/1/atlas-drawing.min.js", + "/assets/js/azure-maps-swipe-map.min.js", + ] + ); +}); + +test("deduplicates concurrent and completed loads", async () => { + const documentRef = fakeDocument(); + + const first = loadAzureMaps(documentRef); + const second = loadAzureMaps(documentRef); + assert.equal(first, second); + await first; + await loadAzureMaps(documentRef); + + assert.equal(documentRef.elements.length, 5); +}); + +test("reuses an existing loaded asset", async () => { + const documentRef = fakeDocument(); + const existing = documentRef.createElement("link"); + existing.href = + "https://atlas.microsoft.com/sdk/javascript/mapcontrol/3/atlas.min.css"; + existing.dataset.azureMapsHref = existing.href; + existing.dataset.loaded = "true"; + documentRef.elements.push(existing); + + await loadAzureMaps(documentRef); + + assert.equal( + documentRef.elements.filter((element) => element.href === existing.href) + .length, + 1 + ); +}); + +test("waits for an existing asset that is still loading", async () => { + const documentRef = fakeDocument(); + const existing = documentRef.createElement("link"); + existing.href = + "https://atlas.microsoft.com/sdk/javascript/mapcontrol/3/atlas.min.css"; + existing.dataset.azureMapsHref = existing.href; + documentRef.elements.push(existing); + + const loading = loadAzureMaps(documentRef); + queueMicrotask(() => existing.listeners.get("load")?.()); + await loading; + + assert.equal( + documentRef.elements.filter((element) => element.href === existing.href) + .length, + 1 + ); +}); + +test("removes a failed asset and allows retry", async () => { + const failedAsset = + "https://atlas.microsoft.com/sdk/javascript/mapcontrol/3/atlas.min.js"; + const documentRef = fakeDocument({ failOnce: failedAsset }); + + await assert.rejects(loadAzureMaps(documentRef), /Unable to load/); + assert.equal( + documentRef.elements.some((element) => element.src === failedAsset), + false + ); + + await loadAzureMaps(documentRef); + assert.equal( + documentRef.elements.filter((element) => element.src === failedAsset).length, + 1 + ); +}); + +test("reports a failed stylesheet URL", async () => { + const failedAsset = + "https://atlas.microsoft.com/sdk/javascript/mapcontrol/3/atlas.min.css"; + const documentRef = fakeDocument({ failOnce: failedAsset }); + + await assert.rejects(loadAzureMaps(documentRef), (error) => { + assert.equal( + error.message, + `Unable to load Azure Maps asset: ${failedAsset}` + ); + return true; + }); +}); + +test("uses the global document by default", async () => { + const documentRef = fakeDocument(); + globalThis.document = documentRef; + + try { + await loadAzureMaps(); + } finally { + delete globalThis.document; + } + + assert.equal(documentRef.elements.length, 5); +}); \ No newline at end of file