Feat/bounded entity linkage (STIT-573) by AlexAxthelm · Pull Request #200 · RMI/stitch

AlexAxthelm · 2026-07-29T13:15:18Z

Summary

Fixes STIT-573 — entity linkage failed at production data scale. The old pass (POST /api/v1/start) pulled the entire resource table into memory and ran the whole reconciliation synchronously in one request, so it blew up on the full GEM + WoodMac + regulator (ccr) dataset while working fine at dev/staging volumes (long time on OOM).

This reworks it into a bounded, per-resource match plus a background bulk job:

Matching is done one resource at a time — search the API for same-name candidates, confirm same country, emit a merge candidate — so peak memory is a single field's candidate block, independent of dataset size.
The full-dataset pass runs as a background job (202 + job_id, polled for status), so it's non-blocking and pollable at scale instead of holding a request open.

What changed

stitch-client

list_oil_gas_fields_page accepts server-side filters (q/name/country).
New iter_oil_gas_fields streaming iterator (bounded memory) and list_merge_candidates.

entity-linkage service

New matcher (matching.py): per-resource blocking on normalized name + country, producing merge candidates. Name matching widens with the API's case-insensitive q (ILIKE) and re-narrows client-side to preserve the exact casefold/strip blocking (a DB-side normalized-name match is a tracked follow-up).
New endpoints: POST /api/v1/oil-gas-fields/{id}/link (synchronous single-resource link); POST /api/v1/oil-gas-fields/link (launches a background linkage run — 202, returns job_id; 409 if one is already running); GET /api/v1/oil-gas-fields/link/status (polls run state + result summary).
In-memory background job manager (jobs.py), borrowed from the stitch-etl-poc start/status pattern.
Removed the old in-memory POST /api/v1/start pass.

frontend

EntityLinkagePage reworked to the async job flow (start → auto-poll status), reusing the ETL page's design via a shared StateBadge component.

Breaking change

POST /api/v1/start is removed — callers now get a 404. It was an internal pre-scale endpoint and the frontend has been migrated to the new endpoints; no external consumers are expected.

Testing

make check and CI green (lint, format, pytest, frontend unit tests, docker builds, lockfile, migration drift).
Unit coverage: matcher (blocking, country confirmation, min_length guard, dedup/skip), job manager (success/failure/concurrent-start), client (filters, streaming, list), async start/status/409/404, and the reworked frontend page.
Equivalence: the new bulk pass produces the same merge-candidate set as the old /start grouping on the seed data.
End-to-end scale run at ~100K resources (via the SEED_FAKER_POST_COUNT ladder in deployments/PERFORMANCE.md) — confirm the run completes with bounded memory at a volume where the old path failed.

AI assistance

Claude Code was used to explore the codebase, design the approach, implement the change, and write the tests.

Follow-ups (separate tickets)

DB-side normalized-name match, to retire the superset + client-side refilter.
Persisted job state (Redis/DB) before the service scales beyond one worker.
Bounded concurrency + a per-run read cache to speed up the full 100K run.

🤖 Generated with Claude Code

github-actions · 2026-07-29T13:20:05Z

CD summary `5a60ab0`

Frontend: https://witty-mushroom-017a3dc1e-200.westus2.1.azurestaticapps.net

Deployments (4)

service	url	fqdn
api	open	`pr-200-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io`
entity-linkage	open	`pr-200-entity-linkage.purplegrass-c07d0a94.westus2.azurecontainerapps.io`
frontend	https://witty-mushroom-017a3dc1e-200.westus2.1.azurestaticapps.net
stitch-llm	open	`pr-200-stitch-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io`

Database (1)

db_name	postgres_host	postgres_port	postgres_db
pr_200	`stitch-dev.postgres.database.azure.com`	`5432`	`pr_200`

Jobs (2)

job	image	postgres_db	api_url	auth_mode
db-migrations	`ghcr.io/rmi/stitch-api:pr-200@sha256:eb15abcfa986e436d06ada2d2502d8f59db5d03eaa6b4ea55bdb9620b2ff53ca`	`pr_200`
seed	`ghcr.io/rmi/stitch-seed:pr-200@sha256:917973f214efd3cb6a3ac71773ffbb7084b1febc841b54d454fbdc1a2f4d91ed`		`https://pr-200-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io/api/v1`	`stitch-client-bearer-token`

Images (4)

build_time	commit_time	git_sha	image	image_digest
2026-07-29T13:15:44Z	2026-07-29T13:15:18Z	`f137747`	`ghcr.io/rmi/stitch-api:pr-200`	`ghcr.io/rmi/stitch-api:pr-200@sha256:eb15abcfa986e436d06ada2d2502d8f59db5d03eaa6b4ea55bdb9620b2ff53ca`
2026-07-29T13:15:42Z	2026-07-29T13:15:18Z	`f137747`	`ghcr.io/rmi/stitch-entity-linkage:pr-200`	`ghcr.io/rmi/stitch-entity-linkage:pr-200@sha256:576a4b5d07abd76d7cbcd476d912da5cff47eb969bc12bc42b71a021936569ba`
2026-07-29T13:15:40Z	2026-07-29T13:15:18Z	`f137747`	`ghcr.io/rmi/stitch-seed:pr-200`	`ghcr.io/rmi/stitch-seed:pr-200@sha256:917973f214efd3cb6a3ac71773ffbb7084b1febc841b54d454fbdc1a2f4d91ed`
2026-07-29T13:15:43Z	2026-07-29T13:15:18Z	`f137747`	`ghcr.io/rmi/stitch-stitch-llm:pr-200`	`ghcr.io/rmi/stitch-stitch-llm:pr-200@sha256:444c958a2ec542d10a9f33fed95edef47102482bbf86bf043cd48469e9a5fd31`

Copilot

Pull request overview

This PR introduces a bounded-memory entity-linkage flow by adding per-resource and bulk “link” endpoints to the entity-linkage service, and extending the Stitch client to support server-side filtering + streaming iteration so production-scale runs don’t require materializing the full dataset in memory.

Changes:

Extend AsyncStitchClient (and the entity-linkage StitchApiClient wrapper) to support q/name/country filters, streaming iteration (iter_oil_gas_fields), and listing merge candidates.
Add bounded per-resource and bulk linkage endpoints (/oil-gas-fields/{id}/link and /oil-gas-fields/link) plus a new matching module to drive the linkage pass.
Refactor shared normalization/user-label helpers into entities.py and add unit/integration tests for the new behavior.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
PLAN.md	Design/approach document for bounded-memory linkage and success criteria.
packages/stitch-client/src/stitch/client/async_client.py	Adds filter params, streaming iterator, and `list_merge_candidates()`; adds `_expect_list()`.
packages/stitch-client/tests/test_async_client.py	Adds tests for filter param forwarding, streaming iteration, and `list_merge_candidates()`.
deployments/entity-linkage/src/stitch/entity_linkage/entities.py	Adds `normalize_name`, `normalize_country`, `user_label`, and new response models for link endpoints.
deployments/entity-linkage/src/stitch/entity_linkage/client.py	Mirrors client filters/streaming + merge-candidate listing in the entity-linkage wrapper.
deployments/entity-linkage/src/stitch/entity_linkage/matching.py	New bounded-matcher implementation and bulk driver.
deployments/entity-linkage/src/stitch/entity_linkage/routers/link.py	New API endpoints for per-resource and bulk linking with permission enforcement.
deployments/entity-linkage/src/stitch/entity_linkage/routers/start.py	Refactors to use shared helpers (`normalize_country`, `user_label`).
deployments/entity-linkage/src/stitch/entity_linkage/main.py	Mounts the new link router under `/api/v1`.
deployments/entity-linkage/tests/test_matching.py	Unit tests for matching/linking logic (fingerprints, dedupe, skip behavior).
deployments/entity-linkage/tests/test_link_api.py	API-level tests for new endpoints + auth/502 translation.
deployments/entity-linkage/tests/test_start.py	Updates tests to use refactored helper functions.
deployments/entity-linkage/README.md	Updates service docs to describe new endpoints and bounded-memory behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

github-actions · 2026-07-29T14:35:46Z

CD summary `6ffd2be`

Frontend: https://witty-mushroom-017a3dc1e-200.westus2.1.azurestaticapps.net

Deployments (4)

service	url	fqdn
api	open	`pr-200-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io`
entity-linkage	open	`pr-200-entity-linkage.purplegrass-c07d0a94.westus2.azurecontainerapps.io`
frontend	https://witty-mushroom-017a3dc1e-200.westus2.1.azurestaticapps.net
stitch-llm	open	`pr-200-stitch-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io`

Database (1)

db_name	postgres_host	postgres_port	postgres_db
pr_200	`stitch-dev.postgres.database.azure.com`	`5432`	`pr_200`

Jobs (1)

job	image	postgres_db
db-migrations	`ghcr.io/rmi/stitch-api:pr-200@sha256:4e3695660daa558199d0f86a93d1f3e9b8f3d1cb262ca357f09093cc7c6a030c`	`pr_200`

Images (4)

build_time	commit_time	git_sha	image	image_digest
2026-07-29T14:31:18Z	2026-07-29T14:30:53Z	`d838187`	`ghcr.io/rmi/stitch-api:pr-200`	`ghcr.io/rmi/stitch-api:pr-200@sha256:4e3695660daa558199d0f86a93d1f3e9b8f3d1cb262ca357f09093cc7c6a030c`
2026-07-29T14:31:12Z	2026-07-29T14:30:53Z	`d838187`	`ghcr.io/rmi/stitch-entity-linkage:pr-200`	`ghcr.io/rmi/stitch-entity-linkage:pr-200@sha256:578df8d3feaa18afc3a35fa80cd3b485f15e3369885f3319cbb692234103c794`
2026-07-29T14:31:18Z	2026-07-29T14:30:53Z	`d838187`	`ghcr.io/rmi/stitch-seed:pr-200`	`ghcr.io/rmi/stitch-seed:pr-200@sha256:76dcdf929ecee92fd69afdaf708142ff9c03f3701a9c49d9d6d102c9d2c94d29`
2026-07-29T14:31:14Z	2026-07-29T14:30:53Z	`d838187`	`ghcr.io/rmi/stitch-stitch-llm:pr-200`	`ghcr.io/rmi/stitch-stitch-llm:pr-200@sha256:7b2b0078e63b79be6442835abb77aa44aeaeaae0a6febae5eb0e620beb62bc92`

github-actions · 2026-08-04T16:30:27Z

CD summary `05fecbd`

Deployments (2)

service	url	fqdn
api	open	`pr-0200-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io`
stitch-llm	open	`pr-0200-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io`

Database (1)

db_name	postgres_host	postgres_port	postgres_db
pr_0200	`stitch-dev.postgres.database.azure.com`	`5432`	`pr_0200`

Jobs (2)

job	image	postgres_db	api_url	auth_mode
db-migrations	`ghcr.io/rmi/stitch-api:pr-0200@sha256:7247908ba0af4a9bfe7d19e1e342d8ab98fc18e31fc9ecee2ae029a3ab9b25d1`	`pr_0200`
seed	`ghcr.io/rmi/stitch-seed:pr-0200@sha256:da9ade27a9f520e18a3ba2695b36e537563aac4e05360b91b5c525acfedfd53c`		`https://pr-0200-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io/api/v1`	`stitch-client-bearer-token`

Images (3)

build_time	commit_time	git_sha	image	image_digest
2026-08-04T16:27:38Z	2026-08-04T16:27:23Z	`dc5b326`	`ghcr.io/rmi/stitch-api:pr-0200`	`ghcr.io/rmi/stitch-api:pr-0200@sha256:7247908ba0af4a9bfe7d19e1e342d8ab98fc18e31fc9ecee2ae029a3ab9b25d1`
2026-08-04T16:27:44Z	2026-08-04T16:27:23Z	`dc5b326`	`ghcr.io/rmi/stitch-seed:pr-0200`	`ghcr.io/rmi/stitch-seed:pr-0200@sha256:da9ade27a9f520e18a3ba2695b36e537563aac4e05360b91b5c525acfedfd53c`
2026-08-04T16:27:39Z	2026-08-04T16:27:23Z	`dc5b326`	`ghcr.io/rmi/stitch-stitch-llm:pr-0200`	`ghcr.io/rmi/stitch-stitch-llm:pr-0200@sha256:629339227f0bde27de0d600d005934e92f8c9c99a5d14a69992d20f2910a76f5`

github-actions · 2026-08-05T10:53:55Z

CD summary `294ef5a`

Database (1)

db_name	postgres_host	postgres_port	postgres_db
pr_0200	`stitch-dev.postgres.database.azure.com`	`5432`	`pr_0200`

Images (4)

build_time	commit_time	git_sha	image	image_digest
2026-08-05T10:52:40Z	2026-08-05T10:52:10Z	`57f08b0`	`ghcr.io/rmi/stitch-api:pr-0200`	`ghcr.io/rmi/stitch-api:pr-0200@sha256:a956f11a9ec966a1ba81acbd76fbaade48cff995dc24bbd5fd0624a4a2ee5700`
2026-08-05T10:52:38Z	2026-08-05T10:52:10Z	`57f08b0`	`ghcr.io/rmi/stitch-entity-linkage:pr-0200`	`ghcr.io/rmi/stitch-entity-linkage:pr-0200@sha256:b62d7647912f98cfe16f9d28d8d0977999284cd6e42064693e9a758673ee931f`
2026-08-05T10:52:33Z	2026-08-05T10:52:10Z	`57f08b0`	`ghcr.io/rmi/stitch-seed:pr-0200`	`ghcr.io/rmi/stitch-seed:pr-0200@sha256:72c19c7a9c46c8b0ba569b33b25c89d0a849eb3130b658246be60d1f905d4421`
2026-08-05T10:52:35Z	2026-08-05T10:52:10Z	`57f08b0`	`ghcr.io/rmi/stitch-stitch-llm:pr-0200`	`ghcr.io/rmi/stitch-stitch-llm:pr-0200@sha256:ff88e12e47c429daac6751af1553a2a181b53a04a590c6ccd61834086f9caced`

github-actions · 2026-08-05T12:20:10Z

CD summary `1b9f9a4`

Frontend: https://witty-mushroom-017a3dc1e-200.westus2.1.azurestaticapps.net

Deployments (4)

service	url	fqdn
api	open	`pr-0200-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io`
entity-linkage	open	`pr-0200-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io`
frontend	https://witty-mushroom-017a3dc1e-200.westus2.1.azurestaticapps.net
stitch-llm	open	`pr-0200-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io`

Database (1)

db_name	postgres_host	postgres_port	postgres_db
pr_0200	`stitch-dev.postgres.database.azure.com`	`5432`	`pr_0200`

Jobs (1)

job	image	postgres_db
db-migrations	`ghcr.io/rmi/stitch-api:pr-0200@sha256:d0aea2f6e217b2028b7809101895c6ee0656bf0502f9a10021afbcd42eeb50aa`	`pr_0200`

Images (4)

build_time	commit_time	git_sha	image	image_digest
2026-08-05T12:15:14Z	2026-08-05T12:14:42Z	`10f81dd`	`ghcr.io/rmi/stitch-api:pr-0200`	`ghcr.io/rmi/stitch-api:pr-0200@sha256:d0aea2f6e217b2028b7809101895c6ee0656bf0502f9a10021afbcd42eeb50aa`
2026-08-05T12:15:12Z	2026-08-05T12:14:42Z	`10f81dd`	`ghcr.io/rmi/stitch-entity-linkage:pr-0200`	`ghcr.io/rmi/stitch-entity-linkage:pr-0200@sha256:9c3a2f46d20709c90c523f47c2e29cc9a217caa2e332a4e1a937523ea2bc43c1`
2026-08-05T12:15:03Z	2026-08-05T12:14:42Z	`10f81dd`	`ghcr.io/rmi/stitch-seed:pr-0200`	`ghcr.io/rmi/stitch-seed:pr-0200@sha256:78c2c5de39e2f106339c3de575edf0eaae4eeca00a3be8095b7b4ac95dca95e3`
2026-08-05T12:15:02Z	2026-08-05T12:14:42Z	`10f81dd`	`ghcr.io/rmi/stitch-stitch-llm:pr-0200`	`ghcr.io/rmi/stitch-stitch-llm:pr-0200@sha256:8900ed73370f6bb748dbac5c31089f0d3c38f6c57102521fbf1dc0205e86a0a5`

github-actions · 2026-08-05T12:59:50Z

CD summary `a93a1ca`

Frontend: https://witty-mushroom-017a3dc1e-200.westus2.1.azurestaticapps.net

Deployments (4)

service	url	fqdn
api	open	`pr-0200-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io`
entity-linkage	open	`pr-0200-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io`
frontend	https://witty-mushroom-017a3dc1e-200.westus2.1.azurestaticapps.net
stitch-llm	open	`pr-0200-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io`

Database (1)

db_name	postgres_host	postgres_port	postgres_db
pr_0200	`stitch-dev.postgres.database.azure.com`	`5432`	`pr_0200`

Jobs (1)

job	image	postgres_db
db-migrations	`ghcr.io/rmi/stitch-api:pr-0200@sha256:6df47010856a025471c77fb7b49de8d53073cebcadfa8e07f44a6f70a513c4bd`	`pr_0200`

Images (4)

build_time	commit_time	git_sha	image	image_digest
2026-08-05T12:55:01Z	2026-08-05T12:54:37Z	`1d7e0ed`	`ghcr.io/rmi/stitch-api:pr-0200`	`ghcr.io/rmi/stitch-api:pr-0200@sha256:6df47010856a025471c77fb7b49de8d53073cebcadfa8e07f44a6f70a513c4bd`
2026-08-05T12:54:59Z	2026-08-05T12:54:37Z	`1d7e0ed`	`ghcr.io/rmi/stitch-entity-linkage:pr-0200`	`ghcr.io/rmi/stitch-entity-linkage:pr-0200@sha256:0d28ce629ba73e079afec86be1bbfc98b86bc039f44ff98fb35baadfb2f8c5fa`
2026-08-05T12:55:06Z	2026-08-05T12:54:37Z	`1d7e0ed`	`ghcr.io/rmi/stitch-seed:pr-0200`	`ghcr.io/rmi/stitch-seed:pr-0200@sha256:747098a3e55a773025710592c4da8be1e9fe39445ee6faabc5866e1374050277`
2026-08-05T12:54:56Z	2026-08-05T12:54:37Z	`1d7e0ed`	`ghcr.io/rmi/stitch-stitch-llm:pr-0200`	`ghcr.io/rmi/stitch-stitch-llm:pr-0200@sha256:f71e209bf2e6291f28fb7393eaec2515b281e10299c3fab69fcfd3c9139131f5`

Copilot

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated no new comments.

Suppressed comments (1)

deployments/entity-linkage/src/stitch/entity_linkage/matching.py:69

find_match_group_for_resource passes the raw seed.name into the API q filter. If the stored name has leading/trailing whitespace (e.g. " Ghawar "), q will include those spaces and the substring search can miss true same-name candidates that differ only by surrounding whitespace, undermining the intended normalize/strip blocking. Strip the seed name before using it as q.

    async for candidate in client.iter_oil_gas_fields(q=seed.name):

github-actions · 2026-08-05T13:41:31Z

CD summary `467c5c5`

Frontend: https://witty-mushroom-017a3dc1e-200.westus2.1.azurestaticapps.net

Deployments (4)

service	url	fqdn
api	open	`pr-0200-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io`
entity-linkage	open	`pr-0200-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io`
frontend	https://witty-mushroom-017a3dc1e-200.westus2.1.azurestaticapps.net
stitch-llm	open	`pr-0200-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io`

Database (1)

db_name	postgres_host	postgres_port	postgres_db
pr_0200	`stitch-dev.postgres.database.azure.com`	`5432`	`pr_0200`

Jobs (1)

job	image	postgres_db
db-migrations	`ghcr.io/rmi/stitch-api:pr-0200@sha256:1ef06c4428ea0c393eec0a4f6754113cd588c36725a101f608456ef33ae149c9`	`pr_0200`

Images (4)

build_time	commit_time	git_sha	image	image_digest
2026-08-05T13:37:09Z	2026-08-05T13:36:43Z	`2254534`	`ghcr.io/rmi/stitch-api:pr-0200`	`ghcr.io/rmi/stitch-api:pr-0200@sha256:1ef06c4428ea0c393eec0a4f6754113cd588c36725a101f608456ef33ae149c9`
2026-08-05T13:37:07Z	2026-08-05T13:36:43Z	`2254534`	`ghcr.io/rmi/stitch-entity-linkage:pr-0200`	`ghcr.io/rmi/stitch-entity-linkage:pr-0200@sha256:386d6055efac1fc8b84c8a9c438594d122d4992ec5bec278ddb6afe20e5bb794`
2026-08-05T13:37:08Z	2026-08-05T13:36:43Z	`2254534`	`ghcr.io/rmi/stitch-seed:pr-0200`	`ghcr.io/rmi/stitch-seed:pr-0200@sha256:9ff1828ed259f05fa7ad35a0715aba08dff414d89b9f5deb2bb89853c8e63551`
2026-08-05T13:37:08Z	2026-08-05T13:36:43Z	`2254534`	`ghcr.io/rmi/stitch-stitch-llm:pr-0200`	`ghcr.io/rmi/stitch-stitch-llm:pr-0200@sha256:ae35e85f0e56fb0161f7988d507ca28fdd94d66d9f27601b0e490f675dc9b73c`

Reuse the jobs infrastructure from the ETL repo.

github-actions · 2026-08-05T14:20:53Z

CD summary `090b45c`

Frontend: https://witty-mushroom-017a3dc1e-200.westus2.1.azurestaticapps.net

Deployments (4)

service	url	fqdn
api	open	`pr-0200-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io`
entity-linkage	open	`pr-0200-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io`
frontend	https://witty-mushroom-017a3dc1e-200.westus2.1.azurestaticapps.net
stitch-llm	open	`pr-0200-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io`

Database (1)

db_name	postgres_host	postgres_port	postgres_db
pr_0200	`stitch-dev.postgres.database.azure.com`	`5432`	`pr_0200`

Jobs (1)

job	image	postgres_db
db-migrations	`ghcr.io/rmi/stitch-api:pr-0200@sha256:7ce7fdd133fe9915b0af67e44a98303d4c9773d2c42c4573980c9d4264c045a9`	`pr_0200`

Images (4)

build_time	commit_time	git_sha	image	image_digest
2026-08-05T14:16:41Z	2026-08-05T14:16:13Z	`6b90ec5`	`ghcr.io/rmi/stitch-api:pr-0200`	`ghcr.io/rmi/stitch-api:pr-0200@sha256:7ce7fdd133fe9915b0af67e44a98303d4c9773d2c42c4573980c9d4264c045a9`
2026-08-05T14:16:36Z	2026-08-05T14:16:13Z	`6b90ec5`	`ghcr.io/rmi/stitch-entity-linkage:pr-0200`	`ghcr.io/rmi/stitch-entity-linkage:pr-0200@sha256:3400e01252a0a12a8e6098dad71b0955f166def916f74aacc696e65e1359db5f`
2026-08-05T14:16:45Z	2026-08-05T14:16:13Z	`6b90ec5`	`ghcr.io/rmi/stitch-seed:pr-0200`	`ghcr.io/rmi/stitch-seed:pr-0200@sha256:4701f5787fff83603ae6642d33f53a9a39d89818a5e7b6de17527cd93688904d`
2026-08-05T14:16:44Z	2026-08-05T14:16:13Z	`6b90ec5`	`ghcr.io/rmi/stitch-stitch-llm:pr-0200`	`ghcr.io/rmi/stitch-stitch-llm:pr-0200@sha256:24ed173e8e395d76520e6d783907374ae13ecd1e9e487bca3e20b1b1ce6097f8`

Copilot

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated no new comments.

Suppressed comments (2)

deployments/stitch-frontend/src/pages/EntityLinkagePage.jsx:214

The auto-poll uses setInterval to call an async fetchStatus() without awaiting it, so slow network/token fetches can cause overlapping in-flight polls (and unnecessary load). A setTimeout loop that schedules the next poll only after the previous one completes avoids request pile-ups.

  // Auto-poll while a run is active; stop once it reaches a terminal state.
  useEffect(() => {
    if (record?.state !== "running") return undefined;

    const id = setInterval(() => {
      fetchStatus();
    }, POLL_INTERVAL_MS);

    return () => clearInterval(id);
  }, [record?.state, fetchStatus]);

packages/stitch-client/tests/test_async_client.py:322

This test asserts an exact query-string (including parameter order). That’s brittle because URL param ordering can change even when the request is correct; comparing parsed params avoids false negatives.

    def handler(request: httpx.Request) -> httpx.Response:
        captured["query"] = request.url.query.decode("utf-8")
        return httpx.Response(200, json={"items": [], "total_pages": 1})

    client, raw_client = make_client(handler)

    await client.list_oil_gas_fields_page(q="Ghawar")

    assert captured["query"] == "page=1&page_size=50&q=Ghawar"

AlexAxthelm · 2026-08-05T17:36:58Z

@mbarlow12 , @jdhoffa Note to look at PR 220 as the test environment here. Noting that this is very slow in its current form, but the turtle is working

mbarlow12

Handful of small NB observations/suggestions. LGTM.

Given the problem description (and so it's recorded somewhere), it does seem like this really wants a direct db connection regardless of whether it lives as a separate service or rolled into stitch-api.

It also occured to me that we don't need to store merge candidates in the DB at all. The UI/UX would not change, and we essentially do one or more similarity searches to retrieve the candidates at call time (same matching logic found here) and present them to the user. Approval would then be a POST to the old merge endpoint, gated on permissions. Candidates not explicitly handled dissolve and become available for subsequent linkage searches. Having ephemeral merge candidates ensures that we never retain matches based on old/stale data. It also aligns more closely with the intended mechanics of REST and would let us drop some DB tables and server logic.. I realized that while we can present entity linkage to the user as a separate service, it's really a composition of fetching groups of similar resources then merging those resources.

Note: we can still retain the candidate/approval process but it becomes entirely human user-centric.

mbarlow12 · 2026-08-07T03:25:12Z

+    # (and vice versa), missing exactly the whitespace variants this blocking is
+    # meant to catch. ``seed_name`` is stripped + casefolded; ILIKE is already
+    # case-insensitive, so the casefold is harmless and the strip is what matters.
+    same_name_ids: set[int] = set()


NB: could initialize this as {resource_id} and drop 77.

mbarlow12 · 2026-08-07T03:32:21Z

+    *,
+    apply_merges: bool,
+    known_existing: set[str] | None,
+) -> tuple[bool, bool]:


NB: purely for readability, you could consider using a NamedTuple here:

class SubmitResult(NamedTuple): created: bool skipped: bool

Still a tuple at runtime but has object semantics in source/editors. Makes intent a little clearer at declaration.

mbarlow12 · 2026-08-07T03:41:18Z

+    """
+    if not apply_merges:
+        return (False, False)
+    if known_existing is not None and merge_fingerprint(resource_ids) in known_existing:


NB (I think): Do we guard against individual resource id overlap? Say [10, 20] come in and [15, 20] is already in the queue, how do we handle that? I may have missed how this is accounted for.

mbarlow12 · 2026-08-07T03:47:57Z

    bearer_token: str | None


+def normalize_name(name: str | None) -> str | None:


NB: if we're ok with different casing for country, we could collapse to a single normalize_str(...) function.

AlexAxthelm temporarily deployed to development July 29, 2026 13:15 — with GitHub Actions Inactive

AlexAxthelm temporarily deployed to development July 29, 2026 13:16 — with GitHub Actions Inactive

AlexAxthelm temporarily deployed to development July 29, 2026 13:17 — with GitHub Actions Inactive

AlexAxthelm temporarily deployed to development July 29, 2026 13:19 — with GitHub Actions Inactive

AlexAxthelm requested a review from Copilot July 29, 2026 14:29

Copilot started reviewing on behalf of AlexAxthelm July 29, 2026 14:30 View session

AlexAxthelm temporarily deployed to development July 29, 2026 14:31 — with GitHub Actions Inactive

AlexAxthelm temporarily deployed to development July 29, 2026 14:32 — with GitHub Actions Inactive

AlexAxthelm temporarily deployed to development July 29, 2026 14:33 — with GitHub Actions Inactive

Copilot AI reviewed Jul 29, 2026

View reviewed changes

AlexAxthelm temporarily deployed to development July 29, 2026 14:34 — with GitHub Actions Inactive

AlexAxthelm changed the title ~~Feat/bounded entity linkage~~ Feat/bounded entity linkage (STIT-573) Jul 29, 2026

AlexAxthelm temporarily deployed to development July 29, 2026 15:18 — with GitHub Actions Inactive

AlexAxthelm temporarily deployed to development July 29, 2026 15:20 — with GitHub Actions Inactive

AlexAxthelm temporarily deployed to development July 29, 2026 15:21 — with GitHub Actions Inactive

AlexAxthelm temporarily deployed to development July 29, 2026 15:22 — with GitHub Actions Inactive

AlexAxthelm temporarily deployed to development August 4, 2026 16:29 — with GitHub Actions Inactive

AlexAxthelm temporarily deployed to development August 5, 2026 10:52 — with GitHub Actions Inactive

AlexAxthelm had a problem deploying to development August 5, 2026 10:53 — with GitHub Actions Failure

AlexAxthelm temporarily deployed to development August 5, 2026 12:14 — with GitHub Actions Inactive

AlexAxthelm temporarily deployed to development August 5, 2026 12:16 — with GitHub Actions Inactive

Copilot AI reviewed Aug 5, 2026

View reviewed changes

AlexAxthelm added 11 commits August 5, 2026 16:16

PLAN

bcd1af3

feat(entity-linkage): instroduce Bounded-memory entity linkage

4ae9b75

feat(entity-linkage): re-use jobs infrastructure

06a4305

Reuse the jobs infrastructure from the ETL repo.

feat(frontend): use new jobs-based endpoint for EL

28aa764

refactor(entity-linkage): remove start endpoint (now unused)

ffeef49

rm PLAN

2da27d5

fix dry-run behavior

3f0a758

stream superset to keep bounded memory

7983a64

fail fast on malformed list

1ec8c98

adopt normalized search

c7ff35a

pass 404 error correctly

090b45c

Copilot AI reviewed Aug 5, 2026

View reviewed changes

mbarlow12 approved these changes Aug 7, 2026

View reviewed changes

		bearer_token: str \| None


		def normalize_name(name: str \| None) -> str \| None:

Uh oh!

Conversation

AlexAxthelm commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed

Breaking change

Testing

AI assistance

Follow-ups (separate tickets)

Uh oh!

github-actions Bot commented Jul 29, 2026

CD summary 5a60ab0

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 29, 2026

CD summary 6ffd2be

Uh oh!

github-actions Bot commented Aug 4, 2026

CD summary 05fecbd

Uh oh!

github-actions Bot commented Aug 5, 2026

CD summary 294ef5a

Uh oh!

github-actions Bot commented Aug 5, 2026

CD summary 1b9f9a4

Uh oh!

github-actions Bot commented Aug 5, 2026

CD summary a93a1ca

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

github-actions Bot commented Aug 5, 2026

CD summary 467c5c5

Uh oh!

github-actions Bot commented Aug 5, 2026

CD summary 090b45c

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

AlexAxthelm commented Aug 5, 2026

Uh oh!

mbarlow12 left a comment

Choose a reason for hiding this comment

Uh oh!

mbarlow12 Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

mbarlow12 Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

mbarlow12 Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

mbarlow12 Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AlexAxthelm commented Jul 29, 2026 •

edited

Loading

CD summary `5a60ab0`

CD summary `6ffd2be`

CD summary `05fecbd`

CD summary `294ef5a`

CD summary `1b9f9a4`

CD summary `a93a1ca`

CD summary `467c5c5`

CD summary `090b45c`