[ENG-43320] Add metadata-extractor flow doc + harden ConfigLoader errors

[vishalk9]

Two coupled changes for the data-plane metadata-extractor job.

1. New doc: docs/metadata-extractor-flow.md

Canonical end-to-end walk-through of the Push-Model extractor, intended for engineers extending it, on-callers debugging it, and the lakeview-docs QMD collection that mounts this folder into the OneHouse AI agent platform. Sections:

• Process lifecycle — Main → CliParser → Guice → continuous vs once mode → shutdown
• Table discovery — TableDiscoveryService recursive walk + format detector dispatch + scheduling intervals
• Metadata read path — hoodie.properties, active timeline, archived timeline V1 vs V2 (LSM manifest)
• Pre-signed URL fetch — exact endpoint, headers, request shape, related endpoints table
• Upload path — small-file buffer vs large-file streaming branches, no compression, parallel within batch
• HTTP transport — retry decision diagram, backoff math, configurable knobs table
• Checkpoint state — per-table Checkpoint fields, fetch/upsert cadence, incrementality semantics
• Error-handling matrix — every failure mode mapped to its observable behaviour and source location
• Concurrency model — pool table (ForkJoinPool + 2 ScheduledExecutorServices)
• Cost levers — full-scan archived V1, no de-dup, tiny archived batch size, etc.
• Configuration reference — every YAML key with required/optional + defaults + validation
• Test coverage map — per-class table marking thin vs covered, ranked gaps
• Debugging recipes — "where did this file go", "why is this table stuck", "why are we re-uploading", "config didn't parse"

Linked from docs/index.md.

2. ConfigLoader hardening

ConfigLoader.loadConfigFromJsonNode previously crashed with a buried NullPointerException when YAML omitted the version field — the outer catch (Exception e) { throw new RuntimeException(\"Failed to load config\", e) } wrapped the NPE in a message that didn't say what was actually wrong, and didn't include the config file path.

Replaced the silent NPE with explicit, actionable errors:

Input	Old behaviour	New behaviour
Empty / unparseable YAML	NPE on null jsonNode	IllegalArgumentException(\"Config is empty or could not be parsed\")
YAML without version field	NPE on null.asText()	IllegalArgumentException(\"Config missing required 'version' field\")
YAML with version: \"\" or version: null	NPE / silent IllegalArg	IllegalArgumentException(\"Config missing required 'version' field\")
YAML with version: V99 (unknown enum)	Java's default \"No enum constant ConfigVersion.V99\"	IllegalArgumentException(\"Unsupported config version: V99\")
Missing config file	\"Failed to load config\"	\"Failed to load config from /path/to/missing.yaml\"

5 tests added to ConfigLoaderTest cover all five paths above.

Regression risk

Zero for valid configs. The fix tightens error messages on already-broken inputs (which used to crash with cryptic stack traces). The exception type thrown by loadConfigFromString / loadConfigFromConfigFile is unchanged (RuntimeException wrapping the new IllegalArgumentException), so any caller that catches RuntimeException continues to work. If a caller was relying on getCause() being a Jackson exception for the empty-YAML case, they would now get an IllegalArgumentException — no such callers found in this repo (ConfigLoader is only called from Main and ConfigRefresher, neither inspects the cause).

Test plan

• ./gradlew :lakeview:test --tests ai.onehouse.config.* — 16 tests green (11 existing + 5 new).
• ./gradlew :lakeview:spotlessCheck — green.
• CI: build, sonar, coverage.

Related

• Companion PR [ENG-43320] Fix InputStream leak in PresignedUrlFileUploader + error-path tests #192 (Fix InputStream leak in PresignedUrlFileUploader + error-path tests) — same ENG-43320 effort, different file.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
87.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud