Skip to content

feat: auto-update index mappings and validate document fields#190

Merged
atheurer merged 1 commit into
masterfrom
auto-update-index-mappings
Jun 5, 2026
Merged

feat: auto-update index mappings and validate document fields#190
atheurer merged 1 commit into
masterfrom
auto-update-index-mappings

Conversation

@atheurer

@atheurer atheurer commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Summary

  • During add-run.js getinfo pass, scan all document fields and validate they exist in indexDefs. Unknown fields cause a hard failure with a clear error message before any indexing occurs.
  • After validation, auto-update existing index mappings with any new fields from indexDefs via the PUT _mapping API (additive/idempotent), so users no longer need to manually add mappings when CDM is updated.
  • Fix getDocType() to return a consistent response object for v7/v8 (was returning a bare string, inconsistent with v9).

Test plan

  • getMappingFieldPaths() correctly identifies valid fields (queue, nic) and rejects unknown fields (bogus)
  • add-run.js with valid NDJSON data prints "Document field validation passed" and completes successfully
  • add-run.js with a document containing param.bogus_field fails with clear error: param: param.bogus_field
  • Run data remains present in OpenSearch after re-indexing

Closes #189

🤖 Generated with Claude Code

…add-run

During the getinfo pass, scan all document fields and validate they exist
in indexDefs. Unknown fields cause a hard failure before any indexing.
After validation, update existing index mappings with any new fields from
indexDefs via the PUT _mapping API (additive/idempotent). Also fixes
getDocType() to return a consistent response object for v7/v8.

Closes #189

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@project-crucible-tracking project-crucible-tracking Bot moved this to In Progress in Crucible Tracking Jun 5, 2026
@atheurer atheurer requested a review from k-rister June 5, 2026 15:38
@atheurer atheurer self-assigned this Jun 5, 2026
@atheurer atheurer merged commit f57cf6a into master Jun 5, 2026
35 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Crucible Tracking Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Auto add new mappings in existing indices

2 participants