Skip to content

Add personal-data skills; document ETL/infra procedures - #51

Open
oinkspook wants to merge 1 commit into
mainfrom
skills/personal-data-and-infra
Open

Add personal-data skills; document ETL/infra procedures#51
oinkspook wants to merge 1 commit into
mainfrom
skills/personal-data-and-infra

Conversation

@oinkspook

Copy link
Copy Markdown
Contributor

Pull Request Description

Type of Change

  • Bug fix
  • New feature
  • Enhancement
  • Documentation update
  • Other (please specify)

Description of Changes

Codifies procedures that were documented (in corneel_notes.txt and the instroom/1cijferho implementations) but not yet captured as skills. All content is grounded in real, verified code — not planning notes.

New knowledge skills

  • browser-pseudonymize — in-browser Pyodide/WebAssembly encryption of sensitive columns (BSN/PGN/onderwijsnummer) before upload, in a Streamlit app. Captures the encrypt-vs-pseudonymize distinction, the "never keyless-hash a BSN" rule, the Pyodide bootstrap/caching, base64 JS-payload injection, and the st_js_blocking session-state latching gotcha. Grounded in 1cijferho/src/frontend/Files/.
  • identifier-mapping — split a record into a derived UUID + dual Postgres tables (secret_sensitive/secret_regular, FK cascade) + a MinIO object. Server-side keyed-HMAC UUID derivation; three read-back resolution levels. Grounded in 1cijferho/src/eencijferho/io/personal_data.py.

Extended skills

  • surf-sdp-helm-flux — new "Batch workloads" section: scheduled CronJob + on-demand Job, the unique-name-per-release mechanism, image-by-digest + config/secret wiring. Flags field bugs in the reference instroom chart.
  • etl-pipeline — cross-pipeline trigger-token setup (INSTROOM_CONFIG_TRIGGER_TOKEN).
  • surfdrive — public-share creation (where SURFDRIVE_SHARE_TOKEN/PASSWORD come from).

Related Issues

None.

Comparison: Before and After

Before:

Personal-data handling (browser encryption, UUID/dual-store) and the batch-job / trigger-token / surfdrive-link procedures lived only in code and corneel_notes.txt; no skill surfaced them.

After:

Two new skills + three extended skills codify these patterns, with the security-sensitive parts (encrypt vs pseudonymize, keyed server-side hashing) made explicit.

Testing Instructions

Skills are markdown (no executable change). To validate:

  1. Confirm each SKILL.md frontmatter description triggers on its intended prompts.
  2. Cross-references (/browser-pseudonymize, /identifier-mapping, /sdp-secrets-management, /etl-pipeline) resolve to existing skills.

Validation

  • styler::style_active_file() has been run — N/A: no R code

Dependencies

None.

Additional Information

  • identifier-mapping explicitly marks the restricted DB user, stored procedures, and INT-9/unique constraints as design intent, not implemented — it does not claim code that is absent.
  • Known chart bugs in instroom-config (duplicate successfulJobsHistoryLimit, misused ttlSecondsAfterFinished, CronJob-only keys on a Job) are documented in the skill but not fixed here — that is a separate change in the instroom-config repo.
  • Relates to the merge backlog noted in docs/skill-gaps.md: please review promptly rather than letting it queue.

Checklist

  • I have tested these changes locally
  • I have updated the documentation accordingly
  • My code follows the project's coding standards
  • I have added/updated tests to cover my changes (if applicable) — N/A: docs-only
  • The code is accompanied by comments
  • I have linked this PR to relevant issues (if applicable)

🤖 Generated with Claude Code

Codify documented-but-uncovered procedures into skills, sourced from
corneel_notes.txt and the instroom/1cijferho implementations.

New knowledge skills:
- browser-pseudonymize: in-browser Pyodide/WASM encryption of sensitive
  columns before upload; encrypt-vs-pseudonymize distinction; never
  keyless-hash a BSN.
- identifier-mapping: UUID + dual Postgres table + MinIO split; keyed
  server-side HMAC UUID derivation. Marks restricted-user/stored-procs/
  constraints as design intent, not implemented.

Extended skills:
- surf-sdp-helm-flux: batch workloads section (scheduled CronJob +
  on-demand Job, unique-name-per-release mechanism); flags field bugs in
  the reference instroom chart.
- etl-pipeline: cross-pipeline trigger-token setup (INSTROOM_CONFIG_TRIGGER_TOKEN).
- surfdrive: public-share creation (where SHARE_TOKEN/PASSWORD come from).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@oinkspook oinkspook self-assigned this Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant