diff --git a/README.md b/README.md index e4b13bf..aa75a2a 100644 --- a/README.md +++ b/README.md @@ -74,6 +74,69 @@ Every metadata directory is pre-wired into `objectstack.config.ts`, empty ones included: add your entry to the named array in your own `src//index.ts` and leave the config alone. It is the one file parallel branches collide on. +## Import your existing list + +Every customer already has the list — a spreadsheet per role, usually. Getting +it in is the platform's **Import** button on each object list, not anything +Duly wrote: upload, confirm the mapping, import. The CSVs in +[`samples/`](samples) are shaped to go straight through it, and they describe +the same fictional manufacturer as `pnpm demo`. + +| Sample | Import it on | Rows | +|:---|:---|:---| +| [`samples/business-units.csv`](samples/business-units.csv) | Setup → People & Organization → Business Units | 6 | +| [`samples/people.csv`](samples/people.csv) | Setup → People & Organization → Users | 12 | +| [`samples/catalog-items.csv`](samples/catalog-items.csv) | Duly → Setup → Role catalog | 21 | +| [`samples/duties.csv`](samples/duties.csv) | Duly → Setup → All duties | 19 | + +### Three steps + +1. **Put your people and units in first.** A duty's owner and business unit are + looked up **by name**, so the rows have to exist before the duty file can + land. In a real deployment they arrive from your directory; on a fresh + `pnpm dev` database, import `business-units.csv` and then `people.csv` + through the same Import button. +2. **Import the role catalog** — `catalog-items.csv` on Role catalog. This is + the list itself: what each position owes, how often, with how much grace. + No lookups, so it goes into an empty app as-is. +3. **Import the duties** — `duties.csv` on All duties. This is the catalog + instantiated onto named people, and it is where the natural keys resolve. + +Each step is the same three screens — Upload → Mapping → Preview → import — +and the count of created rows is reported at the end, with any refused row +named and downloadable. + +### What the columns have to say + +- **Headers are field API names** (`position_code`, `due_offset_days`). The + wizard auto-matches every one of them at high confidence. The **Download + template** link on the Upload step gives you the same columns as labels + instead; both are accepted. +- **Lookups are written as names, not ids.** `owner` takes a person's name + (`Priya Raman`) or their email; `business_unit` takes the unit's **name** + (`Northgate Quality` — its code will not resolve); `catalog_item` takes the + catalog item's name. This is the same natural-key rule the seed loader uses. +- **A name that matches nothing skips that row and says so**, with a + *Download failed rows* file to fix and re-import. Nothing is linked to a + best guess. +- **Blank means "leave unset"**, so the object's defaults apply. That is what + lets one file carry all three duty forms: a `standing` row leaves the five + cadence columns empty and lands with them all null, which is exactly what + `standing_no_frequency` requires. +- **Read-only columns are never written.** They are visible in the mapping + step as `— Skip —` or `(match only)`, so a column that will not land says so + before you import. +- **Re-importing needs the match option.** *When a row matches an existing + record* defaults to *Always create new*; running the same file twice + otherwise gives you two copies. + +`test/import-samples.test.ts` holds every sample header to the object's own +schema, so renaming a field fails the build instead of quietly importing a +blank column. + +**[The full walk, screen by screen, with what each step was measured to +do →](docs/import/walkthrough.md)** + ## Verify before you ship ```bash @@ -101,7 +164,9 @@ src/security/ positions, permission sets, sharing rules src/mappings/ src/data/ catalog import and seed fixtures src/translations/ en (source) · zh-CN scripts/ pnpm demo — prepare the database, then start with the example loaded +samples/ CSVs for the platform's standard Import (see above) docs/product/ positioning, data model, design principles +docs/import/ the recorded import walk, screen by screen ``` ## Documentation @@ -109,6 +174,7 @@ docs/product/ positioning, data model, design principles - [Positioning](docs/product/positioning.md) — who this is for and what it is not - [Data model](docs/product/data-model.md) — the five objects and why each exists - [Design principles](docs/product/design-principles.md) — the constraints above, argued +- [Importing an existing list](docs/import/walkthrough.md) — the standard Import, walked and measured - [Roadmap](docs/roadmap.md) — milestones M0–M4 ## License diff --git a/docs/import/01-upload.png b/docs/import/01-upload.png new file mode 100644 index 0000000..8cd6f67 Binary files /dev/null and b/docs/import/01-upload.png differ diff --git a/docs/import/02-mapping.png b/docs/import/02-mapping.png new file mode 100644 index 0000000..9ef8da6 Binary files /dev/null and b/docs/import/02-mapping.png differ diff --git a/docs/import/03-preview.png b/docs/import/03-preview.png new file mode 100644 index 0000000..4069a9c Binary files /dev/null and b/docs/import/03-preview.png differ diff --git a/docs/import/04-result-catalog.png b/docs/import/04-result-catalog.png new file mode 100644 index 0000000..fa8a9d4 Binary files /dev/null and b/docs/import/04-result-catalog.png differ diff --git a/docs/import/05-mapping-duties.png b/docs/import/05-mapping-duties.png new file mode 100644 index 0000000..13e173f Binary files /dev/null and b/docs/import/05-mapping-duties.png differ diff --git a/docs/import/06-result-duties.png b/docs/import/06-result-duties.png new file mode 100644 index 0000000..ee62e0a Binary files /dev/null and b/docs/import/06-result-duties.png differ diff --git a/docs/import/07-duties-list.png b/docs/import/07-duties-list.png new file mode 100644 index 0000000..01fa13f Binary files /dev/null and b/docs/import/07-duties-list.png differ diff --git a/docs/import/08-unresolved-lookup.png b/docs/import/08-unresolved-lookup.png new file mode 100644 index 0000000..abd37c6 Binary files /dev/null and b/docs/import/08-unresolved-lookup.png differ diff --git a/docs/import/09-result-business-units.png b/docs/import/09-result-business-units.png new file mode 100644 index 0000000..2ed4dfb Binary files /dev/null and b/docs/import/09-result-business-units.png differ diff --git a/docs/import/10-result-people.png b/docs/import/10-result-people.png new file mode 100644 index 0000000..3263bdc Binary files /dev/null and b/docs/import/10-result-people.png differ diff --git a/docs/import/walkthrough.md b/docs/import/walkthrough.md new file mode 100644 index 0000000..64253bd --- /dev/null +++ b/docs/import/walkthrough.md @@ -0,0 +1,213 @@ +# Importing an existing duty list — the recorded walk + +What a pre-sales demo of "import the list you already have" actually looks +like, screen by screen, and what the platform's Import was **measured** to do +with each column. + +Duly writes no import code. The **Import** button on every object list is the +platform's (`@objectstack/connector-rest` 17.2.0); everything below is that +button, driven in a browser against a clean database. + +- **Recorded on:** `pnpm dev` (not `pnpm demo` — see [Why a clean + database](#why-a-clean-database)), fresh `.objectstack/data`, signed in as + the seeded `admin@objectos.ai`. +- **Files:** the four CSVs in [`samples/`](../../samples). +- **Result:** 6 + 12 + 21 + 19 rows, `0 skipped`. + +--- + +## The wizard + +Three screens, identical on every object. + +### 1. Upload + +`Drag & drop a CSV or Excel file here` — plus a **Download template** link that +emits a CSV with the object's columns, required ones marked `*`. + +![Import step 1 — upload](01-upload.png) + +The template writes **labels** as headers (`Duty *`, `Offset (days, 0 = anchor +day)`). The samples in this repo write **API names** (`name`, +`due_offset_days`) instead — both auto-match, and API names are stable across +locales, which is why the samples use them. + +### 2. Mapping + +![Import step 2 — mapping](02-mapping.png) + +`Auto-matched 11 column(s)` for `catalog-items.csv` and 14 for `duties.csv` — +every column, all at *High confidence*, nothing to adjust by hand. + +Two statuses are worth knowing, because both mean "this column will not be +written" and neither is an error: + +| Shown as | Example | Meaning | +|:---|:---|:---| +| `— Skip —` / `Skipped` | `sys_user.manager_id` | The column cannot be mapped at all. The row still imports, without it. | +| `… (match only)` | `duly_duty.last_dispatched_period` | Mappable as a *match key* for update/upsert, never written. Measured: the import reported `1 created` and the column read back `null`. | + +Both are read-only fields. **A skipped column is the quiet failure mode this +whole page exists to make loud** — the import succeeds, the records land, and +one column is simply blank. `test/import-samples.test.ts` fails the build if a +sample header ever stops naming a writable field. + +### 3. Preview + +![Import step 3 — preview](03-preview.png) + +The parsed rows, plus the import options. The one that matters for a repeat +run: + +**When a row matches an existing record** — `Always create new` (the default), +`Update existing (skip if no match)`, `Update if matched, else create`. On the +default, running the same file twice gives you two copies. Re-importing over an +earlier load means picking one of the other two and matching on `name`. + +The button is labelled with the count: `Import 21 Rows`. + +### Result + +![Role catalog imported — 21 created](04-result-catalog.png) + +--- + +## The four files, in order + +Order is not a style preference. Each file's lookups resolve against rows the +previous file created. + +### 1 · Business units — `samples/business-units.csv` → `sys_business_unit` + +Setup → People & Organization → **Business Units** → Import. + +![6 business units created](09-result-business-units.png) + +`parent_business_unit_id` is written as the **parent's name**, and resolves +*within the same file* — `Northgate Plant` finds `Ardenline Group` from a row +above it. (The importer flushes pending creates and retries a miss on the same +object, so a child after its parent is safe.) + +### 2 · People — `samples/people.csv` → `sys_user` + +Setup → People & Organization → **Users** → Import. + +![12 users created](10-result-people.png) + +Name and email only. These are **directory rows, not logins** — nobody can sign +in as them. Real people get invited through Setup → Users → Invite User. + +`manager_id` is deliberately **not** a column: it is read-only and the mapping +step drops it to `— Skip —`. Set the reporting chain in Setup, not in the CSV. + +### 3 · Role catalog — `samples/catalog-items.csv` → `duly_catalog_item` + +Duly → Setup → **Role catalog** → Import. 21 rows, no lookups at all — this +file imports into an otherwise empty database. + +### 4 · Duties — `samples/duties.csv` → `duly_duty` + +Duly → Setup → **All duties** → Import. Three lookups per row, all by natural +key. + +![Import Duty — 19 created](06-result-duties.png) + +![All duties — 19 records with owners resolved](07-duties-list.png) + +Every `owner` resolved to a person, every `business_unit` to a unit, every +`catalog_item` to a template — read back through the API to confirm it is the +right row and not merely *a* row: + +``` +Lifting equipment check — Line C owner=Yuki Tanabe bu=Northgate Operations catalog_item=Lifting equipment check +Keep the permit register current — Ardenline owner=Nadia Ilves bu=Ardenline Group catalog_item=Keep the permit register current +Monthly site performance note owner=Tomas Bergh bu=Northgate Plant catalog_item=null +``` + +--- + +## How lookups resolve — measured, not assumed + +The importer tries, in order: an exact `id`, the target object's display field, +then `name`, `title`, `label`, `full_name`, `email`, `username`. The first +field to match wins; **more than one match stops the row rather than linking +the first**. + +| Column | Target | Write it as | Measured | +|:---|:---|:---|:---| +| `duly_duty.owner` | `sys_user` | the person's **name** (`Priya Raman`) | resolves | +| `duly_duty.owner` | `sys_user` | their **email** (`priya.raman@ardenline.example`) | resolves | +| `duly_duty.business_unit` | `sys_business_unit` | the unit's **name** (`Northgate Quality`) | resolves | +| `duly_duty.business_unit` | `sys_business_unit` | the unit's **code** (`NGP-QA`) | **does not resolve** | +| `duly_duty.catalog_item` | `duly_catalog_item` | the item's **name** | resolves | +| `sys_business_unit.parent_business_unit_id` | `sys_business_unit` | the parent's **name** | resolves, same file | + +**This is the same rule the seed loader uses.** `src/data/org.seed.ts` resolves +`duly_task.owner` and `duly_duty.owner` as natural keys against `sys_user.name`; +the Import UI was the open question, and it matches. One format for both paths. + +### When it does not resolve + +The row is skipped and named. Nothing is guessed, nothing is left dangling. + +![1 created, 2 skipped, with the reason per row](08-unresolved-lookup.png) + +``` +Row 2: Owner: No matching record for "Nobody Here" +Row 3: Business unit: No matching record for "NGP-QA" +``` + +**Download failed rows** hands back just those rows, so a partial load is +finished by fixing that file and importing it again. + +This is also exactly what happens if you skip steps 1 and 2. Measured, on a +genuinely clean database with one user (`Dev Admin`) and no business units: +`duties.csv` imports **0 created, 19 skipped**, one `Owner: No matching record` +per row. Loud, per-row, recoverable — but the people have to be there first. + +--- + +## Blank cells and the conditional defaults + +A blank cell means **"leave this field unset"**, not "write null". The object's +`defaultValue` then decides, which is what makes one flat CSV able to carry +three duty forms without tripping the cadence rules in +`src/objects/catalog-item.object.ts`: + +| Row form | Blank cadence cells | Landed as | +|:---|:---|:---| +| `standing` | frequency, anchor, offset, lead, grace | all five `null` — `standing_no_frequency` and friends satisfied | +| `one_off` | anchor, offset, lead | `null`; `grace_days` keeps the authored value (a one-off's task has a real due date) | +| `recurring` | — | as written | + +Measured over the 21 imported catalog items: `recurring` 18, `standing` 2, +`one_off` 1; frequencies daily 1, weekly 3, fortnightly 1, monthly 9, +quarterly 2, semi-annual 1, annual 2. + +--- + +## Why a clean database + +`pnpm demo` loads the same fictional organisation — Ardenline Group — as a +seed. Walking the import on top of it proves nothing: the rows are already +there, and with the default `Always create new` you get two of each. + +`pnpm dev` starts empty, which is also what a real deployment starts from, so +the walk above is the customer's first hour rather than a demo of a demo. + +```bash +rm -rf .objectstack/data +pnpm dev +``` + +## What is deliberately not here + +- **No import handler, action or job.** The platform's Import is the interface. + A Duly-specific import path would be a second dialect of a thing that already + works. +- **No Excel parsing.** The platform's endpoint accepts `.xlsx` itself; CSV is + what these samples ship as because it diffs. +- **No `sys_business_unit_member` rows.** Membership is what + `sys_user.primary_business_unit_id` is derived from, and it is Setup's + surface, not this walk's — `duly_duty.business_unit` is written directly from + the CSV and does not depend on it. diff --git a/samples/business-units.csv b/samples/business-units.csv new file mode 100644 index 0000000..e6f599c --- /dev/null +++ b/samples/business-units.csv @@ -0,0 +1,7 @@ +name,code,kind,parent_business_unit_id,active +Ardenline Group,ARD,company,,true +Northgate Plant,NGP,division,Ardenline Group,true +Riverside Plant,RVP,division,Ardenline Group,true +Central Office,CEN,division,Ardenline Group,true +Northgate Operations,NGP-OPS,department,Northgate Plant,true +Northgate Quality,NGP-QA,department,Northgate Plant,true diff --git a/samples/catalog-items.csv b/samples/catalog-items.csv new file mode 100644 index 0000000..0594ed4 --- /dev/null +++ b/samples/catalog-items.csv @@ -0,0 +1,22 @@ +name,position_code,form,frequency,due_anchor,due_offset_days,lead_days,grace_days,regulation_ref,description,active +Emissions return,plant_compliance_officer,recurring,monthly,period_end,-5,10,3,Group Environment Standard GE-02 §5,"Submit the site emissions figures for the month, with the meter readings they were derived from.",true +Waste transfer log review,plant_compliance_officer,recurring,monthly,period_start,4,7,2,Group Environment Standard GE-04 §2,"Check every transfer note raised last month against the carrier register; flag anything unmatched.",true +Effluent sampling record,plant_compliance_officer,recurring,weekly,period_start,1,3,1,Site Discharge Consent DC-11 cl.4,Draw and log the weekly outfall sample. Record the result even when it is within limits.,true +Permit condition review,plant_compliance_officer,recurring,quarterly,period_end,-10,21,5,Group Environment Standard GE-09 §1,"Walk the permit conditions one by one and record, for each, the evidence that it was met this quarter.",true +Site environmental audit,plant_compliance_officer,recurring,semiannual,period_end,0,150,10,Group Assurance Plan AP-3 §6,"Full walk-round audit against the group environmental standard, with findings and owners.",true +Annual environmental statement,plant_compliance_officer,recurring,annual,period_end,-30,60,14,Group Environment Standard GE-01 §8,Compile the year's environmental performance into the statement the group publishes.,true +Keep the permit register current,plant_compliance_officer,standing,,,,,,Group Environment Standard GE-09 §4,"The register reflects the permits actually in force — no expiry passes without the entry being updated. Never ""done""; attested, not ticked.",true +Daily line start-up check,shift_supervisor,recurring,daily,period_start,0,1,0,Works Instruction WI-101 §2,Confirm the line is safe to start and record the guard and interlock checks before the first run of the day.,true +Shift handover record,shift_supervisor,recurring,weekly,period_start,0,2,0,Works Instruction WI-120 §3,"Written handover for every shift change in the week: state of the line, anything left open.",true +Line safety walk,shift_supervisor,recurring,monthly,period_start,2,7,2,Site Safety Standard SS-07 §2,Walk the line against the safety checklist with an operator present. Log what you fixed on the spot.,true +Toolbox talk record,shift_supervisor,recurring,monthly,period_start,9,7,3,Site Safety Standard SS-07 §5,Run one toolbox talk with the shift and record who attended.,true +Lifting equipment check,shift_supervisor,recurring,quarterly,period_start,5,14,5,Works Instruction WI-204 §1,"Visual check and tag review of every sling, hoist and eyebolt on the line.",true +Contractor induction refresh,shift_supervisor,recurring,annual,period_end,-60,120,21,Site Safety Standard SS-15 §3,"Re-run the site induction for every contractor still holding a pass, and retire the passes nobody claimed.",true +Answer the duty phone,shift_supervisor,standing,,,,,,Works Instruction WI-002 §1,The out-of-hours phone is carried and answered. There is no version of this that is ever finished.,true +Overtime justification summary,shift_supervisor,recurring,monthly,period_end,-2,7,3,People Policy PP-22 cl.6,One line per overtime shift worked: why it was needed and what it covered.,true +Calibration verification,quality_technician,recurring,monthly,period_start,6,7,2,Quality Manual QM-31 §4,"Verify each instrument against its reference standard and record the deviation, in range or not.",true +Retained sample review,quality_technician,recurring,fortnightly,period_start,2,5,1,Quality Manual QM-18 §2,Inspect the retained samples due for review and dispose of anything past its retention window.,true +Nonconformance log review,quality_technician,recurring,monthly,period_start,1,7,2,Quality Manual QM-05 §3,Review every nonconformance raised last month and confirm each one has an owner and a closing date.,true +Cleaning verification swabs,quality_technician,recurring,weekly,period_start,3,3,1,Quality Manual QM-22 §7,Swab the changeover points after the weekly clean and log the plate counts.,true +Instrument drift check,quality_technician,recurring,monthly,period_start,8,7,2,,Compare this month's calibration deviations against the last three and note any instrument trending out.,true +Commissioning file handover,quality_technician,one_off,,,,,7,Project Standard PS-06 §5,"Hand the commissioning file to operations: as-built drawings, test records, spares list, signed off.",false diff --git a/samples/duties.csv b/samples/duties.csv new file mode 100644 index 0000000..191358b --- /dev/null +++ b/samples/duties.csv @@ -0,0 +1,20 @@ +name,form,owner,business_unit,source,catalog_item,frequency,due_anchor,due_offset_days,lead_days,grace_days,timezone,status,description +Annual environmental statement — Ardenline,recurring,Priya Raman,Northgate Quality,catalog,Annual environmental statement,annual,period_end,-30,60,14,Europe/Berlin,active,Compile the year's environmental performance into the statement the group publishes. +Answer the duty phone — Northgate Quality,standing,Priya Raman,Northgate Quality,catalog,Answer the duty phone,,,,,,Europe/Berlin,active,The out-of-hours phone is carried and answered. There is no version of this that is ever finished. +Calibration verification — Lab 1,recurring,Rosa Delgado,Northgate Quality,catalog,Calibration verification,monthly,period_start,6,7,2,Europe/Berlin,active,"Verify each instrument against its reference standard and record the deviation, in range or not." +Retained sample review — Lab 1,recurring,Rosa Delgado,Northgate Quality,catalog,Retained sample review,fortnightly,period_start,2,5,1,Europe/Berlin,active,Inspect the retained samples due for review and dispose of anything past its retention window. +Nonconformance log review — Northgate Quality,recurring,Rosa Delgado,Northgate Quality,catalog,Nonconformance log review,monthly,period_start,1,7,2,Europe/Berlin,active,Review every nonconformance raised last month and confirm each one has an owner and a closing date. +Calibration verification — Lab 2,recurring,Ibrahim Chaudhry,Northgate Quality,catalog,Calibration verification,monthly,period_start,6,7,2,Europe/Berlin,active,"Verify each instrument against its reference standard and record the deviation, in range or not." +Instrument drift check — Lab 2,recurring,Ibrahim Chaudhry,Northgate Quality,catalog,Instrument drift check,monthly,period_start,8,7,2,Europe/Berlin,active,Compare this month's calibration deviations against the last three and note any instrument trending out. +Shift handover record — Line A,recurring,Marek Dvorak,Northgate Operations,catalog,Shift handover record,weekly,period_start,0,2,0,Europe/Berlin,active,"Written handover for every shift change in the week: state of the line, anything left open." +Line safety walk — Line A,recurring,Marek Dvorak,Northgate Operations,catalog,Line safety walk,monthly,period_start,2,7,2,Europe/Berlin,active,Walk the line against the safety checklist with an operator present. Log what you fixed on the spot. +Line safety walk — Line B,recurring,Sami Okonkwo,Northgate Operations,catalog,Line safety walk,monthly,period_start,2,7,2,Europe/Berlin,active,Walk the line against the safety checklist with an operator present. Log what you fixed on the spot. +Contractor induction refresh — Northgate,recurring,Sami Okonkwo,Northgate Operations,catalog,Contractor induction refresh,annual,period_end,-60,120,21,Europe/Berlin,active,"Re-run the site induction for every contractor still holding a pass, and retire the passes nobody claimed." +Lifting equipment check — Line C,recurring,Yuki Tanabe,Northgate Operations,catalog,Lifting equipment check,quarterly,period_start,5,14,5,Europe/Berlin,active,"Visual check and tag review of every sling, hoist and eyebolt on the line." +Emissions return — Riverside,recurring,Ana Ferreira,Riverside Plant,catalog,Emissions return,monthly,period_end,-5,10,3,UTC,active,"Submit the site emissions figures for the month, with the meter readings they were derived from." +Waste transfer log review — Riverside,recurring,Ana Ferreira,Riverside Plant,catalog,Waste transfer log review,monthly,period_start,4,7,2,UTC,paused,"Check every transfer note raised last month against the carrier register; flag anything unmatched." +Line safety walk — Riverside,recurring,Greta Lindqvist,Riverside Plant,catalog,Line safety walk,monthly,period_start,2,7,2,UTC,active,Walk the line against the safety checklist with an operator present. Log what you fixed on the spot. +Nonconformance log review — Riverside,recurring,Elin Halvorsen,Riverside Plant,catalog,Nonconformance log review,monthly,period_start,1,7,2,UTC,active,Review every nonconformance raised last month and confirm each one has an owner and a closing date. +Keep the permit register current — Ardenline,standing,Nadia Ilves,Ardenline Group,catalog,Keep the permit register current,,,,,,UTC,active,The register reflects the permits actually in force — no expiry passes without the entry being updated. +Monthly site performance note,recurring,Tomas Bergh,Northgate Plant,self,,monthly,period_end,-1,5,0,Europe/Berlin,active,"A page on how the site actually ran this month — written for myself, not for a report." +Commissioning file handover — Riverside upgrade,one_off,Owen Pryce,Central Office,catalog,Commissioning file handover,,,,,7,UTC,active,"Hand the commissioning file to operations: as-built drawings, test records, spares list, signed off." diff --git a/samples/people.csv b/samples/people.csv new file mode 100644 index 0000000..8422684 --- /dev/null +++ b/samples/people.csv @@ -0,0 +1,13 @@ +name,email +Nadia Ilves,nadia.ilves@ardenline.example +Tomas Bergh,tomas.bergh@ardenline.example +Elin Halvorsen,elin.halvorsen@ardenline.example +Owen Pryce,owen.pryce@ardenline.example +Marek Dvorak,marek.dvorak@ardenline.example +Priya Raman,priya.raman@ardenline.example +Sami Okonkwo,sami.okonkwo@ardenline.example +Yuki Tanabe,yuki.tanabe@ardenline.example +Rosa Delgado,rosa.delgado@ardenline.example +Ibrahim Chaudhry,ibrahim.chaudhry@ardenline.example +Ana Ferreira,ana.ferreira@ardenline.example +Greta Lindqvist,greta.lindqvist@ardenline.example diff --git a/test/import-samples.test.ts b/test/import-samples.test.ts new file mode 100644 index 0000000..f3f212e --- /dev/null +++ b/test/import-samples.test.ts @@ -0,0 +1,147 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { readFileSync } from 'node:fs'; + +import { describe, expect, it } from 'vitest'; + +import { SysBusinessUnit, SysUser } from '@objectstack/platform-objects/identity'; + +import { CatalogItem, Duty } from '../src/objects/index.js'; + +/** + * The `samples/` CSVs are the artefact the pre-sales walk-through hands an + * evaluator: "here is your existing duty list, import it". They go in through + * the PLATFORM's standard Import button on each object list — Duly writes no + * import code (#19) — so nothing in this repo would notice the day a field is + * renamed underneath them. + * + * And the failure would be quiet. Measured on `@objectstack/connector-rest` + * 17.2.0 against a live `pnpm dev`: a column the mapping step cannot place is + * marked `— Skip —` and the row still imports, minus that column. So a renamed + * `position_code` does not produce a red import; it produces 21 catalog items + * with no position, and a demo that looks like it worked. + * + * This file is the tripwire: every header in every shipped sample must name a + * field the target object actually has. + * + * ── Read off the schema, never hand-copied ─────────────────────────────── + * The field lists below are derived from the `ObjectSchema` objects + * themselves. A hand-maintained list of "the columns the sample uses" would be + * a second copy of the same fact, and the copy is what goes stale. + * + * ── Two tiers, because ownership differs ───────────────────────────────── + * `duly_catalog_item` and `duly_duty` are objects THIS APP declares, so the + * test can hold the samples to what the import will actually WRITE: declared + * fields minus `readonly`. Measured on a live import, both directions: + * + * - `duly_duty.last_dispatched_period` (`readonly: true`) maps in the + * wizard as **"Last dispatched period (match only)"**, the import reports + * `1 created`, and the column reads back `null`. Mapped, reported, + * not written. + * - every non-readonly column in both samples landed: 21 and 19 rows, `0 + * skipped`, with cadence, timezone, status and source all as authored. + * + * `sys_user` and `sys_business_unit` are the PLATFORM's, and the platform's + * own `readonly` flag does not predict what its import writes there: + * `sys_user.email` is `readonly: true` and IS written (12 rows created, each + * with its address), while `sys_user.manager_id` is `readonly: true` and is + * dropped to `— Skip —`. Encoding that split here would be this app + * describing a table it does not own — the same mistake `src/data/org.seed.ts` + * declines to make with `defineSeed`. So the platform samples are held to the + * weaker, uncontested property: every header names a DECLARED field. That + * still catches the rename this file exists for. + */ + +const samplePath = (file: string): URL => new URL(`../samples/${file}`, import.meta.url); + +/** + * The header row of a CSV, as field names. + * + * Quote-aware because the platform's own template download quotes any header + * containing a comma (`"Offset (days, 0 = anchor day)"`), and a sample + * regenerated from that template would carry the same shape. BOM-stripped for + * the same reason — the template ships one. + */ +function csvHeader(file: string): string[] { + const firstLine = readFileSync(samplePath(file), 'utf8') + .replace(/^\uFEFF/, '') + .split(/\r?\n/) + .find((line) => line.trim() !== ''); + if (firstLine === undefined) throw new Error(`samples/${file} is empty`); + + const out: string[] = []; + let cur = ''; + let quoted = false; + for (let i = 0; i < firstLine.length; i++) { + const ch = firstLine[i]; + if (quoted) { + if (ch === '"') { + if (firstLine[i + 1] === '"') { cur += '"'; i++; } else { quoted = false; } + } else cur += ch; + continue; + } + if (ch === '"') { quoted = true; continue; } + if (ch === ',') { out.push(cur.trim()); cur = ''; continue; } + cur += ch; + } + out.push(cur.trim()); + return out; +} + +/** `[fieldName, definition]` for every field the schema declares. */ +function fieldEntries(schema: unknown): Array<[string, { readonly?: boolean; required?: boolean }]> { + const fields = (schema as { fields?: Record> }).fields; + if (!fields || typeof fields !== 'object') throw new Error('schema declares no fields'); + return Object.entries(fields).map(([key, def]) => [ + typeof def?.name === 'string' ? (def.name as string) : key, + def as { readonly?: boolean; required?: boolean }, + ]); +} + +const declaredFields = (schema: unknown): string[] => fieldEntries(schema).map(([name]) => name); + +/** Declared minus `readonly` — what the import will actually write. */ +const writableFields = (schema: unknown): string[] => + fieldEntries(schema).filter(([, def]) => def?.readonly !== true).map(([name]) => name); + +/** Fields a row cannot omit AND a caller may set. Both halves matter: a + * `required` field that is also `readonly` is the platform's to write. */ +const requiredWritableFields = (schema: unknown): string[] => + fieldEntries(schema) + .filter(([, def]) => def?.required === true && def?.readonly !== true) + .map(([name]) => name); + +describe('samples/ — the duty list an evaluator imports (#19)', () => { + describe.each([ + { file: 'catalog-items.csv', object: 'duly_catalog_item', schema: CatalogItem }, + { file: 'duties.csv', object: 'duly_duty', schema: Duty }, + ])('$file → $object', ({ file, schema }) => { + it('every header names a field the import will write', () => { + const writable = writableFields(schema); + const unknown = csvHeader(file).filter((h) => !writable.includes(h)); + // Named rather than counted: the failure message has to say WHICH column + // stopped resolving, because the symptom in the app is a silently blank + // one. + expect(unknown).toEqual([]); + }); + + it('carries every field a row cannot be created without', () => { + const header = csvHeader(file); + const missing = requiredWritableFields(schema).filter((f) => !header.includes(f)); + // The other direction of the same rot: a NEW required field leaves the + // sample syntactically fine and refused row by row at import time. + expect(missing).toEqual([]); + }); + }); + + describe.each([ + { file: 'business-units.csv', object: 'sys_business_unit', schema: SysBusinessUnit }, + { file: 'people.csv', object: 'sys_user', schema: SysUser }, + ])('$file → $object (platform object — declared-fields tier)', ({ file, schema }) => { + it('every header names a declared field', () => { + const declared = declaredFields(schema); + const unknown = csvHeader(file).filter((h) => !declared.includes(h)); + expect(unknown).toEqual([]); + }); + }); +}); diff --git a/test/node-builtins.d.ts b/test/node-builtins.d.ts new file mode 100644 index 0000000..3cc8017 --- /dev/null +++ b/test/node-builtins.d.ts @@ -0,0 +1,22 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The one Node API a test in this repo reads, typed narrowly. + * + * `@types/node` is deliberately NOT a dependency here — `src/data/index.ts` + * states the reason and sets the precedent, declaring the single `process.env` + * property its gate needs rather than pulling the whole Node type surface into + * a metadata package. `test/import-samples.test.ts` needs exactly one function, + * to read the shipped `samples/*.csv` off disk, so it gets the same treatment. + * + * This has to live in a `.d.ts`: an ambient `declare module` inside a file that + * is itself a module is read as an AUGMENTATION, and augmenting a module that + * does not resolve is an error rather than a declaration. + * + * Widen it only for something a test actually calls. The value of the narrow + * form is that it cannot silently license a second, larger Node dependency. + */ +declare module 'node:fs' { + /** Read a whole file as text. `URL` so the caller can resolve against `import.meta.url`. */ + export function readFileSync(path: URL | string, encoding: 'utf8'): string; +}