Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ coverage/
*.log
*.csv
!test/fixtures/Account.csv
!test/fixtures/Account-en.csv
8 changes: 5 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,9 @@ Run `pnpm lint && pnpm typecheck && pnpm test` before proposing a change; that i
```
src/
csv/ tokenizer (papaparse) + Node row stream + shared options
dialects/ Dialect interface, French dialect, DialectRegistry
dialects/ Dialect interface, French/English/layout-fallback dialects, DialectRegistry
records/ RawRecord — a normalised, dialect-agnostic row
classify/ Matcher registry + matchers (trade, fx, dividend, fees, interest, cash)
classify/ Matcher registry + matchers (trade, fx, dividend, fees, interest, cash, structural)
group/ GroupingStrategy pipeline + Transaction union
validate/ per-currency balance reconciliation
portfolio/ positions, FIFO realized P/L, cash and fee roll-ups
Expand All @@ -54,6 +54,8 @@ examples/dashboard/

The pipeline is `tokenizeCsv` → `mapRow` (dialect) → `ClassifierRegistry.classify` → `groupMovements` (strategy list) → optional `reconcileBalances` / `summarizePortfolio`. Every stage is swappable through `ParseOptions`; adding a locale, a description or a grouping rule should mean registering something, never editing a `switch`.

Header language, number format and description language are three independent axes of a DEGIRO export — changing the interface language rewrites line 1 and nothing else. Keep them independent in code: dialects own the header and the number/date formats, matchers own the wording, and neither may assume the other. `genericDialect` closes the set by matching the column layout alone; anything heuristic sets `heuristic: true` so `ParseResult.warnings` can say so.

## Rules that are load-bearing

- **The root entry imports no Node builtin.** `src/index.ts` and everything it reaches must stay free of `node:*`; anything needing a filesystem or streams goes behind `src/node.ts`. `test/entrypoints.test.ts` guards this and has caught a regression before.
Expand All @@ -69,7 +71,7 @@ The pipeline is `tokenizeCsv` → `mapRow` (dialect) → `ClassifierRegistry.cla
- TypeScript is strict with `noUncheckedIndexedAccess` and `verbatimModuleSyntax`. Type-only imports must use `import type` — `@typescript-eslint/consistent-type-imports` is an error, not a warning.
- Prettier: single quotes, semicolons, trailing commas, 100 columns. `.csv` fixtures and `CHANGELOG.md` are excluded and must not be reformatted.
- `examples/**/components/ui/**` is generated by the shadcn CLI and kept verbatim so it stays upgradable; two lint rules are disabled there. Do not hand-edit those files.
- `test/fixtures/Account.csv` is synthetic but byte-sensitive: French decimals, `U+202F` thousands separators, double-spaced product names, and running balances that reconcile exactly. Editing it usually means fixing the balances too. A real `Account.csv` at the repo root is git-ignored.
- `test/fixtures/Account.csv` is synthetic but byte-sensitive: French decimals, `U+202F` thousands separators, double-spaced product names, and running balances that reconcile exactly. Editing it usually means fixing the balances too. `test/fixtures/Account-en.csv` is the same body behind the English header; keep the two bodies identical, since a test asserts they classify the same. Both are unignored by name in `.gitignore`. A real `Account.csv` at the repo root is git-ignored.
- New behaviour needs a test in `test/` (library) or `examples/dashboard/test/` (analytics). The analytics layer imports nothing from React precisely so it stays testable headlessly.

## Git and PRs
Expand Down
63 changes: 51 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ A modern, ESM-only TypeScript library that turns a DEGIRO account statement into

- 🧮 **Exact money** with [`big.js`](https://github.com/MikeMcl/big.js) — no float drift
- 🧩 **Extensible** dialects, classifiers and grouping strategies
- 🌐 **French and English exports** out of the box, and a layout-detecting fallback for every other language
- 🧠 **Typed domain model** — discriminated unions for movements & transactions
- 🪶 **Lenient parsing** — per-row problems are collected, never thrown
- 🌊 **Streaming** parser for very large files
Expand Down Expand Up @@ -108,6 +109,40 @@ const result = await parseDegiroStream(createReadStream('./Account.csv'));

Parsing is **lenient**: a row with an unparseable date is dropped and reported as an `error`; a partially-parseable amount becomes a `warning`. The only thrown conditions are an empty input and a header that matches no dialect (`UnknownDialectError`).

## Supported exports

DEGIRO localizes the **header row** of `Account.csv` to the interface language. Two dialects recognise a header by its words, and a third recognises the file by its shape when neither does:

| Dialect | `id` | Header |
| ---------------- | --------- | --------------------------------------------------------------------------------- |
| `frenchDialect` | `fr` | `Date,Heure,Date de,Produit,Code ISIN,Description,FX,Mouvements,,Solde,,ID Ordre` |
| `englishDialect` | `en` | `Date,Time,Value date,Product,ISIN,Description,FX,Change,,Balance,,Order Id` |
| `genericDialect` | `generic` | any header in DEGIRO's column layout, whatever the labels say |

All three share the positional column layout, and the two language-aware ones share `DD-MM-YYYY` dates. They differ in how they read numbers: `frenchDialect` expects space thousands separators and a comma decimal mark, while `englishDialect` accepts either that or the US `1,060.20` form — DEGIRO does **not** switch the body of the file to English number formatting when you switch the interface language, so an English export can carry either. When a value contains both `,` and `.`, the last one is the decimal mark; a lone separator is a decimal mark unless the value is a run of exact 3-digit groups (`1,060,200`).

Descriptions are a separate axis: switching the interface language rewrites the header but leaves the description text alone, so an English-header export from a French account still says `Frais DEGIRO de courtage`. The built-in matchers therefore recognise both languages regardless of which dialect read the header. Anything they do not recognise becomes an `unknown` movement — never a dropped row.

### The fallback, for every other language

DEGIRO's column layout is the same in every language: twelve columns, where the mutation and the balance each pair a currency cell with an amount cell under a single header label — so the 9th and 11th header cells (0-based indices 8 and 10) are always empty. `genericDialect` recognises that shape and nothing else, reading no header text at all, and interprets dates (`DD-MM-YYYY`, `DD/MM/YYYY`, `YYYY-MM-DD`) and numbers by the same per-value rules as the English dialect. It is registered last, so a French or English export never reaches it.

Because it is a guess, it says so: `dialect.heuristic` is `true` and a warning lands on `ParseResult.warnings` naming the line it was decided on. Nothing silently depends on it.

```ts
const result = parseDegiroCsv(dutchCsv);
result.dialect.id; // 'generic'
result.dialect.heuristic; // true
result.warnings[0].message; // 'No dialect recognised this header, so the "generic" fallback ...'
```

Two smaller guesses ride along:

- If the header tokenizes to a single cell, `parseDegiroCsv` retries with `;` and then a tab before giving up. Passing `delimiter` explicitly turns this off.
- `structuralTradeMatcher` and `structuralFxTradeMatcher` recover trades from the _shape_ of a description — `<qty> <product>@<price> <CCY> (<ISIN>)` — and take the side from the sign of the mutation, money out being a buy. Both run at a negative priority, after every language-aware matcher has declined, so `Koop 1.060 SMI ETF@106,02 CHF (CH0019852802)` classifies as a buy without anyone teaching the library Dutch.

Descriptions with no such shape — a dividend, a fee, a deposit in a language the matchers do not know — stay `unknown` with their amounts intact, so balances still reconcile and cash still adds up. Register a matcher for the wording to name them.

## Domain model

### Movements
Expand Down Expand Up @@ -237,27 +272,31 @@ Every stage is pluggable. You rarely need to fork the library to support a new e

### Custom dialect (new locale / layout)

French and English are built in; here is a third locale — Dutch headers, dot thousands, comma decimals:

```ts
import { parseDegiroCsv, frenchDialect, parseFrenchDateTime, type Dialect } from 'libdegiro';
import { parseDegiroCsv, DEGIRO_COLUMNS, parseDegiroDateTime, type Dialect } from 'libdegiro';

const englishDialect: Dialect = {
id: 'en',
label: 'DEGIRO English',
columns: frenchDialect.columns, // same positional layout
matches: (header) => header.includes('Change') && header.includes('Balance'),
const dutchDialect: Dialect = {
id: 'nl',
label: 'DEGIRO Dutch',
columns: DEGIRO_COLUMNS, // the shared positional layout
matches: (header) => header.includes('Omschrijving') && header.includes('Mutatie'),
parseDecimal: (raw) => {
const n = raw.trim().replace(/,/g, ''); // US thousands
const n = raw.trim().replace(/\./g, '').replace(',', '.'); // 1.060,20
return /^-?\d+(\.\d+)?$/.test(n) ? n : null;
},
parseDateTime: parseFrenchDateTime,
parseDate: (d) => parseFrenchDateTime(d),
parseDateTime: parseDegiroDateTime,
parseDate: (d) => parseDegiroDateTime(d),
};

parseDegiroCsv(csv, { dialects: [englishDialect] });
parseDegiroCsv(csv, { dialects: [dutchDialect] });
// or force it, skipping detection:
parseDegiroCsv(csv, { dialect: englishDialect });
parseDegiroCsv(csv, { dialect: dutchDialect });
```

A dialect registered through `dialects` replaces the built-ins, including the fallback; pass `createDefaultDialectRegistry().register(dutchDialect, { prepend: true })` to keep them.

### Custom classifier (new movement description)

```ts
Expand Down Expand Up @@ -318,4 +357,4 @@ pnpm build # tsdown -> dist/ (ESM + .d.ts + sourcemaps)

### Test fixture

`test/fixtures/Account.csv` is a **synthetic** statement, not a real export. It mirrors the shape of a genuine French DEGIRO file — column layout, movement types, order-id grouping, French decimals with `U+202F` thousands separators, double-spaced product names — and its running balances reconcile exactly, but every figure, date, ISIN and order id is fabricated. Drop your own `Account.csv` at the repo root to try the library against real data; it is git-ignored.
`test/fixtures/Account.csv` is a **synthetic** statement, not a real export. It mirrors the shape of a genuine French DEGIRO file — column layout, movement types, order-id grouping, French decimals with `U+202F` thousands separators, double-spaced product names — and its running balances reconcile exactly, but every figure, date, ISIN and order id is fabricated. `test/fixtures/Account-en.csv` is the same statement behind an English header, which is exactly what DEGIRO produces when you switch the interface language. Drop your own `Account.csv` at the repo root to try the library against real data; it is git-ignored.
43 changes: 40 additions & 3 deletions examples/dashboard/src/components/dropzone.tsx
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
import { useCallback, useRef, useState, type DragEvent } from 'react';
import { FileUp, ShieldCheck } from 'lucide-react';
import { useCallback, useRef, useState, type DragEvent, type ReactNode } from 'react';
import { Download, FileUp, ShieldCheck } from 'lucide-react';
import { Button } from '@/components/ui/button';
import { cn } from '@/lib/utils';
import { SAMPLE_FILE_NAME, sampleCsv } from '@/lib/sample';
import { useStatement } from '@/state/statement-context';

function Ui({ children }: { children: ReactNode }) {
return <span className="text-foreground font-medium">{children}</span>;
}

export function Dropzone() {
const { load, state } = useStatement();
const [dragging, setDragging] = useState(false);
Expand Down Expand Up @@ -53,7 +57,7 @@ export function Dropzone() {
<div className="space-y-1">
<p className="font-medium">Drop your DEGIRO Account.csv here</p>
<p className="text-muted-foreground text-sm">
Export it from DEGIRO under Inbox → Account statement.
English and French statements are both recognised.
</p>
</div>

Expand Down Expand Up @@ -83,6 +87,39 @@ export function Dropzone() {
) : null}
</div>

<section className="text-muted-foreground w-full space-y-3 text-sm">
<h2 className="text-foreground flex items-center gap-2 font-medium">
<Download className="size-4 shrink-0" aria-hidden />
Exporting Account.csv from DEGIRO
</h2>
<ol className="list-decimal space-y-2 pl-5">
<li>
Sign in to <Ui>degiro.com</Ui> in a browser. The statement export lives in the web
client.
</li>
<li>
Open <Ui>Inbox</Ui>, then the <Ui>Account statement</Ui> tab.
</li>
<li>
Set <Ui>Start date</Ui> to the day you opened the account, or anything earlier, and{' '}
<Ui>End date</Ui> to today. Every number here is computed from the rows in the file, so
a narrower range silently gives you partial positions, fees and realized P/L.
</li>
<li>
Leave <Ui>Curr.</Ui> on <Ui>All</Ui> and the product search empty, so no currency or
instrument is filtered out.
</li>
<li>
<Ui>Hide cash movements</Ui> only changes the table on screen. The export contains every
row either way, and this dashboard needs those rows to reconcile your balances.
</li>
<li>
Click the download button at the top right of the table and choose <Ui>CSV</Ui>. Drop
the file it saves — <Ui>Account.csv</Ui> — above.
</li>
</ol>
</section>

<p className="text-muted-foreground flex items-center gap-2 text-xs">
<ShieldCheck className="size-4 shrink-0" aria-hidden />
Your statement is parsed in this tab and never uploaded. The page blocks all network access,
Expand Down
7 changes: 4 additions & 3 deletions examples/dashboard/src/components/sections/health.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import type { BalanceDiscrepancy } from 'libdegiro';
import { CheckCircle2, Copy, Info, TriangleAlert } from 'lucide-react';
import {
describeHealthNotes,
plural,
describeHealthProblems,
diagnosticsText,
explainDiscrepancy,
Expand Down Expand Up @@ -109,9 +110,9 @@ export function HealthSection() {
</AlertTitle>
<AlertDescription className="flex flex-col gap-2">
<span>
Parsed {health.rows} rows using the {result.dialect.id} dialect. {health.errors.length}{' '}
errors, {health.warnings.length} warnings, {health.unknown.length} unrecognised
descriptions.
Parsed {plural(health.rows, 'row')} using the {result.dialect.id} dialect.{' '}
{plural(health.errors.length, 'error')}, {plural(health.warnings.length, 'warning')},{' '}
{plural(health.unknown.length, 'unrecognised description')}.
</span>
{problems.length > 0 ? (
<ul className="list-disc space-y-1 pl-4">
Expand Down
25 changes: 20 additions & 5 deletions examples/dashboard/src/lib/analytics/health.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ export interface HealthReport {
readonly unparseableExchanges: number;
readonly reconciliation: ReconciliationReport;
readonly range: DateRange | null;
/** `true` when the header matched no language and the layout fallback read the file. */
readonly heuristicDialect: boolean;
readonly ok: boolean;
}

Expand Down Expand Up @@ -54,6 +56,7 @@ export function buildHealthReport(result: ParseResult, unparseableExchanges: num
unparseableExchanges,
reconciliation,
range: statementRange(result.movements),
heuristicDialect: result.dialect.heuristic === true,
ok:
result.errors.length === 0 &&
unknown.length === 0 &&
Expand All @@ -62,7 +65,8 @@ export function buildHealthReport(result: ParseResult, unparseableExchanges: num
};
}

const plural = (count: number, noun: string): string => `${count} ${noun}${count === 1 ? '' : 's'}`;
export const plural = (count: number, noun: string): string =>
`${count} ${noun}${count === 1 ? '' : 's'}`;

/**
* Why `ok` is false, in the user's terms.
Expand Down Expand Up @@ -107,11 +111,22 @@ export function describeHealthProblems(report: HealthReport): string[] {
* ignore the panel; hiding it entirely leaves a real number unexplained.
*/
export function describeHealthNotes(report: HealthReport): string[] {
const notes: string[] = [];

if (report.heuristicDialect) {
notes.push(
'No dialect recognised this header, so the file was read by its column layout alone and its dates and amounts were interpreted by guesswork. Dates, amounts and balances above are worth a spot-check against the statement.',
);
}

const rounding = report.reconciliation.rounding.length;
if (rounding === 0) return [];
return [
`${plural(rounding, 'balance transition')} ${rounding === 1 ? 'is' : 'are'} off by less than one centime. DEGIRO rounds a half-unit price down in its amount column and up in its balance column, so the statement disagrees with itself. Nothing here is a parsing error.`,
];
if (rounding > 0) {
notes.push(
`${plural(rounding, 'balance transition')} ${rounding === 1 ? 'is' : 'are'} off by less than one centime. DEGIRO rounds a half-unit price down in its amount column and up in its balance column, so the statement disagrees with itself. Nothing here is a parsing error.`,
);
}

return notes;
}

/** The one-sentence reading of a discrepancy: what it means and what to do. */
Expand Down
22 changes: 22 additions & 0 deletions examples/dashboard/test/positions.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,28 @@ describe('describeHealthProblems', () => {
expect(describeHealthNotes(report)[0]).toContain('1 balance transition is off');
});

it('warns when the layout fallback read the file, ahead of any rounding note', () => {
const dutchCsv = [
'Datum,Tijd,Valutadatum,Product,ISIN,Omschrijving,FX,Mutatie,,Saldo,,Order Id',
'18-11-2024,00:00,18-11-2024,,,Storting,,CHF,"9.000,00",CHF,"9.000,00",',
'',
].join('\n');
const parsed = parseDegiroCsv(dutchCsv);
const report = withDiscrepancies(buildHealthReport(parsed, 0), [
discrepancy('rounding', '-0.01'),
]);

expect(report.heuristicDialect).toBe(true);
const notes = describeHealthNotes(report);
expect(notes).toHaveLength(2);
expect(notes[0]).toContain('read by its column layout alone');
expect(notes[1]).toContain('1 balance transition is off');
});

it('says nothing about the dialect when the header was recognised', () => {
expect(buildHealthReport(result, 0).heuristicDialect).toBe(false);
});

it('explains a half-centime rounding from the row that caused it', () => {
const entry = discrepancy('rounding', '-0.01', {
statedMutation: new Money('-1528.28', 'CHF'),
Expand Down
Loading