[high] fix(tsk): stop every fls-driven extractor skipping deleted files

[elhoim]

BLUF

• Priority: high.
• Every fls-driven extractor silently skips deleted files. Deleted files are the evidence an examiner came for — a cleared Security.evtx, recoverable as a deleted entry, was never handed to icat.
• Root cause: fls prints the deleted marker as a separate * token after the type pair, before the inode. Eleven copies of a row regex across six modules spelled that as [rd]/[rd*], treating * as a character inside the type pair — so the pattern expected a digit where real output has *.
• The loss was silent: undeleted files in the same directory matched fine, so extraction looked successful.
• The same spelling also restricted types to r and d, dropping symlinks (l/l) and the virtual $OrphanFiles node (V/V) that holds recovered deleted content.
• Fix: one parse_fls_rows in mulder/patterns.py, mirroring the parse_mmls_rows merged in [high] fix(tsk): correct mmls partition-row regex and share one parser #68, with all eleven call sites pointed at it.

The bug

Genuine fls -r -p output, The Sleuth Kit 4.12.1, from a purpose-built ext4 image with two deleted files:

d/d 13:	Users
d/d 14:	Users/alice
r/r 15:	Users/alice/NTUSER.DAT
r/r * 16:	Users/alice/capture.pcap
l/l 17:	Users/alice/link_to_passwd
d/d 11:	lost+found
d/d 18:	Windows
d/d 19:	Windows/System32
d/d 20:	Windows/System32/winevt
d/d 21:	Windows/System32/winevt/Logs
r/r * 22:	Windows/System32/winevt/Logs/Deleted.evtx
r/r 23:	Windows/System32/winevt/Logs/Security.evtx
r/r 24:	Windows/System32/winevt/Logs/System.evtx
V/V 1025:	$OrphanFiles

The regex, repeated eleven times:

r"^[rd]/[rd*]\s+(\d+(?:-\d+-\d+)?):\s+(.+)\s*$"

Run against that output, it matches ten of the fourteen rows and misses:

$OrphanFiles
Users/alice/capture.pcap                          <- deleted
Users/alice/link_to_passwd                        <- symlink
Windows/System32/winevt/Logs/Deleted.evtx         <- deleted

Security.evtx and System.evtx in the same directory matched, which is why nothing ever looked wrong.

The eleven copies, all identical in substance:

src/mulder/server/tools/artifacts.py            (6 copies: browser history x4, plist x2)
src/mulder/server/tools/extract/registry.py     (1)
src/mulder/server/tools/extract/disk_pcap.py    (1)
src/mulder/server/tools/extract/app_files.py    (1)
src/mulder/server/tools/extract/evtx.py         (1)
src/mulder/server/tools/extract/tsk.py          (1)

The fix

One parser in mulder/patterns.py, directly alongside parse_mmls_rows:

FLS_ROW_RE: re.Pattern[str] = re.compile(
    r"^(?P<name_type>[A-Za-z\-])/(?P<meta_type>[A-Za-z\-])\s+"
    r"(?P<deleted>\*\s+)?"
    r"(?P<inode>\d+(?:-\d+-\d+)?):\t"
    r"(?P<path>.+?)\s*$",
    re.MULTILINE,
)

FlsEntry carries deleted as data rather than discarding it, and exposes base_inode (the NTFS 6083-128-1 form reduced to 6083, which is what icat wants). fls_file_entries is the extractor-facing view: directories dropped, everything else kept — including - (unknown) types, which is how orphaned files present.

The separator between inode and path is a literal tab, which is what makes paths containing spaces safe to parse.

In artifacts.py, _find_inodes_by_pattern becomes _find_inodes_by_path — callers now pass only a path pattern:

    browser_patterns = [
        (r"/Chrome/.*?/History$", "chrome"),
        (r"/Firefox/Profiles/.*?/places\.sqlite$", "firefox"),
        (r"/Safari/History\.db$", "safari"),
        (r"/Google/Chrome/.*?/History$", "chrome"),
    ]

The row grammar is stated once, so a future fix lands in one place.

One adjacent dead check, removed

app_files.py filtered by size using _FLS_SIZE_RE = re.compile(r"\t(\d+)\t") against the fls listing. fls -r -p emits no size column — every row has exactly one tab — so max_file_size_kb could never fire. Verified against the real output above: zero matches.

Removing the duplicated row regex forces the issue, so leaving a provably-dead filter in place was not an option. The threshold now applies to the bytes icat returns, where the size is actually known. The repo's own test documented the inertness — test_size_limit_with_tab_delimited_sizes asserted all three files matched despite its name — and is renamed to say what it checks.

Deliberately out of scope

• fls -r -m bodyfile output (run_mactime) is a different, pipe-delimited format and is untouched.
• The mmls parser is [high] fix(tsk): correct mmls partition-row regex and share one parser #68's concern, already merged; disk.py and server/tools/tsk.py needed no change here.
• No framework, no new abstraction beyond the single parser function, and no second status taxonomy.

Verification

• Real-tool grounding: built an ext4 image with mkfs.ext4 -d, deleted two files with debugfs (no mount, no root), and ran the real fls from The Sleuth Kit 4.12.1. Both the broken and fixed patterns were run against that captured output; the fixture in tests/test_fls_parsing.py is that output verbatim.
• uvx pre-commit run --all-files — ruff, ruff-format, mypy all pass. (Staged first: --all-files enumerates via git ls-files and cannot see untracked files.)
• pytest tests/ -q — 874 passed, 0 failed. No deselect was needed; test_disk_pcap.py::...::test_size_filtering_skips_large_pcaps passes here, so this branch is green on the full suite.
• Premise check — with src/mulder/server/tools/extract/evtx.py restored to origin/main and the new tests kept, the behavioural tests fail on the assertion (not on an import or signature mismatch, because _extract_evtx_from_image's signature is unchanged):

FAILED tests/test_evtx_deleted_extraction.py::test_a_deleted_evtx_is_extracted
E   AssertionError: assert 'Windows_System32_winevt_Logs_Deleted.evtx' in {'Windows_System32_winevt_Logs_Security.evtx'}

FAILED tests/test_evtx_deleted_extraction.py::test_icat_is_asked_for_the_deleted_inode
E   AssertionError: icat was never asked for the deleted inode: [['icat', '/evidence/disk.dd', '23']]
    assert '22' in {'23'}

FAILED tests/test_evtx_deleted_extraction.py::test_a_timeout_on_one_file_does_not_abort_the_rest
E   AssertionError: assert '22' in ['23']

3 failed, 2 passed

The two that pass on both trees are the narrowness tests — the undeleted sibling still extracts, directories still are not — so they pin the fix rather than the bug. test_the_old_spelling_really_did_miss_them embeds the previous regex directly, so the premise stays pinned even after the old spelling is gone from the tree.

Context

Recovered from closed PR #77. That branch was stacked on #68 and carried its mmls work; #68 has since merged, so this is rebuilt fresh from current main (2e5432c) with only the fls half — and re-verified against the real tool rather than cherry-picked.

This is the same shape as #68, which was merged: one shared parser replacing duplicated regexes that had each inherited the same misreading of a TSK output format. It is a single concern, not a framework. Checked against all fifteen open PRs: only #86 touches artifacts.py, for read-only SQLite URI escaping — a disjoint concern, different lines, no conflict.

🤖 Generated with Claude Code