Development

Development

How to work on RE-Toolkit: setting up, adding stages and tools, testing, and the gates a change must clear.

This page is the practical companion to CONTRIBUTING.md, which holds the standards, and to Architecture and Design, which explains why the design is what it is.

Contents

• Setting up
• Running from a checkout
• Repository layout
• Adding an analysis stage
• Adding a scoring signal
• Adding a tool to the installer
• Adding a file type
• Testing
• Repository checks
• Diagnostic tools
• Definition of done
• Releasing

Setting up

git clone https://github.com/Sandler73/RE-Toolkit.git
cd RE-Toolkit

# Provision the toolchain, but keep /opt/retoolkit free for your checkout
sudo ./install-retoolkit.sh --skip-source

# Development dependencies
sudo apt install shellcheck shfmt bats
python3 -m pip install --user pytest ruff

Work in a disposable virtual machine. You will be running hostile files against code you are actively changing.

Running from a checkout

Point the library and stage directories at your working tree so the driver uses your code rather than an installed copy:

export RETOOLKIT_LIB_DIR=~/dev/retoolkit/lib
export RETOOLKIT_STAGES_DIR=~/dev/retoolkit/stages/static
~/dev/retoolkit/analyze-binaries.sh -t sample.exe -o ./out --log-level=debug

Debug logging is worth keeping on during development, since skip reasons and tool-resolution decisions are logged at that level.

Repository layout

Path	Contents
install-retoolkit.sh	Layered provisioner
analyze-binaries.sh	Driver: argument parsing and orchestration
GhidraDump.py	Ghidra postscript
lib/	Shared modules sourced by the driver
stages/static/	One file per stage
tools/	Development and repository-check utilities
tests/bats/	Shell test suite
tests/python/	Python test suite
wiki/	Wiki source, published to the separate wiki repository

Module responsibilities are described in Architecture and Design. The rule that keeps them honest: a stage never decides whether it should run, only how to do its job.

Adding an analysis stage

1. Choose a number

Stage filenames are numbered by where their output belongs in a directory listing, not by execution order. Place a new stage where an analyst would expect to find its output.

2. Create the file

stages/static/NN-name.sh, defining exactly one function, with a complete header block. The required sections are Synopsis, Description, Execution Parameters, Provides, Notes, and Version. tools/check-headers.py enforces this.

#!/usr/bin/env bash
# =============================================================================
# stages/static/NN-name.sh
# =============================================================================
#
# Synopsis:
#     One sentence describing what this stage does.
#
# Description:
#     What it does, why it is built this way, and anything a maintainer would
#     otherwise have to rediscover by reading the code.
#
#     Sourced by analyze-binaries.sh; not directly executable.
#
# Execution Parameters:
#     $1  target   Path to the sandboxed copy of the target binary.
#     $2  outdir   Path to this target's output directory.
#
# Provides:
#     stage_name()
#
# Output subtrees:
#     ${outdir}/NN-name/
#
# Skip controls:
#     SKIP_NAME
#
# Notes:
#     Release history is recorded in CHANGELOG.md.
#
# Version:
#     3.7.3 - 2026-05-03
# =============================================================================

stage_name() {
    local target="$1" outdir="$2"
    local out="${outdir}/NN-name"

    [[ "${SKIP_NAME:-0}" == "1" ]] && { log_info "NN-name: skipped"; return 0; }

    if ! command -v sometool >/dev/null 2>&1; then
        log_warn "NN-name: sometool not installed, skipping"
        return 0
    fi

    mkdir -p "$out"
    run_tool "sometool" "$TOOL_TIMEOUT" sometool --analyze "$target" \
        > "${out}/sometool.txt"

    return 0
}

The contract every stage honors:

• Take $1 target and $2 outdir
• Write only inside your own output subdirectory
• Invoke tools only through run_tool or run_shell
• Honor the skip control
• Skip cleanly with a logged reason when a tool is unavailable
• Return, never exit
• Never modify the target

exit is the failure that matters most: it terminates the driver and discards every remaining target in a batch.

3. Wire it into the dispatcher

Add the call in lib/dispatch.sh, in the branches for the types it applies to, positioned correctly in the runtime order. Consult the order documented in that file's header before inserting.

4. Register the skip control

In analyze-binaries.sh argument parsing:

--no-name)           SKIP_NAME=1; shift ;;

5. Feed the summary

A stage whose findings never reach _summary.json has not delivered anything an analyst can use. If the stage produces findings that should influence the verdict, parse its output in stages/static/85-summary.sh and contribute weighted signals.

6. Document it

Regenerate the Stage Reference and add a changelog entry. The reference page is generated from stage headers, so an accurate header produces accurate documentation.

Adding a scoring signal

Signals are contributed in stages/static/85-summary.sh:

add_signal("signal_name", 20, f"evidence: {specific_observation}")

Three rules:

Evidence must be specific. "Suspicious imports" is not evidence. "Imports VirtualAllocEx, WriteProcessMemory, CreateRemoteThread" is.

Do not double-count. If your signal describes the same underlying observation as an existing one, suppress the narrower signal when the composite fires. Otherwise one fact inflates a score by being counted several ways.

Update the test mirror. The scoring model is covered by a golden-sample regression test that deliberately mirrors the implementation, because the logic lives inside a bash heredoc and cannot be imported. Changing a weight or a band threshold means updating tests/python/test-scoring-model.py in the same commit.

Adding a tool to the installer

Prefer the distribution package. Use a source or vendor install only when the tool is genuinely not packaged.

• Add it to the appropriate layer
• Provide a source-build fallback where a rolling distribution may not carry it
• Make the install idempotent: detect an existing install and skip it
• Add it to the post-install verification table
• Record provenance for dual-use tooling: where it came from, and what verifies it
• Never fetch over plain HTTP

Then integrate it into a stage through run_tool, and parse its output into the summary. Installing a tool that nothing invokes adds install time and no capability.

Adding a file type

1. Add detection to lib/detect-type.sh, ordered so more specific signatures win
2. Add a dispatch branch in lib/dispatch.sh
3. Add or extend the stages that handle it
4. Add parsing to the summary stage
5. Document the type in the README table and the wiki

Detection ordering is the part most easily got wrong. A packed PE matches both the packer signature and the PE signature, so the packer test must come first.

Testing

# Shell tests
bats tests/bats

# Python tests
python3 -m pytest tests/python -v

# Everything CI runs, locally
shellcheck $(git ls-files '*.sh')
shfmt -d -i 4 -ci $(git ls-files '*.sh')
ruff check .
python3 tools/check-no-emdash.py
python3 tools/check-headers.py
python3 tools/check-version-consistency.py

Test against a real binary before claiming a stage works. Synthetic fixtures verify that code runs; they do not verify that a tool produces what the parser expects from a real target. Most defects found in practice were cases where a tool exited zero and produced output the parser could not use.

Repository checks

Three checkers enforce standards that review alone does not catch reliably.

Check	Enforces
tools/check-no-emdash.py	No U+2014 anywhere in the repository
tools/check-no-disclosures.py	No personal, host, credential, or engagement detail
tools/check-headers.py	Every code file carries a complete header block
tools/check-version-consistency.py	One version across constants, headers, and changelog

The disclosure check deserves a note, because what it catches does not look like a mistake. Development happens against real targets on a real workstation, so an operator home path in a usage example, a workstation hostname in a bug note, or the filename of an analyzed sample all read as ordinary technical detail. They pass review precisely because they are accurate. Published, they disclose who analyzed what, from which machine, and when.

The check covers four categories: identity, host, credentials, and engagement, plus authorship. The authorship category keeps build tooling out of the published repository: assistant or vendor attribution, model names, generated-by markers, build-sandbox paths, and co-author trailers.

Two deliberate non-rules are worth stating, because both look like gaps:

• "LLM" is not flagged. GhidraDump.py documents that its plain-text output is designed to be fed to a language model. That is a product design statement, not an attribution, and flagging it would be a false positive that teaches reviewers to ignore the check.
• Allowlisting is scoped to the matched text, never the whole line. One benign token must not excuse a finding beside it. A machine-generated commit trailer carries a bot address that is legitimately allowlisted, and under line scope that address silently suppressed the attribution marker sitting next to it on the same line.

Use /path/to/..., sample.exe, or example.com in examples, and attribute findings to "the operator". The check exempts only itself and its own test, both of which must contain the patterns to work at all. There is deliberately no inline suppression directive: one would let any line silence the check.

tools/validate-mermaid.mjs parses every mermaid diagram in the documentation. GitHub renders diagrams server-side, so an invalid diagram does not fail a build; it renders as an error box for a reader. The check makes that a build failure instead.

All four run in CI. Run them locally before opening a pull request.

ShellCheck policy and backlog

ShellCheck runs in two passes with different consequences.

Pass	Severity	Effect
Blocking	error	Fails the build
Advisory	warning	Reported, does not fail the build

The repository is clean at error severity. It carries a backlog of warning-severity findings, and being explicit about why is more useful than implying they were overlooked.

Most of the backlog is SC2164, a pushd or popd whose failure is not handled, concentrated in install-retoolkit.sh. ShellCheck's suggested remedies are || exit or || return, and neither is correct there:

• Those call sites are at top level in the installer's main flow, not inside a function, so return is invalid.
• exit contradicts a deliberate design decision. The installer does not use set -e because it must survive an individual component failure, record it, and report everything unresolved in the verification table. Aborting on a failed pushd would leave a half-provisioned system and no summary.

The correct fix is to restructure each site so the failure is handled and the install continues, which is real work rather than a flag. It is tracked separately and should not be done blind, since it changes control flow in code that can only be exercised by a full install on a real host.

One site sits inside a stage function, where return would be valid. It is deliberately left alone as well. The stage cannot be exercised without a provisioned host and a real sample, and changing control flow in working analysis code to satisfy a linter is not a trade this project makes. Linting describes the code; it does not license rewriting code that cannot be tested.

New code is expected to be warning-clean. Check before opening a pull request:

make shellcheck            # blocking gate, must pass
make shellcheck-warnings   # advisory, review your own additions

Python linting

ruff.toml holds the configuration. Repository tooling and tests are held to the full rule set. Two files carry justified per-file ignores:

• GhidraDump.py must run under both Jython 2.7 and CPython 3, and executes inside a Ghidra script context where names such as currentProgram and getScriptArgs are injected at runtime. They are undefined to a static reader and defined at execution. Import order is also deliberate, because a compatibility preamble must run before any Ghidra module is touched.
• tools/ghidra-diagnostic.py imports late and conditionally on purpose: it probes the environment in stages, so an import that raises is a diagnostic result rather than a failure.

Add a suppression only when the reason is structural, and record the reason alongside it.

Why shfmt is not a gate

shfmt is available as make format but is not part of lint, check, or CI. This codebase predates it and does not follow its default style. Adopting it would rewrite roughly 26,000 lines of working analysis shell to satisfy a formatter the project never used, which is churn with real regression risk and no correctness benefit. The target exists for anyone who wants the diff; matching it is not required.

Diagnostic tools

Tool	Purpose
tools/ghidra-diagnostic.py	Exercise the Ghidra path in isolation
tools/stage30-repro.sh	Reproduce a Ghidra stage run
tools/file-stalker.sh	Watch filesystem activity during a run
tools/rename-trace.sh	Trace rename operations
tools/v13-bisect.sh	Bisect a behavior change across stages

file-stalker.sh is the tool to reach for when you suspect a stage is writing outside its output directory, which is a contract violation worth catching early.

Definition of done

A change is not finished until all of these hold:

• bash -n passes on every modified shell file
• shellcheck reports no new findings
• Both test suites pass
• tools/check-no-emdash.py exits 0
• tools/check-no-disclosures.py exits 0
• tools/check-headers.py exits 0
• tools/check-version-consistency.py exits 0
• Header blocks are accurate and complete
• Documentation updated: wiki for behavior, CHANGELOG.md for history
• Any new tool invocation is bounded and recorded in the ledger
• Exercised against a real binary

State in the pull request what you verified and what you did not. A change verified in part is acceptable when the gap is declared; a change claimed as verified when it was not is not.

Releasing

1. Update RETOOLKIT_VERSION in install-retoolkit.sh and ANALYZER_VERSION in analyze-binaries.sh
2. Update the Version line in every header block the change touched
3. Add a CHANGELOG.md entry with the date and the substantive changes
4. Update the version stated in wiki/_Footer.md and wiki/Home.md
5. Run every repository check and both test suites
6. Tag the release as vX.Y.Z

Tagging triggers the release workflow, which re-validates version consistency, header compliance, and the em-dash guard, confirms the tag matches RETOOLKIT_VERSION, confirms a changelog entry exists, runs the test suites, and then builds and publishes the archives with checksums. A tag whose version does not match the source fails the build rather than publishing a mislabeled release.

Versioning follows Semantic Versioning 2.0.0.