Phantomatics

Coordinated inauthentic behaviour (CIB) detection for social-media data. Finds which accounts act as one organism, keeps that strictly separate from who is behind them, and tests itself against an adversary that adapts.

Name after Stanisław Lem's Summa Technologiae — fantomatyka: engineered realities indistinguishable from the real thing.

---

What it does, and what it deliberately does not

Phantomatics answers one question well: given a snapshot of posts, which accounts are coordinating? It does this by fusing five independent signals (shared sources, shared infrastructure, reused media, near-duplicate text, tight co-retweet timing) into a weighted graph and detecting dense communities.

It does not tell you who runs those accounts. That step — attribution to a named actor — is the epistemically weakest and most consequential part of this problem: a false "foreign agent" label attached to a real person is real harm and, in the EU, real liability. So attribution is never automated here. The system produces coordination suspicion with an explicit confidence and evidence trail, and hands the final leap to a human.

This separation is not a disclaimer bolted on top; it is enforced by the architecture and by tests (test_report_never_claims_attribution).

Responsible use

This is defensive CTI tooling, in the lineage of open academic work on coordination detection (Pacheco & Menczer / OSoMe, Luceri & Ferrara) and public investigative work (DFRLab, EU DisinfoLab, Graphika). It is meant for research, threat intelligence, and platform-integrity work.

• It detects coordination, which is not the same as inauthenticity or malice — legitimate newsrooms, activist networks, and ad campaigns also coordinate. A detected cluster is a lead, not a verdict.
• Do not use its output to publicly accuse, deplatform, or target individuals without human review and corroborating evidence.
• If you point it at real data, you are responsible for the legal basis (in the EU, GDPR applies to personal data even when public; a narrow research/legitimate- interest basis is required).
• The bundled data is fully synthetic. Handles, domains (e.g. the fake typosquat wiadomosci-p0lska.info), and narratives are generated — none are real indicators of compromise.

Install

git clone https://github.com/jerzy99jerzy/phantomatics
cd phantomatics
pip install -e ".[dev]"

Python 3.11+. Core deps: DuckDB, PyArrow, python-igraph, leidenalg, SciPy.

Quickstart

# 1. generate the synthetic worlds (nothing runs against real data by default)
phantomatics fixture --out data                 # naive world
phantomatics fixture --out data --adversarial   # adaptation-hardened world
phantomatics fixture --out data --multi-origin  # source-distribution world

# 2. detect coordination
phantomatics detect --snapshot data/events.parquet \
    --truth data/ground_truth.json --out reports

# 3. origin-level meta-clustering (catches distributed-source evasion)
phantomatics meta --snapshot data/events_multi.parquet \
    --truth data/ground_truth_multi.json --out reports

To run against your own data, point --snapshot at any Parquet file matching EVENT_SCHEMA (see phantomatics/schema.py). A vendor adapter is a schema translation and nothing above it changes.

Real data

An adapter for the public IRA / FiveThirtyEight dataset is included:

phantomatics ingest --input IRAhandle_tweets_1.csv --format ira --out data --limit 40000
phantomatics detect --snapshot data/events.parquet --out reports

ingest prints a channel-availability report: real dumps are lossy (no perceptual hashes, retweet structure stripped, URLs shortened), so several channels go dark and recovery must be read with that in mind. Confronting this dataset is what added the content_hashtag channel — hashtags survive export when retweet structure does not — and taught the project that a single attributed actor is multi-narrative, so clean recovery means splitting it into sub-operations, not recovering one monolith. See docs/SPRINT6_REAL_DATA.md.

How it works (one paragraph)

Every channel is expressed as a sparse account × token incidence matrix B (token = a shared domain, source, media hash, text bucket, or co-retweet window), IDF-weighted so rare shared tokens count more than popular ones. The coordination graph is the sparse projection A = B·Bᵀ — no O(k²) pairwise materialization, which also removes a denial-of-service vector against the detector itself. Leiden finds communities; each is scored by density × corroboration × mass, where the three factors block three distinct attacks (single-channel adaptation, detector poisoning, size dominance). Directed retweet flow then assigns roles: origin (seeder), amplifier, bridge (the laundering node where botnet content crosses into authentic reach), target. When an adversary distributes the narrative across many sources to defeat the shared-source signal, the same projection one level up (origin × narrative) recovers the coordination, because the narrative is the payload and cannot be rotated.

Full detail: docs/MECHANISMS.md.

Adversary-adaptation testing

The system is validated against three synthetic worlds of increasing difficulty, each with explicit ground truth:

world	what it tests	recovery
naive	clean signal, all channels present	F1 = 0.96
adversarial	domain rotation + jitter + unique LLM text	F1 = 0.92
multi-origin	distributed sources (shared-source defeated)	meta F1 = 1.00
multi-narrative	one actor, several parallel narratives	clean sub-cluster split
doppelganger*	infra-centric op (typosquat domains + recycled assets)	F1 = 0.97

* Doppelganger world is synthetic, built from documented TTPs — not real IOCs. It mirrors the IRA channel profile: infra + phash dominate, hashtags near-zero, proving the channel architecture generalizes across threat models. A --profile flag (full_api / public_dump) re-weights channels per source, since a public dump strips retweet structure that a full API feed keeps.

The adversarial world is a first-class citizen, not an afterthought: it is what actually validated the channel weights. On the naive world alone, "infra holds longest" looked true; under adaptation it was false — shared infra holds, but domain rotation is cheap. That reversal is why shared_source is weighted highest.

Project layout

phantomatics/        the package
  schema.py          normalized event schema (the adapter boundary)
  adapters/          real-data adapters (IRA / FiveThirtyEight)
  fixture.py         three synthetic worlds + ground truth
  channels.py        pairwise channels (test oracle only, not user-facing)
  text.py            SimHash / Hamming primitives
  bipartite.py       sparse account×token core (the efficient engine)
  detect.py          pairwise detection (correctness oracle for the bipartite engine)
  detect_bipartite.py  bipartite detection (the engine)
  scoring.py         single home for weights + the scoring formula
  roles.py           L3 role assignment from directed retweet flow
  meta.py            origin-level meta-clustering (Sprint 4)
  evaluate.py        recovery vs ground truth
  cli.py             fixture / detect / meta commands
tests/               end-to-end smoke tests (run through the CLI)
tools/               sweep_roles.py (threshold calibration)
docs/                design notes and analysis

Status and limitations

Alpha. All validation is on synthetic fixtures. The honest expectation for a real feed: lower recall (public ground-truth is a survivorship-biased sample of operators who already got caught) and lower precision (richer organic noise). Wiring real datasets (Twitter/X IO archives, Doppelganger domain lists) for offline validation is the next step and will likely break something — that is the point of doing it.

Design notes, code review, and the engine-equivalence analysis are in docs/.

License

See LICENSE.