Skip to content

Latest commit

 

History

11 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

interview-synthesis

An agent skill for incremental customer interview synthesis.

Most synthesis workflows assume you finish all your interviews, then analyze. Real discovery doesn't work that way — you talk to two people, learn something, talk to two more. This skill is built for that: code 1–3 transcripts at a time into a running evidence base, then regenerate a synthesis report whenever you want a current read.

Sized for studies up to roughly 15 interviews.

Status: untested. Written but never run against real interview data. See Status below before relying on it.

The one rule that matters

Append the evidence. Regenerate the interpretation.

Coded excerpts are append-only — once P03's words are on disk with a participant ID, they never change. But themes, findings and the report are re-derived from the entire evidence base every time, never edited in place.

This exists to defeat anchoring. If each new interview only ever gets filed under themes that already exist, the study freezes around whatever the first two people happened to say, and interview 9 that contradicts interview 2 gets quietly absorbed instead of breaking the theme open.

Modes

Mode What it does
init Set up a study — objective, topic areas, segments, evidence threshold
add Fold in 1–3 new transcripts; extend the codebook; report what changed
report Re-derive themes from all evidence and regenerate the report
status Saturation curve, thin topics, and whether to keep interviewing

What it codes for

15 indicator types across behavior, motivation, commercial signal, meaning, and the ones that break your model. A few that carry the most weight:

  • workaround — the hack they built to survive today. The highest-signal code in the set: someone who built a spreadsheet has already paid for a solution in time.
  • trigger — what changed that made them start looking. Often more actionable than what they say they want.
  • messaging — captures what they repeat back, what they ask about, and what they ignore entirely. The third is the one everyone forgets to record.
  • surprise — with a hard check: zero surprises in an interview means it was a pitch, or the coding was confirmation-seeking.

Every excerpt is rated strong / moderate / weak, where strong means unprompted, specific, and about actual past behavior. A finding may never rest on weak evidence alone, and volume of weak never sums to strong.

Report shape

  1. Summary — top findings with confidence labels and counts, what's working, what changed since the last report, what's still unknown, and a concrete recommendation.
  2. One section per topic area — defined per study in config.md, not hardcoded. Existing flow, pain points, messaging testing, pricing sensitivity, whatever the study needs. A topic with zero evidence still gets a section saying so.
  3. Follow-up questions, limits, appendices.

Ten red-flag checks run as a gate before any report is shown — including three specific to incremental studies: anchoring on early interviews, coverage passed off as consensus, and stale evidence when the product ships changes mid-study.

Install

git clone https://github.com/AgLyx3/interview-synthesis.git ~/.agent-skills/interview-synthesis
ln -sfn ~/.agent-skills/interview-synthesis ~/.claude/skills/interview-synthesis

Swap the target for ~/.codex/skills/ or any other agent's skills directory — the skill is plain Markdown with no binaries or dependencies.

Layout

SKILL.md                     Modes, workspace layout, the core rule
references/coding.md         Indicator taxonomy, evidence strength, excerpt format
references/themes.md         Theme derivation, confidence, saturation, follow-ups
references/report.md         Report template
references/red-flags.md      Quality gate — run before showing any report
scripts/tally.py             Counts the evidence base (stdlib only, no deps)
assets/config-template.md    Per-study config
evals/evals.json             Test prompts and assertions
evals/fixtures/              Three synthetic 30–40 min interviews for exercising the skill

Counting is scripted, interpretation isn't

scripts/tally.py reads evidence/ and returns the code × participant matrix, which codes clear the evidence threshold, the strength distribution, per-topic coverage, and new codes per participant:

python3 scripts/tally.py research/my-study
python3 scripts/tally.py research/my-study --json

Tallying by hand across a dozen files is exactly the arithmetic that goes quietly wrong — a miscount doesn't announce itself, it just produces a theme with the wrong prevalence. The script also warns about malformed excerpt headers, participants unaccounted for on a topic, and codes that clear the threshold on weak evidence alone.

Status

Exercised against synthetic data; never run on real interview data.

The eval suite ran on 2026-08-23: three chained evals (initaddreport) against a no-skill baseline, 28 assertions. With the skill, 27/28 — the miss was an assertion that turned out not to be applicable, since every topic area drew evidence. Baseline scored 10/28: competent prose, but it rewrote its findings file on the second interview, kept no per-participant evidence, and had no codebook, log, or saturation signal. Cost about 2× the tokens and 3× the wall-clock of the baseline.

That run found three real defects — inconsistent code granularity, append-only conflicting with codebook merges, and brittle topic joining in tally.py — all since fixed.

The limitation that remains: the fixtures were written by the same author as the skill, so they share its assumptions about what a transcript looks like. Real interviews are messier — worse transcription, more digression, less tidy structure. Passing on synthetic data is evidence the mechanics hold together, not evidence the analysis is good.

The indicator taxonomy, the ≥3-excerpts-from-≥2-participants threshold, the 4–8 theme target and the report length caps are all reasoned defaults, not numbers tuned against actual output. Treat the first real study as a shakedown run, and change the defaults in place rather than working around them.

Credit

Assembled from three prior skills rather than written from nothing — see ATTRIBUTION.md.

License

MIT for the original work, with one file carrying an additional restriction inherited from upstream. See LICENSE and ATTRIBUTION.md.

About

Incremental customer interview synthesis skill for AI coding agents — code 1-3 transcripts at a time into a running evidence base, regenerate the report on demand

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages