Skip to content

Replace the Apple-vs-Adapty install gap with a standard metric set - #4

Open
KQLYYY wants to merge 1 commit into
mainfrom
remove-install-gap-standard-metrics
Open

Replace the Apple-vs-Adapty install gap with a standard metric set#4
KQLYYY wants to merge 1 commit into
mainfrom
remove-install-gap-standard-metrics

Conversation

@KQLYYY

@KQLYYY KQLYYY commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Release Apple Ads plugin 0.3.0.

1. Remove the Apple-vs-Adapty install comparison

Every review and audit ended with a section like:

Apple (total_installs): 5,045. Adapty installs: 4,123. Absolute gap: 922 (18.3% of Apple's count). Apple and Adapty use different attribution and event definitions, so this gap is a comparison signal, not proof either system is wrong — not investigated further.

It compares two numbers built on different attribution models and different event definitions, then immediately disclaims that the difference means nothing and that nobody will look into it. It reads like a finding, and it occupied a headline output section in both the weekly check-in and the account-health audit.

Removed from:

  • skills/apple-ads/references/playbooks/weekly-review.md and skills/apple-ads-audit/references/playbooks/account-health.md — formula blocks, output sections, examples, gap-specific failure modes. Neither playbook requests adapty_installs any more.
  • skills/apple-ads-audit/SKILL.md — the ## Simple install comparison section, the routing bullet, and the skill description, including the "why are Apple and Adapty installs different" trigger (replaced with "which campaigns are wasting spend").
  • Routing and commands: audit references/INDEX.md, commands/asa-audit.md, commands/asa-review.md, and bid-optimization.md, which also fetched both install counts.
  • Docs: README.md + both translations, docs/playbooks/README ×3, examples/prompts.md, docs/IMPROVEMENT-IDEAS.md.

Rather than only deleting it, each of the three workflow files now states the non-goal explicitly, and lint-workflows.mjs forbidTexts adapty_installs, absolute_gap, relative_gap and different attribution and event definitions so it cannot return quietly.

2. Give the weekly review a real metric set

weekly-review.md requested exactly four things: spend, total_installs, adapty_installs and one selected success metric. Impressions, taps, avg CPT, CPI, cost per trial and cost per paid were all available in the same call and none were requested.

Both reporting workflows now report one fixed set, in order:

Row Metric
Spend spend
Impressions impressions
Taps taps
Avg CPT avg_cpt
Installs total_installs
CPI total_avg_cpi
Cost per trial cost_per_trial
Cost per paid cost_per_paid

The whole set rides the same single metrics overview call the playbooks already made, so the weekly review's four-call budget and the audit's three-call budget are unchanged. The same set goes on the campaign-outlier call and into account-health.md, so /asa-audit and /asa-review now report performance identically. The linter requires all eight names in both playbooks and in asa-review.md.

Two decisions worth reviewing:

  • CPI is Apple's total_avg_cpi field, read rather than computed. It counts redownloads. The apple-ads-benchmarks skill publishes spend-per-download under the name CPA and explicitly forbids aliasing that to CPI, so both playbooks now warn against placing the two side by side without checking the denominator.
  • cost_per_trial is gated on a new required input — "does the app offer a free trial?", asked before the success-metric menu. When the answer is no, the row is dropped and the metric is not offered. The linter requires that question string in both playbooks.

Not changed

skills/apple-ads/references/asa-metrics.md is GENERATED and synced from adaptyteam/adapty-cli. total_installs and adapty_installs stay in the metric vocabulary — they remain valid CLI metrics. Only the workflow built on their difference is gone.

Verification

  • lint-playbooks.mjs, lint-workflows.mjs, test-cowork-preflight.mjs, lint-i18n.mjs and both manifest JSON checks pass locally — the full lint.yml job.
  • A repo-wide grep for adapty install, adapty_installs, absolute_gap, relative_gap, different attribution and event definitions, install comparison and install gap leaves only the intentional non-goal prose, the linter's own guard strings, and the generated metric vocabulary.
  • Each new linter contract was confirmed to actually fail: unbacktick a metric name → missing-contract error; reintroduce adapty_installs → forbidden-behavior error; reword the free-trial gate → missing-contract error.

Not verified: no end-to-end /asa-review or /asa-audit run against a live account — that needs the adapty CLI and a connected org, so the rendered output shape is unproven in practice.

🤖 Generated with Claude Code

The install comparison was noise that read like a finding. It put two
numbers from different attribution models and different event definitions
side by side, then disclaimed that the difference meant nothing and that
nobody would investigate it — while occupying a headline section in both
the weekly check-in and the account-health audit.

Remove it from every skill, playbook, command, doc and translation. Each
of the three workflow files now carries an explicit non-goal sentence
instead, and lint-workflows forbids `adapty_installs`, `absolute_gap`,
`relative_gap` and the attribution disclaimer so it cannot creep back.

In its place, both reporting workflows report one fixed metric set:
spend, impressions, taps, avg_cpt, total_installs, total_avg_cpi,
cost_per_trial and cost_per_paid. The weekly review previously requested
only four metrics; the rest ride the same single `metrics overview` call,
so the four-call and three-call budgets are unchanged. The linter requires
all eight names in both playbooks and in the /asa-review command.

CPI is Apple's own `total_avg_cpi` field, read rather than computed. It
counts redownloads, and the benchmarks skill publishes spend per download
under the name CPA and forbids aliasing the two — both playbooks now warn
against comparing them without checking the denominator.

cost_per_trial is gated on a new required input: whether the app offers a
free trial. When it does not, the row is dropped and the metric is not
offered as a success metric.

`asa-metrics.md` is untouched — it is generated, and `adapty_installs`
remains a valid CLI metric. Only the workflow built on its difference from
Apple's count is gone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants