Skip to content

[agentic-token-optimizer] Daily Agentic Workflow AIC Usage Audit β€” Optimization RecommendationsΒ #234

Description

@github-actions

🎯 Target Workflow

Daily Agentic Workflow AIC Usage Audit (agentic-token-audit) β€” selected as highest-AIC eligible workflow. All workflows were optimized within 14 days; this one has the longest elapsed time since review (2026-06-23, 10 days ago) and the second-highest 7-day AIC spend.

πŸ“Š Spend Profile (2026-06-27 β†’ 2026-07-03, 5 runs)

Metric Value
Total AIC 1,052.33
Avg AIC / run 210.47
Total tokens (1 instrumented run) 653,717
Avg turns / run (instrumented) ~16
Total action minutes 43 min
Error rate 0 / 5 runs

Token coverage is limited: 4 of 5 runs reported null token usage. AIC figures are complete; per-token analysis is based on run Β§28662299837.

Observability insight (medium severity β€” execution drift): the workflow varied from 0 to 17 turns across runs, flagged as "changing task shape or unstable prompts."


πŸ† Ranked Recommendations

1. Trim RunData Schema Table Β· 5–8 AIC/run

Phase 1 documents 14 RunData fields; the processing script uses only 6 (workflow_name, aic, token_usage, turns, error_count, warning_count, url). The 8 unused fields (effective_tokens, duration, branch, display_title, head_sha, logs_path, classification, event) add ~180 prompt tokens per run.

Action: Replace the 14-field table with a 6-field version. Remove the deprecation note on effective_tokens.


2. Merge Phases 1 + 2 Into One Execution Block Β· 10–15 AIC/run

Phase 2 (persist snapshot + update rolling-summary) contains only 2 file operations that are trivially appended to the Phase 1 Python script. The current structure forces the agent to:

  1. Write + run process_audit.py
  2. Read Phase 2 instructions
  3. Copy audit_snapshot.json and update rolling-summary.json (separate turns)

Action: Extend the Phase 1 script spec to also write YYYY-MM-DD.json to repo-memory and update rolling-summary.json. Remove Phase 2 as a separate section.

Evidence: ~2 turns saved per run; at 16 turns/run that is ~12% reduction.


3. Batch Both Charts Into One Python Script Β· 8–12 AIC/run

Phase 3 describes two charts sequentially, causing the agent to write and run two separate Python scripts and make two upload_asset calls across ~4 turns before it can build the issue.

Action: Replace with a single instruction to write gen_charts.py that creates both PNGs in one execution, then make both upload_asset calls before Phase 4.

Evidence: Current sequential description and repeated PYTHONPATH reminder suggest 2 extra scripting turns per run.


4. Consolidate PYTHONPATH Instructions Β· 3–5 AIC/run

The PYTHONPATH prefix is mentioned 3 times: once as a bullet in "Chart requirements," once as a verbatim example command, and implicitly required in Phase 1. This causes agents to re-read and re-derive the same path across turns.

Action: Add one setup note at the top of Phase 1: export PYTHONPATH=/tmp/gh-aw/token-audit/site-packages${PYTHONPATH:+:$PYTHONPATH}. Remove the repeated mentions from Phase 3.


5. Compress Phase 4 Report Template Β· 5–8 AIC/run

The Phase 4 template is ~50 lines of verbatim markdown with explicit placeholder strings loaded into context on every Phase 4 turn.

Action: Replace with a compact requirements list (5 bullet points) naming each required section and its key fields. Remove explicit placeholder names β€” instruct the agent to embed chart URLs inline as they are returned.

Evidence: Current template is ~600 prompt tokens. A requirements list reduces this to ~150 tokens.


πŸ”§ Structural Optimization β€” Sub-Agent Candidates

The workflow has no existing ## agent: blocks and 4 major prompt sections.

Phase 1 β€” Python Data Processing Β· Score 9/10 (Strong)

Dimension Score Rationale
Independence 3 Only needs pre-downloaded workflow-logs.json
Small-model adequacy 2 Extractive Python aggregation with defined output schema
Parallelism 2 First phase, starts immediately
Size 2 30+ line script spec

Why a smaller model fits: group-by, aggregate, sort, write JSON β€” fully specified in/out. A Haiku-class model handles this class of task reliably.

Proposed invocation (add to prompt, replacing Phase 1 prose):

## agent: process-logs
model: haiku
task: |
  Read /tmp/gh-aw/token-audit/workflow-logs.json. Filter status=="completed".
  Group by workflow_name; compute run_count, total_ai_credits, avg_ai_credits,
  total_tokens, avg_tokens, total_turns, avg_turns, total_action_minutes,
  error_count, warning_count. Sort desc by total_ai_credits.
  Write audit_snapshot.json (schema: {date, period_days, overall, workflows}).
  Copy to /tmp/gh-aw/repo-memory/default/YYYY-MM-DD.json.
  Update rolling-summary.json (last 90 entries).
  Treat null aic/token_usage as 0.

Estimated additional savings: 15–20 AIC/run (smaller model for merged Phases 1+2).

Phase 3 β€” Chart Generation Β· Score 7/10 (Moderate)

Dimension Score
Independence 1 (needs Phase 1 output)
Small-model adequacy 3 (mechanical matplotlib)
Parallelism 1
Size 2

Recommend implementing Recommendation 3 (batched script) first; revisit sub-agent extraction if turns remain high.


πŸ’‘ Summary of Expected Savings

Recommendation Est. AIC/run saved
1. Trim schema table 5–8
2. Merge Phases 1+2 10–15
3. Batch charts 8–12
4. Consolidate PYTHONPATH 3–5
5. Compress template 5–8
Total (without sub-agent) 31–48
+ Phase 1 sub-agent +15–20
Total (with sub-agent) 46–68

Conservative estimate: ~40 AIC/run saved (19% reduction from 210 avg AIC/run).


⚠️ Caveats

  • Turn data covers only 1 of 5 runs; savings estimates assume it is representative.
  • "Execution drift" insight (0–17 turns) likely reflects metadata gaps in 4 runs rather than genuine zero-turn executions.
  • Sub-agent invocation requires ## agent: block support in the runtime.
Run-level evidence
Run Date AIC Turns Tokens API Calls
Β§28376941750 2026-06-29 188.90 β€” β€” 10
Β§28446558922 2026-06-30 229.14 β€” β€” 10
Β§28520469613 2026-07-01 198.38 β€” β€” 10
Β§28592118147 2026-07-02 212.66 β€” β€” 10
Β§28662299837 2026-07-03 223.25 17 653,717 10

References: Β§28662299837 Β· Β§28592118147 Β· Β§28376941750

Generated by Agentic Workflow AIC Usage Optimizer Β· 282.8 AIC Β· ⊞ 21.6K Β· β—·

  • expires on Jul 10, 2026, 3:16 PM UTC

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions