Skip to content

Fix quality_avg to compute mean final_score instead of completion rate - #3

Closed
rafaelob with Copilot wants to merge 2 commits into
improvementsfrom
copilot/sub-pr-1-again
Closed

Fix quality_avg to compute mean final_score instead of completion rate#3
rafaelob with Copilot wants to merge 2 commits into
improvementsfrom
copilot/sub-pr-1-again

Conversation

Copilot AI commented Feb 19, 2026

Copy link
Copy Markdown

quality_avg was computing completion rate (completed_count / total * 100) rather than the average of actual quality scores, making it a misleading metric.

Change

  • Extract final_score values from completed traces where final_score is not None
  • Compute arithmetic mean of those scores as quality_avg
# Before — completion rate, not quality
sum(1 for t in recent_traces if t.status == "completed") / max(len(recent_traces), 1) * 100

# After — true average quality score
completed_quality_scores = [
    t.final_score
    for t in recent_traces
    if t.status == "completed" and t.final_score is not None
]
quality_avg_value = sum(completed_quality_scores) / max(len(completed_quality_scores), 1)

final_score is already on a 0–100 scale, so no additional scaling is applied. Falls back to 0.0 when no scored traces exist.


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

… rate

Co-authored-by: rafaelob <814981+rafaelob@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix quality_avg calculation in Sprints 24-26 PR Fix quality_avg to compute mean final_score instead of completion rate Feb 19, 2026
Copilot AI requested a review from rafaelob February 19, 2026 20:41
@rafaelob rafaelob closed this Feb 19, 2026
@rafaelob
rafaelob deleted the copilot/sub-pr-1-again branch February 19, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants