Skip to content

Fix quality_avg to average final_score instead of computing completion rate - #2

Closed
rafaelob with Copilot wants to merge 2 commits into
improvementsfrom
copilot/sub-pr-1
Closed

Fix quality_avg to average final_score instead of computing completion rate#2
rafaelob with Copilot wants to merge 2 commits into
improvementsfrom
copilot/sub-pr-1

Conversation

Copilot AI commented Feb 19, 2026

Copy link
Copy Markdown

quality_avg was reporting completion rate (completed traces / total traces × 100) rather than the average quality score from final_score values on each trace.

Changes

  • observability.py: Replace the completion-rate formula with a proper average of final_score across completed traces that have a non-None score
# Before — measures completion rate, not quality
"quality_avg": round(
    sum(1 for t in recent_traces if t.status == "completed")
    / max(len(recent_traces), 1)
    * 100,
    1,
),

# After — true average of final_score values
completed_quality_scores = [
    t.final_score
    for t in recent_traces
    if t.status == "completed" and t.final_score is not None
]
quality_avg_value = (
    (sum(completed_quality_scores) / max(len(completed_quality_scores), 1)) * 100
)
"quality_avg": round(quality_avg_value, 1),

Falls back to 0 when no qualifying scores exist via max(len(...), 1).


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

…on rate

Co-authored-by: rafaelob <814981+rafaelob@users.noreply.github.com>
Copilot AI changed the title [WIP] WIP Address feedback on RBAC, Sign Languages, Skills, and Braille implementation Fix quality_avg to average final_score instead of computing completion rate Feb 19, 2026
Copilot AI requested a review from rafaelob February 19, 2026 20:28
@rafaelob rafaelob closed this Feb 19, 2026
@rafaelob
rafaelob deleted the copilot/sub-pr-1 branch February 19, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants