Skip to content

feat(router): submit Krusch Cascade Router to RouterArena leaderboard - #169

Open
kruschdev wants to merge 5 commits into
RouteWorks:mainfrom
kruschdev:submit/krusch-cascade-router
Open

feat(router): submit Krusch Cascade Router to RouterArena leaderboard#169
kruschdev wants to merge 5 commits into
RouteWorks:mainfrom
kruschdev:submit/krusch-cascade-router

Conversation

@kruschdev

@kruschdev kruschdev commented Jul 25, 2026

Copy link
Copy Markdown

Router Submission: Krusch Cascade Router

📌 Overview

Krusch Cascade Router is an open-source, framework-agnostic LLM router designed for high-efficiency agentic workflows. It eliminates the TTFT (Time-To-First-Token) latency penalty of extra router LLM calls by combining a sub-50ms predictive prompt classifier (evaluating prompt length, syntax/code blocks, mathematical density, and cognitive task keywords) with speculative logprob confidence thresholding.


📊 Benchmark Evaluation Results

Evaluated on RouterArena dataset (sub_10 split, 1,618 total entries):

  • 🚀 Acc-Cost Arena Score: 65.98
  • 📈 Average Accuracy: 65.23%
  • 💰 Cost per 1K Queries: $0.0675 ($0.0000675 / query)
  • ⚖️ Workload Balance: 50.0% Fast (gpt-4o-mini) / 50.0% Heavy (gemini-2.0-flash-001)

💡 Empirical Domain Routing Strategy

  • STEM & Complex Reasoning Escalation: Multi-step math problems (AIME, GSM8K, MATH), code generation (LiveCodeBench), MMLU-Pro reasoning (72.88%), and medical/scientific QA (MedMCQA, PubMedQA) are routed to gemini-2.0-flash-001.
  • Fast Edge Model Routing: Geography (GeoBench 83.64%), Social QA (SocialiQA 78.69%), and Multilingual Translation (WMT19) are directed to gpt-4o-mini for maximum accuracy and cost efficiency.

📁 Submitted Files

  • router_inference/config/krusch-cascade-router.json
  • router_inference/router/krusch_cascade_adapter.py
  • router_inference/predictions/krusch-cascade-router.json
  • Registered in leaderboard_manifest.yaml

@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant