feat(router): add LLM routing with cost optimization and pretrained configs

[bsbodden]

Extends SemanticRouter with LLM model selection, cost-optimized routing, and pretrained configurations — routing queries to the right model using Redis vector search.

• "hello, how are you?" → GPT-4.1 Nano ($0.10/M tokens)
• "explain garbage collection" → Claude Sonnet 4.5 ($3/M tokens)
• "architect a distributed system" → Claude Opus 4.5 ($5/M tokens)

Design

LLM routing is integrated directly into SemanticRouter. When a Route includes an optional model field, the router returns the LiteLLM-compatible model identifier alongside the match, with a confidence score derived from vector distance (1 - distance/2).

Schema extensions (all optional, no breaking changes):

• Route gains model: Optional[str] and metadata: Dict
• RouteMatch gains model, confidence, alternatives, and metadata fields
• RoutingConfig gains cost_optimization, cost_weight, and default_route
• Callable pattern: router(query) returns a RouteMatch
• route_many() returns multiple ranked matches
• Full async parity via AsyncSemanticRouter

Routes without a model field work exactly as before — existing SemanticRouter usage is unaffected.

Usage

Basic LLM routing:

from redisvl.extensions.router import SemanticRouter, Route

routes = [
    Route(name="simple", model="openai/gpt-4.1-nano",
          references=["hello", "hi"], distance_threshold=0.5),
    Route(name="expert", model="anthropic/claude-opus-4-5",
          references=["architect a system", "design an algorithm"],
          distance_threshold=0.7),
]
router = SemanticRouter(name="llm-router", routes=routes,
                        redis_url="redis://localhost:6379")

match = router("hello there")
print(match.model)       # openai/gpt-4.1-nano
print(match.confidence)  # 0.81

Pretrained config — ships with a 3-route config (simple/standard/expert) mapped to Bloom's Taxonomy levels, with pre-computed sentence-transformers/all-mpnet-base-v2 embeddings:

router = SemanticRouter.from_pretrained("default", redis_url="redis://localhost:6379")

Cost-optimized routing — when multiple routes match with similar distances, a cost penalty biases toward cheaper models:

from redisvl.extensions.router.schema import RoutingConfig

router = SemanticRouter(
    name="cost-router", routes=routes,
    routing_config=RoutingConfig(cost_optimization=True, cost_weight=0.3),
    redis_url="redis://localhost:6379",
)

Async:

router = await AsyncSemanticRouter.create(
    name="async-router", routes=routes, redis_url="redis://localhost:6379")
match = await router("hello")

Export/import with embeddings:

router.export_with_embeddings("my_router.json")
loaded = SemanticRouter.from_pretrained("my_router.json", redis_url="redis://localhost:6379")

Files changed

Area	Files
Core router	redisvl/extensions/router/semantic.py, schema.py, __init__.py
Pretrained configs	redisvl/extensions/router/pretrained/__init__.py, default.json
Query support	redisvl/query/query.py, redisvl/utils/full_text_query_helper.py
Tests	tests/unit/test_llm_router_schema.py, tests/integration/conftest.py, tests/unit/conftest.py
Docs	docs/user_guide/13_llm_router.ipynb
Tooling	scripts/generate_pretrained_config.py

[Copilot]

Pull request overview

Copilot reviewed 16 out of 18 changed files in this pull request and generated 10 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[cursor]

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

[cursor]

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

[rbs333]

I think we still have some disconnects on the design of this feature. Let's maybe set up some time to talk through it.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

[jit-ci]

🛡️ Jit Security Scan Results

✅ No security findings were detected in this PR

---

^{Security scan by Jit}

[Copilot]

Pull request overview

Copilot reviewed 16 out of 18 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

[Copilot]

Pull request overview

Copilot reviewed 16 out of 18 changed files in this pull request and generated 4 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

[bsbodden]

@rbs333 Reworked the PR based on your feedback. Here's where things stand:

1. No separate class — SemanticRouter is the API

All LLM routing logic (cost optimization, confidence scoring, from_pretrained(), export_with_embeddings()) lives directly in SemanticRouter. No separate LLMRouter class.

2. "Tier" → "Route"

Route with an optional model field is the only concept. All code, tests, and the notebook use Route/route consistently.

3. Callable pattern

router(query) throughout — no router.route().

4. from_pretrained() on SemanticRouter

from redisvl.extensions.router import SemanticRouter

router = SemanticRouter.from_pretrained("default", redis_url="redis://localhost:6379")
match = router("hello")  # -> RouteMatch(name="simple", model="openai/gpt-4.1-nano", ...)

5. Dead code removed

• llm_router/router.py (1,528 lines) — deleted
• llm_router/schema.py (206 lines) — deleted
• llm_router/DESIGN.md — deleted

6. Notebook fully rewritten

Every cell uses SemanticRouter, Route, router(), match.name.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

[Copilot]

Pull request overview

Copilot reviewed 12 out of 14 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[rbs333]

Looking good! I think now it just needs an update on the docs and then a quick latency check to make sure we're not adding overhead vs previous router lookup speed.

[rbs333]

Looks good 👍

[rbs333]

are these changes for the router?