ContextOS(MCP Server) – Model Context Protocol Orchestrator

Trust‑driven name that promises clear, actionable insights

Overview

The MCP Server is a production-grade Model Context Protocol implementation designed to work with multiple local LLMs (including vLLM) for flexible and efficient AI inference.

It acts as a gateway between clients and multiple AI models, orchestrating:

• Prompt routing based on capability, performance, and load
• Structured output handling for predictable results
• Streaming token delivery for low-latency responses

---

Features

• Multi-LLM Routing – Dynamically select the optimal model for each request.
• Real-Time Streaming – Push generated tokens to clients over WebSocket.
• Production-Ready – Containerized, observable, and horizontally scalable.

---

Architecture

%%{
  init: {
    'theme': 'base',
    'themeVariables': {
      'primaryColor': '#BB2528',
      'primaryTextColor': '#fff',
      'primaryBorderColor': '#7C0000',
      'lineColor': '#F8B229',
      'secondaryColor': '#006100',
      'tertiaryColor': '#fff',
      'lineWidth': 8,
      'fontSize':30
    }
  }
}%%
flowchart LR
    Client[React Client] --> API[HTTP/WebSocket API Gateway]
    API --> OR[Core Orchestrator]
    OR --> LR[LLM Router]
    LR --> L1[vLLM]
    LR --> L2[Mistral]
    LR --> L3[Qwen]
    OR --> Cache[Redis]
    OR --> Storage[PostgreSQL]

Loading

---

Quick Start

Prerequisites

• Go 1.22+
• Docker & Docker Compose
• Redis and PostgreSQL
• Local LLM server(s) (e.g., vLLM, Mistral)

Installation

git clone https://github.com/echenim/MarketClarityOS.git server
cd server
go mod tidy

Local Development

docker-compose up -d
go run cmd/server/main.go

---

Configuration

Edit configs/config.local.yaml to set:

• LLM endpoints
• Redis/PostgreSQL connection details

---

License

MIT License – see LICENSE for details.