🛢️ Oil Market Monitoring Agent (Oil-Agent)

An advanced autonomous AI system designed to track geopolitical and industrial events impacting global oil prices (Brent/WTI). This agent leverages a hybrid architecture combining smolagents for intelligence gathering and DSPy for structured synthesis and continuous prompt optimization.

---

🛡️ Current Status: Stable Version (v1.0-stable)

• Validated Complete Cycle: Research (smolagents) + Synthesis (DSPy) + Verification.
• Anti-Hallucination Filtering: Automatic event validation against raw sources (verify_event_truthfulness).
• Context Management: Automatic history compression to maintain performance.
• Reliability: 100% of research tools systematically executed before any synthesis.

---

🚀 Key Features

• Autonomous Intelligence Gathering: Uses CodeAgent with a suite of specialized tools to browse web, search news, and parse RSS feeds.
• Structured Synthesis (DSPy): Processes raw intelligence into high-quality, validated JSON events using a modular DSPy pipeline.
• Continuous Learning: Automatically saves successful traces to a local dataset for offline optimization and few-shot boosting.
• Real-Time Monitoring: Integrated with RSS feeds, VIX volatility index, and daily news filters for maximum reactivity.
• Robust Alerting: Sends rich HTML/Text alerts via SMTP (Postfix) when high-impact events are detected (Impact Score ≥ 6).
• Failure Resilience: Implements robust JSON parsing with auto-repair and backup mechanisms for data persistence.
• Automatic LLM Server Management: Automatically starts and stops llama-server when the agent runs, ensuring optimal resource usage.

---

🏗️ Architecture

The system operates in a two-stage pipeline:

1. Phase 1: Research (smolagents)
   • The CodeAgent executes Python logic to orchestrate multiple tools.
   • It gathers "raw intelligence" from specialized sources (Iran tensions, refinery damage, OPEC decisions, etc.).
2. Phase 2: Synthesis (DSPy)
   • A dspy.Module (OilEventAnalyzer) takes raw text and current market context.
   • It performs a Chain of Thought (CoT) reasoning to extract structured events.
   • Validates output using Pydantic models to ensure strict data integrity.

Toolset

Category	Description
Geopolitics	IranConflictTool, GeopoliticalEscalationTool
Supply Chain	RefineryDamageTool, ShippingDisruptionTool, OPECSupplyTool
Market Data	OilPriceTool, VIXTool (Market Fear Index)
News	RecentNewsTool, RSSFeedTool, DuckDuckGoSearchTool

---

🛠️ Installation

1. Prerequisites

• llama-server (llama.cpp): Install llama-server and download required model:

  # Download llama-server for Windows
  # From: https://github.com/ggerganov/llama.cpp/releases
  llama-server.exe

  # Download Qwen3.5-9B model (GGUF format)
  # From: https://huggingface.co/Qwen/Qwen2.5-7B-Instruct-GGUF
  # Place in: C:\Modeles_LLM\Qwen3.5-9B-Q4_K_S.gguf

• uv: The project uses uv for lightning-fast dependency management.

2. Setup

# Clone repository
git clone https://github.com/youruser/oil-agent.git
cd oil_agent

# Sync dependencies (creates .venv automatically)
uv sync

# Activate environment
# Windows:
.venv\Scripts\activate
# Linux/macOS:
source .venv/bin/activate

---

⚙️ Configuration

The project uses config.json for all configuration. Edit this file to customize behavior:

{
  "model": {
    "name": "qwen3.5-9b",
    "path": "C:\\Modeles_LLM\\Qwen3.5-9B-Q4_K_S.gguf",
    "api_base": "http://127.0.0.1:8080",
    "num_ctx": 65536,
    "provider": "llama.cpp",
    "dspy_timeout": 300
  },
  "llama_server": {
    "executable": "llama-server.exe",
    "n_gpu_layers": -1,
    "n_threads": 0,
    "ctx_size": 65536,
    "batch_size": 512,
    "ubatch_size": 128,
    "cache_type_k": "f16",
    "cache_type_v": "f16",
    "host": "0.0.0.0",
    "port": 8080
  },
  "email": {
    "smtp_host": "localhost",
    "smtp_port": 25,
    "email_from": "oil-monitor@localhost",
    "email_to": "your-email@example.com",
    "email_subject_prefix": "[OIL-ALERT]",
    "send_emails": false
  },
  "persistence": {
    "history_file": "logs/email_history.json",
    "events_db": "logs/events_seen.json",
    "dataset_file": "data/oil_intelligence_dataset.jsonl"
  },
  "monitoring": {
    "alert_threshold": 6,
    "news_sources": [
      "reuters.com",
      "bloomberg.com",
      "apnews.com",
      "bbc.com",
      "ft.com",
      "wsj.com"
    ],
    "timezone": "Europe/Paris",
    "recent_news_hours": 24
  }
}

Important Configuration Options:

• model.num_ctx: Context window size (default: 65536 tokens). Increase if you encounter context overflow errors.
• model.dspy_timeout: DSPy processing timeout in seconds (default: 600). Adjust based on your data volume:
  • Development/Test: 120 seconds (2 minutes)
  • Production normal: 600 seconds (10 minutes)
  • Large volumes: 900 seconds (15 minutes)
• model.max_intelligence_chars: Maximum intelligence size for DSPy processing in characters (default: 20000). Adjust based on your model's capabilities:
  • Development/Test: 20000 characters (~5000 tokens) - default recommended
  • Production normal: 30000 characters (~7500 tokens) - for detailed analysis
  • Large volumes: 50000 characters (~12500 tokens) - for complex analysis
• llama_server.n_gpu_layers: Number of layers to offload to GPU (-1 = all layers, 0 = CPU only).
• email.send_emails: Set to false for testing, true for production.
• monitoring.alert_threshold: Minimum impact score (0-10) to trigger email alerts.

---

📈 DSPy Optimization (Learning)

One of project's core strengths is its ability to learn from experience.

1. Dataset Collection

Every time the agent runs, it saves a (raw_intelligence, structured_output) pair into data/oil_intelligence_dataset.jsonl.

2. Running Optimization

Once you have collected at least 5-10 examples, run the optimizer to improve synthesis logic:

uv run python optimize_agent.py

This script uses BootstrapFewShot teleprompter to:

• Evaluate candidate "demos" from your dataset.
• Select the most effective examples based on a custom metric (structure validation + content relevance).
• Save optimized weights to data/oil_analyzer_optimized.json.

The next time oil_agent.py runs, it will automatically load these optimized weights to provide superior analysis.

DSPy Timeout Monitoring

The agent now includes automatic monitoring of DSPy processing time to prevent timeout issues:

• Automatic Time Tracking: Logs start and end times for DSPy processing
• Timeout Configuration: Configurable via model.dspy_timeout in config.json (default: 600 seconds)
  • Important: The timeout is passed to LiteLLM via litellm_params={"timeout": ...} in configure_dspy()
• Proximity Warning: Issues a warning when processing time exceeds 80% of configured timeout
• Example Logs:

  🔄 Début traitement DSPy (intelligence: 35978 caractères)
  ✅ Traitement DSPy terminé en 45.23s
  

This monitoring helps identify when timeout needs adjustment for your specific workload.

Intelligence Reduction for DSPy

To improve DSPy performance with large intelligence volumes, agent now includes automatic intelligence reduction:

• Automatic Reduction: Reduces raw intelligence before DSPy processing
• Configurable Limit: Set model.max_intelligence_chars in config.json (default: 20000)
• Reduction Strategy:
  • Keeps essential headers and summaries
  • Removes duplicates and empty lines
  • Limits total size to configured maximum
• Logging: Logs reduction ratio (original → reduced size)
• Example Logs:

  📦 Intelligence réduite pour DSPy : 36170 → 18945 caractères
  

This helps prevent timeout errors by providing DSPy with a manageable input size.

---

🖥️ Usage

Run a Monitoring Cycle

uv run python oil_agent.py

Automatic LLM Server Management: The agent automatically starts llama-server when needed and stops it when finished. No manual intervention required.

View Alert History

uv run python oil_agent.py history

Test LLM Server Connection

python test_llama_server.py

This script tests:

• llama-server connectivity (health endpoint)
• DSPy integration
• smolagents integration
• Automatic server start/stop

---

📝 License

MIT License. See LICENSE for details.