💡 Idea: Query Result Cache for BLite.Server

[mrdevrobot]

💡 Idea: Query Result Cache for BLite.Server

Category: Performance / BLite.Server

---

Problem

Every query to BLite.Server — whether via REST (POST /query, GET /documents) or gRPC (Query) — hits the storage engine unconditionally, even when the same query is repeated many times against data that hasn't changed. In read-heavy scenarios (dashboards, reporting, repeated API calls) this wastes CPU and I/O for no benefit.

---

Proposed solution

An opt-in, collection-scoped, write-invalidated query cache backed by IMemoryCache.

Key design points:

• Deterministic cache keys — include dbId (cross-tenant isolation), physical collection name, and a SHA-256 hash of the query payload.
• Write-triggered invalidation — any Insert, Update, Delete, or BulkWrite on a collection immediately evicts all cached results for that collection via a CancellationChangeToken. Transaction commit also triggers invalidation.
• Configurable per deployment — enabled/disabled globally, with configurable sliding expiration, absolute expiration, max cache size (bytes), and a max result-set size threshold (skip caching if result > N documents).
• Zero overhead when disabled — the cache path is guarded by a single boolean check.
• Excluded from caching: FindById (already O(1) B-Tree), queries inside an active transaction, vector search endpoints.

Configuration sketch (appsettings.json):

"QueryCache": {
  "Enabled": true,
  "SlidingExpirationSeconds": 30,
  "AbsoluteExpirationSeconds": 300,
  "MaxSizeBytes": 67108864,
  "MaxResultSetSize": 500
}

Affected surfaces: REST BLQL endpoints, REST GET /documents, gRPC DynamicService.Query, gRPC DocumentService.Query.

---

Why it matters

• Reduces engine load drastically for read-heavy workloads
• No application-side caching required — the server handles it transparently
• Safe by design: invalidation is tied to the write path, not a TTL guess

---

Open questions

• Should cache stats be exposed via a /metrics or admin endpoint?
• Should per-collection opt-in/opt-out be supported (e.g., "QueryCache": { "ExcludeCollections": ["logs"] })?
• Cache stampede protection (e.g., single-flight pattern) for high-concurrency scenarios?