10 02 Dual Source Provider Pattern

Dual-Source Provider Pattern

Overview

The Dual-Source Provider Pattern provides a unified lookup mechanism that combines:

1. Configured entities - Loaded via IOptionsMonitor from database or configuration files
2. Materialized entities - Runtime-registered instances created during application execution

This pattern enables hot-reload of configuration while supporting runtime additions, with thread-safe access and ID-based lookups.

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                        {Domain}Provider                              │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  ┌──────────────────────┐    ┌──────────────────────┐              │
│  │   MATERIALIZED       │    │     CONFIGURED       │              │
│  │                      │    │                      │              │
│  │ ConcurrentDictionary │    │   IOptionsMonitor    │              │
│  │   <string, T>        │    │   <List<TConfig>>    │              │
│  │                      │    │         │            │              │
│  │ • Runtime registered │    │         ▼            │              │
│  │ • Mutable            │    │  ┌──────────────┐    │              │
│  │ • Checked first      │    │  │ {Domain}Index│    │              │
│  │                      │    │  │  (immutable) │    │              │
│  └──────────────────────┘    │  │              │    │              │
│                              │  │ • ById dict  │    │              │
│                              │  │ • ByName dict│    │              │
│                              │  └──────────────┘    │              │
│                              │         ▲            │              │
│                              │         │            │              │
│                              │    OnChange()        │              │
│                              │    rebuilds index    │              │
│                              └──────────────────────┘              │
│                                                                      │
│  Lookup Order: Materialized → Configured                            │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

Key Components

1. Index Structure

The index uses an immutable record for thread-safe reads:

private sealed record {Domain}Index(
    IReadOnlyDictionary<Guid, TConfig> ItemsById,
    IReadOnlyDictionary<string, TConfig> ItemsByName);

2. Thread Safety

• ConcurrentDictionary for runtime registrations
• ReaderWriterLockSlim for index access during rebuilds
• Immutable index records allow lock-free reads most of the time

3. Hot Reload

Configuration changes trigger automatic index rebuilding:

_options.OnChange(_ => RebuildConfiguredIndex());

Lookup Priority

1. Materialized (runtime-registered) - Checked first
2. Configured (IOptionsMonitor) - Checked second

This priority allows runtime overrides of configured entities.

ID-Based vs Name-Based Lookups

Lookup Type	Use Case	Example
By ID (Guid)	FK references, durable identity	GetById(Guid id)
By Name (string)	User-facing, configuration binding	GetByName(string name)

Both are supported for backward compatibility and different use cases.

Implementation Checklist

When implementing a new provider with this pattern:

• Add IOptionsMonitor<List<TConfig>> field
• Add ConcurrentDictionary<string, T> for materialized entities
• Add ReaderWriterLockSlim for index access
• Create immutable {Domain}Index record
• Implement backward-compatible constructor (without IOptionsMonitor)
• Implement full constructor with IOptionsMonitor
• Subscribe to OnChange in constructor
• Implement BuildConfiguredIndex() method
• Implement RebuildConfiguredIndex() method with write lock
• Implement ID-based lookup methods
• Maintain existing name-based lookups
• Add MessageLogging methods for operations
• Add Register/Unregister methods for runtime additions

Reference Implementations

Provider	Project	EventId Range
DataStoreProvider	FractalDataWorks.Services.Data	5090-5095
DataSetProvider	FractalDataWorks.Services.Data	5120-5127
WorkflowProvider	FractalDataWorks.Services.Workflows	7850-7858
PipelineConfigurationProvider	FractalDataWorks.Etl.Pipelines	8170-8177
ChainDefinitionProvider	FractalDataWorks.Data.Transformations	N/A (existing)

DI Registration

// Phase 1a: Configure
services.Configure<List<{Domain}Configuration>>(
    configuration.GetSection("{Domain}s"));

// Phase 1b: Register
services.AddSingleton<I{Domain}Provider, {Domain}Provider>();

The provider resolves its IOptionsMonitor via constructor injection.

Skill Reference

See .claude/skills/dual-source-provider/SKILL.md for implementation templates and detailed guidance.

Related Patterns

• ServiceTypeCollection Pattern - For plugin architectures
• ManagedConfiguration - For database-backed configuration
• Three-Phase Registration - For service domain setup