Add comprehensive telemetry to all content providers

[Copilot]

Overview

This PR implements comprehensive logging and metrics reporting across all content providers (Bluesky, Twitter, Blazot, YouTube, YouTubeChat, AzureQueue, Mastodon, and TwitchChat) to improve observability and debugging capabilities.

Changes Made

Enhanced ProviderInstrumentation Class

Added new telemetry capabilities to TagzApp.Common.Telemetry.ProviderInstrumentation:

• ConnectionStatusChangesCounter: Tracks provider status transitions with provider and status tags
• ConnectionStatusGauge: Observable gauge showing real-time provider health (0=Disabled, 1=Unhealthy, 2=Degraded, 3=Healthy)
• Thread-safe status tracking with dictionary-based state management

Provider Updates

Updated all 8 content providers to include:

Logging:

• Connection lifecycle events (startup, shutdown, configuration changes)
• New message discovery with message counts
• Error conditions with detailed context
• Consistent log prefixes for filtering (e.g., "Bluesky:", "Twitter:", "TwitchChat:")

Metrics:

• Message received counts tagged by provider and author
• Connection status change events
• Real-time health status monitoring

Example logging output:

Bluesky: Starting connection to AT Protocol
Bluesky: Retrieved 5 new messages
Twitter: Retrieved 12 new tweets
TwitchChat: Client is not connected - check credentials

OpenTelemetry Integration

• Added tagzapp-provider-metrics meter to TagzApp.ServiceDefaults configuration
• Metrics automatically collected and exposed via OpenTelemetry exporters
• Compatible with existing tracing infrastructure

Documentation Updates

Added comprehensive telemetry requirements to guide future provider implementations:

• .github/copilot-instructions.md: Added "Telemetry Requirements" section with:

  • Required logging patterns with code examples
  • Required metrics with usage samples
  • OpenTelemetry integration details
  • Logging conventions and standards

• doc/Provider-Configuration-Pattern.md: Added detailed "Telemetry Requirements" section with:

  • Required dependencies (ILogger, ProviderInstrumentation)
  • Complete logging examples for all scenarios
  • Metrics documentation with code samples
  • References to existing providers as implementation examples
  • Updated migration checklist to include telemetry implementation

Benefits

1. Real-time Monitoring: Operators can track provider health and message throughput
2. Faster Debugging: Detailed logging helps identify connection issues quickly
3. Performance Insights: Metrics enable tracking of message volumes and provider performance
4. Consistency: All providers now follow the same telemetry pattern
5. Standard Guidelines: Future providers have clear telemetry implementation requirements

Testing

• ✅ All builds succeed
• ✅ Unit tests pass (30 passed, 2 expected failures due to external API dependencies, 2 skipped)
• ✅ Code formatted per project standards
• ✅ No breaking changes to existing functionality

Example Metrics

Operators can now query metrics like:

• messages-received{provider="twitter", author="@user"} - Message counts per provider/author
• connection-status-changes{provider="bluesky", status="Healthy"} - Status transition events
• connection-status{provider="mastodon"} - Current health status

Fixes #522

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Update existing content providers with comprehensive telemetry</issue_title>
<issue_description>All content providers need to be updated so that they provide comprehensive logging and metrics reporting. I'd like to see logging for new messages discovered, connection status changes. I'd like to see metrics reported for messages received as well</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes #522

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[Copilot]

Pull Request Overview

This PR implements comprehensive telemetry (logging and metrics) across all content providers to improve observability and debugging capabilities. The changes add structured logging for connection lifecycle events and message discovery, plus metrics tracking for message counts and provider health status.

Key changes:

• Enhanced ProviderInstrumentation with connection status tracking and gauges
• Added consistent telemetry to all 8 content providers (Bluesky, Twitter, Blazot, YouTube, YouTubeChat, AzureQueue, Mastodon, TwitchChat)
• Integrated new telemetry meter into OpenTelemetry configuration

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/TagzApp.ServiceDefaults/Extensions.cs	Registers the new telemetry meter with OpenTelemetry
src/TagzApp.Common/Telemetry/ProviderInstrumentation.cs	Adds connection status tracking capabilities with counters and gauges
src/TagzApp.Providers.Youtube/YoutubeProvider.cs	Implements telemetry logging and metrics for YouTube provider
src/TagzApp.Providers.YouTubeChat/YouTubeChatProvider.cs	Implements telemetry logging and metrics for YouTube Chat provider
src/TagzApp.Providers.Twitter/TwitterProvider.cs	Implements telemetry logging and metrics for Twitter provider
src/TagzApp.Providers.TwitchChat/TwitchChatProvider.cs	Implements telemetry logging and metrics for Twitch Chat provider
src/TagzApp.Providers.Mastodon/MastodonProvider.cs	Implements telemetry logging and metrics for Mastodon provider
src/TagzApp.Providers.Bluesky/BlueskyProvider.cs	Implements telemetry logging and metrics for Bluesky provider
src/TagzApp.Providers.Blazot/BlazotProvider.cs	Implements telemetry logging and metrics for Blazot provider
src/TagzApp.Providers.AzureQueue/AzureQueueProvider.cs	Implements telemetry logging and metrics for Azure Queue provider

[csharpfritz]

@copilot Let's also enhance our copilot instructions files to indicate that new content providers should also provide telemetry using this technique so that we have standard metrics

[Copilot]

@copilot Let's also enhance our copilot instructions files to indicate that new content providers should also provide telemetry using this technique so that we have standard metrics

Done! I've updated both documentation files to include comprehensive telemetry requirements:

.github/copilot-instructions.md - Added "Telemetry Requirements" section with:

• Required logging patterns with examples
• Required metrics with code samples
• Available metrics from ProviderInstrumentation
• OpenTelemetry integration details

doc/Provider-Configuration-Pattern.md - Added detailed "Telemetry Requirements" section with:

• Required dependencies (ILogger, ProviderInstrumentation)
• Logging conventions and examples for all scenarios
• Complete metrics documentation
• Reference to existing providers as examples

All future providers will now have clear guidance on implementing the standard telemetry pattern. Commit: c37b34a (documentation updates in subsequent commit)