fix: increase redis timeout and bull mq lock duration for long video …

[IliasHad]

Summary

• Background jobs failing with READONLY You can't write against a read only replica errors
• Jobs being marked as stalled when processing videos longer than 1-2 hours

Changes

Redis Configuration:

• Increased commandTimeout from 60s to 120s for long-running operations
• Added reconnection logic for READONLY, OOM, and LOADING errors

Worker Lock Settings:

• Increased lockDuration to 6 hours (transcription), (frame analysis), (audio and visual embedding)
• Set lockRenewTime to 30s to prevent false stall detection
• Adjusted stalledInterval to 2 minutes for better crash detection

Job Cleanup:

• Reduced completed job retention from 24h to 30min
• Limited completed job count from 100 to 5
• Limited failed job count to 20

Code Improvements:

• Moved job chaining inside worker functions for atomic operations (transcription worker)

Impact

• Supports video processing jobs up to 6+ hours without timeouts
• Reduces Redis memory pressure through aggressive cleanup
• Eliminates false stall detection while maintaining crash recovery

More details here

Credits: #8