FootFa - Football News Automation System

A comprehensive Python-based system that automatically scrapes football news from multiple sources, processes content with AI, and posts to Telegram channels.

Features

🚀 Multi-Source News Scraping

• Goal.com, BBC Sport, Marca, ESPN FC, Sky Sports, The Guardian, AS.com, L'Equipe
• Robust CSS selectors with fallback mechanisms
• Intelligent duplicate detection

🤖 AI-Powered Content Processing

• Google Gemini integration for content summarization
• Automatic sentiment analysis
• Smart content enhancement

📱 Telegram Integration

• Automated posting to Telegram channels
• Image support with base64 encoding
• Caption length optimization

🔥 Firebase Backend

• Real-time Firestore database
• Timestamp tracking for efficient scraping
• Scalable cloud architecture

⏰ Intelligent Scheduling

• APScheduler for automated execution
• Configurable intervals
• Graceful error handling and logging

Quick Start

Prerequisites

• Python 3.8+
• Google Firebase project
• Telegram Bot Token
• Google Gemini API key

Installation

1. Clone the repository

   git clone https://github.com/mociatto/FootFa.git
   cd FootFa

2. Create virtual environment

   python3 -m venv .venv
   source .venv/bin/activate  # On Windows: .venv\Scripts\activate

3. Install dependencies

   pip install -r requirements.txt

4. Setup Firebase

   • Create a Firebase project at console.firebase.google.com
   • Enable Firestore Database
   • Create a service account and download the JSON key
   • Save it as service-account-key.json in the project root

5. Configure API Keys

   Edit config.py with your credentials:

   # Firebase
   FIREBASE_SERVICE_ACCOUNT_PATH = "service-account-key.json"
   
   # Telegram
   TELEGRAM_BOT_TOKEN = "your_telegram_bot_token"
   TELEGRAM_CHANNEL_ID = "@your_channel_username"
   
   # Google Gemini AI
   GEMINI_API_KEY = "your_gemini_api_key"

6. Create Firestore Index

   Run the system once to get the index creation URL:

   python3 main.py

   Click the provided Firebase console link to create the required composite index.

Usage

Run once:

python3 main.py

Test individual components:

# Test website scraping
python3 website_scraper.py

# Test Telegram posting
python3 telegram_poster.py

# Test AI processing
python3 ai_processor.py

# Test Firestore operations
python3 firestore_manager.py

Project Structure

FootFa/
├── main.py                 # Main scheduler application
├── website_scraper.py      # Multi-site news scraping
├── firestore_manager.py    # Database operations
├── ai_processor.py         # Content AI processing
├── telegram_poster.py      # Telegram posting
├── image_finder.py         # Image search and processing
├── config.py              # Configuration settings
├── requirements.txt       # Python dependencies
├── .gitignore            # Git ignore patterns
└── README.md             # This file

Configuration

Supported News Sources

• Goal.com - International football news
• BBC Sport Football - Premier League and international
• Marca - Spanish football focus
• ESPN FC - American sports perspective
• Sky Sports Football - British coverage
• The Guardian Football - In-depth analysis
• AS.com - Spanish and international
• L'Equipe - French football coverage

Scheduling

Default: Every 30 minutes Configure in config.py:

NEWS_FETCH_INTERVAL_MINUTES = 30

Content Processing

• Automatic summarization
• Duplicate detection
• Sentiment analysis
• Image optimization
• Caption formatting

API Requirements

Firebase

1. Create project at Firebase Console
2. Enable Firestore Database
3. Create service account key

Telegram Bot

1. Message @BotFather on Telegram
2. Create new bot with /newbot
3. Get your bot token
4. Add bot to your channel as administrator

Google Gemini

1. Visit Google AI Studio
2. Create API key
3. Add to config.py

Deployment

Local Development

source .venv/bin/activate
python3 main.py

Production Deployment

• Deploy to cloud services (AWS, GCP, Azure)
• Use process managers (PM2, systemd)
• Set up monitoring and logging
• Configure environment variables

Troubleshooting

Common Issues

Import Errors

pip install -r requirements.txt

Firebase Permission Errors

• Verify service account key path
• Check Firestore security rules

Telegram API Errors

• Verify bot token
• Ensure bot is channel administrator
• Check channel username format (@channel_name)

Website Scraping Issues

• Sites may change their HTML structure
• CSS selectors in website_scraper.py may need updates
• Some sites implement anti-scraping measures

Logs and Debugging

The system provides comprehensive logging:

• Import/execution errors
• API responses
• Database operations
• Scheduling events

Contributing

1. Fork the repository
2. Create feature branch (git checkout -b feature/new-feature)
3. Commit changes (git commit -am 'Add new feature')
4. Push to branch (git push origin feature/new-feature)
5. Create Pull Request

License

This project is for educational and personal use. Please respect the terms of service of all integrated APIs and websites.

Support

For issues and questions:

• Check existing GitHub issues
• Create new issue with detailed description
• Include error logs and configuration (remove sensitive data)

---

⚽ Happy Football News Automation! ⚽