🚀 LP-SLAM-TECC

Language-Perception SLAM with
Text Error Correction (TECC) + RAG + LLM

Semantic SLAM system that understands text in the environment

GitHub Repository

---

📌 Abstract

Traditional SLAM systems focus primarily on geometry and ignore semantic information present in the environment. LP-SLAM-TECC enhances SLAM by integrating scene text understanding into the mapping process.

The system detects text from images, corrects OCR errors using Text Error Correction & Classification (TECC), enriches understanding using Retrieval-Augmented Generation (RAG), and stores results in a semantic map. This enables robots to understand signs like EXIT, room numbers, and labels in a human-like way.

---

🧠 System Architecture

Camera / Image
      ↓
OCR (EasyOCR)
      ↓
TECC (Correction + Classification)
      ↓
RAG (FAISS + Knowledge Base)
      ↓
Semantic Map

---

✨ Key Features

• 🔍 Robust OCR using EasyOCR
• 🛠 Text Error Correction (handles OCR noise)
• 🧠 Semantic Classification using LLM-ready TECC
• 📚 Retrieval-Augmented Generation (RAG)
• 🗺 In-memory Semantic Map for SLAM
• 🌐 Interactive Web UI (FastAPI + HTML/CSS/JS)

---

📂 Project Structure

LP-SLAM-TECC/
├── app.py                  # FastAPI backend
├── requirements.txt
├── knowledge_base/
│   └── rag_data.jsonl
├── slam_engine/
│   ├── text_detection.py
│   ├── tecc_model.py
│   ├── rag_engine.py
│   └── tecc_db.json
├── static/
│   ├── app.js
│   └── ui.css
├── templates/
│   └── index.html
├── build_rag_index.py
├── rag_index.faiss
├── test_pipeline.py
└── README.md

---

⚙️ Installation & Setup

1️⃣ Clone the Repository

git clone https://github.com/ysujith728/LP-SLAM-TECC.git
cd LP-SLAM-TECC

2️⃣ Create Virtual Environment

python -m venv venv
venv\Scripts\activate

3️⃣ Install Dependencies

pip install -r requirements.txt

---

🚀 Running the Project

uvicorn app:app --reload

Open your browser and go to: http://127.0.0.1:8000

---

🧪 Example Output

Input image containing text:

EXIT

Semantic Map Output:

{
  "text": "EXIT",
  "label": "exit_sign",
  "meaning": "Indicates an emergency exit used for evacuation."
}

---

🗺 Semantic Mapping

The semantic map stores:

• Detected text
• Corrected text
• Semantic label
• Contextual meaning

This allows the SLAM system to reason about the environment instead of just mapping walls.

---

🎯 Applications

• 🤖 Indoor robot navigation
• 🏢 Smart buildings
• ♿ Assistive navigation systems
• 📍 Human-aware SLAM research

---

🔬 Research Contribution

• Introduces text as a first-class SLAM landmark
• Demonstrates TECC + RAG synergy
• Reduces OCR misclassification errors
• Bridges vision, language, and mapping

---

👨‍💻 Author

Sujith Y
B.Tech CSE
GitHub: ysujith728

---

📜 License

This project is intended for academic and research purposes.

---

⭐ If you like this project, consider starring the repository ⭐