Home

NanoChat Wiki

Welcome to the NanoChat Wiki. NanoChat is a lightweight, modern Android chat application built with Jetpack Compose and Material 3, designed to provide a unified interface for multiple AI inference backends.

---

📖 Table of Contents

1. Project Vision
2. Architecture Overview
3. Inference Backends
4. Data & Persistence
5. Development Guide
6. Roadmap

---

1. Project Vision

NanoChat's core philosophy is: Use the smallest, fastest model that can answer the question. It provides a single interface to interact with three distinct backends, switchable on the fly:

• Nano: Local Gemini Nano via Google AICore (zero cost, private).
• Model: Locally downloaded open-source LLMs (via MediaPipe).
• Remote: OpenAI-compatible cloud APIs (fallback for complexity).

Why This Exists

Targeted at devices like the Pixel 9, NanoChat leverages on-device AI capabilities (AICore) to provide a "JARVIS-style" experience that works offline for everyday queries while offering cloud scaling when needed.

---

2. Architecture Overview

NanoChat follows a clean MVVM architecture with a repository pattern to abstract inference logic.

Component Diagram

┌─────────────────────────────────────────────────────────────────┐
│  ChatScreen  ──  FilterChip: [Nano] [Model] [Remote]           │
│                                                                  │
│  ChatViewModel                                                   │
│    └── ChatRepository                                            │
│          └── InferenceClient (interface)                         │
│                ├── LocalInferenceClient   → AICore / Gemini Nano│
│                ├── DownloadedModelClient  → MediaPipe LLM       │
│                └── RemoteInferenceClient  → OkHttp SSE stream   │
│                                                                  │
│  Room DB  (ChatSession + ChatMessage + DownloadedModel)          │
│  DataStore (InferenceMode, Non-secret settings)                 │
│  EncryptedPrefs (API Keys, Secure Tokens)                       │
└─────────────────────────────────────────────────────────────────┘

Key Interfaces

• InferenceClient: The primary contract for all backends.
  • isAvailable(): Returns availability status.
  • streamChat(history, prompt): Returns a Flow of string chunks.

---

3. Inference Backends

🟢 Nano (AICore)

• Engine: Gemini Nano via com.google.ai.edge.aicore.
• Setup: Must be enabled in Developer Options → Gemini Nano.
• Characteristics: Foreground-only, subject to OS-level quotas, no cost, fully offline.
• Context: History is flattened into a single prompt (default window: last 10 turns).

🔵 Model (Local LLM) - Planned

• Engine: MediaPipe LLM Inference (LiteRT).
• Storage: .task files stored in internal or external storage.
• Catalog: Support for Qwen2.5, DeepSeek-R1, Phi-2, and Gemma.
• Format: Uses ChatML (<|user|> / <|assistant|>).

☁️ Remote (Cloud)

• Engine: OkHttp with SSE (Server-Sent Events).
• Compatibility: Any OpenAI-compatible endpoint.
• Features: Token-by-token streaming, configurable base URL, and model name.

---

4. Data & Persistence

Local Database (Room)

• ChatSession: Metadata for conversations (ID, title, timestamp).
• ChatMessage: Individual messages (role, content, timestamp, session ID).
• DownloadedModel: Tracking local model files and download progress.

Preferences Split

To maintain security, settings are split:

• DataStore: Non-sensitive data like InferenceMode, baseUrl, and modelName.
• EncryptedSharedPreferences: Sensitive secrets like apiKey and huggingFaceToken.

---

5. Development Guide

Tech Stack

• UI: Jetpack Compose, Material 3 (Expressive)
• Language: Kotlin 2.1.10
• Build: Gradle 9.3.1 / AGP 9.1.0
• Minimum SDK: 31 (Android 12)

Building the Project

1. Open in Android Studio Ladybug or newer.
2. Ensure JDK 17 is configured.
3. Use the Gradle wrapper:
   • POSIX: ./gradlew assembleDebug
   • Windows: .\gradlew.bat assembleDebug

---

6. Roadmap

✅ Milestone 1: Foundations

• 3-tab shell: Chat, Models (Stub), Settings.
• Room persistence for sessions and messages.
• Remote backend with SSE streaming.
• AICore (Nano) integration.
• Secure storage for API keys.

⏳ Milestone 2: Local Models

• MediaPipe LLM Inference integration.
• Model catalog and download manager.
• Storage management (Move to SD card).
• Enhanced error handling and retry logic.

🚀 Future

• Voice input/output.
• Image/Multimodal support (Gemini Nano with Multimodality).
• UI Polish & Animations.