Game Character AI Plugin for G-Assist

This plugin allows G-Assist to take screenshots of your game and use AI vision to identify characters and answer questions about them. It supports multiple AI backends including Groq, LM Studio, and optional Supabase character database integration.

Features

• Character Analysis: Takes a screenshot and identifies game characters using AI vision
• Question Answering: Ask specific questions about characters' lore, abilities, story, and appearance
• Context Awareness: Remembers the last analyzed character for follow-up questions
• Game Integration: Automatically detects the current game being played
• Multiple AI Backends: Supports Groq API, LM Studio, and optional Supabase database
• Smart Window Detection: Intelligent game window identification using AI

Setup

Prerequisites

1. Install Python 3.10 or higher
2. Choose your AI backend (at least one required):
   • Groq API (recommended for fast inference)
   • LM Studio (for local inference)
   • Supabase (optional character database)

Installation

1. Run the setup script:

   setup.bat

2. Configure your API keys:

   • Copy config.json.example to config.json
   • Edit config.json and add your API keys:

   {
     "groq_api_key": "your_groq_api_key_here",
     "supabase_endpoint_url": "your_supabase_endpoint_here",
     "supabase_auth_header": "Bearer your_supabase_token_here",
     "lm_studio_endpoint": "http://localhost:1234/v1/chat/completions",
     "lm_studio_api_key": "lm-studio"
   }

3. Get API Keys:

   • Groq API: Visit https://console.groq.com/keys to get a free API key
   • LM Studio: Download from https://lmstudio.ai/ and run locally with a vision model
   • Supabase: Optional - set up your own character database endpoint

4. Build the plugin:

   build.bat

5. Install the plugin: Copy the contents of dist\game-character-ai\ to:

   %PROGRAMDATA%\NVIDIA Corporation\nvtopps\rise\adapters\game-character-ai\
   

Usage

Once installed, you can use these voice commands with G-Assist:

• "Analyze the character on screen" - Takes a screenshot and identifies the character
• "Who is this character and what are their abilities?" - Identifies character and answers specific question
• "What's the backstory of this character?" - Analyzes character focusing on lore
• "Clear character context" - Resets to analyze a new character

Configuration Options

The config.json file supports these options:

Required (at least one)

• groq_api_key: Your Groq API key for fast cloud inference
• lm_studio_endpoint: LM Studio endpoint for local inference (default: "http://localhost:1234/v1/chat/completions")

Optional

• supabase_endpoint_url: Your Supabase character database endpoint
• supabase_auth_header: Authorization header for Supabase
• lm_studio_api_key: API key for LM Studio (default: "lm-studio")

AI Backend Details

Groq (Recommended)

• Pros: Fast inference, free tier available, no local setup required
• Cons: Requires internet connection, API limits
• Model: llama-3.3-70b-versatile
• Setup: Get free API key from https://console.groq.com/keys

LM Studio (Local)

• Pros: Fully local, no API limits, privacy-focused
• Cons: Requires powerful hardware, slower inference
• Recommended Model: Gemma 2 27B Vision or similar vision-capable model
• Setup: Download LM Studio and load a vision model

Supabase Database (Optional)

• Purpose: Enhanced character recognition using pre-built character database
• Benefit: More accurate character identification for popular games
• Setup: Deploy your own Supabase function for character similarity search

Window Detection Priority

The plugin uses smart window detection in this order:

1. Game Window - Using AI-powered window detection via Groq
2. Foreground Window - Currently active large window
3. Full Screen - Fallback to entire screen capture

Requirements

• Windows 10/11
• Python 3.10+
• At least one AI backend configured:
  • Groq API key (free tier available), OR
  • LM Studio with vision model, OR
  • Both for redundancy
• Active internet connection (for Groq/Supabase)

Troubleshooting

API Issues

• Groq API Error: Verify your API key is valid and has quota remaining
• LM Studio Connection: Ensure LM Studio is running and has a vision model loaded
• Supabase Error: Check your endpoint URL and authorization token

Screenshot Issues

• Character Not Identified: Try to center the character in view with good lighting
• Window Not Found: Ensure the game window is visible and not minimized
• Screenshot Error: Ensure no full-screen exclusive applications are blocking capture

Configuration Issues

• Missing config.json: Copy from config.json.example and configure your keys
• Invalid JSON: Validate your config.json syntax
• No AI Backend: At least one of Groq or LM Studio must be configured

Privacy & Security

• Screenshots: Captured screenshots are sent to configured AI services (Groq/Supabase)
• Local Processing: LM Studio processes everything locally if preferred
• API Keys: Store securely in config.json (never commit to version control)
• Logging: Activity is logged to game-character-ai.log in your user profile

Development

To contribute or modify:

1. Clone the repository
2. Install dependencies: pip install -r requirements.txt
3. Configure: Copy config.json.example to config.json and add your keys
4. Test: Run python plugin.py directly for debugging
5. Build: Use build.bat to create distribution

License

Licensed under the Apache License, Version 2.0. See LICENSE file for details.