Custom provider implementations for Llama Stack that extend the capabilities of AI applications with specialized safety and content filtering features.
- Overview
- Building and publishing
- Features
- Quick Start
- Configuration
- Usage Examples
- Contributing
- License
- Documentation
This repository contains custom providers for Llama Stack, including:
- Question Validity Shield: Ensures queries are related to OpenShift/Ansible topics (It can be Configured for other Platforms).
- Redaction Shield: Automatically detects and redacts sensitive information from user messages
- Additional safety and content filtering providers
Manual procedure, assuming an existing PyPI API token available:
## Generate distribution archives to be uploaded into Python registry
pdm run python -m build
## Upload distribution archives into Python registry
pdm run python -m twine upload --repository ${PYTHON_REGISTRY} dist/*
- Pattern-based redaction: YAML-configurable regex patterns for flexible content filtering
- Automatic detection: Detects credit card numbers, API keys, tokens, passwords, and custom patterns
- Topic validation: Ensures queries are related to specified topics (OpenShift/Ansible) (It can be configured for other Platforms)
- LLM-powered classification: Uses AI to determine query relevance
- Customizable responses: Configure custom messages for invalid queries
- Python >= 3.12
- Llama Stack == 0.4.3
- pydantic >= 2.10.6
-
Clone the repository
git clone https://github.com/lightspeed-core/lightspeed-providers.git cd lightspeed-providers -
Install dependencies
pip install -e . -
Install Llama Stack (if not already installed)
pip install llama-stack
Usage: make <OPTIONS> ... <TARGETS>
Available targets are:
install Install all dependencies using uv
test Run the unit tests
test-unit Run the unit tests
test-e2e Run end to end tests for the service
test-solr Run Solr vector_io tests
check-types Checks type hints in sources
security-check Check the project for security issues
lint Lint source code, including tests
format Format the code into unified format
shellcheck Run shellcheck
black Check source code using Black code formatter
pylint Check source code using Pylint static code analyser
pyright Check source code using Pyright static type checker
docstyle Check the docstring style using Docstyle checker
ruff Check source code using Ruff linter
verify Run all linters
clean Clean build artifacts
build Build package
help Show this help screen
- Install the Python package
pip install lightspeed_stack_providers
- Configure your run.yaml (see Configuration section below)
Add to your run.yaml file:
# External providers configuration
external_providers_dir: ${env.EXTERNAL_PROVIDERS_DIR:/providers.d}
# Changes in the providers
providers:
safety:
- provider_id: llama-guard
provider_type: inline::llama-guard
config:
excluded_categories: []
# For Redaction shields
- provider_id: lightspeed_redaction
provider_type: inline::lightspeed_redaction
config:
case_sensitive: false
rules:
- pattern: "(?i)(password|passwd)[\\s:=]+[^\\s]+"
replacement: "[REDACTED_PASSWORD]"
- pattern: "(?i)(registry|image):\\s*([\\w\\d\\.-]+)(:[\\w\\d\\.-]+)?"
replacement: "\\1: [REDACTED_IMAGE]"
- pattern: "(?i)(url|endpoint):\\s*https?://[\\w\\.-]+(:\\d+)?(/[\\w\\d\\.-]*)*"
replacement: "\\1: [REDACTED_URL]"
- pattern: "\\b(?:\\d{1,3}\\.){3}\\d{1,3}\\b"
replacement: "[REDACTED_IP]"
- pattern: "(?i)(api_key|secret)[\\s:=]+[a-zA-Z0-9\\-_]{16,}"
replacement: "[REDACTED_SECRET]"
- pattern: "(?i)(ssh-rsa|ssh-ed25519)\\s+[A-Za-z0-9+/=]+"
replacement: "[REDACTED_SSH_KEY]"
# for question validity
- provider_id: lightspeed_question_validity
provider_type: inline::lightspeed_question_validity
config:
model_id: ${env.INFERENCE_MODEL}
model_prompt: |-
Instructions:
- You are a question classifying tool
- You are an expert in ansible
- Your job is to determine where or a user's question is related to ansible technologies and to provide a one-word response
- If a question appears to be related to ansible technologies, answer with the word ${allowed}, otherwise answer with the word ${rejected}
- Do not explain your answer, just provide the one-word response
Example Question:
Why is the sky blue?
Example Response:
${rejected}
Example Question:
Can you help generate an ansible playbook to install an ansible collection?
Example Response:
${allowed}
Example Question:
Can you help write an ansible role to install an ansible collection?
Example Response:
${allowed}
Question:
${message}
Response:
invalid_question_response: |-
Hi, I'm the Ansible Lightspeed Intelligent Assistant, I can help you with questions about Ansible,
please ask me a question related to Ansible.
# changes in the agents :
shields:
- shield_id: lightspeed_question_validity-shield
provider_id: lightspeed_question_validity
- shield_id: redaction-shield
provider_id: lightspeed_redaction
provider_shield_id: lightspeed-redaction-shield# Test the redaction shield
curl -X POST "http://localhost:8321/v1/safety/run-shield" \
-H "Content-Type: application/json" \
-d '{
"shield_id": "redaction-shield",
"messages": [
{
"role": "user",
"content": "My API key is abc123xyz and password is secret456"
}
]
}'- Create provider directory
mkdir -p ./providers.d/inline/safety/ curl -o ./providers.d/inline/safety/lightspeed_question_validity.yaml https://raw.githubusercontent.com/lightspeed-core/lightspeed- providers/refs/heads/main/resources/external_providers/inline/safety/lightspeed_question_validity.yaml curl -o ./providers.d/inline/safety/lightspeed_question_validity.yaml https://raw.githubusercontent.com/lightspeed-core/lightspeed- providers/refs/heads/main/resources/external_providers/inline/safety/lightspeed_redaction.yaml
- Add external provider definition
# resources/external_providers/your_provider.yaml module: lightspeed_stack_providers.providers.inline.safety.your_provider config_class: lightspeed_stack_providers.providers.inline.safety.your_provider.config.YourProviderConfig pip_packages: ["lightspeed_stack_providers"] api_dependencies: - safety
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
This project is licensed under the MIT License - see the LICENSE file for details.