AgentFarm DB to Memory System Converter

Release Notes - AgentFarm DB to Memory System Converter v0.1.2

Overview

The AgentFarm DB to Memory System Converter is a comprehensive module that enables the conversion of data from AgentFarm SQLite databases into a structured memory system. This initial release provides robust functionality for importing agent metadata and memories with support for tiering strategies, validation, and configurable error handling.

Key Features

Core Functionality

• Agent Import System

  • Extracts agent metadata including position, resources, health, and genome data
  • Supports batch processing for efficient memory usage
  • Configurable error handling modes (fail, skip, log)
  • Validation of required fields and data integrity

• Memory Import System

  • Processes different types of memories (states, actions, social interactions)
  • Preserves metadata during conversion
  • Supports incremental and selective imports
  • Batch processing for large datasets

• Memory Tiering

  • Multiple tiering strategies:
    • Simple: All memories in Short-Term Memory (STM)
    • Step-Based: Time decay distribution across tiers
    • Importance-Aware: Combines time decay with importance scores
  • Configurable tier thresholds and importance metrics

Technical Features

• Database Management

  • SQLite database connection pooling
  • Session management with context managers
  • Schema validation and integrity checks
  • Support for both file-based and in-memory databases

• Configuration System

  • Type-safe configuration using dataclasses
  • Comprehensive validation of configuration values
  • Default values for all settings
  • Support for custom memory type mappings

• Error Handling

  • Multiple error handling modes (fail, skip, log)
  • Detailed error logging and reporting
  • Graceful error recovery options
  • Validation error handling

Usage Example

from converter import from_agent_farm

# Basic usage with default configuration
memory_system = from_agent_farm("path/to/agentfarm.db")

# Custom configuration
memory_system = from_agent_farm(
    db_path="path/to/agentfarm.db",
    config={
        "validate": True,
        "error_handling": "fail",
        "batch_size": 200,
        "tiering_strategy_type": "importance_aware",
        "memory_config": {
            "use_mock_redis": False,
            "stm_config": {
                "memory_limit": 1000,
                "ttl": 3600
            }
        }
    }
)

Dependencies

• SQLAlchemy
• Redis

Known Limitations

• Currently only supports SQLite databases
• Memory tiering strategies are fixed and not extensible
• Limited support for custom memory types
• No built-in support for data migration between versions

SimilaritySearchStrategy Release v0.1.1

The SimilaritySearchStrategy is a powerful semantic search mechanism for the agent memory system. It enables finding memories based on semantic similarity using vector embeddings, providing advanced search capabilities across different memory tiers with sophisticated scoring and filtering.

Key Features

• Semantic similarity search: Find memories based on semantic meaning using vector embeddings
• Multi-tier search: Search across STM, IM, and LTM memory tiers
• Advanced scoring system:
  • Cosine similarity scoring between query and memory vectors
  • Configurable minimum score thresholds
  • Score normalization (0.0-1.0 range)
  • Score-based result ordering
• Flexible query support:
  • Text string queries
  • Dictionary state queries
  • Direct vector input
• Metadata filtering: Combine semantic search with metadata filters
• Memory tier targeting: Search specific tiers or across all tiers
• Result limiting: Control number of returned results
• Duplicate handling: Smart handling of duplicate memories across tiers

Validation

The implementation has been extensively validated with:

• Comprehensive test suite covering basic and advanced functionality
• Edge case testing for robustness
• Memory tier transition testing
• Metadata filtering validation
• Content structure testing

Documentation

Detailed documentation is provided in validation/search/similarity/validation.md including:

• Basic usage examples
• Advanced search techniques
• Test results and validation methodology
• Edge case handling
• Memory tier transition scenarios

Getting Started

from memory.search.strategies.similarity import SimilaritySearchStrategy

# Initialize the strategy with your memory system
strategy = SimilaritySearchStrategy(memory_system=memory_system)

# Basic text query search
results = strategy.search(
    query="machine learning model accuracy",
    agent_id="agent-123",
    limit=10
)

# Search with metadata filter and tier specification
results = strategy.search(
    query="data processing pipeline",
    agent_id="agent-123",
    metadata_filter={"type": "process"},
    tier="stm",
    min_score=0.4
)

Test Results

The validation suite has been executed successfully, with key results including:

• ✅ Basic text and dictionary query functionality
• ✅ Metadata filtering and tier-specific search
• ✅ Score threshold filtering
• ✅ Multi-tier search capabilities
• ✅ Edge case handling
• ✅ Memory tier transition scenarios
• ✅ Complex metadata filtering
• ✅ Special character handling

Conclusion

The SimilaritySearchStrategy implementation has been thoroughly validated across functional requirements and edge cases. The test suite provides confidence in the robustness of the implementation and establishes a baseline for regression testing as the codebase evolves.

The validation confirms that the strategy successfully handles various memory structures, search patterns, and retrieval scenarios while properly combining semantic similarity with traditional filtering techniques. All tests pass with the expected results, confirming that the implementation fulfills its designed purpose effectively.

AttributeSearchStrategy Release v0.1.0

Overview

The AttributeSearchStrategy is a flexible search mechanism for the agent memory system. It allows searching for memories based on content attributes and metadata fields, providing advanced filtering and relevance scoring capabilities.

Key Features

• Content-based search: Find memories containing specific text or patterns
• Metadata filtering: Filter memories by metadata fields like type, tags, importance
• Regular expression support: Powerful pattern matching
• Multiple scoring algorithms:
  • Length ratio scoring: Score based on ratio of query length to field length
  • Term frequency scoring: Prioritize based on term frequency
  • BM25 ranking algorithm: Sophisticated information retrieval scoring
  • Binary scoring: Simple match/no-match scoring
• Tier-specific search: Target specific memory tiers (STM, IM, LTM)
• Case sensitivity control: Enable or disable case-sensitive matching
• Conditional matching: Support for both AND/OR logic via match_all parameter
• Array field matching: Search within array elements like tags

Validation

The implementation has been extensively validated with:

• Comprehensive test suite covering basic and advanced functionality
• Edge case testing for robustness
• Performance testing under various conditions

Documentation

Detailed documentation is provided in attribute_search_guide.md including:

• Basic usage examples
• Advanced search techniques
• Best practices
• Example use cases

Getting Started

from memory.search.strategies.attribute import AttributeSearchStrategy

# Initialize the strategy with your memory stores
strategy = AttributeSearchStrategy(
    stm_store=stm_store,
    im_store=im_store,
    ltm_store=ltm_store
)

# Basic content search
results = strategy.search(
    query="meeting",
    agent_id="agent-123",
    limit=10
)

# Search with metadata filter
results = strategy.search(
    query="project",
    agent_id="agent-123",
    metadata_filter={"content.metadata.importance": "high"}
)