Skip to content

waylondev/apexflow

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

175 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ApexFlow

A high-performance framework built on "Everything is Flow" design philosophy, leveraging Kotlin's modern features for clean, solid, and scalable data processing.

🎯 Core Design Philosophy

Everything is Flow

Every operation in ApexFlow is represented as a Flow<T>, enabling seamless composition and reactive stream processing.

Clean & SOLID Principles

  • Single Responsibility: Each component has one clear purpose
  • Open/Closed: Extensible through plugins and composition
  • Liskov Substitution: Interchangeable components
  • Interface Segregation: Focused, minimal interfaces
  • Dependency Inversion: Flow-based dependency management

⚡ Performance Advantage

Streaming vs Sequential Processing

Traditional Sequential Approach

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Read All  │    │    Wait     │    │   Write All │
│   Pages     │────│   (Idle)    │────│   Pages     │
│   (T1)      │    │             │    │   (T2)      │
└─────────────┘    └─────────────┘    └─────────────┘

Total Time: T1 + T2

ApexFlow Streaming Approach

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Read      │    │   Process   │    │   Write     │
│  Page 1     │────│   Page 1    │────│   Page 1    │
└─────────────┘    └─────────────┘    └─────────────┘
        │                │                │
┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Read      │    │   Process   │    │   Write     │
│  Page 2     │────│   Page 2    │────│   Page 2    │
└─────────────┘    └─────────────┘    └─────────────┘
        │                │                │
┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Read      │    │   Process   │    │   Write     │
│  Page 3     │────│   Page 3    │────│   Page 3    │
└─────────────┘    └─────────────┘    └─────────────┘

Total Time: max(T1, T2)

Key Performance Benefits

Theoretical Performance Gain

📊 Example: 406-page PDF conversion
- Traditional: 90s (50s reading + 40s writing)
- ApexFlow: 50s (reading and writing overlap)
- Theoretical Gain: 1.8x faster

Scaling with Complexity

As workflows become more complex, ApexFlow's advantage grows exponentially:

Workflow Steps Traditional ApexFlow Theoretical Gain
2 steps T1 + T2 max(T1,T2) (T1+T2)/max(T1,T2)
5 steps Σ(T1..T5) max(T1..T5) Σ(T1..T5)/max(T1..T5)
N steps Σ(T1..TN) max(T1..TN) Σ(T1..TN)/max(T1..TN)

Why This Works

  • No Idle Time: Reading and writing happen simultaneously
  • Reduced Memory Footprint: Process pages one at a time
  • Backpressure Handling: Automatic flow control
  • Parallel Execution: Leverage modern hardware efficiently

🏗️ Architecture

Component Composition

// Simple flow composition with + operator
val pipeline = pdfReader + imageProcessor + tiffWriter

// Enhance with plugins
val monitoredPipeline = pipeline
    .withPluginTiming()
    .withPluginPerformanceMonitoring()

Plugin System

  • Timing Plugin: Measure execution time per component
  • Logging Plugin: Structured logging throughout the pipeline
  • Performance Monitoring: Real-time metrics collection
  • Custom Plugins: Extensible plugin architecture

🚀 Key Features

Flow-Based Processing

  • Backpressure-Aware: Automatic flow control
  • Resource Management: Automatic cleanup with use()
  • Error Handling: Resilient stream processing
  • Concurrent Execution: Parallel processing with coroutines

Simple & Intuitive API

Basic Conversion

// PDF to TIFF
file.toTiff(outputFile)

// TIFF to PDF  
file.toPdf(outputFile)

// With custom configuration
file.toTiff(
    outputFile,
    pdfConfig = { dpi = 300f, skipBlankPages = true },
    tiffConfig = { compressionType = "JPEG", compressionQuality = 90f }
)

file.toPdf(
    outputFile,
    tiffConfig = { bufferSize = 10 },
    pdfConfig = { jpegQuality = 0.95f }
)

Advanced Usage

// Using ApexFlow DSL for more control
val converter = apexPdfToTiff(
    pdfConfig = { dpi = 300f },
    tiffConfig = { compressionType = "JPEG" }
)
converter.convert(inputFile, outputFile)

// Component composition with + operator
val pdfReader = ApexPdfReader.fromFile(inputFile)
val tiffWriter = ApexTiffWriter.toFile(outputFile)
val pipeline = pdfReader + tiffWriter
pipeline.execute()

📈 Benefits

Performance

  • Reduced Latency: Stream processing eliminates wait times
  • Memory Efficiency: Process pages one at a time, no full dataset loading
  • Scalability: Performance scales with workflow complexity

Development Experience

  • Clean Code: Minimal boilerplate, maximum clarity
  • Composability: Reusable components with + operator
  • Testability: Isolated, testable flow components
  • Maintainability: Clear separation of concerns

Operational Excellence

  • Monitoring: Built-in performance metrics
  • Debugging: Flow visualization and tracing
  • Extensibility: Plugin-based architecture

💡 Use Cases

  • Document Conversion: PDF ↔ TIFF with streaming processing
  • Data Pipelines: ETL workflows with multiple processing steps
  • Real-time Processing: Continuous data stream handling
  • Batch Processing: Large dataset processing with memory efficiency

ApexFlow: Where complexity meets simplicity through the power of flow composition.

About

A high-performance framework built on "Everything is Flow" design philosophy, leveraging Kotlin's modern features for clean, solid, and scalable data processing.

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors

Languages