OmniBioAI Workflow Bundles

Overview

omnibioai-workflow-bundles is the canonical repository for engine-agnostic, immutable bioinformatics workflow bundles used by the OmniBioAI Workflow Registry Service.

This repository exists for authoring and version control only and is not required in deployed OmniBioAI environments.

This repository contains source-controlled workflow bundle templates that are:

• Authored and reviewed in Git
• Packaged as immutable bundles
• Uploaded via CLI into OmniBioAI
• Stored as objects in OmniObjectService
• Indexed as metadata in the Workflow Registry

Important: This repository is not accessed at runtime by OmniBioAI plugins or services. Runtime execution always resolves workflows via the Workflow Registry + OmniObjectService, never directly from Git.

---

What Is a Workflow Bundle?

A workflow bundle is an immutable artifact that encapsulates everything required to execute a single computational workflow using a specific engine.

Each bundle contains:

• A workflow definition (WDL / Nextflow / Snakemake / CWL)
• Engine-specific configuration files
• Optional helper scripts or resources
• A strict manifest.json describing metadata and entrypoints

Immutability applies after upload; bundles may evolve in Git, but each upload produces a new immutable artifact.

Once uploaded to OmniBioAI, a bundle is:

• Zipped
• Stored as an object (identified by object_id)
• Referenced immutably by the Workflow Registry

Bundles are never modified after registration.

---

Supported Workflow Engines

This repository supports workflow bundles for the following engines:

• WDL (Cromwell-compatible)
• Nextflow
• Snakemake
• CWL

Each bundle targets one engine only. Equivalent workflows using different engines are stored as separate bundles.

---

Repository Structure

omnibioai-workflow-bundles/
├── circrna/
│   └── omnibioai_circrna_circtools_wdl_v1/
│       ├── manifest.json
│       ├── workflow/
│       │   └── circrna_circtools_v1.wdl
│       ├── config/
│       │   └── inputs.json
│       ├── docker/
│       └── README.md
│
├── rnaseq/
├── chipseq/
├── single_cell/
└── README.md

Key rules:

• Each directory under a domain (circrna, rnaseq, etc.) is a single workflow bundle
• Directory names are human-friendly only
• Canonical identity comes from manifest.json, not paths

---

Bundle Identity and Versioning

Each workflow version is uniquely identified by the tuple:

(category, engine, name, version)

Example

category: rnaseq
engine: wdl
name: rnaseq_wdl_v1
version: 1.0.0

Uploading a new version:

• Creates a new bundle
• Generates a new object_id
• Inserts a new registry entry
• Leaves older versions untouched

---

Manifest Contract (manifest.json)

Every bundle must include a manifest.json file that defines its canonical metadata.

Example:

{
  "name": "circrna_circtools_wdl_v1",
  "display_name": "circRNA detection using circtools (WDL)",
  "category": "circrna",
  "engine": "wdl",
  "version": "1.0.0",
  "entrypoint": "workflow/circrna_circtools_v1.wdl",
  "configs": ["config/inputs.json"],
  "description": "circRNA detection and quantification using circtools",
  "container_image": "biocontainers/circtools:latest"
}

The manifest is the single source of truth for the Workflow Registry.

---

Relationship to the Workflow Registry

The Workflow Registry Service is the authoritative metadata index for all workflows in OmniBioAI.

Separation of Responsibilities

Component	Responsibility
omnibioai-workflow-bundles	Authoring & version control
Workflow Upload CLI	Validation & ingestion
Workflow Registry	Metadata & discoverability
OmniObjectService	Storage of bundle bytes
Execution Services	Materialization & execution

Registry = metadata Object store = bytes

The registry never stores workflow files and never knows filesystem paths — only object_id.

---

Workflow Ingestion (CLI-first)

Bundles from this repository are uploaded using the OmniBioAI CLI.

Staging Layout (example)

input/
  circrna_circtools_wdl_v1/
    manifest.json
    workflow/
      circrna_circtools_v1.wdl
    config/
      inputs.json

Upload Command

python manage.py workflow_upload \
  --bundle input/circrna_circtools_wdl_v1 \
  --created-by manish \
  --enable

What Happens Internally

1. Validate bundle layout
2. Parse manifest.json
3. Zip the bundle
4. Store zip via OmniObjectService
5. Receive object_id
6. Insert immutable registry entry

---

How Plugins Use Workflows

OmniBioAI plugins never access this repository directly.

Instead:

1. Plugin queries WorkflowCatalogService

2. User selects workflow + version

3. Plugin passes workflow ID to execution service

4. Execution service:

   • Resolves registry entry
   • Fetches bundle via object_id
   • Materializes workflow for execution

This ensures:

• No filesystem scanning
• No path-based inference
• Full reproducibility and auditability

---

Design Principles

• ID-first architecture
• Immutable workflow bundles
• Metadata-driven discovery
• CLI-first ingestion
• Strict separation of registry, storage, and execution

---

Intended Audience

This repository is intended for OmniBioAI developers and workflow authors. End users interact with workflows exclusively via the OmniBioAI UI and APIs.

---

What This Repository Is NOT

• ❌ A runtime workflow store
• ❌ A plugin repository
• ❌ A database-backed system
• ❌ A mutable workflow workspace

---

One-Sentence Summary

omnibioai-workflow-bundles provides version-controlled, engine-specific workflow bundles that are ingested into the OmniBioAI Workflow Registry as immutable artifacts, enabling reproducible, metadata-driven pipeline execution across multiple engines and backends.