From 68349cbef189a2cf2d881fbed72a48ac6235a13a Mon Sep 17 00:00:00 2001 From: Emil Rossing Date: Mon, 30 Mar 2026 10:27:23 +0200 Subject: [PATCH] feat(docs): Added some roadmap notes, plus text for review --- ROADMAP.md | 249 +++++++++++++++++++++ ROADMAP_notes.md | 288 ++++++++++++++++++++++++ formulus/formplayer_question_types.md | 12 +- ode_product_roadmap.md | 249 +++++++++++++++++++++ ode_public_architecture_page.md | 311 ++++++++++++++++++++++++++ 5 files changed, 1099 insertions(+), 10 deletions(-) create mode 100644 ROADMAP.md create mode 100644 ROADMAP_notes.md create mode 100644 ode_product_roadmap.md create mode 100644 ode_public_architecture_page.md diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 000000000..50b89dc68 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,249 @@ +# ODE Product Roadmap + +## Core Philosophy + +ODE is built around two fundamental principles: + +### 1. Runtime Flexibility +Users can build and run tailored data collection experiences. + +### 2. Data Sovereignty +Data should **flow through the system**, not be trapped inside it. + +> **Synkronus is a data clearinghouse and synchronization service.** +> It facilitates data exchange between clients while enabling users to extract, retain, and work with their data locally. + +--- + +# Platform Overview + +## Core Backend + +### Synkronus +Data clearinghouse and synchronization service. + +**Responsibilities:** +- sync protocol (pull/push) +- authentication & user management +- app bundle distribution +- data export +- API for all clients + +--- + +## Clients (Current) + +### Formulus +Mobile data collection runtime. + +- JSONForms-based data capture +- custom app execution (webview) +- offline-first sync with Synkronus + +--- + +### Portal +Web-based operational client (React app). + +- user and access management +- app bundle upload +- data export +- system administration + +> Portal is a standard API client — no privileged backend access. + +--- + +### CLI (Synkronus CLI) +Scriptable power-user and automation client. + +- sync operations +- deployment workflows +- CI/CD integration +- admin tasks + +--- + +# New Components + +## Phase 1 — Local Custody (Immediate Priority) + +### ODE Custodian (MVP via CLI) + +**Purpose:** +Enable users to **pull, retain, and curate data locally**, reinforcing Synkronus as a clearinghouse. + +**Core capabilities:** +- pull data from Synkronus +- store locally in JSON-first format +- inspect and modify records +- push corrected records back + +**Initial implementation:** +- CLI-driven workflows +- optional Python helper scripts + +**Example workflow:** +```bash +synk pull --out ./dataset +# inspect / modify JSON +synk push-json ./corrected-records +``` + +**Outcome:** +- users gain true data ownership +- enables offline workflows +- establishes trust in ODE architecture + +--- + +## Phase 2 — Structured Local Store + +### Custodian Data Layer + +**Purpose:** +Introduce a more robust local datastore. + +**Design:** +- JSON as canonical record format +- optional structured store (e.g. SQLite + JSON blobs) +- sync metadata tracking (dirty state, etc.) + +**Capabilities:** +- track local modifications +- identify "dirty" records +- prepare push payloads +- support efficient querying + +--- + +## Phase 3 — Desktop Custodian + +### ODE Custodian (Desktop UI) + +**Purpose:** +Provide a user-friendly interface for data stewardship. + +**Suggested tech:** +- Tauri (webview-based desktop app) + +**Features:** +- browse local datasets +- search and filter records +- JSON viewer/editor +- validation tools +- push corrections to Synkronus +- sync status visibility + +**Design principles:** +- safe editing (no accidental writes) +- clear separation of local vs remote state +- audit-friendly workflows + +--- + +## Phase 4 — Workshop (Development Environment) + +### ODE Workshop + +**Purpose:** +Provide a desktop harness / workbench for running custom apps in a Formulus-like environment. + +ODE Workshop enables developers to run, test, and debug custom applications locally against a runtime that mirrors Formulus. + +**Core capabilities:** +- load custom app bundles +- run apps in a webview +- inject a `formulusAPI` compatible interface +- use mock, local, or synced data +- debug and iterate rapidly + +**Key idea:** +> Develop custom apps against a local runtime that mirrors Formulus, then deploy with confidence. + +**Stretch goals:** +- JSONForms editor +- FormPlayer desktop runtime +- live preview and test harness workflows + +--- + +## Phase 5 — Integration Layer + +**Shared foundations between Custodian and Workshop:** + +- local datastore +- sync engine (extracted from CLI) +- `formulusAPI` abstraction +- authentication/session handling + +**Architecture goal:** +> One backend, one sync contract, multiple specialized clients. + +--- + +# Product Structure + +## Backend +- Synkronus (data clearinghouse) + +## Clients +- Formulus (collection) +- Portal (admin/ops) +- CLI (automation) +- Custodian (local data stewardship) +- Workshop (custom app development) + +--- + +# Key Design Decisions + +### 1. Synkronus is not a data warehouse +It is a clearinghouse — data is expected to leave. + +### 2. JSON is the canonical data format +Tabular representations are derived, not primary. + +### 3. Local custody is first-class +Users should always be able to: +- extract +- store +- analyze +- curate + +### 4. One API, many clients +All components interact through the same backend API. + +### 5. Separation of concerns +- Custodian = data stewardship +- Workshop = development +- Portal = operations + +--- + +# Immediate Next Steps + +1. Implement CLI-based: + - `synk pull` + - `synk push-json` + +2. Define local JSON storage convention + +3. Provide basic scripting examples (Python) + +4. Validate real-world workflow: + - pull → inspect → modify → push + +--- + +# Strategic Positioning + +ODE is: + +> An open, extensible, local-first data collection and processing ecosystem + +With: +- flexible runtime (Formulus + custom apps) +- clean sync backbone (Synkronus) +- user-owned data workflows (Custodian) +- developer tooling (Workshop) diff --git a/ROADMAP_notes.md b/ROADMAP_notes.md new file mode 100644 index 000000000..94bbd6be1 --- /dev/null +++ b/ROADMAP_notes.md @@ -0,0 +1,288 @@ +# 🧭 ODE Architecture & Product Division (Decision Log) + +## 🎯 Purpose + +This document summarizes how ODE functionality is divided across components, based on: + +- user profiles +- responsibilities +- current implementation +- planned evolution + +It serves as a **companion to the roadmap**, explaining *why* functionality lives where it does. + +--- + +# 🧠 Core Architectural Principle + +> **Synkronus is a data clearinghouse, not a data warehouse.** + +This leads to three key consequences: + +1. Data is expected to **flow through**, not remain centrally stored +2. Users must be able to **extract and retain local copies** +3. Functionality is distributed across **multiple specialized clients** + +--- + +# 🧱 Current State (What Exists Today) + +## 1. Synkronus (Backend) + +### Role +Central synchronization and coordination service. + +### Responsibilities +- pull/push sync protocol +- authentication and user management +- app bundle hosting +- data export +- API for all clients + +### Notes +- No privileged clients — everything uses the same API +- Acts as **hub**, not final storage + +--- + +## 2. Formulus (Mobile Client) + +### User Profile +- field workers +- data collectors + +### Responsibilities +- render JSONForms +- run custom apps (webview) +- collect observations +- offline-first operation +- sync with Synkronus + +### Key Insight +Formulus is the **runtime environment for data collection**, not a general-purpose data tool. + +--- + +## 3. Portal (Web Client) + +### User Profile +- administrators +- junior developers +- operational staff + +### Responsibilities +- user management +- app bundle upload +- data export +- system administration + +### Technical Note +- React app served as static assets +- uses same API as CLI +- no special backend access + +### Positioning +Portal is the **operational interface** to Synkronus. + +--- + +## 4. CLI (Synkronus CLI) + +### User Profile +- power users +- senior developers +- automation / CI workflows + +### Responsibilities +- sync operations +- deployment workflows +- scripting and automation +- admin tasks + +### Positioning +CLI is the **scriptable control surface** of the platform. + +--- + +# 🚧 Identified Gap + +## Problem + +Current system lacks a clear solution for: + +- pulling data out of Synkronus in a structured way +- keeping a **durable local copy** +- modifying records safely +- pushing corrected data back + +## Consequence + +Without this: +- "clearinghouse" positioning is weak +- users risk relying on Synkronus as storage +- analysis workflows are fragmented + +--- + +# 🆕 New Component: ODE Custodian + +## Role + +> **Local data stewardship and custody tool** + +--- + +## User Profile + +- analysts +- data managers +- technical project leads + +--- + +## Responsibilities + +### Core +- pull data from Synkronus +- store locally (JSON-first) +- maintain durable dataset + +### Data Stewardship +- inspect records +- identify issues +- modify JSON documents +- track local changes ("dirty" state) + +### Sync +- prepare push payloads +- push corrected records back + +--- + +## Design Principles + +- local-first +- JSON is canonical +- safe editing (avoid accidental writes) +- explicit sync state (clean vs dirty) + +--- + +## Implementation Phases + +### Phase 1 (Immediate) +- CLI-based workflows +- JSON files as local store +- optional Python scripting + +### Phase 2 +- structured local store (e.g. SQLite + JSON) +- dirty tracking +- query capabilities + +### Phase 3 +- desktop UI (Tauri) +- browsing, editing, validation + +--- + +## Positioning + +Custodian is: + +> **The place where data lives after leaving Synkronus** + +--- + +# 🆕 New Component: ODE Workshop + +## Role + +> **Development environment for custom apps and forms** + +--- + +## User Profile + +- developers +- solution designers +- advanced implementers + +--- + +## Responsibilities + +### Development +- load custom app bundles +- run apps locally + +### Simulation +- inject `formulusAPI` +- provide mock or real data + +### Testing +- validate app behavior +- debug interactions + +--- + +## Future Capabilities + +- JSONForms editor +- FormPlayer runtime +- live preview workflows + +--- + +## Positioning + +Workshop is: + +> **Where collection experiences are built and tested** + +--- + +# 🧩 Functional Division + +## By Responsibility + +| Function | Component | +|--------------------------------|------------------| +| Data collection | Formulus | +| Sync + coordination | Synkronus | +| Administration | Portal / CLI | +| Automation | CLI | +| Local data custody | Custodian | +| Data cleaning & correction | Custodian | +| Custom app development | Workshop | + +--- + +## By User Type + +| User Type | Primary Tool | +|---------------------|---------------------| +| Field worker | Formulus | +| Admin / ops | Portal | +| Power user / DevOps | CLI | +| Analyst | Custodian | +| Developer | Workshop | + +--- + +# 🔗 Relationships Between Components + +## Shared Principles + +- all clients use the **same Synkronus API** +- no hidden or privileged pathways +- consistent data model (JSON observations) + +--- + +## Data Flow + +```text +Formulus → Synkronus → Custodian → Analysis tools + ↑ + (push corrections) \ No newline at end of file diff --git a/formulus/formplayer_question_types.md b/formulus/formplayer_question_types.md index ee7445b53..7a02aa5af 100644 --- a/formulus/formplayer_question_types.md +++ b/formulus/formplayer_question_types.md @@ -102,16 +102,8 @@ Allows users to scan QR codes or enter QR code data manually. **Supported Barcode Formats:** -- **QR Code** - Quick Response codes (most common) -- **Code 128** - Linear barcode used for product identification -- **Code 39** - Alphanumeric barcode standard -- **EAN-13** - European Article Number (product barcodes) -- **EAN-8** - Short version of EAN-13 -- **UPC-A** - Universal Product Code (North American products) -- **UPC-E** - Compressed version of UPC-A -- **Data Matrix** - 2D barcode for small items -- **PDF417** - Stacked linear barcode (driver's licenses, ID cards) -- **Aztec** - 2D barcode used for transport tickets +- **QR Code** - Quick Response codes + **Data Structure:** The QR code field stores a simple string value: diff --git a/ode_product_roadmap.md b/ode_product_roadmap.md new file mode 100644 index 000000000..50b89dc68 --- /dev/null +++ b/ode_product_roadmap.md @@ -0,0 +1,249 @@ +# ODE Product Roadmap + +## Core Philosophy + +ODE is built around two fundamental principles: + +### 1. Runtime Flexibility +Users can build and run tailored data collection experiences. + +### 2. Data Sovereignty +Data should **flow through the system**, not be trapped inside it. + +> **Synkronus is a data clearinghouse and synchronization service.** +> It facilitates data exchange between clients while enabling users to extract, retain, and work with their data locally. + +--- + +# Platform Overview + +## Core Backend + +### Synkronus +Data clearinghouse and synchronization service. + +**Responsibilities:** +- sync protocol (pull/push) +- authentication & user management +- app bundle distribution +- data export +- API for all clients + +--- + +## Clients (Current) + +### Formulus +Mobile data collection runtime. + +- JSONForms-based data capture +- custom app execution (webview) +- offline-first sync with Synkronus + +--- + +### Portal +Web-based operational client (React app). + +- user and access management +- app bundle upload +- data export +- system administration + +> Portal is a standard API client — no privileged backend access. + +--- + +### CLI (Synkronus CLI) +Scriptable power-user and automation client. + +- sync operations +- deployment workflows +- CI/CD integration +- admin tasks + +--- + +# New Components + +## Phase 1 — Local Custody (Immediate Priority) + +### ODE Custodian (MVP via CLI) + +**Purpose:** +Enable users to **pull, retain, and curate data locally**, reinforcing Synkronus as a clearinghouse. + +**Core capabilities:** +- pull data from Synkronus +- store locally in JSON-first format +- inspect and modify records +- push corrected records back + +**Initial implementation:** +- CLI-driven workflows +- optional Python helper scripts + +**Example workflow:** +```bash +synk pull --out ./dataset +# inspect / modify JSON +synk push-json ./corrected-records +``` + +**Outcome:** +- users gain true data ownership +- enables offline workflows +- establishes trust in ODE architecture + +--- + +## Phase 2 — Structured Local Store + +### Custodian Data Layer + +**Purpose:** +Introduce a more robust local datastore. + +**Design:** +- JSON as canonical record format +- optional structured store (e.g. SQLite + JSON blobs) +- sync metadata tracking (dirty state, etc.) + +**Capabilities:** +- track local modifications +- identify "dirty" records +- prepare push payloads +- support efficient querying + +--- + +## Phase 3 — Desktop Custodian + +### ODE Custodian (Desktop UI) + +**Purpose:** +Provide a user-friendly interface for data stewardship. + +**Suggested tech:** +- Tauri (webview-based desktop app) + +**Features:** +- browse local datasets +- search and filter records +- JSON viewer/editor +- validation tools +- push corrections to Synkronus +- sync status visibility + +**Design principles:** +- safe editing (no accidental writes) +- clear separation of local vs remote state +- audit-friendly workflows + +--- + +## Phase 4 — Workshop (Development Environment) + +### ODE Workshop + +**Purpose:** +Provide a desktop harness / workbench for running custom apps in a Formulus-like environment. + +ODE Workshop enables developers to run, test, and debug custom applications locally against a runtime that mirrors Formulus. + +**Core capabilities:** +- load custom app bundles +- run apps in a webview +- inject a `formulusAPI` compatible interface +- use mock, local, or synced data +- debug and iterate rapidly + +**Key idea:** +> Develop custom apps against a local runtime that mirrors Formulus, then deploy with confidence. + +**Stretch goals:** +- JSONForms editor +- FormPlayer desktop runtime +- live preview and test harness workflows + +--- + +## Phase 5 — Integration Layer + +**Shared foundations between Custodian and Workshop:** + +- local datastore +- sync engine (extracted from CLI) +- `formulusAPI` abstraction +- authentication/session handling + +**Architecture goal:** +> One backend, one sync contract, multiple specialized clients. + +--- + +# Product Structure + +## Backend +- Synkronus (data clearinghouse) + +## Clients +- Formulus (collection) +- Portal (admin/ops) +- CLI (automation) +- Custodian (local data stewardship) +- Workshop (custom app development) + +--- + +# Key Design Decisions + +### 1. Synkronus is not a data warehouse +It is a clearinghouse — data is expected to leave. + +### 2. JSON is the canonical data format +Tabular representations are derived, not primary. + +### 3. Local custody is first-class +Users should always be able to: +- extract +- store +- analyze +- curate + +### 4. One API, many clients +All components interact through the same backend API. + +### 5. Separation of concerns +- Custodian = data stewardship +- Workshop = development +- Portal = operations + +--- + +# Immediate Next Steps + +1. Implement CLI-based: + - `synk pull` + - `synk push-json` + +2. Define local JSON storage convention + +3. Provide basic scripting examples (Python) + +4. Validate real-world workflow: + - pull → inspect → modify → push + +--- + +# Strategic Positioning + +ODE is: + +> An open, extensible, local-first data collection and processing ecosystem + +With: +- flexible runtime (Formulus + custom apps) +- clean sync backbone (Synkronus) +- user-owned data workflows (Custodian) +- developer tooling (Workshop) diff --git a/ode_public_architecture_page.md b/ode_public_architecture_page.md new file mode 100644 index 000000000..a43214732 --- /dev/null +++ b/ode_public_architecture_page.md @@ -0,0 +1,311 @@ +# ODE Architecture + +ODE is an open, extensible, local-first data collection and processing ecosystem. + +It is designed around a simple principle: + +> **Data should flow through the system, not be trapped inside it.** + +That principle shapes both the architecture and the product strategy behind ODE. + +--- + +## The Big Picture + +ODE is a coordinated set of components that work together through a shared API and synchronization model - like a well-playing jazz ensemble. + +At a high level, the ecosystem consists of: + +- **Formulus** for offline-first data collection +- **Synkronus** for synchronization and coordination +- **Portal** for web-based administration and operations +- **CLI** for scripting, automation, and power-user workflows +- **ODE Custodian** for local data stewardship and curation *(planned)* +- **ODE Workshop** for custom app and form development *(planned)* + +--- + +## Core Philosophy + +### Runtime flexibility + +ODE supports tailored data collection experiences rather than forcing every project into a single rigid UI. + +This is enabled through: + +- JSONForms-based form rendering +- custom applications running inside a webview +- a shared API surface for interacting with local data and runtime features + +### Data sovereignty + +ODE is designed so that users can pull data out, retain it locally, analyze it in their own environment, and optionally push curated corrections back. + +This is why **Synkronus is best understood as a data clearinghouse and synchronization service, not a data warehouse**. + +--- + +## Existing Components + +## Synkronus + +**Synkronus** is the backend synchronization and coordination service at the center of ODE. + +Its responsibilities include: + +- pull/push synchronization +- authentication and user management +- app bundle distribution +- export functionality +- a shared API for ODE clients + +Synkronus acts as the **hub** of the system, but not as the final resting place for data. It facilitates exchange between clients and supports local ownership of data outside the server. + +--- + +## Formulus + +**Formulus** is the mobile runtime for field data collection. + +It is responsible for: + +- rendering JSONForms-based forms +- running custom apps inside a webview +- collecting observations +- working offline-first +- synchronizing with Synkronus + +Formulus is focused on the collection experience. It is not intended to be the administrative interface or the primary environment for downstream data stewardship. + +--- + +## Portal + +**Portal** is the browser-based operational client for Synkronus. + +It supports tasks such as: + +- user management +- app bundle upload +- data export +- administrative operations + +Portal is a standard client of the same Synkronus API used by other ODE components. It does not rely on privileged side-channel access to the backend. + +This makes it a clean, transparent part of the overall architecture rather than a special-case admin panel. + +--- + +## CLI + +The **Synkronus CLI** is the scriptable and automation-friendly client in the ODE ecosystem. + +It is particularly useful for: + +- power users +- senior developers +- CI/CD workflows +- deployment and automation +- administrative tasks +- direct sync workflows + +The CLI and Portal are complementary. They serve different user preferences while relying on the same underlying API. + +--- + +## Planned Components + +## ODE Custodian + +**ODE Custodian** is the planned local data stewardship tool in the ODE ecosystem. + +Its purpose is to make the "clearinghouse" model operational and credible. + +Custodian is intended to allow users to: + +- pull data from Synkronus +- keep a durable local copy +- inspect records locally +- modify JSON records where appropriate +- track local changes +- push corrected records back to Synkronus + +The first steps toward Custodian are expected to be CLI-driven, with simple local JSON workflows and scripting support. Over time, this can evolve toward a richer local data layer and a desktop interface. + +Custodian is aimed at people who need to **retain, curate, and steward real project data** outside the server. + +--- + +## ODE Workshop + +**ODE Workshop** is the planned development environment for custom ODE apps and forms. + +Its purpose is to make development faster, more transparent, and more pleasant than relying entirely on mobile-device iteration loops. + +Workshop is envisioned to support: + +- loading custom app bundles locally +- running them in a desktop webview +- injecting a compatible `formulusAPI` +- using mock or real local data +- debugging and rapid iteration +- future JSONForms editing workflows +- future FormPlayer-based development loops + +Workshop is aimed at developers, advanced implementers, and solution designers who need a focused environment for building and testing collection experiences. + +--- + +## Functional Division + +ODE deliberately separates responsibilities rather than forcing every workflow into a single application. + +### Collection +Handled by **Formulus** + +### Synchronization and coordination +Handled by **Synkronus** + +### Administration and operations +Handled by **Portal** and **CLI** + +### Local data stewardship and correction +Planned for **ODE Custodian** + +### Custom app and form development +Planned for **ODE Workshop** + +This separation helps keep each tool understandable and aligned with the needs of its users. + +--- + +## User Profiles + +Different ODE components are optimized for different kinds of users. + +### Field workers +Primarily use **Formulus** + +### Administrators and operational staff +Primarily use **Portal** + +### Power users, senior developers, and automation workflows +Primarily use the **CLI** + +### Analysts, data managers, and technical project leads +Will primarily benefit from **ODE Custodian** + +### Developers and solution designers +Will primarily benefit from **ODE Workshop** + +--- + +## Why the Architecture Matters + +Many systems in this space do one part well: + +- data collection +- administrative workflows +- analysis +- extensibility +- local ownership + +ODE aims to connect these concerns without forcing them into the same interface. + +This architecture matters because it creates room for: + +- robust offline-first collection +- clear synchronization boundaries +- local data ownership +- extensible project-specific experiences +- both graphical and scriptable workflows + +--- + +## Architecture Principle: One Backend, Many Clients + +A core design decision in ODE is that user-facing components should, where practical, act as clients of the same Synkronus API. + +This means: + +- no hidden privileged pathways +- cleaner security boundaries +- easier reuse across components +- more consistent behavior between tools +- clearer mental models for users and developers + +In practice, this already holds for: + +- Portal +- CLI +- Formulus + +And it is expected to guide the design of: + +- Custodian +- Workshop + +--- + +## Data Flow + +A simplified data flow looks like this: + +```text +Formulus → Synkronus → Custodian → analysis environment + ↑ + push curated corrections +``` + +This reflects the intended role of Synkronus as a clearinghouse: data is collected, synchronized, extracted, retained locally, analyzed, and in some cases corrected and pushed back. + +--- + +## Development Flow + +A simplified development flow looks like this: + +```text +Workshop → Formulus → Synkronus +``` + +Workshop is intended to shorten the path from custom app and form development to real runtime behavior in Formulus. + +--- + +## What Already Exists vs What Is Envisioned + +### Already in place + +- **Synkronus** as synchronization and coordination backend +- **Formulus** as offline-first mobile client +- **Portal** as web-based operational client +- **CLI** as scriptable and automation-friendly client +- a shared API model across current clients +- export support from Synkronus + +### In active product thinking and future development + +- **ODE Custodian** for local data custody, cleaning, and push-back workflows +- **ODE Workshop** for custom app and form development +- richer local data handling and dirty tracking +- desktop tooling based on a webview-friendly stack +- stronger bridges between runtime, operations, and data stewardship + +--- + +## Strategic Positioning + +ODE is more than a form app and more than a sync server. + +It is an effort to provide: + +- a flexible collection runtime +- a clear synchronization backbone +- local-first data ownership +- specialized clients for different roles +- an extensible platform for tailored solutions + +In short: + +> **ODE is an open, extensible, local-first data collection and processing ecosystem.**