Artificial Intelligence based Earth System Data Assimilation (AIESDA)

Welcome to the AIESDA Wiki

AIESDA (Artificial Intelligence based Earth System Data Assimilation) is a next-generation framework designed to integrate AI Foundation Models into traditional Numerical Weather Prediction (NWP) and Data Assimilation (DA) workflows. The goal of this project is to provide Data Assimilation engine based on JEDI and to bridge it seamlessly with Dynamical Forecast Systems (Bharat, Mithuna) as well as cutting-edge AI Foundation Models (GraphCast, Pangu-Weather, etc.).

🚀 Key Features

Modular and Object Oriented Design

Seperation of concern

Entry Level Data Identity Verification

🛠 Installation

git clone https://github.com/NCMRWF/aiesda.git
cd aiesda
make install

Package directory structure after installation

aiesda_build_2026.1/
├── lib/
│   ├── aiesda/
│   │   ├── __init__.py    (from setup.py build)
│   │   ├── VERSION        (copied manually)
│   │   ├── nml/           (synced assets)
│   │   ├── yaml/          (synced assets)
│   │   ├── pylib/         (synced assets)
│   │   ├── pydic/
│   │   ├── scripts/
│   │   ├── jobs/
│   │   ├── pallets/
│   │   ├── docs/ 
│   │   └── ...
│   └── [site-packages]    (compiled python code)
└───bin/
    ├─── ...
    └─── ...
    
jedi_build_2026.1/
├── lib/
│   ├──
│   └─── ...
└───bin/
    ├─── ...  
    └─── ...
    

🛠 Management & Automation (Makefile)

The project utilizes a centralized Makefile to handle the development lifecycle. This ensures that the source area remains clean and that builds are site-aware (HPC vs. Local).

⚙️ Configuration Variables

Set these variables at runtime to override defaults:

• SITE: Target environment (docker [default] or arunika).
• MSG: Custom commit message for releases.

---

🚀 Available Commands

Command	Description
make help	Displays the interactive help menu.
make sync	Pulls latest source. Handles SSH tunnel on elogin nodes.
make install	Builds and installs the package to the "Away" directory.
make clean	Surgically removes the current version and build artifacts.
make update	Sync → Clean → Install. The standard daily refresh.
make release	Test → Bump Version → Archive. Production push to Git.
make test	Runs the aiesda-dev-cycle-test.sh suite.

---

📖 Usage Examples

1. Daily Development Sync

To synchronize your local environment with the latest remote changes:

make update SITE=arunika

🚦 Quick Start

import xarray
from aidaconf import ModelPassport

# Load a raw forecast file
ds = xarray.open_dataset("pangu_forecast.nc")

# Identify and Verify via Passport
interface = ModelPassport.identify(ds)

# Standardize for JEDI
standard_ds = interface.prepare_state(ds)

🛠 Adding a New Model to the Registry

To register a new model, update the MODEL_REGISTRY in aidadic.py:

"new_model_name": {
    "interface_class": "ailib.NewModelInterface",
    "required_vars": ["t", "q", "u", "v"],
    "horizontal_res": 0.1,
    "vertical_levels": "standard_grid_key",
    "allow_nans": False,
    "mapping": {"air_temperature": "t", ...}
}

🗺 Documentation Navigation

🏠 Home

🏗️ Architecture

🪜 Development Roadmap

🛠 Contribution Guide