Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
3ad2d8b
refactor: consolidate submodules and Makefile as central command center
jdsika Mar 9, 2026
0952e3f
feat: input_manifest.json pipeline, EVES submodule, cross-platform ov…
jdsika Mar 10, 2026
5247dc1
feat: track generated/input blueprint with LFS, add 3d_preview assets
jdsika Mar 10, 2026
9974692
chore: update sl-5-8-asset-tools submodule (filename + bjson fixes)
jdsika Mar 10, 2026
c0a0273
feat: add opendriveconverter as submodule for Python re-implementation
jdsika Mar 10, 2026
fe0e4d4
docs: clarify internet only needed for geocoding, not SHACL/ontologies
jdsika Mar 10, 2026
808b370
chore: remove hand-crafted asset/, update submodule pointers
jdsika Mar 10, 2026
77b3f39
chore: update sl-5-8-asset-tools submodule pointer
jdsika Mar 10, 2026
d09e413
chore: update sl-5-8-asset-tools submodule, skip validate when no asset
jdsika Mar 11, 2026
4e3fa1b
chore: update submodule pointers
jdsika Mar 11, 2026
09dad83
chore: update sl-5-8-asset-tools submodule pointer
jdsika Mar 11, 2026
0931390
docs: fix stale references in copilot-instructions and remove pyproje…
jdsika Mar 11, 2026
89cb427
chore: update sl-5-8-asset-tools submodule pointer
jdsika Mar 11, 2026
a9f143e
fix: update filenames to match pipeline output renames
jdsika Mar 11, 2026
349d85a
chore: remove dead code and update stale references
jdsika Mar 11, 2026
4c74dda
refactor: cascade make setup to submodule Makefile
jdsika Mar 11, 2026
0692fbd
refactor: simplify Makefile — remove duplicated logic, drop make asse…
jdsika Mar 11, 2026
f6065b3
fix: EVES submodule to HTTPS, validate all generated assets
jdsika Mar 11, 2026
74f6c09
chore: remove orphan .flake8 (no Python files in root repo)
jdsika Mar 11, 2026
542c469
chore: update sl-5-8-asset-tools submodule pointer
jdsika Mar 11, 2026
dd0492f
feat: overhaul asset pipeline — OMB validation, structured logging, d…
jdsika Mar 13, 2026
eb4da46
chore: update sl-5-8-asset-tools submodule (CI fix)
jdsika Mar 13, 2026
bdc0474
chore: update sl-5-8-asset-tools submodule (CI + example inputs)
jdsika Mar 13, 2026
e76b05e
refactor: subcommand groups, wizard docs, and clean up generated inputs
jdsika Mar 13, 2026
d91143b
feat: delegate generate to sl-5-8 via INPUT_DIR, add prereqs + docs
jdsika Mar 16, 2026
d08f241
docs: audit fixes — remove ghost setup qc, fix FAQ, fix double setup
jdsika Mar 16, 2026
d5e6a51
refactor: remove 'generate validate' alias, use 'make validate' only
jdsika Mar 16, 2026
9490584
docs: document EVES submodule, fix stale Known Limitations
jdsika Mar 16, 2026
a0cb0aa
fix: Makefile cross-platform and consistency improvements
jdsika Mar 16, 2026
72a3d86
chore: update sl-5-8-asset-tools submodule
jdsika Mar 16, 2026
08523b7
docs: document asset workflow, two creation paths, and Gaia-X role
jdsika Mar 16, 2026
21b4c00
chore: update sl-5-8-asset-tools submodule
jdsika Mar 16, 2026
5830dff
chore: add markdown linting and formatting
jdsika Mar 16, 2026
ef3227c
chore: update sl-5-8-asset-tools submodule (CI fix)
jdsika Mar 16, 2026
afbcb8b
chore: update sl-5-8-asset-tools submodule (CI fix v2)
jdsika Mar 16, 2026
48061d6
chore: update sl-5-8-asset-tools submodule (wizard API fix)
jdsika Mar 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .flake8

This file was deleted.

7 changes: 7 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Enforce LF line endings for all text files
* text=auto eol=lf

# Git LFS - track large binary assets in generated/input/
generated/input/**/*.png filter=lfs diff=lfs merge=lfs -text
generated/input/**/*.pdf filter=lfs diff=lfs merge=lfs -text
generated/input/**/*.xodr filter=lfs diff=lfs merge=lfs -text
82 changes: 82 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# HD-Map Asset Example — Copilot Instructions

## Project Overview

This is a **reference asset repository** for onboarding HD-Map simulation data into the ENVITED-X Dataspace. Users stage input files in `generated/input/`, then `make generate` runs the sl-5-8 pipeline to produce a complete, EVES-003-conformant asset in `generated/output/`.

Assets follow the [EVES-003](https://ascs-ev.github.io/EVES/EVES-003/eves-003.html) specification.

## Asset Creation Flow

### Input → Output

Users provide an OpenDRIVE `.xodr` file and an `input_manifest.json` (plus optional images, docs, LICENSE) in `generated/input/`. The pipeline auto-extracts metadata, runs quality checks, generates GeoJSON previews, and packages everything into an EVES-003 asset in `generated/output/`.

### Two Paths

1. **Automated (default):** `make generate` — pipeline auto-extracts all metadata from the `.xodr` and input manifest. No user interaction needed.
2. **Wizard-assisted (optional):** `make wizard` starts a SHACL-driven web UI (Podman containers) at `http://localhost:4200` for interactively enriching metadata. Users can also edit `input_manifest.json` and `metadata/hdmap.json` by hand as an alternative to the wizard.

### Gaia-X Integration

The pipeline adds [Gaia-X Trust Framework](https://gaia-x.eu/) vocabulary to every asset (`gx:name`, `gx:license`, `gx:copyrightOwnedBy`, `gx:resourcePolicy`) inside `metadata/hdmap.json`. These properties live in closed GX-compliant nodes, while domain-specific properties (road types, lane counts, format version) go in open ENVITED-X wrapper shapes. The wizard's SHACL forms reflect both layers. Users don't need to understand Gaia-X — the pipeline handles compliance automatically.

### Pipeline Stages (18 total)

`meta_data_extractor` → `jsonLD_creator` → `shacl_combiner` → `wizard_caller` (disabled by default) → `jsonLD_validator` → `qualitychecker_caller` (ASAM + OpenMSL) → `xodr_routing_creator` → `xodr_to_geojson_caller` → `asset_reducer` → `structure_creator` → `manifest jsonLD_creator` → `jsonLD_validator`

Key config files are in `submodules/sl-5-8-asset-tools/configs/`. Each stage can be enabled/disabled via `process.json`.

## Repository Structure

- `generated/input/` — Staged pipeline inputs (manifest, `.xodr` files, media, docs)
- `generated/output/` — Pipeline output: complete EVES-003 asset ready for validation and release
- `submodules/sl-5-8-asset-tools/` — Asset creation and processing tools (git submodule from [openMSL/sl-5-8-asset-tools](https://github.com/openMSL/sl-5-8-asset-tools))
- `submodules/ontology-management-base/` — Nested submodule: SHACL shapes, OWL ontologies, JSON-LD contexts, and Python validation tools (from [ASCS-eV/ontology-management-base](https://github.com/ASCS-eV/ontology-management-base))

## Setup and Validation

```bash
make setup
make validate
```

All commands are exposed via `make` targets -- run `make help` for the full list.

## Linting (pre-commit hooks)

Configured in `.pre-commit-config.yaml` — all hooks delegate to `make` targets.

## Key Conventions

### Asset Structure (EVES-003)

Every asset must contain: `simulation-data/`, `metadata/`, `media/`, `documentation/`, and a `manifest.json` at the root. Optional: `validation-reports/`.

### JSON-LD Metadata

- `manifest.json` — Content registry linking all asset files with access roles (`isOwner`, `isRegistered`, `isPublic`) and categories (`isSimulationData`, `isMetadata`, `isMedia`, etc.)
- `metadata/hdmap.json` — Domain-specific metadata (format, content, quantity, quality, data source, georeference) conforming to the HD-Map SHACL shapes from ontology-management-base

Both files use typed `@value`/`@type` pairs for literals and reference ontologies via `@context` prefixes like `hdmap:`, `manifest:`, `envited-x:`, `georeference:`, `gx:`.

### Submodules

Two direct submodules:

- `submodules/sl-5-8-asset-tools` — Asset creation and processing tools (which contains `ontology-management-base` as a nested submodule)
- `submodules/EVES` — The [EVES specification](https://ascs-ev.github.io/EVES/EVES-003/eves-003.html) that defines the structure and requirements for Simulation Assets in the ENVITED-X Dataspace

After cloning, initialize with:

```bash
make setup
```

### Commits

This project uses [DCO sign-off](CONTRIBUTING.md). All commits require `Signed-off-by` — use `git commit -s`. Do **not** add `Co-authored-by: Copilot` trailers.

## Release Workflow

The GitHub Actions workflow (`.github/workflows/release.yml`) triggers on version tags (`v*.*.*`), runs `make setup && make generate && make validate`, and uploads the pipeline-generated `asset.zip` as a GitHub release artifact.
48 changes: 18 additions & 30 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,48 +5,36 @@ on:
tags:
- 'v*.*.*' # Triggers the workflow on version tags

permissions:
contents: write

jobs:
release:
runs-on: ubuntu-latest

steps:
- name: Checkout repository (+ download lfs dependencies)
- name: Checkout repository
uses: actions/checkout@v4
with:
lfs: true
submodules: recursive

- name: Checkout LFS objects
run: git lfs checkout

- name: Set up Node.js
uses: actions/setup-node@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
node-version: '20'

- name: Install zip
run: sudo apt-get install -y zip
python-version: '3.12'

- name: Create zip file
- name: Setup, generate, and validate
run: |
cd asset
zip -r ../Testfeld_Niedersachsen_ALKS_xodr_sample.zip ./*

- name: Create Release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ github.ref }}
release_name: Release ${{ github.ref }}
draft: false
prerelease: false

- name: Upload Release Asset
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
make setup
make generate
make validate

- name: Create Release and Upload Asset
uses: softprops/action-gh-release@v2
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./Testfeld_Niedersachsen_ALKS_xodr_sample.zip
asset_name: Testfeld_Niedersachsen_ALKS_xodr_sample.zip
asset_content_type: application/zip
files: "generated/output/**/*.zip"
generate_release_notes: true
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
/.venv/
output.log
*.zip
/generated/output/
!/generated/input/
10 changes: 7 additions & 3 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
[submodule "ontology-management-base"]
path = ontology-management-base
url = https://github.com/GAIA-X4PLC-AAD/ontology-management-base.git
[submodule "submodules/sl-5-8-asset-tools"]
path = submodules/sl-5-8-asset-tools
url = https://github.com/openMSL/sl-5-8-asset-tools.git
branch = main
[submodule "submodules/EVES"]
path = submodules/EVES
url = https://github.com/ASCS-eV/EVES.git
27 changes: 27 additions & 0 deletions .markdownlint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Markdownlint configuration
# https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md
default: true

# Allow long lines — tables and descriptive prose need room
MD013:
line_length: 300
code_blocks: false
tables: false

# Allow inline HTML for collapsible sections and formatting
MD033:
allowed_elements:
- a
- br
- details
- img
- noscript
- strong
- summary
- sup

# Disable table column style — compact pipes are fine
MD060: false

# Allow bold text as step labels (e.g., **Step 1 — ...**)
MD036: false
50 changes: 10 additions & 40 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,45 +2,15 @@
repos:
- repo: local
hooks:
- id: black
name: black
entry: black
- id: validate
name: validate asset JSON-LD
entry: make validate
language: system
types: [python]
args: ["--config=pyproject.toml"]
pass_filenames: true
exclude: ontology-management-base/

- id: isort
name: isort
entry: isort
pass_filenames: false
always_run: true
- id: lint-md
name: lint markdown
entry: make lint-md
language: system
types: [python]
args: ["--settings=pyproject.toml"]
pass_filenames: true
exclude: ontology-management-base/

- id: flake8
name: flake8
entry: flake8
language: system
types: [python]
args: ["--config=.flake8"]
pass_filenames: true
exclude: ontology-management-base/

- id: jsonld-lint
name: JSON-LD Linter
entry: python ontology-management-base/src/jsonld_lint.py
language: system
types: [json]
files: \.json$
exclude: ontology-management-base/

- id: turtle-lint
name: Turtle Linter
entry: python ontology-management-base/src/turtle_lint.py
language: system
types: [text]
files: \.(ttl)$
exclude: ontology-management-base/
pass_filenames: false
types: [markdown]
Loading