refactor: consolidate tooling, pipeline blueprint, and asset alignment

[jdsika]

Summary

Full consolidation of the hd-map-asset-example repository: tooling, pipeline, generated asset structure, and submodule alignment. The sl-5-8-asset-tools pipeline has been overhauled with OMB-backed validation, structured logging, CID-named archives, Gaia-X metadata enrichment, and an opt-in deterministic mode.

Root Repository Changes

Build System & Tooling

• Add Makefile as central command center (setup, generate, validate, clean, help)
• Add pre-commit hooks with SHACL/JSON-LD validation (replaces black/isort/flake8)
• Remove orphan .flake8 and unused pyproject.toml
• Add .gitattributes enforcing LF line endings + LFS tracking rules
• Align make help with debug logging and deterministic mode hints

Asset Structure (EVES-003)

• Remove hand-crafted asset/ directory (replaced by pipeline-generated output)
• Add generated/input/ with input_manifest.json (JSON-LD), LFS-tracked simulation data, media, docs, LICENSE
• Pipeline output goes to generated/output/ (gitignored)

Submodules

• Consolidate into submodules/ (sl-5-8-asset-tools, EVES)
• Remove stale ontology-management-base root pointer (nested inside sl-5-8)
• EVES submodule switched to HTTPS

CI/CD

• Rewrite release.yml: Python/Make-based pipeline replaces Node.js/zip
• Uses make setup && make generate && make validate, uploads CID-named .zip via softprops/action-gh-release@v2

Documentation

• Rewrite README.md with setup/usage/structure docs
• Add .github/copilot-instructions.md

---

sl-5-8-asset-tools Submodule Changes (30 files, +1515 -285)

Validation Overhaul

• Replace legacy RDF/SHACL validation with OMB-backed offline loading (utils/rdf.py)
• Simplify jsonLD_validator to a thin OMB wrapper (from ~200 lines to ~50)
• Fix recursive SHACL dependency loading in jsonLD_creator (BFS over imports, filtered to ENVITED-X/Gaia-X namespaces)
• Add split JSON syntax validation stages: extractor output + structure output checked separately
• Fix make validate to auto-discover and validate generated examples (was running OMB test catalog)

Pipeline Reporting & Logging

• New utils/pipeline_reporting.py (~530 lines): stage-oriented output with [01/15] START/OK/FAIL formatting, tool-specific summarizers for OMB validation, QC results, GeoJSON conversion
• New utils/log_config.py: SL58_LOG_MODE=debug for verbose subprocess diagnostics (command lines, stdout/stderr, tracebacks)
• Refactored utils/subprocess.py: CommandResult/CommandError types, renamed log_output parameter
• ASCII-only output throughout

CID-Named Archives

• New utils/cid.py: compute CIDv1 (base32, raw, SHA-256) from file content
• Archive output renamed from asset.zip to <CID>.zip in the example directory (one level up from the asset folder)
• .gitignore updated with examples/**/*.zip

Gaia-X Metadata Enrichment

• New meta_data_extractor/gaiax.py: extracts license, copyright owner, and resource policy from sibling LICENSE files
• Wired into both XODR and XOSC extractors
• Generates real manifest UUID (replaces placeholder did:web:registry.gaia-x.eu:Manifest:uuid)

Deterministic Mode (Opt-in)

• SL58_DETERMINISTIC=1 enables reproducible output: same input → same UUIDs, timestamps, ZIP bytes, CID
• utils/ids.py: UUID5 seeded by SL58_INPUT_HASH (content hash of all input files) with counter for intra-process uniqueness
• structure_creator/main.py: SL58_SOURCE_MTIME for generated file timestamps
• create_zip(): sorted file iteration + fixed ZIP entry timestamps
• Default mode: random UUID4 + real timestamps (each generation is a unique asset instance)

QC Checker Fixes

• Pin qc-opendrive to fork fixing crash on roads without <link> element (upstream PR Handle missing optional road links in contact-point check asam-ev/qc-opendrive#139)
• Improved XQAR parsing and internal error detection in qualitychecker_caller

Other Fixes

• Fix GeoJSON output formatting (indent=2, was compact one-liners up to 5MB)
• Fix platform compatibility: os.sep in display paths, quoted $(MAKE) for Windows
• Fix non-ASCII characters in log output
• Harmonize Makefile: remove run alias, align help text
• Add .gitattributes enforcing LF line endings
• Update OMB submodule pointer (3 upstream fixes)

---

Diff Stats (vs main)

Root repository

 .gitattributes              |    7 +
 .github/copilot-instructions.md |   56 +
 .github/workflows/release.yml  |   48 +-
 .gitignore                  |    3 +
 .gitmodules                 |   10 +-
 .pre-commit-config.yaml     |   46 +-
 Makefile                    |  191 +
 README.md                   |  404 +-
 46 files changed, 1248 insertions(+), 11745 deletions(-)

sl-5-8-asset-tools submodule (expanded)

 .gitattributes                   |   2 +
 .gitignore                       |   3 +-
 Makefile                         | 116 +++--
 asset_extraction/main.py        | 165 ++++++-
 configs/process.json             |  11 +-
 jsonLD_creator/main.py           |  65 ++-
 jsonLD_validator/main.py         |  96 ++--
 meta_data_extractor/gaiax.py     | 127 +++++
 qualitychecker_caller/main.py    | 148 ++++--
 structure_creator/main.py        |  43 +-
 utils/cid.py                     |  15 +
 utils/ids.py                     |  20 +-
 utils/log_config.py              | 118 ++++-
 utils/pipeline_reporting.py      | 529 +++++++++++++++++++++
 utils/rdf.py                     | 124 ++++-
 utils/subprocess.py              |  86 +++-
 30 files changed, 1515 insertions(+), 285 deletions(-)

Commits

• 8d38c32 feat: overhaul asset pipeline — OMB validation, structured logging, deterministic mode
• 542c469 chore: update sl-5-8-asset-tools submodule pointer
• 74f6c09 chore: remove orphan .flake8 (no Python files in root repo)
• f6065b3 fix: EVES submodule to HTTPS, validate all generated assets
• 0692fbd refactor: simplify Makefile — remove duplicated logic, drop make asset zip
• 4c74dda refactor: cascade make setup to submodule Makefile
• 349d85a chore: remove dead code and update stale references
• a9f143e fix: update filenames to match pipeline output renames
• 89cb427 chore: update sl-5-8-asset-tools submodule pointer
• 0931390 docs: fix stale references in copilot-instructions and remove pyproject.toml
• 09dad83 chore: update sl-5-8-asset-tools submodule pointer
• 4e3fa1b chore: update submodule pointers
• d09e413 chore: update sl-5-8-asset-tools submodule, skip validate when no asset
• 77b3f39 chore: update sl-5-8-asset-tools submodule pointer
• 808b370 chore: remove hand-crafted asset/, update submodule pointers
• fe0e4d4 docs: clarify internet only needed for geocoding, not SHACL/ontologies
• c0a0273 feat: add opendriveconverter as submodule for Python re-implementation
• 9974692 chore: update sl-5-8-asset-tools submodule (filename + bjson fixes)
• 5247dc1 feat: track generated/input blueprint with LFS, add 3d_preview assets
• 0952e3f feat: input_manifest.json pipeline, EVES submodule, cross-platform overhaul
• 3ad2d8b refactor: consolidate submodules and Makefile as central command center

Known Follow-ups

• OMB Makefile quoting fixes need a separate upstream PR to ASCS-eV/ontology-management-base
• QC fork pin should be updated to @main once upstream PR asam-ev/qc-opendrive#139 is merged