Docker Compose environments for developing modern data platform architectures using Kafka, Flink, Spark, Iceberg, OpenLineage, OpenMetadata, Pinot, ClickHouse, StarRocks + Kpow & Flex by Factor House

Tools for processing sql queries metadata.

NextGen DataMotion Lineage

Covid19 and Iowa Liquor Sales analysis at BigQuery using dbt, Airflow, Marquez, Google Cloud and other modern data stack tools

Simple python script for easy local airflow deployment with docker. Packed with additional components. Will be adding more going forward.

This Project aims to enhance Spark Lineage Generation in Open Metadata by creating Lineage edges from Table-to-Table with Column level lineage as well as Container to Table Lineage.

Testing a Docker deployment of Marquez and OpenLineage

Deploy MWAA with OpenLineage data lineage tracking using AWS CDK - Airflow 3.0.6, Marquez HA

dbt plugin that accelerates incident resolution with automated test failure correlation

Data Reliability & Governance Framework — quality controls, observability evidence, incident playbooks, and ISO-ready audit mapping.

Exploring how to surface dlt pipeline metadata via Airflow's OpenLineage integration.

Scan AWS Glue PySpark jobs in GitHub repositories to generate data lineage using AST analysis and LLMs.

DataCatalog to explore end-to-end lineage with Airbyte, Airflow, dbt, BigQuery

Incident correlation engine for data teams — connects test failures to job runs via OpenLineage

Accelerate Great Expectations incident resolution with automated correlation. Links validation failures to upstream data issues and provides direct navigation from incident to root cause.

Airflow plugin that accelerates incident resolution with automated pipeline correlation