lumen / data-platform / observabilityall systems operational

Lumen data platform

Fictional company's internal data mesh on an open-source lakehouse · 5 domains ·

SAMPLE DATA · ILLUSTRATIVE BUILD BY WOODFROG

Lineage graph

source → bronze → silver → gold → product → consumer

Every table, data product and consumer, linked by what reads from what. Hover a node to trace its full upstream and downstream path; red nodes have a failing check today.

Pipeline runs

LAST 24 HOURS · UTC

SucceededRetriedFailedRunning

Data quality tests

LAST 7 DAYS

Test outcomes by quality dimension. Timeliness carries most failures, driven by the late ERP extract.

PassWarnFail

Freshness SLA

DATA PRODUCT × DAY · LAST 21 DAYS

Each cell is a day's worst lag against that product's SLA. Green met, amber within 2x, red breached.

Compute cost

Spend by domain and workload (month to date). Hover a block for the cost per query.

Adoption

WEEKLY ACTIVE CONSUMERS · 26 WEEKS

Distinct people and services querying data products each week, stacked by domain. Spike in week 18 is the self-serve BI launch.

Incidents

LAST 90 DAYS · TIME TO RESOLVE

Each incident from detection to resolution, sized by severity. Dashed line is the rolling 30-day mean time to resolve.

Sev 1Sev 2Sev 3

Data product catalogue

CLICK HEADERS TO SORT

Demonstration build. Lumen and every figure are synthetic, modelled on a mid-size company running an open-source lakehouse (Iceberg tables, Spark and Trino compute, dbt transformations, Airflow orchestration). Built by Woodfrog: column-level lineage parsed from query logs, dbt and Great Expectations test results, warehouse billing exports and catalogue metadata, joined into one observability model.