The Lakehouse, Live

Iceberg on Ceph, Nessie, Trino, dbt and OpenMetadata 2.0 as deployed 2026-09-30 · Author: George Ezejiofor

The Lakehouse, Live Iceberg on Ceph, Nessie, Trino, dbt and OpenMetadata 2.0 as deployed 2026-09-30 · Author: George Ezejiofor People · browser · Kubeflow notebooks · Architecture component People browser · Kubeflow notebooks Keycloak · realm openmetadata · Architecture component · Google · GitHub Keycloak realm openmetadata Google · GitHub OpenMetadata 2.0.2 · catalog · lineage · tests · governance · OIDC confidential OpenMetadata 2.0.2 catalog · lineage · tests OIDC confidential OpenSearch 3.4.0 · 3 nodes · db pool · governance · operator 2.8.0 OpenSearch 3.4.0 3 nodes · db pool operator 2.8.0 KServe · football-match-outcome · Architecture component KServe football-match-outcome OM ingestion CronJobs · om-ingest-trino · om-ingest-dbt · governance · 10:00 · 10:15 OM ingestion CronJobs om-ingest-trino · om-ingest-dbt 10:00 · 10:15 dbt-build CronJob · dbt-trino · audit branch · lakehouse core · 09:30 dbt-build CronJob dbt-trino · audit branch 09:30 Trino 451 · Stackable · coordinator only · lakehouse core · autoTls :8443 Trino 451 Stackable · coordinator only autoTls :8443 MLflow · tracking · model registry · Architecture component MLflow tracking · model registry dbt-artifacts · CephFS PVC · RWX · governance · manifest · catalog dbt-artifacts CephFS PVC · RWX manifest · catalog Project Nessie · Iceberg REST · Git-for-data · lakehouse core · 2 replicas Project Nessie Iceberg REST · Git-for-data 2 replicas Ceph RGW · s3.georgehomelab.com · TLS · Architecture component · mlops-data-lake Ceph RGW s3.georgehomelab.com · TLS mlops-data-lake CloudNativePG · nessie · mlflow · openmetadata · Architecture component · 3 instances CloudNativePG nessie · mlflow · openmetadata 3 instances log in OIDC index entities + lineage read crawl dbt build publish Iceberg REST warehouse nessie_db model lakehouse core governance Legend Backend Database Cloud Security External

Live since 2026-09-30

  • • OpenMetadata 2.0.2 on OpenSearch 3.4.0
  • • Trino creates Iceberg tables (own S3 keys)
  • • dbt builds on Nessie branch dbt-audit, merges on green

Secrets

  • • Terraform → Vault → VSO → Secret
  • • nothing credential-shaped in Git

Not live yet

  • • Hermes data copilot (DGX Spark)
  • • Trino workers (coordinator only)
  • • Spark ETL on a schedule