What does tablespec define?

tablespec defines the source-semantic ingested bronze contract: types, validation criteria, relationships, aliases, keys, and compiled runtime artifacts from one UMF.

Where does silver begin?

Silver begins after the source contract is complete, with cross-source conformance, survivorship, entity resolution, enrichment, and dimensional modeling.

Where bronze ends and silver begins

TABLESPEC / SOURCE CONTRACT

Definition of done for ingested bronze

tablespec is for data engineers and platform teams who need source data ready for downstream work. It defines a source-semantic ingested bronze contract: the source meaning is preserved, while types, validation, keys, and relationships are declared. Then it compiles one Universal Metadata Format (UMF) spec into SQL, dbt, Lakeflow, schema, validation, Excel review workbooks, and static guidebook artifacts.

Start with a UMF View the worked example

compile path review surface

source flat files, dumps, APIs records as delivered by another system

raw source records transport shape kept for audit and replay

ingested bronze typed, validated, keyed source meaning captured in Delta-compatible tables

silver conform, resolve, enrich business choices across one or more sources

UMFtables/claims/table.yaml

SQLclaims.ingest.sql

dbtmodels/claims.sql

Lakeflowpipeline.yml

GXsuite.json

Guidebookindex.html

types declared validation generated keys recorded relationships defined artifacts committed

EVALUATE

One source-table spec, every generated artifact

Define the source table once

Capture names, source types, nullability, keys, relationships, aliases, and provenance in Universal Metadata Format.

Compile committed artifacts

Generate the SQL DDL, ingest transform, dbt model, Lakeflow pipeline, JSON Schema, Great Expectations suite, Excel workbook, and guidebook from that spec.

Validate before silver starts

Run checks against raw source records and typed ingested tables before silver models add business rules.

WORKED EXAMPLE

Medical claims from UMF to reviewable artifacts

Start with a split-format UMF directory for `medical_claims`: one table-level YAML file plus one YAML file per column. tablespec validates that source-table contract, then emits raw-to-ingested SQL, a dbt project, a Great Expectations suite, and a guidebook from the same spec.

Open the example

terminal deterministic output

tablespec validate tables/
tablespec generate tables/medical_claims -f ingest > claims.ingest.sql
tablespec emit tables/ out/dbt --backend dbt --dialect databricks
tablespec validation-sync tables/medical_claims --out gx/
tablespec guidebook tables/ -o site/guidebook

DECIDE

Bronze records source meaning. Silver records business choices.

Question Ingested bronze Silver

Renames columns? No. Preserve source names. Yes, when conformance requires it.

Resolves duplicates across sources? No. Yes.

Defines business survivorship? No. Yes.

Feeds downstream systems? Yes. Typed, validated, keyed Delta-compatible artifacts. Consumes bronze and applies governed business decisions.

OPERATE

Compile the UMF contract. Review the diff.

Install tablespec, write one UMF source-table spec, generate runtime artifacts, then inspect the bronze boundary before silver work begins.

Get started Read raw, ingested, and silver