In a workspace

Use these steps when you have a Databricks workspace. The paths are the same demos and opt-in harnesses already in the repository — ordered for operators.

Shared setup

1. Clone or sync the repo into the workspace (Git folder or databricks workspace import-dir).

2. Build a wheel and upload it where the notebooks can install it:

   uv build
   # example destinations used by the demos:
   # dbfs:/FileStore/tablespec-demo/   or a UC volume path

3. Prefer a single-user cluster on a recent LTS runtime (Python ≥ 3.12 for current wheels; DBR 16.4+ is a good default).

1. Bootstrap existing tables (Path A)

When tables already live in Unity Catalog / Spark:

from tablespec import bootstrap_from_tables

artifacts = bootstrap_from_tables(
    spark,
    ["catalog.schema.member", "catalog.schema.claims"],
    out_dir="/tmp/tablespec-bootstrap",
    profile=True,
    dialect="databricks",  # Spark-family SQL; accepted public dialect
)

bootstrap_from_tables reflects schema into UMF, optionally enriches validation from the native profiler, compiles the committed artifact tree, and returns the manifest. See also Getting Started for Path B (authored specs) without Spark.

2. Northwind — JDBC discovery end to end

Goal: point tablespec at a SQL Server database; get one validated UMF per table, Excel workbooks, sample data, typed land, staged validation.

Notebooks: notebooks/northwind-demo/

Requirements: single-node cluster so localhost JDBC works; single-user access mode. Credentials in UMF are secret refs only.

Local stand-in (no workspace): Docker-gated uv run pytest tests/integration/test_northwind_e2e.py.

3. Kaggle-style flat file — delimited onboarding

Goal: stage a CSV on a UC volume, land all-STRING raw, profile, author UMF, export a workbook, compile artifacts, run staged validation.

Notebooks: notebooks/kaggle-demo/

Default dataset is NYC Airbnb open data; widgets swap URL/path for other CSVs.

4. SEC 10-K — embeddings and JSON facts

Goal: govern a corpus table with EMBEDDING(1024) and an XBRL facts table via source: kind: json, with dimensionality validation on the embedding column.

Notebooks: notebooks/sec-10k-demo/

Use embedding_mode=fake when the workspace has no Foundation Model API access. Specs never embed endpoints or credentials.

Example specs: examples/sec10k_corpus.yaml, examples/sec10k_companyfacts.yaml.

5. Opt-in serverless / workspace conformance

Goal: when credentials are present, prove dbt/LDP deploy + read-back parity against the shared Spark oracle corpus.

Default make test never requires a workspace. The lane is opt-in:

export DATABRICKS_HOST=https://<workspace>
export DATABRICKS_HTTP_PATH=/sql/1.0/warehouses/<id>
export DATABRICKS_TOKEN=<pat>
# adapters: dbt-databricks, databricks-sdk, databricks-sql-connector

uv run pytest -m databricks_e2e -q

Without credentials, those tests skip with a named reason — they do not silently pass. The unit gate for the skip path is tests/unit/test_databricks_e2e_gate.py.

6. Deploy the profiling app

Separate checklist: Deploy the app.

Checkpoints

Next