Databricks Platform Substitution Reference

If you are adopting HELIX data artifacts (data-prd, data-architecture, data-quality-expectations) on a platform other than Databricks, substitute the platform-specific terms below. The artifact templates and prompts stay the same; only the named platform features change.

Catalog / Governance

Databricks Concept	Snowflake Equivalent	BigQuery Equivalent	On-Prem / Other
Unity Catalog (UC) hierarchy (metastore → catalog → schema → volume/table)	Database + Schema + Table	Project + Dataset + Table	Database + Schema
UC fine-grained access (row filters, column masks)	Row-access policies + masking policies	Authorized views + column-level security	Database views, RLS extensions
UC lineage	Object dependencies + Account Usage	Data lineage in Dataplex	Manual lineage in dbt/Marquez

Ingestion / Streaming

Databricks Concept	Snowflake Equivalent	BigQuery Equivalent	On-Prem / Other
Auto Loader (incremental cloud-file ingestion)	Snowpipe or native connectors	Dataflow, BigQuery Connector Hub, Streaming Inserts	Apache NiFi, Kafka connectors
Streaming Tables (declarative streaming with EXPECT)	Stream-triggered materialized views + native checks	Dataflow with Beam assertions	Apache Flink with custom state management
Spark Structured Streaming	Snowflake streams + tasks	Dataflow	Apache Spark / Flink

Orchestration / Pipelines

Databricks Concept	Snowflake Equivalent	BigQuery Equivalent	On-Prem / Other
Databricks Workflows / Jobs	Snowflake Tasks	Cloud Composer (Airflow) or Cloud Workflows	Apache Airflow, Dagster, dbt Cloud
Delta Live Tables / SDP pipeline orchestration	Dynamic Tables	Dataform	dbt, Dagster assets

Quality / Contracts

Databricks Concept	Snowflake Equivalent	BigQuery Equivalent	On-Prem / Other
SDP `EXPECT ... ON VIOLATION ...`	Data Quality checks + Task error handling	BigQuery Data Quality API + Cloud Workflows	dbt tests, Great Expectations, custom assertions
SDP Genie test generation	dbt auto-generate tests from table samples	BigQuery Data Catalog insights	dbt, custom metadata scanning
Lakeview dashboards (quality monitoring)	Snowflake Dashboards	Looker, Data Studio	Grafana, custom dashboards

Storage / Formats

Databricks Concept	Snowflake Equivalent	BigQuery Equivalent	On-Prem / Other
Delta Lake format	Iceberg or proprietary formats	Native BigQuery tables	Apache Parquet, Iceberg, Hudi
Medallion layers (Bronze / Silver / Gold)	Same pattern applies universally	Same pattern applies universally	Same pattern applies universally

Compute / Cost

Databricks Concept	Snowflake Equivalent	BigQuery Equivalent	On-Prem / Other
DBU budget estimation	Credit consumption	On-demand or flat-rate pricing	Compute resource allocation
All-purpose / Jobs / Serverless SQL compute tiers	Warehouses (XS through 6XL)	On-demand vs reservation slots	Cluster sizing

Innsigle seal: model-primary by HELIX

The signature covers the markdown source of this page, not these HTML bytes. This page quotes that seal; verify it against the source file.

Composition: model-primary
Issuer: HELIX helix
Signing key: ed25519:b0865d76d834a52c48506414d16f4e5a (build key)
Signed source: research/databricks-platform-substitution.md
Signed: 2026-09-23T14:11:58Z
Content digest: sha256:31d17865…17d3f765

This build key is endorsed by the human key for build signing; the signature is not a detector and not a truth guarantee.

Raw attestation JSON

{
  "payload": {
    "innsigle": "1",
    "type": "https://innsigle.dev/claim/colophon/v1",
    "issued_at": "2026-09-23T14:11:58Z",
    "issuer": {
      "id": "helix",
      "name": "HELIX",
      "key_id": "ed25519:b0865d76d834a52c48506414d16f4e5a",
      "key_url": "https://documentdrivendx.github.io/helix/.well-known/innsigle/keys.json"
    },
    "subjects": [
      {
        "uri": "https://documentdrivendx.github.io/helix/research/databricks-platform-substitution/",
        "digest": {
          "alg": "sha256",
          "value": "31d178656c59032c7887e8edeea78e62aabf0ee0f2a9cb58fe8e7f1917d3f765"
        }
      }
    ],
    "colophon": {
      "schema_version": "1",
      "composition": "model-primary",
      "ingredients": [
        {
          "kind": "model",
          "name": "Claude",
          "role": "draft"
        },
        {
          "kind": "tool",
          "name": "sloptimizer",
          "role": "rewrite"
        },
        {
          "kind": "human",
          "name": "operator",
          "role": "structure-edit"
        }
      ],
      "notes": null
    }
  },
  "payload_encoding": "json",
  "signatures": [
    {
      "key_id": "ed25519:b0865d76d834a52c48506414d16f4e5a",
      "alg": "ed25519",
      "sig": "qykP-5uP_QwkDg1KKKvMnVtUHVVTJvD2M9aDrUiS_vbwgGlrE1r5ZxsUSKIT03C2dEaoV3g5Btx1i0Emk2ssBg",
      "signed_at": "2026-09-23T14:11:58Z"
    }
  ]
}

attestation file · issuer keys