DLT vs Workflows: Who Owns the Table?

Teams new to Databricks often ask which one to standardise on: Delta Live Tables (now part of Lakeflow as Declarative Pipelines) or Workflows (Lakeflow Jobs). It is the wrong question. They solve different problems, and the clean way to choose is to ask who owns the table.

DLT owns every table it declares. Its schema, data quality rules, retries, dependencies and backfill behaviour live inside the pipeline definition. Workflows owns nothing. It runs tasks in an order, with retries and parameters, and does not care what those tasks produce.

Use DLT when the table is the product

If a person or a system downstream reads the output, DLT should usually own it. That covers most of the medallion architecture:

• Bronze to silver to gold chains. You declare each table as a query over the previous one, and DLT works out the dependency graph and the order.
• Change data capture. APPLY CHANGES INTO handles inserts, updates, deletes and out-of-order events, and gives you SCD type 1 or type 2 without hand-written MERGE logic.
• Data quality. Expectations declare what valid data looks like and what to do when a row fails: keep it and record it, drop it, or stop the pipeline.
• Streaming and batch together. The same pipeline can read a stream into bronze and maintain batch-style aggregates in gold.

import dlt
 
@dlt.table(comment="Customers, cleaned and deduplicated")
@dlt.expect_or_drop("valid_id", "customer_id IS NOT NULL")
@dlt.expect("valid_email", "email LIKE '%@%'")
def silver_customers():
    return dlt.read_stream("bronze_customers").dropDuplicates(["customer_id"])

The expectations do two jobs. They protect consumers, and they produce metrics in the pipeline event log, so you can chart how many rows failed each rule per run. That chart is the most useful data quality dashboard most teams will ever build.

Use Workflows when the job is the product

Workflows is the orchestrator. Use it when the value is in running steps, not in maintaining a table:

• ML training and batch scoring that read gold tables and write models or predictions.
• Exports to external systems: files to a partner, rows to a CRM, a message to a queue.
• Conditional logic and fan-out: if/else tasks, for_each over a list of sources, retries on one branch only.
• Hand-offs between tools, such as waiting for an Azure Data Factory copy to land before transformation starts.

They compose

The two are not alternatives. A Workflow can run a DLT pipeline as one task in a larger graph:

This is the shape we recommend for most platforms: DLT owns the tables, Workflows owns the schedule and everything around it.

A decision checklist

Ask these in order:

1. Will someone query the output as a table? Yes: DLT.
2. Does the output need quality rules that block bad data? Yes: DLT.
3. Is it CDC from a source system? Yes: DLT with APPLY CHANGES INTO.
4. Is it orchestration, side effects or branching? Yes: Workflows.
5. Does it need to run arbitrary code against external systems? Yes: Workflows task.

Costs and operations

A few practical notes from running both:

• Use serverless or job compute, never all-purpose clusters, for scheduled work. All-purpose compute is for people, not pipelines.
• Triggered DLT pipelines are usually cheaper than continuous ones. Run continuous mode only where latency truly matters.
• Keep pipeline definitions in Databricks Asset Bundles, so DLT pipelines and Workflows are deployed together, with the same variables per environment.
• Alert on expectation failures, not just on pipeline failures. A pipeline that succeeds while dropping 30% of rows is a bigger problem than one that fails loudly.

The short version

If it is a table people rely on, let DLT own it. If it is a sequence of steps, let Workflows run it. And let Workflows run DLT when you need both.