Time Travel, Zero-Copy Clones, and Git-Style Branching

In the previous chapter you saw how Iceberg, Delta, and Hudi turn a folder of Parquet files on S3 into a real ACID table by writing a metadata layer alongside the data. That metadata layer — Iceberg's snapshot tree, Delta's transaction log, Hudi's timeline — is the foundation for everything in this chapter.

The thesis is simple. Snapshots are cheap. Once you commit to immutable data files plus a pointer-based metadata model, the operations that classical row-store databases find expensive (consistent snapshots, full table copies, branched experimentation) become a few KB of metadata writes. You are not duplicating gigabytes of rows; you are duplicating a pointer.

This is a categorical shift in what data engineering can look like. On a Postgres OLTP database, "give me a copy of the production database for QA" is a multi-hour pg_dump/pg_restore cycle, doubles your storage bill, and is stale the moment you finish. On an Iceberg lakehouse, the same operation is one SQL statement, completes in seconds, and costs nothing extra until QA starts writing.

Time travel: querying the past

Every table format that descends from the snapshot model gives you time travel. The exact syntax varies by engine, but the semantics are the same: name a past version (by snapshot id, version number, or timestamp), and the reader resolves the metadata for that version and reads the data files alive at that commit.

The picture above is the whole mental model. Why this works: data files are never mutated in place in a lakehouse table format. A new commit either adds files or marks old ones as removed (via a remove entry, not by deleting from S3). The set of files alive at snapshot T2 is just (files added up to T2) − (files removed up to T2). Because the underlying Parquet has not changed, querying T2 today gives bit-identical results to what a query at T2 would have returned then.

In Iceberg, time travel is exposed three ways:

-- By snapshot id (the most precise; survives clock skew, time zones)
SELECT * FROM orders VERSION AS OF 7283450273948574234;

-- By timestamp (resolves to the latest snapshot at or before that wall-clock time)
SELECT * FROM orders TIMESTAMP AS OF '2024-04-24 18:00:00';

-- By branch or tag name (covered later in this chapter)
SELECT * FROM orders.branch_dev;

Snowflake's Time Travel feature predates Iceberg by several years and uses the same conceptual model under the hood — a sequence of micro-partition snapshots, each with a retention window (1 day on Standard edition, up to 90 days on Enterprise). Snowflake exposes:

SELECT * FROM transactions AT (TIMESTAMP => '2024-04-24 23:59:59'::TIMESTAMP);
SELECT * FROM transactions BEFORE (STATEMENT => '01a2b3c4-...');  -- before a specific query
SELECT * FROM transactions AT (OFFSET => -60 * 5);                -- 5 minutes ago

Delta's syntax (Spark/Databricks):

SELECT * FROM transactions VERSION AS OF 47;
SELECT * FROM transactions TIMESTAMP AS OF '2024-04-24 23:59:59';

In Python with pyiceberg:

from pyiceberg.catalog import load_catalog
import datetime as dt

catalog = load_catalog("prod_glue")
table = catalog.load_table("warehouse.orders")

# All snapshots, newest first
for snap in sorted(table.snapshots(), key=lambda s: s.timestamp_ms, reverse=True)[:5]:
    print(snap.snapshot_id, dt.datetime.fromtimestamp(snap.timestamp_ms / 1000))

# Time-travel read by timestamp
target_ts = dt.datetime(2024, 4, 24, 23, 59, 59)
target_ms = int(target_ts.timestamp() * 1000)
old_snapshot = max(
    (s for s in table.snapshots() if s.timestamp_ms <= target_ms),
    key=lambda s: s.timestamp_ms,
)
df = table.scan(snapshot_id=old_snapshot.snapshot_id).to_pandas()
print(f"Recovered {len(df):,} rows from snapshot {old_snapshot.snapshot_id}")

What can you actually do with time travel?

Recover from accidental deletes and updates. Someone runs DELETE FROM transactions WHERE created_at > '2024-04-24' thinking it was a dev table — and it was production. Without time travel, you reach for last night's backup, restore to a side table, and copy the missing rows back. With time travel, you query the table as it was 5 minutes before the bad delete and re-insert the rows in one statement.

Reproduce model-training datasets. Three months after deploying ML model v3, performance degrades. You want to know exactly what data the model was trained on so you can debug. If you logged the snapshot id at training time, time travel reproduces the training set bit-for-bit, even if the table has been backfilled, schema-evolved, or compacted in the meantime.

Audit and compliance. "Show me the customer record for ID 4827 as it existed on 31 March 2024." For RBI- or SEBI-regulated workloads in India, time travel turns audit trails from a separate change-history table into a property of the table itself.

Zero-copy cloning: creating a new table that points to the same files

A clone is what happens when you take the time-travel idea and give the resulting view-of-the-past its own writeable identity. Snowflake calls it Zero-Copy Cloning; Databricks calls it Delta Shallow Clone; Iceberg achieves the same effect through CREATE TABLE ... AS SELECT over a snapshot, or via Nessie branches.

Snowflake syntax:

-- Clone the entire `analytics` schema (every table, view, stage) for QA:
CREATE SCHEMA analytics_qa CLONE analytics;

-- Clone a single table at a specific point in the past — clone + time travel combined:
CREATE TABLE orders_yesterday CLONE orders AT (TIMESTAMP => DATEADD(day, -1, CURRENT_TIMESTAMP()));

Delta shallow clone (Spark SQL):

CREATE TABLE prod.orders_dev SHALLOW CLONE prod.orders;
-- Or DEEP CLONE if you really want a full copy (rare; usually for cross-region disaster recovery)
CREATE TABLE backup.orders DEEP CLONE prod.orders;

Iceberg via Nessie or branches:

-- Create a branch from the current main snapshot (semantically a clone):
ALTER TABLE warehouse.orders CREATE BRANCH dev;
-- Now writes to `warehouse.orders.branch_dev` are isolated from main.

The economics matter. A 50 TB production gold table on S3 in ap-south-1 costs roughly USD 1,150 per month (50,000 * $0.023). A traditional CTAS copy doubles that to USD 2,300. A zero-copy clone costs nothing on day one. If the dev team writes 200 GB of new data over a month, the clone's incremental cost is just 200 * $0.023 = $4.60 — three orders of magnitude cheaper than the deep copy. Why this matters for Indian startups: the typical engineering org runs 3–5 non-production environments (dev, staging, perf, integration, training-data-snapshot). With zero-copy clones, you can give each environment a real production-fidelity dataset without multiplying your S3 bill.

What "shallow" clones do not protect against: if you run a VACUUM or expire_snapshots on the source table aggressively enough to delete files that the clone still references, the clone breaks. Iceberg and Delta both refuse to expire files referenced by any live snapshot or branch they know about, but they only know about objects in their metadata. A clone in a different catalog instance won't be visible to the source's vacuum. This is a footgun worth flagging in any operations runbook.

Git-style branching: the lakehouse as a versioned filesystem

Once you have snapshots and clones, the next logical step is to give branches first-class identities, the way git does for source code. Project Nessie — built by the Dremio team and now an open-source project — was the first catalog to do this end-to-end for Iceberg. Iceberg itself added native branches and tags in spec v2; Delta Lake has had similar work in progress.

The semantic shift is small but profound: a table is no longer a single sequence of snapshots, but a graph of snapshots, with named branches that can advance independently and merge back together.

The Nessie API exposes this directly:

# pseudocode using pynessie
from pynessie import init
client = init()

# Branch off main
client.create_branch("backfill_experiment", source_ref="main")

# Run the experimental pipeline against the branch — no impact on main
spark.sql("USE REFERENCE backfill_experiment IN nessie")
spark.sql("INSERT INTO warehouse.orders SELECT * FROM staging.cdc_backfill")

# Validate metrics on the branch
metrics = run_quality_suite("nessie.backfill_experiment.warehouse.orders")
if metrics["dedup_rate"] > 0.999 and metrics["row_count_delta"] < 0.05:
    client.merge("backfill_experiment", "main")
else:
    client.delete_branch("backfill_experiment")  # discard the experiment

What does this enable that simple cloning does not?

Cross-table atomic experiments. A clone is per-table. A Nessie branch is per-catalog — you can branch the entire warehouse, modify five tables together on the branch, and merge the whole change set as one logical commit. This matters for pipelines that touch dimensions and facts together; you cannot meaningfully clone "orders" without also cloning "customers" if the joins must stay consistent.

Reproducible pipeline runs. Tag a Nessie commit pipeline_run_2024_04_25_07_00. Six months later, point any query engine at that tag and the entire warehouse will look exactly as it did when that pipeline ran — every dimension table, every fact table, every reference dataset.

Safe schema migrations. Branch, run the migration on the branch, run downstream queries on the branch, verify metrics, then merge. If something breaks, drop the branch — main never moved.

A/B testing pipelines. Run pipeline v1 on main, v2 on branch_v2. Compute metrics on both branches in parallel. Merge whichever wins. Drop the loser.

Implementation: how snapshots, clones, and branches actually work

All three features rest on the same physical structure.

1. Every table has a sequence of snapshots (Iceberg) or versions (Snowflake, Delta). A snapshot/version is a small metadata object: a snapshot id, a timestamp, a parent snapshot id, a schema id, and a pointer to the manifest list (Iceberg) or transaction log entry (Delta) that enumerates the live data files.
2. Snapshots reference data files, which are immutable. Once a Parquet file is written, its bytes never change. The only way the file becomes invisible is for a future snapshot to record a remove action.
3. Time travel at snapshot id S is just: load the metadata for S, walk its manifests, read the resulting Parquet files. Costs one metadata read plus the actual data read.
4. Cloning is creating a new table whose initial snapshot's manifests point at the same data files as the source's snapshot. The catalog stores a separate table identity, but the underlying object-store files are shared. The first write to the clone allocates new files; the first delete on the clone records a remove in the clone's metadata only — the source's metadata is untouched.
5. Branches are named snapshot pointers. main is conventionally the "production" branch, but there is nothing magical about the name — it is just a row in the catalog mapping a string to a snapshot id. Advancing a branch is an atomic compare-and-swap of that pointer. Merging is computing the union of file sets between two branches and producing a new snapshot whose parents include both branch tips.

In Iceberg's branching and tagging model, every snapshot can have multiple named refs pointing at it. main is a built-in branch. Any other branch you create coexists in the same metadata tree. A snapshot that is referenced by any live branch or tag is considered alive and will not be expired.

The cost trade-off: snapshot accumulation and vacuum

The price you pay for cheap snapshots is that data files cannot be deleted from S3 until no snapshot references them. A naive Iceberg or Delta table that runs for a year without cleanup will accumulate gigabytes — sometimes terabytes — of metadata and dead data files.

The remedies:

Iceberg: expire_snapshots removes snapshots older than a retention threshold and physically deletes data files no longer referenced by any live snapshot.

table.expire_snapshots() \
    .expire_older_than(int((datetime.now() - timedelta(days=7)).timestamp() * 1000)) \
    .retain_last(50) \
    .commit()

Delta: VACUUM table RETAIN 168 HOURS (the default 168 = 7 days; you can go lower but Databricks warns against it because of in-flight readers).

Snowflake: storage costs include both Time Travel and Fail-safe storage automatically. Time Travel retention is configurable per table (DATA_RETENTION_TIME_IN_DAYS = 0..90); Fail-safe is a fixed extra 7 days that you cannot disable on the Enterprise tier.

A reasonable default policy for a production lakehouse:

• Keep 7 days of snapshots for time-travel recovery (covers most accidents-discovered-Monday-morning).
• Keep one snapshot per day for the past 30 days (audit window).
• Keep a tagged snapshot per quarter forever (regulatory).
• Vacuum nightly.

Why "snapshots are cheap, but only if you sweep them": every retained snapshot keeps every data file it references alive on S3. A table that takes 100 GB of live data but has 90 days of snapshot history with heavy churn could easily be paying for 500 GB of storage. Vacuum is not optional in production.

-- What's the most recent snapshot before the bad delete (which fired at 23:48:14)?
SELECT snapshot_id, committed_at, summary
FROM "warehouse"."transactions$snapshots"
WHERE committed_at < TIMESTAMP '2024-04-24 23:48:00 IST'
ORDER BY committed_at DESC LIMIT 5;

-- How many rows were live at that snapshot?
SELECT COUNT(*) FROM transactions FOR VERSION AS OF 7283450273948574234;
-- Returns: 412,889,431

-- And how many right now (post-delete)?
SELECT COUNT(*) FROM transactions;
-- Returns: 400,847,555  (delta = 12,041,876, matches the alert)

-- Restore the missing rows in one statement.
INSERT INTO transactions
SELECT * FROM transactions FOR VERSION AS OF 7283450273948574234
WHERE txn_id NOT IN (SELECT txn_id FROM transactions);

-- Snapshot the gold table at last night's batch close, clone for ML.
CREATE TABLE ml_sandbox.transactions_v3_train CLONE warehouse.transactions
  AT (TIMESTAMP => '2024-05-08 02:00:00 IST');
GRANT SELECT, INSERT, ALTER ON ml_sandbox.transactions_v3_train TO ROLE data_scientists;

Real-world workflows worth memorising

Three patterns recur across every Indian data platform team that adopts these features.

Pattern 1: Recovery from accidental DELETE / UPDATE. Capture the bad commit's timestamp. Find the immediately preceding snapshot. Insert back the rows that disappeared, or use RESTORE TABLE ... TO VERSION AS OF (Delta) for a wholesale rollback. Always do this before running vacuum — the data files for the previous snapshot must still exist on S3.

Pattern 2: Clone-test-swap schema migration. Clone the target table. Apply the migration to the clone. Run downstream queries against the clone. Validate row counts, null rates, sample joins. If everything passes, atomically rename: clone becomes prod, prod becomes archive. If anything fails, drop the clone — production never moved.

Pattern 3: Branch-experiment-merge for pipeline changes. Create a Nessie branch (or Iceberg branch). Run the new pipeline version against the branch. Compute the same metrics on main and on the branch in parallel. Compare. Merge if the branch wins, drop if it loses. This is the closest thing data engineering has to a feature branch in software engineering, and it is what makes A/B testing of pipelines themselves (not just product features) possible.

Going deeper

Merge conflicts in a lakehouse

Git's three-way merge works because source files are ordered sequences of lines. Two branches that touch different lines can be merged automatically; only conflicting line-level edits need human resolution.

Lakehouse branches work on tables, where the unit is a row, not a line. A row-level conflict — both branches updated the same (primary_key, version) — is harder to detect and resolve. Iceberg's current branch model is therefore "fast-forward or rebase," not "three-way merge": you can merge a branch back into main only if main has not advanced since the branch was created, or you accept that the branch's state wholly replaces the touched files. Nessie is more sophisticated and supports per-table fast-forward merges, but full row-level conflict resolution is still an open research area.

The practical implication: branches are most useful for isolated experiments — give the branch full ownership of the tables it touches for the duration of the experiment, then merge cleanly when no concurrent writers have moved main forward. Long-lived branches with concurrent writes on both sides will eventually require a manual reconciliation pass, just like a long-lived feature branch in git.

Why Snowflake's clone is per-object, not per-catalog

Snowflake's zero-copy clone descends from its column-store micro-partition design rather than from open table formats. Each Snowflake table is a sequence of immutable micro-partitions; cloning a table creates a new table object whose initial micro-partition list points at the same physical micro-partitions as the source. Modifications to the clone allocate new micro-partitions only.

But Snowflake's catalog is centralised and fully managed; you cannot directly inspect the on-disk format. As a result, Snowflake's clone is per-object (table, schema, or database) but does not have a Nessie-style cross-table branching model. You can clone a whole database to get a snapshot of every table together, which is most of what cross-table branching gives you, but you cannot then "merge" the clone back into the source — you would have to manually copy the changed objects, or rename-swap whole schemas.

The Iceberg + Nessie combination is more flexible (true git-like branching across a multi-table catalog) but operationally heavier. Snowflake is more turnkey but limited to clone-and-swap workflows.

The performance cost of deep snapshot history

Time-travel reads and snapshot enumeration both have overhead that scales with snapshot history. An Iceberg table with 100,000 retained snapshots has a 100,000-entry snapshot list in its top-level metadata JSON; reading that file becomes a megabyte-class I/O. Some Iceberg engines load the full snapshot list lazily (only when a time-travel query is issued), but planning queries on such a table is measurably slower than on a freshly compacted one.

A practical limit: most production teams retain at most 1,000 snapshots, and tag the rest if they need to keep specific commits findable. expire_snapshots with sensible retention is part of every healthy lakehouse's nightly maintenance.

References

1. Snowflake Time Travel — official documentation — query syntax, retention policies, and the relationship to Fail-safe storage.
2. Snowflake Zero-Copy Cloning — clone semantics for tables, schemas, and databases; storage accounting.
3. Apache Iceberg — Branching and Tagging — native branch and tag refs in the v2 spec, with SQL syntax for CREATE BRANCH, FAST FORWARD, and time-travel by ref name.
4. Databricks — Introducing Delta Time Travel for Large-Scale Data Lakes — the original Delta time-travel announcement, with the rationale and example queries.
5. Project Nessie — git-style branching and tagging catalog for Iceberg, including the REST API and CLI used by Spark, Flink, and Trino.
6. Jacques Nadeau et al., Nessie: Git-Inspired Catalog for the Data Lakehouse — the Dremio whitepaper introducing Nessie's design and consistency model.