Fluid Forge
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
  • Introduction

    • Home
    • Why Fluid Forge
    • Getting Started
    • Snowflake Quickstart
    • See it run
    • Forge Data Model
    • Vision & Roadmap
    • Playground
    • FAQ
  • Concepts

    • Concepts
    • Builds, Exposes, Bindings
    • What is a contract?
    • Quality, SLAs & Lineage
    • Governance & Policy
    • Agent Policy (LLM/AI governance)
    • Providers vs Platforms
    • Fluid Forge vs alternatives
  • Data Products

    • Consume a Data Product
    • Product Types — SDP, ADP, CDP
  • Walkthroughs

    • Walkthrough: Local Development
    • Source-Aligned: Postgres → DuckDB → Parquet
    • AI Forge And Data-Model Journeys
    • Walkthrough: MCP Output Port
    • Walkthrough: Deploy to Google Cloud Platform
    • Walkthrough: Snowflake Team Collaboration
    • Declarative Airflow DAG Generation - The FLUID Way
    • Generating Orchestration Code from Contracts
    • Jenkins CI/CD for FLUID Data Products
    • Universal Pipeline
    • The 11-Stage Pipeline
    • End-to-End Walkthrough: Catalog → Contract → Transformation
  • CLI Reference

    • CLI Reference
    • Core workflow

      • fluid init
      • fluid demo
      • fluid forge
      • fluid validate
      • fluid plan
      • fluid apply
      • fluid diff
      • fluid status
    • Build & ship

      • fluid bundle
      • fluid generate
      • fluid generate artifacts
      • fluid validate-artifacts
      • fluid verify-signature
      • fluid generate iac
      • fluid generate-airflow
      • fluid generate-pipeline
      • fluid generate vector
      • fluid viz-graph
      • fluid publish
      • fluid ship
      • fluid rollback
      • fluid schedule-sync
    • AI & Agents

      • fluid ai
      • fluid agents
      • fluid mission
      • fluid mcp
      • fluid memory
      • fluid stats
      • fluid skills
    • Quality & governance

      • fluid test
      • fluid verify
      • fluid contract-tests
      • fluid contract-validation
      • fluid policy
      • fluid policy check
      • fluid policy compile
      • fluid policy apply
    • Standards & interoperability

      • fluid odps
      • fluid odps-bitol
      • fluid odcs
      • fluid export
      • fluid export-odps
      • fluid exporters
      • fluid import
      • fluid market
      • fluid datamesh-manager
    • Project & workspace

      • fluid product-new
      • fluid product-add
      • fluid workspace
      • fluid contract
      • fluid split
      • fluid config
      • fluid providers
      • fluid plugins
      • fluid provider-init
      • fluid auth
      • fluid secrets
      • fluid ide
      • fluid scaffold-ci
      • fluid scaffold-composer
      • fluid scaffold-ide
      • fluid docs
      • fluid runs
      • fluid retention
      • fluid describe
      • fluid doctor
      • fluid roadmap
      • fluid version
    • Catalog adapters

      • Source Catalog Integration (V1.5)
      • Publishing to a Catalog — Overview
      • BigQuery Catalog
      • Snowflake Horizon Catalog
      • Databricks Unity Catalog
      • Google Dataplex Catalog
      • AWS Glue Data Catalog
      • DataHub Catalog
      • Data Mesh Manager Catalog
      • OpenMetadata Catalog
    • CLI by task

      • CLI by task
      • Add quality rules
      • Add agent governance
      • Debug a failed pipeline run
      • Switch clouds with one line
  • Recipes

    • Recipes
    • Recipe — add a quality rule
    • Recipe — switch clouds with one line
    • Recipe — tag PII in your schema
    • Write a contract that consumes another contract
    • Generate per-environment overlays
  • SDK & Plugins

    • SDK & Plugins
    • Quickstart — your first plugin
    • Examples

      • Runnable examples
      • Example: hello-scaffold — the minimal viable plugin
      • Example: gitlab-ci-scaffold — generate a complete CI project
      • Example: steward-validator — a custom governance rule
      • Example: prod-key-guard — apply-time invariant check
    • Journeys

      • Journeys
      • Your own CI/CD

        • You have your own CI/CD setup, no problem
        • GitLab CI — the bundle template
        • GitHub Actions — the bundle template
        • Jenkins — the bundle template
        • CircleCI — the bundle template
      • You have a strict project layout, no problem
      • You have governance rules, no problem
      • You want a check at apply time, no problem
    • Reference

      • Reference
      • Roles reference
      • Entry points reference
      • Trust model
      • Packaging
      • Companion packages
  • Providers

    • Providers
    • Provider Architecture
    • GCP Provider
    • AWS Provider
    • Snowflake Provider
    • Local Provider
    • Creating Custom Providers
    • Provider Roadmap
  • AI & Agents

    • MCP Server
    • Built-in And Custom Forge Guidance
    • Forge Discovery Guide
    • Forge Memory Guide
    • Authoring Forge Tools
    • Guided fluid forge UX
    • LLM Providers
    • LiteLLM Backend
    • Capability Warnings
    • Cost Tracking
    • FLUID Forge Contract GPT Packet
    • Agentic Primitives
  • Operate & Deploy

    • Operating in CI
    • Production Troubleshooting
    • Airflow Integration
    • Blueprints
    • Source-Aligned Acquisition
  • Govern & Secure

    • Governance, Compliance & the Business Case
    • Governance & Compliance
    • Network Safety
    • Credential Resolver — Security Model
  • Configuration & Reference

    • Environment Variables
    • Typed Errors
    • Typed CLI Errors
    • API Stability — fluid_build.api
  • Architecture & Releases

    • V1.5 Catalog Integration — Architecture Deep-Dive
    • V1.5 + V2 Hardening — Release Notes
  • Project

    • Contributing to Fluid Forge
    • Fluid Forge Docs Baseline: CLI 0.14.0
    • Fluid Forge Docs Baseline: CLI 0.13.0
    • Fluid Forge Docs Baseline: CLI 0.12.0
    • Fluid Forge Docs Baseline: CLI 0.11.0
    • Fluid Forge Docs Baseline: CLI 0.10.0
    • Fluid Forge Docs Baseline: CLI 0.9.0
    • Fluid Forge Docs Baseline: CLI 0.8.11
    • Fluid Forge Docs Baseline: CLI 0.8.10
    • Fluid Forge Docs Baseline: CLI 0.8.9
    • Fluid Forge Docs Baseline: CLI 0.8.8
    • Fluid Forge Docs Baseline: CLI 0.8.7
    • Fluid Forge Docs Baseline: CLI 0.8.6
    • Fluid Forge Docs Baseline: CLI 0.8.5
    • Fluid Forge Docs Baseline: CLI 0.8.4
    • Fluid Forge Docs Baseline: CLI 0.8.3
    • Fluid Forge Docs Baseline: CLI 0.8.0
    • Fluid Forge Docs Baseline: CLI 0.7.11
    • Fluid Forge Docs Baseline: CLI 0.7.9
    • Fluid Forge v0.7.1 - Multi-Provider Export Release

fluid apply

Stage 7 of the 11-stage pipeline. Execute a FLUID contract (or a saved plan) end-to-end: provision infrastructure, run transformations, apply governance, and publish to configured destinations.

0.8.0 adds a 6-mode apply matrix (--mode) with explicit destruction gating (--allow-data-loss) and cryptographic plan-binding (bundleDigest / planDigest verification).

Why it matters Deploying a data product is safe and reversible — the plan you reviewed is the plan that runs, and destructive changes are gated. fluid apply re-verifies the bundleDigest + planDigest before any DDL and refuses a tampered plan; --allow-data-loss is required for destructive operations.

Syntax

fluid apply CONTRACT

CONTRACT can be:

  • A FLUID contract file (e.g. contract.fluid.yaml) — plans and applies in one shot.
  • A saved plan JSON file (e.g. runtime/plan.json) — applies the already-planned actions, with digest verification.

Apply mode (stage-7 dispatch)

--modeWhat it does
dry-runRender the planned DDL without calling the warehouse. No state mutation. Safe in every environment.
create-onlyCREATE … IF NOT EXISTS, plus a pre-check that fails if the target already exists. Use when a fresh project needs a clean provision.
amend (default)ALTER TABLE ADD COLUMN IF NOT EXISTS; views CREATE OR REPLACE. Data preserved; new columns backfilled NULL. The everyday mode.
amend-and-buildSame DDL as amend, plus dbt run / dbt test (or the configured build runner). Transforms refreshed on top of the amended schema.
replaceAuto-snapshot the target, then CREATE OR REPLACE TABLE (Snowflake / BigQuery) or DROP+CREATE in a transaction (Redshift). Destructive: requires --allow-data-loss in non-dev environments or when the target has rows.
replace-and-buildSame as replace, plus a full dbt run --full-refresh to rebuild everything from sources. Destructive.

The build-augmented modes (amend-and-build, replace-and-build) run the configured build runner. Pass --build-id <id> to filter execution to a single build job from the contract's builds[]; when unset, every build runs.

Safety gates

OptionDescription
--allow-data-lossRequired to run replace / replace-and-build when FLUID_ENV != dev or the target already has rows. Two independent risk surfaces (env + population) → two-factor opt-in. Never default.
--no-verify-plan-bindingEmergency escape hatch. Skip the bundleDigest / planDigest verification that stage 7 normally enforces on a saved plan. Logged at WARNING so audit trails catch it. Use only during documented DR procedures.
--no-verify-federationEmergency escape hatch. Skip the federated-consumes[] upstream-digest gate (drift between a pinned upstreamDigest and the live upstream). Logged at WARNING for audit. A distinct trust domain from plan binding — each gate has its own narrowly-scoped waiver.
--adopt-shared-container(since 0.13.0) Confirm taking ownership of a container this contract previously referenced as a shared pool (packaging shared → isolated). Emits a structured packaging_adoption_override audit event; the data-loss gate still applies. See Packaging modes.

Since 0.13.1, the structured override events these gates emit — opentofu_destructive_gate_override (--allow-data-loss) and packaging_adoption_override (--adopt-shared-container) — log at WARNING, so audit pipelines filtering at WARNING-and-above catch them. On 0.13.0 and earlier they logged at INFO; event names and payloads are unchanged.

Packaging modes

Opt-in, new in 0.13.0

Only relevant to contracts that declare fluidVersion: "0.7.6" and carry a packaging block. A contract without one resolves to the LEGACY sentinel, can never transition, and applies exactly as it did before 0.13.0.

A packaging block declares whether this product owns each infrastructure container (isolated) or writes into a pre-existing, platform-owned pool (shared). Full reference: fluid generate iac — Packaging modes.

Changing a container's mode changes who owns it, but OpenTofu only sees a resource that left the configuration and plans a destroy — on a shared pool that reaches every other tenant's data. So apply diffs the resolved ownership model against tofu state list before tofu plan:

TransitionBehaviour
isolated → shared (owned → referenced)Always blocked — there is no flag. apply fails closed and prints copy-pasteable tofu -chdir=<workdir> state rm <address> commands. State surgery touches zero bytes of infrastructure; re-run apply afterwards.
shared → isolated (referenced → owned)Requires --adopt-shared-container. Without the gate, brownfield adoption would tofu import the platform's pool into this product's state with force_destroy restored — the exact blast radius the feature exists to close.
# Blocked — drop the resource from state first, then re-run.
fluid apply runtime/plan.json --provider aws --yes
# → packaging_transition_blocked: aws_s3_bucket.data (owned → referenced)
# →   tofu -chdir=.fluid/iac/aws/<product-id> state rm aws_s3_bucket.data

# Taking ownership of a previously-shared container (audited)
fluid apply runtime/plan.json --provider aws --yes --adopt-shared-container

The printed tofu state rm commands include -chdir pointing at the per-contract working directory (.fluid/iac/<provider>/<id>/), so they run against the right state without you having to find it.

Structured packaging_transition_blocked / packaging_adoption_override events are emitted for CI log scrapers. This guard runs earlier than, and is independent of, the data-loss gate — that gate remains the unconditional last line.

Plan binding

When you pass a saved plan (runtime/plan.json) instead of a contract, apply verifies:

  • bundleDigest in plan.json matches the MANIFEST SHA-256 of the tgz bundle the plan was built from. Mismatch → PlanBindingError(kind="bundle-mismatch") before any DDL executes.
  • planDigest in plan.json matches a re-computed digest of the plan's action list (internal consistency check). Mismatch → PlanBindingError(kind="plan-tamper").

This is the Terraform-style "apply consumes exact plan" guarantee, enforced cryptographically.

Key options

General

OptionDescription
--envApply an environment overlay (dev / staging / prod / …)

Execution control

OptionDescription
--yesSkip confirmation
--dry-runAlias for --mode dry-run
--ensure-opentofu(since 0.8.8) If the tofu binary is missing, provision a pinned, SHA-256-verified OpenTofu build before a cloud apply — no root, gpg, cosign, curl, or unzip needed (Python stdlib only). Idempotent (a usable tofu at/above the engine floor is left untouched) and a no-op for native / local applies. Pin via FLUID_OPENTOFU_VERSION. fluid generate ci bakes this into the generated apply stage so cloud applies work on locked-down / non-root runners.
--timeout TIMEOUTGlobal timeout in minutes
--parallel-phasesExecute independent phases in parallel
--max-workers MAX_WORKERSMaximum workers for parallel execution

Safety and rollback

OptionDescription
--rollback-strategynone, immediate, phase_complete, or full_rollback
--require-approvalRequire explicit approval for destructive work
--backup-stateCreate a backup before execution
--validate-dependenciesValidate dependencies before execution

Reporting

OptionDescription
--reportOutput path for the execution report
--report-formatReport format
--metrics-exportExport metrics to monitoring backends
--notifySend notifications to destinations such as Slack or email

Build execution

OptionDescription
--build-id BUILD_IDFilter build execution to a specific build job by ID from the contract's builds[]. Combine with --mode amend-and-build or --mode replace-and-build. When unset and the mode requires builds, every build runs.
--delay DELAYSeconds between build iterations
--fail-fastStop on first failure
--no-outputSuppress build script output

Debugging and advanced

OptionDescription
--verboseDetailed progress output
--keep-temp-filesKeep temporary files
--workspace-dirCustom workspace directory
--state-file STATE_FILECustom state file location
--config-overrideOverride contract config with JSON
--provider-configPath to provider-specific configuration

Examples

Everyday dev workflow

# Quickstart — default --mode amend, no destructive action
fluid apply contract.fluid.yaml --yes

# Preview-only
fluid apply contract.fluid.yaml --mode dry-run
fluid apply contract.fluid.yaml --dry-run   # same thing

Production 11-stage pipeline (plan-bound)

# Stage 6 produces the plan with bundleDigest + planDigest
fluid plan contract.fluid.yaml --out runtime/plan.json

# Stage 7 verifies both digests before executing any DDL
fluid apply runtime/plan.json --mode amend --env prod --yes

Destructive modes (explicit opt-in required)

# Prod replace — REQUIRES --allow-data-loss (two-factor opt-in)
fluid apply runtime/plan.json --mode replace --env prod --yes --allow-data-loss

# Full rebuild (dbt --full-refresh + destructive DDL)
fluid apply runtime/plan.json --mode replace-and-build --env prod --yes --allow-data-loss

After a replace, use fluid rollback to restore from the auto-snapshot if something goes wrong.

Build-augmented apply (dbt run)

fluid apply runtime/plan.json --mode amend-and-build --env dev --yes

DR escape hatch (skip digest verification — audit-logged)

# Only in documented DR procedures. Emits WARNING to the audit log.
fluid apply runtime/plan.json --mode amend --no-verify-plan-binding --yes

# Skip only the federated-consumes upstream-digest gate
fluid apply runtime/plan.json --mode amend --no-verify-federation --yes

Notes

  • The recommended sequence is bundle → validate → plan → apply.
  • For local-first onboarding, fluid apply contract.fluid.yaml --yes is the shortest path after a quickstart scaffold — default --mode amend is safe.
  • --mode replace / replace-and-build always create an auto-snapshot before destructive DDL. On Snowflake this is a zero-copy CLONE; on BigQuery it's bq cp --force; on Redshift it's CREATE TABLE _backup AS SELECT *. Snapshot names are recorded in .fluid/rollback-state.json.
  • If apply's provider dispatcher logs unknown_action_op for your contract's actions, the provider doesn't yet implement the abstract op. This is a known gap for some high-level ops (e.g. provisionDataset, scheduleTask) and is addressed by a translator layer in providers/<platform>/.
  • For Iceberg exposes, apply provisions the prerequisites dbt refuses to create — the Snowflake EXTERNAL VOLUME and AWS Glue CATALOG INTEGRATION (since 0.13.1), and the GCS warehouse bucket on GCP (since 0.14.0). Names are derived by the same deterministic helper that fluid generate uses for dbt's catalogs.yml, so the objects apply creates are exactly the ones dbt references. See the Snowflake and GCP provider pages.
  • (since 0.14.0) Apply hooks run on the OpenTofu path too, so --env plumbing reaches Snowflake / AWS / GCP cloud applies. Previously hooks ran only on the native apply path.

Extension point: apply hooks

As of 0.8.3, fluid apply runs any apply hook plugins registered via Python entry-points before invoking the providers. Use apply hooks to enforce runtime invariants that can't be checked at validate time — required env vars, image signatures, bundle-digest drift, business-hours gating, anything that depends on the deploy environment rather than the contract content.

A hook that appends an error aborts the apply with exit code 1. Pass --force-pattern-drift to downgrade all hook errors to WARNINGs (audit-logged) and let the apply proceed.

  • Author a hook: SDK & Plugins → Apply hook journey
  • Reference: Entry points → fluid_build.apply_hooks
  • Example: prod-key-guard
Edit this page on GitHub
Last Updated: 7/28/26, 8:03 PM
Contributors: Jeff Watson, fas89, Claude Opus 4.7 (1M context)
Prev
fluid plan
Next
fluid diff