Fluid Forge
Get Started
See it run
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
Demos
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
GitHub
Get Started
See it run
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
Demos
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
GitHub
  • Introduction

    • Home
    • Getting Started
    • Snowflake Quickstart
    • See it run
    • Forge Data Model
    • Vision & Roadmap
    • Playground
    • FAQ
  • Concepts

    • Concepts
    • Builds, Exposes, Bindings
    • What is a contract?
    • Quality, SLAs & Lineage
    • Governance & Policy
    • Agent Policy (LLM/AI governance)
    • Providers vs Platforms
    • Fluid Forge vs alternatives
  • Data Products

    • Product Types — SDP, ADP, CDP
  • Walkthroughs

    • Walkthrough: Local Development
    • Source-Aligned: Postgres → DuckDB → Parquet
    • AI Forge And Data-Model Journeys
    • Walkthrough: Deploy to Google Cloud Platform
    • Walkthrough: Snowflake Team Collaboration
    • Declarative Airflow DAG Generation - The FLUID Way
    • Generating Orchestration Code from Contracts
    • Jenkins CI/CD for FLUID Data Products
    • Universal Pipeline
    • The 11-Stage Pipeline
    • End-to-End Walkthrough: Catalog → Contract → Transformation
  • CLI Reference

    • CLI Reference
    • fluid init
    • fluid demo
    • fluid forge
    • fluid skills
    • fluid status
    • fluid validate
    • fluid plan
    • fluid apply
    • fluid generate
    • fluid generate artifacts
    • fluid validate-artifacts
    • fluid verify-signature
    • fluid generate-airflow
    • fluid generate-pipeline
    • fluid viz-graph
    • fluid odps
    • fluid odps-bitol
    • fluid odcs
    • fluid export
    • fluid export-opds
    • fluid publish
    • fluid datamesh-manager
    • fluid market
    • fluid import
    • fluid policy
    • fluid policy check
    • fluid policy compile
    • fluid policy apply
    • fluid contract-tests
    • fluid contract-validation
    • fluid diff
    • fluid test
    • fluid verify
    • fluid product-new
    • fluid product-add
    • fluid workspace
    • fluid ide
    • fluid ai
    • fluid memory
    • fluid mcp
    • fluid scaffold-ci
    • fluid scaffold-composer
    • fluid scaffold-ide
    • fluid docs
    • fluid config
    • fluid split
    • fluid bundle
    • fluid auth
    • fluid doctor
    • fluid providers
    • fluid provider-init
    • fluid roadmap
    • fluid version
    • fluid runs
    • fluid retention
    • fluid secrets
    • fluid stats
    • fluid contract
    • fluid ship
    • fluid rollback
    • fluid schedule-sync
    • Catalog adapters

      • Source Catalog Integration (V1.5)
      • BigQuery Catalog
      • Snowflake Horizon Catalog
      • Databricks Unity Catalog
      • Google Dataplex Catalog
      • AWS Glue Data Catalog
      • DataHub Catalog
      • Data Mesh Manager Catalog
    • CLI by task

      • CLI by task
      • Add quality rules
      • Add agent governance
      • Debug a failed pipeline run
      • Switch clouds with one line
  • Recipes

    • Recipes
    • Recipe — add a quality rule
    • Recipe — switch clouds with one line
    • Recipe — tag PII in your schema
  • SDK & Plugins

    • SDK & Plugins
    • Quickstart — your first plugin
    • Examples

      • Runnable examples
      • Example: hello-scaffold — the minimal viable plugin
      • Example: gitlab-ci-scaffold — generate a complete CI project
      • Example: steward-validator — a custom governance rule
      • Example: prod-key-guard — apply-time invariant check
    • Journeys

      • Journeys
      • Your own CI/CD

        • You have your own CI/CD setup, no problem
        • GitLab CI — the bundle template
        • GitHub Actions — the bundle template
        • Jenkins — the bundle template
        • CircleCI — the bundle template
      • You have a strict project layout, no problem
      • You have governance rules, no problem
      • You want a check at apply time, no problem
    • Reference

      • Reference
      • Roles reference
      • Entry points reference
      • Trust model
      • Packaging
      • Companion packages
  • Providers

    • Providers
    • Provider Architecture
    • GCP Provider
    • AWS Provider
    • Snowflake Provider
    • Local Provider
    • Creating Custom Providers
    • Provider Roadmap
  • Advanced

    • Blueprints
    • Governance & Compliance
    • Airflow Integration
    • Built-in And Custom Forge Guidance
    • FLUID Forge Contract GPT Packet
    • Forge Discovery Guide
    • Forge Memory Guide
    • LLM Providers
    • Capability Warnings
    • LiteLLM Backend (opt-in)
    • MCP Server
    • Credential Resolver — Security Model
    • Cost Tracking
    • Agentic Primitives
    • Typed Errors
    • Typed CLI Errors
    • Authoring Forge Tools
    • Source-Aligned Acquisition
    • API Stability — fluid_build.api
    • Guided fluid forge UX
    • V1.5 Catalog Integration — Architecture Deep-Dive
    • V1.5 + V2 Hardening — Release Notes
  • Project

    • Contributing to Fluid Forge
    • Fluid Forge Docs Baseline: CLI 0.8.3
    • Fluid Forge Docs Baseline: CLI 0.8.0
    • Fluid Forge Docs Baseline: CLI 0.7.11
    • Fluid Forge Docs Baseline: CLI 0.7.9
    • Fluid Forge v0.7.1 - Multi-Provider Export Release

Task: Switch clouds with one line

You have a working data product on one cloud (say, local DuckDB) and you want to redeploy it on another (say, GCP BigQuery). The whole point of Forge's contract-first model is that this is a one-line YAML change + a re-apply.

Time: ~5 minutes assuming the target cloud is already credentialed.

What you start with

A working contract on the original cloud — for this task, a Bitcoin price tracker already running locally on DuckDB. (The GCP walkthrough builds this contract from scratch; here we start from the finished contract.fluid.yaml.)

fluid apply contract.fluid.yaml --yes
# ✓ Pipeline complete · runtime/out/bitcoin_prices.parquet

That produced a Parquet artifact via the local provider on DuckDB. Now we want the same product on BigQuery.

Step 1 — install the target provider

pip install "data-product-forge[gcp]"
fluid providers
# Lists: local ✓, gcp ✓

GCP credentials are picked up via Application Default Credentials (gcloud auth application-default login). No Forge-specific auth setup.

Step 2 — change one line in the contract

Open contract.fluid.yaml and find the binding block:

exposes:
  - exposeId: bitcoin_prices
    binding:
      platform: local              # ← change this
      format: parquet
      location:
        path: ./runtime/out/bitcoin_prices.parquet

Change to:

exposes:
  - exposeId: bitcoin_prices
    binding:
      platform: gcp                # ← here
      format: bigquery_table
      location:
        project: your-gcp-project
        dataset: crypto
        table: bitcoin_prices
        region: europe-west3

Three keys move (platform, format, location). Everything else — schema, dq.rules, accessPolicy, agentPolicy, sovereignty — stays byte-identical. Confirm with git diff:

git diff contract.fluid.yaml
# - platform: local
# - format: parquet
# - location: { path: ./runtime/out/bitcoin_prices.parquet }
# + platform: gcp
# + format: bigquery_table
# + location: { project: your-gcp-project, dataset: crypto, table: bitcoin_prices, region: europe-west3 }

Step 3 — re-validate against the new platform

fluid validate contract.fluid.yaml
# ✓ Schema 0.7.2 — passed
# ✓ binding.platform=gcp — supported (provider gcp installed)
# ✓ binding.location complete (project, dataset, table)
# ✓ Contract validation passed

Step 4 — preview what will change

fluid plan contract.fluid.yaml --env prod
# Plan summary:
#   + ensure  dataset    crypto (region=europe-west3)
#   + create  table      crypto.bitcoin_prices
#   + grant   role/dataViewer  (from accessPolicy.grants)
#   + run     build      bitcoin_price_ingestion (sql)

The planner is deterministic — same contract + same deployed state = same plan. The plan is bound (cryptographically) to the next apply step (stage 6 → 7 of the canonical pipeline).

Step 5 — apply

fluid apply contract.fluid.yaml --env prod --yes
# ⏳ Acquiring lease on gold.crypto.bitcoin_tracker_v1...
# ⏳ Ensuring BigQuery dataset crypto...
# ✓ dataset crypto exists
# ⏳ Creating table crypto.bitcoin_prices...
# ✓ table crypto.bitcoin_prices created
# ⏳ Running bitcoin_price_ingestion (BigQuery SQL)...
# ✓ transformation complete (24 rows in)
# ⏳ Applying IAM bindings...
# ✓ BigQuery roles/dataViewer granted to group:analysts@company.com
# ✓ Pipeline complete in 4.83 s

The same data product is now on BigQuery. Same schema, same governance, same dq.rules.

Step 6 — verify against the deployed state

fluid verify contract.fluid.yaml --env prod --strict
# ✓ Schema matches deployed state
# ✓ accessPolicy.grants match BigQuery IAM bindings
# ✓ dq.rules satisfied (24 rows pass completeness)
# ✓ Verify passed

What happens if I want to switch again?

Same pattern — change binding.platform to aws (or snowflake), re-apply. The provider for the target cloud handles its own dialect/SDK quirks; the contract stays identical.

What you DIDN'T have to do

  • Rewrite SQL for a new dialect (Forge handles dialect translation when engine: sql; rare cases that genuinely need a hand-tuned dialect can use engine: dbt with target-specific macros)
  • Re-author IAM in a new cloud's syntax (accessPolicy.grants → native IAM happens at policy-apply)
  • Re-write your Airflow DAG (fluid generate schedule --scheduler airflow regenerates with the new platform)
  • Re-test quality rules (the same dq.rules block runs against the new cloud's storage)

See also

  • GCP walkthrough — full GCP-specific deployment guide
  • Snowflake walkthrough — Snowflake team collaboration flow
  • Providers vs platforms — how the abstraction works
  • Same contract demo — frame-perfect cast of this exact swap on screen
Edit this page on GitHub
Last Updated: 5/17/26, 6:51 PM
Contributors: fas89, Claude Opus 4.7 (1M context)
Prev
Debug a failed pipeline run