Fluid Forge
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
  • Introduction

    • Home
    • Why Fluid Forge
    • Getting Started
    • Snowflake Quickstart
    • See it run
    • Forge Data Model
    • Vision & Roadmap
    • Playground
    • FAQ
  • Concepts

    • Concepts
    • Builds, Exposes, Bindings
    • What is a contract?
    • Quality, SLAs & Lineage
    • Governance & Policy
    • Agent Policy (LLM/AI governance)
    • Providers vs Platforms
    • Fluid Forge vs alternatives
  • Data Products

    • Consume a Data Product
    • Product Types — SDP, ADP, CDP
  • Walkthroughs

    • Walkthrough: Local Development
    • Source-Aligned: Postgres → DuckDB → Parquet
    • AI Forge And Data-Model Journeys
    • Walkthrough: MCP Output Port
    • Walkthrough: Deploy to Google Cloud Platform
    • Walkthrough: Snowflake Team Collaboration
    • Declarative Airflow DAG Generation - The FLUID Way
    • Generating Orchestration Code from Contracts
    • Jenkins CI/CD for FLUID Data Products
    • Universal Pipeline
    • The 11-Stage Pipeline
    • End-to-End Walkthrough: Catalog → Contract → Transformation
  • CLI Reference

    • CLI Reference
    • Core workflow

      • fluid init
      • fluid demo
      • fluid forge
      • fluid validate
      • fluid plan
      • fluid apply
      • fluid diff
      • fluid status
    • Build & ship

      • fluid bundle
      • fluid generate
      • fluid generate artifacts
      • fluid validate-artifacts
      • fluid verify-signature
      • fluid generate iac
      • fluid generate-airflow
      • fluid generate-pipeline
      • fluid viz-graph
      • fluid publish
      • fluid ship
      • fluid rollback
      • fluid schedule-sync
    • AI & Agents

      • fluid ai
      • fluid agents
      • fluid mcp
      • fluid memory
      • fluid stats
      • fluid skills
    • Quality & governance

      • fluid test
      • fluid verify
      • fluid contract-tests
      • fluid contract-validation
      • fluid policy
      • fluid policy check
      • fluid policy compile
      • fluid policy apply
    • Standards & interoperability

      • fluid odps
      • fluid odps-bitol
      • fluid odcs
      • fluid export
      • fluid export-odps
      • fluid exporters
      • fluid import
      • fluid market
      • fluid datamesh-manager
    • Project & workspace

      • fluid product-new
      • fluid product-add
      • fluid workspace
      • fluid contract
      • fluid split
      • fluid config
      • fluid providers
      • fluid plugins
      • fluid provider-init
      • fluid auth
      • fluid secrets
      • fluid ide
      • fluid scaffold-ci
      • fluid scaffold-composer
      • fluid scaffold-ide
      • fluid docs
      • fluid runs
      • fluid retention
      • fluid describe
      • fluid doctor
      • fluid roadmap
      • fluid version
    • Catalog adapters

      • Source Catalog Integration (V1.5)
      • Publishing to a Catalog — Overview
      • BigQuery Catalog
      • Snowflake Horizon Catalog
      • Databricks Unity Catalog
      • Google Dataplex Catalog
      • AWS Glue Data Catalog
      • DataHub Catalog
      • Data Mesh Manager Catalog
      • OpenMetadata Catalog
    • CLI by task

      • CLI by task
      • Add quality rules
      • Add agent governance
      • Debug a failed pipeline run
      • Switch clouds with one line
  • Recipes

    • Recipes
    • Recipe — add a quality rule
    • Recipe — switch clouds with one line
    • Recipe — tag PII in your schema
    • Write a contract that consumes another contract
    • Generate per-environment overlays
  • SDK & Plugins

    • SDK & Plugins
    • Quickstart — your first plugin
    • Examples

      • Runnable examples
      • Example: hello-scaffold — the minimal viable plugin
      • Example: gitlab-ci-scaffold — generate a complete CI project
      • Example: steward-validator — a custom governance rule
      • Example: prod-key-guard — apply-time invariant check
    • Journeys

      • Journeys
      • Your own CI/CD

        • You have your own CI/CD setup, no problem
        • GitLab CI — the bundle template
        • GitHub Actions — the bundle template
        • Jenkins — the bundle template
        • CircleCI — the bundle template
      • You have a strict project layout, no problem
      • You have governance rules, no problem
      • You want a check at apply time, no problem
    • Reference

      • Reference
      • Roles reference
      • Entry points reference
      • Trust model
      • Packaging
      • Companion packages
  • Providers

    • Providers
    • Provider Architecture
    • GCP Provider
    • AWS Provider
    • Snowflake Provider
    • Local Provider
    • Creating Custom Providers
    • Provider Roadmap
  • AI & Agents

    • MCP Server
    • Built-in And Custom Forge Guidance
    • Forge Discovery Guide
    • Forge Memory Guide
    • Authoring Forge Tools
    • Guided fluid forge UX
    • LLM Providers
    • LiteLLM Backend
    • Capability Warnings
    • Cost Tracking
    • FLUID Forge Contract GPT Packet
    • Agentic Primitives
  • Operate & Deploy

    • Airflow Integration
    • Blueprints
    • Source-Aligned Acquisition
  • Govern & Secure

    • Governance, Compliance & the Business Case
    • Governance & Compliance
    • Network Safety
    • Credential Resolver — Security Model
  • Configuration & Reference

    • Environment Variables
    • Typed Errors
    • Typed CLI Errors
    • API Stability — fluid_build.api
  • Architecture & Releases

    • V1.5 Catalog Integration — Architecture Deep-Dive
    • V1.5 + V2 Hardening — Release Notes
  • Project

    • Contributing to Fluid Forge
    • Fluid Forge Docs Baseline: CLI 0.9.0
    • Fluid Forge Docs Baseline: CLI 0.8.11
    • Fluid Forge Docs Baseline: CLI 0.8.10
    • Fluid Forge Docs Baseline: CLI 0.8.9
    • Fluid Forge Docs Baseline: CLI 0.8.8
    • Fluid Forge Docs Baseline: CLI 0.8.7
    • Fluid Forge Docs Baseline: CLI 0.8.6
    • Fluid Forge Docs Baseline: CLI 0.8.5
    • Fluid Forge Docs Baseline: CLI 0.8.4
    • Fluid Forge Docs Baseline: CLI 0.8.3
    • Fluid Forge Docs Baseline: CLI 0.8.0
    • Fluid Forge Docs Baseline: CLI 0.7.11
    • Fluid Forge Docs Baseline: CLI 0.7.9
    • Fluid Forge v0.7.1 - Multi-Provider Export Release

Getting Started

Run your first data product locally in a few minutes, then move to cloud targets when you are ready.

Why it matters You ship a real, versioned data product on your laptop in minutes — no cloud account, no platform-team ticket. The same contract.fluid.yaml you run locally is the one you later ship to any cloud; only binding.platform changes.

fluid init my-project --quickstart → validate → plan → apply
Click play above for the full local quickstart in 30 seconds. Or follow along step-by-step below.

What you skip with Fluid Forge

You're about to ship a working data product in 30 seconds. The five-tool stack you'd otherwise need: gone.

  • No Airflow DAG to write or maintain. fluid generate schedule --scheduler airflow|dagster|prefect emits the right artifact.
  • No JVM, no cluster, no heap tuning. engine: duckdb runs embedded.
  • No Snowflake permission sprawl. accessPolicy.grants compiles to native GRANT statements.
  • No Terraform for data IAM. policy-apply emits BigQuery / Snowflake / S3 IAM bindings from the same contract.
  • No 27 questions before you ship. fluid forge infers from your local files; you answer 4.

→ See the full comparison: Forge vs dbt / Dagster / Terraform / Snowpark.

What this guide assumes

  • Python 3.10+
  • pip
  • No cloud credentials required for the local-first path

Install the CLI

The current docs baseline is 0.10.0 — the stable tag was cut on 2026-06-27:

pip install --upgrade data-product-forge
pip install "data-product-forge==0.10.0"    # exact pin

Pre-releases (when published) ship to PyPI as PEP 440 pre-releases — pip install skips them by default. The --pre flag opts in.

Dependency floor (what you'll see in pip install)

0.10.0 pins minimum versions on several deps to close known CVEs:

  • jinja2 >= 3.1.6 (CVE-2025-27516), h11 >= 0.16 (CVE-2025-43859), cryptography >= 46.0.7 (CVE-2026-26007 / 39892 / 34073)
  • litellm >= 1.83.7, < 2 (CVE-2026-42208) — skips the compromised 1.82.7 / 1.82.8 PyPI artifacts
  • mcp >= 1.20 (required for fluid mcp serve sampling), keyring >= 24.0 (now a hard dependency — catalog source secrets default to the OS keyring)

Check the installed CLI and basic system health:

fluid version
fluid doctor

This docs set tracks CLI release 0.10.0. Docs updates land in lockstep with each release; if you're on an older CLI, some --mode / --target flags mentioned here won't be present yet — see the CLI index for what maps to what.

Extending the CLI? Fluid Forge ships three plugin extension points and a companion SDK on PyPI. If your team has its own CI templates, scaffolding standards, or governance rules, see SDK & Plugins.

Stuck on install? Jump to Troubleshooting further down, or open an issue — happy to help.

Understand the version numbers

You will see two different version concepts in the docs:

  • fluid version reports the installed CLI release, such as 0.10.0
  • fluidVersion inside contract.fluid.yaml selects the contract schema version, such as 0.7.5

Which fluidVersion a fresh scaffold emits depends on the path:

  • fluid init --quickstart copies the customer-360 template verbatim, which is pinned at fluidVersion: 0.7.2.
  • fluid init --discover, fluid forge, and fluid product-new go through the factory and emit fluidVersion: 0.7.5 — the latest bundled schema.

The CLI still accepts contracts with fluidVersion 0.4.0, 0.5.7, 0.7.1, 0.7.2, 0.7.3, 0.7.4, and 0.7.5 — run fluid version for the authoritative compatibility list.

Quickstart with fluid init

Create a local project:

fluid init my-project --quickstart
cd my-project

Then run the core workflow:

fluid validate contract.fluid.yaml
fluid plan contract.fluid.yaml
fluid apply contract.fluid.yaml --yes

🎉 You just shipped a data product

That output at output/customer_360.parquet is real. It has a schema, a contract, a validated SQL transformation, declared owners, and a deployment record. You shipped a versioned data product on your laptop in under a minute.

Three things you can do right now:

  1. Switch clouds with one line — change binding.platform: local to binding.platform: gcp in contract.fluid.yaml, run fluid apply again. Same data product, on BigQuery. (GCP walkthrough)
  2. Add an AI policy in 5 lines — append an agentPolicy block (allowedModels, deniedUseCases, auditRequired). Now fluid policy-check enforces who can read this from an LLM. (Agent policy guide)
  3. Tweet your win — share this on X — paste the fluid apply output if you want to flex 😉

Beyond dev — the 11-stage production pipeline

The commands above are the dev on-ramp. For production, Fluid Forge promotes an 11-stage pipeline with cryptographic plan-binding, explicit destruction gating, and supply-chain signing:

1. bundle → 2. validate → 3. generate-artifacts → 4. validate-artifacts
      → 5. diff (drift gate) → 6. plan → 7. apply → 8. policy-apply
      → 9. verify → 10. publish → 11. schedule-sync (Path A only)

See the 11-stage pipeline walkthrough for the full end-to-end flow and fluid generate ci for auto-generating a parameterised pipeline for Jenkins / GitHub Actions / GitLab / Azure DevOps / Bitbucket / CircleCI / Tekton.

What the quickstart gives you

The generated project includes a working contract plus local assets so you can validate and apply immediately. The exact scaffold evolves over time, but the important files are:

my-project/
├── README.md
├── contract.fluid.yaml
├── data/
└── .fluid/

Optional AI-assisted path with fluid forge

If you want the CLI to discover local context and scaffold with LLM help, use fluid forge instead of fluid init:

fluid forge
fluid forge --domain finance
fluid forge --llm-provider openai --llm-model gpt-4.1-mini

Use fluid init for the fastest deterministic quickstart. Use fluid forge when you want discovery, memory, or domain-guided scaffolding.

Promoted next commands

fluid test contract.fluid.yaml
fluid verify contract.fluid.yaml
fluid generate schedule --scheduler airflow
fluid publish contract.fluid.yaml

Compatibility note: fluid generate-airflow still exists, but the promoted orchestration path is fluid generate schedule --scheduler airflow.

Move to providers later

When you are ready to target a provider:

  • GCP guide
  • AWS guide
  • Snowflake quickstart
  • Provider overview

Troubleshooting

fluid: command not found

Try the module entry point:

python -m fluid_build.cli --help

Local quickstart dependencies look incomplete

Run:

fluid doctor --verbose

Unsure what to use next

Use the CLI help pages:

fluid --help
fluid <command> -h

Next steps

  • CLI Reference
  • Local walkthrough
  • Vision

Need help?

  • Questions or ideas? Start a GitHub Discussion
  • Bug or unexpected behavior? Open an issue with what you ran and what you saw
  • Want to contribute? See the contributing guide
Edit this page on GitHub
Last Updated: 6/27/26, 4:58 PM
Contributors: Jeff Watson, jeffwatson-ai, fas89, Claude Opus 4.7, Claude Opus 4.7 (1M context)
Prev
Why Fluid Forge
Next
Snowflake Quickstart