Fluid Forge
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
  • Introduction

    • Home
    • Why Fluid Forge
    • Getting Started
    • Snowflake Quickstart
    • See it run
    • Forge Data Model
    • Vision & Roadmap
    • Playground
    • FAQ
  • Concepts

    • Concepts
    • Builds, Exposes, Bindings
    • What is a contract?
    • Quality, SLAs & Lineage
    • Governance & Policy
    • Agent Policy (LLM/AI governance)
    • Providers vs Platforms
    • Fluid Forge vs alternatives
  • Data Products

    • Consume a Data Product
    • Product Types — SDP, ADP, CDP
  • Walkthroughs

    • Walkthrough: Local Development
    • Source-Aligned: Postgres → DuckDB → Parquet
    • AI Forge And Data-Model Journeys
    • Walkthrough: MCP Output Port
    • Walkthrough: Deploy to Google Cloud Platform
    • Walkthrough: Snowflake Team Collaboration
    • Declarative Airflow DAG Generation - The FLUID Way
    • Generating Orchestration Code from Contracts
    • Jenkins CI/CD for FLUID Data Products
    • Universal Pipeline
    • The 11-Stage Pipeline
    • End-to-End Walkthrough: Catalog → Contract → Transformation
  • CLI Reference

    • CLI Reference
    • Core workflow

      • fluid init
      • fluid demo
      • fluid forge
      • fluid validate
      • fluid plan
      • fluid apply
      • fluid diff
      • fluid status
    • Build & ship

      • fluid bundle
      • fluid generate
      • fluid generate artifacts
      • fluid validate-artifacts
      • fluid verify-signature
      • fluid generate iac
      • fluid generate-airflow
      • fluid generate-pipeline
      • fluid generate vector
      • fluid viz-graph
      • fluid publish
      • fluid ship
      • fluid rollback
      • fluid schedule-sync
    • AI & Agents

      • fluid ai
      • fluid agents
      • fluid mission
      • fluid mcp
      • fluid memory
      • fluid stats
      • fluid skills
    • Quality & governance

      • fluid test
      • fluid verify
      • fluid contract-tests
      • fluid contract-validation
      • fluid policy
      • fluid policy check
      • fluid policy compile
      • fluid policy apply
    • Standards & interoperability

      • fluid odps
      • fluid odps-bitol
      • fluid odcs
      • fluid export
      • fluid export-odps
      • fluid exporters
      • fluid import
      • fluid market
      • fluid datamesh-manager
    • Project & workspace

      • fluid product-new
      • fluid product-add
      • fluid workspace
      • fluid contract
      • fluid split
      • fluid config
      • fluid providers
      • fluid plugins
      • fluid provider-init
      • fluid auth
      • fluid secrets
      • fluid ide
      • fluid scaffold-ci
      • fluid scaffold-composer
      • fluid scaffold-ide
      • fluid docs
      • fluid runs
      • fluid retention
      • fluid describe
      • fluid doctor
      • fluid roadmap
      • fluid version
    • Catalog adapters

      • Source Catalog Integration (V1.5)
      • Publishing to a Catalog — Overview
      • BigQuery Catalog
      • Snowflake Horizon Catalog
      • Databricks Unity Catalog
      • Google Dataplex Catalog
      • AWS Glue Data Catalog
      • DataHub Catalog
      • Data Mesh Manager Catalog
      • OpenMetadata Catalog
    • CLI by task

      • CLI by task
      • Add quality rules
      • Add agent governance
      • Debug a failed pipeline run
      • Switch clouds with one line
  • Recipes

    • Recipes
    • Recipe — add a quality rule
    • Recipe — switch clouds with one line
    • Recipe — tag PII in your schema
    • Write a contract that consumes another contract
    • Generate per-environment overlays
  • SDK & Plugins

    • SDK & Plugins
    • Quickstart — your first plugin
    • Examples

      • Runnable examples
      • Example: hello-scaffold — the minimal viable plugin
      • Example: gitlab-ci-scaffold — generate a complete CI project
      • Example: steward-validator — a custom governance rule
      • Example: prod-key-guard — apply-time invariant check
    • Journeys

      • Journeys
      • Your own CI/CD

        • You have your own CI/CD setup, no problem
        • GitLab CI — the bundle template
        • GitHub Actions — the bundle template
        • Jenkins — the bundle template
        • CircleCI — the bundle template
      • You have a strict project layout, no problem
      • You have governance rules, no problem
      • You want a check at apply time, no problem
    • Reference

      • Reference
      • Roles reference
      • Entry points reference
      • Trust model
      • Packaging
      • Companion packages
  • Providers

    • Providers
    • Provider Architecture
    • GCP Provider
    • AWS Provider
    • Snowflake Provider
    • Local Provider
    • Creating Custom Providers
    • Provider Roadmap
  • AI & Agents

    • MCP Server
    • Built-in And Custom Forge Guidance
    • Forge Discovery Guide
    • Forge Memory Guide
    • Authoring Forge Tools
    • Guided fluid forge UX
    • LLM Providers
    • LiteLLM Backend
    • Capability Warnings
    • Cost Tracking
    • FLUID Forge Contract GPT Packet
    • Agentic Primitives
  • Operate & Deploy

    • Operating in CI
    • Production Troubleshooting
    • Airflow Integration
    • Blueprints
    • Source-Aligned Acquisition
  • Govern & Secure

    • Governance, Compliance & the Business Case
    • Governance & Compliance
    • Network Safety
    • Credential Resolver — Security Model
  • Configuration & Reference

    • Environment Variables
    • Typed Errors
    • Typed CLI Errors
    • API Stability — fluid_build.api
  • Architecture & Releases

    • V1.5 Catalog Integration — Architecture Deep-Dive
    • V1.5 + V2 Hardening — Release Notes
  • Project

    • Contributing to Fluid Forge
    • Fluid Forge Docs Baseline: CLI 0.14.0
    • Fluid Forge Docs Baseline: CLI 0.13.0
    • Fluid Forge Docs Baseline: CLI 0.12.0
    • Fluid Forge Docs Baseline: CLI 0.11.0
    • Fluid Forge Docs Baseline: CLI 0.10.0
    • Fluid Forge Docs Baseline: CLI 0.9.0
    • Fluid Forge Docs Baseline: CLI 0.8.11
    • Fluid Forge Docs Baseline: CLI 0.8.10
    • Fluid Forge Docs Baseline: CLI 0.8.9
    • Fluid Forge Docs Baseline: CLI 0.8.8
    • Fluid Forge Docs Baseline: CLI 0.8.7
    • Fluid Forge Docs Baseline: CLI 0.8.6
    • Fluid Forge Docs Baseline: CLI 0.8.5
    • Fluid Forge Docs Baseline: CLI 0.8.4
    • Fluid Forge Docs Baseline: CLI 0.8.3
    • Fluid Forge Docs Baseline: CLI 0.8.0
    • Fluid Forge Docs Baseline: CLI 0.7.11
    • Fluid Forge Docs Baseline: CLI 0.7.9
    • Fluid Forge v0.7.1 - Multi-Provider Export Release

CLI Reference

This section tracks the promoted command surface shown by fluid --help in forge-cli 0.10.0.

Looking up how to do something (not what a command does)?

The page you actually want is CLI by task → — narrative walkthroughs organized around things you might be trying to accomplish, like "deploy to a new cloud", "add quality rules", "debug a failed run", or "add agent governance". Each task page links to the relevant commands. This page is the alphabetical command reference — better for lookup, worse for onboarding.

Read this first

  • CLI release examples in this section use 0.10.0
  • Contract examples use fluidVersion: 0.7.2
  • fluid version and fluidVersion are different things
  • The pinned CLI version is recorded in docs/.vuepress/cli-version.json and enforced by the cli-consistency workflow.
  • Install: pip install data-product-forge. Stable 0.10.0 is on PyPI; TestPyPI is only for release validation and intentional next-release candidates — see Getting Started for the full install matrix.

The 11-stage pipeline

0.8.0 promotes an eleven-stage production pipeline — each stage is a CI gate that exits non-zero on failure. Each command below maps to exactly one stage:

1. bundle → 2. validate → 3. generate-artifacts → 4. validate-artifacts
      → 5. diff (drift gate) → 6. plan → 7. apply → 8. policy-apply
      → 9. verify → 10. publish → 11. schedule-sync (Path A only)

See Walkthrough → 11-stage pipeline for the full end-to-end flow.

Core Workflow

CommandWhat it is for
fluid initCreate a new project
fluid demoZero-setup ~30 second customer-360 example on local DuckDB
fluid forgeAI-assisted scaffolding
fluid forge data-modelForge a reviewable data model from an intent file, DDL, or source catalog
fluid skillsIndustry knowledge packs that augment fluid forge
Source catalogs (V1.5)Forge directly from Snowflake / Unity / BigQuery / Dataplex / Glue / DataHub / DMM metadata
fluid validateCheck contract syntax and provider rules
fluid planPlan execution (--html, --env, --out)
fluid applyDeploy end-to-end (--mode, --yes, --dry-run)
fluid statusOne-page summary of the product in the current directory

The newcomer path is usually:

fluid init my-project --quickstart
cd my-project
fluid validate contract.fluid.yaml
fluid plan contract.fluid.yaml
fluid apply contract.fluid.yaml --yes

Pipeline Stages

These commands each map to one stage of the 11-stage production pipeline. Most users trigger them via a generated CI pipeline (fluid generate ci) rather than running them by hand, but every stage is usable standalone.

#CommandWhat it does
1fluid bundlePackage contract + sources into a signed tgz bundle (--sign, --attest, --format tgz)
2fluid validateCheck contract syntax and provider rules (--strict, --report)
3fluid generate artifactsFanout: ODCS + ODPS-Bitol + schedule + policy bindings
4fluid validate-artifactsVerify MANIFEST SHA-256 + per-format schema checks
5fluid diffDetect drift from deployed state (--exit-on-drift)
6fluid planPlan execution (--html mermaid DAG, --env, --out)
7fluid applyDeploy (--mode, --allow-data-loss, --yes)
8fluid policy-applyEnforce IAM/GRANT bindings (`--mode check
9fluid verifyConfirm deployed state matches the contract (--strict)
10fluid publishPublish to enterprise data catalogs (--target repeatable)
11fluid schedule-syncPush DAGs to airflow / composer / mwaa / astronomer / prefect / dagster

Safety & Supply Chain

These commands are for production-safety concerns that live alongside the pipeline rather than inside it.

CommandWhat it is for
fluid rollbackRestore from the auto-snapshot taken before apply --mode replace (use --list for read-only discovery)
fluid verify-signatureVerify a Sigstore cosign signature + SLSA attestation on a tgz bundle

Generate & Visualize

CommandWhat it is for
fluid generateUnified generation entry point (transformations, schedules, CI, standards, artifacts)
fluid generate artifactsStage 3 fanout — ODCS + ODPS-Bitol + schedule + policy bindings
fluid generate-pipelineUniversal pipeline scaffolds (legacy alias)
fluid generate-airflowCompatibility shim for generate schedule --scheduler airflow
fluid viz-graphRender the contract as an interactive lineage graph (SVG/HTML/PNG/DOT)

The promoted orchestration path is fluid generate schedule --scheduler airflow.

Standards & Interop

CommandWhat it is for
fluid exportersList the spec-export formats (ODCS / ODPS / ODPS-Bitol) a contract can be serialized to
fluid odpsExport, validate, and inspect the official ODPS (Open Data Product Initiative) format
fluid odps-bitolBitol.io's ODPS variant (Entropy Data marketplace)
fluid odcsBidirectional FLUID ↔ Open Data Contract Standard (ODCS v3.1.0)
fluid exportExport to executable orchestration code (Airflow, Dagster, Prefect)
fluid export-odpsOne-shot ODPS file export shortcut

ODPS and ODCS are spec exporters, not cloud providers — they serialize a contract to an open standard and do not deploy infrastructure. List them with fluid exporters; for deployment targets see fluid providers.

Integrations

CommandWhat it is for
fluid publishPublish to enterprise data catalogs (--target repeatable)
fluid datamesh-managerPublish products, contracts, and Access lineage to Entropy Data / Data Mesh Manager
fluid marketSearch and browse discovered products and blueprints
fluid importScan existing projects and generate FLUID contracts

Quality & Governance

0.8.0 introduces a unified fluid policy {check,compile,apply} subcommand group; the existing hyphenated commands stay registered as one-release compatibility aliases.

CommandWhat it is for
fluid policy checkStatic lint of the contract's policy declarations (same surface as fluid policy-check)
fluid policy compileCompile accessPolicy → provider IAM bindings (runtime/policy/bindings.json)
fluid policy applyDeploy compiled IAM bindings to the target warehouse (stage 8)
fluid contract-testsRun contract-level test scenarios
fluid contract-validationValidate contract structure and references
fluid diffDetect drift from deployed state
fluid testValidate the contract against live resources
fluid verifyVerify deployed resources still match the contract
fluid missionDeclarative goals with deterministic success criteria; mission check is a zero-LLM CI gate

Project & Workspace

CommandWhat it is for
fluid product-newScaffold a new data product
fluid product-addAdd a product to an existing workspace
fluid workspaceManage multi-product workspaces
fluid ideIDE integration helpers
fluid aiAI provider configuration
fluid memoryInspect and manage forge memory namespaces
fluid mcp serveServe forge tools to MCP-compatible clients

CI & Scaffolding

CommandWhat it is for
fluid generate ciGenerate parameterised 11-stage CI pipelines, including Jenkins publish/verify defaults
fluid scaffold-ciLegacy CI/CD scaffolds (superseded by generate ci for 11-stage)
fluid scaffold-composerGenerate Cloud Composer scaffolds
fluid docsBuild / index in-product documentation

Utilities

CommandWhat it is for
fluid configView and set workspace defaults
fluid splitSplit a flat contract into fragments
fluid authManage provider authentication flows
fluid doctorRun built-in health checks
fluid providersList registered infrastructure providers
fluid pluginsList installed plugins per role with allow/block governance status
fluid provider-initInitialize provider-specific configuration
fluid roadmapPrint the branch-local roadmap
fluid versionShow CLI version and environment details

Command discovery

fluid --help
fluid <command> -h

If a page in the docs uses an older command spelling such as fluid compile ... or fluid publish --catalog ..., treat it as historical:

  • fluid compile → use fluid bundle --format yaml (renamed in 0.7.3; bundle is now also the tgz+sign+attest command)
  • fluid publish --catalog X,Y → use fluid publish --target X --target Y (renamed + made repeatable in 0.7.3)
  • fluid policy-check / policy-compile / policy-apply → still work; new idiomatic form is fluid policy {check,compile,apply}

Need a hand with a specific command, or noticing something out of date? Start a discussion or open an issue — docs PRs welcome.

Edit this page on GitHub
Last Updated: 7/18/26, 7:02 PM
Contributors: Jeff Watson, jeffwatson-ai, Claude Opus 4.6 (1M context), fas89, Claude Opus 4.7, Claude Opus 4.7 (1M context)