Fluid Forge
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
  • Introduction

    • Home
    • Why Fluid Forge
    • Getting Started
    • Snowflake Quickstart
    • See it run
    • Forge Data Model
    • Vision & Roadmap
    • Playground
    • FAQ
  • Concepts

    • Concepts
    • Builds, Exposes, Bindings
    • What is a contract?
    • Quality, SLAs & Lineage
    • Governance & Policy
    • Agent Policy (LLM/AI governance)
    • Providers vs Platforms
    • Fluid Forge vs alternatives
  • Data Products

    • Consume a Data Product
    • Product Types — SDP, ADP, CDP
  • Walkthroughs

    • Walkthrough: Local Development
    • Source-Aligned: Postgres → DuckDB → Parquet
    • AI Forge And Data-Model Journeys
    • Walkthrough: MCP Output Port
    • Walkthrough: Deploy to Google Cloud Platform
    • Walkthrough: Snowflake Team Collaboration
    • Declarative Airflow DAG Generation - The FLUID Way
    • Generating Orchestration Code from Contracts
    • Jenkins CI/CD for FLUID Data Products
    • Universal Pipeline
    • The 11-Stage Pipeline
    • End-to-End Walkthrough: Catalog → Contract → Transformation
  • CLI Reference

    • CLI Reference
    • Core workflow

      • fluid init
      • fluid demo
      • fluid forge
      • fluid validate
      • fluid plan
      • fluid apply
      • fluid diff
      • fluid status
    • Build & ship

      • fluid bundle
      • fluid generate
      • fluid generate artifacts
      • fluid validate-artifacts
      • fluid verify-signature
      • fluid generate iac
      • fluid generate-airflow
      • fluid generate-pipeline
      • fluid viz-graph
      • fluid publish
      • fluid ship
      • fluid rollback
      • fluid schedule-sync
    • AI & Agents

      • fluid ai
      • fluid agents
      • fluid mcp
      • fluid memory
      • fluid stats
      • fluid skills
    • Quality & governance

      • fluid test
      • fluid verify
      • fluid contract-tests
      • fluid contract-validation
      • fluid policy
      • fluid policy check
      • fluid policy compile
      • fluid policy apply
    • Standards & interoperability

      • fluid odps
      • fluid odps-bitol
      • fluid odcs
      • fluid export
      • fluid export-odps
      • fluid exporters
      • fluid import
      • fluid market
      • fluid datamesh-manager
    • Project & workspace

      • fluid product-new
      • fluid product-add
      • fluid workspace
      • fluid contract
      • fluid split
      • fluid config
      • fluid providers
      • fluid plugins
      • fluid provider-init
      • fluid auth
      • fluid secrets
      • fluid ide
      • fluid scaffold-ci
      • fluid scaffold-composer
      • fluid scaffold-ide
      • fluid docs
      • fluid runs
      • fluid retention
      • fluid describe
      • fluid doctor
      • fluid roadmap
      • fluid version
    • Catalog adapters

      • Source Catalog Integration (V1.5)
      • Publishing to a Catalog — Overview
      • BigQuery Catalog
      • Snowflake Horizon Catalog
      • Databricks Unity Catalog
      • Google Dataplex Catalog
      • AWS Glue Data Catalog
      • DataHub Catalog
      • Data Mesh Manager Catalog
      • OpenMetadata Catalog
    • CLI by task

      • CLI by task
      • Add quality rules
      • Add agent governance
      • Debug a failed pipeline run
      • Switch clouds with one line
  • Recipes

    • Recipes
    • Recipe — add a quality rule
    • Recipe — switch clouds with one line
    • Recipe — tag PII in your schema
    • Write a contract that consumes another contract
    • Generate per-environment overlays
  • SDK & Plugins

    • SDK & Plugins
    • Quickstart — your first plugin
    • Examples

      • Runnable examples
      • Example: hello-scaffold — the minimal viable plugin
      • Example: gitlab-ci-scaffold — generate a complete CI project
      • Example: steward-validator — a custom governance rule
      • Example: prod-key-guard — apply-time invariant check
    • Journeys

      • Journeys
      • Your own CI/CD

        • You have your own CI/CD setup, no problem
        • GitLab CI — the bundle template
        • GitHub Actions — the bundle template
        • Jenkins — the bundle template
        • CircleCI — the bundle template
      • You have a strict project layout, no problem
      • You have governance rules, no problem
      • You want a check at apply time, no problem
    • Reference

      • Reference
      • Roles reference
      • Entry points reference
      • Trust model
      • Packaging
      • Companion packages
  • Providers

    • Providers
    • Provider Architecture
    • GCP Provider
    • AWS Provider
    • Snowflake Provider
    • Local Provider
    • Creating Custom Providers
    • Provider Roadmap
  • AI & Agents

    • MCP Server
    • Built-in And Custom Forge Guidance
    • Forge Discovery Guide
    • Forge Memory Guide
    • Authoring Forge Tools
    • Guided fluid forge UX
    • LLM Providers
    • LiteLLM Backend
    • Capability Warnings
    • Cost Tracking
    • FLUID Forge Contract GPT Packet
    • Agentic Primitives
  • Operate & Deploy

    • Airflow Integration
    • Blueprints
    • Source-Aligned Acquisition
  • Govern & Secure

    • Governance, Compliance & the Business Case
    • Governance & Compliance
    • Network Safety
    • Credential Resolver — Security Model
  • Configuration & Reference

    • Environment Variables
    • Typed Errors
    • Typed CLI Errors
    • API Stability — fluid_build.api
  • Architecture & Releases

    • V1.5 Catalog Integration — Architecture Deep-Dive
    • V1.5 + V2 Hardening — Release Notes
  • Project

    • Contributing to Fluid Forge
    • Fluid Forge Docs Baseline: CLI 0.9.0
    • Fluid Forge Docs Baseline: CLI 0.8.11
    • Fluid Forge Docs Baseline: CLI 0.8.10
    • Fluid Forge Docs Baseline: CLI 0.8.9
    • Fluid Forge Docs Baseline: CLI 0.8.8
    • Fluid Forge Docs Baseline: CLI 0.8.7
    • Fluid Forge Docs Baseline: CLI 0.8.6
    • Fluid Forge Docs Baseline: CLI 0.8.5
    • Fluid Forge Docs Baseline: CLI 0.8.4
    • Fluid Forge Docs Baseline: CLI 0.8.3
    • Fluid Forge Docs Baseline: CLI 0.8.0
    • Fluid Forge Docs Baseline: CLI 0.7.11
    • Fluid Forge Docs Baseline: CLI 0.7.9
    • Fluid Forge v0.7.1 - Multi-Provider Export Release

CLI Reference

This section tracks the promoted command surface shown by fluid --help in forge-cli 0.10.0.

Looking up how to do something (not what a command does)?

The page you actually want is CLI by task → — narrative walkthroughs organized around things you might be trying to accomplish, like "deploy to a new cloud", "add quality rules", "debug a failed run", or "add agent governance". Each task page links to the relevant commands. This page is the alphabetical command reference — better for lookup, worse for onboarding.

Read this first

  • CLI release examples in this section use 0.10.0
  • Contract examples use fluidVersion: 0.7.2
  • fluid version and fluidVersion are different things
  • The pinned CLI version is recorded in docs/.vuepress/cli-version.json and enforced by the cli-consistency workflow.
  • Install: pip install data-product-forge. Stable 0.10.0 is on PyPI; TestPyPI is only for release validation and intentional next-release candidates — see Getting Started for the full install matrix.

The 11-stage pipeline

0.8.0 promotes an eleven-stage production pipeline — each stage is a CI gate that exits non-zero on failure. Each command below maps to exactly one stage:

1. bundle → 2. validate → 3. generate-artifacts → 4. validate-artifacts
      → 5. diff (drift gate) → 6. plan → 7. apply → 8. policy-apply
      → 9. verify → 10. publish → 11. schedule-sync (Path A only)

See Walkthrough → 11-stage pipeline for the full end-to-end flow.

Core Workflow

CommandWhat it is for
fluid initCreate a new project
fluid demoZero-setup ~30 second customer-360 example on local DuckDB
fluid forgeAI-assisted scaffolding
fluid forge data-modelForge a reviewable data model from an intent file, DDL, or source catalog
fluid skillsIndustry knowledge packs that augment fluid forge
Source catalogs (V1.5)Forge directly from Snowflake / Unity / BigQuery / Dataplex / Glue / DataHub / DMM metadata
fluid validateCheck contract syntax and provider rules
fluid planPlan execution (--html, --env, --out)
fluid applyDeploy end-to-end (--mode, --yes, --dry-run)
fluid statusOne-page summary of the product in the current directory

The newcomer path is usually:

fluid init my-project --quickstart
cd my-project
fluid validate contract.fluid.yaml
fluid plan contract.fluid.yaml
fluid apply contract.fluid.yaml --yes

Pipeline Stages

These commands each map to one stage of the 11-stage production pipeline. Most users trigger them via a generated CI pipeline (fluid generate ci) rather than running them by hand, but every stage is usable standalone.

#CommandWhat it does
1fluid bundlePackage contract + sources into a signed tgz bundle (--sign, --attest, --format tgz)
2fluid validateCheck contract syntax and provider rules (--strict, --report)
3fluid generate artifactsFanout: ODCS + ODPS-Bitol + schedule + policy bindings
4fluid validate-artifactsVerify MANIFEST SHA-256 + per-format schema checks
5fluid diffDetect drift from deployed state (--exit-on-drift)
6fluid planPlan execution (--html mermaid DAG, --env, --out)
7fluid applyDeploy (--mode, --allow-data-loss, --yes)
8fluid policy-applyEnforce IAM/GRANT bindings (`--mode check
9fluid verifyConfirm deployed state matches the contract (--strict)
10fluid publishPublish to enterprise data catalogs (--target repeatable)
11fluid schedule-syncPush DAGs to airflow / composer / mwaa / astronomer / prefect / dagster

Safety & Supply Chain

These commands are for production-safety concerns that live alongside the pipeline rather than inside it.

CommandWhat it is for
fluid rollbackRestore from the auto-snapshot taken before apply --mode replace (use --list for read-only discovery)
fluid verify-signatureVerify a Sigstore cosign signature + SLSA attestation on a tgz bundle

Generate & Visualize

CommandWhat it is for
fluid generateUnified generation entry point (transformations, schedules, CI, standards, artifacts)
fluid generate artifactsStage 3 fanout — ODCS + ODPS-Bitol + schedule + policy bindings
fluid generate-pipelineUniversal pipeline scaffolds (legacy alias)
fluid generate-airflowCompatibility shim for generate schedule --scheduler airflow
fluid viz-graphRender the contract as an interactive lineage graph (SVG/HTML/PNG/DOT)

The promoted orchestration path is fluid generate schedule --scheduler airflow.

Standards & Interop

CommandWhat it is for
fluid exportersList the spec-export formats (ODCS / ODPS / ODPS-Bitol) a contract can be serialized to
fluid odpsExport, validate, and inspect the official ODPS (Open Data Product Initiative) format
fluid odps-bitolBitol.io's ODPS variant (Entropy Data marketplace)
fluid odcsBidirectional FLUID ↔ Open Data Contract Standard (ODCS v3.1.0)
fluid exportExport to executable orchestration code (Airflow, Dagster, Prefect)
fluid export-odpsOne-shot ODPS file export shortcut

ODPS and ODCS are spec exporters, not cloud providers — they serialize a contract to an open standard and do not deploy infrastructure. List them with fluid exporters; for deployment targets see fluid providers.

Integrations

CommandWhat it is for
fluid publishPublish to enterprise data catalogs (--target repeatable)
fluid datamesh-managerPublish products, contracts, and Access lineage to Entropy Data / Data Mesh Manager
fluid marketSearch and browse discovered products and blueprints
fluid importScan existing projects and generate FLUID contracts

Quality & Governance

0.8.0 introduces a unified fluid policy {check,compile,apply} subcommand group; the existing hyphenated commands stay registered as one-release compatibility aliases.

CommandWhat it is for
fluid policy checkStatic lint of the contract's policy declarations (same surface as fluid policy-check)
fluid policy compileCompile accessPolicy → provider IAM bindings (runtime/policy/bindings.json)
fluid policy applyDeploy compiled IAM bindings to the target warehouse (stage 8)
fluid contract-testsRun contract-level test scenarios
fluid contract-validationValidate contract structure and references
fluid diffDetect drift from deployed state
fluid testValidate the contract against live resources
fluid verifyVerify deployed resources still match the contract

Project & Workspace

CommandWhat it is for
fluid product-newScaffold a new data product
fluid product-addAdd a product to an existing workspace
fluid workspaceManage multi-product workspaces
fluid ideIDE integration helpers
fluid aiAI provider configuration
fluid memoryInspect and manage forge memory namespaces
fluid mcp serveServe forge tools to MCP-compatible clients

CI & Scaffolding

CommandWhat it is for
fluid generate ciGenerate parameterised 11-stage CI pipelines, including Jenkins publish/verify defaults
fluid scaffold-ciLegacy CI/CD scaffolds (superseded by generate ci for 11-stage)
fluid scaffold-composerGenerate Cloud Composer scaffolds
fluid docsBuild / index in-product documentation

Utilities

CommandWhat it is for
fluid configView and set workspace defaults
fluid splitSplit a flat contract into fragments
fluid authManage provider authentication flows
fluid doctorRun built-in health checks
fluid providersList registered infrastructure providers
fluid pluginsList installed plugins per role with allow/block governance status
fluid provider-initInitialize provider-specific configuration
fluid roadmapPrint the branch-local roadmap
fluid versionShow CLI version and environment details

Command discovery

fluid --help
fluid <command> -h

If a page in the docs uses an older command spelling such as fluid compile ... or fluid publish --catalog ..., treat it as historical:

  • fluid compile → use fluid bundle --format yaml (renamed in 0.7.3; bundle is now also the tgz+sign+attest command)
  • fluid publish --catalog X,Y → use fluid publish --target X --target Y (renamed + made repeatable in 0.7.3)
  • fluid policy-check / policy-compile / policy-apply → still work; new idiomatic form is fluid policy {check,compile,apply}

Need a hand with a specific command, or noticing something out of date? Start a discussion or open an issue — docs PRs welcome.

Edit this page on GitHub
Last Updated: 6/27/26, 4:58 PM
Contributors: Jeff Watson, jeffwatson-ai, Claude Opus 4.6 (1M context), fas89, Claude Opus 4.7, Claude Opus 4.7 (1M context)