Fluid Forge
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
  • Introduction

    • Home
    • Why Fluid Forge
    • Getting Started
    • Snowflake Quickstart
    • See it run
    • Forge Data Model
    • Vision & Roadmap
    • Playground
    • FAQ
  • Concepts

    • Concepts
    • Builds, Exposes, Bindings
    • What is a contract?
    • Quality, SLAs & Lineage
    • Governance & Policy
    • Agent Policy (LLM/AI governance)
    • Providers vs Platforms
    • Fluid Forge vs alternatives
  • Data Products

    • Consume a Data Product
    • Product Types — SDP, ADP, CDP
  • Walkthroughs

    • Walkthrough: Local Development
    • Source-Aligned: Postgres → DuckDB → Parquet
    • AI Forge And Data-Model Journeys
    • Walkthrough: MCP Output Port
    • Walkthrough: Deploy to Google Cloud Platform
    • Walkthrough: Snowflake Team Collaboration
    • Declarative Airflow DAG Generation - The FLUID Way
    • Generating Orchestration Code from Contracts
    • Jenkins CI/CD for FLUID Data Products
    • Universal Pipeline
    • The 11-Stage Pipeline
    • End-to-End Walkthrough: Catalog → Contract → Transformation
  • CLI Reference

    • CLI Reference
    • Core workflow

      • fluid init
      • fluid demo
      • fluid forge
      • fluid validate
      • fluid plan
      • fluid apply
      • fluid diff
      • fluid status
    • Build & ship

      • fluid bundle
      • fluid generate
      • fluid generate artifacts
      • fluid validate-artifacts
      • fluid verify-signature
      • fluid generate iac
      • fluid generate-airflow
      • fluid generate-pipeline
      • fluid viz-graph
      • fluid publish
      • fluid ship
      • fluid rollback
      • fluid schedule-sync
    • AI & Agents

      • fluid ai
      • fluid agents
      • fluid mcp
      • fluid memory
      • fluid stats
      • fluid skills
    • Quality & governance

      • fluid test
      • fluid verify
      • fluid contract-tests
      • fluid contract-validation
      • fluid policy
      • fluid policy check
      • fluid policy compile
      • fluid policy apply
    • Standards & interoperability

      • fluid odps
      • fluid odps-bitol
      • fluid odcs
      • fluid export
      • fluid export-odps
      • fluid exporters
      • fluid import
      • fluid market
      • fluid datamesh-manager
    • Project & workspace

      • fluid product-new
      • fluid product-add
      • fluid workspace
      • fluid contract
      • fluid split
      • fluid config
      • fluid providers
      • fluid plugins
      • fluid provider-init
      • fluid auth
      • fluid secrets
      • fluid ide
      • fluid scaffold-ci
      • fluid scaffold-composer
      • fluid scaffold-ide
      • fluid docs
      • fluid runs
      • fluid retention
      • fluid describe
      • fluid doctor
      • fluid roadmap
      • fluid version
    • Catalog adapters

      • Source Catalog Integration (V1.5)
      • Publishing to a Catalog — Overview
      • BigQuery Catalog
      • Snowflake Horizon Catalog
      • Databricks Unity Catalog
      • Google Dataplex Catalog
      • AWS Glue Data Catalog
      • DataHub Catalog
      • Data Mesh Manager Catalog
      • OpenMetadata Catalog
    • CLI by task

      • CLI by task
      • Add quality rules
      • Add agent governance
      • Debug a failed pipeline run
      • Switch clouds with one line
  • Recipes

    • Recipes
    • Recipe — add a quality rule
    • Recipe — switch clouds with one line
    • Recipe — tag PII in your schema
    • Write a contract that consumes another contract
    • Generate per-environment overlays
  • SDK & Plugins

    • SDK & Plugins
    • Quickstart — your first plugin
    • Examples

      • Runnable examples
      • Example: hello-scaffold — the minimal viable plugin
      • Example: gitlab-ci-scaffold — generate a complete CI project
      • Example: steward-validator — a custom governance rule
      • Example: prod-key-guard — apply-time invariant check
    • Journeys

      • Journeys
      • Your own CI/CD

        • You have your own CI/CD setup, no problem
        • GitLab CI — the bundle template
        • GitHub Actions — the bundle template
        • Jenkins — the bundle template
        • CircleCI — the bundle template
      • You have a strict project layout, no problem
      • You have governance rules, no problem
      • You want a check at apply time, no problem
    • Reference

      • Reference
      • Roles reference
      • Entry points reference
      • Trust model
      • Packaging
      • Companion packages
  • Providers

    • Providers
    • Provider Architecture
    • GCP Provider
    • AWS Provider
    • Snowflake Provider
    • Local Provider
    • Creating Custom Providers
    • Provider Roadmap
  • AI & Agents

    • MCP Server
    • Built-in And Custom Forge Guidance
    • Forge Discovery Guide
    • Forge Memory Guide
    • Authoring Forge Tools
    • Guided fluid forge UX
    • LLM Providers
    • LiteLLM Backend
    • Capability Warnings
    • Cost Tracking
    • FLUID Forge Contract GPT Packet
    • Agentic Primitives
  • Operate & Deploy

    • Airflow Integration
    • Blueprints
    • Source-Aligned Acquisition
  • Govern & Secure

    • Governance, Compliance & the Business Case
    • Governance & Compliance
    • Network Safety
    • Credential Resolver — Security Model
  • Configuration & Reference

    • Environment Variables
    • Typed Errors
    • Typed CLI Errors
    • API Stability — fluid_build.api
  • Architecture & Releases

    • V1.5 Catalog Integration — Architecture Deep-Dive
    • V1.5 + V2 Hardening — Release Notes
  • Project

    • Contributing to Fluid Forge
    • Fluid Forge Docs Baseline: CLI 0.9.0
    • Fluid Forge Docs Baseline: CLI 0.8.11
    • Fluid Forge Docs Baseline: CLI 0.8.10
    • Fluid Forge Docs Baseline: CLI 0.8.9
    • Fluid Forge Docs Baseline: CLI 0.8.8
    • Fluid Forge Docs Baseline: CLI 0.8.7
    • Fluid Forge Docs Baseline: CLI 0.8.6
    • Fluid Forge Docs Baseline: CLI 0.8.5
    • Fluid Forge Docs Baseline: CLI 0.8.4
    • Fluid Forge Docs Baseline: CLI 0.8.3
    • Fluid Forge Docs Baseline: CLI 0.8.0
    • Fluid Forge Docs Baseline: CLI 0.7.11
    • Fluid Forge Docs Baseline: CLI 0.7.9
    • Fluid Forge v0.7.1 - Multi-Provider Export Release

Why Fluid Forge

A data product is a contract, not a pipeline. Fluid Forge turns one versioned contract.fluid.yaml into a governed, multi-cloud data product — so the same file that makes data trustworthy for your team is what an AI agent reads to consume it safely.

You write the product specification once. The CLI compiles it for your target cloud, validates it in CI before anything ships, and serves it — to people and to agents — under the same declared rules.

pip install data-product-forge
fluid init my-project --quickstart
fluid validate contract.fluid.yaml   # catch a breaking change in review, not at 2am
fluid plan contract.fluid.yaml
fluid apply contract.fluid.yaml --yes # a real, versioned data product — on your laptop, no cloud account

Two ways to read this page

Data & platform leaders: the value pillars are written in plain business terms — skim those and stop. Engineers: every pillar links to the page that proves it, and the 60-second quickstart is the fastest path to a working product.

The problem: five tools, five ways to drift

Shipping one trustworthy data product today usually means five tools and five languages:

You write…in…to declare…
the modeldbtthe schema
the infrastructureTerraformwhere it lands
the scheduleAirflowwhen it runs
the access rulesOPAwho can read it
the masking rulesa warehouse UIwhat's sensitive

That's five places for the same product to disagree. When the schema changes, someone has to remember to update all five — or production breaks at 3am. And before any of it ships there's a queue of tickets, approvals, and hand-rolled IAM. That tax is the same one the data-mesh and data-contract movements each set out to remove.

The shift: write the product, not the pipeline

Forge inverts the work. Instead of writing infrastructure code, you write the product specification — and the pipeline falls out when the CLI compiles it.

One contract.fluid.yaml declares the schema, where it's exposed and bound, how it's built and scheduled, who may read it (accessPolicy), which AI agents may use it and for what (agentPolicy), and where the data may physically live (sovereignty). Trust isn't a promise you make after the fact — it's validate → plan → apply, checked on every change.

The new twist: the contract that earns human trust is exactly what an agent needs to consume the product safely. One artifact, two consumers.

The contract carries the context

Raw data plus a query isn't enough to use a data product correctly — you also need its context: what each field means, how fresh it is, how good it is, where it came from, and who may use it for what. For decades that context lived in people's heads, wikis, and ticket threads — exactly the knowledge a senior engineer supplies in review, and exactly what a new hire (or an AI agent) doesn't have.

The industry now has a name for the gap. The 2026 framing is blunt: the model is table stakes; context is the moat. A capable model handed a real-but-ambiguous field still picks the wrong one, because nothing told it which definition the business stands behind — the difference between a smart model and one with the right context. Without that judgment layer, agents produce fluent, confident, wrong answers.

A Fluid Forge contract is that context — made machine-readable and shipped with the product:

What a consumer (human or agent) needs to knowWhere it lives in the contract
What this data meansexposes[].contract.schema — typed fields, descriptions, sensitivity (PII / PHI)
Whether to trust itdq.rules (completeness, freshness, drift) + exposes[].qos (freshness / availability SLOs)
Who may use it, and for whataccessPolicy (people & services) + agentPolicy (which models, which use-cases)
Where it came fromlineage + the SDP → ADP → CDP consumes[] chain
Where it may physically livesovereignty (jurisdiction, regulation)
Who owns itmetadata.owner + business context

Because the context travels inside the contract — versioned, validated, and portable — it doesn't rot in a wiki or get stranded inside one BI tool. When the product ships, its meaning ships with it. That's what makes the contract so powerful: a downstream consumer doesn't re-derive the meaning, and an agent reads it instead of guessing. Over fluid mcp output-port serve, the same governed surface serves the data and the context that makes it safe to act on.

The model layer is commoditizing; your encoded meaning is not. The contract is where that meaning lives — one artifact your team, your pipelines, and your agents all read the same way.

What you get

One contract, not five tools

Why it matters Collapse the five-tool, five-language stack into one specification — fewer drift incidents, fewer 3am pages, faster delivery. You change the contract and re-apply, instead of editing four systems in lockstep and hoping they agree.

A single contract.fluid.yaml carries the schema, exposes / binding (the infrastructure), the schedule, accessPolicy, agentPolicy, and sovereignty — and compiles to native provider DDL plus OpenTofu infrastructure. → What is a contract?

Trustworthy by construction, enforced in CI

Why it matters Trust becomes something you ship, not an audit you pass. A breaking change surfaces at fluid validate in code review — not after a pipeline fails at 2am.

The validate → plan → apply lifecycle is bound by cryptographic digests (bundleDigest + planDigest), re-verified before any DDL runs; a tampered plan.json is rejected outright. It's a tamper-evident version of the shift-left "catch it before production" promise. → The 11-stage production pipeline

Multi-cloud by default, not as a migration

Why it matters Most organizations are already multi-cloud — one team on Snowflake, another on BigQuery, a third on S3 + Athena. One contract works across all of them, with no per-cloud rewrite and no lock-in at the contract layer.

Change binding.platform and the same contract retargets local (DuckDB) → aws (Athena / Glue) → gcp (BigQuery) → snowflake. Every provider implements the same interface, so the compiled output changes without touching the contract. → Providers

AI agents get a contract, not raw access

Why it matters Agents act on whatever data they're handed — they don't pause to sanity-check it. Governing what an agent may read is a prerequisite for scaling agentic AI, not a compliance afterthought.

agentPolicy declares which models may use a product, and for what, in the same file as human accessPolicy. fluid mcp output-port serve then exposes a governed, read-only surface over the Model Context Protocol — binding one expose of a published product and enforcing the contract's rules on every call. → Serve a data product to AI agents

Data-mesh product thinking, without the re-org tax

Why it matters Get the data-mesh outcomes — domain ownership, reuse, faster time-to-insight — without the months of socio-technical setup and dedicated platform team that stalled most adoptions.

Native Data Mesh productType (SDP / ADP / CDP — source-, aggregate-, and consumer-aligned) sits alongside the medallion Bronze / Silver / Gold layer, with consumes[] composition rules so products build into higher-value products. → Product types

Who it's for — and who it isn't, yet

A fit if you have:

  • Two or more clouds, or a credible chance of a second one
  • Compliance pressure — SOX, GDPR, HIPAA — that makes governance non-optional
  • AI agents reading your data (often your newest and largest consumer)
  • A platform team building a self-serve contract layer for internal users
  • Data-product owners who don't want to learn five tools to ship one product

Not the right tool (yet) if you're:

  • A single-warehouse, single-team analytics shop with no governance pressure — dbt alone is simpler; adopt Forge when cross-tool drift starts to bite
  • Running sub-second streaming — Forge's model is batch and mini-batch (5-minute to 24-hour latency); for sub-second, look at Materialize / RisingWave
  • Expecting a hosted control plane today — Forge is CLI + CI; a hosted UI is on the roadmap, not shipped

We keep an honest Forge vs dbt / Dagster / Terraform / Snowpark breakdown for exactly this reason — that's how a tool earns trust.

See it for yourself

# from nothing to a deployed, validated data product — locally, no cloud account
pip install data-product-forge
fluid init my-project --quickstart && cd my-project
fluid validate contract.fluid.yaml
fluid plan contract.fluid.yaml
fluid apply contract.fluid.yaml --yes
  • Getting started — the local-first path, in a few minutes
  • Consume a data product — already have one to use? Discover it, trust its contract, and read it as a human, a pipeline, or an agent
  • Governance, compliance & the business case — for data & compliance leaders: the audit-and-trust story + an honest, per-provider enforcement matrix
  • Providers — does it support my stack? (BigQuery, Snowflake, Athena, DuckDB)
  • Vision & roadmap — what we believe, and what's shipped vs. planned
  • Apache-2.0, open source, no contributor agreement — no vendor capture at the contract layer.
Edit this page on GitHub
Last Updated: 6/26/26, 10:55 AM
Contributors: fas89
Prev
Home
Next
Getting Started