Fluid Forge
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
  • Introduction

    • Home
    • Why Fluid Forge
    • Getting Started
    • Snowflake Quickstart
    • See it run
    • Forge Data Model
    • Vision & Roadmap
    • Playground
    • FAQ
  • Concepts

    • Concepts
    • Builds, Exposes, Bindings
    • What is a contract?
    • Quality, SLAs & Lineage
    • Governance & Policy
    • Agent Policy (LLM/AI governance)
    • Providers vs Platforms
    • Fluid Forge vs alternatives
  • Data Products

    • Consume a Data Product
    • Product Types — SDP, ADP, CDP
  • Walkthroughs

    • Walkthrough: Local Development
    • Source-Aligned: Postgres → DuckDB → Parquet
    • AI Forge And Data-Model Journeys
    • Walkthrough: MCP Output Port
    • Walkthrough: Deploy to Google Cloud Platform
    • Walkthrough: Snowflake Team Collaboration
    • Declarative Airflow DAG Generation - The FLUID Way
    • Generating Orchestration Code from Contracts
    • Jenkins CI/CD for FLUID Data Products
    • Universal Pipeline
    • The 11-Stage Pipeline
    • End-to-End Walkthrough: Catalog → Contract → Transformation
  • CLI Reference

    • CLI Reference
    • Core workflow

      • fluid init
      • fluid demo
      • fluid forge
      • fluid validate
      • fluid plan
      • fluid apply
      • fluid diff
      • fluid status
    • Build & ship

      • fluid bundle
      • fluid generate
      • fluid generate artifacts
      • fluid validate-artifacts
      • fluid verify-signature
      • fluid generate iac
      • fluid generate-airflow
      • fluid generate-pipeline
      • fluid generate vector
      • fluid viz-graph
      • fluid publish
      • fluid ship
      • fluid rollback
      • fluid schedule-sync
    • AI & Agents

      • fluid ai
      • fluid agents
      • fluid mission
      • fluid mcp
      • fluid memory
      • fluid stats
      • fluid skills
    • Quality & governance

      • fluid test
      • fluid verify
      • fluid contract-tests
      • fluid contract-validation
      • fluid policy
      • fluid policy check
      • fluid policy compile
      • fluid policy apply
    • Standards & interoperability

      • fluid odps
      • fluid odps-bitol
      • fluid odcs
      • fluid export
      • fluid export-odps
      • fluid exporters
      • fluid import
      • fluid market
      • fluid datamesh-manager
    • Project & workspace

      • fluid product-new
      • fluid product-add
      • fluid workspace
      • fluid contract
      • fluid split
      • fluid config
      • fluid providers
      • fluid plugins
      • fluid provider-init
      • fluid auth
      • fluid secrets
      • fluid ide
      • fluid scaffold-ci
      • fluid scaffold-composer
      • fluid scaffold-ide
      • fluid docs
      • fluid runs
      • fluid retention
      • fluid describe
      • fluid doctor
      • fluid roadmap
      • fluid version
    • Catalog adapters

      • Source Catalog Integration (V1.5)
      • Publishing to a Catalog — Overview
      • BigQuery Catalog
      • Snowflake Horizon Catalog
      • Databricks Unity Catalog
      • Google Dataplex Catalog
      • AWS Glue Data Catalog
      • DataHub Catalog
      • Data Mesh Manager Catalog
      • OpenMetadata Catalog
    • CLI by task

      • CLI by task
      • Add quality rules
      • Add agent governance
      • Debug a failed pipeline run
      • Switch clouds with one line
  • Recipes

    • Recipes
    • Recipe — add a quality rule
    • Recipe — switch clouds with one line
    • Recipe — tag PII in your schema
    • Write a contract that consumes another contract
    • Generate per-environment overlays
  • SDK & Plugins

    • SDK & Plugins
    • Quickstart — your first plugin
    • Examples

      • Runnable examples
      • Example: hello-scaffold — the minimal viable plugin
      • Example: gitlab-ci-scaffold — generate a complete CI project
      • Example: steward-validator — a custom governance rule
      • Example: prod-key-guard — apply-time invariant check
    • Journeys

      • Journeys
      • Your own CI/CD

        • You have your own CI/CD setup, no problem
        • GitLab CI — the bundle template
        • GitHub Actions — the bundle template
        • Jenkins — the bundle template
        • CircleCI — the bundle template
      • You have a strict project layout, no problem
      • You have governance rules, no problem
      • You want a check at apply time, no problem
    • Reference

      • Reference
      • Roles reference
      • Entry points reference
      • Trust model
      • Packaging
      • Companion packages
  • Providers

    • Providers
    • Provider Architecture
    • GCP Provider
    • AWS Provider
    • Snowflake Provider
    • Local Provider
    • Creating Custom Providers
    • Provider Roadmap
  • AI & Agents

    • MCP Server
    • Built-in And Custom Forge Guidance
    • Forge Discovery Guide
    • Forge Memory Guide
    • Authoring Forge Tools
    • Guided fluid forge UX
    • LLM Providers
    • LiteLLM Backend
    • Capability Warnings
    • Cost Tracking
    • FLUID Forge Contract GPT Packet
    • Agentic Primitives
  • Operate & Deploy

    • Operating in CI
    • Production Troubleshooting
    • Airflow Integration
    • Blueprints
    • Source-Aligned Acquisition
  • Govern & Secure

    • Governance, Compliance & the Business Case
    • Governance & Compliance
    • Network Safety
    • Credential Resolver — Security Model
  • Configuration & Reference

    • Environment Variables
    • Typed Errors
    • Typed CLI Errors
    • API Stability — fluid_build.api
  • Architecture & Releases

    • V1.5 Catalog Integration — Architecture Deep-Dive
    • V1.5 + V2 Hardening — Release Notes
  • Project

    • Contributing to Fluid Forge
    • Fluid Forge Docs Baseline: CLI 0.14.0
    • Fluid Forge Docs Baseline: CLI 0.13.0
    • Fluid Forge Docs Baseline: CLI 0.12.0
    • Fluid Forge Docs Baseline: CLI 0.11.0
    • Fluid Forge Docs Baseline: CLI 0.10.0
    • Fluid Forge Docs Baseline: CLI 0.9.0
    • Fluid Forge Docs Baseline: CLI 0.8.11
    • Fluid Forge Docs Baseline: CLI 0.8.10
    • Fluid Forge Docs Baseline: CLI 0.8.9
    • Fluid Forge Docs Baseline: CLI 0.8.8
    • Fluid Forge Docs Baseline: CLI 0.8.7
    • Fluid Forge Docs Baseline: CLI 0.8.6
    • Fluid Forge Docs Baseline: CLI 0.8.5
    • Fluid Forge Docs Baseline: CLI 0.8.4
    • Fluid Forge Docs Baseline: CLI 0.8.3
    • Fluid Forge Docs Baseline: CLI 0.8.0
    • Fluid Forge Docs Baseline: CLI 0.7.11
    • Fluid Forge Docs Baseline: CLI 0.7.9
    • Fluid Forge v0.7.1 - Multi-Provider Export Release

fluid mission

Declarative, verifiable goals for a data product. A mission is a YAML file that pairs a plain-language goal with deterministic success_criteria — and the criteria, not the model, decide when the mission is done.

fluid mission check runs those criteria with zero LLM calls, so it drops straight into CI. fluid mission run adds the autonomous loop: the model plans and edits, the code-owned checks re-run against the re-read contract on disk after every cycle, and the run terminates only when they pass (or when a budget, iteration, or stall ceiling fires).

New in 0.13.0

fluid mission ships in v0.13.0. See Release Notes 0.13.0.

Syntax

fluid mission <check|run|trust|list> [options]

Exit codes are shared across check and run:

CodeMeaning
0Scorecard green — every non-advisory criterion passed.
1Scorecard red — at least one non-advisory criterion failed.
2Harness error — bad spec, untrusted spec, unreadable contract, or a failed run.

Subcommands

fluid mission check <spec> [contract]

Load the spec (trust-gated), re-read the on-disk contract, run every success criterion, and render a scorecard. No LLM is involved — this is a pure gate and safe to run in CI.

Argument / OptionDescription
<spec>Mission name (e.g. quality-coverage) or a path to a spec YAML file.
[contract]Contract to verify. Default contract.fluid.yaml.
--jsonEmit the scorecard as JSON instead of the rendered table.

fluid mission run <spec> [contract]

Run the mission autonomously: VERIFY → PLAN → EXECUTE → GATE → PROGRESS, repeating until the deterministic checks pass or a ceiling fires. Needs a configured LLM provider.

Argument / OptionDescription
<spec>Mission name or spec YAML path.
[contract]Contract to work on. Default contract.fluid.yaml.
--resumeRe-enter the newest unfinished run for this mission. VERIFY is idempotent, so resuming just re-verifies what is on disk — there is no replay.
--run-id <id>Target a specific mission run id.
--llm-provider <name>LLM provider (default: your configured provider).
--llm-model <id>LLM model id.
--workspace <dir>Workspace root for receipts and tool confinement (default: auto-detected).
--jsonEmit the outcome as JSON.

fluid mission trust <spec>

One-time, direnv-style approval of a workspace spec: records the file's sha256 in ~/.fluid/mission_trust.json. Editing the file requires re-approval. Built-ins and user-global specs (~/.fluid/missions/) are trusted implicitly — trusting them is a no-op.

fluid mission list

List every discoverable mission spec (built-in, user-global, workspace) with its trust status and description.

Examples

fluid mission list                                   # what's available, and is it trusted?
fluid mission check quality-coverage                 # zero-LLM gate against ./contract.fluid.yaml
fluid mission check gdpr-clean ./contract.fluid.yaml --json   # machine-readable, for CI
fluid mission trust .fluid/missions/my-mission.yaml  # approve a workspace spec (pins its sha256)
fluid mission run gdpr-clean                         # autonomous loop until the checks pass
fluid mission run gdpr-clean --resume                # re-enter a paused run

Built-in missions

Two flagship missions ship with the CLI.

NameGoal
gdpr-cleanEvery PII column is classified with provenance, every MCP-exposed port carries an agentPolicy with retention limits, and the contract validates.
quality-coverageEvery exposed output port carries at least one data-quality rule and the contract validates against its declared fluidVersion schema.

Spec format

A mission spec is snake_case YAML. name, description, goal, and success_criteria are the substance; budgets, gates, tools, and plan_hint configure the autonomous runner. Unknown keys are rejected loudly — a typo must never silently weaken a criterion.

name: quality-coverage
description: Reach data-quality rule coverage on every output port.
goal: >
  Every exposed output port carries at least one data-quality rule and
  the contract validates against its declared fluidVersion schema.
success_criteria:
  - check: validate
  - check: predicate
    path: "exposes[*].contract.dq.rules"
    op: exists
budgets:
  max_usd: 3.00
  max_iterations: 4
  max_wall_seconds: 1200
gates:
  destructive: ask
tools:
  allow:
    - discover_workspace
    - read_sample_schema
    - validate_contract
    - propose_contract
plan_hint:
  - inspect
  - add_dq_rules

success_criteria

Every non-advisory criterion must pass. Set advisory: true to report a criterion on the scorecard without gating on it. Three check types ship in v1:

checkWhat it does
validateRuns the same in-process schema validation fluid validate runs, with exit-0 semantics.
ai_readyReuses the AI-readiness agent. require accepts sensitive_exposes_annotated and missing_descriptions.
predicateEvaluates a dotted path over the contract dict.

The predicate mini-language is deliberately frozen: dotted paths, [*] array fan-out, and the operators eq, ne, lt, lte, gt, gte, exists, contains. No filters, no functions, no extensibility hooks. Fan-out is fail-closed — a port with no dq.rules, an empty list, or no ports at all fails the criterion.

  - check: predicate
    path: "exposes[*].policy.agentPolicy.retentionPolicy.maxRetentionDays"
    op: lte
    value: 30

budgets

KeyMeaning
max_usdHard per-run spend ceiling. Re-summed from on-disk receipts each cycle, so pause/resume cannot reset spend.
max_iterationsMaximum outer cycles.
max_wall_secondsDeadline checked before every step and every check; the remaining time becomes the per-call LLM timeout.

Overshoot is bounded but nonzero — one in-flight call can cross the line.

gates

gates.destructive is ask or deny. On a non-TTY it resolves to deny — the gate fails closed, and --yes never approves a destructive diff. This is what stops a model from satisfying "every column has a description" by deleting columns.

tools

tools.allow is an allowlist, intersected with the live tool registry inside the agent loop. The planner may reorder or drop plan_hint steps; it can never add a tool that is not on the list.

How the loop terminates

The load-bearing inversion: only the code-owned checks may declare success. There is no "finish" action available to the model. Every cycle begins by re-reading and re-hashing the contract on disk and re-running the criteria against it — so:

  • Resume is free. VERIFY is idempotent and reads only from disk. A paused, stalled, or crashed run re-enters at VERIFY with no replay machinery; the scorecard is simultaneously the termination authority and the resume pointer.
  • Self-healing is free. Failing-check diagnostics are recycled verbatim as the next cycle's repair feedback. Verification failure is the repair prompt.
  • Every proposed write passes the destructive gate before it lands.

A run's status stays inside running | paused | complete | failed. A paused run carries a pause_reason of stalled, budget, timeout, iterations, or gate_rejected.

Trust boundary

A mission spec configures autonomous execution — tool allowlist, gate mode, budgets, and the goal text that reaches the planner. A cloned repo shipping .fluid/missions/ must therefore not silently control any of it.

SourceTrust
Built-in (shipped with the CLI)Trusted implicitly.
~/.fluid/missions/ (user-global, outside any repo)Trusted implicitly.
.fluid/missions/ (workspace) and arbitrary pathsRequires fluid mission trust, pinned by the file's sha256.

The gate fails closed: an unpinned or changed spec is refused with a mission_untrusted_spec_refused WARNING before anything the spec configures takes effect. A changed file needs re-approval. There is no bypass environment variable — fluid mission trust is the only way to approve.

Receipts

fluid mission run writes to <workspace>/.fluid/missions/<run-id>/, mirroring the .fluid/agents/<run-id>/ convention:

.fluid/missions/<run-id>/
    manifest.json               run-level state (the resume pointer)
    scorecard.json              latest VERIFY result
    cycles/<n>/scorecard.json   per-cycle scorecard
    cycles/<n>/cost.json        per-cycle cost receipt
    cycles/<n>/plan.json        the planner's step list

A run id is an opaque single path segment, validated as such — this is a security boundary, not cosmetics (0.13.0 security fixes).

Using mission check in CI

check is the CI-safe half: no LLM, no network, no credentials.

- name: Mission gate
  run: |
    pip install data-product-forge==0.13.0
    fluid mission check quality-coverage contract.fluid.yaml

Non-zero exit fails the job. Add --json if you want to post the scorecard as a check annotation.

See also

  • fluid validate — the schema-validation stage the validate check wraps
  • fluid agents — the sibling .fluid/agents/<run-id>/ receipt stack
  • fluid stats — aggregate spend across runs
  • Release Notes 0.13.0
Edit this page on GitHub
Last Updated: 7/18/26, 7:02 PM
Contributors: fas89
Prev
fluid agents
Next
fluid mcp