Fluid Forge
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
  • Introduction

    • Home
    • Why Fluid Forge
    • Getting Started
    • Snowflake Quickstart
    • See it run
    • Forge Data Model
    • Vision & Roadmap
    • Playground
    • FAQ
  • Concepts

    • Concepts
    • Builds, Exposes, Bindings
    • What is a contract?
    • Quality, SLAs & Lineage
    • Governance & Policy
    • Agent Policy (LLM/AI governance)
    • Providers vs Platforms
    • Fluid Forge vs alternatives
  • Data Products

    • Consume a Data Product
    • Product Types — SDP, ADP, CDP
  • Walkthroughs

    • Walkthrough: Local Development
    • Source-Aligned: Postgres → DuckDB → Parquet
    • AI Forge And Data-Model Journeys
    • Walkthrough: MCP Output Port
    • Walkthrough: Deploy to Google Cloud Platform
    • Walkthrough: Snowflake Team Collaboration
    • Declarative Airflow DAG Generation - The FLUID Way
    • Generating Orchestration Code from Contracts
    • Jenkins CI/CD for FLUID Data Products
    • Universal Pipeline
    • The 11-Stage Pipeline
    • End-to-End Walkthrough: Catalog → Contract → Transformation
  • CLI Reference

    • CLI Reference
    • Core workflow

      • fluid init
      • fluid demo
      • fluid forge
      • fluid validate
      • fluid plan
      • fluid apply
      • fluid diff
      • fluid status
    • Build & ship

      • fluid bundle
      • fluid generate
      • fluid generate artifacts
      • fluid validate-artifacts
      • fluid verify-signature
      • fluid generate iac
      • fluid generate-airflow
      • fluid generate-pipeline
      • fluid generate vector
      • fluid viz-graph
      • fluid publish
      • fluid ship
      • fluid rollback
      • fluid schedule-sync
    • AI & Agents

      • fluid ai
      • fluid agents
      • fluid mission
      • fluid mcp
      • fluid memory
      • fluid stats
      • fluid skills
    • Quality & governance

      • fluid test
      • fluid verify
      • fluid contract-tests
      • fluid contract-validation
      • fluid policy
      • fluid policy check
      • fluid policy compile
      • fluid policy apply
    • Standards & interoperability

      • fluid odps
      • fluid odps-bitol
      • fluid odcs
      • fluid export
      • fluid export-odps
      • fluid exporters
      • fluid import
      • fluid market
      • fluid datamesh-manager
    • Project & workspace

      • fluid product-new
      • fluid product-add
      • fluid workspace
      • fluid contract
      • fluid split
      • fluid config
      • fluid providers
      • fluid plugins
      • fluid provider-init
      • fluid auth
      • fluid secrets
      • fluid ide
      • fluid scaffold-ci
      • fluid scaffold-composer
      • fluid scaffold-ide
      • fluid docs
      • fluid runs
      • fluid retention
      • fluid describe
      • fluid doctor
      • fluid roadmap
      • fluid version
    • Catalog adapters

      • Source Catalog Integration (V1.5)
      • Publishing to a Catalog — Overview
      • BigQuery Catalog
      • Snowflake Horizon Catalog
      • Databricks Unity Catalog
      • Google Dataplex Catalog
      • AWS Glue Data Catalog
      • DataHub Catalog
      • Data Mesh Manager Catalog
      • OpenMetadata Catalog
    • CLI by task

      • CLI by task
      • Add quality rules
      • Add agent governance
      • Debug a failed pipeline run
      • Switch clouds with one line
  • Recipes

    • Recipes
    • Recipe — add a quality rule
    • Recipe — switch clouds with one line
    • Recipe — tag PII in your schema
    • Write a contract that consumes another contract
    • Generate per-environment overlays
  • SDK & Plugins

    • SDK & Plugins
    • Quickstart — your first plugin
    • Examples

      • Runnable examples
      • Example: hello-scaffold — the minimal viable plugin
      • Example: gitlab-ci-scaffold — generate a complete CI project
      • Example: steward-validator — a custom governance rule
      • Example: prod-key-guard — apply-time invariant check
    • Journeys

      • Journeys
      • Your own CI/CD

        • You have your own CI/CD setup, no problem
        • GitLab CI — the bundle template
        • GitHub Actions — the bundle template
        • Jenkins — the bundle template
        • CircleCI — the bundle template
      • You have a strict project layout, no problem
      • You have governance rules, no problem
      • You want a check at apply time, no problem
    • Reference

      • Reference
      • Roles reference
      • Entry points reference
      • Trust model
      • Packaging
      • Companion packages
  • Providers

    • Providers
    • Provider Architecture
    • GCP Provider
    • AWS Provider
    • Snowflake Provider
    • Local Provider
    • Creating Custom Providers
    • Provider Roadmap
  • AI & Agents

    • MCP Server
    • Built-in And Custom Forge Guidance
    • Forge Discovery Guide
    • Forge Memory Guide
    • Authoring Forge Tools
    • Guided fluid forge UX
    • LLM Providers
    • LiteLLM Backend
    • Capability Warnings
    • Cost Tracking
    • FLUID Forge Contract GPT Packet
    • Agentic Primitives
  • Operate & Deploy

    • Operating in CI
    • Production Troubleshooting
    • Airflow Integration
    • Blueprints
    • Source-Aligned Acquisition
  • Govern & Secure

    • Governance, Compliance & the Business Case
    • Governance & Compliance
    • Network Safety
    • Credential Resolver — Security Model
  • Configuration & Reference

    • Environment Variables
    • Typed Errors
    • Typed CLI Errors
    • API Stability — fluid_build.api
  • Architecture & Releases

    • V1.5 Catalog Integration — Architecture Deep-Dive
    • V1.5 + V2 Hardening — Release Notes
  • Project

    • Contributing to Fluid Forge
    • Fluid Forge Docs Baseline: CLI 0.14.0
    • Fluid Forge Docs Baseline: CLI 0.13.0
    • Fluid Forge Docs Baseline: CLI 0.12.0
    • Fluid Forge Docs Baseline: CLI 0.11.0
    • Fluid Forge Docs Baseline: CLI 0.10.0
    • Fluid Forge Docs Baseline: CLI 0.9.0
    • Fluid Forge Docs Baseline: CLI 0.8.11
    • Fluid Forge Docs Baseline: CLI 0.8.10
    • Fluid Forge Docs Baseline: CLI 0.8.9
    • Fluid Forge Docs Baseline: CLI 0.8.8
    • Fluid Forge Docs Baseline: CLI 0.8.7
    • Fluid Forge Docs Baseline: CLI 0.8.6
    • Fluid Forge Docs Baseline: CLI 0.8.5
    • Fluid Forge Docs Baseline: CLI 0.8.4
    • Fluid Forge Docs Baseline: CLI 0.8.3
    • Fluid Forge Docs Baseline: CLI 0.8.0
    • Fluid Forge Docs Baseline: CLI 0.7.11
    • Fluid Forge Docs Baseline: CLI 0.7.9
    • Fluid Forge v0.7.1 - Multi-Provider Export Release

fluid verify

Stage 9 of the 11-stage pipeline. Multi-dimensional validation that deployed infrastructure still matches the FLUID contract — schema, data types, constraints, and region all checked independently with severity-based drift assessment.

Syntax

fluid verify CONTRACT

Key options

OptionDescription
--expose, --expose-idVerify only a specific expose
--strictExit non-zero when mismatches are found
--outWrite the JSON verification report to this path. No default — omit it and no report file is written.
--show-diffsShow field-by-field differences
--reconcile-dbt(since 0.11.0) Cross-check the contract schema against the build's dbt project (models/**/schema.yml) and flag drift. Static, warehouse-free.
--reconcile-lineage(since 0.12.0) Cross-check declared lineage against local run evidence and the catalog publish payload. Local-only, no network.
--warn-onlyDowngrade --reconcile-dbt / --reconcile-lineage drift to a warning (exit 0)
--envApply an environment overlay

Dimensional analysis

Each expose is checked across four dimensions:

DimensionWhat it compares
Schema structureColumn names and count vs. the contract
Data typesEach column's warehouse type vs. the FLUID type
Constraintsnullable vs. required
LocationRegion / location matches binding.location.region

Each mismatch is scored for severity:

SeverityWhen it firesRemediation
🔴 CRITICALMissing fields, type mismatches, or region mismatchManual intervention (table recreation / migration).
🟡 WARNINGConstraint mismatches (nullable / required)Manual recommended; non-breaking but worth addressing.
🔵 INFOExtra fields in the table not in the contractAuto-fixable; update the contract if intentional.
🟢 SUCCESSEverything matchesNo action.

Reference-only contracts

For contracts where builds[].pattern is hybrid-reference, reference, or external-reference, the target tables are materialised by an externally-owned dbt or Airflow project — NOT by fluid apply. On the first pipeline run the external DAG hasn't run yet, so a "table not found" result is expected state, not a verification failure.

fluid verify detects this mode via builds[].pattern and downgrades missing-table errors to INFO under --strict:

(contract is reference-only — missing tables will be reported as INFO,
not treated as verification failures)

📋 Verifying: subscriber360_core
   Format: snowflake_table
   Target: TELCO_LAB.TELCO_FLUID_DEMO.SUBSCRIBER360_CORE_V1
   🔵 INFO: Table not found: ... (reference-only — external pipeline owns creation)

The downgrade is narrowly scoped:

  • Only missing-table shapes (result["exists"] is False) are downgraded.
  • Auth / config / connection errors still hard-fail under --strict.
  • Non-reference contracts (imperative / declarative) keep the original hard-fail behaviour.

Reconcile: contract ↔ dbt schema (--reconcile-dbt)

(since 0.11.0) A static, warehouse-free cross-check that the contract's promised columns (exposes[].contract.schema) agree with what the build's dbt project declares (models/**/schema.yml). It never connects to a warehouse and never runs dbt. Drift it surfaces:

  • a column the contract promises that dbt doesn't model,
  • a dbt column the contract never exposes,
  • a declared type that disagrees (compared conservatively via coarse type families, so NUMBER(38,0) vs integer is not flagged across adapters).

Drift exits non-zero unless --warn-only is set — independent of --strict (you asked for the check, so its drift gates on its own). The JSON report (--out) carries the detail under report["reconcile"].

Reconcile: contract ↔ published lineage (--reconcile-lineage)

(since 0.12.0) A local-only cross-check (no network) that the contract's declared lineage — consumes[] upstream refs and exposes[] output ports — agrees with:

  1. Observed run evidence — the run records and cursor state the build runners persist under .fluid/ (which source streams were actually read), and
  2. The publish payload — the exact lineage edges the catalog registrar would push, rebuilt locally from the contract.

Three drift classes:

Drift classSeverityMeaning
declared_but_never_readsoftA consumes[] entry with no run/cursor evidence. Never fails — the product may simply not have run yet; with no run evidence at all the check degrades to a note.
read_but_undeclaredcriticalA stream a runner actually read that is neither a consumes[] ref nor a declared acquisition source stream. Data flowed in that the contract never admits to.
publish_payload_mismatchcriticalThe lineage edges/assets the registrar would publish disagree with the contract.

Critical drift fails the build under --strict unless --warn-only is set; soft drift never fails. dbt run records are handled honestly: their streams are executed nodes, not upstream reads, so they are excluded from the undeclared-read check (with a note) instead of producing false positives. The JSON report carries the detail under report["reconcile_lineage"].

dbt test results (transformation checks)

(since 0.12.0) When the product's build ran via dbt (fluid apply --mode amend-and-build), the runner parses target/run_results.json into run records, and fluid verify gains transformation-side checks next to the acquisition probes: dbt_tests_passed and no_error_severity_failures. Both are critical, so a failing contract-derived dbt test gates the exit code under --strict. See fluid runs status for the run-record view of the same data.

Examples

fluid verify contract.fluid.yaml
fluid verify contract.fluid.yaml --strict
fluid verify contract.fluid.yaml --expose bitcoin_prices
fluid verify contract.fluid.yaml --show-diffs
fluid verify contract.fluid.yaml --env prod --out runtime/verify-report.json --strict
fluid verify contract.fluid.yaml --reconcile-dbt --strict          # contract ↔ dbt schema drift
fluid verify contract.fluid.yaml --reconcile-lineage --strict      # contract ↔ observed/published lineage
fluid verify contract.fluid.yaml --reconcile-dbt --reconcile-lineage --warn-only   # report, never fail

Notes

  • Use verify after apply or in CI when you need contract-to-runtime confidence.
  • Use fluid test when you want broader live-resource validation.
  • The JSON report contains per-expose severity + dimensional breakdown + remediation actions. CI dashboards can key off result["severity"]["level"] for red/amber/green status.
Edit this page on GitHub
Last Updated: 7/18/26, 8:34 AM
Contributors: Jeff Watson, fas89, Claude Opus 4.7 (1M context)
Prev
fluid test
Next
fluid contract-tests