Fluid Forge
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
Why Forge
Concepts
Get Started
  • Consume a Data Product
  • See it run
  • Demos
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • MCP Output Port — Serve to AI Agents
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Agent Policy (concept)
  • MCP Output Port — Serve to Agents
  • MCP deep-dive
  • AI-assisted authoring
  • LLM providers & backends
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
  • Introduction

    • Home
    • Why Fluid Forge
    • Getting Started
    • Snowflake Quickstart
    • See it run
    • Forge Data Model
    • Vision & Roadmap
    • Playground
    • FAQ
  • Concepts

    • Concepts
    • Builds, Exposes, Bindings
    • What is a contract?
    • Quality, SLAs & Lineage
    • Governance & Policy
    • Agent Policy (LLM/AI governance)
    • Providers vs Platforms
    • Fluid Forge vs alternatives
  • Data Products

    • Consume a Data Product
    • Product Types — SDP, ADP, CDP
  • Walkthroughs

    • Walkthrough: Local Development
    • Source-Aligned: Postgres → DuckDB → Parquet
    • AI Forge And Data-Model Journeys
    • Walkthrough: MCP Output Port
    • Walkthrough: Deploy to Google Cloud Platform
    • Walkthrough: Snowflake Team Collaboration
    • Declarative Airflow DAG Generation - The FLUID Way
    • Generating Orchestration Code from Contracts
    • Jenkins CI/CD for FLUID Data Products
    • Universal Pipeline
    • The 11-Stage Pipeline
    • End-to-End Walkthrough: Catalog → Contract → Transformation
  • CLI Reference

    • CLI Reference
    • Core workflow

      • fluid init
      • fluid demo
      • fluid forge
      • fluid validate
      • fluid plan
      • fluid apply
      • fluid diff
      • fluid status
    • Build & ship

      • fluid bundle
      • fluid generate
      • fluid generate artifacts
      • fluid validate-artifacts
      • fluid verify-signature
      • fluid generate iac
      • fluid generate-airflow
      • fluid generate-pipeline
      • fluid viz-graph
      • fluid publish
      • fluid ship
      • fluid rollback
      • fluid schedule-sync
    • AI & Agents

      • fluid ai
      • fluid agents
      • fluid mcp
      • fluid memory
      • fluid stats
      • fluid skills
    • Quality & governance

      • fluid test
      • fluid verify
      • fluid contract-tests
      • fluid contract-validation
      • fluid policy
      • fluid policy check
      • fluid policy compile
      • fluid policy apply
    • Standards & interoperability

      • fluid odps
      • fluid odps-bitol
      • fluid odcs
      • fluid export
      • fluid export-odps
      • fluid exporters
      • fluid import
      • fluid market
      • fluid datamesh-manager
    • Project & workspace

      • fluid product-new
      • fluid product-add
      • fluid workspace
      • fluid contract
      • fluid split
      • fluid config
      • fluid providers
      • fluid plugins
      • fluid provider-init
      • fluid auth
      • fluid secrets
      • fluid ide
      • fluid scaffold-ci
      • fluid scaffold-composer
      • fluid scaffold-ide
      • fluid docs
      • fluid runs
      • fluid retention
      • fluid describe
      • fluid doctor
      • fluid roadmap
      • fluid version
    • Catalog adapters

      • Source Catalog Integration (V1.5)
      • Publishing to a Catalog — Overview
      • BigQuery Catalog
      • Snowflake Horizon Catalog
      • Databricks Unity Catalog
      • Google Dataplex Catalog
      • AWS Glue Data Catalog
      • DataHub Catalog
      • Data Mesh Manager Catalog
      • OpenMetadata Catalog
    • CLI by task

      • CLI by task
      • Add quality rules
      • Add agent governance
      • Debug a failed pipeline run
      • Switch clouds with one line
  • Recipes

    • Recipes
    • Recipe — add a quality rule
    • Recipe — switch clouds with one line
    • Recipe — tag PII in your schema
    • Write a contract that consumes another contract
    • Generate per-environment overlays
  • SDK & Plugins

    • SDK & Plugins
    • Quickstart — your first plugin
    • Examples

      • Runnable examples
      • Example: hello-scaffold — the minimal viable plugin
      • Example: gitlab-ci-scaffold — generate a complete CI project
      • Example: steward-validator — a custom governance rule
      • Example: prod-key-guard — apply-time invariant check
    • Journeys

      • Journeys
      • Your own CI/CD

        • You have your own CI/CD setup, no problem
        • GitLab CI — the bundle template
        • GitHub Actions — the bundle template
        • Jenkins — the bundle template
        • CircleCI — the bundle template
      • You have a strict project layout, no problem
      • You have governance rules, no problem
      • You want a check at apply time, no problem
    • Reference

      • Reference
      • Roles reference
      • Entry points reference
      • Trust model
      • Packaging
      • Companion packages
  • Providers

    • Providers
    • Provider Architecture
    • GCP Provider
    • AWS Provider
    • Snowflake Provider
    • Local Provider
    • Creating Custom Providers
    • Provider Roadmap
  • AI & Agents

    • MCP Server
    • Built-in And Custom Forge Guidance
    • Forge Discovery Guide
    • Forge Memory Guide
    • Authoring Forge Tools
    • Guided fluid forge UX
    • LLM Providers
    • LiteLLM Backend
    • Capability Warnings
    • Cost Tracking
    • FLUID Forge Contract GPT Packet
    • Agentic Primitives
  • Operate & Deploy

    • Airflow Integration
    • Blueprints
    • Source-Aligned Acquisition
  • Govern & Secure

    • Governance, Compliance & the Business Case
    • Governance & Compliance
    • Network Safety
    • Credential Resolver — Security Model
  • Configuration & Reference

    • Environment Variables
    • Typed Errors
    • Typed CLI Errors
    • API Stability — fluid_build.api
  • Architecture & Releases

    • V1.5 Catalog Integration — Architecture Deep-Dive
    • V1.5 + V2 Hardening — Release Notes
  • Project

    • Contributing to Fluid Forge
    • Fluid Forge Docs Baseline: CLI 0.9.0
    • Fluid Forge Docs Baseline: CLI 0.8.11
    • Fluid Forge Docs Baseline: CLI 0.8.10
    • Fluid Forge Docs Baseline: CLI 0.8.9
    • Fluid Forge Docs Baseline: CLI 0.8.8
    • Fluid Forge Docs Baseline: CLI 0.8.7
    • Fluid Forge Docs Baseline: CLI 0.8.6
    • Fluid Forge Docs Baseline: CLI 0.8.5
    • Fluid Forge Docs Baseline: CLI 0.8.4
    • Fluid Forge Docs Baseline: CLI 0.8.3
    • Fluid Forge Docs Baseline: CLI 0.8.0
    • Fluid Forge Docs Baseline: CLI 0.7.11
    • Fluid Forge Docs Baseline: CLI 0.7.9
    • Fluid Forge v0.7.1 - Multi-Provider Export Release

fluid publish

Stage 10 of the 11-stage pipeline. Publish one or more contracts to one or more catalogs.

0.8.0 renames the single-valued --catalog X,Y flag to the repeatable --target <name>[:<endpoint>]. Each --target invocation pushes to one catalog; pass the flag multiple times to push to several at once. --catalog is kept as a deprecation-aliased surface for one release.

Syntax

fluid publish CONTRACT_FILES

CONTRACT_FILES supports one or more paths or glob patterns.

Key options

OptionDescription
--target, -t (repeatable)Target catalog name, with an optional :<endpoint> suffix to override the catalog's default URL. Built-in targets include fluid-command-center, datamesh-manager, dmm, and entropy-data; additional catalog providers can be configured by name. Pass multiple times to push to several catalogs.
--catalog, -cDeprecated one-release alias for --target. Emits a warning; treat it as historical.
--list-catalogsList configured catalogs
--dry-runValidate and preview without publishing
--verify-onlyCheck whether a contract is already published
--forceForce an update
--format, -fOutput format
--verbose, -vDetailed output
--quiet, -qMinimal output
--skip-health-checkSkip catalog health checks
--show-metricsShow detailed metrics

Examples

Single target

fluid publish contract.fluid.yaml --target fluid-command-center

Multiple targets in one call

fluid publish contract.fluid.yaml \
  --target fluid-command-center \
  --target datamesh-manager

The publish report contains a per-target result block, so a partial failure (e.g. DataHub auth problem but DMM succeeded) is distinguishable from a full failure.

Endpoint override (self-hosted catalogs)

fluid publish contract.fluid.yaml \
  --target fluid-command-center:https://cc.internal.acme.com \
  --target datamesh-manager:https://dmm.internal.acme.com

Glob input

fluid publish customer-*.fluid.yaml --target datamesh-manager

Dry-run preview

fluid publish contract.fluid.yaml --target datamesh-manager --dry-run

Notes

  • A typical flow is validate → plan → apply → verify → publish.
  • Use fluid market to verify discoverability after publishing.
  • The legacy form --catalog a,b,c (comma-separated) is not equivalent to --target a --target b --target c; the new flag is repeatable instead of comma-separated for consistency with kubectl / helm / gh conventions.

Publishing to Data Mesh Manager (Entropy Data)

Data Mesh Manager (now Entropy Data) is one of the catalogs Fluid Forge publishes to. It has its own dedicated entry point so data products and data contracts can be published with the right payload shape:

fluid datamesh-manager publish CONTRACT   # or: fluid dmm publish CONTRACT

Setup

Env varRequiredPurpose
DMM_API_KEY✅ YesAPI key. Generate at Profile → Organization → Settings → API Keys.
DMM_API_URLNoBase URL override (default: https://api.entropy-data.com).
DMM_ODPS_LINEAGE_MODENocontract (default) uses Entropy Access agreements for product-to-product lineage; source-system is legacy compatibility mode.
DMM_AUTO_APPROVE_ACCESSNoSet to true only when Access agreements should be approved automatically.
DMM_ALLOW_INSECURE_HTTPNoSet to true only when intentionally publishing to a non-local HTTP endpoint.

Key options

OptionDescription
CONTRACTPath to contract.fluid.yaml (positional, required)
--dry-runValidate and preview the API payload without publishing
--with-contractPublish a companion data contract alongside the data product
--team-id IDOverride team-id resolution; auto-creates the team if it doesn't exist
--data-product-spec VALUEOverride the data product payload spec. Use odps for Entropy ODPS publishes.
--odps-lineage-mode {contract,source-system}Choose ODPS lineage behavior. contract is the default.
--auto-approve-accessApprove generated Access agreements immediately. Intended for local sandboxes, not review-based production flows.
--validation-mode {warn,strict}Gate on pre-publish schema validation. warn (default) logs and continues; strict aborts on any violation. Both modes validate against the contract's own declared fluidVersion, so a 0.5.7 contract is validated against fluid-schema-0.5.7.json, a 0.7.2 contract against fluid-schema-0.7.2.json, etc. — upgrading the CLI never invalidates a contract that was valid against its own version.

Examples

export DMM_API_KEY="..."
fluid dmm publish contract.fluid.yaml
fluid dmm publish contract.fluid.yaml --dry-run
fluid dmm publish contract.fluid.yaml --with-contract
fluid dmm publish contract.fluid.yaml --data-product-spec odps --odps-lineage-mode contract
fluid dmm publish contract.fluid.yaml --validation-mode strict

What gets sent

  • Data product via PUT /api/dataproducts/{id} — built from FLUID metadata, exposes, consumes.
  • Data contract (optional, with --with-contract) via PUT /api/datacontracts/{id} — the ODCS v3.1.0 payload of the contract.
  • Access agreements from product-to-product consumes[] entries. This is the default ODPS lineage path and prevents duplicated SourceSystem graph nodes.
  • Input / output ports mapped from FLUID consumes[] / exposes[]. Product-to-product consumes are Access-only in ODPS mode; explicit source-system consumes remain input ports.
  • PII detection from schema[].sensitivity / classification fields.
  • Multi-provider location mapping — BigQuery, Snowflake, S3, Kafka, Redshift, etc.
  • Safe team bootstrap — if DMM rejects team members because users do not exist yet, the provider retries team creation without members while keeping the contact email.
  • Retries with backoff for transient failures (429, 5xx).

Catalog-adapter route

The same behavior is reachable through the generic catalog surface:

fluid publish contract.fluid.yaml --target datamesh-manager

Use the dedicated fluid dmm publish entry point when you want --validation-mode, --with-contract, --odps-lineage-mode, or --auto-approve-access. Use fluid publish --target datamesh-manager when you're treating DMM as one catalog among several behind a common interface.

Edit this page on GitHub
Last Updated: 4/24/26, 11:39 PM
Contributors: Jeff Watson, jeffwatson-ai, fas89
Prev
fluid viz-graph
Next
fluid ship