Publishing to a Catalog — Overview
v0.8.3 consolidated catalog publishing under one registry. Contracts opt in via a single key in properties.catalog.register; the matching registrar is constructed from environment configuration at apply / publish time. Three publish-side backends are active today.
Looking for source-side catalog reading (forge a contract from existing catalog metadata)? That's a different flow — see the catalog index for the
fluid forge data-model from-sourceadapters (Snowflake, Databricks Unity, BigQuery, Dataplex, AWS Glue, DataHub, Data Mesh Manager).
Opting a contract in
Add register to your contract's properties.catalog block:
# contract.fluid.yaml
properties:
catalog:
register: [datahub, datamesh_manager] # one or more publish-side backends
That's the entire contract-side surface. The acquisitionCatalog schema block carries only target names (it's additionalProperties: false) — endpoints, tokens, and other deployment config are read from the environment.
Active publish-side backends
| Backend | Reference | Notes |
|---|---|---|
datahub | Publishing to DataHub | DataHub GMS REST + MCP. Emits DataProduct + Domain + Datasets + DataContract with full schema, ownership, tags, descriptions. FLUID_LAYER_PROPERTY_ID / FLUID_PRODUCT_TYPE_PROPERTY_ID structured properties surface medallion classification. |
openmetadata | Publishing to OpenMetadata | OpenMetadata REST (Tables + extension fields). |
datamesh_manager | Publishing to Data Mesh Manager | Bidirectional. PUT /api/dataproducts/{id} in ODPS + PUT /api/datacontracts/{product_id}.{expose_id} in ODCS per asset. Emits proper SourceSystem lineage links (Phase 1 of the DMM flow); per-port contractId resolves to a sibling ODCS contract. |
How dispatch works
The publish stage (build_runners/_catalog.py) reads properties.catalog.register from the contract, calls build_registrar(target) for each entry to construct the registrar from env config, and pushes a uniform CatalogPublicationPayload to every backend. The shared payload shape means a contract that publishes cleanly to one backend publishes cleanly to all.
Retired registrars — Glue + Snowflake Horizon
The previously-shipped glue and snowflake_horizon publish-side registrars were retired in v0.8.3 (PR #140). The same metadata they used to push at publish time (table descriptions, per-column comments, FLUID classification tags, contract YAML) is now folded into the IaC layer and emitted into aws_glue_catalog_table and snowflake_table resources directly:
- One source of truth (the contract → IaC pipeline owns the catalog metadata along with the table itself).
- Drift detection comes free from
tofu plan. - No more out-of-band registrar writes fighting IaC state.
Contracts that still list glue or snowflake_horizon under properties.catalog.register will get a "not configured" result from build_registrar. The migration is:
- Drop
glue/snowflake_horizonfromproperties.catalog.register. - Run
fluid applyagainst the cloud provider as usual —aws/gcp/snowflakeproviders auto-route through OpenTofu, which now owns the catalog metadata. Seefluid generate iacfor the emit-and-review path. aws_glue_catalog_table.parametersandsnowflake_tablecomments now carry the same metadata.
Databricks Unity Catalog
A Databricks Unity Catalog read adapter still exists under fluid_build/copilot/catalog/unity.py and powers fluid forge data-model from-source --source unity (see the Unity catalog adapter page). The publish-side Unity registrar was dropped — the OSS Unity Catalog server's strict v0.4+ table-create validation made round-tripping the canonical payload too fragile for a generic publish path. Databricks-hosted UC remains addressable via the upstream Databricks SDK if needed.
See also
fluid publish— the user-facing publish command- Source-side catalog index — forge contracts from catalog metadata
- Network safety — SSRF posture on catalog HTTP calls
fluid generate iac— the IaC layer that absorbed Glue + Snowflake Horizon metadata