Providers
Fluid Forge uses one contract format across local and provider-backed execution targets.
Why it matters Target a new cloud by editing
binding, not the rest of the contract. Change an expose'sbinding(itsplatform,formatandlocation) and the same contract compiles forlocal(DuckDB),aws,gcporsnowflake.
Docs baseline
- CLI release covered by the primary docs:
0.18.1 - Contract schema:
0.7.5is the current stable version;0.7.6is a preview you opt into withfluidVersion: "0.7.6". The GCP retention, encryption andbinding.principalsfields need0.7.6. - Which
fluidVersioneach scaffolding command writes is listed influid init.
Provider overview
fluid providers reports aws, datamesh_manager, gcp, local, redshift and snowflake. These have a provider page:
| Provider | Plan / Apply | Scheduling docs stance | Status |
|---|---|---|---|
| GCP | Yes (OpenTofu) | Prefer fluid generate schedule | Production |
| AWS | Yes (OpenTofu) | Prefer fluid generate schedule | Production |
| Snowflake | Yes (OpenTofu) | Prefer fluid generate schedule | Production |
| Local | Yes (native DuckDB) | Local-first onboarding | Production |
"Production" covers provisioning and builds. Governance differs by cloud: on GCP, fluid apply emits dataset grants, column policy tags and, on 0.7.6, retention and Cloud KMS keys; on Snowflake, as of 0.18.1, accessPolicy grants, column restrictions and masking are not applied. Each provider page lists what its module emits.
ODCS / ODPS are spec exporters, not providers. As of
v0.10.0the open-standards exports (ODCS, ODPS, ODPS-Bitol) are surfaced byfluid exporters— they serialize a contract to a spec and do not deploy infrastructure, so they no longer appear in thefluid providersroster.
Runtime requirement for cloud apply
Since v0.10.0, fluid apply against aws / gcp / snowflake auto-compiles the contract to OpenTofu and delegates to the tofu binary — install tofu ≥ 1.6.0 on PATH. local keeps its native DuckDB apply, no tofu needed. See fluid generate iac.
The CLI surface today is asymmetric for the two spec exporters — fluid odcs exposes export / import / validate / info, while fluid odps-bitol exposes only export / validate / info. The unified fluid odps command covers both specs and adds an import subcommand for Bitol — see fluid odps and fluid odcs.
Compatibility note: fluid generate-airflow still exists, but the primary docs path is fluid generate schedule --scheduler airflow.
Quick start by provider
Each snippet assumes the contract's own binding names that cloud. --provider disambiguates a contract that spans clouds or declares none — it does not retarget one, and since 0.15.0 a --provider that contradicts every cloud the contract declares is rejected before anything is written, on both fluid apply and fluid generate iac. To move a product between clouds, edit binding: the switch-clouds recipe shows the diff, and the sovereignty-platform-swap example carries the same product for AWS, GCP and Snowflake; strip the binding: block from the three files and the remainder is identical.
GCP
pip install "data-product-forge[gcp,local]"
gcloud auth application-default login
fluid apply contract.fluid.yaml --provider gcp --yes
The project and region come from each binding's location.project and location.region; see GCP.
AWS
aws configure
fluid apply contract.fluid.yaml --provider aws --yes
Snowflake
export SNOWFLAKE_ACCOUNT=your_account
export SNOWFLAKE_USER=your_user
fluid apply contract.fluid.yaml --provider snowflake --yes
Local
pip install "data-product-forge[local]"
Then follow the local provider's quick start.
Standards and catalogs
Beyond the cloud and local providers, Fluid Forge round-trips contracts against public data-product standards and publishes them to catalogs.
The full list of spec-export formats is surfaced by fluid exporters. Exporters serialize a contract to a spec and are not cloud providers — for deployment targets see fluid providers.
Standards exchange — fluid odps
The unified fluid odps command dispatches both export and import across the ODPS specs via --spec. The spec exporters themselves are bidirectional at the exporter layer (render + import_contract + validate).
| Spec | Command | What it is |
|---|---|---|
| ODPS — Bitol 1.0.0 | fluid odps export … --spec bitol-1.0.0 / fluid odps import | Bitol variant — 1 ODPS doc + N sibling ODCS contracts (one per output port). |
| ODPS — v4.1 | fluid odps export … --spec odps-v4.1 | Open Data Product Initiative single-file variant. Export-only. |
| ODCS | fluid odcs export / fluid odcs import | Open Data Contract Standard v3.1.0 (Bitol.io). |
Exporter-specific entry points also remain: fluid odps-bitol for the Bitol layout. New scripts should prefer fluid odps with --spec.
Publishing — catalog registrars
v0.8.3 consolidated catalog publishing under one registry. Contracts opt in via properties.catalog.register: [<name>]; three publish-side registrars are active:
| Catalog | Reference |
|---|---|
| DataHub | catalog overview → DataHub publish |
| OpenMetadata | OpenMetadata publish |
| Data Mesh Manager / Entropy Data | DMM publish |
| FLUID Command Center | fluid publish; since 0.17.0 fluid apply also reports each run to the Command Center configured for fluid publish (best effort, never changes the exit code; FLUID_COMMAND_CENTER_ENABLED=false turns it off) |
New in 0.15.0
DataHub customProperties keys lose their dots. fluid.layer → fluid_layer, fluid.productType → fluid_product_type, fluid.version → fluid_version, plus a new fluid_domain — the underscore spelling every other emitter already used, so one property is spelled the same way whichever catalog an analyst is browsing. A saved search, dashboard or ingestion rule keyed on the dotted names must be updated. DataHub structured properties keep their dotted qualifiedName; that is a separate namespace, not an inconsistency.
The same release makes a published contract readable on a default OSS install at all. The ODCS document was linked rather than inlined, but the link resolves only when spec_source_base_url is configured — unset unless an operator supplies it via FLUID_CATALOG_DATAHUB_SPEC_BASE_URL or the catalog config key — and DataContract.rawContract is absent from the OSS GraphQL schema, so on a stock install the contract was neither inlined, nor linked, nor readable. Large specs are now linked when a base URL is set and inlined when it is not, so expect larger entity payloads with no base URL configured: the dataset aspect carries odcs_contract, and the DataProduct aspect fluid_contract and odps_spec. See DataHub publish.
OpenMetadata gained the read half of the loop in the same release — OpenMetadataRegistrar.fetch_odcs_contract(fqn) pulls an ODCS contract back out of the catalog, preferring the verbatim copy fluid published over OpenMetadata's native ODCS export, whose converter drops servers and six other top-level blocks. No CLI command calls it yet: it is a library method, and nothing about fluid publish changes.
The previously-shipped glue and snowflake_horizon registrars were retired in v0.8.3 and folded into the IaC layer — catalog metadata for those targets is now emitted as aws_glue_catalog_table / snowflake_table resources via fluid generate iac. One source of truth, drift-detected by tofu plan.
Notes
- Publishing one contract from two clouds (
--env aws, then--env gcp) updates one Command Center product, keyed by the contract id, so the last publish sets its platform and location.fluid generate ci --no-publish-stage-defaultleaves the publish stage off for a second cloud's pipeline. - Use the provider pages (GCP, AWS, Snowflake, Local) when you need deep target details.
- Use CLI Reference for command syntax.
- Use Getting Started for the local-first workflow.
Need a hand with a specific provider? Start a discussion or open an issue.