Contract field reference, schema 0.7.6 preview
Status: preview. A contract opts in with fluidVersion: "0.7.6"; a contract that declares 0.7.5 is unaffected. This page lists the difference between the 0.7.5 schema and the 0.7.6 schema as bundled with the CLI: added fields, fields whose definition changed, and fields removed. A field not listed here has the same type, required flag, allowed values, default and description in both; for those, use the 0.7.5 reference. A definition used at several paths is listed once, at its first path.
Generated from fluid_build/schemas/fluid-schema-0.7.6.json in the data-product-forge 0.18.1 package (sha256 772bf80070a9409f). It is the schema fluid validate checks a contract against, rendered without edits. To regenerate: python scripts/gen_contract_reference.py.
Each table row is one field, addressed by its path from the contract root: . descends into an object, [] marks the items of a list, and <key> stands for any key of a map. Required means required inside its parent object; the parent can itself be optional. Allowed values lists the enum, pattern and range the schema enforces. An object typed plainly object rejects keys the schema does not list; object (open) accepts extra keys and object (free-form) has no listed keys. A row that begins "Only when ..." exists only under that condition. See how to read this reference.
At a glance
- Added: 48 rows
- Changed: 12 rows
- Removed: 0 rows
Added fields
exposes
| Path | Type | Required | Allowed values | Default | Description |
|---|---|---|---|---|---|
exposes[].policy.privacy.masking[].params.saltEnv | string | no | matches ^[A-Z_][A-Z0-9_]{0,127}$ | "FLUID_PII_HASH_SECRET" | NEW in v0.7.6: hash only. The environment variable holding the salt, at least 16 bytes. Unset or empty, the build is refused: an unsalted SHA-256 of a phone number is reversed by enumerating the numbering plan. |
exposes[].policy.privacy.masking[].params.keyEnv | string | no | matches ^[A-Z_][A-Z0-9_]{0,127}$ | NEW in v0.7.6: tokenize and encrypt. The environment variable holding the key. tokenize defaults to FLUID_PII_TOKENIZATION_KEY and needs at least 32 bytes; encrypt defaults to FLUID_PII_ENCRYPTION_SECRET_KEY and needs the base64 of a 16, 24 or 32-byte AES key (openssl rand -base64 32). Unset or empty, the build is refused. | |
exposes[].policy.privacy.masking[].params.keepFirst | integer | no | min 0 max 64 | 0 | NEW in v0.7.6: mask only. Leading characters left as they are. |
exposes[].policy.privacy.masking[].params.keepLast | integer | no | min 0 max 64 | 4 | NEW in v0.7.6: mask only. Trailing characters left as they are, so +46701234567 lands as ********4567. A value no longer than keepFirst + keepLast is masked entirely. |
exposes[].binding.governance.lakeFormation.bucketPolicy | string | no | one of cross-account, none, all-grantees | "cross-account" | NEW in v0.7.6: Which grants[] principals get a statement in the aws_s3_bucket_policy emitted beside the grants (s3:ListBucket and s3:GetBucketLocation on the bucket, s3:GetObject under location.path). A statement lets its principal read the objects straight from S3, which skips Lake Formation's column, row and cell filters and its revocations. 'cross-account' (the default): only principals whose ARN names an AWS account other than the one running the apply, decided at plan time against the caller identity; same-account grantees get no statement, since on a registered location Lake Formation vends them credentials. 'none': no bucket policy at all; the tightest setting when the location is registered (registerLocation) and every reader, cross-account ones included, queries through a Lake Formation integrated engine, or when the bucket policy is managed elsewhere. 'all-grantees': every grantee, same-account included, which is the output before this field existed and reopens the direct S3 read path. An emitted aws_s3_bucket_policy is authoritative: it replaces every other statement on the bucket. When no grantee is left none is emitted. |
exposes[].binding.packaging | object | no | NEW in v0.7.6: Per-exposure packaging override — key-wise precedence over the contract-wide top-level packaging block (binding.packaging > packaging > absent-legacy). | ||
exposes[].binding.packaging.mode | string | no | one of isolated, shared | NEW in v0.7.6: Blanket ownership mode for every container kind. Default when the block is present: 'isolated'. Per-kind exceptions go in containers. | |
exposes[].binding.packaging.pool | string | no | min length 1 | NEW in v0.7.6: Pool id of the platform-owned tenant pool this product writes into. REQUIRED (enforced by the resolver) whenever any container resolves 'shared' — a pool must be addressable. Propagated as the fluid_pool label/tag for cost attribution. | |
exposes[].binding.packaging.poolManifest | string | no | min length 1 | NEW in v0.7.6: Optional path to the platform team's pool manifest file; snapshotted into the bundle at stage 1 so bundleDigest covers it (any pool-file edit invalidates downstream plans). | |
exposes[].binding.packaging.containers | object | no | NEW in v0.7.6: Per-container-kind ownership overrides — each key wins over mode for that kind, yielding hybrid tiers. Kind↔platform mapping: bucket → aws_s3_bucket / google_storage_bucket; database → snowflake_database AND aws_glue_catalog_database; dataset → google_bigquery_dataset; schema → snowflake_schema; warehouse → snowflake_warehouse; cluster → Confluent environment/cluster (v1: shared only). | ||
exposes[].binding.packaging.containers.bucket | string | no | one of isolated, shared | NEW in v0.7.6: Ownership of the object-store bucket (aws_s3_bucket / google_storage_bucket). | |
exposes[].binding.packaging.containers.database | string | no | one of isolated, shared | NEW in v0.7.6: Ownership of the database container — covers Snowflake database AND AWS glue_database. | |
exposes[].binding.packaging.containers.dataset | string | no | one of isolated, shared | NEW in v0.7.6: Ownership of the BigQuery dataset (google_bigquery_dataset). | |
exposes[].binding.packaging.containers.schema | string | no | one of isolated, shared | NEW in v0.7.6: Ownership of the Snowflake schema (snowflake_schema). | |
exposes[].binding.packaging.containers.warehouse | string | no | one of isolated, shared | NEW in v0.7.6: Ownership of the Snowflake warehouse (snowflake_warehouse) — isolated gives per-product cost attribution. | |
exposes[].binding.packaging.containers.cluster | string | no | one of isolated, shared | NEW in v0.7.6: Ownership of the Confluent environment/cluster. v1 accepts 'shared' only — dedicated-cluster provisioning is not yet supported (the resolver rejects 'isolated'). | |
exposes[].binding.encryption | object | no | NEW in v0.7.6: Encryption at rest for an AWS S3 binding or a GCP BigQuery binding. Absent: nothing is emitted, and the cloud's default applies: SSE-S3 for new S3 objects, Google-managed keys for BigQuery. | ||
exposes[].binding.encryption.kms | string | no | one of product, noneor matches ^alias/[A-Za-z0-9/_-]{1,250}$or matches ^arn:aws[a-z-]*:kms:[a-z0-9-]+:[0-9]{12}:(key|alias)/[A-Za-z0-9/_-]{1,250}$or matches ^projects/[a-z0-9.:-]{1,100}/locations/[a-z0-9-]{1,63}/keyRings/[A-Za-z0-9_-]{1,63}/cryptoKeys/[A-Za-z0-9_-]{1,63}$When platform is gcp: must not match ^(alias/|arn:)When platform is aws: must not match ^projects/ | "product" | NEW in v0.7.6: The KMS key the bucket's objects are encrypted with (SSE-KMS with an S3 Bucket Key). 'product' (the default): fluid apply creates a customer managed key for each bucket this product owns (aws_kms_key, rotation on, alias alias/fluid/<contract id>/<bucket>) and makes it the bucket's default encryption (aws_s3_bucket_server_side_encryption_configuration). Its key policy lets the account's IAM policies decide who may use it (the KMS default statement for the account root) and, when the binding carries governance.lakeFormation, lets the Lake Formation service-linked role AWSServiceRoleForLakeFormationDataAccess encrypt and decrypt, matched by aws:PrincipalArn so the role need not exist yet. A principal querying through Athena with credentials Lake Formation vends then needs no KMS permission of its own; a grantee in another account that the bucket policy lets read the objects directly (governance.lakeFormation.bucketPolicy) is also allowed kms:Decrypt, through S3 only. The writer (the identity the build runs as) needs kms:GenerateDataKey and kms:Decrypt from IAM. tofu destroy deletes the alias and schedules the key for deletion after 7 days, the minimum KMS allows; kms:CancelKeyDeletion recovers it until then. Dropping the block, or naming another key, removes the product key while the bucket keeps the objects encrypted under it, which become unreadable once the deletion window ends unless they are rewritten or the deletion is cancelled; fluid apply refuses that plan without --allow-data-loss. On a shared (pool) bucket the bucket's default encryption is the pool owner's, so 'product' is refused there. 'alias/<name>' or a key or alias ARN: an existing key, looked up at plan time (data aws_kms_key, so the identity running fluid apply needs kms:DescribeKey on it) and made the default encryption of a bucket this product owns by its key ARN, since S3 resolves an alias in the account of whoever writes; the plan fails unless the key is Enabled and a symmetric encryption key (SYMMETRIC_DEFAULT). On a pool bucket it is only checked. Its key policy is its owner's, and must let the Lake Formation role use it when the location is registered: an AWS managed key cannot be used with that role, so alias/aws/s3 is refused with registerLocation, and the plan fails for any key that is not customer managed. 'none': nothing is emitted. fluid verify checks that the key is Enabled and that the objects under the binding's prefix are SSE-KMS with it. On a gcp binding (fluid-schema 0.7.6, BigQuery tables): 'product' creates a Cloud KMS key ring and crypto key for each dataset this product writes (google_kms_key_ring fluid-<contract id>-<dataset> and google_kms_crypto_key 'bigquery', in the dataset's location, EU and US mapping to the europe and us key locations, rotated every 90 days), grants the project's BigQuery service agent roles/cloudkms.cryptoKeyEncrypterDecrypter on it, and makes it the dataset's default_encryption_configuration and the table's encryption_configuration. The project must have the Cloud KMS API (cloudkms.googleapis.com) enabled. A key ring and key cannot be deleted on GCP: tofu destroy removes them from state and schedules the key's versions for destruction (30 days by default), and a later apply adopts the same names. 'projects/<p>/locations/<l>/keyRings/<r>/cryptoKeys/<k>': an existing key in the dataset's location, used as given; its owner grants the BigQuery service agent. 'none': Google-managed keys. An alias/... or arn:... value is refused on a gcp binding, and a Cloud KMS name on an aws binding. Adding a key to a table that exists replaces the table (BigQuery cannot re-key it in place), so fluid apply refuses that plan without --allow-data-loss; the next build lands the data again. fluid verify checks kmsKeyName on the dataset and the table. |
exposes[].binding.principals | object (map) | no | keys: min length 1 When platform is gcp: each value is a string matching ^(user|group|serviceAccount|domain):.+$ or an array whose items are each a string matching ^(user|group|serviceAccount|domain):.+$When platform is aws: each value is a string matching ^arn:aws[a-z0-9-]*:iam:: or an array whose items are each a string matching ^arn:aws[a-z0-9-]*:iam:: | NEW in v0.7.6: Maps the contract's LOGICAL principals (accessPolicy.grants[].principal and exposes[].policy.authz.columnRestrictions[].principal, keyed exactly as the contract writes them) to the identities they stand for on this binding's platform, so the base contract stays cloud-neutral and each environment's overlay binds it. On gcp an identity is an IAM member (user:, group:, serviceAccount: or domain:); on aws an IAM principal ARN. A value is one identity or a list; [] says the principal has no identity on this cloud, and nothing is granted to it there. When the block is present, every principal the contract names for this expose must be a key: an unmapped one is refused, never emitted as written. Without the block the principals are used as written, except that a GCP principal in a reserved top-level domain (.example, .test, .invalid, .localhost) is refused as a placeholder. Modelled on ODCS v3's roles[] bound per server (servers[].roles) and on dbt grants resolved per target. | |
exposes[].binding.principals.<key> | string or array of string | no | string: min length 1 array: unique items, each item: min length 1 | ||
exposes[].lifecycle.expire | boolean | no | false | NEW in v0.7.6: Delete stored data once it is older than retention. Default false, so a contract that declares a retention period does not start deleting data when forge-cli is upgraded. Honoured by the AWS emitter on an S3 binding and by the GCP emitter on a BigQuery table; on a gcp binding that is not a BigQuery table (GCS, Pub/Sub, Iceberg storage) it is refused rather than dropped. AWS: fluid apply writes one rule per expose into the bucket's aws_s3_bucket_lifecycle_configuration, filtered to the binding's prefix (location.path, else {database}/{table}/; a binding with neither is refused, so no rule ever expires a whole bucket). The rule expires the current version retention after it is written, removes a noncurrent version (on a versioning-enabled bucket) one day after it stops being current, and aborts an incomplete multipart upload after 7 days or the retention period if shorter. The lifecycle configuration is AUTHORITATIVE for the whole bucket: S3 keeps one per bucket, so rules that fluid apply did not write are replaced. It is therefore written only for a bucket this product owns; on a shared (pool) bucket (packaging) nothing is written, the pool's owner must hold the rule, and fluid verify checks that an enabled rule covering the prefix expires objects after exactly this period. Objects already older than the period when the rule is first applied are expired on S3's next lifecycle run, typically within a day. Removing expire later deletes the lifecycle configuration, which fluid apply refuses without --allow-data-loss (it refuses every plan that removes a resource holding data or policy; removing an access grant or a policy tag is a revocation and is not gated); tofu destroy deletes it. GCP (fluid-schema 0.7.6): the BigQuery table is partitioned by day (time_partitioning type DAY, on binding.location.partitionBy when it names one date or timestamp column, else by ingestion time) with expiration_ms = retention in days, so BigQuery deletes each partition that long after its day ends. BigQuery counts from the partition's date, not from when its rows were written: by ingestion time (no partitionBy, the default and the S3 rule's semantics) no row is deleted sooner than retention after it landed; with partitionBy, retention is the age of the date in that column, so a backfill of rows whose date is already older than retention lands in expired partitions and is deleted at once. It is never a whole-table expiration, which would delete the product. BigQuery cannot partition a table that exists, so adding expire to a live table replaces it: fluid apply refuses that plan without --allow-data-loss, and the next build lands the data again. A later change of retention is an in-place update. Removing expire from a live table removes the partitioning's replacement trigger, which fluid apply also refuses without --allow-data-loss; BigQuery cannot un-partition a table, so to keep data longer set a longer retention instead. fluid verify checks the live table's partition expiration. | |
exposes[].semantics.measures[].aggParams | object | no | Aggregation parameters (mirrors dbt-semantic-interfaces agg_params). Currently used by agg=percentile. | ||
exposes[].semantics.measures[].aggParams.percentile | number | no | min 0 max 1 | Percentile in [0, 1] for agg=percentile (0.5 = median). Consumers default to 0.5 when omitted. | |
exposes[].semantics.measures[].aggParams.useDiscretePercentile | boolean | no | Use the discrete percentile (PERCENTILE_DISC / use_discrete_percentile) instead of continuous interpolation. |
consumes
| Path | Type | Required | Allowed values | Default | Description |
|---|---|---|---|---|---|
consumes[].upstreamWorkspace | string | no | min length 1 | Cross-mesh only. Id of the federated workspace that owns this upstream, as declared in federation/upstreams.yaml. Set it when the upstream lives in another mesh rather than this workspace; omit it for a local upstream, which is resolved by walking the workspace. | |
consumes[].upstreamDigest | string | no | matches ^sha256:[0-9a-f]{64}$ | Cross-mesh only. The contract digest this product was composed against, as sha256:<64 hex>. fluid apply re-fetches the upstream's live digest and compares: a mismatch means the upstream changed since composition. Required whenever upstreamWorkspace is set. |
consumers
| Path | Type | Required | Allowed values | Default | Description |
|---|---|---|---|---|---|
consumers | array of object | no | NEW in v0.7.6: Declared downstream consumers of this data product — the business artifacts built on it (dashboards, notebooks, analyses, ML apps). Shape follows dbt exposures so existing dbt exposure definitions map field-for-field. Purely declarative and additive: consumers never affect plan/apply. RESERVED, not yet consumed — no code reads this key today; it is carried verbatim into plan.json and the contract COMMENT so it survives round-trip, and is intended to feed lineage terminal nodes and impact statements ('Affects: Revenue dashboard') in a later release. | ||
consumers[].name | string | yes | matches ^[A-Za-z0-9_][A-Za-z0-9_.-]*[A-Za-z0-9_]$|^[A-Za-z0-9_]$ | Stable identifier for this consumer (unique within the contract). | |
consumers[].label | string | no | Human-facing display name, e.g. "Weekly Revenue Dashboard". | ||
consumers[].type | string | yes | one of dashboard, notebook, analysis, ml, application | What kind of artifact consumes this product. | |
consumers[].owner | object | no | |||
consumers[].owner.team | string | no | Owning team for this consumer (routable identity, as in metadata.owner). | ||
consumers[].owner.email | string | no | format email | Contact address for the consumer's owner. | |
consumers[].url | string | no | Link to the live artifact (dashboard URL, notebook, app). | ||
consumers[].maturity | string | no | one of high, medium, low | How production-hardened the consumer is (dbt exposures semantics). | |
consumers[].description | string | no | |||
consumers[].exposeIds | array of string | no | each item: matches ^[A-Za-z0-9_][A-Za-z0-9_.-]*[A-Za-z0-9_]$|^[A-Za-z0-9_]$ | Which output ports this consumer reads. Absent = all exposes. |
packaging
| Path | Type | Required | Allowed values | Default | Description |
|---|---|---|---|---|---|
packaging | object | no | NEW in v0.7.6: Contract-wide packaging default — declarative container ownership (isolated = this product creates and owns its infrastructure containers; shared = containers are pre-existing, platform-owned pools the product writes into but does not own). Overridable per exposure via binding.packaging. An ABSENT block means legacy behavior: the IaC emit is byte-identical to pre-packaging releases. | ||
packaging.mode | string | no | one of isolated, shared | NEW in v0.7.6: Blanket ownership mode for every container kind. Default when the block is present: 'isolated'. Per-kind exceptions go in containers. | |
packaging.pool | string | no | min length 1 | NEW in v0.7.6: Pool id of the platform-owned tenant pool this product writes into. REQUIRED (enforced by the resolver) whenever any container resolves 'shared' — a pool must be addressable. Propagated as the fluid_pool label/tag for cost attribution. | |
packaging.poolManifest | string | no | min length 1 | NEW in v0.7.6: Optional path to the platform team's pool manifest file; snapshotted into the bundle at stage 1 so bundleDigest covers it (any pool-file edit invalidates downstream plans). | |
packaging.containers | object | no | NEW in v0.7.6: Per-container-kind ownership overrides — each key wins over mode for that kind, yielding hybrid tiers. Kind↔platform mapping: bucket → aws_s3_bucket / google_storage_bucket; database → snowflake_database AND aws_glue_catalog_database; dataset → google_bigquery_dataset; schema → snowflake_schema; warehouse → snowflake_warehouse; cluster → Confluent environment/cluster (v1: shared only). | ||
packaging.containers.bucket | string | no | one of isolated, shared | NEW in v0.7.6: Ownership of the object-store bucket (aws_s3_bucket / google_storage_bucket). | |
packaging.containers.database | string | no | one of isolated, shared | NEW in v0.7.6: Ownership of the database container — covers Snowflake database AND AWS glue_database. | |
packaging.containers.dataset | string | no | one of isolated, shared | NEW in v0.7.6: Ownership of the BigQuery dataset (google_bigquery_dataset). | |
packaging.containers.schema | string | no | one of isolated, shared | NEW in v0.7.6: Ownership of the Snowflake schema (snowflake_schema). | |
packaging.containers.warehouse | string | no | one of isolated, shared | NEW in v0.7.6: Ownership of the Snowflake warehouse (snowflake_warehouse) — isolated gives per-product cost attribution. | |
packaging.containers.cluster | string | no | one of isolated, shared | NEW in v0.7.6: Ownership of the Confluent environment/cluster. v1 accepts 'shared' only — dedicated-cluster provisioning is not yet supported (the resolver rejects 'isolated'). |
Changed fields
Each row names the attribute that differs and shows both values.
| Path | Attribute | 0.7.5 | 0.7.6 |
|---|---|---|---|
fluidVersion | allowed values | one of 0.7.3, 0.7.4, 0.7.5matches ^\d+\.\d+(\.\d+)?$ | one of 0.7.3, 0.7.4, 0.7.5, 0.7.6matches ^\d+\.\d+(\.\d+)?$ |
fluidVersion | description | Contract schema version. Accepts '0.7.3' (source-aligned data products + acquisition pattern + ingestion engines) or '0.7.4' (adds the MCP output-port gateway + runtime agentPolicy enforcement). 0.7.4 is fully backward-compatible with 0.7.3 — every 0.7.3 contract validates as 0.7.4 unchanged. | Contract schema version. Accepts '0.7.3' (source-aligned data products + acquisition pattern + ingestion engines) or '0.7.4' (adds the MCP output-port gateway + runtime agentPolicy enforcement). 0.7.4 is fully backward-compatible with 0.7.3 — every 0.7.3 contract validates as 0.7.4 unchanged. '0.7.6' is the current preview. |
exposes[].policy.authz.columnRestrictions | description | Column-level access control. | Column-level access control. A column named in any restriction is restricted. 'deny': the principal may not read the columns. 'allow': the columns are readable only by the principals an allow names. A deny beats an allow, and a restriction never grants access: the readers are the expose's readers. Principals are logical and resolve through binding.principals. NEW in v0.7.6 (the field itself is older): fluid apply enforces it. GCP BigQuery: a Data Catalog taxonomy per product and dataset with fine-grained access control, a policy tag per set of restricted columns that share their readers, attached through the table schema's policyTags, and roles/datacatalog.categoryFineGrainedReader for exactly the expose's readers allowed to read them: the accessPolicy read grantees and the expose's own policy.authz.readers (a denied principal gets an access error on the columns). A restriction on an expose with no reader on the binding is refused, as it would lock the columns for everyone. A restriction's tags and labels are written into the policy tag's description; AWS has no place for them. AWS: each governance.lakeFormation grant with SELECT excludes the restricted columns its principal may not read (table_with_columns.excluded_column_names); a grant's hand-written excludedColumns must agree, and a binding with no Lake Formation grants is refused. fluid verify checks the policy tags and their readers on GCP, and the principals' Lake Formation permissions on AWS. |
exposes[].policy.authz.columnRestrictions[].principal | description | none | Logical principal the restriction applies to, written as in accessPolicy (e.g. group:analysts@company.example); binding.principals maps it per cloud. |
exposes[].policy.authz.columnRestrictions[].columns | description | none | Columns of this expose's schema. |
exposes[].policy.authz.columnRestrictions[].access | description | none | deny: the principal may not read the columns. allow: only the principals an allow names may read them. |
exposes[].policy.privacy.masking | description | none | Columns to treat before they land. The DuckDB acquisition runner rewrites each named column inside the write, so the file, S3 object or BigQuery load and the DLQ receive the treated value; quality gates judge the source value first. A treated column lands as a string, so declare it with a string type (string, or VARCHAR under a schemaPolicy that compares source type names). A rule the build cannot apply (an unset salt or key, k_anonymity, a column the stream lacks, a non-string declared type, the incremental cursor) refuses the build rather than landing the column untreated. fluid verify checks every non-null value of a masked column for its strategy's shape, on a local file and, through Athena, on an S3+Glue table. The MCP output-port gateway drops masked columns from what it serves. |
exposes[].policy.privacy.masking[].strategy | description | none | hash: lowercase hex SHA-256 of salt || value, deterministic so joins work (64 hex characters). mask: every character but the first params.keepFirst and the last params.keepLast replaced by '*', length preserved. tokenize: HMAC-SHA256 token, 32 hex characters, the token the tokenize_pii preLand hook makes. encrypt: AES-GCM, written as 'aesgcm:v1:' + base64url(nonce || ciphertext || tag) with the column name as associated data; reversible with the key, not deterministic. k_anonymity: accepted by the schema but refused at landing: it is a property of a whole table over its quasi-identifiers, not of one value. |
exposes[].policy.privacy.masking[].params | type | object (free-form) | object (open) |
exposes[].policy.privacy.masking[].params | description | none | Strategy parameters. The runner accepts exactly these keys and refuses any other, including a literal salt or key: secrets come only from the environment, never the contract, which is published with the plan, the bundle and the Glue table's fluid_contract parameter. Contracts on 0.7.5 and earlier may use the same keys; their schemas leave params untyped, and the runner checks them at build time. |
exposes[].binding.location.partitionBy | description | NEW in v0.7.5: Iceberg partition columns — a FLUID abstraction over the Iceberg PARTITIONED BY / PartitionSpec (maps to iceberg.tables.default-partition-by). | NEW in v0.7.5: Iceberg partition columns — a FLUID abstraction over the Iceberg PARTITIONED BY / PartitionSpec (maps to iceberg.tables.default-partition-by). On a gcp BigQuery table with exposes[].lifecycle.expire: true (fluid-schema 0.7.6), one DATE, TIMESTAMP or DATETIME column to partition by (time_partitioning.field); without it the table is partitioned by ingestion time. With a column, retention counts from the date in that column, not from landing: BigQuery deletes a partition retention after its date, so a backfill of older rows is deleted at once. Leave it out for the S3 rule's semantics (age since written). |
exposes[].lifecycle | description | none | NEW in v0.7.6: An expose's lifecycle. The same fields as the contract-root lifecycle, plus expire, which is only meaningful where there is storage to expire. |
exposes[].lifecycle.retention | description | ISO-8601 duration (e.g., P1D, PT15M, P2Y6M). | How long this expose's data is kept (ISO-8601 duration, e.g. P30D, P7Y). A declaration unless expire is true. With expire: true on an AWS S3 binding, fluid apply expires the objects under the binding's prefix this long after they are written; on a GCP BigQuery table it deletes each daily partition this long after its day ends. Years and months are counted as 365 and 30 days, and a part of a day rounds up to a whole one. |
consumes | allowed values | none | each item: if upstreamWorkspace is set, upstreamDigest is required |
Removed fields
None. Every 0.7.5 field path is present in 0.7.6.