Contract field reference, schema 0.7.7 preview
Status: preview. A contract opts in with fluidVersion: "0.7.7"; a contract that declares 0.7.6 is unaffected. This page lists the difference between the 0.7.6 schema and the 0.7.7 schema as bundled with the CLI: added fields, fields whose definition changed, and fields removed. A field not listed here has the same type, required flag, allowed values, default and description in both; for those, use the 0.7.6 reference. A definition used at several paths is listed once, at its first path.
Generated from fluid_build/schemas/fluid-schema-0.7.7.json in the data-product-forge 0.29.0 package (sha256 0f11fb44bc9dafbc). It is the schema fluid validate checks a contract against, rendered without edits. To regenerate: python scripts/gen_contract_reference.py.
Each table row is one field, addressed by its path from the contract root: . descends into an object, [] marks the items of a list, and <key> stands for any key of a map. Required means required inside its parent object; the parent can itself be optional. Allowed values lists the enum, pattern and range the schema enforces. An object typed plainly object rejects keys the schema does not list; object (open) accepts extra keys and object (free-form) has no listed keys. A row that begins "Only when ..." exists only under that condition. See how to read this reference.
At a glance
- Added: 142 rows
- Changed: 2 rows
- Removed: 0 rows
Added fields
exposes
| Path | Type | Required | Allowed values | Default | Description |
|---|---|---|---|---|---|
exposes[].contract.schema[].fields | array of object | no | min items 1 | Typed child columns of a struct/row/record. Same fields as exposes[].contract.schema[]: read that section with this path as the prefix. | |
exposes[].contract.schema[].element | object | no | |||
exposes[].contract.schema[].element.type | string | yes | one of 79 valuesstring text varchar varchar2 nvarchar char nchar character clob int integer int2 int4 int8 int16 int32 int64 tinyint smallint mediumint bigint long longint serial bigserial float float4 float8 float32 float64 double real decimal dec numeric number bignumeric money boolean bool bit date time datetime datetime2 smalldatetime timestamp timestamptz timestamp_tz timestamp_ntz timestamp_ltz timestampntz interval year variant object array struct map record row json jsonb super binary varbinary bytes blob bytea raw geography geometry geom point uuid uniqueidentifier guid enum hllor matches a 684-character pattern(?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*((\s*[0-9A-Za-z_,\s'"-]\s))?\s*$ | Column data type. Accepts a canonical FLUID/SQL type name (case-insensitive) or a parameterized form such as decimal(18,4) or varchar(255). The enum lists the canonical bare types; the pattern additionally permits case variants and parameter suffixes. Intentionally generous — it rejects obvious garbage (e.g. MYSTERY_TYPE) while admitting the cross-dialect superset. | |
exposes[].contract.schema[].element.required | boolean | no | |||
exposes[].contract.schema[].element.fields | array of object | no | min items 1 | Same fields as exposes[].contract.schema[]: read that section with this path as the prefix. | |
exposes[].contract.schema[].element.element | object | no | Same fields as exposes[].contract.schema[].element: read that section with this path as the prefix. | ||
exposes[].contract.schema[].element.key | object | no | Same fields as exposes[].contract.schema[].element: read that section with this path as the prefix. | ||
exposes[].contract.schema[].element.value | object | no | Same fields as exposes[].contract.schema[].element: read that section with this path as the prefix. | ||
exposes[].contract.schema[].key | object | no | |||
exposes[].contract.schema[].key.type | string | yes | one of 79 valuesstring text varchar varchar2 nvarchar char nchar character clob int integer int2 int4 int8 int16 int32 int64 tinyint smallint mediumint bigint long longint serial bigserial float float4 float8 float32 float64 double real decimal dec numeric number bignumeric money boolean bool bit date time datetime datetime2 smalldatetime timestamp timestamptz timestamp_tz timestamp_ntz timestamp_ltz timestampntz interval year variant object array struct map record row json jsonb super binary varbinary bytes blob bytea raw geography geometry geom point uuid uniqueidentifier guid enum hllor matches a 684-character pattern(?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*((\s*[0-9A-Za-z_,\s'"-]\s))?\s*$ | Column data type. Accepts a canonical FLUID/SQL type name (case-insensitive) or a parameterized form such as decimal(18,4) or varchar(255). The enum lists the canonical bare types; the pattern additionally permits case variants and parameter suffixes. Intentionally generous — it rejects obvious garbage (e.g. MYSTERY_TYPE) while admitting the cross-dialect superset. | |
exposes[].contract.schema[].key.required | boolean | no | |||
exposes[].contract.schema[].key.fields | array of object | no | min items 1 | Same fields as exposes[].contract.schema[]: read that section with this path as the prefix. | |
exposes[].contract.schema[].key.element | object | no | Same fields as exposes[].contract.schema[].key: read that section with this path as the prefix. | ||
exposes[].contract.schema[].key.key | object | no | Same fields as exposes[].contract.schema[].key: read that section with this path as the prefix. | ||
exposes[].contract.schema[].key.value | object | no | Same fields as exposes[].contract.schema[].key: read that section with this path as the prefix. | ||
exposes[].contract.schema[].value | object | no | |||
exposes[].contract.schema[].value.type | string | yes | one of 79 valuesstring text varchar varchar2 nvarchar char nchar character clob int integer int2 int4 int8 int16 int32 int64 tinyint smallint mediumint bigint long longint serial bigserial float float4 float8 float32 float64 double real decimal dec numeric number bignumeric money boolean bool bit date time datetime datetime2 smalldatetime timestamp timestamptz timestamp_tz timestamp_ntz timestamp_ltz timestampntz interval year variant object array struct map record row json jsonb super binary varbinary bytes blob bytea raw geography geometry geom point uuid uniqueidentifier guid enum hllor matches a 684-character pattern(?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*((\s*[0-9A-Za-z_,\s'"-]\s))?\s*$ | Column data type. Accepts a canonical FLUID/SQL type name (case-insensitive) or a parameterized form such as decimal(18,4) or varchar(255). The enum lists the canonical bare types; the pattern additionally permits case variants and parameter suffixes. Intentionally generous — it rejects obvious garbage (e.g. MYSTERY_TYPE) while admitting the cross-dialect superset. | |
exposes[].contract.schema[].value.required | boolean | no | |||
exposes[].contract.schema[].value.fields | array of object | no | min items 1 | Same fields as exposes[].contract.schema[]: read that section with this path as the prefix. | |
exposes[].contract.schema[].value.element | object | no | Same fields as exposes[].contract.schema[].value: read that section with this path as the prefix. | ||
exposes[].contract.schema[].value.key | object | no | Same fields as exposes[].contract.schema[].value: read that section with this path as the prefix. | ||
exposes[].contract.schema[].value.value | object | no | Same fields as exposes[].contract.schema[].value: read that section with this path as the prefix. |
builds
| Path | Type | Required | Allowed values | Default | Description |
|---|---|---|---|---|---|
builds[].properties.stream | object | no | Only when pattern is embedded-logic. NEW in v0.7.7 (preview): The stream the build reads. On a managed Flink build (parameters.flink.management) sql is optional: the lane writes the query from the mapping instead. Elsewhere the stream describes what the build's own sql reads. | ||
builds[].properties.stream.kind | string | yes | must be kafka | Only when pattern is embedded-logic. The streaming system. Only kafka in this preview. | |
builds[].properties.stream.topic | string | yes | matches ^(?!\.\.?(?![\s\S]))[A-Za-z0-9._-]{1,249}(?![\s\S]) | Only when pattern is embedded-logic. One Kafka topic, by name: Kafka's legal characters (not . or ..), no pattern and no list. | |
builds[].properties.stream.format | string | yes | one of json, avro-confluent | Only when pattern is embedded-logic. How record values are encoded. json: one JSON object per record; a field the schema does not declare is ignored, and a declared field that is missing reads as null. avro-confluent: Avro in the Confluent Schema Registry wire format. The declared schema is the reader schema, so the producers' subject must stay readable by it: FORWARD or FULL compatibility, transitive. | |
builds[].properties.stream.schema | array of object | yes | min items 1 | Only when pattern is embedded-logic. The record value's fields, as columns. Nested data uses fields (struct), element (array) and key/value (map). | |
builds[].properties.stream.schema[].fields | array of object | no | min items 1 | Only when pattern is embedded-logic. Typed child columns of a struct/row/record. Same fields as builds[].properties.stream.schema[]: read that section with this path as the prefix. | |
builds[].properties.stream.schema[].element | object | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.schema[].element.type | string | yes | one of 79 valuesstring text varchar varchar2 nvarchar char nchar character clob int integer int2 int4 int8 int16 int32 int64 tinyint smallint mediumint bigint long longint serial bigserial float float4 float8 float32 float64 double real decimal dec numeric number bignumeric money boolean bool bit date time datetime datetime2 smalldatetime timestamp timestamptz timestamp_tz timestamp_ntz timestamp_ltz timestampntz interval year variant object array struct map record row json jsonb super binary varbinary bytes blob bytea raw geography geometry geom point uuid uniqueidentifier guid enum hllor matches a 684-character pattern(?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*((\s*[0-9A-Za-z_,\s'"-]\s))?\s*$ | Only when pattern is embedded-logic. Column data type. Accepts a canonical FLUID/SQL type name (case-insensitive) or a parameterized form such as decimal(18,4) or varchar(255). The enum lists the canonical bare types; the pattern additionally permits case variants and parameter suffixes. Intentionally generous — it rejects obvious garbage (e.g. MYSTERY_TYPE) while admitting the cross-dialect superset. | |
builds[].properties.stream.schema[].element.required | boolean | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.schema[].element.fields | array of object | no | min items 1 | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.schema[]: read that section with this path as the prefix. | |
builds[].properties.stream.schema[].element.element | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.schema[].element: read that section with this path as the prefix. | ||
builds[].properties.stream.schema[].element.key | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.schema[].element: read that section with this path as the prefix. | ||
builds[].properties.stream.schema[].element.value | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.schema[].element: read that section with this path as the prefix. | ||
builds[].properties.stream.schema[].key | object | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.schema[].key.type | string | yes | one of 79 valuesstring text varchar varchar2 nvarchar char nchar character clob int integer int2 int4 int8 int16 int32 int64 tinyint smallint mediumint bigint long longint serial bigserial float float4 float8 float32 float64 double real decimal dec numeric number bignumeric money boolean bool bit date time datetime datetime2 smalldatetime timestamp timestamptz timestamp_tz timestamp_ntz timestamp_ltz timestampntz interval year variant object array struct map record row json jsonb super binary varbinary bytes blob bytea raw geography geometry geom point uuid uniqueidentifier guid enum hllor matches a 684-character pattern(?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*((\s*[0-9A-Za-z_,\s'"-]\s))?\s*$ | Only when pattern is embedded-logic. Column data type. Accepts a canonical FLUID/SQL type name (case-insensitive) or a parameterized form such as decimal(18,4) or varchar(255). The enum lists the canonical bare types; the pattern additionally permits case variants and parameter suffixes. Intentionally generous — it rejects obvious garbage (e.g. MYSTERY_TYPE) while admitting the cross-dialect superset. | |
builds[].properties.stream.schema[].key.required | boolean | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.schema[].key.fields | array of object | no | min items 1 | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.schema[]: read that section with this path as the prefix. | |
builds[].properties.stream.schema[].key.element | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.schema[].key: read that section with this path as the prefix. | ||
builds[].properties.stream.schema[].key.key | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.schema[].key: read that section with this path as the prefix. | ||
builds[].properties.stream.schema[].key.value | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.schema[].key: read that section with this path as the prefix. | ||
builds[].properties.stream.schema[].value | object | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.schema[].value.type | string | yes | one of 79 valuesstring text varchar varchar2 nvarchar char nchar character clob int integer int2 int4 int8 int16 int32 int64 tinyint smallint mediumint bigint long longint serial bigserial float float4 float8 float32 float64 double real decimal dec numeric number bignumeric money boolean bool bit date time datetime datetime2 smalldatetime timestamp timestamptz timestamp_tz timestamp_ntz timestamp_ltz timestampntz interval year variant object array struct map record row json jsonb super binary varbinary bytes blob bytea raw geography geometry geom point uuid uniqueidentifier guid enum hllor matches a 684-character pattern(?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*((\s*[0-9A-Za-z_,\s'"-]\s))?\s*$ | Only when pattern is embedded-logic. Column data type. Accepts a canonical FLUID/SQL type name (case-insensitive) or a parameterized form such as decimal(18,4) or varchar(255). The enum lists the canonical bare types; the pattern additionally permits case variants and parameter suffixes. Intentionally generous — it rejects obvious garbage (e.g. MYSTERY_TYPE) while admitting the cross-dialect superset. | |
builds[].properties.stream.schema[].value.required | boolean | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.schema[].value.fields | array of object | no | min items 1 | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.schema[]: read that section with this path as the prefix. | |
builds[].properties.stream.schema[].value.element | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.schema[].value: read that section with this path as the prefix. | ||
builds[].properties.stream.schema[].value.key | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.schema[].value: read that section with this path as the prefix. | ||
builds[].properties.stream.schema[].value.value | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.schema[].value: read that section with this path as the prefix. | ||
builds[].properties.stream.schema[].name | string | yes | min length 1 | Only when pattern is embedded-logic. | |
builds[].properties.stream.schema[].type | string | yes | one of 79 valuesstring text varchar varchar2 nvarchar char nchar character clob int integer int2 int4 int8 int16 int32 int64 tinyint smallint mediumint bigint long longint serial bigserial float float4 float8 float32 float64 double real decimal dec numeric number bignumeric money boolean bool bit date time datetime datetime2 smalldatetime timestamp timestamptz timestamp_tz timestamp_ntz timestamp_ltz timestampntz interval year variant object array struct map record row json jsonb super binary varbinary bytes blob bytea raw geography geometry geom point uuid uniqueidentifier guid enum hllor matches a 684-character pattern(?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*((\s*[0-9A-Za-z_,\s'"-]\s))?\s*$ | Only when pattern is embedded-logic. Column data type. Accepts a canonical FLUID/SQL type name (case-insensitive) or a parameterized form such as decimal(18,4) or varchar(255). The enum lists the canonical bare types; the pattern additionally permits case variants and parameter suffixes. Intentionally generous — it rejects obvious garbage (e.g. MYSTERY_TYPE) while admitting the cross-dialect superset. | |
builds[].properties.stream.schema[].required | boolean | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.schema[].description | string | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.schema[].sensitivity | string | no | one of 12 valuesnone internal confidential restricted pii phi cleartext treated anonymized pseudonymized tokenized encrypted | Only when pattern is embedded-logic. Field-level sensitivity classification for improved policy ergonomics. | |
builds[].properties.stream.schema[].semanticType | string | no | Only when pattern is embedded-logic. Semantic meaning (e.g., 'email', 'phone', 'identifier'). | ||
builds[].properties.stream.schema[].businessName | string | no | Only when pattern is embedded-logic. Business-friendly name for this field. | ||
builds[].properties.stream.schema[].businessDefinition | string | no | Only when pattern is embedded-logic. Business definition and meaning of this field. | ||
builds[].properties.stream.schema[].validationRules | array of object | no | Only when pattern is embedded-logic. Field-level validation constraints. | ||
builds[].properties.stream.schema[].validationRules[].type | string | no | one of range, regex, enum, custom | Only when pattern is embedded-logic. | |
builds[].properties.stream.schema[].validationRules[].constraint | string | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.schema[].validationRules[].message | string | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.schema[].quality | array of object | no | each item: min properties 1 | Only when pattern is embedded-logic. NEW in v0.7.6: Column-level data quality rules, in ODCS's own shape — the array that is written straight into schema[].properties[].quality[] of the exported ODCS v3.1.0 contract, and the array an imported ODCS column's rules are read back into, inline on the column rather than in the importer's pass-through bucket: the ODPS importer merges a resolved port's contract block into its output-port stub and discards the sub-import's root extensions, so rules left in that bucket were dropped outright by a FLUID → ODPS → FLUID cycle. The ODCS exporter has always replayed this key verbatim and always read it back on import, but $defs.column is additionalProperties: false and did not define it, so a column written the way the exporter supports was REFUSED by fluid validate: the only way to get a hand-written quality rule into a published contract was to write an invalid contract. Optional and additive — a column without it exports exactly as it does today, with the rules forge derives from required, the primary-key tag and validationRules. Setting it REPLACES those derived rules for that column rather than adding to them, which is what makes an imported ODCS column round-trip byte-for-byte. The authority for an item's contents is ODCS v3.1.0 $defs.DataQuality, deliberately not re-declared here (a copy would drift from the vendored schema, and fluid odcs validate already checks an emitted document against it); the keys below are the common ones. Every rule should carry an id: it is optional in ODCS, but a DQ engine that labels its results by rule id rejects a contract whose rules have none. | |
builds[].properties.stream.schema[].quality[].id | string | no | matches ^[A-Za-z0-9_-]+$ | Only when pattern is embedded-logic. Stable rule identifier, unique within this column's rules. The pattern is ODCS $defs.StableId. Rules forge derives itself are named <column>-<metric>. | |
builds[].properties.stream.schema[].quality[].type | string | no | one of text, library, sql, custom | Only when pattern is embedded-logic. ODCS quality check type. text = prose; library = one of ODCS's predefined metrics; sql = a query whose result is compared; custom = a named engine's own check. | |
builds[].properties.stream.schema[].quality[].metric | string | no | one of nullValues, missingValues, invalidValues, duplicateValues, rowCount | Only when pattern is embedded-logic. type: library only. The five predefined metrics ODCS v3.1.0 defines; a name outside this list is not a library rule. | |
builds[].properties.stream.schema[].quality[].arguments | object (free-form) | no | Only when pattern is embedded-logic. type: library only. Extra arguments for the metric — pattern (a regex) or validValues (a list) for invalidValues, as DataHub's ODCS ingestion source reads them. | ||
builds[].properties.stream.schema[].quality[].query | string | no | Only when pattern is embedded-logic. type: sql only. The query whose result the comparison operator is applied to. | ||
builds[].properties.stream.schema[].quality[].engine | string | no | Only when pattern is embedded-logic. type: custom only. The engine that runs the check, e.g. soda, great-expectations, dbt. | ||
builds[].properties.stream.schema[].quality[].implementation | any | no | Only when pattern is embedded-logic. type: custom only. The check as that engine expresses it. | ||
builds[].properties.stream.schema[].quality[].dimension | string | no | one of accuracy, completeness, conformity, consistency, coverage, timeliness, uniqueness | Only when pattern is embedded-logic. The data quality dimension the rule measures. | |
builds[].properties.stream.schema[].quality[].description | string | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.schema[].quality[].severity | string | no | Only when pattern is embedded-logic. Severity of a failure. ODCS types this as a free string and documents info / warning / error by example. | ||
builds[].properties.stream.schema[].quality[].unit | string | no | Only when pattern is embedded-logic. Unit the comparison is in, usually rows (the default) or percent. | ||
builds[].properties.stream.schema[].tags | array of string | no | unique items each item: matches ^[a-z0-9][a-z0-9-]*[a-z0-9]$|^[a-z0-9]$ | Only when pattern is embedded-logic. Field-level tags for categorization and data governance. | |
builds[].properties.stream.schema[].labels | object (map) | no | Only when pattern is embedded-logic. Field-level labels for metadata and automation. | ||
builds[].properties.stream.schema[].labels.<key> | string | no | Only when pattern is embedded-logic. Label values are always strings for consistency and tooling compatibility. | ||
builds[].properties.stream.key | object | no | Only when pattern is embedded-logic. NEW in v0.7.7 (preview): The record key, read as columns next to the value's. A record without a key reads its key columns as null. | ||
builds[].properties.stream.key.format | string | yes | one of raw, json, avro-confluent | Only when pattern is embedded-logic. raw: the key's bytes as one column, of type string (UTF-8) or binary. json and avro-confluent: a key record whose fields are the schema's columns. | |
builds[].properties.stream.key.schema | array of object | yes | min items 1 | Only when pattern is embedded-logic. The key's fields, as columns: exactly one for raw. | |
builds[].properties.stream.key.schema[].fields | array of object | no | min items 1 | Only when pattern is embedded-logic. Typed child columns of a struct/row/record. Same fields as builds[].properties.stream.key.schema[]: read that section with this path as the prefix. | |
builds[].properties.stream.key.schema[].element | object | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.key.schema[].element.type | string | yes | one of 79 valuesstring text varchar varchar2 nvarchar char nchar character clob int integer int2 int4 int8 int16 int32 int64 tinyint smallint mediumint bigint long longint serial bigserial float float4 float8 float32 float64 double real decimal dec numeric number bignumeric money boolean bool bit date time datetime datetime2 smalldatetime timestamp timestamptz timestamp_tz timestamp_ntz timestamp_ltz timestampntz interval year variant object array struct map record row json jsonb super binary varbinary bytes blob bytea raw geography geometry geom point uuid uniqueidentifier guid enum hllor matches a 684-character pattern(?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*((\s*[0-9A-Za-z_,\s'"-]\s))?\s*$ | Only when pattern is embedded-logic. Column data type. Accepts a canonical FLUID/SQL type name (case-insensitive) or a parameterized form such as decimal(18,4) or varchar(255). The enum lists the canonical bare types; the pattern additionally permits case variants and parameter suffixes. Intentionally generous — it rejects obvious garbage (e.g. MYSTERY_TYPE) while admitting the cross-dialect superset. | |
builds[].properties.stream.key.schema[].element.required | boolean | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.key.schema[].element.fields | array of object | no | min items 1 | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.key.schema[]: read that section with this path as the prefix. | |
builds[].properties.stream.key.schema[].element.element | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.key.schema[].element: read that section with this path as the prefix. | ||
builds[].properties.stream.key.schema[].element.key | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.key.schema[].element: read that section with this path as the prefix. | ||
builds[].properties.stream.key.schema[].element.value | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.key.schema[].element: read that section with this path as the prefix. | ||
builds[].properties.stream.key.schema[].key | object | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.key.schema[].key.type | string | yes | one of 79 valuesstring text varchar varchar2 nvarchar char nchar character clob int integer int2 int4 int8 int16 int32 int64 tinyint smallint mediumint bigint long longint serial bigserial float float4 float8 float32 float64 double real decimal dec numeric number bignumeric money boolean bool bit date time datetime datetime2 smalldatetime timestamp timestamptz timestamp_tz timestamp_ntz timestamp_ltz timestampntz interval year variant object array struct map record row json jsonb super binary varbinary bytes blob bytea raw geography geometry geom point uuid uniqueidentifier guid enum hllor matches a 684-character pattern(?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*((\s*[0-9A-Za-z_,\s'"-]\s))?\s*$ | Only when pattern is embedded-logic. Column data type. Accepts a canonical FLUID/SQL type name (case-insensitive) or a parameterized form such as decimal(18,4) or varchar(255). The enum lists the canonical bare types; the pattern additionally permits case variants and parameter suffixes. Intentionally generous — it rejects obvious garbage (e.g. MYSTERY_TYPE) while admitting the cross-dialect superset. | |
builds[].properties.stream.key.schema[].key.required | boolean | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.key.schema[].key.fields | array of object | no | min items 1 | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.key.schema[]: read that section with this path as the prefix. | |
builds[].properties.stream.key.schema[].key.element | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.key.schema[].key: read that section with this path as the prefix. | ||
builds[].properties.stream.key.schema[].key.key | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.key.schema[].key: read that section with this path as the prefix. | ||
builds[].properties.stream.key.schema[].key.value | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.key.schema[].key: read that section with this path as the prefix. | ||
builds[].properties.stream.key.schema[].value | object | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.key.schema[].value.type | string | yes | one of 79 valuesstring text varchar varchar2 nvarchar char nchar character clob int integer int2 int4 int8 int16 int32 int64 tinyint smallint mediumint bigint long longint serial bigserial float float4 float8 float32 float64 double real decimal dec numeric number bignumeric money boolean bool bit date time datetime datetime2 smalldatetime timestamp timestamptz timestamp_tz timestamp_ntz timestamp_ltz timestampntz interval year variant object array struct map record row json jsonb super binary varbinary bytes blob bytea raw geography geometry geom point uuid uniqueidentifier guid enum hllor matches a 684-character pattern(?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*((\s*[0-9A-Za-z_,\s'"-]\s))?\s*$ | Only when pattern is embedded-logic. Column data type. Accepts a canonical FLUID/SQL type name (case-insensitive) or a parameterized form such as decimal(18,4) or varchar(255). The enum lists the canonical bare types; the pattern additionally permits case variants and parameter suffixes. Intentionally generous — it rejects obvious garbage (e.g. MYSTERY_TYPE) while admitting the cross-dialect superset. | |
builds[].properties.stream.key.schema[].value.required | boolean | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.key.schema[].value.fields | array of object | no | min items 1 | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.key.schema[]: read that section with this path as the prefix. | |
builds[].properties.stream.key.schema[].value.element | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.key.schema[].value: read that section with this path as the prefix. | ||
builds[].properties.stream.key.schema[].value.key | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.key.schema[].value: read that section with this path as the prefix. | ||
builds[].properties.stream.key.schema[].value.value | object | no | Only when pattern is embedded-logic. Same fields as builds[].properties.stream.key.schema[].value: read that section with this path as the prefix. | ||
builds[].properties.stream.key.schema[].name | string | yes | min length 1 | Only when pattern is embedded-logic. | |
builds[].properties.stream.key.schema[].type | string | yes | one of 79 valuesstring text varchar varchar2 nvarchar char nchar character clob int integer int2 int4 int8 int16 int32 int64 tinyint smallint mediumint bigint long longint serial bigserial float float4 float8 float32 float64 double real decimal dec numeric number bignumeric money boolean bool bit date time datetime datetime2 smalldatetime timestamp timestamptz timestamp_tz timestamp_ntz timestamp_ltz timestampntz interval year variant object array struct map record row json jsonb super binary varbinary bytes blob bytea raw geography geometry geom point uuid uniqueidentifier guid enum hllor matches a 684-character pattern(?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*((\s*[0-9A-Za-z_,\s'"-]\s))?\s*$ | Only when pattern is embedded-logic. Column data type. Accepts a canonical FLUID/SQL type name (case-insensitive) or a parameterized form such as decimal(18,4) or varchar(255). The enum lists the canonical bare types; the pattern additionally permits case variants and parameter suffixes. Intentionally generous — it rejects obvious garbage (e.g. MYSTERY_TYPE) while admitting the cross-dialect superset. | |
builds[].properties.stream.key.schema[].required | boolean | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.key.schema[].description | string | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.key.schema[].sensitivity | string | no | one of 12 valuesnone internal confidential restricted pii phi cleartext treated anonymized pseudonymized tokenized encrypted | Only when pattern is embedded-logic. Field-level sensitivity classification for improved policy ergonomics. | |
builds[].properties.stream.key.schema[].semanticType | string | no | Only when pattern is embedded-logic. Semantic meaning (e.g., 'email', 'phone', 'identifier'). | ||
builds[].properties.stream.key.schema[].businessName | string | no | Only when pattern is embedded-logic. Business-friendly name for this field. | ||
builds[].properties.stream.key.schema[].businessDefinition | string | no | Only when pattern is embedded-logic. Business definition and meaning of this field. | ||
builds[].properties.stream.key.schema[].validationRules | array of object | no | Only when pattern is embedded-logic. Field-level validation constraints. | ||
builds[].properties.stream.key.schema[].validationRules[].type | string | no | one of range, regex, enum, custom | Only when pattern is embedded-logic. | |
builds[].properties.stream.key.schema[].validationRules[].constraint | string | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.key.schema[].validationRules[].message | string | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.key.schema[].quality | array of object | no | each item: min properties 1 | Only when pattern is embedded-logic. NEW in v0.7.6: Column-level data quality rules, in ODCS's own shape — the array that is written straight into schema[].properties[].quality[] of the exported ODCS v3.1.0 contract, and the array an imported ODCS column's rules are read back into, inline on the column rather than in the importer's pass-through bucket: the ODPS importer merges a resolved port's contract block into its output-port stub and discards the sub-import's root extensions, so rules left in that bucket were dropped outright by a FLUID → ODPS → FLUID cycle. The ODCS exporter has always replayed this key verbatim and always read it back on import, but $defs.column is additionalProperties: false and did not define it, so a column written the way the exporter supports was REFUSED by fluid validate: the only way to get a hand-written quality rule into a published contract was to write an invalid contract. Optional and additive — a column without it exports exactly as it does today, with the rules forge derives from required, the primary-key tag and validationRules. Setting it REPLACES those derived rules for that column rather than adding to them, which is what makes an imported ODCS column round-trip byte-for-byte. The authority for an item's contents is ODCS v3.1.0 $defs.DataQuality, deliberately not re-declared here (a copy would drift from the vendored schema, and fluid odcs validate already checks an emitted document against it); the keys below are the common ones. Every rule should carry an id: it is optional in ODCS, but a DQ engine that labels its results by rule id rejects a contract whose rules have none. | |
builds[].properties.stream.key.schema[].quality[].id | string | no | matches ^[A-Za-z0-9_-]+$ | Only when pattern is embedded-logic. Stable rule identifier, unique within this column's rules. The pattern is ODCS $defs.StableId. Rules forge derives itself are named <column>-<metric>. | |
builds[].properties.stream.key.schema[].quality[].type | string | no | one of text, library, sql, custom | Only when pattern is embedded-logic. ODCS quality check type. text = prose; library = one of ODCS's predefined metrics; sql = a query whose result is compared; custom = a named engine's own check. | |
builds[].properties.stream.key.schema[].quality[].metric | string | no | one of nullValues, missingValues, invalidValues, duplicateValues, rowCount | Only when pattern is embedded-logic. type: library only. The five predefined metrics ODCS v3.1.0 defines; a name outside this list is not a library rule. | |
builds[].properties.stream.key.schema[].quality[].arguments | object (free-form) | no | Only when pattern is embedded-logic. type: library only. Extra arguments for the metric — pattern (a regex) or validValues (a list) for invalidValues, as DataHub's ODCS ingestion source reads them. | ||
builds[].properties.stream.key.schema[].quality[].query | string | no | Only when pattern is embedded-logic. type: sql only. The query whose result the comparison operator is applied to. | ||
builds[].properties.stream.key.schema[].quality[].engine | string | no | Only when pattern is embedded-logic. type: custom only. The engine that runs the check, e.g. soda, great-expectations, dbt. | ||
builds[].properties.stream.key.schema[].quality[].implementation | any | no | Only when pattern is embedded-logic. type: custom only. The check as that engine expresses it. | ||
builds[].properties.stream.key.schema[].quality[].dimension | string | no | one of accuracy, completeness, conformity, consistency, coverage, timeliness, uniqueness | Only when pattern is embedded-logic. The data quality dimension the rule measures. | |
builds[].properties.stream.key.schema[].quality[].description | string | no | Only when pattern is embedded-logic. | ||
builds[].properties.stream.key.schema[].quality[].severity | string | no | Only when pattern is embedded-logic. Severity of a failure. ODCS types this as a free string and documents info / warning / error by example. | ||
builds[].properties.stream.key.schema[].quality[].unit | string | no | Only when pattern is embedded-logic. Unit the comparison is in, usually rows (the default) or percent. | ||
builds[].properties.stream.key.schema[].tags | array of string | no | unique items each item: matches ^[a-z0-9][a-z0-9-]*[a-z0-9]$|^[a-z0-9]$ | Only when pattern is embedded-logic. Field-level tags for categorization and data governance. | |
builds[].properties.stream.key.schema[].labels | object (map) | no | Only when pattern is embedded-logic. Field-level labels for metadata and automation. | ||
builds[].properties.stream.key.schema[].labels.<key> | string | no | Only when pattern is embedded-logic. Label values are always strings for consistency and tooling compatibility. | ||
builds[].properties.stream.key.prefix | string | no | matches ^[A-Za-z_][A-Za-z0-9_]*(?![\s\S]) | Only when pattern is embedded-logic. Put in front of every key column's name, so a key field can share its name with a value field (a Debezium key repeats the value's id). A mapping names the prefixed column. | |
builds[].properties.stream.metadata | array of object | no | min items 1 | Only when pattern is embedded-logic. Kafka record metadata, each read as a column. | |
builds[].properties.stream.metadata[].name | string | yes | min length 1 | Only when pattern is embedded-logic. The column's name, unique among the stream's value, key and metadata columns. | |
builds[].properties.stream.metadata[].from | string | yes | one of topic, partition, offset, timestamp, timestamp-type, headers, leader-epoch | Only when pattern is embedded-logic. The metadata the column reads. | |
builds[].properties.stream.metadata[].description | string | no | Only when pattern is embedded-logic. | ||
builds[].properties.mapping | object | no | Only when pattern is embedded-logic. NEW in v0.7.7 (preview): How the stream's records become the output's rows. Requires stream, and replaces sql. | ||
builds[].properties.mapping.explode | object | no | Only when pattern is embedded-logic. NEW in v0.7.7 (preview): One output row per element of an array. A record whose array is null or empty produces no row. | ||
builds[].properties.mapping.explode.path | string | yes | matches ^[A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*|\['[^'\\\u0000-\u001f\u007f]{1,256}'\])*(?![\s\S]) | Only when pattern is embedded-logic. The array: a stream column or a nested field, such as items or order.lines. | |
builds[].properties.mapping.explode.as | string | yes | matches ^[A-Za-z_][A-Za-z0-9_]*(?![\s\S]) | Only when pattern is embedded-logic. The element's name in mapping paths: item.sku for a field of a struct element, item for a scalar element. | |
builds[].properties.mapping.columns | array of object | no | min items 1 | Only when pattern is embedded-logic. Output columns, or nested fields of them, that are read from another path, renamed or converted. | |
builds[].properties.mapping.columns[].name | string | yes | matches ^[A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*)*(?![\s\S]) | Only when pattern is embedded-logic. The output column, or a field of a struct output column as a dotted path (customer.id). Mapping a nested field builds that struct from the paths given, so a nested output can be written from fields that are flat or elsewhere in the record. | |
builds[].properties.mapping.columns[].from | string | yes | matches ^[A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*|\['[^'\\\u0000-\u001f\u007f]{1,256}'\])*(?![\s\S]) | Only when pattern is embedded-logic. The source: a stream column (value, key or metadata), a nested field (customer.address.country), a map entry (attributes['channel'], null when the key is absent) or the exploded element (item.sku). | |
builds[].properties.mapping.columns[].cast | string | no | one of 79 valuesstring text varchar varchar2 nvarchar char nchar character clob int integer int2 int4 int8 int16 int32 int64 tinyint smallint mediumint bigint long longint serial bigserial float float4 float8 float32 float64 double real decimal dec numeric number bignumeric money boolean bool bit date time datetime datetime2 smalldatetime timestamp timestamptz timestamp_tz timestamp_ntz timestamp_ltz timestampntz interval year variant object array struct map record row json jsonb super binary varbinary bytes blob bytea raw geography geometry geom point uuid uniqueidentifier guid enum hllor matches a 684-character pattern(?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*((\s*[0-9A-Za-z_,\s'"-]\s))?\s*$ | Only when pattern is embedded-logic. Convert the source to this type: binary to string (UTF-8), or a widening (int to long, float to double, a decimal to a wider precision at the same scale, a timestamp to a finer precision). | |
builds[].properties.mapping.columns[].description | string | no | Only when pattern is embedded-logic. |
Changed fields
Each row names the attribute that differs and shows both values.
| Path | Attribute | 0.7.6 | 0.7.7 |
|---|---|---|---|
fluidVersion | allowed values | one of 0.7.3, 0.7.4, 0.7.5, 0.7.6matches ^\d+\.\d+(\.\d+)?$ | one of 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.7.7matches ^\d+\.\d+(\.\d+)?$ |
fluidVersion | description | Contract schema version. Accepts '0.7.3' (source-aligned data products + acquisition pattern + ingestion engines) or '0.7.4' (adds the MCP output-port gateway + runtime agentPolicy enforcement). 0.7.4 is fully backward-compatible with 0.7.3 — every 0.7.3 contract validates as 0.7.4 unchanged. | Contract schema version. Accepts '0.7.3' (source-aligned data products + acquisition pattern + ingestion engines) or '0.7.4' (adds the MCP output-port gateway + runtime agentPolicy enforcement). 0.7.4 is fully backward-compatible with 0.7.3 — every 0.7.3 contract validates as 0.7.4 unchanged. '0.7.7' is the current preview. |
builds[].properties.sql (only when pattern is embedded-logic) | required | yes | no |
Removed fields
None. Every 0.7.6 field path is present in 0.7.7.