Seam generator and fidelity check: normative specification (v0.1)
1. The contract-schema generator (generator/)
Input: a resolved expose (seam/contract.py: compile_expose) → ordered columns with LogicalType. For each column the generator picks a family from the column's declared aggregate statistics (§3) or, absent statistics, from the type default:
| Column type (6) | Default family | Families allowed (4) |
|---|---|---|
| STRING categorical (declared support or ≤ 64 distinct) | categorical | categorical, empirical-histogram |
| INT | poisson/exponential-rate (counts) or uniform (ids) | categorical, bounded-normal, rate, uniform/empirical |
| DECIMAL(p,s) | bounded-normal | bounded-normal, rate, uniform/empirical, categorical |
| BOOL | categorical (Bernoulli) | categorical |
| DATE / TIMESTAMP | uniform within timeBounds or rate (inter-arrival) | uniform/empirical, rate |
identifier (STRING/uuid with unique, or a one-column STRING key the contract declares: KERNEL-SPEC section 8.1) | derived deterministically from (H, ordinal), never random | none |
Every draw uses the §4 kernel samplers on the path ("sim", sim_id, "gen", exposeId, "row", rowIndex, column). A column declared given another (§2) keeps that path: what it reads from the other column is the level that column drew in the same row, which picks the weights it draws with.
Keys and foreign keys. The columns a FLUID 0.7.7 contract makes keys or foreign keys are produced as KERNEL-SPEC section 8.2 says, so the exposes of one contract are generated together: a key is laid out from the row index and never repeats, and a foreign key takes its parent's key at a parent row. A foreign key that declares a draw draws on its own path as above and holds the parent's nearest key; one that does not picks its parent on the stream of its relationship's first column. Such a column needs no declaration of its own. A one-column key may declare only a range, or bins with binWeights; a column of a several-column key, a range, bins or levels. Rows are produced in rowIndex order; the number of rows is declared (extensions["com.agentics.flux/1"].emits[<exposeId>].rows, §2; default = population size when the emit is scenario-driven).
Two row sources, and the declaration picks. The generator above runs for an emit whose extension block declares a fidelity block: declared statistics decide every value, and the fidelity check is specified over them. An emit that declares no block is scenario-driven: its rows are the run's own moments, and one detector firing for one subject is one row.
The scenario path (seam/mapping.py, stages/pipeline.py):
- the moment is the detector fire, not the journey transition. A transition exists only where a Journey's trigger held and a persona reaction was drawn, so journey-driven rows would silently drop every burst the detector found outside a journey's reach; the detection also carries its matched events, so every mapped column has a source.
- the mapping is resolved at plan time, once per emit, by three steps in fixed order, first match wins: (1) the extension block's
mappingoverride; (2) the vertical's alias table (verticals/<pack>.py, e.g.subscriber_id → subject.id,aborted_at → event.time,amount → event.data.amount,market → subject.market); (3) the column's own name, searched underevent.data.,subject.,subject.traits.in that order. A required column no step resolves isFLX-3302, naming the column, before any row exists; an optional one renders JSONnull. - an INT column's source is checked in the same step: it must be integral (KERNEL-SPEC §9), read off the declared families and never off the values a run drew, or the emit is
FLX-3303, naming the column and the source, before any row exists. ADecimalfrom an integral source reaches the renderer as the exactintit equals. - the moment's event is the last matched event of the detection: the one that completed the pattern.
- rows are scheduled through KERNEL-SPEC §7's scheduler at
phase = EMIT, keyed on that event's instant, and written through the sameEmitWriter;outputDigestis unchanged (§8 of KERNEL-SPEC, computed above the sink). - a declared
rowscaps a scenario-driven stream; a run with fewer moments emits what it has and never pads.
Coverage. Whichever way an emit's rows are produced, every column has one producer, and the plan says which: declared (its entry under fidelity.columns decides the draw, or, for a foreign key, decides the value it then moves to the parent's nearest key), copied (a runtime attribute, by one of the three steps above), derived (an identifier, from (H, ordinal); a key the contract declares, laid out from the row index; a foreign key whose parent row its relationship picks), profile (reserved; nothing produces it yet) or default. A column no declaration reaches takes its type's default and is a default: an optional column the mapping resolves to nothing (null), or a DATE or TIMESTAMP column that declares no draw of its own (§1's table: uniform within timeBounds). A default is named by plan, recorded by apply in column-provenance.json and listed under "What Was Not Proven" in EVIDENCE.md. Under --strict-coverage it is FLX-3304, naming every such column of every emit, before any row exists. plan.json records the digest of the record as columnProvenanceDigest (the raw-bytes digest of the file, as seedTreeDigest is of seed-paths.ndjson), so planDigest, the run id and the run digest cover it. apply refuses to write a record the plan does not name, and ship and verify read the record only through that digest and refuse a missing or moved one with FLX-2115; a plan without the key binds no record, and the report says so. The setting changes whether a run is accepted and never what an accepted run produces: the same bundle and seed give the same plan.json, runDigest and rows with --strict-coverage on or off, at any --jobs.
Strict coverage in the bundle. A Simulation can ask for the same refusal itself: strictCoverage: true at the top level of the namespace (§2). The run is strict when the flag or the declaration asks, and the flag cannot turn a declaration off, because the declaration is part of the bundle the run's inputDigest covers and a run that could switch it off would not be the run the bundle declares. The refusal names which one asked, so the way out is plain: drop the flag, remove the key, or both. The value is a boolean and nothing else (FLX-3105); false and a null declare nothing. The declaration is a bundle edit, so it moves inputDigest and with it the run id, but plan.json has the same keys with it as without (the record's columnProvenanceDigest included), and a run it accepts makes the rows and every component digest it makes without it.
2. The fidelity block (lives in Simulation.extensions["com.agentics.flux/1"].emits[<exposeId>].fidelity)
fidelity is the block's only spelling in a document. The subsystem keeps its own module name (flux_engine.concordance), and that name stays inside the package: a second accepted key would be two digests of one contract (the same reason §2 refuses a bare support list below) and it would put a reserved word into a file an author writes.
Why it does not sit in spec.emits[]. FLUX 0.5.0 declares Simulation.spec.emits[] items with additionalProperties: false, so a bundle carrying fidelity or rows there is refused by the published validator and by the packer; the bundle cannot ship. RFC-02's document-level extensions map is the one sanctioned extension point (propertyNames = reverse-dns/major, contents owned by the namespace), so this engine's declarations live there and every bundle stays valid against the unmodified 0.5.0 schema.
The namespace carries five keys and no others: emits, keyed by exposeId as below; profile, which pins a profile document by its digest (PROFILE-SPEC.md §6); scorecard, which declares thresholds on the scores that profile gives the run (SCORECARD-SPEC.md §4); parameters, the parameter register, which says what each number the run reads rests on (PARAMETER-REGISTER-SPEC.md); and strictCoverage, a boolean (§1). Any other key is refused (FLX-3105). A new block starts at the top level of the namespace when what it declares belongs to the Simulation rather than to one expose. An emit's block carries five keys and no others: rows, mapping, fidelity, invariants and rules. invariants holds FLUX settings for the rules of the emit's contract (exposes[].contract.invariants[]), each keyed by the invariant's id (EXPR-SPEC-2.md §1, §6). An invariant's settings carry generates, the column a bounds or difference form generates. Step 3 of ADR 0011 adds two more keys and no others: distribution, the draw of a bounds form relative to a parent value (EXPR-SPEC-2.md §6, rules over a relationship), and target, the percent of covered rows that violate (EXPR-SPEC-2.md §7). The setting for a rule's violations that EXPR-SPEC-2.md §7 names comes with a later step; until then a violation's candidates are drawn from the column's own statistics. rules holds the emit's FLUX-side rules, a list of entries that each name their language (EXPR-SPEC-2.md §1).
extensions:
com.agentics.flux/1:
emits:
payment_recovery_moment: # keyed by the emit's exposeId
rows: 1000 # §1's declared row count
mapping: # optional per-column source overrides (§1)
aborted_at: event.time
invariants: # optional FLUX settings for the contract's rules, keyed by the
date_of_key: # invariant's id (EXPR-SPEC-2.md §1)
generates: d_date # the column a check of difference form generates
rules: # optional FLUX-side rules (EXPR-SPEC-2.md §1), a list
- id: day_of_month
language: flux-expr/3
derive: d_dom
expr: EXTRACT(DAY FROM d_date)
fidelity:
encoding: { fractionalBits: 16, rounding: floor }
records: { min: 1000, max: 100000 }
columns:
amount: { range: { min: "0", max: "10000" }, null: exclude }
market: { support: { values: ["IE","GB","FR","DE"] }, maxCardinality: 8, null: fail }
aborted_at: { range: { min: "2026-07-01T00:00:00Z", max: "2026-09-30T23:59:59Z" }, null: fail }
properties:
- { property: mean, column: amount, tolerance: { type: relative, value: "0.02" } }
- { property: marginalHistogram, column: amount, bins: ["0","50","100","250","1000","10000"], tolerance: { type: distance, measure: totalVariation, max: "0.05" } }
- { property: support, column: market, tolerance: { type: count, max: 0 } }
- { property: mean, column: amount, groupBy: market, tolerance: { type: relative, value: "0.03" } }
budget: { maxChecksPerState: 20, maxStateAge: P30D }
failureAction: withholdBundle
spec:
emits:
- productRef: telco.gold.payment_recovery_moment
exposeId: payment_recovery_moment
fluidVersion: "^0.7.3"
Rules enforced by fluxsim validate (offline): every column and groupBy is a member of the referenced expose schema; every key of extensions[...].emits is an exposeId the Simulation actually emits; bins are strictly increasing and partition the declared range; a levelHistogram names a support column and carries no bins; a quantile may carry bins, and no other kind but a marginalHistogram does; a correlation property declares two columns and a minVariance; every property's parameters are present (a contract missing any of them is refused before any statistic is formed). Numbers are strings (exact decimals), never floats. Written out without an exponent, a number has at most 1000 digits, not counting a leading 0.: "1E+99999999" names a number of a hundred million digits and is refused (FLX-5030) by its length, before its value is formed.
A quantile may carry bins. They follow a marginalHistogram's rules: strictly increasing, partitioning the declared range, on the grid. A quantile with bins is formed and compared by quantile/2 (ADR 0012), which takes the order statistic as quantile/1 does and states, in place of its value, the bin that holds it and the records below and through that bin. Its tolerance is absolute, as every quantile's is, and the check passes when the synthetic order statistic lies within τ′ grid steps of the reference bin (§5). A quantile without bins is quantile/1, unchanged.
columns.<name> is one schema, read by two consumers. range, support, maxCardinality and null are read by the check (§5); bins, binWeights, binWeightsGiven, family, given, identifierLength, levelWeightsGiven, levels, mean, nullFraction, rate, stddev and unique are read by the §1 generator. The set is closed and shared (concordance.predicate.COLUMN_KEYS): a key outside it is refused by both, and neither refuses a key the other owns. A column entry that declares neither range nor support declares nothing the check can compare; it is legal (the generator still reads it), and a property naming such a column is FLX-5034.
support has exactly two spellings. { values: [...] } writes the levels out. declared takes them from the FLUID column type's own enum list: type: "enum('IE','GB')", the only structured value set FLUID 0.7.5 carries (validationRules[].constraint is free text with no grammar). A column that spells declared over a type naming no levels is refused (FLX-5034) rather than resolved to an empty support, because an empty support makes every value a non-member and fails the check for the wrong reason. A bare list is refused: two spellings of one declaration would be two digests of one contract.
levelHistogram counts a categorical column. It names a support column, whose declared levels, NFC-normalised and sorted, are its cells, as a marginalHistogram's declared bins are its cells. It carries no bins, and its tolerance is a total-variation distance. With groupBy (a second support column) it is a two-way count table of the two columns: one count per (group, level) cell, compared group by group as a grouped marginalHistogram is. A value of either column outside its declared support has no cell, so it is refused (FLX-5005) rather than counted, as a value outside a declared range is. Every group needs records.min records, as every group does. A declared level the state never holds is a cell of count zero, so the table's shape comes from the declaration and never from the data.
given draws a column conditionally on another. given names one other column of the same expose, and the column declaring it carries one weight table per level of that column, in place of its unconditional weights:
- a histogram column (declared
bins) declaresbinWeightsGivenin place ofbinWeights: for each level of the given column, onebinWeightslist over the same edges, P(bin | level). A groupedmarginalHistogramproperty (groupBythe given column, over the same bins) states exactly these lists: its (group, bin) counts; - a categorical column declares
levelWeightsGivenin place oflevels: for each level of the given column, one mapping of level to weight, P(level | level), from a two-way count table. Every mapping names the same levels.
The weights are counts or shares, normalised per level of the given column as binWeights is, and every list or mapping carries some weight. The given column is a categorical with no nullFraction, so each row has exactly one level to look up, and the table names exactly the levels it can draw. The links form a forest: a column has at most one given, a cycle is refused, and a histogram column is always a leaf, because a bin is not a level. A column is drawn after the column it is given, whatever the declared column order; every cell keeps its own §1 path, so column-first generation and slices are unchanged, and a root of the forest (a column drawn given nothing) draws exactly the values it drew without the links. A declaration the generator cannot draw is refused at plan time with FLX-3402. A plan that declares no given documents none of the three keys, so its generatorDigest and its rows are the ones it had before they existed.
flux_engine.calibrate.structure suggests the forest offline, from two-way count tables alone: a Chow-Liu tree (Chow and Liu, 1968), the maximum spanning tree of pairwise mutual information, which is the selection step of the MST synthesizer (McKenna, Miklau and Sheldon, 2021) without its privacy noise, with every numeric column a leaf. It reads counts and writes the columns.<name> entries; it never reads a record. Counts can still single out a rare combination: a cell of one or two counts describes a handful of real records, so the owner applies their own disclosure rule, such as a minimum cell count, before declaring a table.
What the check covers of a given link. Both kinds of link are checked end to end, by a grouped property over the drawn column with groupBy the given column. A histogram drawn given a categorical is checked by a grouped marginalHistogram over the same bins, and a categorical drawn given a categorical by a grouped levelHistogram. Each compares D′'s (group, cell) counts with S's, group by group. They are the counts the link is declared from: a grouped marginalHistogram's (group, bin) counts are binWeightsGiven's lists, and a grouped levelHistogram's (group, level) counts are levelWeightsGiven's mappings. flux_engine.calibrate.structure reads them back as two-way tables (pair_table_from_groups, pair_table_from_level_groups), so the forest can be suggested and declared from a committed S. A root's own levels are checked by an ungrouped levelHistogram over it. A pair of columns that no declared property counts together is not checked.
3. Digests
H=jcs-documentdigest of{"contract": <parsed .fluid.yml>, "emit": <emits[] entry>, "extension": <the emit's entry inextensions["com.agentics.flux/1"].emits, when it has one>, "columns": [declared column names in order]}. Theextensionkey is present only when the emit has such an entry, so a bundle that declares none keeps theH, the seed root and the whole stream it had before the block existed. Computed byfluxsim plan, printed, and stored inplan.json; the run seed derives from it (KERNEL-SPEC §3).G= the same with the excluded set removed from the emit entry and from the extension block: everytolerance, everyminVariance,budget.maxStateAge.G ⊂ H. A change that leavesGunchanged keeps the committed statistic reusable.εper property, computed by the compiler fromencoding,columns[].range,records.min(andminVariancefor correlations) by interval propagation;τ′ = τ − ε;ε ≥ τ→FLX-5002at plan time;τ′rounded onto the grid in the tightening direction.- Both
HandGand everyεare written toplan.json. The emit's entry carries the compiled block underfidelity, the key the block is declared under:exposeId,H,G,encoding,fieldCapacityBits, and per property itsepsilon,margin(τ′),maxNonMembersandmagnitudeBits. An emit that declares no block carries no such key.
4. Boundary side (concordance/boundary/, invoked by fluxsim apply --boundary from Playback.sourceStream)
- Committed state: the owner fixes the record set (
file://slice in v0.1; partition/sequence boundary recorded);stateOrdinalincrements per state. - C1: per-column Merkle root (sha256 in v0.1; the leaf function is a port so it can become circuit-friendly later) over fixed-point-encoded values padded to
records.maxwith the reserved padding encoding; nulls carry the null encoding; a validity flag per position. - S (derived statistic): per property the aggregate quantities only: bucket counts on the declared edges, level counts on the declared support,
Sx,Sy,Sxy,Sxx,Syy, the sorted committed support, the quantile witness (for a quantile with bins, the bin that holds it and the records below and through that bin, never the witness),n,kper property and per group. - C2: sha256 over canonical
Splus a per-state blinding value that never leaves the boundary. - What leaves the boundary in v0.1:
S,C1,C2,stateOrdinal, record count (or declared range), commitment time. Never a record. The check counter (budget.maxChecksPerState) lives here and refuses beyond budget (FLX-5010).
5. Verifier (concordance/verifier.py, ClearStatistic backend)
Inputs: the contract set (recomputes H, G), S, C1, C2, and D′ (recomputes D′'s statistics on the same grid and D′'s digest = outputDigest). Evaluates each property in the integer cross-multiplied form:
- mean:
|Sx·n′ − Sx′·k| ≤ τ′·k·n′(absolute) or the relative homogenised form; - total variation:
Σ_bins |n′·c − k·c′| ≤ 2·τ′·k·n′; - level histogram: the same form over the declared levels,
Σ_levels |n′·c − k·c′| ≤ 2·τ′·k·n′, withε = 0; - correlation:
L·R ≤ A ≤ U·Rsplit by signs, withQx ≥ k²·v,Qy ≥ k²·v; - quantile: order-statistic rule on both sides; with bins (
quantile/2), the rule read at the reference bin's edges,binBelow < j ≤ binThroughwithj = ⌈level·k⌉, the synthetic order statistic's own ranks, ande_i − τ′ ≤ w′ ≤ t_i + τ′for that statisticw′, wheret_iis the bin's largest grid value (e_{i+1} − 1, or the last edge for the last bin); - support: count of non-members ≤
max. Fail-closed conditions:k = 0, count outsiderecords, group belowrecords.min, a value outside its range or its declared support, a value that needs more than 1000 digits written out (counted as §2 counts a declared number, and refused before it is formed), a column absent,ε ≥ τ. No proof is generated in v0.1; theProofSystemport has one implementation,none, and the record says so.
Each comparison above is one registered statistic definition (flux_stats), so a property's reference can also come from a profile (PROFILE-SPEC.md §7) and be compared in the same integer form. There a group or a cell the profile suppressed is censored, never scored as zero, and a profile that holds no statistic of the property's shape is refused (FLX-5037). The record of §6 is written for a committed state only. verify and ship score the profile path's outcomes in a scorecard per property, table, dimension and overall (SCORECARD-SPEC.md).
6. The fidelity record (concordance/record.py): one per verified emit, in the evidence pack
{
"recordVersion": 1,
"reference": { "productRef", "exposeId", "fluidVersion", "contractDigest" },
"H": "sha256:…", "G": "sha256:…",
"committedState": { "ordinal", "C1", "C2", "recordCount" | "recordRange", "committedAt", "checkOrdinal" },
"synthetic": { "outputDigest", "rows", "generatorDigest" },
"properties": [ { "property", "column(s)", "groupBy?", "tolerance", "margin", "epsilon", "outcome": "within|outside" } ],
"proofSystem": "none",
"scope": {
"certifies": "statistical relation between the synthetic dataset and the committed state under the registered properties",
"doesNotCertify": ["privacy or re-identification resistance of the synthetic dataset", "fairness or any property of the source dataset itself", "any property of a system trained on or tested with the synthetic dataset", "that the synthetic dataset was generated independently of the source", "that the committed state is the whole source dataset", "predictive validity of any simulation outcome"]
},
"columnsCovered": [...],
"signature": { "alg": "ed25519", "keyId", "sig" }
}
The scope statement is generated from a fixed template, never hand-edited; its doesNotCertify list is the register-discipline guard. The record is signed by the verifier key (ed25519 via cryptography), and travels in the evidence pack under run/fidelity-record-<exposeId>.json.
7. The gate (concordance/gate.py, invoked by fluxsim ship)
Recompute H over the contract set; verify the record's binding to H and its signature; recompute D′'s digest and statistics from the rows actually placed in the pack; re-evaluate every property; check committedAt against budget.maxStateAge. On any failure: write no bundle, exit 1 with FLX-8001, emit a CloudEvent com.agentics.flux.ship.withheld to the configured sink (stdout NDJSON by default). On success: pack (staged copy; the .fluid.yml copied byte-for-byte) and print the public re-verification commands.
When the run's plan pins a profile, ship takes it with --profile and compares every emit's properties with it as well (PROFILE-SPEC.md §7). An evaluation that is not within withholds the emit, and so does a missed scorecard threshold or a measured score below the emit's minCredibility (SCORECARD-SPEC.md §5). With a committed slice too, the emit ships only when both references find it within. The comparison with a profile writes no record; the scorecard it feeds travels in the pack as run/scorecard.json.
8. What a change does
| Change | Effect |
|---|---|
tolerance / minVariance / maxStateAge only | H moves, G unchanged → re-run the check against the cached S, new record |
| any other declaration in the block or the contract | G moves → new committed state required (FLX-5011) |
generator (engine kernelVersion) | generatorDigest moves → re-materialise D′, new record against the same state |
| appended source records | new committed state (ordinal+1); additive quantities folded, quantiles recomputed |