ADR 0009: the statistics are a distribution of their own, flux-stats
Status
Accepted 2026-11-16. Amended 2026-11-17: the record reader's pure half moves into flux-stats too, as flux_stats.reading, and the fidelity block's grammar follows as flux_stats.fidelity (see the amendments below). Amended 2026-11-18: the exact YAML read of a bundle and the canonical form move too, as flux_stats.exact_yaml and flux_stats.canonical, with PyYAML as the optional yaml extra, and a bundle's files are held to size caps before they are read.
Context
ADR 0003, as amended on 2026-11-16, says the engine never reads a record of the data a bundle imitates. A separate profiler reads records only where they live and emits only aggregates. The profiler is the forge-cli extra fluid profile, and it depends on the statistics alone, so the FLUX repositories hold no data-access code.
The statistics were already one module: flux_engine.stats held one definition per statistic, the column metrics and the profile document reader, and the fidelity check's statistic and verifier went through it. It imported the standard library and two kernel modules, flux_engine.core.errors (the FLX-nnnn catalog) and flux_engine.core.fixedpoint (the fixed-point grid), and an import-linter contract and a fresh-interpreter test held that closure. But the only way to install it was to install the engine, with numpy, pydantic, jsonschema, cryptography, flux-spec and the rest. A profiler that runs next to the data should carry none of that.
Decision
- Names. The distribution is
flux-statsand the import package isflux_stats. Both names were free on PyPI on 2026-11-16, and they followflux-specandflux_spec. - What moves.
flux_engine/statsbecomesflux_stats, andflux_engine/core/fixedpoint.pybecomesflux_stats.fixedpoint. The catalog mechanism offlux_engine/core/errors.py(the bands,register,FluxError) becomesflux_stats.errors.flux_engine.core.errorsstays: it re-exports those same objects and keeps the engine's list of the modules that own codes (ensure_core_codes_loaded). A process therefore holds one catalog and oneFluxErrorclass, every code and message reads as before, andfluxsim explainis unchanged. - No aliases.
flux_engine.statswas never in a release, so nothing re-exports it, andflux_engine.core.fixedpointgoes with no shim. Each statistic has one name. - Layout. One repository, one uv workspace.
packages/flux-statsholds its ownpyproject.toml, asrc/flux_statspackage and its own tests. The rootpyproject.tomldeclares it as a workspace member and resolves the engine's requirement to it, and oneuv.lockcovers both. A change that touches both lands in one pull request. - Versions. Lockstep. One
vX.Y.Ztag releases both distributions atX.Y.Z, and the engine depends on exactly that release (flux-stats==X.Y.Z): its verdicts are computed by these definitions, so a range would pair an engine with statistics it was never tested with. The release guard refuses a tree or a tag where the engine's version, flux-stats's version and the pin disagree. The release workflow builds, attests and attaches four files, and uploads flux-stats to PyPI before the engine. - Enforcement. import-linter now has both root packages, and its flux_stats contract forbids the whole engine and the modules the kernel may not use (
random,uuid,socket,time,os,secrets, andhashlib, which stayscore.canonical's). The kernel AST scan reads flux_stats too. A fresh interpreter that importsflux_statsloads nothing outside the standard library. The CI jobflux-stats aloneinstalls the wheel and its test group where no engine is installed and runs the package's tests. The packaging smoke installs the flux-stats wheel alone, with no index, and reads the example profile.
Prior art
Checked on 2026-11-16 before choosing (search receipts in the pull request):
- opentelemetry-python keeps
opentelemetry-apiandopentelemetry-sdkin sibling directories with onepyproject.tomleach, declares them in a root[tool.uv.workspace], releases them at one version, and pins the API from the SDK exactly (opentelemetry-api == 1.46.0.dev). - pydantic moved
pydantic-coreinto the pydantic repository (the old pydantic-core repository is archived), declares it as a workspace member and pins it exactly (pydantic-core==2.50.0), without matching version numbers. - Apache Airflow declares
airflow-coreand its other distributions as members of one root uv workspace. - LangChain keeps
langchain-coreinlibs/corewith its own version and depends on it by range (langchain-core>=1.8.0,<2.0.0). That is the alternative this ADR does not take, for the reason in decision 5. - The uv workspace documentation (members,
{ workspace = true }sources, one lockfile) and PyPI's Trusted Publishing documentation (one publisher entry per project; a pending publisher for a new name reserves nothing).
Consequences
pip install flux-statsinstalls the statistics with no third-party dependency, once a release carries it.pip install flux-sim-engineinstalls both. The engine's core gains no third-party dependency.- The name
flux-statswas free on PyPI, and the engine's exact pin would have made a squatted release run code on any plain-pip install from source. The owner registered it on 2026-11-16, before the exact pin was published, with a placeholder 0.0.1 that holds no code. Before the first release that carries flux-stats, its trusted publisher has to name this repository's release workflow (docs/RELEASING.md). - Installing from a checkout with pip names the member first (
pip install -e packages/flux-stats -e ".[dev]");uv syncneeds nothing extra. CI, the nightly and the release workflow do the former. - mutmut 3.8.1 names a mutant by its path and strips only a
src/prefix, so the weekly run mutates flux-stats frompackages/flux-stats, against that package's own tests, and the engine's modules from the root as before. The floors offlux_stats.fixedpoint,flux_stats.definitionsandflux_stats.registryare now held by the suite that ships with the package. - flux-stats supports Python 3.10 and newer, one release below the engine's floor (the maintainers, 2026-11-16):
fluid profileruns inside forge-cli, which still supports 3.10. The CI jobflux-stats aloneruns on 3.10, so nothing newer creeps in, and the floor rises when forge-cli drops 3.10. --committed-slice, the engine's one path that reads records, stays in the engine untilfluid profilecan profile a slice file.
Amendment of 2026-11-17
The maintainers decided on 2026-11-17 that the record reader's pure half moves too, in a pull request of its own after the first.
- What moves.
flux_stats.readingholds how the fidelity check reads a record: the cell a column names, refusing an absent column (cell); a number as the exact rational it spells (exact); a range column's value, a number or an instant (read_exact), checked against the declared range and put on the grid (encode_value); a category as its NFC string (category); the declared levels in their one order (level_cells); the null policy (NullPolicy,read_column,read_pairs); and the groups (group_keys,UNGROUPED). With it gonfcandparse_rfc3339_ms, which it calls,ValueDomain, and the two codes it raises that the engine registered,FLX-5022andFLX-5031, with their titles and explanations unchanged. Each function asks a column only for what it reads: a name (NamedColumn) to read a cell or a category, a name and a domain (ValueColumn) to read a range column's value, and the five attributes ofColumnfor the record-set readers. No signature carries one of the engine's compiled types, and a profiler, which declares no range, reads a value without inventing one. - What stays. The engine's names for these are imports of the same objects:
parse_rfc3339_msandFLX_5022influx_engine.core.clock,nfcinflux_engine.core.canonical,NullPolicyandValueDomaininflux_engine.concordance.predicate,FLX_5031influx_engine.concordance.errors, andencode_value,level_cells,RowandUNGROUPEDin the boundary's statistic module. Some forty engine modules and tests call them, and keeping the names keeps one implementation without touching any of them, asflux_engine.core.errorsdoes for the catalog. Which columns a property reads and which definition measures them stays in the engine (compute_property_statistic), and so does the boundary, which reaches the records. - One change of behaviour.
parse_rfc3339_msreads the pattern's groups into thedatetimeconstructor instead of callingdatetime.fromisoformat, whose accepted forms differ between Python versions: 3.10 refusesZand a fraction of other than 3 or 6 digits, and 3.14 accepts hour 24. On 3.11 to 3.13, the versions the engine is tested on, it returns what it returned before, value for value and refusal for refusal, on the 198,219 inputs of the pull request's differential check, and it returns the same on 3.10 and 3.14. A date-time whose instant falls outside the years 1 to 9999 once moved to UTC (0001-01-01T00:00:00+00:01) used to escape as anOverflowError; it is now refused withFLX-5022.
Consequences of the amendment:
fluid profilereads a value by calling these functions, so forge-cli implements no reading of its own.- The weekly mutation run mutates
flux_stats.readingwith flux-stats's own tests, at the reader's floor of 95%. - ADR 0003's one exception stands. The boundary still reads committed records inside the engine, and the engine depends on flux-stats, so installing the engine still installs a record reader.
- Two risks moved with the code. The first is closed:
exactformed theFractionof a decimal before anything bounded its size, so a cell such as1e99999999999took unbounded time and memory, as it did in the engine's reader. Since a follow-up on 2026-11-17, before forge-cli reads with the module and before any release carries it,exactrefuses a number that needs more than 1000 digits written out (FLX-5005), counted bydigits_written_out, the declaration reader's own count, before forming it. The second stays open:nfcnormalises with the running Python's Unicode database (13.0 on 3.10, 16.0 on 3.14), so a string holding a character newer than one of two Pythons can be keyed differently by each. A follow-up records the Unicode version with a profile and with a run. - The parser keeps one leniency of the call it replaces, so that no value changes: an offset's minutes are added to its hours, so
+00:60reads as+01:00where RFC 3339 caps the minutes at 59. Tightening it is a change of behaviour of its own.
Amendment of 2026-11-17: the fidelity block
The maintainers decided on 2026-11-17 that the fidelity block's grammar moves too, in a pull request of its own after the reader's.
- What moves.
flux_stats.fidelityholds how a Simulation's fidelity block is read: the expose it is checked against (ExposeSchema,DeclaredExpose,expose_from_contract), the closed sets (ColumnKind,PropertyKind,ToleranceKind,FailureAction), the declarations (Encoding,RecordsRange,ColumnDeclaration,Tolerance,PropertyDeclaration,Budget,FidelityBlock), the one key set of a column (COLUMN_KEYS),read_support_levels,parse_duration_ms,require_number_lengthandparse_fidelity_block. With it godeclared_enum_values, which reads the levels of anenum(...)type and whichsupport: declaredcalls, and the codes the grammar raises,FLX-3001,FLX-5030,FLX-5032,FLX-5033andFLX-5034, with their titles and explanations unchanged. The module takesnfcandparse_rfc3339_msfromflux_stats.readingandFluxErrorfromflux_stats.errors, which are the objects the engine already re-exported. - What stays. The engine's names for these are imports of the same objects: every name of
flux_engine.concordance.predicate, the four codes influx_engine.concordance.errors, andFLX-3001anddeclared_enum_valuesinflux_engine.seam.contract. No engine module or test changes its imports. Which columns a property reads and which definition measures them stays in the engine (compute_property_statistic, which also keeps therecords.minfloors and the property ids). - The per-group measurement moves with it.
measure_groupforms one group of one property from the parsedPropertyDeclaration, the declared columns and the grid: support with itsmaxCardinalityrefusal, a level histogram with its refusal of a value outside the support, a correlation from its pairs, and a mean, a histogram on edges put on the grid, or a quantile from the encoded values. It was the boundary's_group_statistic, which read nothing of its compiled property but the declaration. The boundary calls it, with the same codes and messages. - No change of behaviour. The code is moved, not rewritten, and
fluxsim explainprints the same text for each code.
Consequences of the amendment:
- A profiler such as forge-cli's
fluid profile --bundlereads a Simulation's fidelity block withflux_stats.fidelityand forms each group of a property withmeasure_group, so it implements no parser and no group reader of its own. - The weekly mutation run gains
flux_stats.fidelity, mutated with flux-stats's own tests. Its floor is the 75% target until a first run records its kill rate, and it rises when that rate is known. import flux_statsnow loads the module, so every code it registers is in the catalog without the engine; the closure a fresh interpreter loads grows by one module and stays standard library only.
Amendment of 2026-11-18: the bundle read
The maintainers decided on 2026-11-18 that flux-stats exports the read of a bundle that the engine takes its digests over, so that forge-cli's profiler and the engine read a bundle with one reader. flux-stats's core stays standard library only, and an optional extra brings PyYAML.
- What moves.
flux_stats.canonicalholds the canonical form of KERNEL-SPEC §2.1:canonicalise,coerce_key,render_decimal,JCS_MAX_SAFE_INTEGERand the codes they raise,FLX-1001andFLX-1003toFLX-1007, with their titles and explanations unchanged.flux_stats.exact_yamlholds the exact YAML read:ExactLoader(no float, no datetime, every number exact) andFLX-3105, withload_documents, the read of a text's documents, andcheck_documents, which holds a text to the read's bounds before another reader reads it. Both take text and open no file. - What stays.
flux_engine.core.canonicalre-exports those names and keeps the steps that need more than the standard library or that make a digest: the RFC 8785 bytes (rfc8785), the sha256 (hashlib), masking and field paths.flux_engine.seam.resolvere-exportsExactLoaderandFLX_3105, and keeps which files a bundle reads, how it opens them, and the budget each of the bundle's two reads shares across its files. - The extra.
flux_stats.exact_yamlimports the standard library alone, soimport flux_statsloads nothing else and still registers every code. PyYAML is imported when a document is first read orExactLoaderis first named, and without it either raises anImportErrorthat namespip install 'flux-stats[yaml]'. The engine depends on PyYAML itself, so its requirements do not change. PyYAML is in flux-stats'stestgroup, so theflux-stats alonejob runs the read's tests. - Bounds. The read carries over the bounds forge-cli's port of it gained in review. A document may write out at most 1,000,000 values once its aliases and merge keys are written out, and nest at most 64 levels, counted through its aliases; a document that holds itself through an alias is refused. The merge keys of one read of a bundle copy at most 100,000 pairs. A number needs at most 1,000 digits written out. Text nested too deeply for the parser is refused instead of raising
RecursionError. Each bound refuses withFLX-3105, naming the file and the line, never a value. The largest bundle in this repository writes out 1,106 values, nests 11 levels and uses no merge key, and a bundle within the bounds reads to the same documents as before. The read also bounds the mappings the merge keys of a document name (1,000,000, each counted for every mapping that holds the key) and the characters a document writes out (64 MiB), since a string is one value however long it is and a number is spelled in full: it counts as the digits its decimal is spelled in when that is more than its text, and a zero counts the digits its scale spells, since the canonical form keeps the scale (digits_written_out, which the fidelity block's readers share). The main branch of the FLUX validator holds a document to the same bounds since 2026-11-19 (Agenticstiger/flux#37, not yet released), counting a number as its text. - The validator's read. The FLUX reference validator reads a bundle with a plain SafeLoader, which bounds nothing in any release so far. So before it reads a file, the engine holds the file's text to the bounds with
check_documents, decoded as the validator decodes it (in the locale's encoding, asPath.open()opens it): a file decoded two ways can hold a document that one decoding shows and the other hides in a comment. The check also builds each integer, float and boolean scalar as that read would, so one it cannot build is refused by its position. A file that does not decode, or that holds a character YAML does not allow, is refused there, because the validator meets either only where the file holds it, after building every document before it. A path that is not a regular file is refused without being read. - File sizes. Each read takes a file whole, so the maintainers decided on 2026-11-18 that a bundle's files are held to two caps by their sizes, checked before a byte is read: 4 MiB for one file (
MAX_FILE_BYTESinflux_engine.seam.resolve, the cap forge-cli's profiler holds a bundle's files to), and 256 MiB for its documents and lockfile together (MAX_BUNDLE_READ_BYTES, flux_spec's cap on an archive's members), so a directory and the archive packed from it meet one limit. Both are constants, as every other bound is.load_bundletakes every file's size withstatbefore it opens any, and each read (read_bundle_file) takes the file'sstatagain, refuses a path that is not a regular file, such as a FIFO or a device, with no byte read, and reads at most one byte past the cap. Past either cap, the read refuses withFLX-3105, naming the file and its size or the bundle's total. In an evidence archive,verify_evidenceholds each document, lockfile and pack product to the file cap by the size its header declares, before flux_spec reads it, and flux_spec's own cap of the same size holds the archive's members together.
Consequences of the amendment:
- forge-cli's
fluid profile --bundlereads a Simulation with these modules once a flux-stats release carries them, and itsprofileextra asks forflux-stats[yaml], so its own port of the read goes. - The flux_stats import-linter contract does not change.
exact_yamlimportssys, for CPython's limit on an integer's digits (4300 on Python 3.10.0 to 3.10.6, which have none), and PyYAML only inside functions. - The check and the validator open a file separately, so a file changed between the two is read unchecked. A bundle is read from a directory the person running the engine controls, which this accepts.