Fluid Forge
Get Started
See it run
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
Demos
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
GitHub
Get Started
See it run
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
Demos
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
GitHub
  • Introduction

    • Home
    • Getting Started
    • Snowflake Quickstart
    • See it run
    • Forge Data Model
    • Vision & Roadmap
    • Playground
    • FAQ
  • Concepts

    • Concepts
    • Builds, Exposes, Bindings
    • What is a contract?
    • Quality, SLAs & Lineage
    • Governance & Policy
    • Agent Policy (LLM/AI governance)
    • Providers vs Platforms
    • Fluid Forge vs alternatives
  • Data Products

    • Product Types — SDP, ADP, CDP
  • Walkthroughs

    • Walkthrough: Local Development
    • Source-Aligned: Postgres → DuckDB → Parquet
    • AI Forge And Data-Model Journeys
    • Walkthrough: Deploy to Google Cloud Platform
    • Walkthrough: Snowflake Team Collaboration
    • Declarative Airflow DAG Generation - The FLUID Way
    • Generating Orchestration Code from Contracts
    • Jenkins CI/CD for FLUID Data Products
    • Universal Pipeline
    • The 11-Stage Pipeline
    • End-to-End Walkthrough: Catalog → Contract → Transformation
  • CLI Reference

    • CLI Reference
    • fluid init
    • fluid demo
    • fluid forge
    • fluid skills
    • fluid status
    • fluid validate
    • fluid plan
    • fluid apply
    • fluid generate
    • fluid generate artifacts
    • fluid validate-artifacts
    • fluid verify-signature
    • fluid generate-airflow
    • fluid generate-pipeline
    • fluid viz-graph
    • fluid odps
    • fluid odps-bitol
    • fluid odcs
    • fluid export
    • fluid export-opds
    • fluid publish
    • fluid datamesh-manager
    • fluid market
    • fluid import
    • fluid policy
    • fluid policy check
    • fluid policy compile
    • fluid policy apply
    • fluid contract-tests
    • fluid contract-validation
    • fluid diff
    • fluid test
    • fluid verify
    • fluid product-new
    • fluid product-add
    • fluid workspace
    • fluid ide
    • fluid ai
    • fluid memory
    • fluid mcp
    • fluid scaffold-ci
    • fluid scaffold-composer
    • fluid scaffold-ide
    • fluid docs
    • fluid config
    • fluid split
    • fluid bundle
    • fluid auth
    • fluid doctor
    • fluid providers
    • fluid provider-init
    • fluid roadmap
    • fluid version
    • fluid runs
    • fluid retention
    • fluid secrets
    • fluid stats
    • fluid contract
    • fluid ship
    • fluid rollback
    • fluid schedule-sync
    • Catalog adapters

      • Source Catalog Integration (V1.5)
      • BigQuery Catalog
      • Snowflake Horizon Catalog
      • Databricks Unity Catalog
      • Google Dataplex Catalog
      • AWS Glue Data Catalog
      • DataHub Catalog
      • Data Mesh Manager Catalog
    • CLI by task

      • CLI by task
      • Add quality rules
      • Add agent governance
      • Debug a failed pipeline run
      • Switch clouds with one line
  • Recipes

    • Recipes
    • Recipe — add a quality rule
    • Recipe — switch clouds with one line
    • Recipe — tag PII in your schema
  • SDK & Plugins

    • SDK & Plugins
    • Quickstart — your first plugin
    • Examples

      • Runnable examples
      • Example: hello-scaffold — the minimal viable plugin
      • Example: gitlab-ci-scaffold — generate a complete CI project
      • Example: steward-validator — a custom governance rule
      • Example: prod-key-guard — apply-time invariant check
    • Journeys

      • Journeys
      • Your own CI/CD

        • You have your own CI/CD setup, no problem
        • GitLab CI — the bundle template
        • GitHub Actions — the bundle template
        • Jenkins — the bundle template
        • CircleCI — the bundle template
      • You have a strict project layout, no problem
      • You have governance rules, no problem
      • You want a check at apply time, no problem
    • Reference

      • Reference
      • Roles reference
      • Entry points reference
      • Trust model
      • Packaging
      • Companion packages
  • Providers

    • Providers
    • Provider Architecture
    • GCP Provider
    • AWS Provider
    • Snowflake Provider
    • Local Provider
    • Creating Custom Providers
    • Provider Roadmap
  • Advanced

    • Blueprints
    • Governance & Compliance
    • Airflow Integration
    • Built-in And Custom Forge Guidance
    • FLUID Forge Contract GPT Packet
    • Forge Discovery Guide
    • Forge Memory Guide
    • LLM Providers
    • Capability Warnings
    • LiteLLM Backend (opt-in)
    • MCP Server
    • Credential Resolver — Security Model
    • Cost Tracking
    • Agentic Primitives
    • Typed Errors
    • Typed CLI Errors
    • Authoring Forge Tools
    • Source-Aligned Acquisition
    • API Stability — fluid_build.api
    • Guided fluid forge UX
    • V1.5 Catalog Integration — Architecture Deep-Dive
    • V1.5 + V2 Hardening — Release Notes
  • Project

    • Contributing to Fluid Forge
    • Fluid Forge Docs Baseline: CLI 0.8.3
    • Fluid Forge Docs Baseline: CLI 0.8.0
    • Fluid Forge Docs Baseline: CLI 0.7.11
    • Fluid Forge Docs Baseline: CLI 0.7.9
    • Fluid Forge v0.7.1 - Multi-Provider Export Release

🎬 CLI demos

14 frame-perfect SVG casts — install through deploy, local through Snowflake, the AI copilot, agentPolicy enforcement, day-2 incident response, and agent-loop compaction. Each one carries a takeaway popup with the punchline numbers. Click play — the SVG only animates after you opt in (no autoplay, no JS).

Convinced? → Install in 30 seconds. Want longer-form proof of specific workflows? → See it run (5 narrative scenarios, ~50 s each, with takeaway numbers).


Install + run, locally

Start here. No cloud account, no credit card, ~30 seconds end-to-end.

fluid init my-project --quickstart → validate → plan → apply
Install the CLI, scaffold the Customer 360 Analytics contract from the quickstart template, validate it against the 0.7.2 schema, preview the plan, and apply it against the local DuckDB provider. Two exposes — a master table and a high-value-customer view — land as Parquet under output/.

Same contract, different cloud

Swap binding.platform and re-deploy. The contract, schema, dq.rules, and the multi-stage build all stay byte-identical — only the cloud-specific binding fields change.

GCP / BigQuery

GCP quickstart — install, swap one line, deploy
From the local Customer 360 contract to a fully-deployed BigQuery dataset, in seconds. The `git diff` shows only the binding block changing — schema, dq.rules, and the 5-stage build all stay byte-identical.

AWS / Athena

AWS quickstart — S3 + Glue + Athena
Same Customer 360 contract, AWS provider extra installed, binding swapped to the AWS platform. S3 bucket provisioned, Glue catalog auto-created, Athena made queryable — all from fluid apply.

Snowflake

Snowflake quickstart — dry-run flow
The dry version: env-file credentials, contract validation, plan preview, and apply --mode dry-run rendering DDL without firing it. For the live-auth version see snowflake-real below.

AI copilot — full Gemini-powered flow

The fluid forge AI copilot generating a finance-domain contract end-to-end: project memory loaded, finance domain expertise pack applied (SOX + GDPR), local context discovered, a Gemini streaming call, and the contract emerging block-by-block with the agentPolicy gate. The hand-scripted version below mirrors the real-API flow at frame-perfect fidelity; a real-capture script (scripts/demos/forge_gemini_real_capture.py) is preserved for users who want to record an actual session.

fluid forge --domain finance --llm-provider gemini --llm-model gemini-2.5-flash
The full agent flow: memory → domain pack → discovery → streaming → contract emit (schema, dq.rules, accessPolicy, agentPolicy, sovereignty) → auto-validation → memory persist.

Snowflake live-auth dry-run

The snowflake-biz-lab flow at full fidelity: env credentials sourced, real validate --strict, plan against the live account, apply --mode dry-run rendering DDL without firing it, then policy-apply --mode check over the compiled IAM bindings.

Snowflake — validate → plan → apply --mode dry-run → policy-apply --mode check
Live auth (account=acme-demo placeholder; the scrubber substitutes the real account name). No DDL fires, no RBAC mutates — just the auth + connectivity + dry-render flow you'd run before a real production deploy.

AI copilot — interview shape only

fluid forge --blank skips the LLM call entirely and just scaffolds the structured stub for the chosen domain. Useful when you know what you want and don't need an LLM round-trip.

fluid forge --blank --domain finance
The blank skeleton with finance-domain defaults pre-seeded: SOX/GDPR regulatory framework, 'training' and 'fine_tuning' denied use cases, Gold layer assignment. Fill in the expose blocks yourself — no LLM call.

Policy + IAM compilation

The policy-check → generate artifacts → policy-apply --mode check triple. Validates the access policy, compiles to native cloud IAM (BigQuery/Snowflake/AWS), and runs the bindings in check-only mode (no live IAM mutations).

policy-check → generate artifacts → policy-apply --mode check
Three commands, full policy round-trip from declarative `accessPolicy.grants` in YAML to native cloud IAM JSON, then a dry-run that shows exactly which bindings would apply against the deployed state.

agentPolicy enforcement (LLM / AI governance)

Declare agentPolicy in YAML, validate it, see the enforcement summary, watch a replay of agent reads (allow/deny) against the policy.

agentPolicy — declare, validate, gate (validate → policy-check → audit)
The YAML block (allowedModels, deniedUseCases, canStore, auditRequired) → validate → policy-check enforcement summary → 4 replayed agent reads (gpt-4 allowed, claude-3 + training denied, unlisted model denied, gemini summarization allowed).

Long-form scenario casts

The 5 casts below pair with the See it run page — each tells a story (problem → CLI flow → punchline) at ~30-50 seconds with takeaway numbers.

$0.03 per data product — three providers, one contract

fluid forge data-model — same intent, three providers, real cost figures
Eight lines of intent YAML. Anthropic, OpenAI, Ollama — same flag swapped each time. All three emit the same valid contract, the same dbt project layout. Real production token counts and costs.

Six months → sixty seconds — source-aligned Bronze

fluid init --discover postgres://... — Bronze contract in 60 seconds
Connect, scan information_schema (47 ms), infer 28 tables / 143 columns / 12 PII candidates / 8 foreign keys, emit a complete Bronze contract.fluid.yaml, validate, apply against embedded DuckDB. 6.2 seconds total. Then show the engine swap path.

23 questions, skipped — guided UX

fluid forge — guided UX in action
47 ms welcome scan finds 3 CSVs + 2 dbt models + 1 README, infers domain (finance) and PII (5 columns). 5-mode picker. 4 questions answered (most accept the inferred default with ↵). Cost-cap progress in real time ($0.000 → $0.021 of $0.050 cap). Pre-write panel shows exactly what will + won't change.

3am Slack ping → ship in 90 seconds

3am Slack ping → ship in 90 seconds
PagerDuty: freshness SLA breached. fluid runs status shows 3 consecutive failures. fluid runs logs --component dlq surfaces the root cause. fluid runs diff shows what changed since the last OK run. One-line contract fix. fluid ship. 87 seconds end-to-end. Recovered 12,361 rows.

$0.50 → $0.05 — agent-loop compaction

Agent-loop compaction — three strategies, real before/after costs
20-turn baseline: $0.503/run, super-linear context bloat (5K → 67K → 298K tokens). Three strategies side-by-side: truncate (5.8× cheaper), summarize (9.3× cheaper), hybrid (10.5× cheaper). One env var: FLUID_COMPACTION_STRATEGY=hybrid.

How the casts are produced

The pipeline that built each SVG above:

   scripts/demos/<name>.py            ← cast generator
                ↓
        /tmp/casts/<name>.cast.raw    ← raw asciinema cast (gitignored)
                ↓
        scripts/cast-v3-to-v2.py      ← format conversion (asciinema 3.x → 2.x)
                ↓
        scripts/scrub-cast.py         ← strip API keys, JWTs, env-shaped secrets
                ↓
        svg-term --in <cast>          ← render to animated SVG (no --window;
                                          our <CliCast> component supplies
                                          the terminal chrome)
                ↓
   docs/.vuepress/public/demos/<name>.svg   ← the only file that gets committed

Two passes of secret-scanning happen:

  1. scrub-cast.py redacts known formats (AIza…, sk-…, sk-ant-…, JWTs, KEY=…/SECRET=…/PASSWORD=… 16+ char values) and substitutes literal $SNOWFLAKE_ACCOUNT/$SNOWFLAKE_USER/$GEMINI_API_KEY env values for friendly placeholders.
  2. Final-SVG grep in generate-demos.sh re-scans the output before keeping the file. If any leak pattern matches in the post-scrub SVG, the file is deleted and the build fails.

The .cast.raw working files live in /tmp/casts/ (gitignored) and are deleted at the end of each render.

To regenerate everything:

scripts/generate-demos.sh                 # regenerate every cast
scripts/generate-demos.sh --safe-only     # only the credential-free casts
scripts/generate-demos.sh forge-gemini    # one specific cast

Source for each cast lives at scripts/demos/<name>.py — review or fork freely.

Edit this page on GitHub
Last Updated: 5/17/26, 6:51 PM
Contributors: fas89, Claude Opus 4.7 (1M context)