Fluid Forge
Get Started
See it run
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
Demos
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
GitHub
Get Started
See it run
  • Local (DuckDB)
  • Source-Aligned (Postgres → DuckDB)
  • AI Forge + Data Models
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
  • 11-Stage Production Pipeline
  • Catalog Forge End-to-End
CLI Reference
  • Overview
  • Quickstart
  • Examples
  • Your own CI
  • Your own scaffolding
  • Custom validator
  • Apply hook
  • Reference
Demos
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
GitHub
  • Introduction

    • Home
    • Getting Started
    • Snowflake Quickstart
    • See it run
    • Forge Data Model
    • Vision & Roadmap
    • Playground
    • FAQ
  • Concepts

    • Concepts
    • Builds, Exposes, Bindings
    • What is a contract?
    • Quality, SLAs & Lineage
    • Governance & Policy
    • Agent Policy (LLM/AI governance)
    • Providers vs Platforms
    • Fluid Forge vs alternatives
  • Data Products

    • Product Types — SDP, ADP, CDP
  • Walkthroughs

    • Walkthrough: Local Development
    • Source-Aligned: Postgres → DuckDB → Parquet
    • AI Forge And Data-Model Journeys
    • Walkthrough: Deploy to Google Cloud Platform
    • Walkthrough: Snowflake Team Collaboration
    • Declarative Airflow DAG Generation - The FLUID Way
    • Generating Orchestration Code from Contracts
    • Jenkins CI/CD for FLUID Data Products
    • Universal Pipeline
    • The 11-Stage Pipeline
    • End-to-End Walkthrough: Catalog → Contract → Transformation
  • CLI Reference

    • CLI Reference
    • fluid init
    • fluid demo
    • fluid forge
    • fluid skills
    • fluid status
    • fluid validate
    • fluid plan
    • fluid apply
    • fluid generate
    • fluid generate artifacts
    • fluid validate-artifacts
    • fluid verify-signature
    • fluid generate-airflow
    • fluid generate-pipeline
    • fluid viz-graph
    • fluid odps
    • fluid odps-bitol
    • fluid odcs
    • fluid export
    • fluid export-opds
    • fluid publish
    • fluid datamesh-manager
    • fluid market
    • fluid import
    • fluid policy
    • fluid policy check
    • fluid policy compile
    • fluid policy apply
    • fluid contract-tests
    • fluid contract-validation
    • fluid diff
    • fluid test
    • fluid verify
    • fluid product-new
    • fluid product-add
    • fluid workspace
    • fluid ide
    • fluid ai
    • fluid memory
    • fluid mcp
    • fluid scaffold-ci
    • fluid scaffold-composer
    • fluid scaffold-ide
    • fluid docs
    • fluid config
    • fluid split
    • fluid bundle
    • fluid auth
    • fluid doctor
    • fluid providers
    • fluid provider-init
    • fluid roadmap
    • fluid version
    • fluid runs
    • fluid retention
    • fluid secrets
    • fluid stats
    • fluid contract
    • fluid ship
    • fluid rollback
    • fluid schedule-sync
    • Catalog adapters

      • Source Catalog Integration (V1.5)
      • BigQuery Catalog
      • Snowflake Horizon Catalog
      • Databricks Unity Catalog
      • Google Dataplex Catalog
      • AWS Glue Data Catalog
      • DataHub Catalog
      • Data Mesh Manager Catalog
    • CLI by task

      • CLI by task
      • Add quality rules
      • Add agent governance
      • Debug a failed pipeline run
      • Switch clouds with one line
  • Recipes

    • Recipes
    • Recipe — add a quality rule
    • Recipe — switch clouds with one line
    • Recipe — tag PII in your schema
  • SDK & Plugins

    • SDK & Plugins
    • Quickstart — your first plugin
    • Examples

      • Runnable examples
      • Example: hello-scaffold — the minimal viable plugin
      • Example: gitlab-ci-scaffold — generate a complete CI project
      • Example: steward-validator — a custom governance rule
      • Example: prod-key-guard — apply-time invariant check
    • Journeys

      • Journeys
      • Your own CI/CD

        • You have your own CI/CD setup, no problem
        • GitLab CI — the bundle template
        • GitHub Actions — the bundle template
        • Jenkins — the bundle template
        • CircleCI — the bundle template
      • You have a strict project layout, no problem
      • You have governance rules, no problem
      • You want a check at apply time, no problem
    • Reference

      • Reference
      • Roles reference
      • Entry points reference
      • Trust model
      • Packaging
      • Companion packages
  • Providers

    • Providers
    • Provider Architecture
    • GCP Provider
    • AWS Provider
    • Snowflake Provider
    • Local Provider
    • Creating Custom Providers
    • Provider Roadmap
  • Advanced

    • Blueprints
    • Governance & Compliance
    • Airflow Integration
    • Built-in And Custom Forge Guidance
    • FLUID Forge Contract GPT Packet
    • Forge Discovery Guide
    • Forge Memory Guide
    • LLM Providers
    • Capability Warnings
    • LiteLLM Backend (opt-in)
    • MCP Server
    • Credential Resolver — Security Model
    • Cost Tracking
    • Agentic Primitives
    • Typed Errors
    • Typed CLI Errors
    • Authoring Forge Tools
    • Source-Aligned Acquisition
    • API Stability — fluid_build.api
    • Guided fluid forge UX
    • V1.5 Catalog Integration — Architecture Deep-Dive
    • V1.5 + V2 Hardening — Release Notes
  • Project

    • Contributing to Fluid Forge
    • Fluid Forge Docs Baseline: CLI 0.8.3
    • Fluid Forge Docs Baseline: CLI 0.8.0
    • Fluid Forge Docs Baseline: CLI 0.7.11
    • Fluid Forge Docs Baseline: CLI 0.7.9
    • Fluid Forge v0.7.1 - Multi-Provider Export Release

SDK & Plugins

data-product-forge already does the boring parts — validate, plan, apply, publish. This section is for everything else: when your team needs the CLI to do something specific to your organization.

Your CI templates aren't the ones forge ships? Plug your own in. Your team's project layout is opinionated? Generate it from a contract. You have governance rules nobody else has? Run them at validate time. You want a safety check right before deploy? Add an apply hook.

You don't fork the CLI. You write a small Python package, register one entry-point, pip install it, and forge picks it up.

Quick: Quickstart · Examples · Your own CI · Your own scaffolding · Custom validator · Apply hook · Roles · Entry points · Trust model · Packaging · Companion packages

What you'll be building

A plugin is a normal Python package — pyproject.toml, src/, tests/ — with one extra line telling pip "here's a thing forge can discover":

[project.entry-points."fluid_build.custom_scaffolds"]
my-scaffold = "my_pkg.scaffold:MyScaffold"

After pip install, the CLI sees your plugin automatically. No registration step. No config file. No fork of data-product-forge.

End-to-end it looks like this:

your-package
└── src/my_pkg/scaffold.py   ─┐
                              │   subclass an SDK role:
                              ▼   CustomScaffold / Validator / InfraProvider / CatalogAdapter
   ┌──────────────────────────────┐
   │ class MyScaffold(...):       │
   │     def plan(self, contract):│
   │         return [...]          │
   └──────────────────────────────┘
                              │   pyproject.toml entry-point
                              ▼
         pip install your-package
                              │
                              ▼
   ┌──────────────────────────────┐
   │   data-product-forge (CLI)   │ ──▶ discovers your plugin
   │   fluid validate / apply /   │
   │   generate / publish         │
   └──────────────────────────────┘

That's the whole mental model. The rest of this page is "which role do I subclass" and "which entry-point group do I use" — both small decisions.

A complete, runnable example, top to bottom

This is everything. Paste it into a fresh directory and run.

# src/hello_scaffold/scaffold.py
from fluid_sdk import ContractHelper, CustomScaffold, write_file_action


class HelloScaffold(CustomScaffold):
    name = "hello"

    def plan(self, contract):
        c = ContractHelper(contract)
        return [
            write_file_action(
                path="README.md",
                content=f"# {c.name}\n\n{c.description}\n".encode("utf-8"),
            ).to_dict(),
        ]
# pyproject.toml
[build-system]
requires = ["setuptools>=68.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "hello-scaffold"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = ["data-product-forge-sdk>=0.9,<1"]

[project.entry-points."fluid_build.custom_scaffolds"]
hello = "hello_scaffold.scaffold:HelloScaffold"

[tool.setuptools.packages.find]
where = ["src"]
# Install everything you need
pip install data-product-forge data-product-forge-custom-scaffold
pip install -e .

# In any fluid project's contract.fluid.yaml:
#   extensions:
#     customScaffold:
#       libraries: [{id: hi, source: {kind: entrypoint, name: hello}}]
#       patterns: [{use: hi:main}]

fluid generate custom-scaffold
# ✓ 1 file written, 0 failed
#   README.md

That's it. ~15 lines of Python, two TOML stanzas, one CLI command. Read the quickstart for the same thing with each step explained, plus what to check when something doesn't work.

Which role do I subclass?

Four roles, one mental model. Each role is a thin subclass of BasePlugin that pins a role tag and provides role-shaped helpers. Pick the one that matches what you're producing:

You want to…RoleHook it intoExample
Generate files from a contract (CI configs, app code, IaC stacks, docs)CustomScaffoldfluid generate custom-scaffoldhello-scaffold, gitlab-ci-scaffold
Enforce governance / compliance / cost rules at author-timeValidatorfluid validatesteward-validator
Add support for a new cloud platform or data warehouseInfraProviderfluid apply—
Sync product metadata into your catalog (DataHub, Atlan, Collibra…)CatalogAdapterfluid publish—

Class signatures and inherited methods: Roles reference.

Pick the journey that matches your problem

Each guide opens with the real problem you might have, then walks you to a working plugin end-to-end:

You have…ReadWhat you build
Your own CI/CD templatesyour-own-ciA scaffold bundle that emits your CI files from contract data (GitLab / GitHub Actions / Jenkins variants)
A strict project layoutyour-own-scaffoldingA scaffold for the whole project skeleton (README, tests, Dockerfile, app source)
Governance rulescustom-validatorA Validator plugin (e.g. "every Gold product MUST declare a data steward")
A safety check that must run at applyapply-hookAn apply hook that aborts deploy when an invariant is violated (with override flag)

What's on PyPI today

PackageVersionWhat it doesInstall
data-product-forge0.8.3The CLI (this docs set)pip install data-product-forge
data-product-forge-sdk0.9.0Plugin SDK — zero-dependency ABCs + conformance harnesspip install data-product-forge-sdk (import: from fluid_sdk import …)
data-product-forge-custom-scaffold0.1.0Reference custom-scaffold engine (Jinja+YAML or Python plugins)pip install data-product-forge-custom-scaffold

The SDK and scaffold ship as version-pinned standalone packages. A first stable cut (1.0.0 / 0.2.0) is planned after a validation window — feel free to consume them today, just pin the upper bound. See Companion Packages for the dual-naming details (PyPI: data-product-forge-sdk, import path: fluid_sdk).

What about security?

Plugins are uncontained Python loaded into the CLI process. Trust in a plugin = trust in whatever pip resolved when you installed it. The CLI does not sandbox plugin code, time-limit it, or restrict what it can do.

What the CLI does defend against, automatically:

  • Crash containment — a plugin that raises an exception cannot crash the CLI.
  • Contract mutation — apply hooks get a copy.deepcopy() of the contract, not the live reference.
  • Credential leak in error messages — plugin exception text is scrubbed before reaching logs.

Full statement: Trust model. Read it before installing community plugins.

Reference

  • Roles — what each role gives you, what you override
  • Entry points — the entry-point groups (3 CLI hooks + 4 role-level) with signatures and failure model
  • Trust model — what we defend against, what we don't
  • Packaging — pyproject.toml template, py.typed, conformance harness, publishing checklist
  • Companion packages — what's on PyPI, dual-naming, version pinning

Source repos

  • SDK: Agenticstiger/forge-cli-sdk — also contains three runnable example plugins under examples/.
  • Custom-scaffold engine: Agenticstiger/data-product-forge-custom-scaffold — reference bundle in tests/fixtures/reference_bundle/.

The example walkthroughs on these pages mirror the upstream examples/ directories — they're the truth source.

Edit this page on GitHub
Last Updated: 5/13/26, 6:01 AM
Contributors: fas89
Next
Quickstart — your first plugin