Build & Setup¶

VS Code setup¶

Open bxp.code-workspace rather than the plain folder: it carries the recommended-extension list, and VS Code offers to install the whole set on first open (Extensions → Recommended, or Show Recommended Extensions from the command palette).

Extension	Purpose
`ziglang.vscode-zig`	Zig language support: syntax, build tasks, and the bundled ZLS (completions, go-to-definition, inline diagnostics)
`dart-code.dart-code`	Dart language support: analyzer diagnostics, formatting, debugging
`dart-code.flutter`	Flutter tooling: device selection, hot reload on save, the widget inspector, and F5 launch with a full debugger
`blueglassblock.better-json5`	Syntax highlighting for the JSON5 config files bxp reads
`zainchen.json`	JSON formatting and tree view — plain-JSON payloads such as MCP wire dumps and .mcp.json
`mechatroner.rainbow-csv`	Column-aware CSV viewer — helpful when reading source exports
`bierner.markdown-mermaid`	Renders Mermaid diagrams in the Markdown preview, which is how the architecture pages are written
`bpruitt-goddard.mermaid-markdown-syntax-highlighting`	Syntax highlighting inside those Mermaid fences

That table is not retyped here — it is generated from the same bxp.code-workspace list the editor reads, each row's purpose being the comment above the entry. Adding an extension for yourself and adding it to this page are one action.

ZLS (Zig Language Server) comes with ziglang.vscode-zig — the extension fetches a ZLS matching your Zig toolchain, giving completions, go-to-definition and inline diagnostics with nothing else to install.

The Dart and Flutter extensions matter only if you touch bxp-gui; the desktop side has its own prerequisites and first-run walkthrough in GUI setup.

The workspace opens a second folder

bxp.code-workspace also lists ../DEV, a scratch folder that is not part of this repository. VS Code shows it greyed out if you do not have one — harmless, and removable from File → Remove Folder from Workspace.

Verify Zig language version¶

The required Zig version is pinned in build.zig.zon (minimum_zig_version), and CI installs exactly that version — .github/workflows/ci.yml is the second place to look if you want to see the pin in use. Install that toolchain from ziglang.org/download, or with a version manager such as zigup if you keep several around; the ZLS the Zig extension fetches follows whichever zig is on your PATH.

bxp-core has two external (fetch) dependencies — uucode (MIT), the Unicode case-mapping tables behind UPPER/LOWER, and zig_libs (MIT), supplying the whole primitive layer: the datefmt date core, the tz IANA offset lookup, the encoding code-page transcoder, the json5 preprocessor, the decimal fixed-point numeric core, the numparse grouped- number parser, the zipstream ZIP reader, the csvstream CSV reader, the diagnostics collector and the regex engine behind REGEX_MATCH/REGEX_EXTRACT (linear-time, ReDoS-safe), plus minisign and procrun (re-exported for bxp-gui-bridge) and mcp (the JSON-RPC transport re-exported for bxp-mcp). All are pinned in bxp-core/build.zig.zon. The fetches are cached after the first build; CI runners have network.

In VS Code terminal:

zig version   # must satisfy build.zig.zon's minimum_zig_version

Claude Code setup¶

BXP development in Zig works with Claude Code, locally. What the repository commits for it:

CLAUDE.md files — the root one plus one per package (bxp-cli/, bxp-core/, bxp-mcp/, bxp-gui/, bxp-gui-bridge/, bxp-gui/packages/json5_ast/) and docs/examples/. Claude loads them automatically; the root file's Project rules section is the one to read first.
.claude/rules/zig-pitfalls.md — Zig lessons specific to this repository (helpers to use, false-green checks, no zig fmt).
.claude/skills/zig-0.17/ — the zig-0.17 API-reference skill for the Zig version in .zigversion, vendored from an audited release of zaxified/zig-skills. Claude loads it before writing or reviewing Zig code. Never edit these files: the zig-skill workflow fails on any difference from the release named in .claude/skills/zig-0.17/.zig-skills-source.json. To move to a newer release, run that repository's scripts/install.py from the new tag and commit the result.

Everything else under .claude/ (settings, local permissions) stays on your machine and is ignored by git.

Repository layout¶

Generated by walking the repository, with each entry's note read from the file's own header — a //! line in Zig, the comment under a script's shebang, the description: front matter of a Markdown page. Directories are described in the generator's own catalog. Nothing here is retyped, so a new file arrives on this page with the commit that adds it.

bxp/                                # monorepo root (git root)
├── bxp-cli/                        # user-facing CLI binary — the conversion engine
│   ├── src/                        # arg parsing, config dispatch, the processing pipeline
│   │   ├── cli_docs.zig            # Co-located documentation catalogs for bxp-cli's CLI surface: the flag table (the `--help` renderer's source) and the process exit codes
│   │   ├── main.zig                # Broker eXchange Parser — CLI entry point
│   │   └── pipeline.zig            # Processing pipeline for bxp-cli: per-template file conversion and xlsx pre-pass
│   ├── CLAUDE.md
│   ├── build.zig
│   └── build.zig.zon
├── bxp-core/                       # internal shared library (no binary of its own)
│   ├── src/                        # the domain layer: parsers, evaluator, config, trace, inspect
│   │   ├── btrace.zig              # Binary trace stream — compact metadata emitted by `bxp-cli --trace`
│   │   ├── config.zig              # Configuration types and JSON loader for bxp
│   │   ├── docs.zig                # Aggregates and serializes the documentation surface `inspect.docsJson` exposes to the GUI, and renders the Markdown reference tables
│   │   ├── expr.zig                # Expression evaluator used to compute input_schema variable values
│   │   ├── inspect.zig             # inspect — shared stateless inspection core for bxp-mcp and the GUI bridge
│   │   ├── json.zig                # Streaming JSON array-of-objects reader for bxp-cli
│   │   ├── module_docs.zig         # Architecture catalogs for bxp-core: the module inventory and the stateless `inspect` surface
│   │   ├── unicode.zig             # In-house Unicode text operations for expr.zig builtins
│   │   ├── wasm.zig                # wasm32 export wrapper around the stateless inspect core
│   │   └── xlsx.zig                # Converts Excel .xlsx workbooks to CSV
│   ├── CLAUDE.md
│   ├── build.zig
│   └── build.zig.zon
├── bxp-mcp/                        # MCP server (JSON-RPC 2.0 over stdio) for AI agents
│   ├── src/                        # entry point, tool catalog, bxp_simulate orchestration
│   │   ├── main.zig                # bxp-mcp — entry point
│   │   ├── sim.zig                 # bxp-mcp — bxp_simulate orchestration
│   │   └── tools.zig               # bxp-mcp — tool catalog + handlers (in-process, no spawn)
│   ├── CLAUDE.md
│   ├── build.zig
│   └── build.zig.zon
├── bxp-gui/                        # Flutter desktop app (Linux / macOS / Windows)
│   ├── fonts/                      # the bundled UI font, embedded rather than fetched at runtime
│   ├── installer/                  # NSIS script for the Windows installer
│   ├── lib/                        # Dart source: services (FFI + prefs + updater), store, ui
│   ├── linux/                      # Linux Flutter shell + CMake hooks
│   ├── macos/                      # macOS Flutter shell
│   ├── packages/                   # path-dep Dart packages — today just json5_ast
│   ├── test/                       # widget + unit tests, incl. the cross-runner expression corpus
│   ├── tool/                       # developer probes, e.g. the Windows bridge stream check
│   ├── web/                        # web Flutter shell (not shipped; keeps `flutter` happy)
│   ├── windows/                    # Windows Flutter shell
│   ├── CHANGELOG.md
│   ├── CLAUDE.md
│   ├── README.md
│   ├── analysis_options.yaml
│   ├── devtools_options.yaml
│   └── pubspec.yaml
├── bxp-gui-bridge/                 # Zig FFI shared library — the GUI's single backend
│   ├── src/                        # C-ABI exports and the marshalling around them
│   │   ├── main.zig                # bxp-gui-bridge — Dart FFI shim that proxies bxp-core inspect ops / bxp-cli calls
│   │   └── ops.zig                 # Catalog of the bridge's C-ABI surface
│   ├── test/                       # re-exec helper binary driving the subprocess tests
│   │   └── test_helper.zig         # bridge-test-helper — controllable child process for bridge unit tests
│   ├── CLAUDE.md
│   ├── build.zig
│   └── build.zig.zon
├── tools/                          # build-time documentation generators (never distributed)
│   ├── dart-doc-gen/               # renders the Dart GuiToolDoc pages (runs as a flutter test)
│   └── zig-doc-gen/                # renders every Zig catalog and @typeInfo page into docs/
├── datasets/                       # anonymized sample data + expected output, gated by test-07
├── docs/                           # the MkDocs site — this documentation
│   ├── ai/                         # agent-facing guides: authoring a broker, gui-mcp, handoff
│   ├── assets/                     # site assets: stylesheets, the playground JS, the wasm engine
│   ├── dev/                        # developer documentation: build, test, debug, architecture, release
│   ├── examples/                   # the runnable example tree plus its Examples section pages
│   ├── getting-started/            # install, first conversion, the shipped template library
│   ├── gui/                        # bxp-gui user guide: features, preferences, updates, troubleshooting
│   ├── guide/                      # user guide: templates, expressions, dates, routing, targets
│   ├── includes/                   # generated fragments pulled into hand-written pages
│   ├── reference/                  # generated reference pages — do not edit by hand
│   └── index.md                    # Convert tabular exports — CSV, XLSX or JSON — into the shape another tool expects, with declarative JSON5 templates and no code
├── resources/                      # files shipped inside the release archives
│   ├── console/                    # the sample config bundled with the console archives
│   ├── desktop/                    # the Linux launcher template bundled with the desktop archives
│   └── icons/                      # SVG variants + build-icons.sh, the single source for app icons
├── scripts/                        # test, release and documentation tooling
│   ├── bench/                      # developer-only benchmark matrix (not part of test.sh)
│   │   ├── results/                # recorded benchmark runs — the baseline a regression is measured against
│   │   ├── bench.sh                # BXP stress-test matrix runner
│   │   ├── gen-xlsx.py             # Synthetic .xlsx generator for profiling bxp-core/src/xlsx.zig ingest
│   │   ├── gen.py                  # Generate synthetic input.in.csv + bxp-cli.json for a single bench run
│   │   ├── verify-output.sh        # One-off output verification for the chunking refactor
│   │   ├── xlsx-bench.json
│   │   └── xlsx-fanout-bench.json
│   ├── docs/                       # documentation support: site generation, wasm playground, checks
│   │   ├── mermaid-check/          # the mermaid-fence parser behind check-formatting.sh
│   │   ├── check-formatting.sh     # Documentation mermaid check — PRE-RELEASE ONLY
│   │   ├── check-wasm-parity.py    # Compare the wasm and native expression evaluators over the shared corpus
│   │   ├── check-wasm-parity.sh    # wasm/native parity gate for the docs expression scratchpad
│   │   ├── gen-docs.sh             # Generate the MkDocs documentation site and (default) serve it locally
│   │   ├── gen-examples-index.py   # Generate the Examples index pages for the MkDocs site
│   │   ├── gen-trees.py            # Generate the repository tree, the test-phase table and the VS Code extension table
│   │   ├── gen-wasm-playground.sh  # Build the docs playground's wasm engine into docs/assets/wasm/
│   │   └── requirements.txt
│   ├── README.md
│   ├── release-01-console.sh       # Build console release archives for Linux, Windows, and macOS
│   ├── release-02-desktop.sh       # Build the bxp-desktop release bundle for the current host OS
│   ├── release-03-checksums.sh     # Generate (or verify) SHA256SUMS for the release artifacts in a dir tree
│   ├── release-changelog.sh        # Bump versions across all 6 manifests + generate a CHANGELOG.md entry from commits since the last release tag, in one step
│   ├── release-tag.sh              # Cut a release tag from the version already written into the manifests by `release-changelog.sh`, and push it
│   ├── release.sh                  # Run the local release-build flow
│   ├── test-01-console.sh          # Console-side build + unit tests (bxp-core, bxp-cli, json5_ast)
│   ├── test-02-mcp.sh              # bxp-mcp build + unit tests + JSON-RPC smoke
│   ├── test-03-bridge.sh           # bxp-gui-bridge unit tests
│   ├── test-04-desktop.sh          # Desktop-side tests: the Flutter app and the embedded json5_ast package
│   ├── test-05-bench-guard.sh      # Performance regression guard
│   ├── test-06-expr-corpus.sh      # Expression corpus regression gate
│   ├── test-06-expr-corpus.txt
│   ├── test-07-datasets.sh         # Dataset regression: run bxp-cli against every datasets/<template>/sample.json and diff every generated *.csvx against the matching *.expected fixture
│   ├── test-08-docs-examples.py    # Checker behind scripts/test-08-docs-examples.sh — see that script's header
│   ├── test-08-docs-examples.sh    # Docs example-page gate: the clickable expressions must actually work
│   ├── test-09-examples.sh         # Example regression: run bxp-cli against every docs/examples/<tier>/<name>/ sample.json and diff each produced output against its committed *.expected golden
│   ├── test-lib.sh                 # Sourced; not executable on its own
│   └── test.sh                     # Run the full test suite
├── .github/                        # CI configuration
│   ├── ISSUE_TEMPLATE/             # GitHub issue forms
│   ├── workflows/                  # the CI, docs-publish and release pipelines
│   └── PULL_REQUEST_TEMPLATE.md
├── README.md
├── CHANGELOG.md
├── CONTRIBUTING.md
├── SECURITY.md
├── CODE_OF_CONDUCT.md
├── LICENSE.md
├── CLAUDE.md
└── mkdocs.yml

Clone and build¶

# Clone this repository
git clone https://github.com/zaxified/bxp.git

# Build bxp-cli (fetches dependencies on first run)
cd ./bxp/bxp-cli
zig build

# Run
./zig-out/bin/bxp-cli --help

Running bxp-cli without arguments processes every template defined in bxp-cli.json in the current working directory. \ The typical dev workflow:

# From the monorepo root
./bxp-cli/zig-out/bin/bxp-cli --config ./datasets/anycoin_to_wealthfolio/sample.json --debug

Build the documentation site¶

The site you are reading is MkDocs Material, driven by one script:

# Regenerate every generated page + fragment, build, and serve on :8000
bash scripts/docs/gen-docs.sh

bash scripts/docs/gen-docs.sh --build   # regenerate + build into site/, no serve
bash scripts/docs/gen-docs.sh --check   # drift guard: fail if a generated page
                                        # differs from a fresh generation

The Python toolchain lives in a local venv the script expects to find, and is pinned in scripts/docs/requirements.txt — the same file the Pages workflow installs from, so a page that builds on a laptop builds on the runner:

python3 -m venv .venv-docs
.venv-docs/bin/pip install -r scripts/docs/requirements.txt

The social plugin renders the Open Graph preview cards through Cairo, so cairosvg and pillow in that file are not optional extras — and on Linux they need system libraries the venv cannot supply:

sudo apt-get install -y libcairo2-dev libfreetype6-dev libffi-dev \
    libjpeg-dev libpng-dev

That is the same list .github/workflows/docs.yml installs. Without them mkdocs build --strict fails inside the plugin rather than on your page.

Two things to know before editing:

Regenerating needs a working Zig and Flutter toolchain. The reference pages come out of tools/zig-doc-gen, and four GUI pages out of tools/dart-doc-gen, which runs as a flutter test. Editing hand-written prose does not need either — building the whole site does.
--check is a CI gate, not a test phase. .github/workflows/docs.yml runs the drift check, the wasm/native parity check and a --strict build; scripts/test.sh deliberately does not. See the note in Testing for why.

Markdown formatting is hand-maintained (prettier and markdownlint reflow MkDocs-specific syntax and break the rendered pages). bash scripts/docs/check-formatting.sh parses the Mermaid fences and is run before a release rather than on every commit.

The wasm playground target¶

bxp-core also builds for wasm32-freestanding, which is what powers the clickable expressions on the docs site — the reader's browser runs bxp's own evaluator rather than a JavaScript re-implementation. It is an opt-in target, never part of install:

cd bxp-core && zig build wasm -Dtarget=wasm32-freestanding -Doptimize=ReleaseSmall

In practice you do not run that by hand: scripts/docs/gen-wasm-playground.sh builds it into docs/assets/wasm/bxp-eval.wasm, and scripts/docs/gen-docs.sh calls that script for you. The .wasm is a build artifact and stays untracked — a checkout regenerates it, and a docs build without it leaves the panel reporting that it could not load the engine.

Agreement with the native engine is measured rather than assumed: scripts/docs/check-wasm-parity.sh runs the cross-runner expression corpus through both and requires byte-identical results. It needs a JS runtime, so it is a docs-workflow step rather than a test-NN phase.

Run the test suite¶

# From the monorepo root — runs unit tests + all regression tests
bash scripts/test.sh

Nine phases covering the bxp-core unit tests and the CLI build, the MCP server, the bridge, the Flutter desktop app, a coarse perf guard, the expression corpus, and three regression gates — datasets, the expressions printed on the example pages, and the examples' committed output. Testing has the full breakdown (generated from the phase scripts' own headers, so it counts itself), the individual sub-suite commands, and how to add a regression.