Skip to content

Building from source

Build SourceOrrery from source: the tools to install on Windows, macOS and Linux, cloning, the CLI and the C ABI, the SDK test vault, each SDK and the conformance harness.

10 min read

Everything SourceOrrery ships, the CLI, the C ABI and the ten SDKs, builds from one repository. Build it yourself to try a change that is not released, to run on a platform the release does not cover, or to check that what you run is what the source says. This page covers what to install, how to get the code, and how to build and test each part; every SDK page ends with the steps for its own language.

For project setup with an installed CLI, a rebuild is unnecessary:

  1. Option 1: Set up with the CLI. Follow the first-run guide.
  2. Option 2: Set up with an AI coding agent. Run sourceorrery init --integration all, then ask your agent to configure the project. Current skills can be copied into a project independently of CLI packages.

Every build compiles Rust, so every build needs:

Tool Version Why
Git any recent to clone the repository
Python 3.11 or newer to run the version-stamping script
Rust, installed with rustup 1.91.1 or newer the core. rust-toolchain.toml selects the stable channel with rustfmt and clippy; rustup installs it on the first cargo command in the checkout
A C compiler the bundled SQLite is compiled from C: the Visual Studio Build Tools on Windows, the Xcode Command Line Tools on macOS, gcc or clang on Linux
pkg-config and the D-Bus headers Linux only the Secret Service (the OS credential store); not needed for a build with --no-default-features
Bash the script that creates the test vault: Git Bash on Windows
Terminal window
winget install --id Git.Git -e # Git and Git Bash
winget install --id Rustlang.Rustup -e
winget install --id Microsoft.VisualStudio.2022.BuildTools -e --override "--wait --passive --add Microsoft.VisualStudio.Workload.VCTools --includeRecommended"

The last line installs the MSVC compiler and the Windows SDK (the Desktop development with C++ workload), which Rust’s default Windows toolchain links with. Open a new terminal afterwards so that cargo is on PATH.

Each SDK needs its own toolchain on top:

SDK Also needs
Python Python 3.11 or newer, with pip and venv
JavaScript / TypeScript Node.js 20 or newer, with npm
Rust nothing more
Java a JDK, 22 or newer (CI uses Temurin 25), and Maven
.NET the .NET 8 SDK or newer for the library; the .NET 10 SDK for its tests and example, which target net10.0
Go Go 1.24 or newer and a C compiler for cgo: gcc or clang, or MinGW-w64 gcc on Windows
C the C compiler above
C++ a C++17 compiler; CMake 3.16 or newer for the example and the test
PHP PHP 8.2 or newer with the ffi extension, and Composer
Conformance harness Python 3, whichever SDK it drives

Check out the v0.3.1 tag to build the published release source. Use main when contributing changes toward the next release.

Terminal window
git clone https://github.com/Netizine/sourceorrery
cd sourceorrery
git checkout v0.3.1

A build from source carries the version committed at its commit: 0.3.1 at the v0.3.1 tag, and on main the version these pages describe. The release workflow stamps the version it publishes itself; to stamp one into a local build, use Python 3.11 or newer:

Terminal window
python scripts/release/set-version.py 0.3.1 --keep-node-links

The script updates package manifests and lockfiles. --keep-node-links preserves the Node package’s local platform-package links, so npm ci continues to work from this checkout. The release workflow omits that option when preparing packages for publication.

Terminal window
cargo build --release -p sourceorrery-cli -p sourceorrery-ffi

The first build downloads and compiles every dependency and takes a few minutes. What lands in target/release/:

File What it is
sourceorrery (sourceorrery.exe on Windows) the CLI
libsourceorrery_ffi.so (Linux), libsourceorrery_ffi.dylib (macOS), sourceorrery_ffi.dll with its import library sourceorrery_ffi.dll.lib (Windows) the C ABI as a shared library
libsourceorrery_ffi.a (Linux, macOS), sourceorrery_ffi.lib (Windows) the C ABI as a static library

The headers are crates/sourceorrery-ffi/include/sourceorrery.h and, for C++, bindings/cpp/include/sourceorrery.hpp.

On Windows, .cargo/config.toml links the C runtime statically (-C target-feature=+crt-static), so the CLI and the DLL run without the Visual C++ Redistributable, and the static sourceorrery_ffi.lib is for programs compiled with /MT. A RUSTFLAGS variable replaces those flags; python scripts/release/check-pe-imports.py target/release/sourceorrery.exe target/release/sourceorrery_ffi.dll says whether a build still imports a C runtime DLL.

To put the CLI on your PATH (in ~/.cargo/bin):

Terminal window
cargo install --locked --path crates/sourceorrery-cli
Crate Default features Without them
sourceorrery-cli os-keyring, ui, remote, otlp --no-default-features: vaults open with their recovery keys only, no screens (ui, setup’s and init’s), no providers and no OTLP export; add --features ui to keep the screens
sourceorrery-ffi, sourceorrery os-keyring, remote --no-default-features: vaults open with their recovery keys only, and no providers

A build without os-keyring needs no D-Bus headers on Linux and suits servers and containers, which have no credential store (Headless).

The supported combinations are the defaults, the headless build of each crate, and each feature on its own on top of it (provider-vault, provider-aws, provider-azure, remote, ui, otlp, os-keyring); the Python and Node extensions build with and without their defaults. scripts/ci/feature-matrix.sh builds every one of them and runs the headless crates’ tests; CI’s msrv job runs it, and the workspace’s tests, on the minimum Rust, 1.91.1, which every manifest declares as rust-version. To do the same locally:

Terminal window
rustup toolchain install 1.91.1 --profile minimal
RUSTUP_TOOLCHAIN=1.91.1 cargo test --workspace --locked
RUSTUP_TOOLCHAIN=1.91.1 bash scripts/ci/feature-matrix.sh

The Rust workspace, with the same checks CI runs:

Terminal window
cargo fmt --all -- --check
cargo clippy --workspace --all-targets --locked -- -D warnings
cargo test --workspace --locked

The tests open vaults in temporary directories with their recovery keys (or an in-memory credential store); they never touch your credential store or your own vaults.

scripts/dev/keyring-smoke.sh (macOS, Linux) and scripts/dev/keyring-smoke.ps1 (Windows) exercise this machine’s key in its credential store, in an isolated home: create a vault, read through the store, open it elsewhere with the recovery key, destroy with and without --keep-os-key, with the store inspected after each step. They print PASS or FAIL per check and never a value; exit 2 means the store is not available in this session (no desktop, no session bus). CI runs them headlessly on all three platforms (the keyring job); run one by hand in a desktop session to validate the store you use day to day:

Terminal window
cargo build -p sourceorrery-cli
bash scripts/dev/keyring-smoke.sh # macOS Keychain, GNOME Keyring, KWallet, KeePassXC
pwsh scripts/dev/keyring-smoke.ps1 # Windows Credential Manager

On Linux the store checks need secret-tool (libsecret-tools); without it the functional checks still run.

Every SDK’s tests, its example and the conformance harness read a vault that the CLI creates, so build the CLI and the C ABI first (above), then, in Bash (Git Bash on Windows):

Terminal window
eval "$(bash scripts/ci/fixture-vault.sh)"

scripts/ci/fixture-vault.sh creates a throwaway home directory with a vault named ci that opens with its recovery key, holding a secret in every state the tests check: two versions of database-password, a disabled newest version, an expired, a not-yet-valid and a deleted secret. It prints export lines, which eval applies to the current shell only:

Variable Value
SOURCEORRERY_HOME the throwaway home directory
SOURCEORRERY_RECOVERY_KEY the vault’s recovery key
SOURCEORRERY_TEST_VAULT ci
SOURCEORRERY_TEST_MANIFEST a sourceorrery.toml that names the ci vault
SOURCEORRERY_TEST_OLD_VERSION the oldest version of database-password
SOURCEORRERY_NATIVE_DIR target/release, where the C ABI library is
SOURCEORRERY_CLI the CLI that created the vault
SOURCEORRERY_TEST_REMOTE_MANIFEST given one provider’s vault create flags after the home, a manifest naming secrets kept at that provider through the vault’s connection (the provider jobs)

Run each SDK’s tests from that shell. Never put real secrets in this vault: its recovery key is in that shell’s environment and the job’s log.

From the repository root, in the shell where you ran the fixture script:

SDK Build and test
Rust cargo test -p sourceorrery
Python pip install ./bindings/python pytest, then python -m pytest bindings/python/tests (in a virtual environment)
JavaScript cd bindings/node && npm ci && npm run build && npm test && npm run typecheck
Java cd bindings/java && mvn test
.NET dotnet test bindings/dotnet/SourceOrrery.sln
Go cd bindings/go && CGO_LDFLAGS="-L$SOURCEORRERY_NATIVE_DIR" LD_LIBRARY_PATH="$SOURCEORRERY_NATIVE_DIR" DYLD_LIBRARY_PATH="$SOURCEORRERY_NATIVE_DIR" go test ./...
C++ cmake -S bindings/cpp -B build/cpp -DSOURCEORRERY_NATIVE_DIR="$SOURCEORRERY_NATIVE_DIR" && cmake --build build/cpp --config Release, then ctest --test-dir build/cpp -C Release with the library on the loader path
C compile crates/sourceorrery-ffi/examples/get_secret.c against the header and the library and run it
PHP php bindings/php/tests/run.php

Each SDK page has the full sequence, with the example and the platform details. On Windows, the C and C++ steps use MSVC’s cl, which is on PATH only in a Developer Command Prompt for VS: start Git Bash from there ("C:\Program Files\Git\bin\bash.exe") so that the compiler and the fixture’s variables are in one shell.

One list of scenarios, tests/conformance/sdk/scenarios.json, holds every SDK to the same behaviour. harness.py runs a small runner program per SDK through all of them and judges what it reports:

Terminal window
python tests/conformance/sdk/harness.py --runner python

The interpreted runners (python, node, java, php) need only the SDK built as above. The compiled ones are built first:

Runner Build it with
rust cargo build --release -p sourceorrery --example conformance
dotnet dotnet build bindings/dotnet/conformance -c Release
go go build -o conformance-go ./conformance in bindings/go (conformance-go.exe on Windows)
c, cpp compile crates/sourceorrery-ffi/examples/conformance.c and bindings/cpp/conformance/run.cpp into build/conformance_c and build/conformance_cpp

The protocol and how to add a scenario are in tests/conformance/sdk/README.md (opens in a new tab).

The CI images are Debian-based. To check a change on Linux without a Linux machine, run the tests in the official Rust image with Docker:

Terminal window
docker run --rm -v "$PWD:/work" -w /work rust:1-bookworm \
bash -c 'apt-get update && apt-get install -y libdbus-1-dev pkg-config && cargo test --workspace --locked'

In Git Bash on Windows, put MSYS_NO_PATHCONV=1 in front of docker so that /work is not rewritten as a Windows path. The build writes to target/ in your checkout; add -e CARGO_TARGET_DIR=/tmp/target to keep the Linux build apart from your own.

site/ is the Astro Starlight site at sourceorrery.ai; docs/ is its source. With Node.js 22.12 or newer:

Terminal window
cd site
npm ci
npm run dev # http://localhost:4321
npm run build # build, then check every internal link

Details: site/README.md (opens in a new tab).