# llvm-dsdl User Manual -- complete text Every page of the manual at https://opencyphal-garage.github.io/llvm-dsdl/, concatenated in navigation order. Generated by tools/docs/build_llms_index.py; the link index alone is at llms.txt. The showroom's generated type gallery is not included here. Those two dozen pages carry the full generated code for every type in eight language/profile variants and are four times the size of the manual itself, which would make this file expensive to fetch for the many readers who want the manual. Every one of them is listed with its URL in llms.txt. ================================================================================================== Source: docs/index.md URL: https://opencyphal-garage.github.io/llvm-dsdl/ ================================================================================================== # llvm-dsdl User Manual Compiler tooling for OpenCyphal DSDL, built on MLIR and LLVM. `llvm-dsdl` ships three user-facing tools: - `dsdlc` for compile/codegen workflows - `dsdl-opt` for dialect/pass pipeline work - `dsdld` for language-server workflows ## Contents | Section | Read it when | Starts at | |---|---|---| | [Get Started](start/install.md) | You have not run the tools yet. Two pages, read in order, ending with generated code on disk. | [Install](start/install.md) · [Quick Start](start/quickstart.md) | | [Reference](reference/index.md) | You know what you want and need the exact spelling, or the exact guarantee. | [Overview](reference/index.md) | | [Development](development/contributing.md) | You are changing this repository. | [Contributing](development/contributing.md) · [Testing and CI](development/testing.md) | | [For Agents](agents/index.md) | You are a coding agent or a documentation indexer. | [Agent Guide](agents/index.md) | ## Canonical Documents - Project README: [README.md](https://github.com/OpenCyphal-Garage/llvm-dsdl/blob/main/README.md) - Design details: [DESIGN.md](https://github.com/OpenCyphal-Garage/llvm-dsdl/blob/main/DESIGN.md) - Contributor workflow: [CONTRIBUTING.md](https://github.com/OpenCyphal-Garage/llvm-dsdl/blob/main/CONTRIBUTING.md) ================================================================================================== Source: docs/start/install.md URL: https://opencyphal-garage.github.io/llvm-dsdl/start/install/ ================================================================================================== # Install Two ways to get `dsdlc`, `dsdl-opt`, and `dsdld` onto a machine: download a release build, or build from source. Take the release binaries unless you intend to work on the compiler itself — they carry the LLVM runtime they need, so nothing has to be installed first. ## Option 1: Use release binaries Download binaries from: - Put `dsdlc`/`dsdl-opt`/`dsdld` on your `PATH`. ## Option 2: Build from source ```bash git clone https://github.com/OpenCyphal-Garage/llvm-dsdl.git cd llvm-dsdl git submodule update --init --recursive cmake --workflow --preset install-bin-release-ci ``` Installed binaries are typically under: - `build/matrix/ci/install/bin` ## Prerequisites - CMake 3.25+ - Ninja - LLVM + MLIR CMake packages (`LLVMConfig.cmake`, `MLIRConfig.cmake`) - C++20-capable toolchain See the full developer prerequisites in [Contributing](../development/contributing.md). ================================================================================================== Source: docs/start/quickstart.md URL: https://opencyphal-garage.github.io/llvm-dsdl/start/quickstart/ ================================================================================================== # Quick Start Compile a DSDL namespace into generated code. Each block below is a complete command; run them against a root namespace directory of your own, or against the `lanyard` namespace in the [Showroom](../showroom/index.md) if you do not have one yet. ## Show tool help ```bash dsdlc --help dsdlc --version ``` ## Generate C from a namespace ```bash dsdlc --target-language c path/to/root_namespace --outdir out/c ``` ## Generate profile-agnostic C++ ABI object code ```bash dsdlc --target-language obj path/to/root_namespace \ --obj-abi-language cpp \ --target-endianness little \ --jobs 8 \ --outdir out/obj-cpp ``` ## See the output before you write anything The [Showroom](../showroom/index.md) is a namespace of plausible vendor-specific drone datatypes generated into every supported language and profile. Build the whole tree with: ```bash cmake --build --target showroom ``` ## Next - CLI options: [dsdlc](../reference/commands/dsdlc.md) - Backend behaviour: [Backend Overview](../reference/codegen/backends.md) - Object backend details: [Object Backend](../reference/codegen/object.md) ================================================================================================== Source: docs/reference/index.md URL: https://opencyphal-garage.github.io/llvm-dsdl/reference/ ================================================================================================== # Reference Everything the tools promise, arranged for lookup rather than for reading through. | Group | Contents | |---|---| | [Commands](commands/dsdlc.md) | Switch names, accepted values, and what each tool does with them. | | [Code Generation](codegen/backends.md) | What each target language emits, and the ordering contract the emitters hold to. | | [Language Server](lsp/lint-rules.md) | `dsdld`'s rule catalog, index schema, ranking, logging, and AI surface. | | [Guarantees](guarantees/parity-matrix.md) | What holds across languages, what happens on malformed input, what is reproducible, and what you can verify about a downloaded artifact. | | [Showroom](../showroom/index.md) | Real definitions compiled into every language and profile, with their wire layout. | ## Commands - [`dsdlc`](commands/dsdlc.md) — the compiler and codegen driver - [`dsdl-opt`](commands/dsdl-opt.md) — dialect and pass-pipeline work - [`dsdld`](commands/dsdld.md) — the language server ## Code generation - [Backends](codegen/backends.md) — the target languages and what each produces - [Object Backend](codegen/object.md) — compiled `.o`/`.a` output, ABI, and endianness - [Emit Order](codegen/emit-order.md) — the canonical serialize/deserialize step order every backend renders ## Language server - [Lint Rules](lsp/lint-rules.md) — the rule catalog and suppression schema - [Index Schema](lsp/index-schema.md) — what the workspace index stores - [Ranking Model](lsp/ranking-model.md) — how completions and symbol results are ordered - [Logging](lsp/logging.md) — the structured log channel - [AI Operation](lsp/ai-operation.md) — modes, policy gates, and configuration - [AI Data Flow](lsp/ai-data-flow.md) — what enters the AI surface, what leaves it, what is retained ## Guarantees - [Cross-Language Parity](guarantees/parity-matrix.md) — which backends are covered for which behaviours - [Malformed Input](guarantees/malformed-input.md) — what each backend does with input it should reject - [Determinism](guarantees/determinism.md) — what is byte-reproducible, and across which axes - [Supply Chain](guarantees/supply-chain.md) — the LLVM version lock, SBOM, and what you can verify about a release ### The guarantee pages are generated `parity-matrix`, `malformed-input`, and `determinism`, along with the [Consistency Lint](../development/convergence-scorecard.md) under Development, are written by the report generators under `tools/convergence/` and rebuilt whenever the site is published. Each page carries a **gating mode** banner. ================================================================================================== Source: docs/reference/commands/dsdlc.md URL: https://opencyphal-garage.github.io/llvm-dsdl/reference/commands/dsdlc/ ================================================================================================== # `dsdlc` `dsdlc` is the primary compiler/codegen driver. Run `dsdlc --help` for languages, target syntax, and the full option set. This page covers behaviour that switch descriptions do not carry. ## Support code Support code is everything a backend emits that is not derived from a definition. It is rendered from content compiled into `dsdlc`, and `--generate-support` selects when it is written. | Backend | Support artifacts | | --- | --- | | `c` | `dsdl_runtime.h` | | `cpp` | `dsdl_runtime.h`, `dsdl_runtime.hpp` (per profile) | | `rust` | `Cargo.toml`, `src/dsdl_runtime.rs`, `src/dsdl_runtime_semantic_wrappers.rs` | | `go` | `go.mod`, `dsdlruntime/dsdl_runtime.go` | | `ts` | `package.json`, `dsdl_runtime.ts` | | `python` | `pyproject.toml`, `_dsdl_runtime.py`, `_runtime_loader.py`, `py.typed` | Under `never`, Python still writes the `__init__.py` chain its generated modules import through. ## Dependency files `-MD` writes a make-style `.d` beside each generated output listing the `.dsdl` files it was built from: its own definition plus the transitive closure of the composite types it references. Some outputs have no `.dsdl` source to name: definitions from the [embedded `uavcan` catalog](#embedded-uavcan-catalog) and all [support code](#support-code) are compiled into the binary. Those rules name **the `dsdlc` executable** as their prerequisite instead — upgrading the compiler rebuilds what it produced. An output mixing local and embedded definitions lists its real inputs and the executable. Every path emitted in a depfile or by `--list-inputs` exists on disk, so both can be fed to a build system verbatim. ## Pruning stale output `dsdlc` writes what it is asked for; without `--prune-manifest` it does not remove what it wrote last time. Delete a definition and its generated header stays in `--outdir`, still on the include path, so code that names a type nobody defines any more keeps compiling. `--prune-manifest ` closes that. The run records its outputs in `` and, on the next run, deletes the outputs the previous manifest listed that it no longer produces. Directories emptied by pruning are removed too, so a deleted namespace leaves no shape behind. **One manifest per invocation, not per output directory.** Generation [decomposes](#support-code) — a build may split one namespace across several runs for support, the embedded catalog, and definitions — and a run owns only the files it emits. A run that swept `--outdir` would delete the files its siblings had just written, so each tranche is given its own manifest and prunes only what it owns: ```bash dsdlc -l c --outdir gen --generate-support only --prune-manifest .dsdlc/support dsdlc -l c --outdir gen --generate-support never --prune-manifest .dsdlc/builtin +uavcan.node dsdlc -l c --outdir gen --generate-support never --omit-dependencies \ --prune-manifest .dsdlc/types dsdl/myns ``` Removals are confined to `--outdir`: a manifest naming anything outside it is a hard error rather than a deletion, since a manifest is an input and an input that can name any path is a way to turn a stale file into an arbitrary `rm`. A manifest in an unrecognised format is treated as absent — a format change should cost one stale file, not every configured tree. The flag is ignored under `--dry-run` and the `--list-*` modes, which imply it. A dry run that deleted files while reporting that it wrote none would be worse than either. ## Embedded uavcan catalog For the standard `uavcan.*` namespace, `dsdlc` ships an embedded catalog used by the `mlir` and codegen targets. Types referencing core `uavcan` definitions resolve without external `uavcan` source roots. The catalog is consulted automatically during dependency resolution, and can be named directly as a target with the `+` sigil. `+` targets behave as explicit targets: their dependency closure is generated too, and `--omit-dependencies` restricts output to what was named. They mix freely with filesystem targets, and a local definition sharing a type key shadows the embedded one. Namespace matching is anchored at a dot boundary, so `+uavcan.n` selects nothing rather than standing in for `+uavcan.node`. A selector matching nothing is an error with a did-you-mean; an unavailable version reports the versions the catalog carries. ## Deprecation A definition marked `@deprecated` generates a `Deprecated: …` notice in its documentation comment and an `IS_DEPRECATED` metadata constant (`DSDL_IS_DEPRECATED` in TypeScript and Python, `_IS_DEPRECATED_` in C), in every language. Go recognises the `Deprecated: ` doc paragraph, and TypeScript is additionally given a `/** @deprecated … */` JSDoc block. C, C++, and Rust additionally get a language-native attribute — `__attribute__((deprecated))`, `[[deprecated]]`, and `#[deprecated]` respectively — so naming the type produces a compiler diagnostic. This is **on by default**. Each generated file suppresses deprecation diagnostics across its own body (`#pragma GCC diagnostic ignored "-Wdeprecated-declarations"`, or `#![allow(deprecated)]` in Rust). The suppression is scoped to the generated file: including generated headers is clean under `-Werror`, and only your own code naming a deprecated type is diagnosed. The `obj` backend never emits these attributes. ================================================================================================== Source: docs/reference/commands/dsdl-opt.md URL: https://opencyphal-garage.github.io/llvm-dsdl/reference/commands/dsdl-opt/ ================================================================================================== # `dsdl-opt` `dsdl-opt` is the pass-driver for the `dsdl` MLIR dialect. Use it to inspect and transform lowered DSDL IR through specific pass pipelines. ## Typical use ```bash dsdlc --target-language mlir path/to/ns > input.mlir dsdl-opt input.mlir -pass-pipeline='builtin.module(lower-dsdl-exec,dsdl-annotate-aliasability,dsdl-legalize-endianness)' ``` ## Uses - Debug pass behaviour in isolation - Validate contract attributes across pipeline boundaries - Build reproducible IR test cases for lit/unit tests ## Related - [Architecture](../../development/architecture.md) — the dialect as a contract boundary, and the pass sequence these pipelines are built from ================================================================================================== Source: docs/reference/commands/dsdld.md URL: https://opencyphal-garage.github.io/llvm-dsdl/reference/commands/dsdld/ ================================================================================================== # `dsdld` `dsdld` is the DSDL language server (LSP over stdio JSON-RPC). It is editor/client configured (settings-driven), not primarily CLI-flag driven. ## Run ```bash dsdld ``` ## Typical client settings - `roots` - `lookupDirs` - lint enablement - AI feature mode and trace settings ## Editor integration - Neovim: configure via `nvim-lspconfig` ## Related operator docs - [AI Operation](../lsp/ai-operation.md) - [Index Schema](../lsp/index-schema.md) - [Lint Rules](../lsp/lint-rules.md) ================================================================================================== Source: docs/reference/codegen/backends.md URL: https://opencyphal-garage.github.io/llvm-dsdl/reference/codegen/backends/ ================================================================================================== # Backend Overview `dsdlc` shares one frontend/semantic pipeline and dispatches to multiple backend emitters. ## Targets - `c`: C headers + implementation - `cpp`: C++ model/runtime outputs (`std`, `pmr`, `autosar`) - `rust`: crate/module output with profile/runtime controls - `go`: module/package output - `ts`: typed model + runtime helpers - `python`: package + runtime loader/specialization - `obj`: compiled `.o` + optional `.a` ## Shared principles - One semantic interpretation per DSDL source set - Contract-validated lowering boundaries - Deterministic file planning - Backend parity coverage in CI ## Object code path The `obj` lane runs executable lowering and emits compiled artifacts via host compiler toolchains. See [Object Backend](object.md) for endianness, ABI, and artifact details. ================================================================================================== Source: docs/reference/codegen/object.md URL: https://opencyphal-garage.github.io/llvm-dsdl/reference/codegen/object/ ================================================================================================== # Object Backend (`-l obj`) The object backend compiles generated sources into static objects and optional archives. ## ABI lanes ## C lane (`--obj-abi-language c`) - Stages generated C under `.obj_stage_c` - Compiles C translation units to `.o` - Optionally archives into `.a` - Publishes the staged headers into `--outdir` ## Headers The staged sources are an intermediate this backend compiles itself, but the headers are the only way to call what ends up in the archive, so they are copied into `--outdir` and reported by `--list-outputs` like any other output. The layout matches what the `c` backend produces, so `-I` plus the archive is a complete interface from a single invocation. The staging directories remain, and remain private: nothing should read `.obj_stage_c` or `.obj_stage_cpp`, and their contents are not declared outputs. ## C++ lane (`--obj-abi-language cpp`) - Uses canonical profile-agnostic C++ ABI types - Exports C++ ABI symbols plus C-callable shim symbols - Stages under `.obj_stage_cpp` (including nested C stage) ## Endianness semantics The DSDL wire format is little-endian on every target, so `serialize_`/`deserialize_` perform explicit little-endian bit assembly and are independent of host endianness. `--target-endianness` selects the codegen/legalization strategy; it does not change wire-contract semantics. There is no byte-swap step. - `little`: all fast paths are available, including the zero-copy view helpers (`try_deserialize_view_` / `try_serialize_view_`) for alias-eligible fixed-size sealed layouts. - `big`: `serialize_` / `deserialize_` are fully supported and produce byte-identical wire output to `little` (verified by the object-backend smoke test). The zero-copy view helpers are disabled and return `-DSDL_RUNTIME_ERROR_INVALID_ARGUMENT`. **Note:** the big-endian path is validated by compiling the `LLVMDSDL_TARGET_ENDIANNESS_BIG` code path and asserting wire byte-parity on a little-endian host; it is not yet exercised on real big-endian hardware in CI. ## Example ```bash dsdlc --target-language obj path/to/ns \ --obj-abi-language cpp \ --target-endianness little \ --obj-archive-name my_dsdl \ --jobs 12 \ --outdir out/obj ``` ================================================================================================== Source: docs/reference/codegen/emit-order.md URL: https://opencyphal-garage.github.io/llvm-dsdl/reference/codegen/emit-order/ ================================================================================================== # Canonical emit order The reference **abstract** serialize/deserialize step order that every string backend (Rust, Go, C++, TypeScript, Python) follows. This is a live contract, not a historical plan: the shared render template produces this order by construction, and the emit-order verifier (`tools/convergence/emit_order_verifier.py`, ctest `llvmdsdl-emit-order-verifier`) independently pins it on every build. This prose is the human-readable projection of the machine-checked model in [spec/dafny/CyphalSerdes.dfy](https://github.com/OpenCyphal-Garage/llvm-dsdl/blob/main/spec/dafny/CyphalSerdes.dfy), which *proves* (unbounded, by induction) that serialize/deserialize round-trip and that the read path is bounds-safe, and defines the accepted op orderings (`SerOrderOK`/`DeOrderOK`). The model is the source of truth; this doc is the readable shadow. **Where the order lives in code.** `buildUnionSectionSteps` and `buildFieldEmitSteps` ([`include/llvmdsdl/CodeGen/EmitStep.h`](https://github.com/OpenCyphal-Garage/llvm-dsdl/blob/main/include/llvmdsdl/CodeGen/EmitStep.h)) build the step list/tree; `renderUnionSection` and `renderFieldSteps` ([`lib/CodeGen/EmitStepRender.cpp`](https://github.com/OpenCyphal-Garage/llvm-dsdl/blob/main/lib/CodeGen/EmitStepRender.cpp)) own every cross-statement ordering decision, recursively. Per-backend `UnionSectionSpelling` / `FieldStepSpelling` classes contain *zero sequencing* — only leaf statement idioms. A backend cannot reorder these steps; it can only exercise a declared right (see [Accepted differences](#accepted-differences)). ## Abstract order vs spelling The emit-order verifier checks **abstract op order**, not surface text. Two backends agree if they perform the same ordered sequence of abstract ops, regardless of how each op is spelled. - **Abstract op** (the emit-order verifier asserts these, cross-backend identical): `VALIDATE_TAG`, `MASK_TAG`, `WRITE_TAG`, `ADVANCE`, `SWITCH`, `CASE`, `ALIGN`, `WRITE_SCALAR`, … (full list below). - **Spelling** (the shared-render-template visitor varies these; the emit-order verifier is deliberately blind to them): whether a mask is a separate statement or an inline sub-expression; `match` vs `switch` vs `if/elif`; `Result`/`Err` vs `(rc,0)` vs negative-int return vs `throw`/`raise`; identifier names; indentation. Example: in `serialize` of a union tag, Rust/Go write `set_uxx(buf, off, mask(tag), bits)` (mask folded into the write argument) while TS emits `tag = mask(tag);` then `writeUnsigned(...tag...)` (mask as its own statement). **Both are `VALIDATE_TAG → MASK_TAG → WRITE_TAG`** — identical abstract order, different spelling. The emit-order verifier treats them as equal. This case doubles as a verifier insensitivity check (D1 below). ## Two invariant principles 1. **Write path validates before it emits; read path reads before it validates.** On serialize, a value is range/tag/length-validated *before* the corresponding `WRITE_*`. On deserialize, the raw bits are `READ_*` first, then masked/validated. This asymmetry is intentional and appears identically for the union tag and the array length prefix. 2. **`ADVANCE` (`offset_bits += N`) happens immediately after the matching `WRITE_*`/`READ_*`, never before.** Alignment padding (`ALIGN`) is emitted *before* a field's value ops. ## Section level ### Struct (non-union) — serialize and deserialize For each lowered field step, in lowered order: ``` (padding step) → ALIGN(pad.bits) → PAD(write/skip zero bits) → ADVANCE (value step) → ALIGN(field.alignmentBits) → end of section → ALIGN(8) # trailing pad to byte boundary ``` ### Union — serialize ``` VALIDATE_TAG # unionTagValidate(tag); error-branch on failure MASK_TAG # unionTagMask(tag) WRITE_TAG(tagBits) # set_uxx / writeUnsigned ADVANCE(tagBits) SWITCH(tag) { CASE(optionIndex): ALIGN(option.alignmentBits)