Skip to main content

Developer Guide

<!-- markdownlint-disable MD013 -->
<!--
SPDX-License-Identifier: Apache-2.0
SPDX-FileCopyrightText: 2026 ndaal Gesellschaft für Sicherheit in der Informationstechnik mbH & Co KG, Cologne
-->
<!-- markdownlint-enable MD013 -->

# Developer Guide — BSI Grundschutz++ OSCAL Viewer

This guide is for people who change the viewer: its code, templates,
translations, tests and release tooling. For running and deploying the
binary see the Administrator Guide; for using the web UI see the User
Guide; for diagnosing a misbehaving installation see the Troubleshooting
Guide. All four guides ship inside the binary and are shown in the
**Info** menu.

## 1. Architecture at a glance

The viewer is **one Rust crate that builds one binary**. Everything it
needs at runtime is compiled in: the Grundschutz++ OSCAL catalog, the
cross-framework mappings, the templates, the CSS, the fonts, the icons,
the README, the changelog and these guides. Copying the binary is a
complete installation.

A request travels through these layers:

```text
TCP accept (src/main.rs)
  -> connection limit + handshake/header timeouts
  -> TLS 1.3 only (src/tls.rs, rustls + aws-lc-rs)
  -> Host-header allowlist (src/host_guard.rs, anti-DNS-rebinding)
  -> router (src/router.rs, matchit)
  -> handler (src/routes/*.rs, src/dialog.rs)
  -> Askama template (src/templates/**)
  -> response with security headers (src/template_response.rs)
```

The main modules:

| Module | Responsibility |
| --- | --- |
| `src/main.rs` | Entry point: CLI parsing, logging, startup checks, the accept loop |
| `src/cli.rs` | Command-line argument parsing |
| `src/router.rs` | The HTTP router built on `matchit` |
| `src/routes/` | Page handlers: home, controls, practices, search, stats, export, annotations, health |
| `src/dialog.rs` | The Info-menu pages (About, System Information, legal pages, Changelog, README, the four guides) |
| `src/oscal.rs` | Typed model of the OSCAL 1.1.3 subset the BSI catalog uses |
| `src/index.rs` | Flattened, query-friendly view over the embedded catalog |
| `src/filter.rs`, `src/search.rs` | Control-list filters and the built-in full-text search |
| `src/meili.rs` | Optional Meilisearch client (off unless `--meili-url` is set) |
| `src/changes.rs` | The "what changed" overlay between two catalog editions |
| `src/export.rs`, `src/export_pdf.rs` | Multi-format export, including Unicode-faithful PDF |
| `src/store.rs` | SQLite annotation store (bundled `rusqlite`) |
| `src/dbdump.rs`, `src/sidecar.rs` | Database dumps and their checksum sidecars |
| `src/crypto.rs` | Optional encryption at rest for annotation bodies and images |
| `src/scan.rs`, `src/yara_fetch.rs` | Upload scanning through `clamscan` and `yr` |
| `src/i18n.rs` | The 49-language UI, one TSV per language in `src/i18n_data/` |
| `src/updater.rs` | `--check-update` and `--self-update` |
| `src/static_files.rs` | Embedded static assets |
| `src/paths.rs` | The `ndaal/` working-directory layout |

`src/lib.rs` exposes the modules as a library so the integration tests,
fuzz targets and harness crates can call them directly.

## 2. Repository layout

| Path | Content |
| --- | --- |
| `src/` | The crate: Rust modules, `templates/`, `i18n_data/` |
| `data/` | The embedded catalog, its change overlay and their licence sidecars |
| `documentation/` | Guides, arc42 views, threat model, code wiki, gate reports |
| `tests/` | Integration tests, `tests/scripts/` gate runners, BDD features, `trycmd` cases |
| `fuzz/` | `cargo-fuzz` targets (one per untrusted-input surface) |
| `afl/`, `hfuzz/`, `fuzz-harness/` | The same surfaces under AFL++ and honggfuzz |
| `kani-harness/`, `loom-harness/` | Model checking and concurrency permutation harnesses |
| `loadtest/`, `criterion-benches/`, `callgrind-harness/` | Load test and benchmarks |
| `scripts/` | Maintenance scripts, each with a sibling `.bats` test and a README |
| `release/` | Release scripts and the cumulative `SHA256SUMS` |
| `.githooks/` | The pre-commit hook and the Git LFS hooks |
| `supply-chain/` | The `cargo vet` audit store |
| `csaf/` | CSAF 2.1 advisories, five checksum sidecars each |
| `skills/` | Vendored engineering guidance; not part of the build |

The harness crates are standalone crates with their own `[workspace]`
table. That is deliberate: `--cfg loom`, Kani and the fuzz engines need
build settings the main crate must not inherit. Check those directories
before concluding that a tool is missing from the project.

## 3. Design principles

- **Offline first.** No CDN, no remote font, no analytics, no runtime
  download the user did not ask for. Fonts (Roboto, Roboto Mono),
  Bootstrap, Bootstrap Icons and HTMX are embedded.
- **Safe by default.** The listener binds `127.0.0.1:8228` over TLS 1.3
  only. A non-loopback bind is refused unless `--allow-non-loopback` is
  given, because the viewer has no authentication.
- **No unsafe code.** `src/lib.rs` and `src/main.rs` carry
  `#![forbid(unsafe_code)]`. The only `unsafe` in the binary lives in
  audited dependencies.
- **One TLS stack.** rustls with the `aws-lc-rs` provider serves the
  listener, the Meilisearch client and the self-update client. Do not
  add `reqwest`, `openssl` or a second crypto provider.
- **Escape, never render, untrusted text.** The README, the changelog
  and the guides are shown as escaped Markdown inside `<pre>`. Nothing
  reaches the browser as markup unless a template wrote it.
- **Capability-based file access.** Paths that come from outside the
  binary (CLI, environment, uploads, dumps) go through `cap-std`
  directory handles or component validation. Never join an untrusted
  string onto a path and open it.
- **Fail loudly.** A gate that skips says so. A restore without a valid
  checksum sidecar is refused. A self-update without a matching
  checksum does not install.
- **Documentation travels with the binary.** Guides are embedded with
  `include_str!`, so the page always matches the code that serves it.
- **Idempotent tooling.** Every script can run twice with the same
  result: catalog refresh, release steps, CSAF repair.

## 4. Naming conventions

| Item | Convention | Example |
| --- | --- | --- |
| Rust modules and functions | `snake_case` | `dialog_system_info` |
| Types | `UpperCamelCase` | `DocPageTemplate` |
| Constants holding an embed | `SCREAMING_SNAKE_CASE` + `_MD`/`_TEXT` | `DEVELOPER_GUIDE_MD` |
| Routes | lowercase, `-` between words | `/system-info` |
| i18n keys | `<area>.<item>` | `info.developer` |
| Shell scripts | `snake_case.sh` with sibling `.bats` and `.README.md` | `update_catalog.sh` |
| Skill directories | `kebab-case` | `rust-info-developer-guide` |
| Guides | lowercase `snake_case.md` in `documentation/` | `troubleshooting_guide.md` |
| Release binaries | `grundschutz-oscal-viewer-<version>-<triple>[.exe]` | `…-1.6.60-x86_64-unknown-linux-musl` |
| Tags | `v<version>` | `v1.6.60` |
| Checksum sidecars | `<file>.<algo>` with hyphenated algorithm names | `.sha-256`, `.sha3-512` |

No path may contain whitespace or non-ASCII characters.

## 5. Versions, tags and releases

The crate follows Semantic Versioning. One version number covers the
binary, the crates.io package and the GitLab release.

1. Bump `version` in `Cargo.toml`; `Cargo.lock` follows on the next
   build.
2. Open a `## [Unreleased] - <version>` section in `CHANGELOG.md` while
   the work is in progress.
3. **Before building release binaries**, retitle it to
   `## [<version>] - <YYYY-MM-DD>`. The binary embeds `CHANGELOG.md`;
   targets are compiled one after another over several hours, so an
   edit during the build gives different platforms different text.
4. Cross-compile the six targets with `release/build_release_targets.sh`
   (cargo-zigbuild) into `dist/`.
5. Generate the five checksum sidecars per binary and the per-version
   `grundschutz-oscal-viewer-<version>-SHA256SUMS.txt` asset; append the
   new lines to the cumulative `release/SHA256SUMS`.
6. Tag `v<version>` and create the GitLab release with all assets
   (`release/create_release_on_gitlab.sh`).
7. Publish to crates.io (`release/create_release_on_crates.io.sh`).
   Run `cargo publish --dry-run` first: a file that `include_str!` or
   `include_bytes!` embeds but that is missing from the `include` list
   in `Cargo.toml` builds locally and fails on crates.io.
8. Publish the CSAF release announcement under `csaf/`.

`cargo binstall` and `--self-update` read the same asset names. Both
URL patterns live in two places — `[package.metadata.binstall]` in
`Cargo.toml` and `src/updater.rs` — and the updater tests pin them. If
you change an asset name, change both and the release scripts in one
commit.

## 6. Building and running

```sh
cargo build --release
just run                     # cargo run, default flags
cargo run -- --help          # every flag; see Administrator Guide §2
```

The Rust floor is `rust-version = "1.93"` (edition 2021). The UI is at
`https://127.0.0.1:8228/`; the certificate is self-signed and generated
at startup unless you pass your own.

## 7. Templates and the user interface

Templates are Askama 0.16 files in `src/templates/`:

- `layouts/base.html` is the page frame. Its inline scripts are listed
  by hash in the Content-Security-Policy. If you edit an inline script,
  the test `csp_script_hashes_match_inline_scripts` in
  `tests/test_routes.rs` fails until the hash is updated.
- `partials/nav_right.html` holds the language switcher and the **Info**
  menu.
- `pages/*.html` are the pages; `pages/doc.html` is the shared page for
  embedded documents.

The UI follows the ndaal design system (`skills/ndaal-design`): no
invented colours, nothing signalled by colour alone, a keyboard path for
every pointer action, and WCAG 2.2 AA contrast. Decorative icons carry
`aria-hidden="true"`; the text next to them names the item.

### 7.1 The Info menu

The order of the Info menu is fixed by ndaal-design § 8.1 and pinned by
the test `info_menu_follows_the_design_order_and_ends_with_troubleshooting`
in `src/dialog.rs`:

1. About (`/about`)
2. System Information (`/system-info`)
3. Privacy, Security, License, Imprint (after a divider)
4. Changelog, README, Administrator Guide, User Guide, Developer Guide,
   Troubleshooting Guide (after a second divider)

To add an embedded document page:

1. Add the Markdown file under `documentation/`.
2. Add a `pub const …_MD: &str = include_str!(…)` and a handler in
   `src/dialog.rs` that renders `DocPageTemplate`.
3. Register the route in `src/routes/mod.rs`.
4. Add the menu item to `partials/nav_right.html`.
5. Add the label key to **all 49** files in `src/i18n_data/`.
6. Add the file to the `include` list in `Cargo.toml`.
7. Extend the `PAGES` and `DOC_ROUTES` tables in the `dialog.rs` tests.

### 7.2 Translations

Each language is one TSV file, `src/i18n_data/<code>.tsv`, holding
`key<TAB>text` lines. English is the fallback, so a missing key does not
break the page — it silently shows English. Two tests stop that:

- `every_language_table_is_complete` requires every table to carry
  exactly the English key set.
- `every_all_language_is_embedded_in_the_tsv_table` requires every
  language in the switcher to be embedded.

Adding a language needs its file, its row in the fixed-size `TSV` array
in `src/i18n.rs` **and** the bump of that array's length. Without the
row the language appears in the switcher but serves English text.

## 8. Testing workflow

Write the test first and watch it fail, then make it pass. A test that
cannot fail when the behaviour changes is not a test.

| Layer | Where | Run with |
| --- | --- | --- |
| Unit tests | `#[cfg(test)]` modules in `src/` | `cargo test` |
| Integration tests | `tests/*.rs` | `cargo test --all-targets --all-features` |
| Doctests | `///` examples | `cargo test --doc` (not covered by `--all-targets`) |
| Runner | all of the above, isolated per test | `cargo nextest run --all-features` |
| Property tests | `tests/proptest_invariants.rs` | part of `cargo test` |
| CLI contract | `tests/cmd/*.trycmd`, `tests/cli_contract.rs` | part of `cargo test` |
| BDD | `tests/features/*.feature` | `cargo test --test bdd` |
| Compile-time invariants | `tests/static_assertions_invariants.rs` | `cargo build --tests` |
| Fuzzing | `fuzz/fuzz_targets/` (78 targets) | `cargo +nightly fuzz run <target> -- -max_total_time=60` |
| Mutation testing | `.cargo/mutants.toml` | `cargo mutants --file <module>` |
| Model checking | `kani-harness/` | `cargo kani` inside that crate |
| Concurrency | `loom-harness/` | `RUSTFLAGS="--cfg loom" cargo test` inside that crate |
| Load | `loadtest/`, `tests/scripts/test_goose.sh` | against a running viewer |

Practical notes:

- Never use `cargo test --lib` as a gate: it skips every integration
  test and every doctest.
- Run cargo-mutants one module at a time. A "timeout" is not a missed
  mutant; check it before calling it a gap. The target is a mutation
  score of at least 90 % and line coverage of at least 85 %.
- A new parser, decoder or request handler for untrusted input needs a
  fuzz target before it is merged.
- Restores refuse a dump without valid checksum sidecars, so tests and
  fuzz targets that exercise the restore path must write them.

## 9. QA toolchain

### 9.1 The pre-commit hook

Enable it once per clone:

```sh
git config core.hooksPath .githooks
```

Phase 1 repairs what it can (`cargo fmt`, `cargo clippy --fix`,
`cargo update --workspace`) and stages the result. Phase 2 verifies and
aborts the commit on the first failure:

1. `typos` — spelling in code, comments and documentation
2. `cargo fmt --all --check`
3. `cargo clippy --all-targets --all-features -- -D warnings`
4. `cargo test --all-targets` and `cargo test --doc`
5. `cargo doc` — broken intra-doc links
6. `rust-doctor --scope files --base HEAD` — an error-severity finding
   blocks; the score is reported but advisory

Each step has an environment switch (`RUST_HOOK_TYPOS=0` and so on) for
an emergency. Do not make skipping the habit: a skipped gate is a gate
that did not run.

### 9.2 The full gate set

```sh
bash scripts/quality_gates.sh --fast   # dev loop, no live-server gates
bash scripts/quality_gates.sh          # everything, including fuzz smoke
just qa                                # the core set (scripts/run_qa.sh)
```

`quality_gates.sh` runs the native gates (fmt, clippy, tests, doctests,
docs, htmlhint, oxlint), a fuzz smoke pass and the live HTTPS route
sweep, and dispatches the remaining gates to `tests/scripts/test_*.sh`.
A missing tool is reported as SKIP; only a real failure exits 1.

Gate configuration lives at the repository root: `clippy.toml`,
`rustfmt.toml`, `deny.toml`, `audit.toml`, `rust-doctor.toml`,
`.typos.toml`, `.oxlintrc.json`, `.markdownlint.yaml`, `.rumdl.toml`
and `src/templates/.htmlhintrc`.

Two traps that have cost time:

- `typos` matches `extend-exclude` globs against the path as given. The
  hook passes an absolute path, so every glob in `.typos.toml` needs a
  `**/` prefix; `typos .` alone can be green while the hook is red.
- rust-doctor has no path exclusion. `tests/scripts/test_rust_doctor.sh`
  scans a copy of the tree without `skills/`; the hook limits itself to
  the files a commit touches.

### 9.3 Supply chain

`cargo audit`, `cargo deny check`, `cargo vet check` and
`cargo machete` run in the gate set. A new or upgraded dependency
follows `skills/rust-library`: read the real latest version from the
registry, add adoption tests for the surface you use, and record why the
crate was chosen.

## 10. Backward compatibility

Treat these as public interfaces and change them only with a version
bump, a changelog entry and a migration note:

- **Command-line flags and `GSV_*` environment variables.** Add new
  ones freely; never rename or repurpose an existing one. `tests/cmd`
  pins `--help` and `--version`.
- **Routes.** Bookmarks and scripts use them. A moved page keeps a
  redirect.
- **The annotation database schema.** The store migrates forward on
  open; a new column needs a migration and a test that opens an old
  database.
- **Dump format.** A dump written by an older release must still
  restore, provided its sidecars verify.
- **Release asset names.** `cargo binstall` and every installed binary's
  `--self-update` resolve them by name.
- **i18n keys.** Rename a key in all 49 files at once, never in some.

`cargo semver-checks check-release` guards the library API.

## 11. Contribution checklist

Before you ask for a review:

- [ ] The change has a test that failed before the fix.
- [ ] `cargo fmt`, `cargo clippy -D warnings` and all tests pass,
      including `cargo test --doc`.
- [ ] Functions stay within 50 lines, nesting depth 3, 10 branches and
      4 parameters.
- [ ] New untrusted input has a fuzz target and a property test.
- [ ] New user-facing text exists in all 49 TSV files.
- [ ] Every file you embed is listed in the `Cargo.toml` `include` list.
- [ ] New source files carry the SPDX header.
- [ ] `CHANGELOG.md` describes the change for users.
- [ ] The guides, the README and its four translations (`LIESMICH.md`,
      `LISEZMOI.md`, `LEEME.md`, `LEGGIMI.md`) match the change.
- [ ] Markdown passes markdownlint and rumdl.
- [ ] You staged only the files you meant to; no `git add -A`.