Crucible / Docs

Package an application for AOS

An AOS package is a Nix derivation built from source with the AOS package set. Adding an application normally has three parts:

  1. define the package under pkgs/;
  2. expose its runtime interface when apm must activate it;
  3. include it in a system variant or publish it to a registry.

This guide builds a small service package called acme-health-agent. The example is deliberately self-contained so it can be evaluated and built without a separate source repository.

Use Review package security while choosing dependencies, permissions, and an expose contract. The corresponding operator-visible boundary is documented in Understand the package sandbox.

#Define the package

Create pkgs/acme/acme-health-agent.nix:

{
  mkDerivation,
  coreutils,
  writeShellScriptBin,
}: let
  agent = writeShellScriptBin "acme-health-agent" ''
    set -eu

    while true; do
      printf 'acme-health-agent: healthy at %s\n' \
        "$(${coreutils}/bin/date --iso-8601=seconds)"
      ${coreutils}/bin/sleep 60
    done
  '';
in
mkDerivation {
  pname = "acme-health-agent";
  version = "1.0.0";
  src = null;

  runtimeDeps = [
    agent
    coreutils
  ];

  phases = [
    {
      name = "install";
      script = ''
        mkdir -p "$out/bin"
        ln -s ${agent}/bin/acme-health-agent \
          "$out/bin/acme-health-agent"
      '';
    }
  ];

  expose = {
    units."acme-health-agent.service" = {
      description = "Acme host health agent";
      serviceConfig = {
        Type = "simple";
        ExecStart = "${agent}/bin/acme-health-agent";
        Restart = "on-failure";
        RestartSec = "5s";
      };
    };

    permissions = {
      network = "private";
      tcp-bind = [];
      capabilities = [];
      devices = [];
      host-paths = [];
      syscalls = "restricted";
    };
  };

  meta = {
    description = "Acme host health agent";
    license = "Apache-2.0";
  };
}

Package files are discovered recursively. Files and directories whose names begin with _, and files named default.nix, are not published as packages. The file above therefore creates pkgs.acme-health-agent and the flake output pkg-acme-health-agent.

A discovered file that returns a callable package factory instead of a derivation must be listed in packageFactories in pkgs/default.nix. Keep the factory available through pkgs.<name> for its callers, but exclude it from packageNames, the pkg-* flake outputs, and packages.<system>.all; Nix cannot build a function. Use this explicit inventory because dynamically probing every discovered value would evaluate unrelated packages and trigger their IFDs during otherwise isolated builds.

The builder shell is POSIX sh. Keep phase scripts portable, and use explicit AOS package paths in generated scripts. Do not use /bin/bash, /usr/bin/env, host tools, or nixpkgs packages.

#Understand dependencies

Use the dependency field that matches why the package is needed:

FieldUse
buildDepsCompilers, build systems, generators, and other build-only tools
runtimeDepsLibraries and commands needed when the package runs
propagatedDepsDependencies that downstream builds must inherit

mkDerivation already supplies the wrapped compiler and bootstrap tools. List application-specific build tools, libraries, and runtime commands explicitly. Include tools that an upstream configure script probes before the compile even when the build phases do not invoke them directly; for example, declare Perl in buildDeps when configure rejects a missing Perl interpreter. Declare generators such as Bison even when a release archive includes generated sources, because build rules can regenerate them after unpacking or patching. Pass the generator through the upstream configure interface when one exists so the build cannot silently fall back to an undeclared host command. Make excluded optional components explicit through the upstream configure interface. Do not rely on a missing dependency to disable a component: a CMake helper can return after a failed probe while its parent directory continues generating targets with NOTFOUND include or library paths. Assert the component's disabled cache value after configure so an upstream default change cannot silently expand the package feature set. If a dependency is missing from AOS, package it from source rather than reaching into the host or importing nixpkgs.

Select a package's named output, such as library.dev, when a downstream build needs only that output's headers or package metadata. mkDerivation preserves the package's propagatedDeps on every named output, so downstream search paths still contain the transitive dependencies required by those headers and .pc files. Declare those requirements once on the producing package; do not repeat them on each consumer merely because the producer splits its outputs.

mkDerivation also enables the repository hardening profile. Keep that profile unless an upstream representation is incompatible with one specific flag. For example, code that deliberately uses a trailing one-element or zero-length array as variable-length storage can trigger false _FORTIFY_SOURCE aborts under strictflexarrays3. In that case, preserve fortify and the other hardening checks while selecting the compatible flexible-array interpretation:

hardeningDisable = ["strictflexarrays3"];
hardeningEnable = ["strictflexarrays1"];

Document the upstream data layout and reproduce the actual build-time or runtime abort before adding this exception. Do not disable all hardening to work around one incompatible flag.

Pass $NIX_BUILD_CORES to build and install commands that support parallel execution. Some bootstrap installers rebuild tools, and some build systems forward a separate job count to nested builds; pass the configured budget to those entry points as well.

When a compiler or generator fails, preserve its inputs, command, and build log, then reproduce the failure before choosing a workaround. Fix missing dependency edges or shared-output races where a reproducer demonstrates them. Do not treat a successful retry at a lower job count as a diagnosis of compiler corruption or as evidence for a permanent package-specific limit. See the build concurrency notes for packages to watch under resource pressure and the available reproduction results.

For an upstream release, add fetchurl and fakeHash to the package function arguments, keep version beside the source, and replace src = null with:

{fetchurl, fakeHash}: let
  version = "1.0.0";
in {
  src = fetchurl {
    urls = ["https://downloads.example.com/acme-agent-${version}.tar.gz"];
    hash = fakeHash;
  };
}

After adding the package file, obtain its hash with:

nix run . -- prefetch --package acme-health-agent

Replace fakeHash with the printed sha256-... value. Alternatively, --update edits the package file; inspect that diff before committing. Keep the version, source URLs, and hash together in the package file. Prefer more than one trusted source URL when the upstream has a stable mirror.

Extract only the named members a package consumes from a larger vendor bundle. Some archives contain a ./ member whose mode and timestamp tar applies to the existing extraction root; a sandbox executor can allow file creation there without allowing the builder to change mount-owned root metadata. Naming the required members avoids that unrelated metadata operation and keeps unpacking bounded to the package's declared payload.

For a Bazel package built with mkBazelPackage, leave populateBCR enabled when the dependency graph uses Bazel modules. Set populateBCR = false when the package explicitly disables Bzlmod and resolves only a WORKSPACE graph. The optional empty-workspace synchronization otherwise downloads Bazel's built-in repositories for platforms and toolchains the target does not use; those bytes enlarge the fixed-output dependency closure and require needless network policy exceptions. When a pinned Git repository has a commit-identical official mirror on an already-used origin, prefer that mirror and verify the exact commit before changing the recipe. Keep fork-only commits on their canonical origin rather than substituting an unaffiliated mirror. Apply the same rule to release archives: when one upstream owns multiple official hostnames, verify that the pinned payloads are byte-identical and select the hostname already used by the dependency closure instead of widening its network-origin set.

Leave Bazel action placement to the system Bazel configuration when its local default works in the package sandbox. Do not add a global --spawn_strategy=standalone package flag: command-line package flags override the system configuration and prevent an available remote executor from receiving actions. When nested Bazel sandboxing is unavailable and a package needs standalone execution as its portable local fallback, use --spawn_strategy=remote,standalone; Bazel then consumes a remote executor from the system configuration when present and otherwise falls back locally. If one action class cannot run remotely, scope its strategy by mnemonic instead of forcing the entire build local, and document why that exception is required.

Do not retain Bazel's host_platform, internal_platforms_do_not_use, or local_config_* repositories in a fixed-output dependency closure. They describe the fetch executor rather than a source dependency. Remove those repositories and their markers in postFetch; Bazel recreates them for the executor that performs the real build.

Do not let a fixed-output dependency fetch silently fall back between native and pure-language artifacts. When an upstream package treats a declared native extension as optional, use its supported strict-build switch so compilation failure stops the fetch. This makes the dependency hash describe one artifact mode instead of whichever mode happened to build on the fetch executor.

Use equal-length store-path placeholders for binary files. A replacement with a different byte length corrupts offsets recorded in formats such as ELF, but a fixed-output dependency cannot retain store references. Restore the original equal-length path before the offline build uses the artifact.

Preserve permissions while scrubbing fetched repositories. Go module caches make their files and directories read-only to enforce content immutability, so do not make an entire cache recursively writable just to run an in-place rewriter. Render the transformed bytes into scratch space, make only the matched file writable long enough to overwrite it, and restore its exact mode. This avoids requiring write permission on the cache directory and leaves unrelated module content untouched.

When upstream commits a Cargo lockfile and generated Bazel crate repositories, fetch that checked-in graph without setting CARGO_BAZEL_REPIN. Repinning asks rules_rust to resolve compatible versions against the current registry index, so the result can change even when the package source revision does not. Run a repin only while intentionally updating the source and commit the regenerated lock and repository definitions before calculating the new dependency hash. When a local rules_rust patch changes the repository-rule input digest without changing the Cargo.lock-selected graph, advance the generated lock's exact checksum in the source patch phase and assert the new value. Do not enable repinning in production fetches merely to bypass a stale rule digest.

#Expose the runtime interface

The expose attribute is the contract used by APM. It renders a separate activation artifact containing units, firewall rules, configuration, and a permission declaration. A package without expose can be used at image build time, but it cannot be registered under aos.packages or activated as an APM package.

The renderer creates a package target named:

aos-pkg-<package-name>.target

#Author generated package documentation

Configuration reference belongs beside the package's Nix interface. Add a structured documentation value to configModule; do not create a per-package Markdown option guide:

configModule = {
  module = ./_acme-health-agent-config/module.nix;
  documentation = {
    summary = "Health reporting, listener policy, and reload behavior.";
    sections.quickstart = {
      title = "Quick start";
      blocks = [
        {
          kind = "paragraph";
          spans = [
            {
              kind = "text";
              text = "Enable the agent and select its reporting interval.";
            }
          ];
        }
        {
          kind = "code";
          language = "nix";
          text = ''
            {
              acmeHealthAgent.enable = true;
              acmeHealthAgent.interval = "60s";
            }
          '';
        }
      ];
    };
  };
};

The restricted publisher evaluation mechanically extracts option paths, types, defaults, examples, ownership and contribution rules. It cross-checks any package-authored option enrichment against that declared interface, combines it with expose metadata for services, listeners, credentials, paths and capabilities, and emits canonical aos.package-documentation/v1 JSON. Structured prose supports paragraphs, lists, notes and code blocks; raw Markdown, HTML and external includes are intentionally not representable.

Publication stores the canonical JSON as a reference-free Nix store object and binds its NAR and semantic identities into signed package metadata. A prose-only change updates documentation without changing runtime measurement, while an option or runtime-interface change updates the semantic schema digest. Verify the package's generated interface and publication contract with:

nix-build -A checks.package-documentation --no-out-link
nix-build -A checks.package-expose --no-out-link
apr verify --registry <name>

Activating acme-health-agent enables aos-pkg-acme-health-agent.target, which owns the service unit above. Units marked onlyManualStart = true are installed but are not pulled into that target.

Declare the narrowest permissions the service needs. network = "private" gives the package an isolated network namespace. A service that must use the host network needs network = "host" and the appropriate tcp-bind ports. The package renderer rejects inconsistent permissions during evaluation. The port list remains signed audit and socket-listener intent, but host networking is an explicit downgrade from per-package Landlock/eBPF network enforcement; filesystem, MAC, capability, and systemd sandboxing still apply.

#Add an on-host configuration module

Use configModule when host policy must set typed package options at runtime. Keep the module in a local directory containing module.nix; it receives lib, config, and a resolver-supplied outputs attrset. Declare every runtime output that the module interpolates by name:

configModule = {
  src = ./config-module;
  dependencies = {
    bash = bash;
  };
  declares = ["acmeHealth.command"];
  ownsRoots = [{root = "acmeHealth";}];
};

The module refers to that output without importing a package set:

{lib, outputs, ...}: {
  options.acmeHealth.command = lib.mkOption {
    type = lib.types.str;
    default = "${outputs.dependencies.bash}/bin/bash";
  };
}

mkDerivation exposes the resolved map as configModuleDependencies without copying store paths into the config-only output. Publication must bind the same names to their exact runtime outputs:

apr publish "$STORE_PATH" \
  --config-module "$CONFIG_MODULE_PATH" \
  --config-base-lib "$BASE_LIB_PATH" \
  --config-dependency "bash=$BASH_PATH" \
  --registry acme \
  --key-id initial

Each dependency must be a direct reference of the published runtime output. The registry signs the name-to-path map, and on-host evaluation injects that authenticated map as plain strings. It never exposes ambient packages or instantiates a derivation.

#Build and inspect the package

Add the new file to Git before using its flake output; flakes evaluate the tracked source tree:

git add pkgs/acme/acme-health-agent.nix
nix build .#pkg-acme-health-agent

Inspect the payload and rendered activation manifest:

find result -maxdepth 3 -type f -o -type l
nix-build -A pkgs.acme-health-agent.expose -o result-expose
sed -n '1,240p' result-expose/manifest.json

Run repository checks before publishing:

nix run . -- lint
nix run . -- test eval
nix build .#pkg-acme-health-agent

Add package-specific checks under the derivation's checks attribute when a version command, library link, protocol response, or VM behavior can be tested directly. A successful build proves that the output was produced; it does not by itself prove that the service is healthy.

#Integrate the service into a release image

This is a release-maintainer workflow. Users of a published AOS image should install the package from a registry with apm instead. See Build and customize release images for the image build and validation process.

Register the package in a system variant:

# systems/acme-server.nix
{pkgs, ...}: {
  imports = [./server.nix];

  aos.packages.acme-health-agent = {
    package = pkgs.acme-health-agent;
    bundle = true;
    preset = true;
  };
}

bundle = true includes the package and its activation artifact in the image. preset = true enables its package target when AOS seeds the initial system package profile. A preset package must also be bundled.

Build and validate the image as described in the release-image maintainer guide linked above.

After boot, inspect both the APM state and the service:

apm list --installed --system
systemctl status acme-health-agent.service
journalctl -u acme-health-agent.service -b

#Publish and install the package

Build the store path, then publish it from the registry's authoring clone:

STORE_PATH="$(nix build .#pkg-acme-health-agent \
  --no-link --print-out-paths)"

apr publish "$STORE_PATH" \
  --registry acme \
  --description "Acme host health agent" \
  --license Apache-2.0 \
  --maintainer packages@example.com \
  --key-id release

Create and upload a signed registry release using the workflow in Publish packages and releases. Once the consumer has synchronized that registry, declare the service in a machine-wide desired file:

packages = ["acme-health-agent"]

Preview and reconcile the complete desired set:

apm update --system
apm install --system --from ./desired.toml --dry-run
apm install --system --from ./desired.toml --yes
systemctl status acme-health-agent.service

The file is authoritative: packages omitted from it are removed during reconciliation. apm install PACKAGE --system is instead the OS-sysroot install path and rejects an ordinary application package.

#Ship a new version

Change the package version and source hash, build it, and run its checks. Then publish the new store path and a new registry release. On a canary host, refresh metadata and inspect the candidate:

apm update --system
apm list --upgradable --system
apm policy acme-health-agent --system

The current machine-wide desired-package reconciler installs and removes roots, but does not replace an already-present package with a newer registry candidate. apm upgrade --system stages an A/B OS image, not the runtime package profile. Until a machine-wide runtime upgrade operation ships, roll a new image containing the new package or use a release-specific, tested remove-and-reconcile procedure. Do not present that workaround as an atomic upgrade.

Verify the unit and application behavior before advancing more rollout partitions. Registry channels are monotonic: stop a bad rollout and publish a higher, corrected version rather than moving channel partitions backward.