Imported from junioryono/billet (.claude/skills/billet-lifecycle/SKILL.md). Install upstream with npx skills add junioryono/billet --skill billet-lifecycle. Copyright stays with the author.

The local service lifecycle

What this area is

internal/lifeops inspects a Linux host (units, files, running processes; it prints nothing) and converges it; internal/lifeops/launchd is its sibling for macOS launch agents, deliberately not one abstraction because the vocabularies are not shared. cmd/billet/local*.go hold the order the commands act in, which is the whole safety content: up is check, server, node, each started, proved, then enabled; down is seal, wait, stop node, stop server, disable both; uninstall is down plus forgetting the services; backup, restore and recover are in billet-backup-restore. deploy/ holds billet-server.service, billet-node.service, billet-backup.service and .timer, billet-rbd.conf, the two sh.billet.*.plist agents, the packaged billet.yaml template and the package scripts; deploy/units.go exposes the names as constants.

Rules

The units are load-bearing and each line was found by installing a package. Both units pass --config /etc/billet/billet.yaml explicitly (billet never reads the working directory). Both are Type=notify, because only that makes a successful systemctl start mean the process reached READY=1. TimeoutStopSec=88200 is not sized from drain_timeout (a drain has no bound); a SIGKILL mid-drain loses billet's bookkeeping, not the jobs, because the guests keep running and the next process re-adopts them. The server runs as billet with an empty capability set; the node runs as root (Docker's socket is host root, Firecracker needs TAP devices, cgroups, chroots and process signalling) with /srv/jailer, its state directory and its runtime directory the only writable paths under ProtectSystem=strict. lock_dir is set in the packaged config because billet derives the default from HOME, which a service does not have.

The package enables and starts nothing. Installing billet must not connect a machine to GitHub before /etc/billet/billet.yaml says something true. postinstall.sh loads the rbd module, creates the billet account (refusing an existing one whose home is not /var/lib/billet), creates /var/lib/billet as root:root 0755 (the common parent must not let the server rename root-owned node, kernel or upgrade directories), /srv/jailer as root:root 0755 and /etc/billet as root:billet 0750, copies the template only when no config exists, and warns about drop-in overrides that could break Type=notify. /var/lib/billet is created by the script rather than shipped, so a removal cannot delete the identity and the CA key. preremove.sh stops node then server on a real removal and fails the removal if either will not stop. The template lives at /usr/share/billet/billet.yaml, not under /usr/share/doc, which Debian's slim images path-exclude. There is no .apk: Alpine has BusyBox adduser and OpenRC.

On a Mac the node is a launch agent and a root daemon cannot do the job. Since macOS 15 Virtualization.framework needs an unlocked login.keychain (it fails as SecKeyCreateRandomKey_ios failed, Failed to generate keypair, Interaction is not allowed with the Security Server, none naming a keychain); a headless SSH session leaves it locked, so a dedicated node wants automatic login and one first login through Screen Sharing. tart's store is per-user. The agent's PATH is not the shell's (launchd has no Homebrew prefix, so tart and softnet resolve to nothing), and ExitTimeOut is pinned to 88200 because launchd's default is five seconds (not the twenty the man page says) and zero means infinity. billet local up installs, starts, proves and enables the agents; run it as the account that will run the node, never under sudo.

systemctl show hides the privilege prefix. ExecStart=+/usr/bin/billet … runs with full privileges whatever User= says, and -p ExecStart is byte-identical with and without the +. Only -p ExecStartEx shows flags=privileged (systemd v246 and later; an absent answer is uncertainty). Do not parse the unit file instead: systemd accepts whitespace around the key and treats an empty assignment as a list reset, so a hand parser and systemd disagree.

What launchd loaded is not what its plist says. launchd reads a plist once, at bootstrap; launchctl print reports the loaded values, so a node can run a stale five-second drain grace while its file is byte-identical to the shipped one. up compares the loaded job's program, arguments, timeout and whole environment (a redirected TART_HOME makes the inventory describe another machine, which resells live guests). A bootout returns in zero seconds with the process still draining, the service in the domain as state = SIGTERMed with its pid; billet captures the pid first and waits for both the domain and the process. launchctl kill TERM would restart these agents (KeepAlive{SuccessfulExit:false}), so the stop primitive is bootout. The disabled-override database is durable, keyed by label, outlives the plist and accepts labels that never existed: uninstall removes the plist, flushes that removal, then clears the override, because the reverse order leaves a window in which a login starts the node being uninstalled, and leaving it set makes the next install bootstrap a service launchd silently refuses to run. Bootstrap failed: 5: Input/output error means already loaded, still draining or disabled alike; diagnose with print and print-disabled. A parenthesised value ((never exited), (unlimited)) is launchd saying there is no value. launchd reports a pid before the program has run a line, and there is no start timestamp and no sd_notify, so readiness cannot be proved; up says "the same pid survived a settle window" in those words.

up writes nothing structural on systemd and does write the service definition on launchd. The Ansible role renders its own units, so "differs from the packaged unit" is the ordinary state of a role-managed host and is named, never refused. Nothing else renders a plist, so a difference there is an operator's edit or a stale billet, and both want a person. GitHub must be proved, not unrefuted: billet check exits 0 when the probe was skipped or could not complete, and only the explicit verified verdict permits starting a server. An active service is never touched, and deactivating is a node draining, so every ActiveState is mapped explicitly and anything unrecognised refuses. Start, prove, then enable, never enable --now; launchd cannot start a disabled label, so there UnitPlan.EnableBeforeStart declares the weaker order and the unwinding protects the host. Only what this run committed is unwound, on context.WithoutCancel with a timeout, because the likeliest way in is the operator's own interrupt.

When a check cannot be built, observe. Four rounds of modelling what starting one unit does to the other were wrong every time, because the closure is systemd's to compute through units billet has never heard of. Sample both units before and after, and refuse if the one you did not start moved; that catches Conflicts=, a transitive Requires= and [Install] Also= without modelling anything. Sample both even on a node-only host, because that is where a transitive Requires= starting the server matters.

The seal is not the authority; the barrier is. Sealing stops new work being admitted, takes effect at a listener's next poll, and says nothing about what is running. "Is it safe to stop now" is answered by what the deployment holds (alloc.Quiescence) and then by the compute barrier, which asks the machines (billet-capacity). down re-reads the clearance immediately before the first stop, which narrows the gap without closing it; --without-compute-proof prints a different conclusion, and down skips the stage, saying so, when this host's control plane is not running. A node-only host cannot seal or prove anything and says so, naming billet drain --wait against the control plane; refusing to run there was rejected because it pushes operators to plain systemctl stop.

Provenance decides who may clear a seal. A local-down seal is cleared by the next successful up; an operator seal (from billet drain or local recover) is left alone and reported. A row billet cannot read is never sealed over: AdmissionUnknown is the fail-closed zero value, and writing a local-down seal onto it would turn "could not tell" into a seal the next up clears.

Only states that prove no process remains count as stopped. systemctl stop returns when its job completes and says nothing about whether the unit came back; refusing only active and activating accepted deactivating. Allowlist inactive and failed, treat an empty answer as uncertainty and a future state as not proof. A deliberate stop exits 0, or systemctl stop billet-node marks the unit failed after every supported drain. down is several jobs, so a failure part-way reports what was stopped, what was disabled and any unit stopped but still enabled.

One lifecycle command at a time, via a host flock taken immediately before the first mutation. Not at the top, or a config the packaged units cannot use is reported as a failure to create a lock directory; --dry-run never needs it. macOS has no /var/lock and no /usr/local/var, so up refuses with the sudo mkdir -p to run rather than creating root-owned directories.

Measured systemd facts, each of which reads the other way somewhere. An absent unit answers show with exit 0, LoadState=not-found and an empty UnitFileState; an error from show is a different fact and an empty LoadState is neither. StateDirectory= repairs ownership recursively only when the top directory's owner is wrong, so a root-run billet check leaves billet.db and billet.lock root-owned under a correct directory. Names= shows an alias only once its link exists. systemctl disable follows [Install] Also=. OnFailureJobMode is replace and the *Action settings are none on the packaged units, so "refuse if non-empty" refuses every correct host. CapabilityBoundingSet is the full set on the node. Type=exec returns from start as soon as execve succeeds. StateDirectory= and RuntimeDirectory= are LISTS, and show renders one as its entries separated by spaces (RuntimeDirectory=billet/locks billet/registration on the node unit, measured 2026-09-09): lifeops admits a configured directory that is one entry, whole, and refuses the joined text, a prefix of an entry and the entries' parent; a comparison against the rendered value as one path refused every packaged node the day the unit gained its second entry.

billet-upgrade.timer and billet-images-refresh.timer are the one exception to the package enabling nothing. postinstall runs systemctl enable --now on both (and says so when it cannot), preremove disables them first; the rule they except exists to keep an install from connecting a machine to GitHub, and these connect nothing: the upgrade oneshot runs host-upgrade --from-rollout as root every five minutes and acts only on a rollout the ledger records, the images oneshot runs images refresh daily, and both exit doing nothing under release: {automatic: false}. deploy/units.go embeds them and scheduled_test.go pins user, ExecStart, the oneshot shape and the timer's cadence. On a Mac the same two are sh.billet.upgrade (StartInterval 300) and sh.billet.images (StartCalendarInterval daily), no KeepAlive, RunAtLoad; agentOf knows them, Converger.Scheduled() names them, and local up installs and loads both through EnableScheduled after the two services are proved (install, enable, then bootstrap; a loaded idle job is booted out and back in so a changed plist takes effect; a loaded job WITH a process is left alone, because it may be the upgrade transaction). up reports a schedule it could not install as exit 2 like a sealed deployment; uninstall boots them out before removing them; status prints them. up also requires /usr/local/bin writable (UpRequest.BinaryDir, added to requiredDirs), because the updater renames into it as the operator.

up enables the timers as a reported last step outside the unit plan. enableTimers in cmd/billet/localup.go runs after every service is up and proved: the backup and upgrade timers on a control-plane host, the refresh timer on a firecracker node with a Ceph cluster, each enabled and then armed with Converger.StartTimer (start plus an ActiveState=active proof, since a timer has no process for StartAndProve to watch), reported rather than fatal, naming systemctl enable --now. It exists because billet-backup.timer is not enabled by the package (each archive holds a copy of the App key) and a host converged from a tarball ran no postinstall; the timers are deliberately not in the plan up/down prove and unwind, which owns the two services that hold compute and a ledger, and down leaves them running, since a backup of a stopped deployment is the useful kind. systemd only, by ManagerName(); launchd's StartTimer refuses, and the Mac's schedules are enableScheduledAgents. local status reports the update timers' enablement on Linux; billet check reports the backup, upgrade and refresh timers, and the host's converge guard (checkGuard: informational under a day, a WARNING naming converge-guard recover past guardStaleAfter, a WARNING for an unpublished hold or a record whose time cannot be read); billet status prints guard when one is held (printGuard). The guard itself is billet-releases-and-upgrades. The Ansible host role renders and enables the refresh timer in tasks/images-refresh.yml on every node that boots guests, under the same billet_automatic_updates switch as the upgrade timer, and removes it elsewhere. It also renders and enables the backup timer in tasks/backup.yml on every host whose server may run (billet_server_should_run, never the bare flag: a prepared host is not archived), under billet_backup_timer (default true), and removes it elsewhere; the rendered unit carries the same ledger fence as the server unit (RequiresMountsFor= always, plus Requires=/After= on the exact mount unit when a volume is configured) and a drop-in on it is refused by asking systemd for DropInPaths after the reload. The packaged billet-backup.service carries RequiresMountsFor=/var/lib/billet/server in [Unit] for the same reason; deploy/units_test.go pins the directive and its section, because systemd ignores it under [Service].

The rehearsals drive the packaged units on real systemd in containers. scripts/rehearsal-lib.sh (make rollout-rehearsal recover-rehearsal ca-rotation-rehearsal promotion-rehearsal) installs the .deb into privileged ubuntu:24.04 containers with systemd as PID 1, runs billet local up, local down, local recover, resume and host-upgrade there, and asserts on systemctl is-active, journals and the state directory's ownership. A PostgreSQL controller under the packaged unit gets its DSN from a whole unit rendered from the packaged one plus EnvironmentFile=-/etc/billet/server.env, the file the host role renders, never from a drop-in: local up and the host transaction refuse a billet unit with an effective drop-in, and the promotion rehearsal measured that refusal on 2026-09-04 before it rendered the unit the way the role does.

Every systemctl the inspector runs is bounded, and a stop or a start runs under the caller's deadline from the unit's own bound. internal/lifeops/systemctl.go's execRunner cancels with TERM (not exec's immediate KILL), KILLs after the wait delay (WithWaitDelay, ten seconds by default), ends the wait at the same delay for a descendant that keeps the pipes open (the answer refused as not read whole, never a truncated value), and bounds each stream at 1 MiB. Property reads and enablement changes run under WithTimeout; StopAndProve and StartAndProve do not, so every caller passes a deadline: local up and the transaction's StartServices derive theirs from deploy.UnitStartTimeout (120s) plus lifecycleDeadlineMargin, local down from deploy.UnitStopTimeout (88200s, the node's drain) plus the margin, the endpoint migration from --stop-timeout, and the transaction's StopNode deliberately passes none (cmd/billet/lifecycledeadline_test.go pins each through WithObserver; deploy/units_test.go pins the constants against the shipped units). Without the bound a hung systemctl held local up forever, and a cancelled one was KILLed under its own stop job (internal/lifeops/systemctl_bounds_test.go).

A root installer prepares the PACKAGED directory, never the calling account's. billet local prepare is root's and Linux's, and with no configuration to read it prepares <retirement.Root>/server, not config.DefaultServerStateDir(), which answers the USER's config directory (/root/.config/billet for root): the package's postinstall runs before any configuration is seeded, and preparing root's own directory failed on its missing parent and left the install's unit decisions deferred (measured in CI, 2026-09-12). A test that pins hostOS pins retirement.Platform with it (asLinux is useRetirementRoot), or it proves an ordering against a host no fleet runs: off Linux there is no global exclusion and an ordinary ledger open still creates the identity directory, which hid a whole class of Linux-only refusals until CI ran them.

A host is moved onto the authority exclusion by an installer, and only an installer. billet local prepare (root, Linux) is retirement.Bootstrap: init lock, the identity directory created ONLY when the service-account record, the global lock and the status are all positively absent beside it (metadataAbsent; an absent directory beside any of them is a retirement's move or damage and is left absent, reported as IdentityDirAbsent and, in the JSON answer, identity_dir_absent), the global lock created (Hold.Created says whether), the status read (unreadable OR malformed refuses through admitBootstrapStatus; a closed one is returned), the service-account record written from the account the installer VALIDATED (never read back), both locks re-owned by descriptor (RepairLocks answers a non-nil list, and reownLock refuses a second link or another filesystem). Three callers: the package's postinstall (after /var/lib/billet exists, on install and upgrade), local up (under its lifecycle lock, before check opens the ledger; a closed status refuses the start), and the host role's service-account.yml (early on an existing installation, after account.yml on a fresh one with the passwd fact re-read there, and before the transaction's root billet check; the installed-identity refusal AND the unrelated-account refusal live in installed-identity.yml, imported by both, so no record is written for an account either would reject; the command runs through the first executable a PROBE proves knows local prepare (<exe> local capability-probe, whose refusal quotes the word and then names the local subcommands the release knows, so the probe word carries no "prepare" and the match is on the list; the answerer first, then a staged candidate), because the answerer proved only the converge guard, a release before local prepare also answers that, a fresh host's managed binary is installed later, and default('/usr/bin/billet') does not replace the empty default of billet_managed_binary; a closed status or an absent identity directory in the answer STOPS ordinary convergence before account.yml recreates the server directory, with the runbook named, until the retirement's own task file handles such a host). Postinstall makes its unit decisions and its config seeding under the lifecycle lock through an fd-level flock -w 60 9, re-asking the status inside and acting only on a CURRENT answer (a preparation that fails under the lock defers the decisions exactly as contention does), and on contention leaves everything as it was and names dpkg-reconfigure billet as the retry; local up is not that retry, since it loads the configuration first. local up on darwin runs as the login user and does none of this.

• On the reference systemd node, ProtectSystem=strict made /var/lib/billet/upgrades and /usr/bin read-only to the node's children: its first dispatched upgrade refused open /var/lib/billet/upgrades/transaction.lock: read-only file system on 2026-09-06 (#98). setsid does not escape the unit's cgroup; KillMode=mixed kills that child when the updater stops the node. Node upgrades therefore launch through systemd-run in a transient root oneshot of their own; opening more writable paths in the node unit would leave the lifetime defect.

• A focused ExecUpgrader probe on Ubuntu 24.04 arm64/systemd 255.4-1ubuntu8.17 (2026-09-06) received a fixture PostgreSQL variable through systemd-run --setenv=NAME without its value in argv, acknowledged through the node's state-directory socket, stopped a parent unit with ProtectSystem=strict, PrivateTmp=true and KillMode=mixed, and wrote outside the parent's writable paths after that stop returned. Both units exited successfully and the socket was gone. This proves the launch boundary, not the full host-upgrade transaction.

• Ubuntu 24.04 / systemd 255 for every systemd fact above; macOS 26 for every launchd fact, in internal/lifeops/launchd/reallaunchd_test.go.

• launchd default ExitTimeOut: 5 seconds.

• A Setsid child of a launch agent survives bootout of that agent and keeps running after the agent's process exits (macOS 26, realscheduled_test.go); this is what lets the node agent start the updater that boots it out.

• A oneshot with StartInterval and no KeepAlive runs at load and stays loaded, with no process, after exiting 0.

• bootout: returns in 0 seconds with the process still draining.

• A launchd-managed node ran a real Actions job to green and destroyed its guest.

• The restore rehearsal found the ownership gap StateDirectory= cannot repair.

• make systemd-lifecycle is the only test of the lifecycle against a real service manager and the real package; it skips without a working App credential.

Retirement operation admission and stopped boundaries

Retirement uses Inspector.AdmitOperations before intent, resumed mutation and every timer/controller stop and disable and retained-node enable, stop and start. The trusted journal and retirement.Decide determine the remaining sequence. Admission is read-only, rereads sources and path evidence, and grants only the requested target effect. Unreadable, changed or unsupported evidence refuses as could-not-tell before the operation and leaves the phase unchanged. Both systemctl show and typed busctl reads are required. Each operation-admission pass reads all needed properties in one systemctl show invocation per unit: an unfiltered --all read for the retained node and an explicit --property= list for other units. It uses one typed bus invocation per execution unit; standard no-op leaves and timers need no execution-interface read. The retained node uses Properties.GetAll with an empty interface (systemd v255 bus-objects.c:1255) to combine Unit and Service values. Evidence is local to the call, discarded once before the comparison reread and at return; no cache crosses a retirement step. CI remains the runtime validation of the batched reader. Rereads compare definition sources and their contents, unordered relationship sets, directory/setup values and manager policy; activity, jobs, enablement and inverse timer triggers are judged afresh, including after a disable reload. Timestamps, next-elapse values and invocation counters are not definition changes. A policy mismatch names the property and both values; source contents are represented by digests in diagnostics. Quiet timer masks require LoadState=masked, UnitFileState=masked or masked-runtime, ActiveState=inactive and no job, plus FragmentPath=/dev/null or an lstat-proved symlink at FragmentPath whose readlink target is exactly /dev/null, before stop/disable helpers take the positive no-op path. A failed lstat or readlink is could-not-tell (operation-mask-unreadable), never absence. CI on Ubuntu 24.04/systemd 255.4-1ubuntu8.17 (2026-09-15, 55cdd81) measured FragmentPath as the masked timer’s own /etc/systemd/system/ pathname. This operation allowance does not relax the durable stopped-state requirement that refuses a runtime-only mask. Full closed-edge admission also precedes request lock and ledger preparation, EnsureRetiredDir, the staged configuration and intent publication, using protection derived from the installed request configuration before a journal exists and from the recorded journal on resume. Direct trigger checks alone cannot authorize preparation: a path watcher targeting a helper whose OnSuccess= names backup is related through backup’s OnSuccessOf= and refuses before directory creation.

Retirement admits a closed list of edges: each relationship property and its source and destination classes must match an entry in internal/lifeops/operationclosed.go. Every forward, inverse and installation relationship from or to the controller, backup and updater services, both retirement timers, retained node, required billet-network.service, configured billet-dnsmasq@ instances and dedicated ledger mount is checked, including loaded aliases. An unlisted edge refuses with operation-edge-outside-set, naming the property and unit. A familiar endpoint name grants no authority: WantedBy=multi-user.target is admitted for the controller, but OnSuccess=multi-user.target is refused, and the backup cannot install OnSuccess=billet-server.service even while it is allowed to finish. Completion handlers, upholds, namespace joins and reload propagation are outside the list. Stop propagation admits only the dedicated ledger mount’s StopPropagatedFrom= naming the exact device derived from its own What=, and that device’s inverse PropagatesStopTo= toward the ledger mount; every other stop-propagation edge involving a protected unit refuses. The pair is optional: the first CI measurement on Ubuntu 24.04 with systemd 255.4-1ubuntu8.17 (2026-09-15) reported no such stop-propagation edge for the loop-backed ext4 ledger template; the second (55cdd81) reported the exact pair. Retirement never stops a device, so a reported exact pair is a no-op for its operations (mount_add_device_dependencies). Each timer may trigger only its own service. No shipped unit declares installation Also=, so every nonempty assignment refuses, even before a later empty assignment. Every protected role must resolve to a distinct canonical Id, and no role’s Names may include another protected role’s name; a collision refuses with operation-role-collision at every fresh or resumed admission before any operation. Extra aliases of a single role remain accepted. Each graph pass rebuilds its protected set from the original declarations; discovered aliases never become additional declared roles in the canonical-identity comparison.

The manager-effects boundary excludes a unit with no relationship to any protected unit that reacts to billet’s filesystem changes: for example, a .path watching /var/lib/billet/server whose unrelated destination touches the registration directory. This has the same status as a cron job or an inotify daemon watching that directory. It is a stated limit, not a filesystem-watcher discovery feature. A watcher whose Unit= is a protected service does have a relationship and refuses.

Standard units are trusted as the operating system ships them. Their own relationships, drop-ins and .wants/ links are the operating system’s init graph and are outside this admission boundary: root additions under sysinit.target.wants/, including mutually wanted cyclic helpers, have the same status as a cron job or an unrelated watcher. Admission treats listed standard targets, slices, journald sockets and ancestor mounts with their derived device and fsck units as no-op leaves without traversing their graph. Every edge between a protected unit and a standard unit must still be on the edge list, and a standard unit that an operation would start must already be active.

The edge list is derived from billet’s shipped units and role templates and pinned systemd 255 sources: service_add_default_dependencies, timer_add_default_dependencies and timer trigger dependencies, mount_add_default_dependencies and mount_add_device_dependencies, the block-device ordering in unit.c, journald stdio dependencies and slices. The common unit work is unit_add_default_target_dependency and unit_add_{exec,slice,mount}_dependencies; v255 has no function named unit_add_default_dependencies. Standard endpoints end traversal. A delivered start requires an already-active unit; a delivered stop requires an already-inactive unit. Positively absent or masked dependencies cannot execute. Except for ancestor mounts and default slices, listed ordering edges and inverse edges that the operation does not propagate need no activity proof: stopping the controller with Conflicts=shutdown.target cannot start shutdown. An unlisted property refuses even if its standard endpoint is active. The ordering-only names include docker.service, systemd-networkd.service, systemd-remount-fs.service, systemd-tmpfiles-setup.service and the journald socket. The mount at the exact ledger path is protected and its implicit edges are checked, including the exact device and blockdev@ target derived from What=, and active ancestor mounts supplying that exact device path. Ancestor mounts supplying admitted RequiresMountsFor= paths, such as a separate /var on /dev/vdb1, are standard leaves accepted only while active; their own device, blockdev, fstab-passno systemd-fsck@ service, local-fs-pre.target, local-fs.target, systemd-remount-fs.service, root-mount and tmpfs swap.target dependencies are not traversed. A service cannot use this allowance to depend directly on arbitrary devices or unrelated mounts. CI’s real-template control fails loudly on every unlisted edge. The second measurement also pins the role backup’s Requires= and After= on the ledger (billet-backup.service.j2:27-28), including the mount’s RequiredBy= and Before= inverses. Instances of protected templates require and follow only their exact per-template slice: systemd 255’s unit_set_default_slice escapes the already escaped template prefix again and prepends system-; the instance name does not participate. That slice must be loaded, active and job-free. Relationship arrays and Names= from systemctl show quote names containing backslashes, so the shared property reader removes the display quoting once while preserving the unit-name escapes. This is required for both the ledger edges and the DNS slice edges; it also makes graph diagnostics query the actual standard unit instead of a quoted nonexistent name. Images-refresh and Ceph-health are outside retirement’s protected unit set and declare no relationships to its protected roles.

Billet’s own execution contexts still supply the five managed-directory lists and typed directory/command arrays; protected and effect paths share existing-prefix symlink resolution and are rebound before acting. Standard input must be null; output and error must be journal, inherit or null. File, append, truncate, terminal, socket and named-descriptor modes refuse, including for ExecStop and ExecStopPost. PID-file, PAM, log-namespace, namespace-path, root-image, bind and filesystem setup redirection also refuse. These checks apply to billet units even when an operator adds a drop-in. The existing node/server execution-shape and termination inspections remain authoritative for their programs; admission does not interpret arbitrary program semantics.

Retained inputs must survive the handoff: keep node configuration, identity, TLS certificates, keys, CA bundles and every other startup input on persistent storage, never under /run, /tmp or /var/tmp. The required set includes the configuration file itself, every path the node configuration resolves, and every loaded node EnvironmentFiles entry with ignore_errors=no; optional entries (ignore_errors=yes, the unit’s - prefix) are excluded from required-resource capture whether present or absent, but still pass the loaded-unit path rule below. The command reads org.freedesktop.systemd1.Service.EnvironmentFiles through the shared typed bus reader in internal/lifeops, requiring the a(sb) signature and a string/boolean pair for every entry. A proved empty array is supported; missing, malformed or unreadable property evidence refuses as could-not-tell, and mandatory files must be readable. Systemd 255 prints no text key for empty EnvironmentFiles, command arrays or directory-symlink arrays, even with --all; a missing text key is never evidence of an empty array. Bind, image and temporary-filesystem array printers do emit an empty key, but lifecycle inspection still obtains typed evidence before authorizing from their emptiness. Lifecycle hook checks do the same, including ExecReload. Required inputs also stay outside the identity directory being archived: retained-input-archived refuses lexical, resolved or intermediate traversal through that directory, including a parent symlink, before preparation and intent, at every admission and in stopped/archive proofs. The configuration’s own resource identity is captured and compared through archive and immediately before rewrite. The rewrite records its staged replacement inode before renaming it over the configuration, so crash reconciliation accepts that exact replacement and rejects a different inode even with identical bytes. Admission refuses with retained-input-volatile when an input’s lexical path, resolved path, or any traversed existing prefix component or followed symlink lies under one of those roots, comparing lexical and resolved forms (including /var/run resolving to /run). This rule runs at every admission and in the stopped and archive proofs, independently of current unit settings: it covers private temporary trees retained across PrivateTmp=yes → daemon-reload → PrivateTmp=no, links inside another unit’s runtime directory that lead to persistent TLS files, and credentials placed inside the node’s own runtime directory. The captured registration directory and lock records are explicitly disposable because the node recreates them on start; every other captured resource remains required. The node’s own stop may remove its disposable records, but another unit’s teardown may not. Runtime-directory checks protect traversed links as well as resolved targets. Private-tmp scanning and implicit /run/credentials/<canonical unit>/ overlap remain only for cross-unit disposable-record protection, including normal completion of an awaited backup. Existing private trees are scanned for either loaded PrivateTmp value; no cannot prove that a running invocation has no trees to remove. Unknown boot-id, directory or traversal evidence refuses.

The retained node's loaded unit has its own closed path rule. Every admission and stopped/archive proof rereads the complete loaded node property inventory with unfiltered systemctl show --all <unit> (no --property=), plus one typed busctl Properties.GetAll read across its interfaces. Path extraction covers the union of both property inventories, including properties omitted entirely from the show output, and uses typed values to preserve array entries and escaped pathnames. Systemd 255 matches property-filter names literally: * is not a wildcard, and adding --all does not remove a filter. Every absolute path in every property is checked, including conditions, assertions, namespace access lists, working/root directories, environment files, credentials and command arguments. Only FragmentPath, SourcePath, DropInPaths, ControlGroup and ControlGroupId are excluded. A lexical path, resolved path or traversed entry inside the archived identity directory refuses with retained-node-path-archived; optional -, negated ! and triggering | forms receive no exemption. Paths under /run, /tmp or /var/tmp refuse with retained-input-volatile, except the node's own disposable runtime records beneath its resolved /run/<RuntimeDirectory entry> prefixes. A sibling such as /run/billet/somewhere-else is outside that exception. The exact RequiresMountsFor=/var/tmp prerequisite of PrivateTmp=yes is manager setup and is admitted only in that property; descendants and paths in other properties still refuse, and archive traversal checks still apply. Relative directory properties are expanded under their system roots before checking. The real clean sequences log the graph and scanned paths per protected unit before each operation, including the original node command before substituting the test workload; the journald socket is observed as a standard unit and its Listen paths do not become node inputs. Required configuration inputs and mandatory environment files remain persistent even inside those prefixes. Shipped and role-rendered node directory settings are clean controls in the fake and real-systemd sequences.

An unfinished retirement binds the CLI --config operand and the loaded node ExecStart configuration operand to RetainedInvocation.ConfigPath before any mutation, comparing both the recorded lexical pathname and its captured resolved pathname. Moving both operands to another file with identical bytes still refuses with retained-config-path-changed. The configuration's leaf itself must not be a symlink: retained-config-symlink refuses before intent and names the target to configure directly. Persistent parent-directory symlinks outside the archived directory remain supported, including the authorized configuration rewrite and resume after its rename but before its phase write.

Historical handoff proofs govern only unfinished retirements. A done journal dispatches to the completed path without requiring RetainedInvocation, its captured resource identities, its recorded configuration pathname or the old archive-input boundary. Preparation protection is derived from the current admitted serverless configuration, and the existing done postconditions still govern. Older completed journals without invocation evidence, ordinary serverless configuration edits and atomic TLS certificate replacements remain supported.

Full effect admission precedes timer-stop metadata, stopped status, every phase journal write, archive rename and directory flush steps, and the configuration rewrite. The configuration rewrite repeats the complete remaining-sequence admission after its temporary-file flush before recording ConfigReplacement, and repeats it after that journal persistence before the configuration rename. A related watcher introduced at the first boundary must leave the journal unpublished, configuration unchanged and phase unchanged; the temporary is left for inspection when the next step refuses. The tail supplies current done protection through the receipt refresh hook after registration waiting. Completion-ledger preparation validates the existing archive and lock with state.WithExistingLocalState(), creating and chmodding nothing: the archive already contains the controller's directory and lock. Admission is repeated after locator reads and after ledger opening, before row completion; historical completion and acknowledgement do not require current node health.

Admission granularity. Admission cannot close a race against a concurrent root writer: however close an admission is to a mutation, a unit armed between them fires. What admission guarantees is that the host's unit configuration is judged at each distinct retirement step: before each service operation, and before each persistence step (a journal write, a status publication, the stage write, the archive rename, the configuration rewrite, the receipt publication, the completion-ledger preparation and row completion, the acknowledgement, marker clearing, settlement). Within a step, the syscalls a shared primitive performs (temporary files, chmod, sync, lock-file creation) are covered by the step's admission; a unit a root user arms concurrently, while a step runs, is outside the boundary and is detected by the next step's admission. A step never waits (registration wait, lock wait, network call, backup wait) between its admission and its first mutation: a wait ends a step, and admission runs again after it. Timer stop/disable helpers perform their absence/mask observation before final effects admission and submit the admitted command without another query. Other callers keep the existing helper behavior.

Quiet-service checks accept only a backup or updater’s own timer as TriggeredBy, excepted until its stop and required quiet before disable and at every stopped/filesystem proof. Every upholder and every unrelated path, socket, timer or automount trigger refuses, even when inactive. Before backup handling, an activating/start oneshot with a start job may reach the bounded wait, and a failed backup may reach the existing retirement-status reconciliation; only that retirement’s recognised status is cleared. This activity allowance never exempts the backup’s edges, including a backup starting after the driver’s fact snapshot. Stopped and archive proofs still require completed quiescence, no job and no process.

The production stop/disable helpers positively handle a not-found timer before submitting any command: require LoadState=not-found, ActiveState=inactive, empty UnitFileState, empty FragmentPath and no Job. StopAndProve reports Gone: Yes, How: "not-found"; Disable returns without an operation. A read failure, inconsistent absence or any submitted command's error refuses. Role convergence can remove timers, so server-only absence is supported rather than assumed impossible; the real control calls these helpers and never swallows an operation error.

A server declaring the node's runtime directory can remove its registration on stop (systemd 255, measured 2026-09-09 in the node template). Directory ownership overlap therefore refuses before controller stop even without a dependency edge. The original retained invocation, main PID, resolved resource paths and resource identities are durable journal evidence recorded before intent; a resume never derives the original from whichever node now happens to be active. Controller stop and the node handoff check termination policy against the refreshed per-unit snapshot of the final admission pass and accept only KillMode=control-group or mixed. The retained provider is recorded with required service invocations: Firecracker protects billet-network.service, DNS instances derived from the installed trusted and untrusted bridge names, and their state/configuration paths. A required network dependency may receive an idempotent start only while already active; any stop refuses. The shared stopped proof checks current quiet timers and controller enablement, zero controller main PID plus positive cgroup disappearance or empty cgroup.procs throughout its subtree under /sys/fs/cgroup, reconciled backup completion, and the retained invocation, runtime registration and resource identities before stopped status, again after status persistence before stopped journal, and under the authority exclusion immediately before archive. Both archive and archive crash reconciliation repeat the proof after flushes. Config replacement repeats full admission and the original-invocation proof after its temporary-file flush before replacement-inode journaling, and again after journaling immediately before rename. Never await backup under that exclusion, restore an archive, restart a collaterally stopped node, or manufacture missing historical evidence. Unfinished journals without the captured configuration path or original invocation evidence refuse the pre-handoff proof by name; an unmarked resource defaults to required input. Fake boundary witnesses introduce drift after successful operation admission; a later final-done refusal does not prove these boundaries.

internal/lifeops/operationmeasurement_test.go pins the measured quoted ledger edges and their inverses, exact per-template slice identity and active no-op state, and one-layer unit-list decoding; operationevidence_test.go and operationtimer_test.go cover the mask’s own pathname, the direct /dev/null form, wrong targets, regular files and unreadable evidence. The real-systemd witness is internal/lifeops/realoperation_test.go, opt-in with BILLET_TEST_SYSTEMD_OPERATION_EFFECTS=1 on a disposable root Linux host running systemd 255. It renders ledger-mount, server, node, backup, upgrade, network and DNS role templates with Python 3/Jinja2. Shipped and rendered controls perform all nine operations with remaining-sequence and immediate admission, retain DefaultDependencies, and assert final state and unchanged network invocations. Both sequences also have a stable node drop-in with ConditionPathExists= pointing inside the identity directory; it must refuse before the first retirement operation while the node remains in its original invocation. Server-only controls cover absent nodes and masked/absent timers. The ledger template uses an ext4 filesystem on a loop device backed by a file in the test’s temporary directory, detached during cleanup. Separate preventive and bypass fixtures cover an enabled controller restarted by an active multi-user completion anchor, implicit credential-directory removal, OnSuccess timer activation, node/DNS stop propagation, shared runtime directories, socket runtime-directory removal, direct archive path activation and stop-time truncation from a billet service drop-in. A non-billet helper related to the controller must refuse by name. The additional witnesses in internal/lifeops/realoperationalias_test.go cover a controller symlink alias to the node under /run/systemd/system, controller private-tmp cleanup and normal completion of an awaited backup removing a retained file from its private /var/tmp tree, both with and without a reload changing PrivateTmp to no, and runtime-directory links to persistent TLS files. internal/lifeops/operationinput_test.go covers the closed volatile-root rule and traversal evidence; cmd/billet/serverretireinput_test.go drives the full command with production input/record classification and injects volatile traversal at stopped and archive boundaries. cmd/billet/serverretireenvironment_test.go covers typed optionality, required environment-file capture, persistent/optional handoff controls and input drift after a stop wait. cmd/billet/serverretirearchiveinput_test.go covers direct and parent-symlink archive dependence, late traversal at each stopped boundary, configuration identity through archive/rewrite and exact-inode rewrite crash reconciliation. internal/lifeops/operationnodepath_test.go covers every property carrier, optional forms, typed-only conditions omitted entirely by show, repeated unfiltered complete reads, traversal and the actual packaged/role directory declarations. Both manager fakes match property filters literally, including when --all is also present; the command condition witness requires refusal before intent and an unfiltered node inventory read. cmd/billet/serverretirenodepath_test.go drives pre-intent drop-in refusals, late property drift at all stopped boundaries, both configuration operands moving on intent/stopped resume, leaf-symlink refusal, and parent-symlink rewrite/crash-resume controls. The archive/network-resource drift in serverretireadmission_test.go is injected through retireBeforeStoppedProof after archive admission succeeds. The stock fake control includes inactive condition-skipped systemd-firstboot.service beneath active sysinit.target; admission never traverses it. Tests name the mutation they target; only CI execution supplies measured results. Stopped-boundary drifts are injected after preceding operation admission and must prevent the corresponding publication or rename.

Release inspection shares the typed EnvironmentFiles decoder and compares paths and optionality across its process sample. Its remapping checks for BindPaths, BindReadOnlyPaths, MountImages, ExtensionImages and TemporaryFileSystem still reject non-empty text; omission is equivalent to empty for that decision. The policy-group CI step also compiles cmd/billet tests as the runner and runs TestRealSystemd command witnesses as root from their package directory, using BILLET_TEST_SYSTEMD_OPERATION_EFFECTS=1 and a temporary PostgreSQL ledger. TestRealSystemdRetirementWithoutEnvironmentFiles completes the command handoff with the shipped node's real typed environment-file property; its other manager evidence and operations use the request fixture. Enabling this witness without a PostgreSQL DSN fails rather than skips.

server retire --check-node-config is the read-only future-rendering boundary for a settled retained node. Its strict stdin binds schema, run, retiring host, transition, exact rendering bytes/digest and an exact operation document with filesystem, services and units arrays. Its separate node-config verdict binds the guard and both byte digests; no normal done answer is repurposed. It takes only existing trusted locks and inspects records, never uses takeTxLock preparation, repairs status, opens a ledger, creates a directory or stage, writes a receipt, repairs ownership or submits a service action. The shared controller-side postconditions remain strict; only this ordinary-entry observer permits quiet inactive/failed nodes with zero main/control/cgroup processes, no job, persistent enablement and loaded/installed agreement. Missing stopped registration does not require an active node. This grants no drain proof. Normal completion and publication still call the strict retained-node predicate.

server retire --check-settled-entry and --check-settled-closing are separate read-only modes with --config, --run, --retiring-host, --transition, --expected-holder equal to the run and --expected-guard. They accept no stdin document or competing mode. withRetireInspection takes only existing trusted transaction/global locks; observeRetireOrdinaryEntry shares trusted settled records, archived identity/provenance, closure status, absent marker/original identity location, current guard ownership, installed node identity, controller postconditions and current resource protection with node-config admission. A later authorized holder need not equal the historical journal owner. Both modes admit current installed paths and unit effects and bracket those observations with record/configuration checks; neither repairs, publishes, opens a ledger, creates directories, writes receipts or acts on services.

Entry uses observeRetireEntryNode: active shares the full strict done observer; inactive/failed requires explicit zero MainPID and ControlPID, positively empty Job, no contradictory cgroup process evidence, persistent enablement through the shared retained-enablement predicate, and supported loaded/installed execution. Quiet nodes need neither InvocationID nor runtime registration. Known non-quiet activity gets its own settled-entry-node-STATE refusal, positive main/control processes get settled-entry-node-process-present, queued jobs get settled-entry-node-queued-job, and missing/malformed jobs get settled-entry-node-job-observation. Unknown, absent, repeated or contradictory observations remain could-not-tell. Closing adds observeRetirePostconditions after path admission and must require active/current registration; never substitute entry permission or call the status-repairing tail. Frozen done publication, status republication, marker clearing and settlement keep their strict proofs and registration re-proofs.

Entry reads single raw property values: only exactly empty Job= proves no queued job, and whitespace-only or padded evidence is malformed rather than empty. State, enablement, reload, account, unit identity/source and cgroup evidence refuse padding; main/control PIDs require canonical unsigned decimal values, including an explicit 0 for quietness. Ordinary path/effects admission includes the existing read-only node-enable admission even with no proposed service operations, sharing installation admission's source/link checks without submitting an enable or reload. This proves a higher-priority installed fragment cannot silently replace the loaded source while NeedDaemonReload=no; a source disagreement gets settled-entry-node-unit-mismatch. The active-entry command witnesses restart into missing/stale registration and assert retained-node-registration-unproved, independently of closing's extra strict proof.

retirement.SettledVerdict carries schema 1, purpose settled-entry/settled-closing, outcome admitted/verified, invocation/classifier bindings, done/row/settlement/completion facts, observed node activity and state: nothing. It has no postconditions member and cannot parse as a normal tail answer. DecodeSettledVerdict keeps the purposes distinct; DecodeSettledRefusal admits only its refusal branch and matching exit semantics. Command behavioral witnesses use TestRetirement...; TestTheRetireSettledEntryFixturesAreTheCommandsOwn and TestTheRetireSettledClosingFixturesAreTheCommandsOwn deliberately belong to the t shard and print exact production bytes for missing/stale fixtures. CI supplies execution and mutation evidence. The two-real-main interrupted-resume witness and Ansible routing belong to commit 3; command admission alone does not prove that reachability.

Future paths are judged component by component, including a missing prefix followed by .., a protected intermediate resource, and an outside destination. Only positive absence starts hypothetical-parent reasoning; returning to an existing ancestor resumes real observations. Link reads are bounded and rechecked, and permission errors, inconsistent parents and exhausted bounds remain could-not-tell. Recursive operations cannot contain protected ancestors; the common state root is still usable for nonrecursive directory creation. Current trusted records and lifecycle definitions supply protection. Future credentials may be absent. The service's implicit deployment-lock default is environment-dependent and unexported by its owner; this interface requires explicit node.lock_dir rather than guessing from the inspector's environment. Proposed unit replacement initially admits exact installed bytes only; changed unit effects need supported evidence before admission. Generic writes to unit destinations require the proposed unit document.

The shared typed environment reader now supplies release inspection's environment_file_specs alongside its path-only compatibility field; both opening and closing reads preserve optionality, including for quiet units. Known [], absent-unit null and unknown remain distinct. The shared renderer supports zero files or one absolute literal path, excludes control/space/specifier/escape syntax and emits - only for ignore_errors=true. Required files must be positively readable and optional absence stays absent; no credential contents enter the verdict. A persistently masked retired server's account is checked from a trusted descriptor-bound retirement account record against current local names, numeric IDs and home, without demanding executable User/Group from the masked unit or inventing a server path for serverless configuration.

The new walker, node-config command, environment envelope/renderer, strict verdict, account and producer tests each name the removed guard their assertions target. No implementation-time execution establishes their mutation results; CI must do that. TestTheReleaseInspectFixturesAreTheCommandsOwn will report the added envelope bytes. TestTheRetireNodeConfigFixturesAreTheCommandsOwn produces active and quiet node-config answers without hand-authored healthy JSON. This pass changes no Ansible consumers or fixtures.

Where the tests are

• internal/lifeops/*_test.go: the systemctl fake answers only the properties it was asked for (a whole-reply fake makes deleting a property from the production query invisible); fixtures start healthy and break one thing; a healthy host is asserted not refused. os.SameFile type-asserts the concrete FileInfo, so a fixture varies link counts with a real hard link; syscall.Stat_t widths differ per platform, so stats go through a converting helper; OnFailureJobMode belongs in [Unit] or the fixture proves nothing.
• internal/lifeops/launchd/*_test.go and reallaunchd_test.go (each test derives its own label).
• cmd/billet/local_test.go, localup_test.go, localdown_test.go, lifecycle_test.go, drain_test.go, admission_test.go, hostlock_test.go, systemd_test.go; deploy/units_test.go (pins ExitTimeOut to the unit's 88200).
• scripts/test-systemd-lifecycle.sh, scripts/test-package-lifecycle.sh.

Related skills

billet-capacity (the drain and the barrier), billet-backup-restore (the other local commands), billet-providers-local (the Mac and the jailer that shape the units), billet-releases-and-upgrades (what the packages install and the host transaction).