Running in containers

‹ docs index

Containers are where most processkit-using services actually run, and it's the environment with the most sharp edges: which containment mechanism you get depends on privileges the orchestrator may or may not grant, your process is usually PID 1 (with everything that implies for signal delivery and reaping), and the image itself is often minimal (musl/Alpine, sometimes no shell at all). None of this is new machinery — every fact below is already documented in Platform support — this page is the container-shaped tour through it, with the Dockerfile fragments and gotchas that only show up once you actually run the crate inside docker run / Kubernetes.

Which containment mechanism you get

On Linux, ProcessGroup's cgroup backend needs write access to the cgroup v2 hierarchy at the real hierarchy root (see Platform support → containment mechanisms for exactly why). A plain, unprivileged docker run container gets neither: /sys/fs/cgroup is typically mounted read-only, so the crate quietly falls back to the ProcessGroup (POSIX process-group) mechanism — kill-on-drop still works, but the whole-tree accounting and limits capabilities of the cgroup backend do not (confirmed by running the crate inside docker run with no extra flags: mechanism() reports ProcessGroup, and /sys/fs/cgroup refuses even a touch). --privileged (or an equivalent cgroup-namespace delegation) makes the filesystem writable enough for mechanism() to report CgroupV2, but the container's cgroup is still a namespace root, not the real hierarchy root — /proc/self/cgroup reads 0::/ either way — so resource limits stay unenforceable even then.

use processkit::{Mechanism, ProcessGroup};

fn main() -> processkit::Result<()> {
    let group = ProcessGroup::new()?;
    if group.mechanism() == Mechanism::ProcessGroup {
        // The common case inside an unprivileged Linux container: kill-on-drop
        // still holds (see the next section), but `members()`'s CPU/memory
        // totals and `limits` are not available — see the capability
        // matrices in Platform support. `CgroupV2`/`JobObject` get those too.
    }
    Ok(())
}

Never assume a mechanism; check mechanism() if your service's behavior must not silently degrade (e.g. it relies on limits for sandboxing an untrusted child) — see Container resource limits for the fail-fast alternative when a cap truly matters. Whichever mechanism you land on, whole-tree kill-on-drop itself is unconditional — the fallback only narrows accounting and limits, never containment.

PID 1: signals, zombies, and what's contained

Inside a container your process is almost always PID 1 — Docker and Kubernetes don't run a real init unless you ask for one. PID 1 has two kernel duties an ordinary process doesn't: it's the implicit reparent target for every orphaned descendant in the container's PID namespace, and — for signals without an installed handler — some default dispositions are ignored instead of applied (irrelevant to SIGKILL/SIGSTOP, which are never blockable, but relevant to graceful termination — see the next section). Neither duty is something processkit does for you, and neither needs to be:

  • Containment (killing the tree) is unaffected. The whole point of the cgroup/pgroup mechanisms is that kill_all/shutdown/Drop reach every descendant, including ones spawned after your direct child — that's true whether or not your process happens to be PID 1.
  • Reaping an orphan that isn't yours is not. A grandchild your own child forked and then exited without waiting for still needs someone to call wait() on it once it dies, or it lingers as a zombie (kill(pid, 0) still reports it alive) forever. Once that grandchild reparents to PID 1, "someone" has to be PID 1 itself — and an ordinary process, processkit-managed or not, doesn't indiscriminately reap processes it never spawned.

This is exactly the gap Platform support's CI section documents and works around with --init for the crate's own test suite — and it reproduces identically for any container. Run a process that orphans a short-lived grandchild as the container's PID 1 with a plain docker run (no --init), and the grandchild is left a permanent zombie after it exits; the identical container run with --init (which runs tini as the real PID 1, a subreaper) shows no zombie at all — confirmed by running both side by side. Baking tini into the image itself works identically, and is the only option on Kubernetes, which has no --init-equivalent flag:

FROM alpine:latest
RUN apk add --no-cache tini
COPY my-app /usr/local/bin/my-app
ENTRYPOINT ["/sbin/tini", "--", "/usr/local/bin/my-app"]

(Confirmed against a real pk-container-demo image built from this exact pattern: with tini as ENTRYPOINT, an orphaned grandchild is reaped with no --init flag at all; the same image invoked without tini — the bare binary as PID 1 — leaks the identical zombie that the plain docker run case above does.) sh -c 'sleep 0.3 &'-style orphans are the illustrative worst case; in practice this only bites processes with descendants that outlive their direct parent — most single-service containers with no forking children never hit it. When in doubt, run with a subreaper as PID 1; it's a no-op cost otherwise.

Graceful shutdown on the orchestrator's SIGTERM

Docker (docker stop) and Kubernetes both stop a container by sending SIGTERM to PID 1 — your process — then wait a grace period (Docker's --stop-timeout / docker stop -t, Kubernetes' terminationGracePeriodSeconds, both default to a small handful of seconds) before escalating to SIGKILL. That SIGTERM targets your process, not the tree processkit manages — the two are related but distinct signals: catching the orchestrator's SIGTERM is ordinary application code (e.g. tokio::signal::unix::signal with tokio's own signal feature, or a crate like signal-hook); reacting to it by tearing down the child tree gracefully is ProcessGroup::shutdown (or RunningProcess::shutdown for a single start()ed service): SIGTERM the tree, wait up to shutdown_timeout, then SIGKILL any survivor.

use processkit::{Command, ProcessGroup};

#[tokio::main]
async fn main() -> processkit::Result<()> {
    let group = ProcessGroup::new()?;
    let _server = group.start(&Command::new("app-server")).await?;

    // Left to the application: install a handler for the orchestrator's own
    // SIGTERM (tokio::signal::unix::signal(SignalKind::terminate()) with
    // tokio's `signal` feature, or the `signal-hook`/`ctrlc` crates) and
    // resolve this future when it fires.
    wait_for_orchestrator_sigterm().await;

    // SIGTERM the tree, wait shutdown_timeout, SIGKILL stragglers:
    group.shutdown().await?;
    Ok(())
}

async fn wait_for_orchestrator_sigterm() {
    // …
}

(This exact pattern — tokio::signal::unix::signal(SignalKind::terminate()) followed by group.shutdown() — was built and run for real as a small processkit-consuming binary: docker stop on the resulting container delivered SIGTERM to PID 1, the handler fired, shutdown() tore the tree down, and the process exited well inside Docker's default 10-second grace — no SIGKILL needed.)

Two things worth setting deliberately, both already documented at the ProcessGroup/Command level:

  • The orchestrator's grace period must exceed your own. If shutdown_timeout (or a per-Command timeout_grace) is longer than Docker's --stop-timeout / Kubernetes' terminationGracePeriodSeconds, the orchestrator sends the hard SIGKILL to your still-shutting-down process before your own escalation ever gets a chance to run — set the outer grace at least as generous as the inner one.
  • A frozen tree can't shut down gracefully. If you've called group.suspend(), the frozen processes can't run their SIGTERM handler — shutdown() waits out the whole grace and then hard-kills. Resume first; see Platform support's frozen-tree caveat.

Minimal images: musl/Alpine, no shell, no setpriv

Two properties of a lean final image turn out to be non-issues for this crate specifically, because of how it does privilege drop and spawning — confirmed by building and running against rust:alpine/alpine:latest:

  • No setpriv/su-exec/gosu needed for uid()/gid()/groups(). The privilege drop is setgroups → setgid → setuid called directly as raw syscalls inside the child's pre_exec hook (see Running commands → privileges) — the crate never shells out to an external helper binary, so a final stage that lacks one entirely still drops privileges correctly. Confirmed by running the drop (.uid(...).gid(...).groups(...)) inside a container and getting back the target identity with no such binary invoked.
  • No shell needed to spawn. Command/ProcessGroup::spawn always execs the target program directly with an argv array — there is no sh -c step anywhere in the crate's own spawn path (pipelines wire pipes at the OS level, not through a shell either — see Pipelines). A FROM scratch-style final stage with no shell at all works, as long as the programs you spawn are themselves present — the crate not needing a shell doesn't exempt a command you invoke as sh -c "…" yourself.
# Builder: musl/Alpine — the same base the crate's own CI `test-musl` job and
# `just test-musl` build against (see Platform support → CI coverage), so the
# toolchain and libc pairing is already exercised.
FROM rust:alpine AS builder
WORKDIR /src
COPY Cargo.toml Cargo.lock ./
COPY src ./src
RUN cargo build --release

# Runtime: no shell, no setpriv/su-exec/gosu — the crate needs none of them.
FROM alpine:latest
COPY --from=builder /src/target/release/my-app /usr/local/bin/my-app
ENTRYPOINT ["/usr/local/bin/my-app"]

(Built and run for real, end to end: cargo build --release inside rust:alpine, then the resulting binary run from a bare alpine:latest final stage — privilege drop and a no-shell spawn both succeeded exactly as above.) Add tini to the final stage — see PID 1 — if anything you spawn can outlive its own children.

Container resource limits vs the crate's limits

The orchestrator's own cgroup limits (Docker's --memory/--cpus, Kubernetes' resources.limits) apply to the whole container, enforced by the kernel regardless of anything processkit does — a container that hits its memory limit gets OOM-killed independent of any max_memory the crate was asked to set. That outer limit is not the same thing as the crate's own limits feature (max_memory / max_processes / cpu_quota on ProcessGroupOptions), which caps a specific tree within the container and needs this process to sit at the real cgroup v2 hierarchy root — a requirement an ordinary container essentially never meets, privileged or not (see Which containment mechanism you get). Confirmed by actually requesting a limit from inside a container:

plain docker run:       Error::ResourceLimit { kind: Memory, reason: Unenforceable,
                           detail: "…Read-only file system…" }
docker run --privileged: Error::ResourceLimit { kind: Memory, reason: Unenforceable,
                           detail: "…cgroup v2's 'no internal processes' rule…Resource busy…" }

Both fail the same way the crate documents for any non-delegated host — Error::ResourceLimit with reason: LimitReason::Unenforceable — never a silently-unbounded group (see Errors → ResourceLimit and Platform support → containment mechanisms for the delegation requirement in full). In practice: rely on the orchestrator's own memory/CPU limits as the outer boundary for anything running in a container, and reserve the crate's limits feature for hosts where this process genuinely owns the cgroup root — a minimal, non-systemd init on bare metal or a VM, not a container. mechanism()/kill-on-drop containment keep working either way; only the limits cap itself is unavailable.

Next: Platform support · Process groups · docs index