
Async-and-sync child-process management for Python with a kernel-backed no-orphan guarantee: every process you start — and everything it spawns — lives in a kill-on-exit container (a Windows Job Object, a Linux cgroup v2, or a POSIX process group). Subject to the documented POSIX escape caveats, normal completion, errors, timeouts, cancellation, and context-manager exit reap descendants as a unit. Abrupt owner-death coverage is platform-specific and reported explicitly.
Beyond spawning a subprocess: run-and-capture, line streaming, interactive stdin, shell-free pipelines, readiness probes, timeouts & cancellation, supervision with restart/backoff, resource-limited sandboxes, and a mockable runner seam for subprocess-free tests — each in a synchronous and an asyncio-native form.
from processkit import Command
# Require success and get trimmed stdout; a failure raises a typed exception.
version = Command("python", ["--version"]).run()
print(version)
Why processkit?
subprocess and asyncio.subprocess reach (at most) the direct child. The
processes it spawned — a build tool's compiler children, the real payload
behind a wrapper (cmd /c …, sh -c …), a test's helper servers — survive a
timeout, an exception, or a cancelled task, and keep running as orphans.
processkit spawns every child into the operating system's own containment
primitive — a Job Object on Windows, a cgroup v2 on Linux (with a
process-group fallback), a POSIX process group on macOS/BSD — so teardown is
a kernel operation over the whole tree, not a best-effort signal to one pid:
- Nothing escapes silently. Exiting a
with/async withblock reaps every descendant, grandchildren included. Where a mechanism has a genuine weakness (asetsidchild can escape a POSIX process group),ProcessGroup.mechanismreports the active backend instead of pretending — never a silent downgrade. - Sync and async, first-class. The run-&-capture verbs, pipelines, and
supervision each exist as a plain synchronous call and an
a-prefixed asyncio coroutine, sharing one set of types. The inherently-streaming surfaces — live line streaming, interactive stdin, readiness probes — are asyncio-native (awaited on a started process), not duplicated as blocking calls. - Honest results. A non-zero exit is data (
ProcessResult) until you ask for success; a timeout is captured in the result; a cancellation is always an error; every platform divergence raisesUnsupportedor is documented. Raised exceptions carry structured fields and alias the stdlib's (Timeoutis aTimeoutError,ProcessNotFoundaFileNotFoundError,PermissionDeniedaPermissionError). - Testable. One runner seam swaps the real spawner for scripted doubles or record/replay cassettes — no subprocess in your tests.
How it compares
| whole-tree kill-on-exit | async | sync | limits / stats | streaming · pipelines · supervision | |
|---|---|---|---|---|---|
subprocess | — | — | ✓ | — | — |
asyncio.subprocess | — | ✓ | — | — | — |
processkit | ✓ | ✓ | ✓ | ✓ | ✓ |
The first column is the differentiator: a child's descendants are contained and reaped as a unit (Job Object / cgroup v2 / process group), not just the direct child.
Stable API. The public API has been stable since 1.0 and follows Semantic Versioning: breaking changes land only in a new major version, so
1.xupgrades are backward-compatible. See CHANGELOG.md, and ROADMAP.md for how it was built.
Guides
New here? Start with the Cookbook — short task-to-snippet
recipes for everything the package does — then read Running commands
end to end (it's the vocabulary every other guide builds on). Coming from the
standard library? Coming from subprocess maps your existing
subprocess / asyncio.subprocess patterns onto their processkit equivalents.
Reach for the rest as the need arises, and keep Platform support
handy before you ship: it collects every per-OS caveat in one place.
| Guide | Covers |
|---|---|
| Cookbook | "I want to …" → working snippet, for every capability; the fastest way in |
| Coming from subprocess | Side-by-side translation of subprocess / asyncio.subprocess patterns, the exception mapping, and the whole-tree containment the stdlib can't give |
| Running commands | The Command builder end to end — args, env/sandboxing, stdin, stdout/stderr redirection, encodings, output caps, timeouts, privileges — and every consuming verb (output, run, probe, …) with its error semantics |
| Process groups | Kill-on-drop containment: creating groups, spawning, teardown, whole-tree signals, suspend/resume, member listing, resource limits, stats |
| Sandboxing untrusted tools | The agent/LLM-tool recipe: locked-down env → bounded output → group resource limits → timeout → teardown, a checklist, and an honest threat model (what this does and does not protect against) |
| Streaming & interactive I/O | astart() and the live RunningProcess: line streaming, interactive stdin, readiness probes (wait_for_line / wait_for_port / wait_until), per-run profiling |
| Pipelines | Shell-free command pipelines — chain with .pipe() or the pipe operator: wiring, pipefail attribution, chain timeouts, binary tails |
| Timeouts & cancellation | How a deadline is captured vs when it raises, interrupting a blocked sync call (Ctrl+C), and asyncio cancellation that reaps the whole tree |
| Supervision | Keeping a child alive: restart policies, backoff & jitter, stop conditions, outcomes |
| Testing your code | The ProcessRunner seam — ScriptedRunner (incl. scripted streaming start()), record/replay cassettes, the RecordingRunner spy, the CliClient wrapper, and the autoloaded pytest plugin (ready-made fixtures + a no-real-spawn guard) |
| Command-line usage | python -m processkit run -- ...: containment and resource limits for a shell command with no Python to write, flags, exit codes |
| Performance & overhead | Why the workload is syscall-bound, what each benchmark in benchmarks/ measures, how to reproduce them locally, and qualitative throughput/scaling expectations |
| Async runtimes & event loops | Which event loops the asyncio-native surface runs on — asyncio and uvloop (yes), anyio-on-asyncio (yes), native trio / anyio-on-trio / curio (no) — and why |
| Platform support | The containment mechanisms, every per-feature support matrix in one place, and the caveats worth knowing before you ship |
| Troubleshooting | A symptom-to-guide map for resource-limit, signaling, event-loop, cassette, privilege-drop, and teardown errors |
Packaging
Unlike the Rust crate's compile-time feature flags, the Python wheel ships one surface with everything enabled — resource limits, signals/stats, record/replay, and opt-in logging are all present in every published wheel. There is nothing to opt into at install time:
pip install processkit-py # import name: processkit
On a platform without a prebuilt wheel, build from source (uv run maturin develop) — see the
README.
Distributed as abi3 wheels for CPython 3.10+ (one wheel per OS/arch runs on every supported minor version, 3.14 included), plus a version-specific free-threaded wheel for CPython 3.14t (PEP 703). See Platform support for the wheel matrix and the free-threaded note.
The 60-second tour
import asyncio
from processkit import Command, ProcessGroup
# One-shot, sync: capture everything. A non-zero exit is data, not an exception.
head = Command("git", ["rev-parse", "HEAD"]).output()
print("HEAD =", head.stdout.strip(), head.code)
# Success-checking: a non-zero exit / timeout / signal-kill becomes a typed exception.
version = Command("python", ["--version"]).run()
async def main():
# Asyncio: the same verbs with an `a` prefix; cancelling reaps the whole tree.
result = await Command("git", ["status", "--short"]).aoutput()
# Stream a child's stdout; the context manager reaps the tree on exit.
async with await Command("my-build", ["--watch"]).astart() as proc:
async for line in proc.stdout_lines():
print(line)
# Containment: anything started in the group dies with it (grandchildren too).
async with ProcessGroup() as group:
await group.astart(Command("dev-server"))
# async-with exit reaps the whole tree
asyncio.run(main())
API reference
The API reference is the complete, per-symbol index of the
public surface — every class, function, protocol, type alias, and exception,
plus the processkit.testing submodule. It is generated straight from the type
stubs and docstrings (the same source your IDE and mypy read), so it never
drifts from the real API. These guides are the narrative layer on top — they
explain how the pieces compose, with the platform fine print collected in
Platform support. The underlying algorithms (the OS containment
mechanisms, race-free spawn) live in the
processkit Rust crate.
For LLM assistants
This site publishes two machine-readable files following the llmstxt.org convention, so an AI assistant can pull an accurate, current slice of the docs instead of guessing from a stale summary:
- llms.txt — an annotated index of every guide, in reading order, each linked with a one-line description drawn from the guide itself.
- llms-full.txt — the full text of all guides concatenated, for loading the whole documentation set into an assistant's context at once.
Both are generated straight from these guides (scripts/gen_llms_txt.py, drift-guarded
like the API reference) and served as static assets of this book.