Testing your code

Previous: Overview

Code that shells out is miserable to test — unless the subprocess sits behind a seam. In ProcessKit that seam is one small interface, IProcessRunner. It is both the dependency-injection point (production code depends on the interface, not on a concrete spawner) and the test seam (a test hands the same code a subprocess-free double). The default real implementation is JobRunner — each run lands in a fresh kill-on-dispose group; everything in this guide swaps it for a double so your tests never touch the operating system.

The subprocess-free doubles ship in a separate ProcessKit.Testing NuGet package, kept out of the runtime ProcessKit package so its on-disk/JSON record-replay surface never enters your production dependency graph. Add a ProcessKit.Testing package reference to your test project; the types stay in the ProcessKit.Testing namespace.

F#

// One interface, three primitives — each takes a CancellationToken:
type IProcessRunner =
    abstract CaptureStringAsync: Command * System.Threading.CancellationToken -> System.Threading.Tasks.Task<Result<ProcessResult<string>, ProcessError>>
    abstract CaptureBytesAsync: Command * System.Threading.CancellationToken -> System.Threading.Tasks.Task<Result<ProcessResult<byte[]>, ProcessError>>
    abstract SpawnAsync: Command * System.Threading.CancellationToken -> System.Threading.Tasks.Task<Result<RunningProcess, ProcessError>>

C#

// One interface, three primitives — each takes a CancellationToken:
public interface IProcessRunner
{
    Task<FSharpResult<ProcessResult<string>, ProcessError>> CaptureStringAsync(Command command, CancellationToken cancellationToken);
    Task<FSharpResult<ProcessResult<byte[]>, ProcessError>> CaptureBytesAsync(Command command, CancellationToken cancellationToken);
    Task<FSharpResult<RunningProcess, ProcessError>> SpawnAsync(Command command, CancellationToken cancellationToken);
}

Unlike some interfaces, none of the three is defaulted: a hand-rolled IProcessRunner implements all three (the doubles that ship with ProcessKit do this for you). CaptureStringAsync and CaptureBytesAsync are the bulk primitives; SpawnAsync returns a live handle for streaming and probes. These are deliberately named apart from the consuming verbs (OutputStringAsync/RunAsync/StartAsync/…): the verbs layer on top of the primitives — applying the command's Retry policy and the success/parse semantics — so you implement only the three primitives and get the whole verb vocabulary for free. (Calling a verb routes through these primitives; for a single raw capture with no retry, call CaptureStringAsync directly.)

• The IProcessRunner seam
• Running the test suite
• Scripting replies
• Verifying calls
• Custom doubles and mocking frameworks
• What the doubles don't cover
• Record and replay
• CliClient
• Dependency injection

Running the test suite

An ordinary dotnet test ProcessKit.slnx runs the regular F# and C# suites on both target frameworks. The long-running, concurrency-sensitive Stress and Interleaving fixtures are NUnit Explicit, so that command skips them by default. Select a category to opt in:

dotnet test ProcessKit.slnx --filter "Category=Stress"
dotnet test ProcessKit.slnx --filter "Category=Interleaving"

The main CI test job and scripts/verify-all.ps1 retain the same explicit Category!=Stress&Category!=Interleaving filter. Weekly and manually dispatched CI jobs select and run each opt-in category.

The IProcessRunner seam

Write production code against IProcessRunner and let the caller supply the runner. In production that runner is a JobRunner (or a ProcessGroup, which is itself an IProcessRunner, so every run lands in one shared kill-on-dispose group); in a test it is a double.

You rarely call the interface's three methods directly — the Runner module gives every runner the full verb vocabulary, each verb taking (runner, cancellationToken, command):

Everything in the first eight rows reaches a child only through CaptureStringAsync (or CaptureBytesAsync), so it runs hermetically against the subprocess-free doubles below. The last two — firstLine and start — need a live handle and go through SpawnAsync; both ScriptedRunner and RecordReplayRunner serve it (a RecordReplayRunner reconstructs a live handle from the recording), so streaming and readiness code replays too — see what the doubles don't cover.

A custom runner that deliberately supports capture but cannot expose a live handle may throw NotSupportedException synchronously from SpawnAsync. Supervisor treats that specific exception as the capture-only capability marker and falls back to CaptureStringAsync; any other exception is allowed to surface as a runner defect instead of silently disabling live status, graceful stop, and liveness monitoring.

Production code, generic over the runner:

F#

/// HEAD's commit id, run through whatever runner the caller injects.
let head (runner: IProcessRunner) (ct: CancellationToken) =
    Runner.run runner ct (Command.create "git" |> Command.args [ "rev-parse"; "HEAD" ])

C#

/// HEAD's commit id, run through whatever runner the caller injects.
Task<FSharpResult<string, ProcessError>> Head(IProcessRunner runner, CancellationToken ct) =>
    runner.RunAsync(new Command("git").Args(["rev-parse", "HEAD"]), ct);

In production you pass JobRunner(); in a test you pass a double and no process spawns. The retry policy (Command.retry) is applied by the Runner verbs, so a double exercises your retry handling without a subprocess too.

Time-dependent retry, readiness, PtySession pattern-wait, and supervision tests can additionally set Command.TimeProvider(provider) (or Command.timeProvider provider). It defaults to TimeProvider.System; a deterministic provider lets the test advance delays and deadlines without waiting or changing process-wide time. The provider belongs on the command, so a Supervisor and its liveness probes inherit the same clock automatically.

Scripting replies

ScriptedRunner (in the ProcessKit.Testing namespace) is the work-horse double: it returns canned Replys for matched commands. It is immutable and fluent — On / When add rules and Fallback sets a catch-all, each returning a new runner:

F#

let runner =
    (ScriptedRunner())
        // Match when every listed token appears among the command's program and
        // arguments (order-independent):
        .On([ "git"; "rev-parse"; "HEAD" ], Reply.Ok "abc123\n")
        // …or by any predicate over the whole Command:
        .When((fun cmd -> cmd.WorkingDirectory.IsSome), Reply.Fail(128, "fatal: not a git repository"))
        // …with an optional catch-all:
        .Fallback(Reply.Ok "")

C#

var runner = new ScriptedRunner()
    // Match when every listed token appears among the command's program and
    // arguments (order-independent):
    .On(["git", "rev-parse", "HEAD"], Reply.Ok("abc123\n"))
    // …or by any predicate over the whole Command:
    .When(cmd => cmd.WorkingDirectory is not null, Reply.Fail(128, "fatal: not a git repository"))
    // …with an optional catch-all:
    .Fallback(Reply.Ok(""));

The pieces:

• Reply.Ok stdout — exit 0 with that stdout. Reply.Fail(code, stderr) — a non-zero exit with that stderr. Reply.Exit code — an explicit exit code with empty stdout/stderr. Reply.Signalled n — terminated by signal n (Reply.Signalled () when the number is unavailable).
• .WithStdout text / .WithStderr text refine any reply — e.g. Reply.Fail(1, "merge failed").WithStdout("CONFLICT in app.fs") to model a tool that writes to both streams.
• Rule order matters: first match wins. On([ "git"; "rev-parse"; "HEAD" ]) matches any command whose program plus arguments contain all three tokens — it is a subset test, not a positional prefix.
• No matching rule and no Fallback throws — a missing stub fails the test loudly rather than silently returning a default, so an unexpected invocation can't slip through.
• A scripted reply respects the command's OkCodes: the ProcessResult it produces carries the command's accepted-codes, so IsSuccess and the success-requiring verbs honour them.

A test (any framework — the doubles depend on none; this repo's fixtures are NUnit):

F#

[<TestFixture>]
type GitTests() =

    [<Test>]
    member _.``head returns the trimmed sha``() =
        task {
            let runner =
                (ScriptedRunner())
                    .On([ "git"; "rev-parse"; "HEAD" ], Reply.Ok "abc123\n")

            match! head runner CancellationToken.None with
            | Ok sha -> Assert.That(sha, Is.EqualTo "abc123")
            | Error err -> Assert.Fail err.Message
        }

C#

[TestFixture]
public class GitTests
{
    [Test]
    public async Task Head_returns_the_trimmed_sha()
    {
        var runner = new ScriptedRunner()
            .On(["git", "rev-parse", "HEAD"], Reply.Ok("abc123\n"));

        switch (await Head(runner, CancellationToken.None))
        {
            case { IsOk: true, ResultValue: var sha }:  Assert.That(sha, Is.EqualTo("abc123")); break;
            case { IsOk: false, ErrorValue: var err }: Assert.Fail(err.Message); break;
        }
    }
}

A Reply.Fail behaves differently depending on the verb that consumes it — the same honest-result rule as a real run. Through Runner.run / Runner.runUnit (success required) a non-zero exit becomes Error(ProcessError.Exit …); through Runner.outputString it stays Ok with result.IsSuccess = false and the code in result.Code:

F#

task {
    let runner = (ScriptedRunner()).Fallback(Reply.Fail(2, "boom"))
    let grep = Command.create "grep" |> Command.args [ "needle"; "file" ]

    // Success-requiring verb: the non-zero exit surfaces as an error.
    match! Runner.run runner CancellationToken.None grep with
    | Error(ProcessError.Exit(program, code, _, stderr)) -> () // program="grep", code=2, stderr="boom"
    | _ -> ()

    // Honest-result verb: the non-zero exit is data.
    match! Runner.outputString runner CancellationToken.None grep with
    | Ok result -> Assert.That(result.IsSuccess, Is.False)
    | Error err -> Assert.Fail err.Message
}

C#

var runner = new ScriptedRunner().Fallback(Reply.Fail(2, "boom"));
var grep = new Command("grep").Args(["needle", "file"]);

// Success-requiring verb: the non-zero exit surfaces as an error.
if (await runner.RunAsync(grep) is { IsOk: false, ErrorValue: { IsExit: true } }) { } // program="grep", code=2, stderr="boom"

// Honest-result verb: the non-zero exit is data.
switch (await runner.OutputStringAsync(grep))
{
    case { IsOk: true, ResultValue: var output }: Assert.That(output.IsSuccess, Is.False); break;
    case { IsOk: false, ErrorValue: var err }:   Assert.Fail(err.Message); break;
}

Verifying calls

Scripting replies covers the stub side — canned answers so the code under test runs. The mock side is the mirror image: after the run, assert what was called. ScriptedRunner records every command routed through it in Received, a structural, secret-free journal — so a test like "the deploy code ran git commit exactly once" needs no bespoke counting decorator:

F#

task {
    let runner = (ScriptedRunner()).Fallback(Reply.Ok "")

    do! deploy (runner :> IProcessRunner) // the code under test, run through the double

    // Verify by predicate — Matches mirrors On's token semantics (program + args, order-independent):
    Assert.That(runner.CountReceived(fun inv -> inv.Matches [ "git"; "commit" ]), Is.EqualTo 1)

    // …or inspect the journal directly:
    let last = runner.Received |> Seq.last
    Assert.That(last.Program, Is.EqualTo "git")
    Assert.That(last.Verb, Is.EqualTo RunnerVerb.CaptureString)
}

C#

var runner = new ScriptedRunner().Fallback(Reply.Ok(""));

await Deploy(runner); // the code under test

Assert.That(runner.CountReceived(inv => inv.Matches(["git", "commit"])), Is.EqualTo(1));

var last = runner.Received[^1];
Assert.That(last.Program, Is.EqualTo("git"));
Assert.That(last.Verb, Is.EqualTo(RunnerVerb.CaptureString));

Each RecordedInvocation carries the shape of one call: Program, Args, Cwd, EnvNames, HasStdin, Pty, and Verb — which of the three primitives (RunnerVerb.CaptureString / CaptureBytes / Spawn) served it, so you can assert a call streamed (Spawn) rather than buffered.

• CountReceived(predicate) counts matches — the ergonomic "exactly once" check. RecordedInvocation.Matches(tokens) is the same subset test On uses, so a call scripted with On([ "git"; "commit" ], …) verifies with Matches([ "git"; "commit" ]).
• The journal is per instance. On / When / Fallback are fluent and return a new runner with its own empty journal, so record and assert against the same instance you handed the code under test.
• The secret invariant holds — the one the cassettes keep, without the exception a cassette carries (a recorded NotFound's searched PATH; nothing like it is journalled here). Only environment-variable names are recorded (never their values), and only whether stdin was present (never its content). A secret passed as Command.Env("TOKEN", secret) or fed to stdin never lands in Received, so a dumped journal is safe to log or attach to a failing test.

Custom doubles and mocking frameworks

IProcessRunner is a plain interface, so any .NET mocking framework (Moq, NSubstitute, FakeItEasy) can stand in for it — handy when the interaction is what you want to assert (was StartAsync called? with which command?) or when you want to return specific Error outcomes. The error cases are easy: every ProcessError case has a public constructor, so a double can return Error(ProcessError.NotFound("git", None)), Error(ProcessError.Io "..."), and so on directly.

Returning a ProcessResult is almost as easy. Its constructor is internal, but the ProcessResult test factories build one directly: ProcessResult.Success(stdout) for a clean exit, ProcessResult.Failure(stdout, stderr, exitCode) for a non-zero exit, and ProcessResult.Create(stdout, stderr, outcome, duration) for full control over the Outcome (e.g. Outcome.TimedOut). The captured-stdout type is inferred — C# writes ProcessResult.Success("out"), F# writes ProcessResult.Success "out" — and the result behaves like a real one (IsSuccess, EnsureSuccess, Code, …), so they double as fixtures for any code that consumes a ProcessResult:

F#

let fixedSha: IProcessRunner =
    { new IProcessRunner with
        member _.CaptureStringAsync(_, _) = task { return Ok(ProcessResult.Success "abc123\n") }
        member _.CaptureBytesAsync(_, _) = task { return Ok(ProcessResult.Success [| 1uy; 2uy |]) }
        member _.SpawnAsync(_, _) =
            task { return Error(ProcessError.Unsupported "no streaming in this double") } }

C#

public sealed class FixedSha : IProcessRunner
{
    public Task<FSharpResult<ProcessResult<string>, ProcessError>> CaptureStringAsync(Command command, CancellationToken ct) =>
        Task.FromResult(FSharpResult<ProcessResult<string>, ProcessError>.NewOk(ProcessResult.Success("abc123\n")));

    public Task<FSharpResult<ProcessResult<byte[]>, ProcessError>> CaptureBytesAsync(Command command, CancellationToken ct) =>
        Task.FromResult(FSharpResult<ProcessResult<byte[]>, ProcessError>.NewOk(ProcessResult.Success(new byte[] { 1, 2 })));

    public Task<FSharpResult<RunningProcess, ProcessError>> SpawnAsync(Command command, CancellationToken ct) =>
        Task.FromResult(FSharpResult<RunningProcess, ProcessError>.NewError(ProcessError.NewUnsupported("no streaming in this double")));
}

For canned successes wired through a matcher, ScriptedRunner is still the most convenient seam (it builds the result for you). Doubles can also delegate their success path to an inner runner. A custom IProcessRunner written as an object expression — implementing all three methods — composes cleanly. This one injects a single transient failure before delegating, so you can test that retry handling actually retries:

F#

let failOnce (inner: IProcessRunner) : IProcessRunner =
    let mutable calls = 0

    { new IProcessRunner with
        member _.CaptureStringAsync(command, ct) =
            task {
                calls <- calls + 1

                if calls = 1 then
                    return Error(ProcessError.Io "transient blip") // ProcessError.isTransient -> true
                else
                    return! inner.CaptureStringAsync(command, ct)
            }

        member _.CaptureBytesAsync(command, ct) = inner.CaptureBytesAsync(command, ct)
        member _.SpawnAsync(command, ct) = inner.SpawnAsync(command, ct) }

C#

public sealed class FailOnce(IProcessRunner inner) : IProcessRunner
{
    private int calls;

    public Task<FSharpResult<ProcessResult<string>, ProcessError>> CaptureStringAsync(Command command, CancellationToken ct) =>
        ++calls == 1
            ? Task.FromResult(
                FSharpResult<ProcessResult<string>, ProcessError>.NewError(ProcessError.NewIo("transient blip"))) // ProcessError.isTransient -> true
            : inner.CaptureStringAsync(command, ct);

    public Task<FSharpResult<ProcessResult<byte[]>, ProcessError>> CaptureBytesAsync(Command command, CancellationToken ct) =>
        inner.CaptureBytesAsync(command, ct);

    public Task<FSharpResult<RunningProcess, ProcessError>> SpawnAsync(Command command, CancellationToken ct) =>
        inner.SpawnAsync(command, ct);
}

Wrap a ScriptedRunner with it and drive a retrying verb to prove the retry fires — because retry lives in the verb layer over the CaptureStringAsync primitive, failOnce's single transient error is retried away. If a double is bulk-only and you want spawning to be a hard error, return Error(ProcessError.Unsupported "no streaming in this double") from SpawnAsync.

Deterministic fault injection

FaultInjectingRunner packages that decorator pattern for resilience tests. It wraps any IProcessRunner and intercepts the three primitive invocations before delegating. The common "fail N times, then succeed" retry test is direct:

F#

let inner: IProcessRunner = (ScriptedRunner()).Fallback(Reply.Ok "recovered")

let runner: IProcessRunner =
    FaultInjectingRunner(
        inner,
        2,
        FaultInjection.Error(ProcessError.Io "transient network failure")
    )

let command =
    (Command.create "tool").Retry(2, TimeSpan.Zero, Func<ProcessError, bool>(fun error -> error.IsTransient))
let recovered = runner.OutputStringAsync(command)

C#

IProcessRunner inner = new ScriptedRunner().Fallback(Reply.Ok("recovered"));
IProcessRunner runner = new FaultInjectingRunner(
    inner,
    2,
    FaultInjection.Error(ProcessError.NewIo("transient network failure")));

var recovered = runner.OutputStringAsync(
    new Command("tool").Retry(2, TimeSpan.Zero, error => error.IsTransient));

The sequence constructor consumes different FaultInjection values in order and then delegates. FaultInjection.Outcome(...) synthesizes non-zero, signalled, timed-out, or unobserved live/bulk results; FaultInjection.Delegate() is an explicit pass-through step. Seeded(inner, seed, probability, injection) chooses the same invocation indices for the same seed and call order, making probabilistic scenarios reproducible rather than flaky.

WithLatency(duration) delays that one injection through the intercepted command's TimeProvider. Supply a virtual provider, advance its timer, and test timeout/storm-guard behavior without sleeping. Completion calls link both the verb token and Command.CancelOn; SpawnAsync keeps its ordinary one-shot token boundary. A cancelled delayed action returns ProcessError.Cancelled and never reaches the inner runner. InvocationCount is safe to inspect after concurrent calls, although reproducibility still uses invocation order as its sequence axis.

What the doubles don't cover

The subprocess-free doubles center on the bulk primitives. ScriptedRunner and RecordReplayRunner both implement CaptureStringAsync and CaptureBytesAsync, and both serve a FakeProcess from SpawnAsync (so the parts of the live surface a fake can replay — StdoutLinesAsync, the readiness probes — are testable through them). The one gap is recording a live stream: a RecordReplayRunner in record mode returns Error(ProcessError.Unsupported …) from SpawnAsync, because a live stream can't be captured without racing the consumer — record a streaming call through a capture verb, then replay it as a stream. The full live RunningProcess surface (WaitForPortAsync, ProfileAsync, …) and a Pipeline are best tested against a real (possibly trivial) child process; keep the scripted/cassette doubles for everything that flows through the capture primitives.

FakeProcess.WithContentLengthFrames(payloads) is the framed-protocol exception: it builds canonical CRLF Content-Length messages around byte-exact scripted payloads, so a ContentLengthSession can be tested without a server process. Pair it with WithStdinOpen() and assert FakeProcess.StdinBytes to verify frames the client sent.

Pseudo-terminal (PTY) doubles

A Command.Pty run gives the child a single merged stdout+stderr terminal stream — OutputEvent.Stderr is never produced (the two streams are physically one tty device). The doubles model that observable shape:

• FakeProcess.WithPty() marks a fake as a PTY run. On the built handle OutputEventsAsync() then yields only OutputEvent.Stdout, ProcessResult.Stderr is empty, and any text set via WithStderr is folded into the merged stdout stream (a fake cannot reproduce real OS interleaving, so folded stderr simply follows the stdout text) rather than surfaced as a separate event.
• ResizeAsync is a recorded no-op success. On a PTY fake ResizeAsync(cols, rows) returns Ok () (not the typed Unsupported a non-PTY fake returns) and records the geometry — read the last requested (cols, rows) back through FakeProcess.LastResize for assertions.
• Signals are recorded. RunningProcess.Signal appends the requested value to FakeProcess.Signals; StopAsync appends the command's configured StopSignal, and a completion verb cancelled on a command with CancelGrace appends its CancelSignal — so control-flow tests can assert direct, graceful, and cancellation delivery without an OS process.
• ScriptedRunner serves a command built with Command.Pty() as a merged-stream PTY fake automatically, so a scripted PTY scenario reads back the same way.
• Cassettes record a Pty flag and geometry (schema v4) and replay as a merged-stream handle; the WithRedaction hook scrubs the whole merged stream, so an echoed credential never lands in a committed fixture.

task {
    // A PTY fake: one merged stream, resize is a recorded no-op.
    let fake = FakeProcess.Create("tui").WithPty().WithStdout("frame1\nframe2")
    use proc = fake.Build()

    let! _ = proc.ResizeAsync(120, 40)
    Assert.That(fake.LastResize, Is.EqualTo(Some(120, 40)))
    // proc.OutputEventsAsync() yields only OutputEvent.Stdout — never OutputEvent.Stderr.
}

Expect-style sessions against the double

A PtySession works over the built handle just as it does over a real run: it reads the same merged stream raw, so ExpectAsync finds a scripted prompt that carries no line terminator, and SendAsync/SendLineAsync record their bytes into FakeProcess.StdinBytes. Add WithStdinOpen() to give the fake the interactive stdin the send verbs need (a fake built from your own Command.KeepStdinOpen command through ScriptedRunner already has it).

task {
    let fake =
        FakeProcess.Create("installer").WithPty().WithStdinOpen()
            .WithStdout("Welcome\r\nPassword: LEN=6\r\n")

    use proc = fake.Build()
    let session = PtySession proc

    // The prompt has no newline — exactly what WaitForLineAsync cannot deliver.
    let! _ = session.ExpectAsync("Password: ", TimeSpan.FromSeconds 10.0)
    let! _ = session.SendLineAsync "secret"
    let! _ = session.ExpectAsync("LEN=6", TimeSpan.FromSeconds 10.0)

    Assert.That(Encoding.UTF8.GetString fake.StdinBytes, Is.EqualTo "secret\r")
}

The scripted output is complete before the first ExpectAsync runs, so a fake cannot model a child that reacts to what was sent: script the whole conversation's output up front and expect its parts in order. A genuine request/response exchange needs a real Command.Pty run.

Inherent limitation (not papered over). A double has no real terminal, so it cannot make the child observe isatty = true. Any behaviour that depends on the child seeing a tty — a tool switching from line-buffered "dumb" output to full-screen TUI mode, a shell enabling colour, a prompt suppressing echo — is not reproducible with a fake or a cassette; only the observable merged-stream shape is. Test that child-tty behaviour against a real Command.Pty run.

Merged-stderr doubles

A command built with Command.MergeStderr() (or F# Command.mergeStderr) is replayed by ScriptedRunner with the same observable contract as a real OS-level 2>&1: scripted stderr is folded into stdout, ProcessResult.Stderr is empty, and OutputEventsAsync() emits only OutputEvent.Stdout. The fake places scripted stderr after scripted stdout, adding a newline when needed; it cannot reproduce the operating system's real byte interleaving. This is independent of PTY mode, so a merged-stderr command does not acquire PTY-only ResizeAsync support.

Interactive stdin doubles

FakeProcess and ScriptedRunner model interactive stdin for a command built with Command.KeepStdinOpen. TakeStdin() then returns a working Some backed by an in-memory sink; assert the exact bytes written through it with FakeProcess.StdinBytes. This works both when building a FakeProcess directly with Build() and when ScriptedRunner replays a keep-open command.

Record and replay

RecordReplayRunner (also in ProcessKit.Testing) closes the loop: record real runs to a JSON cassette once, then replay them deterministically — fast, hermetic, no subprocess in CI.

F#

task {
    // Record once against the real tool (wraps a real runner), then save:
    let recorder = RecordReplayRunner.Record("fixtures/git.json", JobRunner())
    let! _ = Runner.run recorder CancellationToken.None (Command.create "git" |> Command.arg "--version")
    recorder.Save() |> ignore // Result<unit, ProcessError> — surfaces write errors

    // Replay everywhere else — no subprocess, identical results:
    match RecordReplayRunner.Replay "fixtures/git.json" with
    | Ok replay ->
        match! Runner.run replay CancellationToken.None (Command.create "git" |> Command.arg "--version") with
        | Ok version -> () // the recorded stdout, replayed
        | Error err -> eprintfn $"{err.Message}"
    | Error err -> eprintfn $"{err.Message}"
}

C#

// Record once against the real tool (wraps a real runner), then save:
var recorder = RecordReplayRunner.Record("fixtures/git.json", new JobRunner());
await recorder.RunAsync(new Command("git").Arg("--version"), CancellationToken.None);
recorder.Save(); // Result<unit, ProcessError> — surfaces write errors

// Replay everywhere else — no subprocess, identical results:
var replayResult = RecordReplayRunner.Replay("fixtures/git.json");
if (replayResult is { IsOk: true, ResultValue: var replay })
{
    // the recorded stdout, replayed
    if (await replay.RunAsync(new Command("git").Arg("--version"), CancellationToken.None) is { IsOk: false, ErrorValue: var err })
        Console.Error.WriteLine(err.Message);
}
else if (replayResult is { IsOk: false, ErrorValue: var loadErr })
    Console.Error.WriteLine(loadErr.Message);

Record(path, inner) wraps inner and captures each completed call; Save() writes the cassette (it is also flushed best-effort on dispose — RecordReplayRunner is IDisposable — but only once Complete() has declared the recording finished, see A crash writes nothing before Complete, and Save() is the call that surfaces a write error). Replay(path) returns a Result<RecordReplayRunner, ProcessError> loaded from the file.

Semantics worth knowing before you commit a cassette:

Robust against a corrupt or hostile cassette. A cassette is untrusted input — it may be hand-edited, truncated by a failed write, corrupted on disk, or crafted. Loading (and replaying) one is guaranteed to either succeed or fail with a typed ProcessError — never an unhandled exception, a hang, or runaway memory: a truncated or bit-flipped file, invalid JSON or base64, missing/inconsistent fields, out-of-range sizes, or a format version from the future all resolve to a typed error (a future version is rejected rather than misread). This is enforced by an adversarial, randomized (FsCheck) parsing-robustness suite (CassetteRobustnessTests.fs) alongside the example-based cassette tests.

Only env values are redacted by construction, into the fingerprint. program, args, stdout, stderr, and a recorded failure's own text (its streams, detail, JSON-RPC data, and — the one exception to the env-values rule, an Env("PATH", …) override included — the PATH a NotFound searched) are stored verbatim and can carry secrets (a --password=… flag, a token echoed to output), so review a fixture before committing it. The WithRedaction hook covers the captured text (a string capture's stdout/stderr, a bytes capture's stderr) and every one of those failure fields; program and args are recorded as given unless you add the opt-in WithCommandProjection hook, which decides what those two fields carry on disk without touching what the entry matches on. On Unix the file is written atomically and owner-only (0600 from creation — a temp file renamed into place, so it is never briefly world-readable); on Windows it inherits the containing directory's ACL, so keep secret-bearing fixtures out of world-readable directories. The write is also crash-safe: the new cassette is flushed to disk before it replaces the old one, and on Unix the directory entry the rename creates is fsync'd too, so an interrupted save leaves the previously saved cassette intact rather than a truncated file.

A neat trick: in tests, record against a ScriptedRunner instead of JobRunner() — the whole record → save → replay round trip is then itself hermetic.

Grow a cassette on miss (VCR "new episodes"). RecordReplayRunner.Auto(path, inner) replays what the cassette already holds and, on a miss, delegates to inner, records the result, and grows the file on Save() (or on a dispose after Complete(), like record mode — see A crash writes nothing before Complete) — so you build a cassette up incrementally instead of curating every entry by hand. Existing entries still replay hermetically; only a first-seen call reaches the real tool. A missing (or empty) file starts a fresh cassette. Use strict Replay(path) in CI, where a miss should fail loudly. (Like record mode, Auto can't capture a streaming miss — record such a call through a capture verb first.)

Matching customization & redaction (RecordReplayOptions). Pass an immutable, fluent RecordReplayOptions to Record / Replay / Auto (use the same options on both sides, since they change how invocations are keyed):

• WithFileStdinContentHashing() — key a Stdin.FromFile source by its contents (a SHA-256 of the bytes) instead of its path, so a cassette matches on what was actually fed to the child (and matches a Stdin.FromBytes of the same bytes). Opt-in: the file must exist at record and replay time, and an unreadable file surfaces ProcessError.Stdin.
• WithArgNormalizer(args -> args) — normalize the argument list before matching, so a volatile argument (a temp directory, a nonce) no longer defeats the match — drop it, or rewrite it to a stable placeholder. The raw arguments are still stored verbatim for inspection.
• WithRedaction(text -> text) — scrub captured text before it is written, so a secret echoed to stdout/stderr never reaches disk. Applied at record time to a string capture's stdout/stderr and a bytes capture's stderr; a byte[] stdout capture is stored opaquely (base64) and is not passed through the redactor.
• WithCwdMatching() — restore the working directory (Command.CurrentDir) as part of the match key, so two otherwise-identical invocations that ran in different directories are treated as distinct recordings. CassetteEntry.Cwd always stores the working directory verbatim for inspection regardless of this setting; only its participation in matching is opt-in.
• WithCommandProjection((program, args) -> (program, args)) — project the persisted command line: what this hook returns is what the recording stores in CassetteEntry.Program/Args, so a secret that lives in argv (a --password=… flag, a token in a URL) is kept off disk the way WithRedaction keeps one out of captured output. It cannot break replay, because matching does not go through it: the entry is keyed on a CommandFingerprint — a SHA-256 of the invoked program and its (normalized) arguments, plus the Command.Arg0 override (never projected, since this hook covers only Program/Args) — taken before the projection runs and stored beside the projected text. So two calls whose projections collide stay two recordings, and a reader needs no projection configured to replay a projected cassette (this is a write-side policy). Two things to know: with the raw arguments gone from the file the match key is frozen at record time, so changing WithArgNormalizer afterwards needs a re-record; and the fingerprint is a hash of the real command line, so a low-entropy secret argument (a short PIN) is still brute-forcible from it — the same caveat the environment fingerprint carries. A blank projected program is stored as (redacted) (the format requires an entry to name one); Cwd and captured output are not touched here. It projects what this recorder records — an Auto session that grows an older cassette rewrites none of the rows already in it, so a fixture that already carries a secret needs re-recording, not just reopening.

How a CapturePolicy interacts with a cassette. Command.CapturePolicy (see Hardening → Redacting output at capture) is a command knob, not a cassette one, so it applies on both halves of a round trip and the two doubles stay in step with a live run:

• Recording stores the capture the run produced, so text the policy shaped is written to disk already shaped, with no RecordReplayOptions configured: a string recording's stdout and stderr, and a bytes recording's stderr. A bytes recording's byte[] stdout is shaped by neither the policy (a raw capture hands it no decoded line) nor WithRedaction (which skips that field, as its bullet above says), so it is stored as captured — a bytes fixture can hold whatever the child printed, policy or not. WithRedaction remains the hook for what a policy cannot reach: a recorded typed failure's own fields, and a fixture recorded by a command that had no policy.
• Replaying shapes the recorded text with the replaying command's policy, exactly as the live pump would have shaped the same lines. That is what keeps a cassette hit agreeing with the SpawnAsync replay of the same entry, which re-pumps the recording through the ordinary capture path — and it means adding a policy to an existing test scrubs a fixture recorded before the policy existed. A bytes entry's byte[] stdout stays byte-exact (a raw capture has no decoded line to shape); its stderr is shaped, the same split a live bytes run makes.
• Because the policy therefore runs on both halves, write an idempotent policy — redaction, the case this seam exists for, is. A non-idempotent transform would be applied once at record time and once again on replay.
• The policy is not part of the match key, so it never changes which recording a call replays: swapping or adding one re-shapes what comes back, it does not turn a hit into a miss.

var options = new RecordReplayOptions()
    .WithArgNormalizer(args => args.Where(a => !a.StartsWith("/tmp/")).ToArray())
    .WithRedaction(text => text.Replace(token, "[REDACTED]"))
    // Stored argv only — matching still keys on the real command line.
    .WithCommandProjection((program, args) =>
        (program, args.Select(a => a.Replace(token, "[REDACTED]")).ToArray()));

// Auto (like Replay) returns a Result — it can fail to load an existing cassette.
if (RecordReplayRunner.Auto("fixtures/git.json", new JobRunner(), options) is { IsOk: true, ResultValue: var recorder })
{
    // recorder replays a hit, records a miss, and grows the cassette on Save()...
}

A crash writes nothing before Complete

A cassette keeps program, args, cwd, and captured stdout/stderr verbatim, so the recording a failed run left behind is exactly what you do not want landing on disk by accident. Dispose runs both when a scope ends normally and while the stack unwinds out of a thrown exception or a failed assertion, and .NET gives it no way to tell those apart — so the drop-time flush is gated on Complete(), your own statement that the recording finished as intended:

F#

use recorder = RecordReplayRunner.Record("fixtures/git.json", JobRunner())
// ... the calls you want recorded ...
// ... and the assertions about them, which may fail ...
recorder.Complete() // last: reached only if nothing above threw; the write happens on dispose

C#

using var recorder = RecordReplayRunner.Record("fixtures/git.json", new JobRunner());
// ... the calls you want recorded ...
// ... and the assertions about them, which may fail ...
recorder.Complete(); // last: reached only if nothing above threw; the write happens on dispose

Until Complete() is called, disposing the recorder writes nothing at all: no cassette is created at that path, and one already there is left byte for byte as it was. It is the TransactionScope.Complete() shape, and Auto behaves exactly like Record here.

The mark is the whole gate, so complete last. Dispose reads that mark and nothing else; it is never told how the scope ended. A scope that throws after Complete() is therefore flushed just like one that ended normally — the failed run's verbatim argv and captured output do reach disk — which is why the call belongs after everything that can fail, assertions included, and not directly after the calls being recorded. If a recording must never be written by anything but a call you can point at, leave Complete() out entirely and Save() where you want the file: with no mark set, dispose writes nothing whatever happens.

Save() is unaffected in both directions — it writes immediately and returns the I/O error, neither needing nor setting the completion mark — so a recording that must reach disk however the scope ends is one you Save(). Complete() itself only marks: it does no I/O, never throws, and the write it enables is still the best-effort, silent one dispose has always done (including anything captured after the call, so record first, assert next, and complete last).

CliClient

CliClient is the foundation for a typed wrapper around an external tool (git, gh, kubectl, …): it owns the program name, per-client defaults, and the runner, so your wrapper contributes only the commands and the parsers — and because the runner is injectable, the wrapper tests hermetically with a ScriptedRunner.

Create one with CliClient.create name (or CliClient(name)) and configure it, each call returning a new client:

• .WithDefaults(configure) — apply shared defaults with the full Command builder, e.g. client.WithDefaults(fun c -> c.CurrentDir(repo).Timeout(ts).Env("K", "V")) (timeout, working directory, environment, encoding, ok-codes, retry, logger, …)
• .WithRunner(runner) — run every command through runner instead of the default JobRunner (this is the test seam)

.Command(args) builds a configured Command without running it (the template's defaults applied), and .RunAsync(args) / .OutputStringAsync(args) / .OutputBytesAsync(args) (plus ExitCodeAsync/ProbeAsync/ParseAsync/…) build and run through the client's runner. .EnsureAvailableAsync() is a preflight check — "is the client's program installed?" — with no spawn (see Preflight: is a program installed?); it is always local, never delegated to .WithRunner's runner, so a ScriptedRunner injected for the wrapper's own tests has no bearing on it.

F#

/// A small typed git wrapper. The CliClient is supplied, so tests inject a double.
type Git(client: CliClient) =

    /// HEAD's commit id (trimmed stdout, success required).
    member _.Head(repo: string) =
        client.RunAsync [ "-C"; repo; "rev-parse"; "HEAD" ]

    /// Is the work tree clean? The exit code *is* the answer, so probe it.
    member _.IsClean(repo: string) =
        client.ProbeAsync [ "-C"; repo; "diff"; "--quiet" ]

C#

/// A small typed git wrapper. The CliClient is supplied, so tests inject a double.
public class Git(CliClient client)
{
    /// HEAD's commit id (trimmed stdout, success required).
    public Task<FSharpResult<string, ProcessError>> Head(string repo) =>
        client.RunAsync(["-C", repo, "rev-parse", "HEAD"]);

    /// Is the work tree clean? The exit code *is* the answer, so probe it.
    public Task<FSharpResult<bool, ProcessError>> IsClean(string repo) =>
        client.ProbeAsync(["-C", repo, "diff", "--quiet"]);
}

Production wires the real runner and the per-client defaults:

F#

let git = Git((CliClient.create "git").WithDefaults(fun c -> c.Timeout(TimeSpan.FromSeconds 30.0)))

C#

var git = new Git(new CliClient("git").WithDefaults(c => c.Timeout(TimeSpan.FromSeconds(30))));

…and the wrapper tests against a scripted runner, no subprocess:

F#

[<TestFixture>]
type GitWrapperTests() =

    [<Test>]
    member _.``Head is trimmed``() =
        task {
            let scripted =
                (ScriptedRunner())
                    .On([ "git"; "rev-parse"; "HEAD" ], Reply.Ok "abc123\n")

            let git = Git((CliClient.create "git").WithRunner scripted)

            match! git.Head "/repo" with
            | Ok sha -> Assert.That(sha, Is.EqualTo "abc123")
            | Error err -> Assert.Fail err.Message
        }

C#

[TestFixture]
public class GitWrapperTests
{
    [Test]
    public async Task Head_is_trimmed()
    {
        var scripted = new ScriptedRunner()
            .On(["git", "rev-parse", "HEAD"], Reply.Ok("abc123\n"));

        var git = new Git(new CliClient("git").WithRunner(scripted));

        switch (await git.Head("/repo"))
        {
            case { IsOk: true, ResultValue: var sha }:  Assert.That(sha, Is.EqualTo("abc123")); break;
            case { IsOk: false, ErrorValue: var err }: Assert.Fail(err.Message); break;
        }
    }
}

…or against a cassette recorded from the real tool once.

Dependency injection

The separate ProcessKit.Extensions.DependencyInjection package wires the seam into Microsoft.Extensions.DependencyInjection. AddProcessKit() registers an IProcessRunner in the container — logger-aware when the container already has an ILoggerFactory, so runs emit ProcessKit's lifecycle events with no extra wiring. (See the Dependency injection guide for configured defaults, keyed per-tool clients, and a shared container-managed group.)

F#

services.AddProcessKit() |> ignore

// Consumers depend on the interface — the same seam you test against:
type Deployer(runner: IProcessRunner) =
    member _.Deploy() =
        Runner.run runner CancellationToken.None (Command.create "deploy")

C#

services.AddProcessKit();

// Consumers depend on the interface — the same seam you test against:
public class Deployer(IProcessRunner runner)
{
    public Task<FSharpResult<string, ProcessError>> Deploy() =>
        runner.RunAsync(new Command("deploy"), CancellationToken.None);
}

AddProcessKit registers via TryAdd, so a pre-existing IProcessRunner is left intact: to substitute a double in an integration test, register your ScriptedRunner (or RecordReplayRunner) before calling AddProcessKit, and the real runner backs off. In a plain unit test you usually skip the container entirely and construct Deployer(scriptedRunner) directly — the whole point of depending on the interface.

Next: Observability