Choosing the Right System Boundary for Agent Workspace Forking

Why fork an agent workspace at all? Because useful agents need to try things. A coding agent may compare two implementations, test several debugging hypotheses, or choose between a small patch and a deeper refactor. A computer-use agent may need to explore multiple action paths, recover from a bad click, or return to a known state after an application behaves unexpectedly.

Branching the model's text state is cheap. Branching the environment where its actions take effect is not. A coding workspace accumulates state as work proceeds: the repository changes, dependencies are installed, tests warm caches, development servers start, terminals remain open, and tools build in-memory models of the project. If every attempt starts from a clean image, the agent repeatedly pays to rebuild the context it already had.

The desired operation is closer to this:

fork the current workspace in a couple of seconds
run several attempts in isolated branches
compare their tests, logs, and diffs
keep one result
discard the others

This article is Blog 0 of a technical series about implementing that operation. Before delving into technical details such as memory mappings, page-cache copy-on-write, network policy, or result-promotion semantics, this post introduces how we choose the right system boundary for agent workspace forking.

What is a live agent workspace fork?

A live agent workspace fork creates an isolated, runnable branch of an already-running workspace. At the fork point, the source and branch observe equivalent process, memory, filesystem, terminal, service, and relevant interface state. Afterward, each side can execute independently, and writes in one branch do not silently mutate the other.

This is different from starting another clean sandbox. A clean sandbox begins from an image or template. A live fork begins from the state the agent has already built. That distinction matters when a test server is running, a language server has indexed the project, packages are installed, caches are warm, and intermediate files exist.

There are at least four familiar places where a system could branch: the Git repository, one Unix process, a container, or a virtual machine. They capture different amounts of state and therefore solve different problems.

Git-level fork: the file tree

A Git branch or git worktree is the most familiar form of branching for developers. It gives each attempt a separate view of repository files while preserving history and providing mature tools for diff, conflict detection, review, and merge. For an agent that only edits source code and runs short-lived commands, Git may be enough.

But Git versions a repository, not a running workspace. It does not reproduce running development servers or terminal sessions. It does not clone installed dependencies, warmed package caches, language-server memory, browser state, local databases, background processes, GUI state, open file descriptors, or environment variables already held inside processes. Some of that state may be represented by untracked files; much of it is not a file at all.

A second worktree can rebuild parts of this environment, but rebuilding is not forking. The cost becomes setup time, and subtle differences can appear between the source and the reconstructed branch.

Process fork: the right cost model, at the wrong scope

Unix fork() offers the performance intuition we want. The child initially shares the parent's physical memory pages. Page-table entries are copied, both processes see the same bytes, and the kernel makes a private copy only when one side writes. This is copy-on-write, usually abbreviated CoW.

fork latency ~= fixed setup
             + existing state discovery and reconstruction
             + synchronously captured changed state

post-fork capacity cost ~= state changed after the fork

not: eagerly copy the complete source workspace

This works because speculative branches begin identical to their source. Copy-on-write keeps the capacity cost proportional to subsequent divergence, but it does not eliminate fixed setup, discovery, consistency, or reconstruction work from fork latency.

The scope is the problem. A coding workspace is not one process. It is a process tree containing the agent, shells, test runners, compilers, language servers, development servers, databases, browser helpers, and other background tasks. Their parent-child relationships, process and thread groups, namespaces, signals, descriptors, and shared resources are part of the live state. Calling fork() on one process does not branch that coordinated workspace.

VM-level fork: a complete machine boundary

A virtual-machine snapshot captures a broad boundary: guest-kernel state, guest memory, virtual devices, disks, and processes. This is useful when the primary requirement is strong machine isolation, a separate guest kernel, distributed provisioning, or a clean boundary between tenants.

AgentENV is a good example of this design point. It runs Firecracker microVM environments across machines, supports snapshot-backed pause and resume, and can fork a running environment into independent sandboxes using incremental memory and filesystem snapshot layers. In our same-host evaluation, AgentENV’s complete first-use latency was 295–395 milliseconds across 1–200 workload processes. It increased from 360 milliseconds with no requested dirty allocation to 642 milliseconds at 512 MiB and 1.75 seconds at 2 GiB of dirty memory.

But a VM is intentionally a thick boundary. The host manages a guest machine rather than individual workload processes. AgentENV deliberately shares the host page cache through OverlayBD and ublk, but host-side process inspection, per-process policy, debugging, and direct integration with host tools are less transparent than with containers.

The point is not that VM-level forking is inferior. It solves a different systems problem. If the main question is "How do we provision many strongly isolated agent machines?", a microVM is a natural answer. If the question is "How do we repeatedly branch this already-running Linux development workspace with minimal duplicated state?", a container-level boundary deserves a closer look.

Container-level fork: the live Linux workspace

A container does not emulate a complete machine. Its processes use the host kernel and are isolated through namespaces, cgroups, filesystem views, capabilities, and security policy. That usually avoids per-sandbox guest-kernel and VMM overhead while remaining broad enough to hold the coordinated process tree that Git and single-process fork() miss. TClone's current implementation does require a custom host kernel and runtime stack, so operational simplicity is not automatic.

This boundary also creates useful sharing opportunities. Anonymous memory can begin shared and diverge page by page. Filesystem data can use copy-on-write storage. Read-mostly binaries, libraries, source files, and cached data can reuse host memory. The host can still inspect and manage the branch as Linux processes rather than as an opaque guest machine.

The resulting target is:

reconstruct the process tree in a sibling container
share anonymous memory through copy-on-write
share unchanged file and page-cache state
isolate branch-specific writes and side effects
discard the branch or explicitly promote allowlisted artifacts later

The TClone paper calls this a versioned personal workspace. Its central design decision is to separate fast branch creation from checkpoint serialization. The source pauses long enough to capture metadata and establish sharing, then resumes. The new branch becomes locally usable using copy-on-write state while checkpoint serialization can proceed asynchronously. Local usability, host-flushed completion, and remote durability are separate milestones.

The open-source GenseeAI/os4agent repository carries this design into a container runtime stack. It includes a Linux kernel with page-cache CoW support, extended CRIU components, a crun live-fork path, a TClone-aware conmon, and Podman support for live container cloning. The implementation target is not "another container from the same image." It is another runnable version of the current container.

In Gensee's TClone workflow, Gensee copies the selected workspace and agent configuration into a source container, starts Codex and tmux there, and later forks that live container. Packages installed inside the source become part of subsequent forks. Host packages outside the workspace are not automatically imported; they must come from the container image, be installed inside the source, or be explicitly mounted. An AgentENV integration would need an equivalent guest template and workspace-transfer path, although OCI templates and guest tooling can hide that setup from end users.

Why container-level fork fits coding-agent search

Coding-agent execution increasingly resembles search over program states. From workspace W, the runtime can create W1, W2, and W3, let agents try different approaches, run tests in each branch, compare the resulting diffs and logs, and retain one result.

The branches are highly redundant at creation time. They contain the same repository, dependencies, process tree, caches, and services. Many source-edit experiments diverge modestly, but builds, browsers, databases, and large mutable caches can dirty substantial memory and storage. A useful runtime should share unchanged state while making its synchronous work proportional to the state that must actually be discovered, reconstructed, or captured.

Container-level fork therefore offers a combination that is difficult to get at the other boundaries: richer state than Git, a broader scope than process fork(), and direct host-kernel sharing and inspection without a guest boundary. It also leaves Git in its proper role. Runtime state can be branched at the container layer, while final source changes can still be reviewed and promoted using Git.

Our same-host experiments found opposite scaling variables. AgentENV was nearly insensitive to guest process count because guest process state remained encoded in VM memory, but latency grew with dirty guest memory because selected dirty pages were copied into an immutable memory layer before fork return. TClone retained a substantial per-process reconstruction cost, but its asynchronous local-use path directly shared source memory and scaled more slowly with dirty bytes. AgentENV was faster for process-heavy, moderately dirty workloads; TClone became faster for sufficiently large dirty working sets. Exact crossover points depend on guest size, workload, cache state, and the chosen usable-versus-durable timing boundary.

Why live container fork was a missing primitive

Mainstream container tooling can start a fresh container, snapshot filesystem state, and checkpoint or restore a process tree. Those are useful primitives, but they do not by themselves provide low-latency branching and explicit branch lifecycle or artifact promotion for an already-running workspace.

CRIU is the closest building block. During checkpoint, it collects process resources and writes state, including memory contents and register information, to image files. During restore, it reads those images, recreates the process tree, restores namespaces and resources, fills memory mappings, recreates threads, and resumes execution. That workflow is designed primarily for checkpoint/restore use cases such as migration, recovery, and maintenance.

If used naively for branching, the new branch waits for both sides of that workflow: dump the source state, then reconstruct a runnable copy from the checkpoint. The cost of serializing and restoring the workspace lands directly on every speculative branch point. In our same-host measurements, latest direct TClone return rose from 464 milliseconds at one process to 1.07 seconds at 200 processes, while AgentENV first use rose from 295 to 395 milliseconds. The boundaries are not identical—the TClone number is API return and the AgentENV number is first usable command—but the process-scaling difference is clear.

A live fork needs a different critical path:

briefly pause the source
capture topology and sharing metadata
make a sibling branch runnable from shared state
resume source and branch
copy only as their state diverges
move serialization and durability outside the local-use critical path when semantics permit

Historically, there was little demand for this exact operation. Containers were commonly started from images, scaled as stateless service replicas, checkpointed for migration, or restarted for recovery. There was no broad use case for creating many speculative, mergeable branches of the same live development environment. Agentic execution changes that workload: branching becomes part of the inner loop rather than an occasional infrastructure event.

The difficult parts begin after choosing the boundary

Choosing the container boundary does not make workspace fork equivalent to calling one kernel function. A live workspace combines state with different ownership and consistency rules.

The runtime must recreate a process tree without corrupting parent-child relationships, thread groups, namespace-local identifiers, open resources, or signals. It must share anonymous memory safely across sibling containers, even though ordinary fork() only creates a child process. Filesystem snapshots are not enough because block-level CoW can still duplicate identical file pages in the host page cache. GUI buffers change rapidly and generally need branch-local handling. Existing network connections and external services create side effects that cannot be rolled back by copying local memory.

Result promotion is harder still. Discarding an isolated branch is conceptually simple. Files, Git commits, test results, and logs can be reviewed and explicitly promoted, but process memory, credentials, browser profiles, database transactions, and external effects are not generally safe or meaningful to merge. A coding-agent runtime needs to expose branch lifecycle and allowlisted artifact promotion through the agent's normal workflow, with human approval wherever an action changes authority.

Container isolation has its own tradeoff. Containers share the host kernel, so their security boundary is not identical to a microVM boundary. A practical system needs defense in depth through namespaces, cgroups, capabilities, seccomp, Linux Security Modules, credential scoping, filesystem restrictions, and network controls. The right question is not "Are containers always better?" It is "Does this isolation and sharing boundary match the agent workload and threat model?"

Forking state also forks authority

A full live fork can duplicate ambient authority as well as useful state. API keys, cloud tokens, SSH credentials, browser cookies, authenticated sessions, environment variables, open descriptors, credential-helper caches, transcripts, and copies retained in process memory may all cross the fork boundary. Deleting credential files afterward is not reliable sanitization because live processes and immutable or copy-on-write snapshot layers may retain the same bytes.

Full-state forks should therefore normally remain within one principal or trust domain. Cross-user sharing should start from a credential-free base and copy only explicitly selected workspace artifacts. A credential broker should issue short-lived, branch-specific capabilities after the child has a unique identity and restricted network policy. Snapshot layers require encryption, access control, retention limits, and garbage collection because logical deletion does not guarantee physical erasure.

Containers make destination-specific file, file-descriptor, socket, namespace, and mount transformation easier because the host understands more Linux objects. A VM generally requires an in-guest sanitization agent. Neither boundary can reliably preserve an authenticated live browser or agent process while guaranteeing that all original authority has been removed.

What this technical series will cover

The design principle connecting the series is simple:

make the branch locally usable first
share unchanged pages
keep persistence off the local-use critical path
explicitly promote only selected artifacts

That principle avoids eagerly materializing all memory and filesystem bytes, although the runtime still reconstructs the process tree and kernel-visible resources. The result is a systems primitive that coding agents can use repeatedly while they search.

Frequently asked questions

What is a live agent workspace fork?

It is an isolated, runnable branch of an already-running agent workspace. It preserves supported in-container or in-guest process, memory, filesystem, terminal, service, and relevant interface state at the fork point. External services and side effects are not automatically forked or rolled back, and later changes remain private until selected artifacts are promoted or the branch is discarded.

Why is a Git worktree not a complete workspace fork?

A worktree branches repository files and Git history. It does not branch the process tree, in-memory application state, open descriptors, local services, terminal sessions, browser state, or other live execution context.

How is live container fork different from CRIU checkpoint and restore?

Stock CRIU synchronously serializes checkpoint images and later reconstructs them. TClone extends CRIU to establish cross-container copy-on-write sharing first; when asynchronous persistence is selected, the branch can become locally usable before checkpoint serialization and host-flush completion.

When is a VM-level agent environment the better choice?

A VM or microVM is a strong fit when machine isolation, tenant separation, a full-system environment, process-heavy workloads, distributed sandbox provisioning, or a separate guest kernel dominates the design. Container-level fork is attractive for frequent branching of a same-trust live Linux workspace, especially when direct host integration or large dirty-memory working sets matter.

Sources and further reading

This article draws on the TClone systems note, the full paper TClone: Low-Latency Forking of Live GUI Environments for Computer-Use Agents, the open-source GenseeAI/os4agent runtime repository, the AgentENV repository, and CRIU's checkpoint/restore documentation.