Creating a development sandbox with crosvm

# Creating a development sandbox with crosvm Source: [https://www.erat.org/crosvm.html](https://www.erat.org/crosvm.html) ## Contents - [Introduction](https://www.erat.org/crosvm.html#intro) - [Background](https://www.erat.org/crosvm.html#background) - [Compiling`crosvm`](https://www.erat.org/crosvm.html#compiling) - [Running`crosvm`](https://www.erat.org/crosvm.html#running)- [Host networking](https://www.erat.org/crosvm.html#host-networking) - [Starting the VM](https://www.erat.org/crosvm.html#starting-vm) - [VM networking](https://www.erat.org/crosvm.html#vm-networking) - [Resizing the VM disk](https://www.erat.org/crosvm.html#resizing-disk) - [Managing the VM](https://www.erat.org/crosvm.html#managing-vm) - [Development](https://www.erat.org/crosvm.html#development)- [Connecting to VM servers](https://www.erat.org/crosvm.html#vm-servers) - [Signing and pushing changes](https://www.erat.org/crosvm.html#signing-pushing) - [Conclusion](https://www.erat.org/crosvm.html#conclusion) ## Introduction[¶](https://www.erat.org/crosvm.html#intro) I’m uneasy writing software on a computer that also holds sensitive information like email, passwords, and web browser cookies\. Just in the past few days, I’ve read about people’s lives being upended after[cloning the wrong GitHub repositories](https://nymag.com/intelligencer/article/your-digital-self-is-vulnerable.html)and about[fake interview scams](https://grack.com/blog/2026/06/25/dissecting-a-failed-nation-state-attack/)designed to try to trick developers into running`npm install`to install backdoors\. Some toolchains \(like[Go](https://go.dev/)’s\) at least ensure that compiling a program won’t result in untrusted code being executed, but tests still need to be run\. I try to minimize my use of third\-party dependencies \(Go’s standard library helps here\) and hold off on upgrading libraries for a while after new versions are published, but development tools like language servers, pretty\-printers, and linters still increase the risk of supply\-chain attacks\. One way to deal with this would be by using a completely separate computer for development, but the idea of carrying a second laptop around with me is unappealing — I’d prefer to just have a single piece of hardware to worry about\. ## Background[¶](https://www.erat.org/crosvm.html#background) My primary way of addressing my fears for a long time was by doing development on a[Chromebook](https://en.wikipedia.org/wiki/Chromebook)\.[ChromeOS](https://en.wikipedia.org/wiki/ChromeOS)uses[verified boot](https://www.chromium.org/chromium-os/chromiumos-design-docs/verified-boot/)in conjunction with a[readonly OS partition](https://www.chromium.org/chromium-os/developer-library/reference/device/disk-format/)for its “main” operating system \(read: the Chrome browser and assorted userspace services\), but it also provides[Crostini](https://chromeos.dev/en/linux/linux-on-chromeos-deep-dive), a system for running a separate Linux installation within a virtual machine alongside the main OS\. At least in theory, code running in the VM is unable to interact with the main OS in all but a few restricted ways, like shared directories,[Wayland](https://wayland.freedesktop.org/)windows forwarded via[sommelier](https://chromium.googlesource.com/chromiumos/platform2/+/HEAD/vm_tools/sommelier/), and audio output\. My development process typically involves text mode programs and web servers, so a VM accessed via terminal windows was a natural fit\. Within the Crostini VM, I further tried to limit what[Node\.js](https://nodejs.org/en)programs and other third\-party development tools could do by[using`bubblewrap`to run them in sandboxed environments](https://www.erat.org/node.html)\. In 2025, Google[announced](https://chromeunboxed.com/its-official-google-says-the-android-and-chromeos-merger-is-coming-next-year/)a vague effort to move “the ChromeOS experience” onto Android: > So I think the opportunity for us that we see is how do we accelerate all the AI advancement that we’re doing on Android and bring that to the laptop form factor as rapidly as possible, and also have the laptop and the rest of the Android ecosystem work seamlessly together\. So what we’re doing is we’re basically taking the ChromeOS experience and re\-baselining the technology underneath it on Android\. In early 2026, my use of ChromeOS VMs was negatively affected by a couple of bugs that took months to be fixed:[VMs wouldn’t start after a reboot](https://issues.chromium.org/issues/458443474)and[glyphs in the terminal were sporadically rendered as solid blocks](https://issues.chromium.org/issues/502262228)\. The people of Reddit[found](https://www.reddit.com/r/Crostini/comments/1rl2m2k/if_you_are_not_able_to_start_crostini_after_the/)[workarounds](https://www.reddit.com/r/chromeos/comments/1sgak7g/anyone_else_having_trouble_with_terminal_fonts/)for both issues, but the fact that these bugs shipped in the first place and remained unfixed for so long didn’t bode well for Google keeping the lights on\. In May 2026, Google announced that something called a[Googlebook](https://en.wikipedia.org/wiki/Googlebook)running something else called[Aluminium OS](https://en.wikipedia.org/wiki/Aluminium_OS)would be the successor to Chromebooks\. The promise/threat of a focus on generative AI finally prompted me to start looking for other options\. \(The fact that my latest Chromebook had a 2019\-era CPU was also a factor\.\) I knew that my new development environment would likely take the form of a laptop running[Debian](https://www.debian.org/), but I wasn’t sure how to address my security concerns\. I considered trying to sandbox all my development using`bubblewrap`, but I was uneasy about depending on namespaces for security given the number of Linux kernel CVEs so far this year\. [crosvm](https://github.com/google/crosvm), Crostini’s[hypervisor](https://en.wikipedia.org/wiki/Hypervisor), is available in standalone form and has[a decent amount of documentation](https://crosvm.dev/book/), so I decided to try using it to replicate what I had been doing earlier with ChromeOS\. I’m aware that using Google\-maintained software with “cros” in its name probably isn’t the safest escape route when concerns about the future of ChromeOS were what sent me down this path, but[`crosvm`is still being actively developed as of mid\-2026](https://github.com/google/crosvm/commits/main/), and it sounds like from the documentation like it’s gained at least a bit of traction outside of ChromeOS: > It is now used across multiple products and platforms such as TerminalApp on Android, Cuttlefish and Windows\. If`crosvm`stops receiving updates at some point, I figure that I’ll just try to replicate my setup using a different hypervisor\. \([Firecracker](https://github.com/firecracker-microvm/firecracker), maybe? I’d love to have a non\-Big Tech option here with equally\-strong security claims and support for graphics and sound, should I need them at some point\.\) ## Compiling`crosvm`[¶](https://www.erat.org/crosvm.html#compiling) The[Building Crosvm on Linux](https://crosvm.dev/book/building_crosvm/linux.html)page describes how to check out and compile the`crosvm`source code\. I tried following the “Using the development container” instructions first to build using[Docker](https://www.docker.com/), but the process failed with an unhelpful error\. Rather than debugging further, I switched to[Podman](https://podman.io/): ``` $ sudo apt install --no-install-recommends podman passt uidmap libguestfs-tools $ ./tools/dev_container ./tools/build_release ``` The build process downloaded gigabytes of data, all of which got dumped under`~/\.local/share/containers`\. Eventually, the compiled`crosvm`executable ended up in`~/\.local/share/containers/storage/overlay/5dec291d9e203d8cfa5fb69a301fadd5be2020fb9173e09420444622701788df/diff/scratch/cargo\_target/crosvm/x86\_64\-unknown\-linux\-gnu/x86\_64\-unknown\-linux\-gnu/release/crosvm`\. \(Presumably that ugly 32\-byte hex ID isn’t constant\.\) The executable was dynamically linked against`libvirglrenderer\.so\.1`, which looks like it’s provided by the[libvirglrenderer1](https://packages.debian.org/stable/libvirglrenderer1)Debian package\. It was also linked against`libgfxstream\_backend\.so\.0`, which caused more trouble\. I couldn’t find a prebuilt copy of this library anywhere online \(I even tried poking around in various Android\-development\-related archives provided by Google, since that seems to be where the code originated\)\. The source code looks like it’s available at[https://github\.com/google/gfxstream](https://github.com/google/gfxstream), but rather than spend time trying to build dependencies when I wasn’t even sure that I’d be able to get`crosvm`working, I decided to rebuild without graphics support for now\. \(I noticed later that the`crosvm`build process appears to automatically compile the library to`~/\.local/share/containers/storage/overlay/26a34f735cfaeb84ad3969e2886048652ef2b21f818fe73a3d7b8660f18eaa9c/diff/usr/lib/x86\_64\-linux\-gnu/libgfxstream\_backend\.so\.0\.1\.2`, and I presumably could’ve just used that copy\.\) `crosvm`defines a list of default features in its top\-level`Cargo\.toml`file: ``` default = ["audio", "balloon", "document-features", "gpu", "qcow", "usb", "libvda-stub", "net", "slirp"] ``` It sounds like there’s currently[no way to disable individual default features in Cargo](https://github.com/rust-lang/cargo/issues/3126), so I did another build explicitly requesting just the features that I thought I’d be likely to need: ``` $ ./tools/dev_container cargo build --release --no-default-features --features audio,balloon,net,qcow,usb ``` The resulting`crosvm`binary was dynamically linked against far fewer libraries than before\. It ended up in`~/\.local/share/containers/storage/overlay/5dec291d9e203d8cfa5fb69a301fadd5be2020fb9173e09420444622701788df/diff/scratch/cargo\_target/release/crosvm`this time, and I copied it to a directory listed in`$PATH`\. ## Running`crosvm`[¶](https://www.erat.org/crosvm.html#running) ### OS image The`crosvm`documentation includes both[Example usage](https://crosvm.dev/book/running_crosvm/example_usage.html)and[Advanced usage](https://crosvm.dev/book/running_crosvm/advanced_usage.html)pages\. Based on those instructions, I downloaded the latest`debian\-13\-nocloud\-amd64\-XXXXXXXX\-XXXX\.tar\.xz`file from the[Debian Official Cloud Images](https://cloud.debian.org/images/cloud/)site and extracted it to a`~/temp/crosvm`directory where I planned to store the VM’s data\. I used`virt\-builder`from the[guestfs\-tools](https://packages.debian.org/stable/guestfs-tools)package to extract the kernel and[initrd](https://en.wikipedia.org/wiki/Initial_ramdisk): ``` $ virt-builder --get-kernel ./disk.raw -o . ``` ### Host networking[¶](https://www.erat.org/crosvm.html#host-networking) To give the VM network access,`crosvm`can share a[TAP](https://en.wikipedia.org/wiki/TUN/TAP)interface with it\. I ran these commands from`crosvm`’s[Network](https://crosvm.dev/book/devices/net.html)document to create a TAP interface for 192\.168\.10\.1/24 and route traffic to and from it: ``` $ sudo ip tuntap add mode tap user $USER vnet_hdr crosvm_tap $ sudo ip addr add 192.168.10.1/24 dev crosvm_tap $ sudo ip link set crosvm_tap up $ sudo sysctl net.ipv4.ip_forward=1 $ HOST_DEV=$(ip route get 8.8.8.8 | awk -- '{printf $5}') $ sudo iptables -t nat -A POSTROUTING -o "$HOST_DEV" -j MASQUERADE $ sudo iptables -A FORWARD -i "$HOST_DEV" -o crosvm_tap -m state --state RELATED,ESTABLISHED -j ACCEPT $ sudo iptables -A FORWARD -i crosvm_tap -o "$HOST_DEV" -j ACCEPT ``` Since I’m \(reluctantly\) using[NetworkManager](https://www.networkmanager.dev/)on my computer, I needed some way to make the interface permanent\. Running the following seemed to produce the same result as the previous`ip`commands: ``` $ sudo nmcli con add type tun ifname crosvm_tap con-name crosvm_tap \ mode tap owner $(id -u $USER) \ ipv4.method manual ipv4.addresses 192.168.10.1/24 ``` The`net\.ipv4\.ip\_forward=1``sysctl`setting can be made permanent by creating a`/etc/sysctl\.d/ip\_forward\.conf`file: ``` # Enable forwarding for crosvm TAP networking. net.ipv4.ip_forward = 1 ``` To make the`iptables`rules stick around, I also installed the[iptables\-persistent](https://packages.debian.org/stable/iptables-persistent)package\. ### Starting the VM[¶](https://www.erat.org/crosvm.html#starting-vm) At this point, I was ready to try starting the VM\. From within my`~/temp/crosvm`directory, I ran the following: ``` $ crosvm run \ --socket ./crosvm.sock \ --cpus num-cores=4 \ --mem size=2048 \ --net tap-name=crosvm_tap \ --block ./disk.raw \ --initrd ./initrd.img-* \ --params "root=/dev/vda1" \ ./vmlinuz-* ``` I was greeted with a familiar\-looking Linux boot sequence, followed by prompts for setting the VM’s time zone and root password\. With the above command, the initial`crosvm`process creates a`crosvm\.sock`socket that later`crosvm`invocations can use to interact with the running hypervisor\. ### VM networking[¶](https://www.erat.org/crosvm.html#vm-networking) After logging in as root with the password that I just set, I ran the following to bring up the network interface at 192\.168\.10\.2 in the VM, using 192\.168\.10\.1 as a gateway: ``` # ip addr add 192.168.10.2/24 dev enp0s4 # ip route add default via 192.168.10.1 ``` I also updated the VM’s`/etc/systemd/resolved\.conf`file to contain the following: ``` [Resolve] DNS=8.8.8.8#dns.google 8.8.4.4#dns.google 2001:4860:4860::8888#dns.google 2001:4860:4860::8844#dns.goo gle ``` and ran`systemctl restart systemd\-resolved`\. At this point, I was able to reach the outside world from within the VM, and also ping the VM at 192\.168\.10\.2 from the host OS\. To make the networking changes permanent, I created a`/etc/systemd/network/10\-lan\.network`file in the VM containing the following: ``` [Match] Name=e* Type=ether [Network] Address=192.168.10.2/24 Gateway=192.168.10.1 DNS=8.8.8.8 ``` and then ran`systemctl enable systemd\-networkd \-\-now`\. ### Resizing the VM disk[¶](https://www.erat.org/crosvm.html#resizing-disk) The VM started out using the 3 GiB disk provided by the Debian image\. To make the root partition larger, I followed the instructions from the`crosvm`[Block](https://crosvm.dev/book/devices/block.html)document\. With the VM already running, I ran the following within`~/temp/crosvm`in the host OS to resize the block device to 20 GiB: ``` $ crosvm disk resize 0 21474836480 ./crosvm.sock ``` Inside the VM, I then followed the approach from[this random page](https://www.golinuxcloud.com/extend-resize-primary-partition-non-lvm-linux/#method-1-change-size-of-partition-using-parted-cli-utility), using[parted](https://packages.debian.org/stable/parted)to delete the`vda1`root partition and recreate it starting at the same sector and using the rest of the disk \(luckily, I didn’t have a swap partition in the way as described in that doc\) and then using`resize2fs`from[e2fsprogs](https://packages.debian.org/stable/e2fsprogs)to grow`vda1`to use the new space\. **It is very important to update the VM’s`/etc/fstab`to reference the new partition’s`PARTUUID`\(see`/dev/disk/by\-partuuid`\), not its`UUID`\(from`/dev/disk/by\-uuid`\)\.**If you don’t do this or get it wrong, you’ll end up with a readonly root partition\. I was able to recover from this state by running`mount \-n \-o remount,rw \-t ext4 /dev/vda1 /`so I could write an updated`/etc/fstab`\. ### Managing the VM[¶](https://www.erat.org/crosvm.html#managing-vm) I wanted a straightforward way to start and stop the VM, so I wrote a little script named`manage\_crosvm`: ``` #!/bin/bash -e readonly NUM_CPU=6 readonly MEM=8192 readonly STOP_SEC=60 usage() { echo "Usage: $0 [start|stop] <dir>" exit 2 } [[ $# -ne 2 || $1 = -* ]] && usage readonly cmd=$1 readonly dir=$2 readonly socket=$dir/crosvm.sock name=$(basename "$dir") readonly name cd "$dir" case $cmd in start) if [[ -e $socket ]]; then pids=$(lsof -t "$socket") if [[ -n $pids ]]; then echo "Socket $socket already used by process(es) $pids" exit 1 fi echo "Deleting stale socket $socket" rm -f "$socket" fi echo "Starting crosvm with $NUM_CPU CPUs and $MEM MB" exec crosvm run \ --socket "$socket" \ --cpus num-cores=$NUM_CPU \ --mem size=$MEM \ --net tap-name="${name}_tap" \ --block ./disk.raw \ --initrd ./initrd.img-* \ --params "root=/dev/vda1" \ ./vmlinuz-* ;; stop) if [[ ! -e $socket ]]; then echo "Socket $socket doesn't exist" exit 0 fi echo "Running poweroff in $name" if ! ssh "$name" sudo poweroff; then echo "Running poweroff failed" exit 1 fi echo "Waiting for $socket to go away" end=$((EPOCHSECONDS + STOP_SEC)) while [[ -e $socket ]] && (( EPOCHSECONDS <= end )); do sleep 1 done if [[ ! -e $socket ]]; then echo "crosvm exited cleanly" else echo "crosvm didn't exit; telling it to stop" crosvm stop "$socket" fi ;; *) usage; ;; esac ``` Then I created a trivial[systemd](https://systemd.io/)unit file at`~/\.config/systemd/user/crosvm\.service`for starting and stopping`crosvm`: ``` [Unit] Description=crosvm virtual machine [Service] ExecStart=%h/path/to/manage_crosvm start %h/temp/crosvm ExecStop=%h/path/to/manage_crosvm stop %h/temp/crosvm Restart=no ``` After running`systemctl \-\-user daemon\-reload`, I’m able to run`systemctl \-\-user start crosvm`to start the VM and`systemctl \-\-user stop crosvm`to stop it\. \(Presumably there’s a way to make the job gracefully stop when I log out, but I haven’t looked into that yet\.\) The script tries to gracefully shut the VM down by running`poweroff`via SSH before giving up and running`crosvm stop`\(which forcibly stops the VM immediately, as far as I can tell\)\. ## Development[¶](https://www.erat.org/crosvm.html#development) ### Connecting to VM servers[¶](https://www.erat.org/crosvm.html#vm-servers) I wanted an easy way to let a web browser in the host OS connect to HTTP servers running in the VM\. I added a`192\.168\.10\.2 crosvm`line to`/etc/hosts`in the host OS, but I was still faced with the issue that many servers only bind to 127\.0\.0\.1 by default for local development\. To make it possible to connect to localhost\-bound VM servers from the host OS, I created a`/etc/sysctl\.d/route\_localhost\.conf`file in the VM: ``` # Allow routing incoming traffic to localhost. net.ipv4.conf.all.route_localnet = 1 ``` \(Running`sysctl \-w net\.ipv4\.conf\.all\.route\_localnet=1`as root applies this immediately without needing to reboot\.\) Then I added an iptables rule in the VM to forward incoming connections on ports 8000\-8999 to localhost: ``` # iptables -t nat -A PREROUTING -p tcp \ --dst 192.168.10.2 --dport 8000:8999 \ -j DNAT --to-destination 127.0.0.1:8000-8999 ``` I installed[iptables\-persistent](https://packages.debian.org/stable/iptables-persistent)in the VM as well so I wouldn't need to manually add the rule every time I started the VM\. ### Signing and pushing changes[¶](https://www.erat.org/crosvm.html#signing-pushing) I wasn’t excited to give the VM access to a GPG key or try to forward USB access to a Yubikey to it so it could sign my commits, and I also didn’t like the idea of giving it SSH keys to be able to push the commits to[Codeberg](https://www.erat.org/codeberg.html)or to repositories hosted elsewhere\. After a lot of internal debate, I decided to write a script that I can run in the host OS to pull a repo’s changes out of the VM, display a diff for manual review, sign the changes, and then push them upstream\. The code is a bit hokey and I don’t completely trust its correctness yet, but I ended up with the following`push\_repo`script: ``` #!/bin/bash set -euo pipefail # Prefix and suffix added to repo name to get upstream URL. readonly UPSTREAM_PREFIX=ssh://[email protected]/my-username/ readonly UPSTREAM_SUFFIX=.git # Base for repos on dev system. readonly DEV_HOST=me@crosvm readonly DEV_REPOS_DIR=code # Directory under which repos are cloned locally. readonly REPOS_DIR=$HOME/temp/repos if [[ $# != 1 || $1 == -h || $1 == -help || $1 == --help ]]; then echo "Usage: $0 <repo>" exit 2 fi repo_name=$1 if [[ ! $repo_name =~ ^[a-zA-Z0-9][-_a-zA-Z0-9]*$ ]]; then echo "Invalid repo name" >&2 exit 2 fi readonly repo_dir=$REPOS_DIR/$repo_name readonly dev_dir=$DEV_REPOS_DIR/$repo_name # Create a local checkout if one doesn't already exist. if [[ ! -e $repo_dir ]]; then mkdir -p "$REPOS_DIR" readonly upstream_url=${UPSTREAM_PREFIX}${repo_name}${UPSTREAM_SUFFIX} echo "Cloning $upstream_url" >&2 jj git clone --colocate "$upstream_url" "$repo_dir" readonly dev_url=${DEV_HOST}:${dev_dir} echo "Adding dev remote $dev_url" >&2 jj git remote --quiet -R "$repo_dir" add dev "$dev_url" fi cd "$repo_dir" echo "Fetching changes from dev" >&2 jj git fetch --remote dev # Verify that the changes are okay. jj diff --quiet --from main@origin --to main@dev echo read -r -p "Looks good? [Y/n]: " lgtm if [[ ! $lgtm =~ ^[Yy]?$ ]]; then echo "Aborted" >&2 exit 1 fi # Sign and push the changes. jj bookmark set --quiet -r main@dev main jj sign --ignore-immutable -r 'main ~ main@origin' jj git push --remote origin --bookmark main jj rebase --quiet --onto main echo "Updating dev repo" >&2 # If the dev repo is tracking a public URL, fetch the new changes. # Otherwise, push our local copy of the changes to the dev repo. dev_bookmark= origin=$(ssh "$DEV_HOST" jj -R "$dev_dir" git remote list | sed -nre 's/^origin (.*)/\1/p') if [[ $origin = https://* ]]; then ssh "$DEV_HOST" jj git fetch -R "$dev_dir" --quiet --remote origin dev_bookmark=main@origin else dev_bookmark=upstream jj bookmark set --quiet "$dev_bookmark" -r main jj git push --quiet --remote dev -b "$dev_bookmark" fi # Confirm that the dev repo doesn't contain any uncommitted code, # and then abandon the unsigned changes and update the bookmark # to include the signed changes. ssh "$DEV_HOST" bash <<EOF set -euo pipefail cd '$dev_dir' if [[ -n \$(jj diff --quiet --from '$dev_bookmark' --to main@git) ]]; then echo "$dev_bookmark contains changes against main@origin" >&2 exit 1 fi jj abandon --quiet -r 'main@git ~ $dev_bookmark' jj bookmark set --quiet main -r '$dev_bookmark' jj rebase --quiet --onto main EOF ``` The rough idea is: - In the VM, I do development in a[Jujutsu](https://github.com/jj-vcs/jj)repository, e\.g\.`~/code/my\-repo`\. - When I’m ready to push changes upstream, I set the`main`bookmark in the VM \(e\.g\.`jj bookmark set main \-r @\-`\)\. - In the host OS, I run`push\_repo my\-repo`, which pulls the changes from the VM into`~/temp/repos/my\-repo`and displays a diff\. - If I indicate that I’m happy with the changes, the script signs them, pushes them upstream, and abandons the original unsigned changes in the VM’s repo\. I feel like having signed and unsigned versions of the same changes floating around made this way harder than it needed to be\. I’m still an absolute beginner when it comes to`jj`, so I suspect that there's a better way of doing all of this\. ## Conclusion[¶](https://www.erat.org/crosvm.html#conclusion) So far, this setup is working quite well for me\. The VM takes just a few seconds to start, and basic benchmarking using the[sysbench](https://packages.debian.org/stable/sysbench)tool suggests that performance inside the VM is comparable to what I see outside of it\. Most importantly, code running in the VM has no access to the host OS beyond what’s granted to it by a hypervisor written in a memory\-safe language\.