Skip to content

test(hack): Check that a tailnet can route into an isolated VPC - #955

Draft
privateip wants to merge 1 commit into
mainfrom
feat/edge-tailnet-feasibility
Draft

privateip wants to merge 1 commit into
mainfrom
feat/edge-tailnet-feasibility

Conversation

@privateip

Copy link
Copy Markdown
Collaborator

Summary

A tailnet gateway has to route a tenant's tailnet into exactly one VPC without leaking into another, and nobody has checked whether tailscaled can run that way inside a VRF.

This adds a root-only harness that builds two overlapping VPCs with network namespaces and pinned Headscale and Tailscale images, runs each check, prints pass or fail, and removes everything it created.

It covers enrollment, route approval, source-preserving forwarding, isolation, direct and relay paths at 1280 bytes, control-plane loss and local cut-off while tailscaled keeps running.

Test plan

  • Every check passes on a host with root and Docker, and nothing the harness created is left behind

Related to #944

🤖 Generated with Claude Code

@privateip

Copy link
Copy Markdown
Collaborator Author

Run on Fedora with kernel 7.1.3 as root, with Headscale v0.29.5 and Tailscale v1.104.1 pinned by digest. All eight checks passed, and the harness left no namespaces, containers or processes behind. Full CI also passed at 8665595.

The harness is not part of CI, since it needs root and Docker.

Findings a design has to absorb:

  • tailscaled reopens a pre-created TUN in the binding's pod VRF, but on restart it removes the VRF routes through it, so the attachment owner must restore them.
  • One Headscale instance elects one primary per prefix across all its users, so tenants whose prefixes may overlap need separate instances.
  • An IPv6-only runtime keeps rebinding its sockets, so the management network needs to be dual-stack.

Not run: the hosted Tailscale control plane, which waits on a test tailnet.

@privateip
privateip force-pushed the feat/edge-tailnet-feasibility branch from 8665595 to 8c7d70a Compare October 20, 2026 19:17
Add hack/tailnet-feasibility.sh, a root-only script that builds a tailnet binding topology from network namespaces and pinned Headscale v0.29.5 and Tailscale v1.104.1 images, runs each check and prints PASS/FAIL per check.

Each binding is its own namespace running tailscaled in kernel TUN mode with its own state, socket and node key, attached to its VPC the way a host-owned attachment is: a pod VRF, a veth into the canonical host VRF with explicit routes and permanent neighbors, and tailscale0 as a persistent TUN enslaved to the pod VRF. Two VPCs use the same workload address, and two remote subnet routers in separate tailnets serve the same site prefix.

The checks cover enrollment, subnet route export gated on Headscale approval with SNAT off, route import, forwarding through the pod VRF while control and WireGuard traffic stay on the management interface, source address preservation, isolation between the two VPCs, direct and DERP-only paths with 1280-byte packets and Packet Too Big above the tailscale0 MTU, forwarding through a control-plane outage, local cut-off by an nft drop or by releasing the host attachment while tailscaled keeps running, and replacement by a binding with a new node key.

It also records tailscaled and Headscale behaviors the design has to absorb: tailscaled reopens the persistent TUN on restart but its pod VRF routes must be restored, the binding's own tailnet addresses are unreachable from inside a VRF, one Headscale elects a single primary per prefix across all users, the DERP region needs a host name rather than an IPv6 literal, and an IPv6-only runtime keeps rebinding its sockets.

The script runs files it writes, so its state directory /run/gtn-tailnet and its work directory (default /run/gtn-tailnet/work, overridable with GTN_WORKDIR) must be owned by root with mode 0700, not symlinks, and in a parent other users cannot rename them in; it creates them that way and refuses an existing path that is not, and refuses to adopt a GTN_WORKDIR that is not empty.

One run at a time: it holds an flock on /run/gtn-tailnet/lock. It records every namespace and container it creates, including the named container it copies tailscaled out of, and cleanup on exit, at the start of the next run or with --down removes only what is recorded: it stops processes in the recorded namespaces and those whose pid files still name its own tailscaled or helper server, deletes the recorded namespaces and containers, and removes the work directory it created, or only its own entries in one it adopted. Each cleanup step runs even if an earlier one fails, and it refuses to reuse a gtn-* namespace or container it did not create.

tailscaled shares the host mount namespace, so the script fingerprints /etc/resolv.conf and /var/lib/tailscale before the run and fails the run if either changed. The pulled images stay in the Docker cache, the vrf and tun modules stay loaded and an empty lock file stays in /run/gtn-tailnet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@privateip
privateip force-pushed the feat/edge-tailnet-feasibility branch from 8c7d70a to 6954f0b Compare October 20, 2026 19:21
@privateip

Copy link
Copy Markdown
Collaborator Author

Review fixes are in, and the head is now 6954f0b.

The state and work directories now live under a root-owned, mode 0700 directory in /run. Runs take a lock, cleanup removes only the namespaces and containers the run recorded, and kills only processes whose command line is the harness's own. Every cleanup step now runs even if an earlier one fails, and an explicitly passed work directory must be empty. A host-file guard fails the run if resolv.conf or the Tailscale state directory changes.

The full harness ran again end to end at 8c7d70a. All checks passed and the host was left clean. Pre-created world-writable and non-root work paths were left untouched or refused. The only change since then is the empty-directory refusal, which was tested on its own and left the directory's contents in place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant