Repository navigation
feat: inject sandbox UID via compute driver instead of requiring it in container images #1959
Description
Activity
- addedarea:policyPolicy engine and policy lifecycle workPolicy engine and policy lifecycle workstate:review-readyReady for human reviewReady for human reviewarea:sandboxSandbox runtime and isolation workSandbox runtime and isolation worktest:e2eRequires end-to-end coverageRequires end-to-end coveragearea:clusterRelated to running OpenShell on k3s/dockerRelated to running OpenShell on k3s/docker
on Jun 18, 2026 - added a parent issue
on Jun 22, 2026 🏗️ build-plan
Implementation Plan
Issue type:
feat
Complexity: Medium
Confidence: High — the policy engine and supervisor changes are well-scoped; Kubernetes driver changes follow established patterns. Phase 3 cleanup is trivial once Phases 1–2 land.Summary
Remove the hard dependency on a baked-in
"sandbox"user in container images by allowing numeric UIDs in the process identity fields (run_as_user/run_as_group) within a safe range, and addingsandbox_uid/sandbox_gidconfig to compute drivers that inject the resolved UID at sandbox creation time. On Kubernetes/OpenShift, the driver auto-detects the sandbox UID from namespace SCC annotations.Scope
crates/openshell-policy/src/lib.rs— Relax policy validation: accept numeric UIDs in[1000, 2_000_000_000]alongside"sandbox"; reject UID 0 and system rangescrates/openshell-supervisor-process/src/process.rs— Accept numeric UIDs for privilege dropping without requiring/etc/passwdentry; handle filesystem chown with numeric UIDscrates/openshell-supervisor-process/src/ssh.rs— Derive USER/HOME env vars gracefully when no passwd entry existscrates/openshell-driver-kubernetes/src/config.rs— Addsandbox_uid/sandbox_gidoptional config fields with OpenShift annotation detectioncrates/openshell-driver-kubernetes/src/driver.rs— Inject sandbox UID into supervisor container securityContext and PVC init container chown; resolve from annotations or configcrates/openshell-driver-vm/src/rootfs.rs— Accept configurablesandbox_uid/sandbox_gidinstead of hardcoded constants (add to driver config)examples/bring-your-own-container/Dockerfile— Removegroupadd/useradd; document that images no longer need a baked-in sandbox userdocs/reference/gateway-config.mdx/docs/reference/sandbox-compute-drivers.mdx— Document new driver config options- Tests — Unit tests for policy validation, supervisor UID handling; integration test for K8s driver annotation parsing
Implementation Steps
Phase 1: Policy engine accepts numeric UIDs
- Add constants
MIN_SANDBOX_UID = 1000andMAX_SANDBOX_UID = 2_000_000_000toopenshell-policy/src/lib.rs. - Replace
PolicyViolation::InvalidProcessIdentitywithPolicyViolation::UidNotInSandboxRange { field, value }(or keep existing variant but change error message). The Display impl should clarify that"sandbox"or a valid numeric UID is accepted. - In
validate_sandbox_policy(), instead of hard-checking for"sandbox", accept either:"sandbox"(backward-compatible string name)- A string that parses as
u32within[MIN_SANDBOX_UID, MAX_SANDBOX_UID]
- Reject: UID 0, strings < 1000 or > 2_000_000_000, non-numeric non-"sandbox" values (e.g.,
"root","nobody"). - Update
ensure_sandbox_process_identity()— only default to"sandbox"when empty; never inject a UID. The UID comes from the driver config at runtime. - Add unit tests covering:
"sandbox"(pass), numeric UID in range (pass), UID out of range (fail),"root"(fail),"0"(fail), non-numeric string like"nobody"(fail).
Phase 2: Supervisor accepts numeric UIDs
- In
validate_sandbox_user()(process.rs), accept both name-based and UID-based identities:- If value parses as u32, skip passwd lookup (write OCSF log confirming numeric UID usage)
- If it is
"sandbox", keep existing passwd lookup behavior - Return error for non-numeric non-"sandbox" values that fail passwd lookup
- In
prepare_filesystem(), accept numeric UIDs:- If uid is a parsed u32, pass directly to
nix::unistd::chown()(no passwd lookup needed) - If it is a name, keep existing passwd-based resolution
- If uid is a parsed u32, pass directly to
- In
drop_privileges(), accept numeric UIDs:- If uid parses as u32, use it directly (skip
User::from_name()) - Maintain all defense-in-depth checks (UID/GID verification after drop)
- For groups: if gid is numeric, skip passwd lookup; otherwise keep name-based resolution
- If uid parses as u32, use it directly (skip
- In
ssh.rssession_user_and_home(), when the value is a numeric UID, fall back to default HOME (/sandbox) instead of trying a passwd lookup that will fail. The USER env var should be set to the UID string so the agent session still has a meaningful identifier.
Phase 3: Kubernetes driver config + OpenShift detection
- Add
sandbox_uid: Option<u32>andsandbox_gid: Option<u32>fields toKubernetesComputeConfigincrates/openshell-driver-kubernetes/src/config.rs. Both default to None. - In the K8s driver, implement passive OpenShift SCC detection:
- On sandbox creation, read namespace annotations via
Api<Namespace>::get()(addk8s-apiimport for namespace API) - Parse
openshift.io/sa.scc.uid-rangeannotation (format:<start>/<size>) to extract UID - Parse
openshift.io/sa.scc.supplemental-groupsfor GID if present, otherwise use UID as GID - If annotations are absent or not in OpenShift format, fall back to configured
sandbox_uid/sandbox_gidfrom config - Validate resolved UID/GID against
[MIN_SANDBOX_UID, MAX_SANDBOX_UID](reuse policy constant)
- On sandbox creation, read namespace annotations via
- In the supervisor container spec construction (
apply_supervisor_sideload), pass the resolved sandbox UID to the supervisor via an environment variable (SANDBOX_UID) in addition to setting it in the policy'srun_as_user. The policy field carries the UID to child processes; the env var helps the supervisor validate its own UID. - For PVC init containers that chown workspace data, use the resolved sandbox UID/GID instead of hardcoded user names.
Phase 4: VM driver and examples/docs cleanup
- Add
sandbox_uid/sandbox_gidconfig to the VM driver (check if there's a shared driver config or if it needs its own). Replace hardcodedSANDBOX_UID = 10001in rootfs prep with the configurable value. - In the BYOC example Dockerfile, remove the
groupadd/useraddlines. Update comments to explain that sandbox UIDs are now injected at runtime. - Update gateway config docs (
docs/reference/gateway-config.mdx) to document new driver config options. - Update compute-driver setup docs if applicable.
Test Plan
- Unit tests:
- Policy validation: test all acceptance/rejection paths for
run_as_userandrun_as_group(uid, name, out-of-range, invalid) - Supervisor privilege dropping: test with numeric UID (no passwd entry),
"sandbox"name, and edge cases (gid mismatch verification) - Kubernetes driver: test OpenShift annotation parsing with sample annotations; test fallback to config values; test that resolved UID is within range
- Policy validation: test all acceptance/rejection paths for
- Integration tests:
- Kubernetes driver pod spec generation includes correct
securityContext.runAsUserfor supervisor container andrunAsUser/init container chown with numeric UID - Supervisor process.rs: verify privilege dropping works when uid has no passwd entry (mock
/etc/passwdor test in isolated environment)
- Kubernetes driver pod spec generation includes correct
- E2E tests:
- Update
e2e/rust/tests/custom_image.rs— the example image currently creates user at UID 1000660000. Verify it still works with the new policy engine (the UID is now valid). Optionally add a test that uses an image without a pre-created user, relying entirely on injected UID.
- Update
Risks & Open Questions
- Programs requiring passwd entry — The spike flagged
sudo/sshfailure without/etc/passwd. For now, this is acceptable: the supervisor drops privileges before starting child processes, and most sandbox workloads don't needsudo. If future issues report broken tools, a synthetic passwd write inprepare_filesystem()can be added later. - Docker driver — The Docker driver already uses
--user 0for the supervisor container (confirmed at line 2283). It passes the policy'srun_as_userto child processes. The Docker driver can adopt the configurable UID pattern in a follow-up issue if needed, but it's not in scope for this build. - Image ownership drift — Images built with sandbox user at a different UID will show file ownership as "owned by 1000660000" when viewed from outside the container. This is benign and noted in the spike.
Documentation Impact
crates/openshell-driver-kubernetes/src/config.rs— new config fieldsdocs/reference/gateway-config.mdx— document newsandbox_uid/sandbox_giddriver config options (required by AGENTS.md when adding gateway TOML keys)docs/reference/sandbox-compute-drivers.mdx— if compute-driver setup docs cover UID requirements, update to note the injected UID patternexamples/bring-your-own-container/Dockerfile— comment changes explaining new behavior
Revision 1 — initial plan based on spike investigation (issue #1959)
- added 3 commits that reference this issue
on Jun 26, 2026 5 remaining items
- added a commit that references this issue
on Jul 2, 2026 This issue has had no activity for 14 days and is now marked stale. It may be closed in 7 days if there is no further activity. Comment or remove the state:stale label to keep it open.
- addedstate:staleInactive item at risk of automatic closure.Inactive item at risk of automatic closure.
on Jul 8, 2026 Tracking split:
- #2331 completes Docker and Podman runtime UID/GID injection, shared child environment normalization, and userless-image validation. A single OpenShell PR should close this issue.
- NVIDIA/OpenShell-Community#87 migrates the Community images and release/tag rollout after a compatible OpenShell release is available.
#1959 remains the umbrella for the completed and remaining work; neither child should close it directly.
- removedstate:staleInactive item at risk of automatic closure.Inactive item at risk of automatic closure.
on Jul 17, 2026 Following this closely — we've been prototyping the driver-injected UID path against a stock K8s cluster (no OpenShift) and hit one small plumbing gap that would be worth folding into this issue's scope, or tracking as a follow-up if you'd prefer to keep #1959 tightly scoped.
The Kubernetes driver already accepts
sandbox_uid/sandbox_gid(via TOML at[openshell.drivers.kubernetes]or theOPENSHELL_K8S_SANDBOX_UID/_GIDenv vars — seecrates/openshell-driver-kubernetes/src/main.rs:119-123and src/config.rs). But the helm chart's gateway ConfigMap template (deploy/helm/openshell/templates/gateway-config.yaml) doesn't render either field, even though it already renders every other optionalKubernetesComputeConfigfield with a{{- if .Values... }}pattern (image_pull_policy,image_pull_secrets,workspace_default_storage_size,default_runtime_class_name,app_armor_profile,supervisor_image_pull_policy,provider_spiffe_workload_api_socket_path). So on a helm-installed gateway the only reachable knob today is theopenshift.io/sa.scc.uid-rangenamespace annotation — which does work on vanilla K8s (we're using it that way), but is an awkward, OpenShift-branded lever for non-OpenShift clusters and hides the driver's platform-agnosticsandbox_uidconfig from helm users entirely.Rough shape of the fix (happy to send a PR):
- deploy/helm/openshell/values.yaml: add
server.sandboxUid/server.sandboxGid(both optional, unset by default). - deploy/helm/openshell/templates/gateway-config.yaml: render them under
[openshell.drivers.kubernetes]when set, matching the{{- if .Values... }}pattern already used by the sibling fields listed above.
That would make the driver's own
sandbox_uidconfig a first-class helm value instead of requiring either a namespace annotation or a manual ConfigMap patch, and would line up with the docs/reference/gateway-config.mdx update already listed under "Risks & Open Questions" in the issue body.Happy to open a standalone issue if you'd rather keep #1959 focused on the driver / policy / supervisor work.
- deploy/helm/openshell/values.yaml: add
- added a commit that references this issue
on Aug 24, 2026 This issue has had no activity for 14 days and is now marked stale. It may be closed in 7 days if there is no further activity. Comment or remove the state:stale label to keep it open.
- addedstate:staleInactive item at risk of automatic closure.Inactive item at risk of automatic closure.
on Sep 10, 2026
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsDone
Problem Statement
Every container image used as an OpenShell sandbox must bake in a
sandboxuser and group at a specific UID/GID. The bring-your-own-container docs require UID1000660000, the VM driver hardcodes UID10001, and the policy engine rejects anyrun_as_uservalue other than"sandbox". This creates friction for image authors, prevents compatibility with environments that allocate their own UID ranges (e.g., OpenShift SCCs), and ignores the Kubernetes-nativesecurityContextmechanism for injecting user identity at runtime.The goal is to eliminate the requirement that the sandbox user exist in the container image and instead have the compute driver inject the desired UID/GID at sandbox creation time. On OpenShift, the UID should be auto-detected from namespace SCC annotations.
Technical Context
The sandbox user identity flows through 6 distinct layers today — from image build time, through policy validation, to supervisor privilege dropping. Each layer assumes a user named
"sandbox"exists in the container's/etc/passwd. The supervisor starts as root (UID 0) and drops privileges to the sandbox user viasetuid()/setgid()after setting up network namespaces, Landlock, and seccomp. The Kubernetes driver already constructssecurityContexton the pod spec (settingrunAsUser: 0for the supervisor), so extending it to pass the sandbox UID is architecturally straightforward. The proto already uses astringfield forrun_as_user, so numeric UIDs require no wire format change.Affected Components
crates/openshell-policy/src/lib.rsrun_as_user/run_as_group, currently rejects anything other than"sandbox"crates/openshell-supervisor-process/src/process.rssetuid()/setgid(), chowns filesystemcrates/openshell-supervisor-process/src/ssh.rsUSER/HOMEenv vars from policy usercrates/openshell-driver-kubernetes/src/driver.rssecurityContext, manages PVC init containerscrates/openshell-driver-vm/src/rootfs.rscrates/openshell-driver-docker/src/lib.rs--userondocker runproto/sandbox.protoProcessPolicy.run_as_useras string (no change needed)examples/bring-your-own-container/Technical Investigation
Architecture Overview
The sandbox user identity is established at image build time and consumed at 6 points during sandbox lifecycle:
groupadd/useraddcreates thesandboxuser at a fixed UID in the Dockerfileensure_sandbox_process_identity()defaults emptyrun_as_user/run_as_groupto"sandbox"validate_sandbox_policy()hard-rejects any non-"sandbox"valuevalidate_sandbox_user()callsUser::from_name("sandbox")against/etc/passwddrop_privileges()resolves"sandbox"to a UID viaUser::from_name(), then callssetgid()/setuid()with verificationprepare_filesystem()resolves the sandbox user forchownofread_writedirectoriesThe supervisor runs as root (UID 0) to create network namespaces, set up the proxy, and configure Landlock/seccomp. It drops to the sandbox UID only for child processes. The Kubernetes driver forces
securityContext.runAsUser = 0on the main container for this reason.Code References
openshell-policy/src/lib.rs:660-668ensure_sandbox_process_identity()— defaults empty user/group to"sandbox"openshell-policy/src/lib.rs:756-772validate_sandbox_policy()— hard-rejects non-"sandbox"values forrun_as_user/run_as_groupopenshell-policy/src/lib.rs:680-697PolicyViolationenum — would need newUidOutOfRangevariantopenshell-supervisor-process/src/process.rs:758-786validate_sandbox_user()— callsUser::from_name("sandbox"), fails if missing from imageopenshell-supervisor-process/src/process.rs:892-998drop_privileges()— resolves name → UID viaUser::from_name(), callssetgid()/setuid()with verificationopenshell-supervisor-process/src/process.rs:788-870prepare_filesystem()— resolves sandbox user/group forchownofread_writedirectoriesopenshell-supervisor-process/src/ssh.rs:221-225USER/HOMEfrom policyrun_as_user, defaults to"sandbox"/"/sandbox"openshell-driver-kubernetes/src/driver.rs:970-981securityContext.runAsUser = 0on supervisor containeropenshell-driver-kubernetes/src/driver.rs:994+openshell-driver-vm/src/rootfs.rs:755-772SANDBOX_UID = 10001/SANDBOX_GID = 10001in rootfsproto/sandbox.proto:47-52ProcessPolicy—run_as_userandrun_as_grouparestringfieldsexamples/bring-your-own-container/Dockerfile:20-21groupadd -g 1000660000 sandbox && useradd -m -u 1000660000 -g sandbox sandboxe2e/rust/tests/custom_image.rs:27-281000660000patternCurrent Behavior
When a sandbox is created:
run_as_useris defaulted to"sandbox"if empty, then validated — only"sandbox"is accepted.User::from_name("sandbox")against the container's/etc/passwd. If the user doesn't exist, startup fails with: "sandbox user 'sandbox' not found in image; all sandbox images must include a 'sandbox' user and group".drop_privileges()resolves"sandbox"→ numeric UID viaUser::from_name(), then callssetgid()/setuid()with post-drop verification (defense-in-depth: confirms UID changed, confirms root can't be re-acquired).prepare_filesystem()resolves the sandbox user forchownofread_writedirectories before forking the child process.What Would Need to Change
Policy engine — Relax the hard
"sandbox"string check to also accept numeric UID strings within a platform-level range:Accept
"sandbox"(existing) or anyu32in[MIN_SANDBOX_UID, MAX_SANDBOX_UID]. Reject"root", UID 0, system UIDs below 1000, and non-numeric garbage. AddUidOutOfRangeviolation variant for clear error messages. The range is a platform safety constant, not a per-policy knob.Supervisor —
validate_sandbox_user(),drop_privileges(), andprepare_filesystem()must accept numeric UIDs:u32, skip/etc/passwdlookup and use the UID directly (setuid()/setgid()do not require a passwd entry).USER=sandboxandHOME=/sandboxas defaults when no passwd entry exists.Kubernetes driver — Add
sandbox_uid/sandbox_gidto driver config. Pass the UID to the supervisor through the policy'srun_as_userfield. The supervisor container staysrunAsUser: 0. The PVC init container uses the injected UID for chown.OpenShift SCC-aware UID resolution — On OpenShift, read namespace annotations to auto-select the sandbox UID:
Api<Namespace>::get()(driver doesn't currently do this — requires adding the call).openshift.io/sa.scc.uid-rangeannotation (format:<start>/<size>, e.g.,1000660000/10000). Use range start as sandbox UID.openshift.io/sa.scc.supplemental-groupsfor GID. Fall back to UID range start if absent.sandbox_uid/sandbox_gid.[MIN_SANDBOX_UID, MAX_SANDBOX_UID].This is passive detection (annotation presence) — no explicit "OpenShift mode" config flag needed.
VM driver — Use configurable UID instead of hardcoded
10001. The VM driver controls its own rootfs, so it can continue creating the user at rootfs prep time.BYOC / docs — Remove
groupadd/useraddrequirement from examples and documentation.Alternative Approaches Considered
NSS module in the supervisor — Synthesize a
sandboxpasswd entry via custom NSS module. Adds runtime dependency and complexity. Rejected: directly using numeric UIDs is simpler and more portable.Init container running
useradd— Create the user at container start. Requires the image to have user management tools and writable/etc/passwd. Rejected: many minimal images lack these tools.Better documentation only — Just improve the BYOC docs. Doesn't solve the underlying friction or OpenShift UID range incompatibility.
Patterns to Follow
securityContexton pod specs (driver.rs:970-981) — the sandbox UID injection follows the same JSON manipulation pattern.PolicyViolationenum with descriptive variants andDisplayimpls — the newUidOutOfRangevariant should follow the same pattern.drop_privileges()already has defense-in-depth verification (confirms UID changed, confirms root can't be re-acquired) — numeric UID support must maintain these checks.Proposed Approach
Implement in three phases. Phase 1 teaches the policy engine and supervisor to accept numeric UIDs within a safe range (
[1000, 2_000_000_000]), removing the hard dependency on a/etc/passwdentry. Phase 2 addssandbox_uid/sandbox_gidconfig to the Kubernetes driver and injects it via the policy, with passive OpenShift SCC annotation detection for automatic UID selection on OpenShift clusters. Phase 3 removes the image-side user requirement from examples, docs, and e2e tests.Scope Assessment
featRisks & Open Questions
Programs requiring a passwd entry — Some programs (
sudo,ssh) fail if the running UID has no/etc/passwdentry. Should the supervisor write a synthetic passwd entry at startup before dropping privileges?Home directory creation — If the image doesn't have
/sandboxcreated, who creates it? The supervisor could create and chown it duringprepare_filesystem(), but this needs to happen before the child process starts.File ownership in image layers — Files in the image owned by the old sandbox UID will appear as owned by a different user. Only affects images previously built with the sandbox user.
Security boundary — The range check (
MIN_SANDBOX_UID = 1000throughMAX_SANDBOX_UID = 2_000_000_000) replaces the current"sandbox"string check as the non-root invariant. It rejects UID 0, system UIDs below 1000, and unreasonably large values. The range is enforced as platform-level constants, not per-policy configurable.Docker driver — The Docker driver uses
--userondocker run. Should it also adopt the configurable UID pattern, or is this Kubernetes-only initially?Gateway config docs — If
sandbox_uid/sandbox_gidare added to the Kubernetes driver config,docs/reference/gateway-config.mdxand relevant compute-driver setup docs must be updated.Test Considerations
"sandbox"(pass), numeric UID in range (pass), numeric UID out of range (fail),"root"(fail),"0"(fail), non-numeric string (fail). Supervisor tests must cover numeric UID privilege dropping without passwd entry.securityContextwhensandbox_uidis configured, and that OpenShift annotation parsing works correctly.e2e/rust/tests/custom_image.rsto use an image without a baked-in sandbox user. Verify sandbox creation, privilege dropping, and filesystem ownership work with injected UIDs.openshell-driver-kubernetes/src/driver.rshas tests likesupervisor_sideload_injects_run_as_user_zero()that verifysecurityContext— new tests should follow this pattern. Policy validation tests inopenshell-policy/src/lib.rsusevalidate_sandbox_policy()assertions — extend these.Created by spike investigation. Use
build-from-issueto plan and implement.