Repository navigation
[WIP] feat(windows): MSI bootstrap + self-update packaging - #253
Closed
swarit-stepsecurity wants to merge 35 commits into
Closed
swarit-stepsecurity wants to merge 35 commits into
swarit-stepsecurity wants to merge 35 commits into
Conversation
Point(name) is an inlinable no-op unless built with selfupdate_faults. Tagged builds act on DMG_FAULT (merged with the link-time Baked value): kill, err, hang, diskfull, exit<N>, with optional @user/@System qualifiers. DMG_FAULT_TRACE records every checkpoint reached.
…ecords Layout paths rooted at an install root and a data root; strict x.y.z versions; schema-1 state.json with corrupt-as-missing Load and atomic Save (temp + fsync + MoveFileEx REPLACE_EXISTING|WRITE_THROUGH, or rename + dir fsync) with <prefix>.state-temp/-renamed fault checkpoints; non-blocking LockFileEx/flock update lock; run records capped at 10 with bad-run classification. Kill-at-checkpoint subprocess tests and fuzzers.
Windows: create the directory with, or re-assert, owner SYSTEM and a protected DACL (SYSTEM F, Administrators F, Users RX, inherited by children) via native security APIs, reporting whether drift was fixed. Rejects files and reparse points. Other platforms only create the dir.
…s and version manifest Port the latest-binary client, GitHub-then-mirror download and SSHSIG verifier from step-security#217 for the MSI updater: native arch via IsWow64Process2, env then WinHTTP machine proxy, strict x.y.z targets, signatures checked before download and sha256 after. Version dirs get a manifest so staged versions can be re-verified.
The MSI installs the launcher a second time as stepsecurity-dev-machine-guard-bootstrap.exe. Under that name it resolves the self-update state.json pointer (current -> previous -> seed), launches versions\<v>\task.exe with run-record env, picks the SYSTEM updater with a seed fallback after 3 consecutive failures, and adopts the MSI seed into versions\ without ever failing the MSI. No network or crypto code. The launcher exits 70 when started by the bootstrap and unable to start the agent, and falls back to a sibling agent.exe in version dirs. updatestate gains the frozen pointer reader, the run-chain resolver and a field-preserving state patch.
…ner with our own WinVerifyTrust (revocation off) on Windows plus a portable PKCS#7 read of the PE signature for the leaf subject and countersignature timestamp. SamePublisher pins candidates to the running updater's own signer subject; an unsigned dev build refuses updates unless a selfupdate_testkeys build pins a subject.
On an MSI bootstrap install, install points both scan tasks at `bootstrap.exe run send-telemetry` and (re)creates the SYSTEM task "StepSecurity Dev Machine Guard (Update)" from a full task XML: hourly with a 10 min random delay plus at boot after 15 min, StartWhenAvailable, battery allowed, 30 min limit, IgnoreNew. The registration is verified. Uninstall of a bootstrap install deletes all three tasks, waits up to 60 s for the scan mutex, terminates DMG processes under the install root, and removes versions\, state\ and RunningVersion. Non-bootstrap uninstalls keep their behaviour and never touch the update task.
Product.wxs installs stepsecurity-dev-machine-guard-bootstrap.exe and
seed\<version>\{agent.exe, task.exe, dmg-linux} in place of the top-level
agent and launcher. A deferred SYSTEM adopt-seed CA (Return=ignore) runs
on install, upgrade and repair; the install CA now runs on repair too
(M-7); the uninstall CA is skipped during a major upgrade and can't block
removal (M-6, M-19).
AUTOUPDATE (default 1) is passed to configure, which gains
--auto-update=<bool> and writes auto_update into the machine config.
msi-smoke asserts the layout, the versions/state ACLs, all three tasks,
a bootstrap run, a real SYSTEM update tick, repair, an N to N+1 upgrade
without downgrade, and a clean uninstall.
One tick of S0-S7: lock, sweep and GC (tolerating in-use dirs), ACL re-assert, probation health with rollback and quarantine (permanent, or seven days for an inconclusive probation), target check every 6 h or when a probation closes, verified staging into versions\<v>, SYSTEM smoke test, mutex-guarded switch with deferral reporting. Events are written to a state.json outbox in the same atomic write as the change they report and retried until the backend takes them.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Update events go to run-status as status=update with retries; 4xx drops the event, anything else stays in the updater's outbox. The SYSTEM updater resolves the device id from the hardware and rewrites the device-id file. Scans mark the bootstrap's run record once run-status started is accepted and report update_pointer_invalid when the bootstrap fell back. Adds the agent.prestart checkpoint, configure --auto-update and the auto_update/update_mirror_url/lag/cooldown config fields.
Overlapping launches (hourly and logon tasks on wake) updated whichever record was newest, so the run that scanned could stay open forever and another run could take its verdict. Records now carry the bootstrap's run_id and MarkRunStarted/FinalizeRun match it; the agent reads STEPSEC_RUN_ID.
The previous updater runs during every probation and rewrote state.json from its own struct, erasing whatever a newer version had added. State, probation, quarantine entries and updater now carry unknown keys through Load and Save verbatim.
A valid state that omitted or nulled them loaded with nil maps, so the first write into them would panic.
An unknown @qualifier, an unknown action or a colon-bearing checkpoint without an action left the rule silently unmatched, so a typo in a fault test passed without injecting anything. Only in selfupdate_faults builds.
…ndows CreateFile without FILE_FLAG_BACKUP_SEMANTICS fails on a directory with access denied, so Load returned an I/O error instead of corrupt and the updater skipped its rebuild path.
…vent fields Each update event keeps the execution_id minted when it was queued, so only its own retries reuse it. Events agent-api would refuse (no install_channel, rollback without update_from) are not posted.
A clean run of each scenario (update, rollback) under DMG_FAULT_TRACE lists the checkpoints of every tick; each traced checkpoint is then hit with kill, err, hang and diskfull in a subprocess, followed by clean ticks, asserting convergence and its event, hygiene, no flip-flop and the frozen bootstrap read against an in-process fake API. A state corruption matrix covers the updater side of the bootstrap cases. The matrix found that a rolled-back version's dir survived until the next tick, so GC now also runs right after a rollback. Runs on Linux and windows-latest.
A second os=linux call could resolve a different version if the release gate moved between calls. dmg-linux now uses linux_checksum and signed_linux_checksum from the windows response; missing fields fail verification.
… backend During an outage the primary updater exits 1 three times and the fallback would then report a broken updater that was never broken. The fallback now reports it only on a tick where its own latest-binary call succeeded.
…/win-msi-self-update-boot Brings in the core updater (C1-C4) so the bootstrap builds against its updatestate API. Conflicts in cli/config (--auto-update) and tests.yml resolved in favour of core; the bootstrap is adapted in the merge so it compiles: release file names (updatestate.AgentFile/LauncherFile), run records keyed by run_id (STEPSEC_RUN_ID), unknown-field-preserving Save instead of the local patch helper, and a manifest for the adopted seed.
…te=false The seed holds stepsecurity-dev-machine-guard.exe, stepsecurity-dev-machine-guard-task.exe and dmg-linux, the same names as versions\<v>\. AUTOUPDATE=0 runs configure --auto-update=false on install, upgrade and repair; the default writes nothing. msi-smoke waits only for msiexec itself with a bound, dumps the log and DMG processes on a hang, checks the adopted seed's manifest, and uploads logs when cancelled too.
…reboot A running scan holds the bootstrap open, so a quiet repair or upgrade saw a file in use and Windows Installer rebooted the machine. util:CloseApplication now terminates the bootstrap, launcher and agent before InstallValidate, and the Intune wrappers and msi-smoke pass /norestart. Also: bootstrap run launches without STEPSEC_RUN_ID when the run record can't be appended (D13), and drops an unused helper.
AUTOUPDATE has no default. 0 or 1 runs configure --auto-update=<value> on install, upgrade and repair; unset touches nothing. msi-smoke installs with 0, repairs with 1 and upgrades without the property, asserting false, true and unchanged.
swarit-stepsecurity
force-pushed
the
swarit/feat/wt/win-msi-self-update-boot
branch
from
October 19, 2026 14:01
59f2fa5 to
09bbb4d
Compare
…xisting state R-B-1. LoadPointers no longer runs the full schema-1 Validate: only schema (an integer >= 1) and a strict current decide whether state.json is usable, an invalid previous is dropped, and probation/updater are read best-effort. A future writer that puts RC versions or new shapes in other fields can't strand first-generation bootstraps on the seed. adopt-seed only creates a state when none exists. An existing state is changed through the preserving Save, or left byte-for-byte alone when this build can't round-trip it.
R-B-2. AppendRun and FinalizeRun now run under runs\<v>.json.lock (LockFileEx/flock, bounded 5 s wait), so the hourly and Logon tasks firing together no longer lose each other's records. On timeout the bookkeeping is skipped and the scan still launches. The test runs two real processes against one file; without the lock it lost 10-40 of 40 records per process.
R-B-3. The check compared each expected ACE with -contains, so an extra ACE such as Users:Modify still passed. It now requires the owner SYSTEM, a protected DACL and exactly the three explicit ACEs (type, identity, rights, inheritance).
R-B-4. adopt-seed writes HKLM\Software\StepSecurity\RunningVersion (64-bit view) right after it saves a new current, so the value exists from install and the uninstall check that removes it is no longer vacuous. msi-smoke asserts it after install, before uninstall and gone after. A Windows test covers the registry write.
…stop R-B-5. Windows-native tests: SpawnHidden's deadline terminates a child and its grandchild through the job with exit 124; terminateUnder stops a process whose image is under the install root and leaves others alone; waitForInstanceMutex returns at once without a mutex, waits for a scan to release it, and gives up after its bound. The mutex name is a var so the test uses its own.
R-B-6. adopt-seed only refuses a newer seed when the seed's own version has a quarantine entry that is permanent or whose until is still in the future. An expired D5 inconclusive quarantine no longer blocks an MSI push of that version, and other versions' entries never did.
…CM /fa repair R-B-8 and follow-ups: - install with AUTOUPDATE=0 (false); plain SCCM repair msiexec /fa keeps current, restores tasks and leaves it false; upgrade with it unset keeps it false; the README's REINSTALL=ALL REINSTALLMODE=omus AUTOUPDATE=1 re-enables it; a fresh install with it unset doesn't write false. - any other AUTOUPDATE value now reaches configure, which rejects it; the rejection is in the MSI log and Return=ignore keeps the MSI going with the setting unchanged (smoke asserts both).
`(AutoUpdate -eq 'false')` parses as AutoUpdate called with the arguments -eq and 'false', so it returned the value itself and the check passed for any non-empty value. Wrap the call: ((AutoUpdate) -eq …).
…nstall msi-smoke showed uninstall leaving versions\: killing the bootstrap first made its kill-on-close job end the launcher and agent, so TerminateProcess on them failed with ACCESS_DENIED, terminateUnder reported an error without waiting, and RemoveAll ran while their images were still mapped. stopProcess now waits for the exit and only fails if the process is still running; removal of versions\ and state\ retries briefly. Windows test for the exiting-process case.
D18. recordUpdater used the full Load, so a state holding values this build rejects (RC versions in quarantine or probation) stopped consecutive_failures from counting, and the seed-updater fallback never triggered. updatestate.SaveUpdaterRecord reads the state like LoadPointers, rewrites only updater.version/consecutive_failures and keeps every other byte, under the update lock. Test: such a state with three failing updater exits picks the seed on the fourth tick.
D18 follow-up. SaveUpdaterRecord refused any schema other than 1, so on a schema-2 state the failure count was skipped and the seed fallback never fired. Under the additive-only contract updater.version and consecutive_failures keep their meaning in every schema: it now patches them for any schema >= 1 and keeps every other byte, schema included. LoadPointers reads probation/updater best-effort for any schema, so the updater pick honours the counter. Test: a schema-2 state with extra fields and three failing exits picks the seed on the fourth tick.
Member
Author
|
Superseded by #252, which contains this branch (merged in) plus all follow-up fixes. Closing; this draft only carried Windows CI for the bootstrap/MSI work. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Work in progress; opened as a draft to get Windows CI (msi-smoke, windows-native tests). Not for review or merge yet.
stepsecurity-dev-machine-guard-bootstrap.exe run|update|adopt-seed): resolves the self-update state pointer (current, previous, seed) and launches versioned binaries; no network or crypto.Builds on the self-update foundation commits (faultinject, updatestate).