Build/Test Tools: Check invariants while the PHPUnit suite runs. by dmsnell · Pull Request #15480 · WordPress/wordpress-develop

dmsnell · 2026-11-04T22:17:35Z

Generated by Claude using code generated by Codex.
Builds a parallel test suite for ensuring invariants are held during code runs.

Adds an external suite of invariants in tests/invariants/: properties which should hold every time some code runs, such as "a numeric post ID given to get_post() is never negative", or "a query from wpdb::prepare() reaches MySQL with its structure unchanged". The gameshark PHP extension calls their hooks just before or just after each call to a function or method, while the whole existing test suite runs unchanged. An invariant is defined with WP_Invariant: a description, any number of before and after hooks on functions or methods, and a state shared by those hooks. A before and an after hook on the same target are paired per call, so that an invariant can relate a call's inputs to its result, snapshot state before a call and compare it afterwards, or keep a table of calls and check another call against it. Calls made directly from test code are not checked by default, since tests give public functions invalid input on purpose. Violations are only reported, in the summary of the new PHPUnit Invariants workflow and as an artifact. It runs on PHP 8.5 only, for pushes and pull requests to trunk, and again when the `invariant-tests` label is added. It builds the extension, at the commit in tests/invariants/GAMESHARK_REF, inside the PHP container of the local Docker environment. Self-tests of the support code run with the extension but without WordPress, including a check of the SQL lexer against the real wpdb::prepare(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

+jobs:
+  prepare-gutenberg:
+    name: Prepare Gutenberg Assets
+    uses: ./.github/workflows/reusable-prepare-gutenberg.yml


+          package-manager-cache: false
+
+      - name: Set up npm
+        uses: ./.github/actions/setup-npm


github-actions · 2026-11-04T22:21:53Z

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

All changes will be lost when closing a tab with a Playground instance.
All changes will be lost when refreshing the page.
A fresh instance is created each time the link below is clicked.
Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

…parsers. Adds invariants covering slashed and unslashed data, authentication, permissions, type errors, HTML and XML parsing, and PCRE use. In priority order: slashed data given to functions which unslash it, and slashed data stored as given; the parse tree of what wpdb::prepare() returns; wp_kses() output, esc_url() schemes, and esc_attr() breaking out of attributes; auth cookies validated for the user they name; do_not_allow never granted; no WP_Error passed as an ID; queries MySQL would reject; PCRE functions failing silently; XML parsed with entity substitution or entity declarations; esc_xml() output which is not well-formed; WP_HTML_Processor::normalize() disagreeing with a browser; and the results of wp_authenticate(), wp_hash_password(), wp_verify_nonce(), map_meta_cap() and has_cap(). Where it helps, an invariant compares what WordPress produced with how an independent parser reads it: Dom\HTMLDocument for HTML, Uri\WhatWg\Url for URLs, Dom\XMLDocument for XML, and the MySQL parser by Jan Jakeš from WordPress/sqlite-database-integration, which is generated from MySQL's own grammar. The suite now needs PHP 8.5, which it already ran on. The parser is checked out, like the extension, at the commit in tests/invariants/DEPENDENCIES, into .invariants/, which replaces .gameshark/ and GAMESHARK_REF. The support code gains before_each() and after_each(), strings_in(), depth() to find the calls a target made itself, and WP_INVARIANTS_SKIP to leave out spec files: the PCRE and XML invariants hook built-in functions, which slows every built-in call. Calls from test code are now filtered when a violation is recorded rather than before each check, so that a stack is only read when there is something to report. The README explains why checks must not have side effects, with the cases found in core. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

With the invariants, the suite runs several times slower, and a long quiet stretch in the log could be a slow test or a stuck one. The PHPUnit extension now reports progress every 1,000 tests, and every test which takes longer than 30 seconds, on STDERR. It keeps the name of the running test in invariants-current-test.txt, which the workflow shows when the run fails or is cancelled. The job's timeout is raised from 60 to 180 minutes. The extension's warning about hooks on built-in functions is now turned off in its INI file. GAMESHARK_INVARIANTS_WARN_BUILTINS cannot turn it off: the extension's INI default for the same setting takes precedence. Whether a hook is paired is now worked out once, when its spec is loaded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… functions. The PHPUnit Invariants run crashed with a segmentation fault at test 327, Tests_Abilities_API_WpRegisterCoreContentAbility::test_ids_combine_with_a_status_filter. The gameshark extension, at a7f9b79, reads every argument of a call from where PHP keeps declared arguments. A userland function's extra arguments, such as those user_can() and map_meta_cap() collect in ...$args, are kept after its local variables, so hooks were given undefined values for them. This self-test calls a variadic fixture and checks the arguments before and after hooks see. It fails with the extension at a7f9b79, so the self-tests step stops the run early instead of the suite crashing partway through, and passes once the extension reads extra arguments as func_get_args() does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… every argument. Hooks on userland functions which take more arguments than they declare, such as `user_can()`, read the extra arguments from the wrong place in the call frame and could crash PHP. The fixed extension passes them as the function received them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…es its text. `wpdb/prepared-reaches-mysql` looked for each recently prepared query in the queries sent to MySQL, and reported it as broken wherever its text did not end on a token. `wp_set_object_terms()` prepares the same query for each term, so the query for term 1 was found inside the query for term 11, and reported. Text which only matches part of a number or a name, or lies inside one string, quoted identifier or comment, is now another query. A prepared query is still reported where it starts or ends inside a string, quoted identifier or comment, which it would break into or out of, and every place it appears is checked, not only the first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… browser reads. New helpers let invariants look at more without changing what they look at: - `WP_Invariant::property()` reads a private or protected property with reflection, which runs none of the object's code. - `WP_Invariant::methods_in()` lists methods declared in a directory's files, read as tokens without loading them, as targets for a spec file. - `WP_Invariant::has_callbacks()` reads `$wp_filter` without running a hook. - `WP_Invariant_Call::caller_args()` gives the arguments of the target's caller. The oracles can now say which host a browser goes to for a URL, how it reads one start tag (inside a TEMPLATE, where tags such as TD are kept), the value it gives a TEXTAREA, the nodes it builds in a BODY, and whether an XML document is well-formed. The MySQL oracle lists the tables a query names, and keeps the latest parse trees, so that several invariants can share one parse. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…t a browser. In priority order: - redirect/host-agrees-with-browser: when wp_validate_redirect() lets a location through, a browser goes to the host parse_url() found. - rest/permission-result-is-bool-or-error: every core REST permission check returns true, false or a WP_Error. The server treats any other value but null as allowed. - blocks/attributes-stay-in-comment: serialized block attributes stay in their comment, as a browser reads it, and decode back to the attributes. - slashing/wpdb-receives-unslashed: wpdb is never given a string wp_slash() just returned. - types/no-false-ids: get_post() and get_comment() are never given false or an empty string, which mean the current global object. - types/metadata-object-id: metadata object IDs are whole numbers, which absint() would otherwise quietly change. - escaping/esc-js-stays-in-string and escaping/esc-textarea-reads-back. - formatting/strip-all-tags-has-no-elements. - html/get-attribute-agrees-with-browser and html/decoder-agrees-with-browser: the HTML API reads attributes and character references as lexbor does. - blocks/serialized-blocks-parse-back: a round trip through the block parser. - wpdb/tables-are-prefixed: queries name only prefixed tables, as MySQL's grammar parses them. - option/names-fit-columns: option names, transient names and meta keys fit their columns, which MySQL would otherwise quietly cut short. - json/no-silent-failure: wp_json_encode() returns a string. - auth/check-password-accepts-own-hash: wp_check_password() accepts a password with its own hash. wp_hash_password() trims the password and wp_check_password() does not, so a password with whitespace at its start or end is rejected; this shows that. - kses/idempotent-when-reapplied, xml-output/well-formed, files/upload-names-safe and query/counts-consistent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…h the HTML API. WP_Invariant_HTML_API reads HTML with WordPress's tag processor, which follows the tokenizer of the HTML standard and, unlike Dom\HTMLDocument, gives byte offsets. It counts the markup in HTML, finds its text nodes, and finds where tags, tag names, attributes, attribute values, comments and DOCTYPE declarations start. It is independent of the regular expressions and string functions which the invariants check with it, but not of the HTML API itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…up, regex HTML parsing and path traversal. - kses/input-not-escaped: the text given to wp_kses() has no escaped tag or comment, such as `<p` or `&WordPress#60;/p`. - formatting/no-new-markup: wp_kses(), wpautop() and wptexturize() return no more of any tag, comment or registered shortcode than they were given, apart from the P and BR tags wpautop() adds. - pcre/matches-no-html-syntax: no match or group which a PCRE function in WordPress returns, and no replacement or split, starts where the HTML API finds HTML syntax, unless the pattern would start there without it. Each place is reported once. - files/validated-paths-stay-inside: a path validate_file() accepts never climbs above the directory it is read from. - filesystem/paths-stay-inside: WordPress gives the file system no path outside the directories it runs in. This hooks built-in functions, and WP_INVARIANTS_SKIP=filesystem leaves it out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ll hooks. The build adds hooks with the target `*`, which run for every user-space function declared in the files they are given, and fixes two ways post hooks were lost: for a function whose first call came from a hook, and for every call once enough generators had been called, whose frames were never removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…and returns. With WP_INVARIANTS_TYPES=1, as in CI, two catch-all hooks count the types of the arguments and return values of every function and method declared in src/wp-includes and src/wp-admin, apart from bundled third-party libraries. They are counted apart for calls from WordPress and from tests, and shown next to the native, PHPDoc and PHPStan types each function declares, in a second report, invariants-types.md and invariants-types.json, for adding and correcting type annotations. - `true` and `false` are shown apart until both are seen, then as `bool`. - Up to 10 distinct strings shorter than 32 bytes are shown as literals. - Arrays and stdClass objects are shown as records of their first-level keys and the types of their values, up to 64 keys, or as lists of the types of their elements. - Private functions and methods are included, and marked. - Tests which run in a separate process save what they counted for the report. WP_Invariant::functions_in() finds the functions and methods from the source, so that those which were never called are listed too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ger corrupts the heap, and give PHP all the memory. The pinned build of the extension read the arguments of trampolines, which stand in for __call() when an inaccessible method such as WP_List_Table::extra_tablenav() is called, from past the end of their frame. Its catch-all hooks copied what was there, which corrupted the heap partway through the suite. It also kept the frames of calls which threw, which held on to their arguments and gave them to later calls. The extension's ini also lifts PHP's memory limit, which WordPress only does once it loads, and the build step logs how much memory the container has. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… generators. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ive distinct ones are seen. A slot counts up to four distinct ints apart, such as `0` or `-1`, as it does with strings; a fifth distinct int widens them all to `int`. Floats are always `float`. Processes' counts are merged by the same rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…heir properties. Once a slot's ints or strings are no longer literals, the report splits them by the properties of their values, counted from the first one seen: - ints are `zero` or `non-zero`; - strings are `empty`, or have the most specific of `html` (the Tag Processor finds a token other than text), `ref` (an HTML character reference), `url` (an absolute WHATWG URL), `file` or `path` (probably a file path, which does or does not exist), `unslash` or `slash` (what wp_unslash() or wp_slash() would change) and `-utf8` (not valid UTF-8), or else `non-empty`. The JSON rows carry the tags too. The classification uses no PCRE, so it leaves preg_last_error() alone, and remembers recent short strings, but not whether their files exist. A URL with an opaque path, such as `mailto:a@example.org`, always parses, so it is not given to the parser: PHP 8.5's parser keeps the memory of each one until the request ends, which nearly doubled the peak memory of some groups. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ues, scopes of calls, paths and attacker-shaped values. - WP_Invariant_Request finds the values a request sent: the strings in the superglobals, the URI, query string and headers, and the parameters of the REST requests being dispatched, as sent and unslashed. - WP_Invariant_Security reads addresses as internal or not in any IPv4 or IPv6 form, the classes in serialized data and which could run code, and file names and contents which a web server could run. - WP_Invariant::frame_mark() and running() keep a scope for the run of a call without an after hook, which would not run when it throws. - The path helpers of the filesystem invariants move to WP_Invariant. - WP_Invariant::functions_calling() finds the functions whose own code calls a function, as targets, and WP_Invariant::cached() keeps such a result while the source does not change, sharing source_key() with the types report. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…L, SQL, unserialize() and email. - output/request-values-stay-text: a request value printed in HTML, or in the HTML of a JSON response, holds no HTML syntax there, as the HTML API reads the whole output: reflected XSS. - wpdb/request-values-stay-literals: a request value in a query sent to MySQL is inside one string literal or quoted identifier: SQL injection which never went through wpdb::prepare(). - serialize/no-foreign-objects: unserialize() gets no objects of classes with magic methods which this run did not serialize, and no request values: PHP object injection. - mail/no-header-injection: wp_mail() gets no addresses, subject or headers with line breaks which would add headers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…and a capability before they change anything. Inside an AJAX, admin-post or admin action handler, functions which change posts, options, users, terms, comments, plugins or themes must come after a nonce check which passed there, against cross-site request forgery, and after a capability check which passed, as must those in REST callbacks. A REST callback must also run as the user its permission check ran for. Public handlers, a user's own user meta, transients, cron, rewrite rules, salts, and writes inside other writes are left out. WP_INVARIANTS_REPORT_NAME names the report of a second run, for the AJAX tests, which the main run leaves out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ve entries written outside their destination. - uploads/nothing-executable: nothing written into the uploads directory has a name or contents a web server could run, such as photo.php.jpg, .htaccess or <?php. - archives/extract-inside-destination: while unzip_file() or copy_dir() runs, every path written is inside its destination: zip slip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… made with a secure random number generator. - secrets/never-sent: no salt, password hash, password reset or request key, session token, auth cookie or application password WordPress makes, and nothing shaped like a password hash, appears in a REST or JSON response, printed HTML, a wp_die() message, a header or the error log, unless the request sent it. - random/secrets-use-csprng: no mt_rand(), rand(), uniqid(), lcg_value(), str_shuffle(), array_rand() or shuffle() runs inside functions which make secrets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…s a field is already safe. While kses filters post content, excerpts and titles, comments, term descriptions or user descriptions, what reaches the database through wpdb::insert() or update() has no element or comment kses would remove, no event handler attribute, and no URL with a scheme kses does not allow: stored XSS through paths which skip the filters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…e their own output, and sites left switched. - http/validated-urls-stay-external: a URL wp_http_validate_url() accepts goes to the host it checked, as a WHATWG URL parser reads it, and not to an internal address in any form. - sanitize/idempotent: sanitize_key(), sanitize_title() and the other sanitizers return what they are given when given what they returned, and keep only the characters they promise. - multisite/blog-restored: a function which calls switch_to_blog() has restored the site by the time it returns. It runs only in multisite runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… purpose, and run them and the AJAX tests in CI. Each canary breaks an invariant through code which, with WP_INVARIANTS_CANARIES=1, counts as code under test, and checks that it is reported; its twin does the same safely, and checks that nothing is. They run with WordPress loaded, as `npm run test:php:invariants:canaries`. The workflow also runs the AJAX tests with the invariants, with a report of their own, since the request handler invariants mostly find their handlers there. The README describes the security invariants. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Strings which is_numeric() accepts, such as '42', '-1.5', '1e3' or ' 7', now have the property `numeric`, so that the report shows where a function documented as taking an int is given numeric strings, as get_post() is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ed markup out of two invariants. - pcre/matches-no-html-syntax no longer reports the PCRE calls which wpautop(), wptexturize() and wp_html_split() make themselves, since they are known to parse HTML with regular expressions. Calls from the functions they use are still reported. WP_Invariant_Call::caller_function() tells which function made a call. - kses/input-not-escaped now reports escaped markup in what wp_kses() is given only where an HTML escaping function returned it: PHP's htmlspecialchars() or htmlentities(), or WordPress's esc_html(), esc_attr(), esc_textarea(), esc_xml() or _wp_specialchars(). Escaped markup which content arrived with, as a user may type it, is left out. Canaries cover both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-advanced-security Bot found potential problems Nov 4, 2026

View reviewed changes

dmsnell added the invariant-tests label Nov 4, 2026

dmsnell force-pushed the testing/add-gameshark branch 2 times, most recently from ca22c3f to 89dafe4 Compare November 5, 2026 00:26