feat(performer): run WP-CLI via php with an opcache file cache - #72
Open
luismulinari wants to merge 1 commit into
Open
feat(performer): run WP-CLI via php with an opcache file cache#72luismulinari wants to merge 1 commit into
luismulinari wants to merge 1 commit into
Conversation
Without -fpm-url, every WP-CLI invocation spawned a fresh php process that
recompiled WordPress and every plugin from source. On a large customer
codebase that compile step was ~0.7s of a 1.4s list-due-batch call, which
is the reason the php-fpm shim was introduced in the first place.
processCommand now always executes:
php -d opcache.enable_cli=1 -d opcache.file_cache_only=1 \
-d opcache.file_cache=<tmpdir>/cron-control-runner-opcache \
<wp-cli-path> <args...>
so each process loads precompiled opcodes from disk. Timestamp validation
is left at its default so a changed file is always recompiled. The cache
dir lives under the OS temp dir and is created at startup.
Measured on a dev-env site (p50 per list-due-batch): plain wp-cli 221ms,
php-fpm shim ~130ms, wp-cli with file cache ~93ms. On a large codebase
the plain call dropped from 1.39s to 0.71s.
The FPM path and remote WP-CLI are unchanged. Running php explicitly also
lets -wp-cli-path point at an extracted wp-cli boot-fs.php, which avoids
the per-call phar open and signature check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
The main idea behind this PR is to remove the complexity of the PHP-FPM path. Today the runner invokes WP-CLI through a FastCGI shim (
processCommandWithFPM→/var/wpvip/fpm-cron-runner.php) that fakes a CLI environment inside an FPM worker. That shim exists for one reason: it was the only way to get opcache for WP-CLI, since a freshphpprocess recompiles WordPress and every plugin on each call (Automattic/Cron-Control#208).It keeps causing trouble, because customer code runs inside the response path: long headers wedging the response read (PLTFRM-2424, #62), output escaping
ob_start()(Cron-Control#211), FPM worker limits and timeouts applying to cron, and exit codes turned into HTTP statuses. None of that is fixable while cron runs through FPM.This PR makes the plain
execpath fast enough that the shim is no longer needed, so it can be turned off (-fpm-urlempty) and later deleted along with thegofastdependency, the two FPM flags, the FPM metric, and the devcontainer shim.What changes
Without
-fpm-url,processCommandnow always runs WP-CLI as:Each short-lived process loads precompiled opcodes from disk instead of compiling from source. Timestamp validation is left at its default, so a changed file is recompiled and the cache never serves stale code. The cache directory is created under the OS temp dir at startup; if that fails, the error is logged and php runs uncached.
phpis resolved fromPATH, the same way thewpshebang already does it.-wp-cli-pathmay also point at an extracted wp-cli tree'sphp/boot-fs.php, which skips the per-call phar open and signature check (~10 ms).remote.go) are untouched.Measurements
Testing
go test ./performer/ ./orchestrator/(new tests cover the exact argument list, thePATHdefault, cache dir creation, and aprocessCommandround trip through a fake php shim, so php is not required on CI).go build ./...needsGOOS=linux; the darwin failure inremote.gois pre-existing.To try it locally: run the runner without
-fpm-url, then check that$TMPDIR/cron-control-runner-opcachefills with.binfiles after the firstlist-due-batch.Follow-ups
-fpm-urlempty on one deployment and compare the existingget_site_events/run_eventlatency histograms.processCommandWithFPM, the FPM flags and metric,gofast, and the devcontainer shim once the soak looks good.