Opened 4 years ago
Last modified 4 years ago
#5275 new enhancement
Smarter nightly refreshing
| Reported by: |
|
Owned by: | |
|---|---|---|---|
| Milestone: | Priority: | normal | |
| Component: | General | Keywords: | 2nd-opinion |
| Cc: |
Description
It's not uncommon for external repositories to configure the PHPUnit test suite for WordPress to be copied directly from trunk.
For example, the install script used in the WP-CLI scaffold command follows this approach. If trunk or nightly is specified as the version, the test suite is copied using svn co directly from trunk. However, when downloading WordPress' nightly version, the https://wordpress.org/nightly-builds/wordpress-latest.zip package is used instead (assuming for performance reasons).
This causes any test setup configured using the WP-CLI scaffold command to fail until the nightly is refreshed any time files are moved or added in Core that tests rely on.
There were two situations where this came up this week:
- PHPMailer update, which moved the libraries files into a new directory.
- The XML Sitemap feature plugin being merged.
Manually refreshing the nightly fixed all issues reported by external projects.
There is an upstream issue for this on the WP-CLI Scaffold repo, but that would only apply to projects going forward, or ones that update their install script.
Is it possible to set up "smarter" nightly refreshing where the nightly package gets rebuilt automatically if files are added, moved, or the tests/phpunit/includes directory is modified? That would ensure issues are failures in external projects have the potential to fail for a much smaller window than files are shuffled around or added.
This is somewhat related to #1218. If the nightly was built more frequently, this problem would generally be resolved.
However, another way to look at this issue is that many CI setups for plugins and themes consistently rely on using the test suite files from trunk to test against the latest nightly build, which is created from a different point in history than the test suite files. Could we create a build of the test suite files at the same time that the latest nightly gets generated so folks could use that in their test suites instead of the files from trunk?