Add beta/RC offers to core version API check

[afragen]

@pbiron and I are implementing a version of the WordPress Beta Plugin that allows for the selection of the Beta/RC releases of either point or unstable. The idea is to allow Beta Tester users to stay on a beta/RC release and not get autoupdated to the nightly. Updates would only show when there is a new beta/RC release.

It would be much simpler if the API could return offers for beta-rc. Which would be the most current released beta/RC for the queried version.

[dd32]

The idea is to allow Beta Tester users to stay on a beta/RC release and not get autoupdated to the nightly. Updates would only show when there is a new beta/RC release.

While I have wanted that feature forever, I think it should be handled slightly differently:

• Nightly updates once >= Beta1
• Nightly updates once >= RC1

There can be significant bugs fixed between Beta's and RC's and getting those updates to those users ASAP will both improve their experience of the Beta/RC and also reduce the number of duplicate bugs being reported ("Ah yes, that's the 20th ticket this week, it'll be fixed in Beta2 which is delayed another week while we fix all these other bugs" - a scenario that shouldn't happen, but if it does, isn't great to the end-user.

[afragen]

@dd32 if you’re interested you can help beta test The branch of the Beta Tester plugin that has this feature.

The idea is that you can switch back and forth between getting updates from point, bleeding edge, and beta/RC for whichever stream you are currently on. Obviously there’s a lot we have to do to furnish the beta/RC download and having it as an offer on the API would make this more consistent and simpler.

It’s very simple to switch between the nightlies and the beta/RC. If you’re on the beta/RC when a new zip is created an update will show.

Ping either @pbiron or me on Slack and we can give you more details.

I don’t disagree that sometimes the nightlies are a better experience but many devs only are only testing against a beta or the RC. Hopefully this is a simpler way for that to happen.

[dd32]

Replying to afragen:

I don’t disagree that sometimes the nightlies are a better experience but many devs only are only testing against a beta or the RC. Hopefully this is a simpler way for that to happen.

Just because that's what people ask for, doesn't mean that's what people need. A lot of people may not realise that the nightlies between Beta's are just as stable / usable.

I'll have a think about it, and see if I can think of a easy way to make it work in the API code (It's not the cleanest code, and adding things like this can be complicated).

[afragen]

Just because that's what people ask for, doesn't mean that's what people need. A lot of people may not realise that the nightlies between Beta's are just as stable / usable.

Truer words I haven’t heard lately 😉

I'll have a think about it, and see if I can think of a easy way to make it work in the API code (It's not the cleanest code, and adding things like this can be complicated).

I understand and appreciate anything you might be able to do. Taking a peek at the Beta Tester branch maybe give you a better idea of what we’re trying to do and maybe your experience can help us.

[pbiron]

My reasons for wanting this are two-fold:

1. on the day betas/RCs are built when people are asked to test the packages before they are announced, most folks use wp-cli to install the packages. Not everyone is comfortable using wp-cli (or able, for whatever reason, to use it on a particular site they want to install the beta/RC package on). Being able to just click "Update Now" on the "WordPress Updates" screen would greatly increase the pool of people able to test the packages prior to release.
   • This goal would be satisfied by @dd32's "slightly different handling".

2. while this doesn't happen often, it has happened to me a few times: I install a beta/RC on a site and begin in-depth testing. I find something that doesn't appear to work correctly but it's not easy to figure out why and so it will take a few days to really dig in and investigate. When I do get back to it I can't reproduce what I thought was a problem and I spend considerable time trying to determine whether the problem disappeared because it was "data dependant" (e.g., only surfaces when there is a post with certain characteristics and in the mean time I've edited the post that showed the problem so that it no longer has those characteristics) or the site is on the bleeding edge nightlies stream and a commit after the beta/RC was released has fixed the problem
   • The fact that the problem was fixed by a subsequent commit that will be included in the next beta/RC is good for the project (as @dd32 points out), but being autoupdated to nightlies after the beta/RC is installed can complicate testing strategies
   • Hence, many times I want to keep a site on that particular beta/RC until the next one is released.
   • Of course, I (and others who are in my position) could simply disable the Beta Tester plugin on such sites after the beta/RC is updated to, but then we'd have to remember to re-activate it when the next beta/RC is released and it would be much simpler to just have a setting that says "don't autoupdate until the next beta/RC is available.

[pbiron]

As a side note: the fact that such a "big deal" is made when each beta/RC is released (the "release party" as they've been called recently, make/core posts for them, listing them on ​https://wordpress.org/download/releases/#betas, twitter announcements, etc) suggests that the WP project treats them differently (and is asking testers to treat them with "more respect") than the nightlies that are built on the days they are released (or there after).

So, if the thinking is that people should (almost) always be testing with the nightlies after beta1 is released, then I think the messaging around the beta/RC releases should be changed (toned down?).

[pbiron]

My thinking on what the change to the API would look like is: it would take an extra parameter (e.g., ?beta-rc=true) to cause it to return beta/RC package download URLs in the development and autoupdate offers and to not return those offers if, based on the ?version= param, the next beta/RC isn't available.

[dd32]

Replying to pbiron:

Being able to just click "Update Now" on the "WordPress Updates" screen would greatly increase the pool of people able to test the packages prior to release.

That's a good point, often that all happens before the nightlies are updated. "Fun" fact - The Beta/RC packages are never exposed by the update api's, only nightlies, that's part of what makes it hard to expose Beta/RC's directly. That also means that we'd be "releasing" the Beta/RC packages before it's actually 'intended' to be released.

I install a beta/RC on a site and begin in-depth testing. I find something that doesn't appear to work correctly but it's not easy to figure out why and so it will take a few days to really dig in and investigate. When I do get back to it I can't reproduce what I thought was a problem and I spend considerable time trying to determine whether the problem disappeared because it was "data dependant"

That is more about automatic updates than Beta/RC vs Beta/RC+Nightlies inbetween.
The scenario you describe would also happen if you didn't get back to it until after the next Beta/RC - which could be as long as a few weeks or as short as two days.

The other side of it, is that a user doesn't run into that problem in the first place, as they install Beta1+1 day which had fixed it ahead of them running into it.

It varies between releases, but I've seen lots of "annoying" bugs in my time that exist in Beta1/2 that get reported a lot with the same response of "Update to the nightly, or wait for Beta X+1 to be released next week" being given multiple times a day to affected users.

One initial thought that could help with your specific situation is Beta/RC's having a "living" changelog - A log of recent changes made since Beta1, a list of "known problems" (Defects that are still open on the milestone) - for developers who are testing the beta/rc to refer to.

Replying to pbiron:

My thinking on what the change to the API would look like is: it would take an extra parameter (e.g., ?beta-rc=true) to cause it to return beta/RC package download URLs in the development and autoupdate offers and to not return those offers if, based on the ?version= param, the next beta/RC isn't available.

That's mostly exactly what I was thinking, ?testing=beta and ?testing=rc and only returning >= $version+0.1-Beta or >= $version+0.1-RC.
It would even be possible to return these along-side the "current" version, so that the end-user had the option to install it through their Updates dashboard or not (ie. autoupdates disabled for next-branch, but keep getting their current branch automatic updates)

[afragen]

Closing in favor of #5412