Proposal to update plugin star rating calculation

[JeffPaul]

I believe the existing star rating for a plugin (i.e., "4.2 out of 5 stars.") is based on the lifetime average of reviews. Since some of those reviews may be many years old and not reflective of more recent updates for a plugin, I propose that we consider an update to this calculation that takes into account the time since a review.

Instead of a simple average, I propose that we calculate a time-weighted average where newer reviews contribute more to the final score and older reviews gradually fade in influence.

A formula for this could be something like:

Time-weighted average = (sum of (rating x weight)) / (sum of weights)

...where the rating is each individual review (e.g. 1 - 5 stars) and weight is determined by how recent the review is.

Potential weightings to consider could be:

Reviews less than 6 months old: weight = 1.0
6-12 months old: weight = 0.75
1-2 years old: weight = 0.5
2-3 years old: weight = 0.25
3-5 years old: weight = 0.1

To ensure some statistical relevance and to try and avoid manipulation, we could only show the time-weighted average if a plugin has at least 10 reviews in the past 12 months ...OR... If it has at least 30 total reviews across all time periods. A potential fallback here could be something like "Not enough recent reviews to generate a current rating" and instead display the historical average as is currently displayed.

Some other potential adjustments to consider:

• Release-Based Reset Weighting: When a plugin releases a major version, consider re-weighting reviews so that pre-major-release reviews carry less weight. I'm not a huge fan of this as it potentially allows for manipulation and the fact that versioning is already a loaded topic (e.g., SemVer vs. WPver).
• Display Two Ratings: a Current Rating (i.e., time-weighted 12-month view) and a Lifetime Rating (i.e., simple historical average). This mirrors how Amazon sometimes displays "recent reviews" and "overall ratings".

Some of the benefits of an updated calculation are that it encourages plugin maintainers to improve their plugins, it prevents stale or no-longer-relevant reviews from skewing perception, and it can help users make better-informed plugin decisions.

I'm curious what folks think of an algorithm like this and if there are any iterations to the weighting time ranges, the weights, and the minimum review threshold... thoughts?

[codente]

Interesting proposal!

How would edited reviews/ratings factor in? Let's say I reviewed it years ago and then try it again and want to change my review/rating years later? Or even just confirm that my review/rating still stands.

[JeffPaul]

How would edited reviews/ratings factor in? Let's say I reviewed it years ago and then try it again and want to change my review/rating years later?

If a review was edited, then perhaps we take into account the date that it was updated and utilize that as the date for the time-weighted time range (instead of the older, original review date)?

[dd32]

@codente: Let's say I reviewed it years ago and then try it again and want to change my review/rating years later?

Realistically, edited reviews shouldn't be handled like they are today; see #7064

---

Duplicate of #1181, although things may have changed since I posted #1181:comment:25

Noting some thoughts I posted on #6851 as well:

Another suggestion is that more recent reviews can have extra weight applied to them, so instead of a review from 4 years ago counting the same as one yesterday, yesterdays review has a much higher impact on the overall score. That would allow for having a singular rating, but still allowing recent reviews to affect the overall score, while not negatively affecting those with few reviews.
I have previously looked into that, and it changed some plugin ratings significantly, but the majority were mostly unaffected (or only moved 0.1~0.5 stars).

Additionally:

The biggest problem with reviews-over-time is that the majority of the plugins have very few ratings, relatively, making it difficult to have a singular rating let alone multiple of them.

I'm going to close this as a duplicate of #1181 if you could both post your thoughts and revive that ticket, that'd be appreciative!