WordPress.org

Making WordPress.org

Opened 3 years ago

Last modified 2 months ago

#1181 new enhancement

Enhancement for Reviews/Ratings: Use a Rolling Time Frame for more current reviews

Reported by: cais Owned by:
Milestone: Priority: low
Component: Support Forums Keywords:
Cc:

Description

Currently the reviews of plugins (and themes for that matter) go back to a static point where there is also a comment that was made with the review. This is helpful but I believe it would be more beneficial to have the review be on a rolling time frame, perhaps matching the automatic closing of support topics without activity (one year?)

Also to note, plugins that have not been updated in two years or longer will not appear in general search results. Given this idea, an old plugin with very strong reviews could be misleading to users with its quality and stability if used with current WordPress versions.

Given how rapidly plugins, themes, and the general WordPress environment in which they operate changes, putting the focus on relatively current reviews would ensure that they more accurately reflect current state and performance of themes and plugins.

Change History (22)

#1 @scottwyden
3 years ago

This is a fantastic idea!

#2 @Otto42
3 years ago

Not sure I'd go for one year.

Old doesn't mean broken. WordPress strives for backwards compatibility, and the fact that lots of 5-6 year old plugins still work perfectly is proof of that. Yes, there is some breakage, but if a plugin is actually broken, we'd rather remove it from the listings instead of simply making everything "old" hidden.

The 2 year standard for listing has worked relatively well, but reviews don't necessarily seem to have that apply to me. A review is still a legit review, even if it's two years old. Making ratings "expire" doesn't seem to me that it would actually surface better results.

#3 @cais
3 years ago

I'm not saying the review was not legitimate when it was written, but if there have been updates and improvements to the plugin (or theme) after the fact that would have that review no longer be "correct" then a rolling time frame would help to improve the review (and rating) results.

I see current reviews as more relevant than dated reviews, especially for current plugins ... I'd even accept a rolling time frame dating back from the last release of the plugin.

#4 @Otto42
3 years ago

Well, we do show most recent reviews first, although I'd like to change that a bit and try to surface better reviews. Substance over date, sort of thing.

#5 @cais
3 years ago

Substance would obviously be the key; and I see the date as being relevant to how substantial the review is ... I'm not so sure they can be separated and still hold the same value.

#6 @deconf
2 years ago

I feel like counting latest X reviews will better reflect the current state of plugins. In my opinion "expired" reviews should stay in place but not count.

A review is still a legit review, even if it's two years old.

@Otto42 A user can always update his review. By updating it, he can make his opinion count again. The advantage is that his updated review will reflect the current state of the plugin, instead of a two years old one.

The ability of leaving new reviews or updating old ones also covers the case of old plugins. It really doesn't matter how old the plugin is, the reviews will always show its current state.

Counting latest X reviews may also help in cases where plugins with lots of positive reviews fall into the hands of wrong people or choose a wrong path.

My worry about counting reviews based on time frames is that in some cases the number of reviews could be so low that it won't really reflect the truth about a plugin.

Last edited 2 years ago by deconf (previous) (diff)

#7 in reply to: ↑ description ; follow-ups: @SergeyBiryukov
15 months ago

  • Keywords reporter-feedback added

Replying to cais:

Currently the reviews of plugins (and themes for that matter) go back to a static point where there is also a comment that was made with the review. This is helpful but I believe it would be more beneficial to have the review be on a rolling time frame, perhaps matching the automatic closing of support topics without activity (one year?)

As noted above in comment:6, a user can update their review at any time to reflect their recent experience.

Is there any other particular changes that should be done here?

#8 in reply to: ↑ 7 ; follow-up: @cais
15 months ago

  • Keywords reporter-feedback removed

Replying to SergeyBiryukov:

As noted above in comment:6, a user can update their review at any time to reflect their recent experience.

Although my review philosophy is not in question, I'm fairly certain I have written at least one review in all my years working in and around WordPress although I have no way to easily find it since it has been more than two years since one was written and my "activity" report only goes back two years(?!)

Granted a user may be able to update their review and/or change their star rating (up or down as their want), making those reviews older than two years still be considered while not easily found with the current interfaces does present challenges thus supporting the point of using a rolling time frame.

That is not to say older reviews, especially for the example of utility plugins without an update in 5 or 6 years (as examples, such as noted above) are not valid or useful, they are more edge cases in this scenario; and, if I understand the search functionality of the plugin directory these same older plugins would not be found unless their specific name is searched. This would take them out of the equation as far as comparing current plugins offering the same functionality where many potential users have little more to decide with other than recent reviews and current star ratings.

#9 in reply to: ↑ 7 ; follow-up: @samuelsidler
15 months ago

Replying to SergeyBiryukov:

As noted above in comment:6, a user can update their review at any time to reflect their recent experience.

While users can, anecdotally they very rarely actually update their reviews.

Is there any other particular changes that should be done here?

I think a rolling timeline makes sense. We might want to consider different timeline lengths, depending on active usage. For example, if a plugin has 1 million+ active users, a shorter rolling window might make sense, whereas a plugin with 10,000+ active users might get a longer rolling window (due to fewer reviews).

@joostdevalk and I discussed this recently and I know he has thoughts on it.

#10 in reply to: ↑ 9 @joostdevalk
15 months ago

@joostdevalk and I discussed this recently and I know he has thoughts on it.

To be honest, I think we should mimic the App Store model. If a plugin hasn't had updates for 3 years, a 3 year old review can still be valid. However if a plugin has been updated every 2 weeks, a review written 6 months ago might already be completely invalid.

So, I'd suggest we start recording a version number next to each review. We keep a record of the last X version numbers and only show reviews for those X versions. Of course this would allow plugin developers to game the system by doing X releases to get rid of 1* reviews. So maybe we should add some more... Anti gaming rules.

#11 follow-ups: @samuelsidler
15 months ago

Hm, good point. Perhaps a mix of releases and time? One thing to consider is that we don't necessarily know what version a user is reviewing when they leave a review, unlike the App Store.

#12 in reply to: ↑ 11 @joostdevalk
15 months ago

Replying to samuelsidler:

Hm, good point. Perhaps a mix of releases and time? One thing to consider is that we don't necessarily know what version a user is reviewing when they leave a review, unlike the App Store.

We should probably fix that too. Would fix a whole lot of sock puppeting if people could only review a plugin they had actually installed on their site...

#13 in reply to: ↑ 11 @SergeyBiryukov
15 months ago

Replying to samuelsidler:

One thing to consider is that we don't necessarily know what version a user is reviewing when they leave a review, unlike the App Store.

Related: #152

#14 in reply to: ↑ 8 ; follow-up: @SergeyBiryukov
15 months ago

Replying to cais:

I'm fairly certain I have written at least one review in all my years working in and around WordPress although I have no way to easily find it since it has been more than two years since one was written

Related: #2007

#15 in reply to: ↑ 14 @photocrati
15 months ago

Replying to SergeyBiryukov:

Replying to cais:

I'm fairly certain I have written at least one review in all my years working in and around WordPress although I have no way to easily find it since it has been more than two years since one was written

Related: #2007

:grin:

#16 follow-up: @SergeyBiryukov
15 months ago

Replying to cais:

I'm fairly certain I have written at least one review in all my years working in and around WordPress although I have no way to easily find it since it has been more than two years since one was written

#2007 is now live: https://wordpress.org/support/users/cais/reviews/

#17 in reply to: ↑ 16 @cais
15 months ago

Replying to SergeyBiryukov:

#2007 is now live: https://wordpress.org/support/users/cais/reviews/

Thanks for the update! Nice to see that these reviews are editable even going as far back as the few I have written although I had to look at the reviews several times to see where to edit (I know it's a great big blue banner ;) ... it just didn't register as the interface is different than the standard forums view of reviews).

#18 @SergeyBiryukov
12 months ago

Replying to samuelsidler:

One thing to consider is that we don't necessarily know what version a user is reviewing when they leave a review, unlike the App Store.

#152 has a patch to address this, anything I can do to help move it forward?

#19 follow-up: @Otto42
11 months ago

One thing to consider is that we don't necessarily know what version a user is reviewing when they leave a review, unlike the App Store.

A consideration brought up is that this is highly subject to gaming via artificial version inflation. People will bump versions of apps specifically to "reset" their reviews.

#20 in reply to: ↑ 19 @SergeyBiryukov
11 months ago

Replying to Otto42:

A consideration brought up is that this is highly subject to gaming via artificial version inflation. People will bump versions of apps specifically to "reset" their reviews.

Yes, this was mentioned in comment:10 as well.

The patch on #152 doesn't reset reviews though, it just records the plugin/theme version for support topics and reviews, which would make support a bit easier, and might also come in handy for the new plugin developer dashboard that has been discussed at the latest Plugins Team meeting.

Having the versions data, we can decide on the best way to use it while preventing plugin developers from gaming the system. Otherwise, both #152 and this ticket can be closed as maybelater or wontfix.

This ticket was mentioned in Slack in #forums by clorith. View the logs.


2 months ago

#22 @Clorith
2 months ago

  • Priority changed from normal to low
Note: See TracTickets for help on using tickets.