Making WordPress.org

Opened 4 years ago

Last modified 4 years ago

#5196 new enhancement

New or enhanced rating system discussions

Reported by: rhialto's profile Rhialto Owned by:
Milestone: Priority: normal
Component: Plugin Directory Keywords:
Cc:

Description

I want to open up discussions for a better plugin's rating system. The current star rating can be abused by some devs who suddenly completly change a plugin which is suddenly not appreciated but still keep enjoying good reputation because of a lot of 5 stars rating given all the previous years. I think this issue should have been looked into a while ago but the good news is it's never too late to revisit the problem and think on ways to improve. WP is still going strong and will only keep growing with more plugins being added every week but also plugins being updated at any time. Any opinion on this? Ideas?

Change History (9)

#1 follow-up: @joyously
4 years ago

How would you prevent what you are calling abuse? What changes are you advocating?

#2 in reply to: ↑ 1 @Rhialto
4 years ago

Replying to joyously:

How would you prevent what you are calling abuse? What changes are you advocating?

I just wanted to bring this up for discussions, some may aldreay have good ideas on ways to improve. Do you agree the current system was never enhanced since introduction and could get some improvements?

#3 follow-up: @joyously
4 years ago

I'm not ever reading reviews or looking at star ratings because I'm a programmer, so I look at the code. But most people do look at the ratings.
It sort of sounds like you are suggesting some sort of sliding time frame that the ratings are calculated from, so that the rating average is from current ratings (within that time frame) instead of the lifetime of the plugin. This would more accurately reflect the current state of the plugin, but the way around it (since people will game everything) is to close the plugin and start a new one.
Perhaps if the rating was tied to a version number, but how do you average those or display them all separately?

#4 in reply to: ↑ 3 @Rhialto
4 years ago

Replying to joyously:

It sort of sounds like you are suggesting some sort of sliding time frame that the ratings are calculated from, so that the rating average is from current ratings (within that time frame) instead of the lifetime of the plugin. This would more accurately reflect the current state of the plugin.

Exactly this.

but the way around it (since people will game everything) is to close the plugin and start a new one.

Well as an example, that's what latest 1 star ratings are saying while complaining about latest changes on a plugin I was watching but I don't think it is actually possible to force a plugin author to return a plugin to a previous state and start a new one.

This seems an issue with quite a few plugins from comments I was reading here and there and that triggered me into looking for a solution and there am I after begin redirected from support forum to slack to here. :-)

Perhaps if the rating was tied to a version number, but how do you average those or display them all separately?

That's what I was also thinking about, but just like you I didn't know how to solve the puzzle. With all the talented people working on WP, no doubt someone will come up with propositions.

#5 @Rhialto
4 years ago

Now that I think about it, Amazon also use a star rating systems but with a bit more code... https://www.amazon.com/gp/help/customer/display.html?nodeId=GQUXAMY73JFRVJHE
There could be inspiration ideas from it? They also have to deal with product updates.

Last edited 4 years ago by Rhialto (previous) (diff)

#6 @dd32
4 years ago

See Also: #1181 (Use a Rolling Time Frame for more current reviews).

I did some back-of-the-napkin maths a while back as shown in that ticket, and in general aside for a handful of plugins there was very little benefit to the overall ratings by applying a 6month rolling timeframe for reviews.
Having more data points (reviews) per theme/plugin would make it easier to change something here.

#7 @conradish
4 years ago

Was about to start a new ticket on this. But spotted this.

Ok, so I just had a plugin "Stealth upgrade" an entirely new unwanted bloatware multifunction plugin over the original very popular lightweight single use plugin I had installed years ago. There wasn't really any clue in the "view version x.x.x details" or on the plugin page itself, though reading the latest 1 star reviews made it clear what had happened and that it was now very unpopular. The plugin team suggested I post here with an idea I had to help with this.

How about including a simple line graph on the plugin details page (and/or even the "view version x.x.x details" from installed plugins page) showing average review score over time? If say a seven day average was used, this would help show a sudden drop or rise in plugin experience for the user and immediately flag up that something might not be quite right.

Y axis would be star rating 1 to 5. X axis might be the last 12 months, or whatever works.

#8 follow-up: @Rhialto
4 years ago

You had me for a second, I use a plugin called Stealh Update and immediatly logged in to see if it was updated but yours is called Stealth Upgrade.

I like your idea except for the fact that when the update shows up in the appropriate section and you are just 1 click away to update you have no clue what is about to happen. Usually you think geat, an update! But then after update is complete you realize the horror. With a backup you can usually go back to a previous version but if we could find a way to raise a flag right before clicking the update button maybe?

On another similar note, when many 1 star comments suddenly show up for a specific plugin, why it does not pop a red flag for an inspection by external staff who can then maybe take appropriate actions with the plugin author if fault is found?

#9 in reply to: ↑ 8 @conradish
4 years ago

Replying to Rhialto:

You had me for a second, I use a plugin called Stealh Update and immediatly logged in to see if it was updated but yours is called Stealth Upgrade.


No, sorry, I meant the plugin did a stealth update; new plugin owner, new plugin functions all delivered under old plugin update.

I like your idea except for the fact that when the update shows up in the appropriate section and you are just 1 click away to update you have no clue what is about to happen.

When a plugin is due for an update you have two options "update now" and "view version x.x.x details". I always check the details, which shows the changelog, and I'm suggesting contains a graph that would highlight any changes in a plugins rating.

On another similar note, when many 1 star comments suddenly show up for a specific plugin, why it does not pop a red flag for an inspection by external staff who can then maybe take appropriate actions with the plugin author if fault is found?

Because this could be competitors leaving a bad review. Plugins could be easily sabotaged. But ultimately this is what I am suggesting we as users have, which will help all of us, to dig a bit deeper.

Note: See TracTickets for help on using tickets.