Making WordPress.org

Opened 7 years ago

Closed 6 years ago

Last modified 2 years ago

#2632 closed enhancement (worksforme)

Plugin Directory: Remove full output of last two reviews

Reported by: brianhogg's profile brianhogg Owned by:
Milestone: Priority: normal
Component: Plugin Directory Keywords: ux-feedback has-patch 2nd-opinion
Cc:

Description

The new design lists the full output of the last two reviews, which actually takes up a lot more real-estate than the unexpanded full description (ie. on https://wordpress.org/plugins/the-events-calendar-shortcode). The text of the last two reviews is not representative of the overall state of the plugin, especially when many will leave a negative review before even asking for help.

If the full review text must remain for some reason, then the full thread including any replies should also be added?

Attachments (1)

2632.diff (715 bytes) - added by brianhogg 7 years ago.
Patch to remove the plugin homepage full reviews section

Download all attachments as: .zip

Change History (28)

#1 @Ipstenu
7 years ago

  • Keywords ux-feedback needs-patch added
  • Milestone changed from Plugin Directory v3.0 to Plugin Directory v3 - Future
  • Type changed from defect to enhancement

Moving this to future, only because it's not mission critical to consider this rollout complete and ready for itterations.

Truncation like an excerpt would be best IMO. A decent balance and very similar to Amazon.

#2 @Otto42
7 years ago

It also doesn't necessarily have to be the most recent 2 reviews. Not that I have any workable algorithms for selecting quality reviews to display at present, but, you know, we could perhaps surface something else there.

#3 @lukecavanagh
7 years ago

Could quality reviews not be based on, the two last reviews that had a rating of 4 and above?

Last edited 7 years ago by lukecavanagh (previous) (diff)

#4 @brianhogg
7 years ago

Certainly showing positive reviews would be better than negative, though number of stars doesn't necessarily mean the text content of the review is positive.

Amazon and others have a "was this helpful for you?" rating system for the ratings/reviews, which lets them show the helpful positive (and negative) reviews. Without a similar system on wp.org, I feel they should not be shown on the main plugin page but be kept to showing the average, along with a link to the full listing of reviews where people can see the review along with the whole thread of any replies.

#5 @Shelob9
7 years ago

I'd really like to see this new feature removed. I don't think it helps the end user and it is ripe for abuse.

Last two reviews isn't a good algorithm for selecting a representative sample of reviews. I could see how 2 most helpful reviews or 2 most interesting reviews or 2 most upvoted reviews, or 2 reviews from my trust worthy users would be helpful. But this algorithm doesn't make sense.

Who it does help is those taking advantage of the blackmail support request -- reporting a bug as a 1 star review. This has been a tactic to game the system that has been abused for a long-time, and this new feature only helps. Also, it helps competitors of a plugin who can not insert any arbitrary content into the main plugin page of the plugin whose reputation they wish to harm.

So, I'd like to see this removed, with strong placement of the spread of reviews across the stars, and an average. For now, this doesn't help developers or end-users and only helps the trolls.

#6 @Shelob9
7 years ago

For example, a user tries out my plugin. It causes some sort of conflict. They open a support ticket on WordPress.org, with too little details for me to help them, I ask for more details, giving them the information they would need to find the error: https://wordpress.org/support/topic/pages-404-error-after-install-plugin/

Instead of responding, they elevate it to a blackmail support request: https://wordpress.org/support/topic/please-dont-install-it/ Not to say that my plugin isn't without bugs or conflicts, but I think it's safe to assume, we didn't get to 50k active installs by turning people's websites into 404s and our code isn't so horrible that it makes websites into 404s without even being installed. But this new design ads to my plugin page a warning that installing this plugin will make your website 404, even after you uninstall this.

Maybe, I have an over-inflated sense of the value of opinions of those who have been a part of the community for a long time, develop plugins or themes and contribute to WordPress core. But, I don't think the opinion of a user who registered for WordPress.org yesterday, as is the case here, should be given that kind of value. That doesn't seem like smart systems design.

So assuming that I'm really not that bad of a developer, who does this help?

Does it help those browsing for plugins? Nope, it's not a fair assessment.

Does it help my existing users? No. I know have to give this user more attention then I would someone who would help in hopes I can kiss their butt hard enough to get them to change the star rating.

Does it help my competitors? Yes, and that's fine, I like many of them, but not sure this is the point.

Does it help the user who is experiencing this conflict? Not anymore than opening the support request did, because I still don't know what is wrong and can't really help without knowing what errors they have or what version of my plugin they have.

Last edited 7 years ago by Shelob9 (previous) (diff)

#7 @pderksen
7 years ago

Another example of too much emphasis/real estate for just one user's negative review:

https://wplift.com/new-wp-plugin-repository-community-feedback (Reviews section down the the post a bit)

Screenshot: https://wplift.com/wp-content/uploads/2016/07/Stripe-Bad-review-691x1024.png

#8 @brianhogg
7 years ago

Possible for someone to post a dump of their local wordpressorg_dev? The current meta VVV install isn't installing a usable database with a single plugin page (and when I install I can't even get a blank plugin directory page, the table isn't created).

@Otto42 @Ipstenu @lukecavanagh

#9 @strangerstudios
7 years ago

Are +1 posts allowed on here? In any case, I completely agree with @brianhogg here (emphasis mine):

Amazon and others have a "was this helpful for you?" rating system for the ratings/reviews, which lets them show the helpful positive (and negative) reviews. Without a similar system on wp.org, I feel they should not be shown on the main plugin page but be kept to showing the average, along with a link to the full listing of reviews where people can see the review along with the whole thread of any replies.

A quick fix removing the reviews would be appropriate until a more thought out version is ready.

#10 @brianhogg
7 years ago

I just need either a dump of someone's wordpressorg_dev with some plugins that include reviews/ratings, or details on the format of the ratings post meta data so I can try to manually build it. Then I can try submitting a patch for this (and possibly other tickets).

The current repo included with the meta add-on for VVV might not be up to date. With the data and code I have, things like the ChangeLog are showing in full vs. with a 'read more', but maybe that's actually something that's been recently changed but not yet live.

#12 @pigeonhut
7 years ago

wow, is the only commnent to that last one

#13 @Ipstenu
7 years ago

If you just want to +1 or say 'wow' please use the "Watch this ticket" option instead. While we appreciate everyone's insight here, the more people reply to say "Me too!" the harder it gets to keep up with the productive end of the conversation. Basically you make it spammy :) It's unintentional, I know, but that's why we love the watch stars! That tells us "These are people who care about this."

There needs to be some sort of balance here. I agree, showing the last two is a bit unfair, because anyone can be subject to the Vote Mob Mentality that happens sometimes. Note: This goes both ways. I've seen far more cases of someone spamming their own reviews by incentivizing them in order to get all 5 stars than I have them being hit by competitors being jerks.

Looking at Amazon, I see "top" reviews and IIRC those are calculated by a combination of recent + 'this review was helpful/unhelpful' marks. In the long run, I think if we added that kind of functionality, which would allow users to flag reviews as helpful/unhelpful would be a better approach. but... Even with that in mind, I see a lot of Amazon reviews that have a 'top' of low-starred reviews, which tells me that ... people are people.

Now. We're pretty much all in agreement that this can be better. What we need to figure out is a workable and more just algorithm or even an idea of what shape this would be to make things fair.

What won't work is allowing developers to pick the reviews. Sorry but that's too ripe for abuse and we have more trouble than it's worth today with developers using unfair and unethical practices for garnering more reviews. Too many devs have spoiled that one.

We don't currently have a way to track helpful/unhelpful reviews, though, which means we're starting from ground zero. Otto, is that even a possibility?

#14 @samuelsidler
7 years ago

One idea that @dd32 mentioned is to look at the average rating and show a similarly or greater-than rated review. For example, if a plugin is rated 3.5 stars, show the most recent >3.5 rated reviews. We'd probably need to put some limits on this, like only do this when a plugin has >10 ratings or something, but that might be a good starting point.

Is anyone interested in working through some ideas, perhaps in a meeting in the meta channel? We can start with what the ideal would be, work backwards from there, and also look at more immediate solutions if the ideal isn't feasible in the short term. (For example, rating reviews (up/down voting reviews, that is) probably isn't feasible at the moment, for technical reasons.)

I think it'd be best to move this conversation to a meeting (or two) so the discussion is easier to follow, then write a proposal post here, or on make/meta.

#16 @brianhogg
7 years ago

@samuelsidler certainly interested in coming up with some ideas on #meta but for now, in the absence of any fair algorithm, can this be removed? I'm still trying to get a WP meta environment set up so I can submit a patch, if someone can share their local wordpressorg_dev table with some reviews added to one or more plugins I can do that, and be able to help submit other patches.

Yet another example of this happening, on my plugin from someone who never asked for support and is now the top review:

https://wordpress.org/support/topic/customizability-2/

#17 @Ipstenu
7 years ago

Brian, in lieu of having the database (or a test DB which we really should have...), I can only recommend you do what I did and fake it. That is, I created a dozen plugins and populate it manually. It took some trial and error, but it did work enough to give me a test environment so I could make a few patches. This is one of the changes you can find and test without a full DB.

It shows up as the first review, not the 'top' review. I'm not trying to be pedantic, and I understand what you mean. I just want to make sure to use the clearest English I can so we're all on the same page :)

Last edited 7 years ago by Ipstenu (previous) (diff)

#18 @Shelob9
7 years ago

I really like the proposal that Sam has made. I think it's the best possible system without major new features. I think it would be a good compromise.

I would like to suggest that it be random, not most recent in the range allowed. I'd be happy to discuss further in #meta if someone proposes a time.

I also agree strongly with Brian that the reviews display should be temporarily removed to avoid feeding the trolls further.

#19 @brianhogg
7 years ago

@Ipstenu no worries on the wording :)

I do have several plugins but there's no way to add reviews without hacking something into the post meta, that I can see. The link to add a review is hard-coded to go to wordpress.org and there's no wordpressorg.dev/support in the WordPress meta VVV environment anyway

I can dig around the code further to see what that format should be, if I can't get it from someone who has it locally (ie. the rating and ratings post meta for a plugin, with associated user ID of the reviewer).

I'm also wondering if what's in github and/or the WordPress meta has other inconsistencies, since there's some odd things like the ChangeLog not having a read more link on my local environment.

Last edited 7 years ago by brianhogg (previous) (diff)

@brianhogg
7 years ago

Patch to remove the plugin homepage full reviews section

#20 @brianhogg
7 years ago

  • Keywords has-patch added; needs-patch removed

Added a patch to remove the full reviews display until we can get a new system in there (or leave out indefinitely and use the sidebar widget display). Possible for someone to commit?

To test I manually added a wporg_419_posts and ratings table along with a couple user accounts for the reviews to be associated with. This can be found in a dump of my local wordpressorg_dev database here:

https://raw.githubusercontent.com/brianhogg/wordpressorg_dev_meta_database/master/wordpressorg_dev.sql

#21 @jb510
7 years ago

More than a +1. Three related thoughts here.

First, any "two reviews" is going to be a problem whether chosen by chronology or by approximate rating (Sam's suggestion). I think most would agree that the build of reviews are unfortunately crap. Hence the odds are that "any two reviews" selected by either means are going to be crap. The really useful reviews aren't the ones that are near the mean or most recent. They're the ones that say something meaningful to the person looking at using the plugin. The only way to make "two reviews" helpful to rank them by helpfulness. That is to do a "This review was Helpful / not helpful" system and then highlight the most helpful positive and negative reviews. Essentially clone Amazon.

Second, when one does click through to "Read all XX reviews" they get a very unfriendly BBPress list of titles with no excerpts, so they now are presented with less useful information than before which they're clicking in/our of endless to find useful information. I know this is outside the scope here, but excerpts, at least the first sentence would be super helpful on that page, along with "helpfulness" rankings.

Third, what I really think should happen is the review get pulled out of the main column and put into the sidebar below the progress bars (5/4/3/2/1 star review bars). Then below the progress bars show the most helpful positive and critical review. As an added bonus this would also help with the excessively long single page format.

In the mean time, I'd take the reviews w/ excerpts completely of the single page view. Just show a link to "read more". It's not ideal, but at least it's not misleading.

#22 @alvarogois
7 years ago

I totally agree with @jb510, reviews are useless or even counter-productive as it is. Not to mention quite unfair to the plugin developer when the last two are just rubbish or trolling. It should be out of the single page view.

The review system in itself is difficult and there're lots of examples of people who just don't know how to use it or use it as a support/complain system instead. The reviews only have meaning beyond the visual impact if you care to read it and assess.

#23 follow-up: @Cybr
7 years ago

I'd say that we should get the option to pick out and therefore pin two reviews of our own choosing.

I kinda like the idea that they're displayed in the plugin's home context :).

With this, maybe even change the title "Reviews" to "What others say".

#24 in reply to: ↑ 23 ; follow-up: @jb510
7 years ago

Replying to Cybr:

I'd say that we should get the option to pick out and therefore pin two reviews of our own choosing.

and the next day 90% of authors write one sock puppet review and select it... It's got to be crowd sourced.

Last edited 7 years ago by jb510 (previous) (diff)

#25 in reply to: ↑ 24 @Cybr
7 years ago

Replying to jb510:

and the next day 90% of authors write one sock puppet review and select it... It's got to be crowd sourced.

Good point! Back to square one :D

#26 @tellyworth
6 years ago

  • Keywords 2nd-opinion added

#27 @obenland
6 years ago

  • Milestone Plugin Directory v3 - Future deleted
  • Resolution set to worksforme
  • Status changed from new to closed

Since the ticket was opened reviews were moved into a separate tab, with now six reviews shown in their entirety, maintaining the link to all reviews. While I'm sure the reviews tab could be improved further, I think that has addressed the most immediate issues expressed here. Let's circle back to this when the single plugins page undergoes its next iteration.

Note: See TracTickets for help on using tickets.