Making WordPress.org

Opened 7 years ago

Last modified 6 weeks ago

#2753 new enhancement

Plugin directory search sorting and filtering

Reported by: tellyworth's profile tellyworth Owned by:
Milestone: Improved Search Priority: high
Component: Plugin Directory Keywords: needs-design has-ui-feedback needs-screenshots has-patch
Cc:

Description

We bumped sorting and filtering from the v3 launch, so it's time to revisit.

Implementing both is straightforward on the back-end, the main issue is the UI: where to put it, how to make it helpful and accessible. This is a general ticket for discussion and progress.

Attachments (4)

mock-search-results-sort.jpg (106.9 KB) - added by joyously 7 years ago.
How about if the sort is very simple, at the top.
mock-Search-results-sort-mobile.jpg (57.2 KB) - added by joyously 7 years ago.
And a sort control on a small screen
mock-Search-Results-Refine.jpg (174.1 KB) - added by joyously 7 years ago.
The filters could be hidden until the Refine Search button is clicked, at which point they appear in an overlay (could slide out from the side or whatever).
mock-Search-Results-Refine-mobile.jpg (87.3 KB) - added by joyously 7 years ago.
The filter overlay on small screen.

Download all attachments as: .zip

Change History (74)

#1 @gibrown
7 years ago

Some proposals for the scope of this change.

There are a number of open issues that involve browsing, sorting, or finding plugins that are orthogonal to just searching. Here's a list of the ones I found:

  • #2624 : view all the plugins for a particular author
  • #2631 : find recently updated plugins
  • #2653 : /browse/new and /browse/updated and /browse
  • #496 : /browse/popular - this one is really about deep paging

I'd like to propose that we solve all of these by adding faceting/filtering and then redirect from those paths to the search with the appropriate parameters for sorting/filtering. Some of these are kinda narrow use cases, but there is some overlap between them all.

List of features we need to add to search to satisfy all of these:

  • matching all plugins with search. If the search is blank then we run a match_all query and get back everything based on whatever filters/sorting are set.
  • filtering by author login
  • descending sort by update date range
  • either descending sort by plugin first publish date, or a filter on plugins created in the last three months
  • descending sort by active_installs or rating, or a combo. (or maybe the default alg with a match_all will work ok?)

Those use cases drive some decisions on what to allow sorting on. I'd suggest we support three cases (there are dozens of possibe fields we shouldn't add too many):

  • Current sort. More or less most popular, though maybe we should be stronger and call it "recommended" or something? It pretty heavily weights getting users support and plugins that stay up to date.
  • Recently updated - sort by plugin_modified
  • Average Rating descending - this should probably be a combo of number of reviews and ave rating so a single 5 star rating does not put a plugin above one with a 4.9 rating from 1000 users. #2686 has some interesting thoughts in it.

I left out "sort by date_gmt" with the thought that a filter can solve that and having a filter for that would make for both a better user experience and help new plugins find users. I was thinking of something similar to this Amazon "New and Upcoming" facet

https://cloudup.com/cUSzYLxRarc

It feels like it does a better job at inviting the user to click than a sorting option would because it gives you a promise that there is something inside.

Some other proposed facets:

  • Average rating: (just stolen from Amazon) 1 and up, 2 and up, 3 and up, 4 and up.
  • WP version: 4.8, 4.7, 4.6, ... (probably only show the 4 most recent versions though the filtering through url hacking should support any version)
  • Active installs. We should probably adjust our bins from what the plugins use. Maybe 100,000+, 10,000-99,999, 1,000-9,999, 100-999, 1-99
  • Percentage of support threads resolved: 75%+, 50-75%, 25-75%, 0-25%
  • Category: there was past discussion about things like free, paid, etc which it sounds like is desirable, but not yet defined. If we define it we could add them here.
  • Percentage of plugin strings translated into the current locale - This data is not in the index, but it does exist: https://translate.wordpress.org/locale/es/default/wp-plugins?s=jetpack

That all kinda feels like a lot to me and I wonder if we should cut some to simplify for a first release and see how it goes (eg do we need WP version? Doesn't feel like it would mean much to an end user). Other fields I've thought about:

  • Authors - would be a lot of noise. We should support filtering on it though and fix the author search
  • Tags look really noisy from what I have seen so far.
  • Total number of updates a plugin has had - kinda interesting to exclude plugins that rarely get updated
  • How many months has the plugin been getting updated for - maybe a better version of the above because less easy to abuse, but I'm not sure if it is useful enough

Other ideas?

A note on deep paging. Elasticsearch imposes a paging limit of 10,000 results. I don't really think there is a good use case for going that far, but 10k seems to work ok and not break so I don't care. We will start to hit this though if we are matching on everything. I think that is fine given that we are now giving the user a bunch of other ways to sort and filter through results.

For mobile, do we just hide the faceting sidebar? At most allow a sorting option? Or maybe the sidebar floats to the bottom of the page?

Some thoughts on timing. This is both a significant and also not that hard project. It is significant in that we need to change the design to add a sidebar, a sorting option, and add a bunch of url param handling. It is not hard in that there is a bunch of code for doing faceting and filtering that is destined for Jetpack in the next month or so that we can build off of. If we make some decisions around design over the next month then I can push ahead with the implementation in a pretty straightforward manner (once we update Jetpack).

This ticket was mentioned in Slack in #meta by gibrown. View the logs.


7 years ago

#3 @gibrown
7 years ago

  • Keywords ui-feedback needs-screenshots added

This ticket was mentioned in Slack in #meta by ipstenu. View the logs.


7 years ago

@joyously
7 years ago

How about if the sort is very simple, at the top.

@joyously
7 years ago

And a sort control on a small screen

@joyously
7 years ago

The filters could be hidden until the Refine Search button is clicked, at which point they appear in an overlay (could slide out from the side or whatever).

@joyously
7 years ago

The filter overlay on small screen.

#5 @joyously
7 years ago

@gibrown, I put the filters that made sense to me into a mockup image (see attachments).
I am of two minds on how the filters should behave. One is that each choice is a link (parameterized search) so the page reloads on each. The other is that it is a form that you set all the parameters and then submit once.
When I have used these types of filters, my preference is one submit, but people with fast connections probably prefer each click to change the page.

#6 follow-up: @joyously
7 years ago

One thing that is unclear is whether it is different code that populates the plugin search results in the admin versus the page that looks the same on wordpress.org.
Does this ticket cover both contexts?

#7 @gibrown
7 years ago

Thanks for adding some mockups. Helps to think through some of the complexities.

For sorting:

  • I'm going back and forth a bit on the word "popularity" and wondering if we should just say "relevance" which is at least open to interpretation
  • I'm not sure what sorting by plugin name would be? Alphabetical? What use cases does that solve for us? Also, we won't always be able to page through all of the results, so I think it would be a broken experience

Looking at the original design some more, I don't really like that the search box is in the upper right of the current design. Makes it hard to find and therefore hard to adjust. Feels to me that it should be in the center of the page, and much bigger than the text that says: "Showing results for:"

We also need someplace to display any filters that have already been added to the search. Maybe just below a more prominent search box with a set of filters using something like https://select2.github.io/examples.html#tags (though maybe something better than that).

For faceting, I think we should have the facets on screen at all times on large screens. That may mean reworking how the current results are displayed, but should make the search more browsable and interactive. I guess on small screens the faceting will have to slide in, or not exist at all.

I'd also love to just skip straight to doing a React UI for the search interface to make interacting much faster, but that would make the project a fair bit bigger I think (or at least it would make it harder for me to implement). I was looking at these for instance: https://react.rocks/tag/Faceted_Search

#8 in reply to: ↑ 6 @gibrown
7 years ago

Replying to joyously:

One thing that is unclear is whether it is different code that populates the plugin search results in the admin versus the page that looks the same on wordpress.org.
Does this ticket cover both contexts?

It is different code. The core plugin search is separate, but it uses the same backend algorithm. I think we should focus on iterating on the design here before looking to improve the UI in core.

This ticket was mentioned in Slack in #meta by gibrown. View the logs.


7 years ago

#10 @gibrown
7 years ago

We had some discussion about this in the plugin dir meeting yesterday. Want to capture two potential options for how to do implementation:

1.

Use the new Jetpack search sidebar widget to provide faceted search and filtering on the search page: https://github.com/Automattic/jetpack/blob/master/modules/search/class.jetpack-search-widget-filters.php

We'd also want to implement the appropriate sorting of results and redesign the layout of the page so it works on mobile.

This is probably the easiest path since it uses a lot of already existing code. An example of filtering and results is here: https://vip.wordpress.com/?s=search&category=case-studies&year=2013&monthnum&day

2.

A React/JS app that let's users interactively refine their searches. I had build something like this in backbone a while ago (https://greg.blog/2012/08/20/quickly-build-faceted-search-with-elasticsearch-and-backbone-js/), but that wouldn't work very well here. Ideally this would be a flexible UI that could be used in other situations also.

This would let us build a much more modern search interface: search as you type, fast responses to changes, etc. But it would also be a fair bit more work since it is all new development. Almost certainly needs some additional help.

We would also need to iterate on the rest api a fair bit. The other benefit to building a great React/JS app here is that it could be reused in wp-admin where most users do their searching for new plugins. This would be a big benefit.

Last edited 7 years ago by gibrown (previous) (diff)

This ticket was mentioned in Slack in #design by joyously. View the logs.


7 years ago

#12 @joyously
7 years ago

@gibrown How about like this site's search and filters? https://www.carvana.com/search

This ticket was mentioned in Slack in #core-php by joyously. View the logs.


7 years ago

This ticket was mentioned in Slack in #meta by sergey. View the logs.


7 years ago

This ticket was mentioned in Slack in #core by joyously. View the logs.


7 years ago

This ticket was mentioned in Slack in #core by swissspidy. View the logs.


7 years ago

This ticket was mentioned in Slack in #design by joyously. View the logs.


7 years ago

This ticket was mentioned in Slack in #design by joyously. View the logs.


7 years ago

This ticket was mentioned in Slack in #design by joyously. View the logs.


7 years ago

#20 @boemedia
7 years ago

We discussed this ticket yesterday in the Design meeting. We think the plugin directory could hugely benefit from a sort & filter feature. I have taken a look at the mock ups in this ticket and here are my findings:

  • the way the filter and sort buttons are designed, aren't coherent. They have a different look and feel and are too distributed on the page
  • the term refined search could be changed into 'filter', since this is a common and well known term for this.
  • the filter options are now presented in a pop up. This is not ideal and I don't think this gives a good user experience. We have to ask the accessibility team for their view on this too

As for the filter options themselves:
Not sure if a filter on 'support threads resolved' would be something users look for. What filters could be interesting?

  • topic (like security, seo, user management, etc.). Someone is looking for a plugin with a specific purpose
  • last updated - we don't like plugins that aren't updated over 3 years (maybe combined with tested with versie x.x)
  • rating (although not sure if this should be a filter option; may work as well as a sort option, just like no. of installs)

This list could be extended, although we should consider that the more filter options people have, it's not always getting clearer at the same time.

For inspiration, we could look at what's done at the themes directory: https://wordpress.org/themes/tags/accessibility-ready+photography/

I also think the booking.com filters are something we could look at, since they've done a lot of UX research: https://www.booking.com/searchresults.en-gb.html?label=gen173nr-1FCAEoggJCAlhYSDNYBGipAYgBAZgBHLgBB8gBD9gBAegBAfgBC5ICAXmoAgM;sid=cc87f6e69f36f712aa9721a1c62012e1;city=-2602512&;ilp=1;d_dcp=1

What I like about the booking.com filter over the themes filter is, that it's there all the time. You can filter and sort from the same screen. However, the page is not mobile friendly, which is something I really like on the themes page.

Last edited 7 years ago by boemedia (previous) (diff)

#21 @boemedia
7 years ago

  • Keywords has-ui-feedback added; ui-feedback removed

This ticket was mentioned in Slack in #accessibility by boemedia. View the logs.


7 years ago

#23 @boemedia
7 years ago

  • Keywords needs-design added

This ticket was mentioned in Slack in #design by boemedia. View the logs.


7 years ago

#25 @gibrown
7 years ago

@boemedia thanks for the review. Very helpful.

My current plan on this is to add the Jetpack Search sidebar widget to use for filtering and sorting. Here's a screenshot:

https://cloudup.com/i3ue6hjqwcf

And you can play with the interaction a bit on this test site: http://gibrown.wpsandbox.me/main/?s=post

We'll need to play with the CSS a bit of course. I think this comes pretty close to matching what your example of booking.com is doing though, so sounds like we are on a similar page.

This does mean that we are going to need a search page that has a sidebar designed and built.

#26 @afercia
7 years ago

A few accessibility considerations, not pretending to be an exhaustive list of potential issues to address:

Sidebar:
If it's going to be a fixed sidebar, I see no a11y concerns. Instead, if it's going to be a "sliding" sidebar, it's highly preferable it is placed in the source right after the toggle button that opens it. This way, it would be in the natural tab order. Otherwise, focus needs to be managed.

"Sort by" select:
It should have a button to submit, see https://core.trac.wordpress.org/ticket/40925

Interaction:

One is that each choice is a link (parameterized search) so the page reloads on each. The other is that it is a form that you set all the parameters and then submit once.

Checkboxes and a submit button are highly preferable. Triggering a search when checking a checkbox produces an unexpected change of context though. Depending on the implementation, there's also the risk the search fails to return the intended results and the risk to hammer the server with multiple requests. For example, quickly checking and unchecking multiple checkboxes can easily fail.
To reproduce on the http://gibrown.wpsandbox.me/main/?s=post example, just quickly check and uncheck dowork with a double click. Although you've left the checkbox unchecked, the search triggers after the first action and you get results for dowork tag=dowork.
Ideally, a submit button should always be used, as that's the only reliable way to confirm the users intention to submit.

Checkboxes:
Consider to group them in a fieldset with a legend and in this case the headings can be removed.
Note: for inspiration, see the themes filtering in core, which has been changed to use fieldsets/legends (no headings) together with some more a11y improvements (e.g. buttons are now buttons and not links <a class="drawer-toggle" href="#">, the submit button is now also at the end of the form, etc. see https://core.trac.wordpress.org/changeset/38640). Instead, the themes directory hasn't been updated with this changes.
If keeping the headings, the heading hierarchy in the page should be correct. Currently, on http://gibrown.wpsandbox.me/main/?s=post I see the headings are h4 and they skip a few levels.

Labeling:

Search as you type:
Note: I'm not taking into considerations the mentioned "search as you type" UI, as it seems it's not going to be implemented for now. However, please consider this kind of UI needs a considerable amount of work to get a minimum level of accessibility and I'm not sure they can be made "fully" accessible. For the similar searches in core, some concerns have been expressed about their real usability, see for example https://core.trac.wordpress.org/ticket/31818 and https://core.trac.wordpress.org/ticket/38211
Also worth considering the following WCAG Success Criterion about keystrokes timings: https://www.w3.org/TR/WCAG21/#keyboard

All functionality of the content is operable through a keyboard interface without requiring specific timings for individual keystrokes ...

#27 @rianrietveld
7 years ago

For inspiration:
An example of an (almost) accessible filter with options is on the search results page of the Newsroom of Encompass Health:
https://newsroom.encompasshealth.com/?filter%5Basset_type%5D=&s=encompass
The filters button toggles the visibility of the search options.

Last edited 7 years ago by rianrietveld (previous) (diff)

This ticket was mentioned in Slack in #meta by tellyworth. View the logs.


6 years ago

#29 @chriscct7
6 years ago

As a tiny bit of feedback, size of zip probably is not a useful metric to almost any average user

This ticket was mentioned in Slack in #meta by joyously. View the logs.


6 years ago

This ticket was mentioned in Slack in #design by joyously. View the logs.


6 years ago

This ticket was mentioned in Slack in #meta by tellyworth. View the logs.


6 years ago

This ticket was mentioned in Slack in #meta by joyously. View the logs.


5 years ago

This ticket was mentioned in Slack in #meta by clorith. View the logs.


4 years ago

#36 @Otto42
4 years ago

#5176 was marked as a duplicate.

This ticket was mentioned in Slack in #forums by joyously. View the logs.


4 years ago

#38 @edo888
4 years ago

Hi,

I have recently noticed that a plugin with about ~200 reviews, ~4.7 stars, 80k+ installations, ~200 support threads, 120 resolved support threads (past 2 months), same keyword in the slug and title, same keyword in the content, recently updated, tested up to the decent wp version is a way ahead of other plugin with ~1500 reviews, ~4.9 stars, 200k+ installations, ~25 support threads, ~25 resolved support threads (past 2 months), same keyword in the slug and title, same keyword in the content, recently updated, tested up to the decent wp version.

I have checked the source code of the ranking algorithm and it seems that the score is based on support_threads_resolved and that support_resolution_percentage is not being taken into account (please correct me if I'm wrong).

My idea is to use the resolution percentage instead. Buggy plugins can have a lot more support threads than plugins with less bugs. Also, we offer live chat support which reduces the number of support threads on the public forums and we get punished for that. :(

Also, it would be great if the plugin author can indicate that there is a live chat support available, so that it can also be a ranking factor.

Thanks!

Last edited 4 years ago by edo888 (previous) (diff)

This ticket was mentioned in Slack in #forums by edo888. View the logs.


4 years ago

#41 in reply to: ↑ 40 @edo888
4 years ago

Replying to gibrown:

@edo888 I added it to https://meta.trac.wordpress.org/ticket/2686#comment:14

Thanks, I'll check.

This ticket was mentioned in Slack in #design by estelaris. View the logs.


4 years ago

#43 in reply to: ↑ description @atmeditation
4 years ago

A filter for plugin search results would be very useful. For example, when I search for "payment gateway", there are 90 pages of plugin results. Some filter criteria that would be helpful to make results more relevant include last updated, active installations, average rating, % support issues resolved in last two months, tested up to, and PHP version.

Replying to tellyworth:

We bumped sorting and filtering from the v3 launch, so it's time to revisit.

Implementing both is straightforward on the back-end, the main issue is the UI: where to put it, how to make it helpful and accessible. This is a general ticket for discussion and progress.

#44 @dd32
4 years ago

In 10233:

Plugin Directory: Search: When searches are performed from /browse/block/ limit the searches to the plugins in that section (block plugins).

This is mostly for debugging and to visually see any search modifications easier.

See https://github.com/WordPress/gutenberg/issues/24910
See #2753.

This ticket was mentioned in Slack in #meta by joyously. View the logs.


4 years ago

This ticket was mentioned in Slack in #meta by tellyworth. View the logs.


4 years ago

This ticket was mentioned in Slack in #forums by joyously. View the logs.


4 years ago

This ticket was mentioned in Slack in #meta by joyously. View the logs.


4 years ago

This ticket was mentioned in Slack in #meta by edo888. View the logs.


4 years ago

#50 @NekoJonez
4 years ago

I'm sorry for the bump but I would love to see a feature like this in the standard WordPress install.

#51 @reelism
4 years ago

Here to add a gratuitous +1 :)

Version 0, edited 4 years ago by reelism (next)

#52 @gibrown
3 years ago

  • Milestone changed from Plugin Directory v3 - Future to Improved Search

#53 @Ipstenu
3 years ago

#5885 was marked as a duplicate.

#54 @mohammadayoub96
3 years ago

Hi I have realized that there is no way to filter plugin on plugin search page, If you can add filter on plugin search It will be good way to search the right plugin we need.
Also In plugin grid list, There should be listed plugin dependencies plugin is using.
Also if possible there be CSS & JS size should be shown, So developer should have a better idea which plugin to use or which not to.
Regards: Ayoub Khan

#55 @mohammadayoub96
3 years ago

Why is there not a way to filter all the plugins by e.g. number of downloads and / or number of reviews/ best rated.
Come on guys! I get that probably the reason for this is because you don’t want to give more power to the most popular ones – and you’re trying to be fair to new authors – but honestly – it would save so much time for us users if we could filter the plugin (and theme) search with some additional parameters.
Having a plugin dependencies list for each plugin could be quite difficult to achieve (though it is a great idea) but why can’t we “order by: most popular” etc…. Surely that would be fairly easy to implement.

This ticket was mentioned in Slack in #meta by funcdesign. View the logs.


2 years ago

This ticket was mentioned in Slack in #meta by joyously. View the logs.


2 years ago

#58 in reply to: ↑ description @fordreamauto
12 months ago

Replying to tellyworth:

We bumped sorting and filtering from the v3 launch, so it's time to revisit.

Implementing both is straightforward on the back-end, the main issue is the UI: where to put it, how to make it helpful and accessible. This is a general ticket for discussion and progress.

Take a look at this site's filter: https://fordreamauto.com/

#59 @dd32
8 months ago

In 13190:

Plugin Directory: Enable the Community/Comercial filters on Search results.

See #2753, #7430.

This ticket was mentioned in PR #202 on WordPress/wordpress.org by @dd32.


8 months ago
#60

  • Keywords has-patch added

This is a work-in-progress draft attempt at seeing if it's possible to integrate the wporg/query-filter block into the plugin directory, and how that would look.

See Meta.trac #2753

  • This PR is not a guarantee of seeing it on w.org/plugins anytime soon.
  • The code is not intended to be production ready.
  • The filtering most likely doesn't work with search, due to no conversion to Jetpack ES args happening (Test with a /tag/.../ url, or possibly /author/)
  • It's likely that we wouldn't want this sorting/filtering hitting WP_Query => DB ever, as it uses meta_queries, which are most likely not performant for our scale. Thats to suggest, that we'd probably want to limit this to ES-backed views only.
Before After
https://github.com/WordPress/wordpress.org/assets/767313/bb0a71fb-4b00-4a1b-a92e-cef54f07258d https://i0.wp.com/github.com/WordPress/wordpress.org/assets/767313/cbf69148-02a0-4875-b08f-936ad09dd3c9
  • Yes, The count is wrong
  • Yes, The screenshots are of two different queries
  • Yes, The search is duplicated
  • Yes, The whitespace is off

#61 @dd32
8 months ago

In 13220:

Plugin Directory: Store a num_ratings post_meta for query filtering.

See #2753.

#62 @dd32
7 months ago

In 13332:

Plugin Directory: Adjust the query customisations to allow for custom sorting.

This doesn't expose any UI, but provides the underlying API required for supporting it.

See #2753.

#63 @dd32
5 months ago

#5082 was marked as a duplicate.

#64 @dd32
5 months ago

  • Priority changed from normal to high

Bumping the priority; This is more important than most of the other Search-related tickets IMHO.

This ticket was mentioned in PR #313 on WordPress/wordpress.org by @dd32.


5 months ago
#65

This is effectively a cutdown version of #202 but specific for sort with the new plugin directory theme.

This enables a sorting dropdown on archives & search.

ZERO effort has been put into styling and getting the dropdowns in the correct locations on either search or archives.

It's worth noting that custom sorting is already enabled for non-search archives, just not exposed with a dropdown, for example, tags/performance/?orderby=rating

Sorting for search is arguably useless, as when you sort you immediately loose relevancy order in the results, and you end up with the same top-20 plugins showing up in every search in the first few places.

This requires a change to Jetpack Search on WordPress.com to allow us to sort by some fields, D148116, notably this is required for the ratings, num_ratings, tested, and active_installs sorts. If you're an Automattician with a WordPress.com sandbox, you can test using the diff D148116 w/ JETPACK__SANDBOX_DOMAIN.

Trac: https://meta.trac.wordpress.org/ticket/2753

This ticket was mentioned in Slack in #meta by dd32. View the logs.


5 months ago

#67 @dufresnesteven
5 months ago

Based on the work from @dd32, I've toggled on experimental filters behind a 'show_filters' query variable in r13692 for non-search views.

There are still things to sort out, which include but not limited to:

  • How do we sort search results?
  • What should be the sort values?
  • What categories are most relevant?
  • ...

#68 @dufresnesteven
5 months ago

In 13693:

wporg-plugins-2024: Remove all category value to fix experimental filters.

Related: r13692
See: #2753

#69 @dd32
4 months ago

Noting this GitHub issue, which requests for filtering to blocks.

https://github.com/WordPress/wordpress.org/issues/334

There is a URL dedicated to block-enabled plugins:
https://wordpress.org/plugins/browse/block/

On that page, only block enabled plugins are filtered. There is a search box on the top. When one enters a term in that box, all plugins that fit search results come out as a results. I would expect that this search box works only on block enabled plugins subset.

To my knoledge, currently there is no way to search only block enabled plugins in directory. I propose to introduce that possibility. This would give greater visibility of blocks from which to choose to users, simliarly to block search inside block editor.

#70 @dd32
6 weeks ago

In 14002:

Plugin Directory: Restore sorting for views that use last_updated to be sorted by date including time.

[13332] incorrectly assigned the column type as a DATE, rather than DATETIME, which has resulted in the relevant queries being sorted by date only.
This restores the order to be by date+time.

See #2753.

Note: See TracTickets for help on using tickets.