Plugin Directory: Pagination issue in Popular Plugins section

[SergeyBiryukov]

​https://wordpress.org/plugins/browse/popular/ displays 2,049 pages at the bottom, but any page greater than 99 gives you a 404 error.

[SergeyBiryukov]

Possibly related to #59.

[Otto42]

Actually, this is intentional. Nobody pages that far except Googlebot, and so it's blocked after that limit to prevent Googlebot from going nuts and killing the thing with those ridiculous requests.

[SergeyBiryukov]

Could we reduce the page count to 99 then?

Pagination in other sections (Newest, Recently Updated, Highest Rated) appears to work as expected.

[kpdesign]

#570 was marked as a duplicate.

[joostdevalk]

This was actually fixed in the recent redesign, if I look at the site now we only show page 1 - 34. Closing as worksforme.

[SergeyBiryukov]

Looks like it's back.

​https://wordpress.org/plugins/browse/popular/page/99/ shows 2,863 pages, but ​https://wordpress.org/plugins/browse/popular/page/100/ produces a 404 error page from nginx.

[SergeyBiryukov]

Also reported in ​#WP40296.

[lukecavanagh]

@SergeyBiryukov

Confirmed, trying to load a page between

​https://wordpress.org/plugins/browse/popular/page/100/
​https://wordpress.org/plugins/browse/popular/page/2863/

Will result in a 404 error in NGNIX.

[Otto42]

Can't reproduce this on my sandbox, works fine there. Perhaps there is a nginx rule on production blocking these.

[SergeyBiryukov]

Replying to Otto42:

Perhaps there is a nginx rule on production blocking these.

Yep, the consensus in ​IRC discussion 3 years ago was to either limit the number of pages or wait for an overhaul:

sams: Preferably we won't 404 it and just limit the number of pages.
Otto42: preferably, we overhaul the thing to not be in bbPress anymore and then pagination won't suck all the DB processing power

The latter happened, so perhaps the rule can be removed now.

[dd32]

Replying to Otto42:

Can't reproduce this on my sandbox, works fine there. Perhaps there is a nginx rule on production blocking these.

It's a block in the LBs which doesn't apply to sandboxes - IIRC the block was put in place due to excess load caused by the high pages.

[Otto42]

#2649 was marked as a duplicate.

[tellyworth]

Limit browse results to 200 pages.

[tellyworth]

The added patch filters found_posts to limit it to 200 pages. That's reflected in pagination. Is there a better number than 200?

[Otto42]

Pagination is a problem basically due to core queries not being indexed well for this. I'd like to figure out a better index instead of cutting it off, but maybe that's not possible? Needs testing.

[SergeyBiryukov]

#2868 was marked as a duplicate.

[SergeyBiryukov]

Replying to Otto42:

Pagination is a problem basically due to core queries not being indexed well for this.

The block was put in place years ago when the directory was still on bbPress 1.x. As far as I know, WordPress core doesn't have any performance issues with high page numbers. Is this still an issue in the new directory?

[SergeyBiryukov]

Nobody pages that far except Googlebot

It's been reported several times by different people on Meta Trac, Core Trac, and support forums, so apparently someone does :) Clicking on the last page of results is a natural curiosity, seeing a default nginx 404 error page there is unexpected and not very user-friendly.

[Otto42]

#2908 was marked as a duplicate.

[KestutisIT]

So, no my ticket was just closed - https://meta.trac.wordpress.org/ticket/2908#comment:1 . And I see this bug is already over 3 years old. I pretty much wanted to know in which page starts plugins with 10,000+ activations, so to know where would my plugin be located after reaching my goal for activations this year. So, believe me - there is bunch of reasons and people who care about this bug. So let's solve this finally after 3 years.

[gibrown]

It is going to be pretty impossible to scale the MySQL queries to handle deep paging of results. Deep paging just doesn't perform very well.

I propose that we address this with a faceted search interface that let's users filter by various plugin data. See #2753

[tobifjellner]

#2944 was marked as a duplicate.

[imemine]

Isn't this bad UX? Can't we just show the page numbers till 99, if we want users to visit only till that?

[SergeyBiryukov]

Replying to tellyworth:

The added patch filters found_posts to limit it to 200 pages. That's reflected in pagination. Is there a better number than 200?

I'd suggest 99, as it's the last working number :). Refreshed the patch, should be good to go: 496.diff​.

[tellyworth]

In 5719:

Plugin directory: limit browse queries to 99 pages of results. Props @SergeyBiryukov

Fixes #496

[SergeyBiryukov]

@cantothemes reported a regression in ​#pluginreview. ​https://wordpress.org/plugins/browse/popular/ still displays 3070 pages, leading to an error on 100+:

404 Not Found

nginx

Caused by [5725]. Plugin_Directory::pre_get_posts()​ sets post_type as an array, so once the in_array() check was removed, the filter no longer does the job.

We could either revert [5725] and cast $wp_query->query_vars['post_type'] to an array before comparing, or just use a string in the first place. It looks like there's no need for it to be an array, so 496.2.diff​ goes with the latter option and uses a strict check while we're at it.

[Otto42]

In 5846:

Plugin Directory: Change check back to array, but add is_array check to prevent it from causing errors when the value is not actually an array. Fixes #496

[Otto42]

Changing it to be not an array causes other errors elsewhere. Not sure why this was done that way, but it was, so changing the check back and adding an is_array check as well is the better choice.