Changeset 11979 for sites/trunk/wordpress.org/public_html/wp-content/mu-plugins/pub/wporg-robots.php

Timestamp:

07/20/2022 12:16:16 AM (2 years ago)

Author:

dd32

Message:

WordPress.org Robots.txt: Correctly limit /plugins/search/* crawl limiting.

In [11722] I added blocking for the plugins/search endpoint, however, I did so by not including a trailing slash.
The result of this is that the rule has blocked access to plugins whose slugs begin with search (i.e. /plugins/search-*/).

By adding a trailing slash here, it correctly applies the Disallow rule to /plugins/search/* instead.

Props digamberpradhan for noticing/reporting the issue.
See #5323.
Fixes #6413.

File:

• sites/trunk/wordpress.org/public_html/wp-content/mu-plugins/pub/wporg-robots.php (modified) (1 diff)

sites/trunk/wordpress.org/public_html/wp-content/mu-plugins/pub/wporg-robots.php

@@ -38 +38 @@
     if ( 'wordpress.org' === $blog_details->domain || defined( 'IS_ROSETTA_NETWORK' ) ) {
         $robots .= "\nUser-agent: *\n" .
-                   "Disallow: /plugins/search\n";
+                   "Disallow: /plugins/search/\n";
     }