Making WordPress.org

Opened 4 months ago

Last modified 10 days ago

#5806 new defect

Remove legacy robots.txt disallow directives

Reported by: jonoaldersonwp Owned by:
Milestone: Priority: low
Component: General Keywords: seo


wp.org + Rosetta site robots.txt files (such as https://wordpress.org/robots.txt, and https://tah.wordpress.org/robots.txt) prevent crawling of the wp-admin directory.

These rules should be removed, for the following reasons:

*A separate ticket will alter where these resources are loaded from and how they're served.


Change History (2)

#1 @dd32
4 months ago

Remove legacy robots.txt disallow directives

These are the current core robots.txt directives, please follow up with a Core ticket if these are not appropriate for all WordPress sites.

I'll take a middle-ground approach here and specifically allow access to /wp-admin/load-*.php because I do know that some WordPress.org themes trigger some WordPress functionality that other sites wouldn't on the front-end.

#2 @dd32
4 months ago

In 11094:

Robots: Alow access to wp-admin/load-*.php as some WordPress.org themes load resources from there.

This should be upstreamed to Core.
See #5806.

Note: See TracTickets for help on using tickets.