WordPress.org

Making WordPress.org

Changes between Initial Version and Version 1 of Ticket #4559


Ignore:
Timestamp:
06/28/2019 09:39:44 AM (2 years ago)
Author:
jonoaldersonwp
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Ticket #4559 – Description

    initial v1  
    1 This consumes huge amounts of crawl budget, for relatively little return.
    2 We'd like to block crawling of it entirely, via robots.txt.
     1This consumes huge amounts of crawl budget, for relatively little return. We'd like to block crawling of it entirely, via robots.txt.
    32
    43At the moment, it shares a robots.txt with other wordpress.org domains, which makes this impossible.
     
    1312}}}
    1413
     14NB: We'll need to be absolutely certain that this is a standalone file, and doesn't bleed through to any other WP domains/contexts, or we'll cause the end of the world.
     15
    1516If/when this is complete, the `?filter` rule can be removed from the shared/global robots.txt file.