Changes between Initial Version and Version 1 of Ticket #4559
- Timestamp:
- 06/28/2019 09:39:44 AM (5 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Ticket #4559 – Description
initial v1 1 This consumes huge amounts of crawl budget, for relatively little return. 2 We'd like to block crawling of it entirely, via robots.txt. 1 This consumes huge amounts of crawl budget, for relatively little return. We'd like to block crawling of it entirely, via robots.txt. 3 2 4 3 At the moment, it shares a robots.txt with other wordpress.org domains, which makes this impossible. … … 13 12 }}} 14 13 14 NB: We'll need to be absolutely certain that this is a standalone file, and doesn't bleed through to any other WP domains/contexts, or we'll cause the end of the world. 15 15 16 If/when this is complete, the `?filter` rule can be removed from the shared/global robots.txt file.