![(please configure the [header_logo] section in trac.ini)](/chrome/site/your_project_logo.png){kind=logo}
Opened 5 years ago
Last modified 3 years ago
#5668 accepted defect (bug)
Add an 'empty' robots.txt file to s.w.org
| Reported by: | jonoaldersonwp | Owned by: | dd32 |
|---|---|---|---|
| Priority: | lowest | Milestone: | |
| Component: | General | Keywords: | seo performance |
| Cc: |
Description
https://s.w.org/robots.txt currently redirects to wordpress.org.
For SEO and performance reasons, we should implement a robots.txt file at this location.
The contents should be:
User-agent: * Disallow:
Change History (3)
#2
@
5 years ago
- Owner set to
- Status new → accepted
[10840] has added the robots.txt, but due to how the CDN is setup for caching, it could take a bit for the changes to take effect.
# Current cached public response $ curl -is https://s.w.org/robots.txt HTTP/2 301 location: https://wordpress.org/ x-nc: HIT mdw 1 # New Response, uncached. $ curl -is https://s.w.org/robots.txt?1 HTTP/2 200 x-nc: MISS mdw 1 User-agent: * Disallow: # New Response, cached. $ curl -is https://s.w.org/robots.txt?1 HTTP/2 200 x-nc: HIT mdw 1 User-agent: * Disallow:
I'm not going to request the CDN be cleared quite yet, as it should self-clear within a few days I expect. I'll keep this open to remind myself.
Note:
See TracTickets
for help on using tickets.
In 10840: