Opened 5 years ago
Closed 5 years ago
#4817 closed defect (bug) (fixed)
Change meta robots directive on systems
Reported by: | jonoaldersonwp | Owned by: | |
---|---|---|---|
Milestone: | Priority: | lowest | |
Component: | Make (Get Involved) / P2 | Keywords: | seo has-patch |
Cc: |
Description
At the moment, systems URLs (such as https://make.wordpress.org/systems/, https://make.wordpress.org/systems/2019/10/01/redirect-downloads-wordpress-org-to-wordpress-org-download-currently/) output a meta robots tag with a value of noindex, nofollow
.
This should be altered to noindex, follow
.
Attachments (2)
Change History (12)
#2
@
5 years ago
That tag/setting won't stop them looking at it. It prevents them from indexing it, and it prevents them from then following any links (though as of ~Oct, the nofollow directive in this case, as far as Google is concerned, is actually just a hint, rather than a directive).
If we want to stop them from looking at it, it needs blocking in robots.txt.
Given that isn't the case, we _do_ want them to follow links (and flow pagerank) _out_ of those pages back into other areas of the WP ecosystem.
If there's truly sensitive stuff here which we don't want crawling at all, it shouldn't be public.
#4
@
5 years ago
This patch (4817.diff
) will require an admin to flip the option in the admin so that the site is considered public - do not discourage search engines. The reason the admin toggle needs to be flipped is because I am using wp_no_robots
which will check the option value and continue using the current behaviour if it is still set to non-public. Setting it to public will tell wp_no_robots
to change behaviour to allow follow
but retain noindex
.
@
5 years ago
An alternative implementation that doesn't require the admin to change the site visibility/search engine option
#5
follow-up:
↓ 6
@
5 years ago
Is this something specific to the make/systems site, or is this a generic suggestion that should be made to all WordPress.org sites if using the discourage search engine option?
If the latter, is this something that should be considered for change in WordPress core? or is nofollow
probably the intended value for the vast majority of cases?
Any private-private site on WordPress.org is protected using a login plugin, and the make/systems site isn't designed to be hidden (We have private posts for anything potentially sensitive) just not encouraged to appear in search results.
#7
follow-up:
↓ 8
@
5 years ago
We updated the "Discourage search engines" behaviour in core relatively recently, to output "noindex, follow". Are we running a recent enough version to have picked that up?
#8
in reply to:
↑ 7
@
5 years ago
Replying to jonoaldersonwp:
We updated the "Discourage search engines" behaviour in core relatively recently, to output "noindex, follow". Are we running a recent enough version to have picked that up?
We are, but that change doesn't do what you think it does.. https://core.trac.wordpress.org/ticket/43590
wp_no_robots()
still outputs noindex,nofollow
unless the option "Discourage search engines" is disabled where it changes to noindex,follow
. But if the option is disabled, wp_no_robots()
isn't called by default, except on embeds and ?replytocom=
links where it's called.
That's not a tag which is specific to the systems blog, it is simply using the WordPress core "Discourage search engines from indexing this site" setting.
Which seems correct, as we don't want search engines indexing that site. Or looking at it in any way.