WordPress.org

Making WordPress.org

Opened 6 months ago

Closed 3 months ago

Last modified 2 months ago

#4994 closed defect (fixed)

Stale themes should be noindex'd

Reported by: jonoaldersonwp Owned by: dd32
Milestone: Priority: high
Component: Theme Directory Keywords: seo
Cc:

Description (last modified by SergeyBiryukov)

As per #4088, themes which haven't been updated in over two years should be noindex'd.

E.g., https://wordpress.org/themes/destro/

Change History (17)

#1 @Otto42
6 months ago

Why 2 years? Let's be honest, the 2 years thing was a number I plucked out of thin air almost 10 years ago when the directories were somewhat different.

Would a different number make more sense?

#2 @jonoaldersonwp
6 months ago

Ah, good question! I just piggybacked on the number in the notification banner, but would be happy with any sensible value.

How long does it take a theme to (likely) go 'stale'? I wonder if a year isn't long enough, and that we might end up impacting the visibility of valid-but-outdated themes? I also wonder if that time-to-stale might change radically over the course of adoption of Gutenberg, and also, decrease over time as block support proliferates...

Are there other opinions/policies on this?

Noindex'ing them also removes them from internal search, right? So we should probably be careful to avoid false-positives.

#3 @SergeyBiryukov
6 months ago

  • Description modified (diff)

#4 @dd32
6 months ago

noindex doesn't remove it from the search, but yes, the themes search includes age as one of it's requirements (bypassing that for a exact match).

The only thing that this will affect is ranking in search engines, something I'm willing to ditch given we don't promote such themes.

Questions on if 2 years is the right flag to use here is valid, but since we're already using that magic number elsewhere for themes we should probably figure that out separately.

#5 @dd32
6 months ago

  • Owner set to dd32
  • Resolution set to fixed
  • Status changed from new to closed

In 9471:

Theme Directory: Output a noindex robots tag for themes marked as potentially outdated.

Fixes #4994.

#6 @jonoaldersonwp
5 months ago

  • Resolution fixed deleted
  • Status changed from closed to reopened

Found a few examples which I think should be noindex'd, but don't appear to be.
Any idea why these are slipping through the net?

https://wordpress.org/themes/guruq/
https://wordpress.org/themes/chinese-love/
https://en-gb.wordpress.org/themes/portfilo/

#7 @ocean90
5 months ago

Hmm. maybe check again? All three themes have the <meta name='robots' content='noindex,follow' /> tag.

#8 @jonoaldersonwp
5 months ago

Interesting. I get radically different markup / meta when logged in vs logged out. Is that intentional?

#9 @dd32
5 months ago

Looks like something to do with caching.

$ curl -is https://wordpress.org/themes/guruq/ | grep '\(robots\|x-nc\)'
x-nc: MISS ord 1
<meta name='robots' content='noindex,follow' />

$ curl -is https://wordpress.org/themes/chinese-love/ | grep '\(robots\|x-nc\)'
x-nc: MISS ord 1
<meta name='robots' content='noindex,follow' />

$ curl -is https://en-gb.wordpress.org/themes/portfilo/ | grep '\(robots\|x-nc\)'
x-nc: MISS ord 2
$ curl -is https://en-gb.wordpress.org/themes/portfilo/ | grep '\(robots\|x-nc\)'
x-nc: MISS ord 2
<meta name='robots' content='noindex,follow' />

It's not nginx caching, so I'm not sure what to make of that at first.

#10 @dd32
5 months ago

Yup, looks like it's some kind of caching, the $theme data isn't available when the header functions run:

$ curl -is https://en-gb.wordpress.org/themes/portfilo/ | grep '\(<meta\|x-nc\)'
x-nc: MISS ord 1
<meta charset="utf-8" />
<meta property="fb:page_id" content="6427302910" />
<meta name="google-site-verification" content="7VWES_-rcHBcmaQis9mSYamPfNwE03f4vyTj4pfuAw0" />
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="referrer" content="always">
<meta name="generator" content="WordPress 5.4-beta3-47420" />
<meta name="description" content="Free WordPress Themes" />

Seconds later:
$ curl -is https://en-gb.wordpress.org/themes/portfilo/ | grep '\(<meta\|x-nc\)'
x-nc: MISS ord 1
<meta charset="utf-8" />
<meta property="fb:page_id" content="6427302910" />
<meta name="google-site-verification" content="7VWES_-rcHBcmaQis9mSYamPfNwE03f4vyTj4pfuAw0" />
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="referrer" content="always">
<meta name="generator" content="WordPress 5.4-beta3-47420" />
<meta property='og:title' content='Portfilo' />
<meta property='og:description' content='Portfilo is a responsive theme designed for businesses to help implement their online presence along with blog. With the help of recommended free plugins, you can showcase your portfolio and testimonials on the website. Theme provides customizer options to change look and feel of your website. Theme provides options to change logo, favicon, color scheme, tyapography, etc. It supports popular plugins like BuddyPress, WooCommerce and WPML. Portfilo theme is translatable so that you can easily localize it in the language you want.' />
<meta property='og:site_name' content='WordPress.org' />
<meta property='og:type' content='website' />
<meta property='og:url' content='https://en-gb.wordpress.org/themes/portfilo/' />
<meta property='og:image' content='https://i0.wp.com/themes.svn.wordpress.org/portfilo/1.0.18/screenshot.png' />
<meta name='twitter:card' content='summary_large_image'>
<meta name='twitter:site' content='@WordPress'>
<meta name='twitter:image' content='https://i0.wp.com/themes.svn.wordpress.org/portfilo/1.0.18/screenshot.png?w=560&amp;strip=all' />
<meta name='robots' content='noindex,follow' />
<meta name="description" content="Portfilo is a responsive theme designed for businesses to help implement their online presence along with blog. With the help of recommended free plugins, you can showcase your portfolio and testimonials on the website. Theme provides customizer options to change look and feel of your website. Theme provides options to change logo, favicon, color scheme,&hellip;" />
Version 0, edited 5 months ago by dd32 (next)

#11 follow-up: @jonoaldersonwp
5 months ago

Yikes! How confident are you that this is only affecting logged-in users? Do we need to worry that this is sporadically the output for logged-out folks (inc Google)?

Either way, it'd be good to fix it for logged-in folks, too; some of those meta tags are important for accessibility, etc.

#12 in reply to: ↑ 11 @dd32
5 months ago

Replying to jonoaldersonwp:

Yikes! How confident are you that this is only affecting logged-in users? Do we need to worry that this is sporadically the output for logged-out folks (inc Google)?

Nah, affects everyone. I have no idea what's going on - Can't fix what I can't figure out.

#13 @jonoaldersonwp
5 months ago

  • Priority changed from low to high

Eek. How do we escalate this and get more eyes/brains on it?
That's going to have a pretty catastrophic effect on performance.

#14 @dd32
3 months ago

#5192 was marked as a duplicate.

#15 @dd32
3 months ago

  • Resolution set to fixed
  • Status changed from reopened to closed

In 9870:

Theme Directory: Switch to using get_queried_object() instead of get_post() due to get_post not returning correctly on the theme directory sometimes.

Also moves the outdated theme noindexing to it's own filter.

See #5173.
Fixes #4994.

#16 @dd32
2 months ago

In 9935:

Theme Directory: Don't unset the global $post variable which causes a lot of "weird" failures of functions on a cold cache.

This should resolve the long-standing "what is going on here" questions on the Theme Directory when it comes to the SEO oddities.

See r9949-dotorg (5 years ago).
See #4994.

#17 @dd32
2 months ago

In 9936:

Theme Directory: Swap back to using get_post() now that it works as expected.

See [9935].
Reverts parts of [9870].
See #5173, #4994.

Note: See TracTickets for help on using tickets.