WordPress.org

Making WordPress.org

Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#245 closed defect (fixed)

Google not indexing core trac anymore?

Reported by: DrewAPicture Owned by:
Milestone: Priority: normal
Component: Trac Keywords:
Cc:

Description

Up until very recently, I've always used Google search with the site operator to search core trac. And for some reason, it's no longer returning very many if any results.

So I wondered if anybody knew if the crawl/robots.txt settings for core trac had changed recently prevent Google from indexing it?

Sample search:
site:core.trac.wordpress.org bug

Results (5): https://www.google.com/#filter=0&hl=en&q=site:core.trac.wordpress.org+bug

Change History (10)

#1 @nacin
6 years ago

Per http://core.trac.wordpress.org/robots.txt, things seem kosher, at least. Should probably not block /changeset, but eh.

#2 @bpetty
6 years ago

I do this a lot too (Google search history says about 24 times in the last couple months), and had noticed last week that it actually gave me search results that something along the lines of "this site has blocked search results, check robots.txt".

The robots.txt file looked kosher to me back when this happened to me last week (and it's exactly the same as now). So I'm really not sure what's up either. Regardless, I'm also not seeing any results for things there should be hundreds of results for, and this is rather critical in my opinion not just for us, but for WordPress users looking up issues too. Someone should hop on Google Webmaster tools (who has proper authorization), and figure out what's wrong.

#3 @ocean90
6 years ago

Someone should hop on Google Webmaster tools (who has proper authorization), and figure out what's wrong.

Yes please. It's really really sad that no trac ticket is indexed.

#4 @nacin
6 years ago

I added Trac to Google webmaster tools. It's basically empty, but everything seems to work fine. This isn't my area of expertise, but I'm definitely wanting to resolve this. I'm happy to add someone who knows more than me to investigate.

#5 follow-up: @nacin
6 years ago

With the help of dd32 I tracked down where bots were getting blocked. Hopefully this will clear up soon. I may submit a sitemap of all tickets.

#6 in reply to: ↑ 5 @DrewAPicture
6 years ago

Replying to nacin:

With the help of dd32 I tracked down where bots were getting blocked. Hopefully this will clear up soon. I may submit a sitemap of all tickets.

Thanks fellas. Sounds like I'll have my Trac-fu back soon.

#7 @joostdevalk
6 years ago

Saw your ping @nacin, sorry for the late response, seems resolved as far as I'm concerned. I think changeset should indeed not be blocked, and the Crawl-delay: 3 is something Google and Bing both ignore AFAIK.

This ticket was mentioned in IRC in #wordpress-meta by sams. View the logs.


6 years ago

#9 @nacin
6 years ago

  • Resolution set to fixed
  • Status changed from new to closed

Seems good to go. Thanks joost also.

Note: See TracTickets for help on using tickets.