Add a robots.txt disallow all rule to ps.w.org

[jonoaldersonwp]

Replace the contents of ​https://ps.w.org/robots.txt with:

User-agent: *
Disallow: /

[Otto42]

The easy way to fix this would be to add the robots.txt to the root SVN. I think I'm the only person with direct access to do that easily.

Is that an acceptable fix?

[dd32]

Committing a file won't work here, as it's the robots.txt served by apache for all SVN hosts, there's a similar generic one for Trac hosts.

@jonoaldersonwp can you confirm what should happen for routes such as these, and if they should differ from ​https://ps.w.org/robots.txt (That's just a cached CDN of plugins.svn.wordpress.org)
​http://plugins.svn.wordpress.org/robots.txt
​http://core.svn.wordpress.org/robots.txt
​http://meta.svn.wordpress.org/robots.txt

Having them have different content is possible, it's just easier to have a single file for all svns.

[jonoaldersonwp]

Hmm. I think we can safely apply the same global disallow rule to each of these. Easy! :)