Create WordPress Open Data page

[rodrigosprimo]

Would it be possible to create a WordPress Open Data page similar to Django Open Data page (​https://code.djangoproject.com/wiki/OpenData)?

I'm asking this because as part of my master's degree I'm researching some aspects of WordPress development. I was able to get the data I wanted from core repository but I wasn't able to extract data from WordPress plugin repository (due to its size and probably some restrictions on the server it takes forever to download the whole repository) and from Trac because I don't have access to its XML-RPC interface.

A WordPress Open Data page would make it easier for researchers, like myself, to extract data and study the project. I'm available to help implementing this if necessary.

[iandunn]

That seems like a nice-to-have, but not like it should be a higher priority than the things we already have planned. I don't feel strongly about it, though, so I'm happy to defer to others.

In the mean time, if you're not already aware of them, there are some tools to make analyzing the plugin and theme repos easier:

• ​https://github.com/markjaquith/WordPress-Plugin-Directory-Slurper
• ​https://github.com/kovshenin/WordPress-Theme-Directory-Slurper

...but be aware of ​https://github.com/markjaquith/WordPress-Plugin-Directory-Slurper/issues/9 and ​https://github.com/markjaquith/WordPress-Plugin-Directory-Slurper/pull/14

By default, it only takes 1-3 days to download the whole thing. That's a relatively short amount of time for that much data. IIRC, with Drew's batching PR, it only takes a couple hours.

[rodrigosprimo]

Thanks for your quick reply.

Maybe we could start with a page documenting what is already available? For example, though it won't be useful for me because I need the repository history, I wasn't aware of those two tools you mentioned. Plugin Mirror (​http://www.pluginmirror.com/) should be on this list as well.

Is enabling Trac XML-RPC interface doable in the near future? I'm assuming this is a simple change and it is what a need the most.

[Otto42]

We already have the Trac XML-RPC interface enabled. However, access to it is strictly limited to accounts that need it. Primarily bots and other scripts that do some of the grunt work for us.

[rodrigosprimo]

@Otto42 would it be possible to give me access to it? I can give you more details of what I'm doing and why I need it. I'm just not sure if this ticket is the best place to continue this part of the conversation.

[Otto42]

You can email me. otto at wordpress dot org

[Otto42]

Access to the trac is given (and was given) on an as needed basis.