Implement Plugin Recommender

[dd32]

We currently support the Plugin Recommendations tab based on the plugins you currently have installed, we should continue supporting this in the new plugin directory.

It's only available through the API, although I believe it's datasource may be a little outdated now.

[dd32]

If we want to drop it or leave it for after launch, we could probably simply replace it with the latest trending plugins instead.

[tellyworth]

In 3550:

Plugin directory: implement a simple recommended plugins API response using Elasticsearch.

This returns a list of plugins with the following criteria:

• Not already installed
• Tested-up-to is close to the requesting site's WP version
• Scoring accounts for similar factors as a regular search

See #1729

[tellyworth]

In 3557:

Plugin directory: [3350] introduced a reference to $_SERVER, which won't work due to caching. Remove it.

See #1729

[tellyworth]

The HTTP_USER_AGENT logic removed in [3557] could be re-introduced in future if the WP version string is passed as a query parameter rather than in HTTP headers.

[gibrown]

implement a simple recommended plugins API response using Elasticsearch.

The alg as implemented will almost entirely be based on popularity rather than any notion of co-occurrence. It's a good start, but I think co-occurrence would be much more powerful.

The easiest way to index this would be to have an index with one doc per site and then look at the uncommonly common plugins using a significant terms aggregation: ​https://www.elastic.co/guide/en/elasticsearch/guide/2.x/significant-terms.html

I played around with this recently on an index of Jetpack sites that we (Automattic) have:

GET jetpack-sites-8/_search
{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "enabled_plugin_slugs": {
              "value": "woocommerce"
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "NAME": {
      "significant_terms": {
        "exclude": ["jetpack","woocommerce"], 
        "field": "enabled_plugin_slugs"
      }
    }
  }
}

That query then gets the following recommendations:

woocommerce-gateway-paypal-express-checkout
woocommerce-gateway-stripe
yith-woocommerce-wishlist
woocommerce-services
contact-form-7
revslider
akismet
js_composer
woocommerce-gateway-paypal-powered-by-braintree
regenerate-thumbnails

Which is pretty good. Akismet is super popular (>5m installs), but woocommerce-gateway-paypal-express-checkout with 300k installs is a much better recommendation. Having the full list of plugins on a site would make the results even better.

However, this requires indexing a doc for every site (which there are many obvious reasons not to do). I think what we want is a set of plugins that are commonly installed with any particular plugin and then put that list into post meta for each plugin. We can then (I think) query that field to find plugins that are commonly installed together.

For filling in the post meta, I'd recommend we try an algorithm like this:

For each plugin P:
   create a count C for each plugin Q that is co-installed with it
   if ( C > 10% of the install count of P ) then put Q into a post meta list for P

10% may be too low. Very popular plugins may have a very long list of plugins. (though maybe not due to it being harder to get to 10%.

I don't really know what the query will look like, but I think I can come up with one once I have this data. :) It probably won't work as well as the one above, but I think it would be an ok v1.

[tellyworth]

We'll get to something like this at some point. For now it's low demand and we don't have a clear way of implementing something that's good enough to warrant the effort.