Leverage the WordPress Cron / Cavalcade to run the plugin imports

[dd32]

Currently the plugin directory is using a stand alone script which runs in a loop to process changes.

Historically this has caused problems during high-frequency-commits, so instead, lets switch the importer to run as a WordPress Cron Job, this will have the advantage that we can run many imports in parallel, it also brings complexity as a plugin shouldn't be imported twice at the same time.

On WordPress.org we're using Cavalcade to offload cron tasks from WordPress which has given jobs the ability to scale, it also means we can be sure that no two individual jobs are running in more than 1 location.

[dd32]

In 4520:

Plugin Directory: Shift SVN Watching, Plugin Imports, and Plugin i18n imports to Jobs.

See #2330.

[dd32]

In 4521:

Plugin Directory: Fix a typo in the i18n importer [4520] by using a consistent variable name.

See #2330

[dd32]

In 4522:

Plugin Directory: Queue up plugin i18n import jobs after the main plugin import runs.

See #2330

[dd32]

In 4523:

Plugin Directory: Flush the Cavalcade jobs cache as these methods need to operate on fresh data from the database - currently they may be unaware of another job added while the current process was running.

See #2330.

[dd32]

This seems to be holding up pretty well.

[obenland]

In 4670:

Plugin Directory: Add missing constant PHP

Fixes a bug where imports can't be processed due to a missing path to php.

See #2330, [4520].

[dd32]

In 7370:

Plugin Directory: When re-queueing the next plugin import job, queue it for a minutes time rather than an hours time.

See ​https://wordpress.slack.com/archives/C02QB8GMM/p1530669226000099 for discussion.
See #2330.