Plugins API Unit Tests

[obenland]

Write client-side Core Unit Tests for the existing API to make sure the new API doesn't break backwards compatibility.

[enej]

wporg plugin api tests

[enej]

@obenland I just attached my initial test for the WPorg plugin api.

I would love some feedback on it before I work on the rest.

To run the tests I use the following command
phpunit --group wporg-plugin-api

[obenland]

Looking good!

A few things that I noticed:

• We should probably test plugins_api() as well, which is in wp-admin/includes/plugin-install.php.
• I'm not sure the download url itself needs to stay the same necessarily, as long as we have one at all (and it provides a zip).
• Can the property tests be combined in a separate method?

I'm sure @dd32 has some thoughts as well.

[enej]

Added tests for plugins_api function and the different actions that you can pass into it.

[obenland]

In 1614.diff​:

• Use Jetpack instead of Hello Dolly to get a richer response.
• Refactored attribute tests to be reusable (except for query_plugins).
• Switched array tests to use assertAttributeInternalType().

Next steps could be to look at ​https://developer.wordpress.org/reference/functions/plugins_api/ and start writing tests for different arguments and check if fields are returned correctly when added or removed in the request.

[enej]

v3 added more tests and assertions

[enej]

In diff wp-org-plugin-api-test-v3.diff
I added more test that test each of the different cases and also added tests for

• tag, author, user, browse, installed_plugins, and fields

Some inconsistancies that i noticed between the documentation and what I got back from the api

1. group fields doesn't seem to do anything since it doesn't show up for most plugins.
2. is_ssl is always set to true by default.
3. hot_categoreis number doesn't do anything. We always return 8 categories.
4. Also the reviews gets returns on the section field. Which is not specified in the documentation.

All the tests in the diff pass but should include all the inconsistencies to be complete.
Let me know if you want me to add assertions for them.

[obenland]

@enej Thank you so much for your work on this! I think this is pretty close to being commit ready.

Was test_plugins_api_function_action_query_plugins_local() supposed to test localized plugin results? I couldn't find a local parameter, only locale.

[obenland]

Should we add tests for the XML endpoints?
Example: ​http://api.wordpress.org/plugins/info/1.0/jetpack.xml

/cc @dd32 @ocean90 @otto42

[obenland]

In 1614.2.diff​ I made all fields required for every request and added an XML test and one test for a v1.1 endpoint.

Should we even add a test for XML returns?
Should we run all tests for both versions, 1.0 and 1.1? For 1.1 obviously only the JSON tests.

[dd32]

hot_categoreis number doesn't do anything. We always return 8 categories.

Just noting that the hot_categories endpoint is an unreleased endpoint and should not be tested. It's output is not yet known/standardised and will change (or be removed) without any back-compat thoughts.

Should we add tests for the XML endpoints?

That depends on what you're testing - The data or the format.
I would assume you'd test the data structure is within "Expected" ranges, so XML/PHP/Json shouldn't really matter.
That being said, all the formats have the potential to encode the data differently, so you would test them all.. but realistically we don't care about that angle.

[pento]

Core really feels like the wrong place to put these tests.

The idea of these tests is clearly to catch changes to the API, but if the tests were in Core, it could be hours (or even days, during RC) between when a change is made to the API, and when the tests are run.

I'd be more inclined to to add them to the API repo along with an appropriate phpunit.xml, so phpunit can easily be run before any changes are committed. (Having a pre-commit hook feel a bit excessive, but may be worth investigating.)

[obenland]

I don't feel strongly either way. Do you want to take point on this? What do you think about the tests themselves?

[pento]

In 3211:

Plugin Directory: Add unit test scaffolding, and API unit tests.

Props enej, obenland, pento.

Fixes #1614.

[pento]

Note: These tests will run in a VVV install of WordPress, not on a WordPress.org sandbox.

They can be easily extended to work on Travis, if we choose to do that in the future.

[dd32]

In 3227:

Plugin Directory: Tests: Remove group from the supported fields, we never actually implemented it. Remove the hot_categories method too, as once again, it was never actually implemented.

See #1614.

[samuelsidler]

Milestone Plugin Directory v3 - M4 deleted