WordPress.org

Making WordPress.org

Opened 22 months ago

Closed 19 months ago

Last modified 9 months ago

#1614 closed enhancement (fixed)

Plugins API Unit Tests

Reported by: obenland Owned by: enej
Milestone: Priority: normal
Component: Plugin Directory Keywords: has-patch
Cc:

Description

Write client-side Core Unit Tests for the existing API to make sure the new API doesn't break backwards compatibility.

Attachments (5)

wporg-plugin-api.php (9.7 KB) - added by enej 22 months ago.
wporg plugin api tests
wp-org-plugin-api-test-v2.diff (17.4 KB) - added by enej 21 months ago.
Added tests for plugins_api function and the different actions that you can pass into it.
1614.diff (8.7 KB) - added by obenland 21 months ago.
wp-org-plugin-api-test-v3.diff (13.6 KB) - added by enej 21 months ago.
v3 added more tests and assertions
1614.2.diff (13.8 KB) - added by obenland 21 months ago.

Download all attachments as: .zip

Change History (31)

#1 @obenland
22 months ago

  • Owner set to enej
  • Status changed from new to assigned
  • Type changed from defect to enhancement

#2 @obenland
22 months ago

  • Component changed from General to Plugin Directory

This ticket was mentioned in Slack in #meta by obenland. View the logs.


22 months ago

@enej
22 months ago

wporg plugin api tests

#4 @enej
22 months ago

@obenland I just attached my initial test for the WPorg plugin api.

I would love some feedback on it before I work on the rest.

To run the tests I use the following command
phpunit --group wporg-plugin-api

#5 @obenland
22 months ago

Looking good!

A few things that I noticed:

  • We should probably test plugins_api() as well, which is in wp-admin/includes/plugin-install.php.
  • I'm not sure the download url itself needs to stay the same necessarily, as long as we have one at all (and it provides a zip).
  • Can the property tests be combined in a separate method?

I'm sure @dd32 has some thoughts as well.

This ticket was mentioned in Slack in #meta by enej. View the logs.


22 months ago

@enej
21 months ago

Added tests for plugins_api function and the different actions that you can pass into it.

This ticket was mentioned in Slack in #meta by obenland. View the logs.


21 months ago

@obenland
21 months ago

#8 @obenland
21 months ago

In 1614.diff:

  • Use Jetpack instead of Hello Dolly to get a richer response.
  • Refactored attribute tests to be reusable (except for query_plugins).
  • Switched array tests to use assertAttributeInternalType().

Next steps could be to look at https://developer.wordpress.org/reference/functions/plugins_api/ and start writing tests for different arguments and check if fields are returned correctly when added or removed in the request.

This ticket was mentioned in Slack in #meta by obenland. View the logs.


21 months ago

@enej
21 months ago

v3 added more tests and assertions

#10 @enej
21 months ago

In diff wp-org-plugin-api-test-v3.diff
I added more test that test each of the different cases and also added tests for

  • tag, author, user, browse, installed_plugins, and fields

Some inconsistancies that i noticed between the documentation and what I got back from the api

  1. group fields doesn't seem to do anything since it doesn't show up for most plugins.
  2. is_ssl is always set to true by default.
  3. hot_categoreis number doesn't do anything. We always return 8 categories.
  4. Also the reviews gets returns on the section field. Which is not specified in the documentation.

All the tests in the diff pass but should include all the inconsistencies to be complete.
Let me know if you want me to add assertions for them.

#11 @obenland
21 months ago

@enej Thank you so much for your work on this! I think this is pretty close to being commit ready.

Was test_plugins_api_function_action_query_plugins_local() supposed to test localized plugin results? I couldn't find a local parameter, only locale.

#12 @obenland
21 months ago

Should we add tests for the XML endpoints?
Example: http://api.wordpress.org/plugins/info/1.0/jetpack.xml

/cc @dd32 @ocean90 @otto42

@obenland
21 months ago

#13 @obenland
21 months ago

  • Keywords has-patch added

In 1614.2.diff I made all fields required for every request and added an XML test and one test for a v1.1 endpoint.

Should we even add a test for XML returns?
Should we run all tests for both versions, 1.0 and 1.1? For 1.1 obviously only the JSON tests.

#14 @dd32
21 months ago

hot_categoreis number doesn't do anything. We always return 8 categories.

Just noting that the hot_categories endpoint is an unreleased endpoint and should not be tested. It's output is not yet known/standardised and will change (or be removed) without any back-compat thoughts.

Should we add tests for the XML endpoints?

That depends on what you're testing - The data or the format.
I would assume you'd test the data structure is within "Expected" ranges, so XML/PHP/Json shouldn't really matter.
That being said, all the formats have the potential to encode the data differently, so you would test them all.. but realistically we don't care about that angle.

This ticket was mentioned in Slack in #meta by obenland. View the logs.


20 months ago

#16 @pento
20 months ago

Core really feels like the wrong place to put these tests.

The idea of these tests is clearly to catch changes to the API, but if the tests were in Core, it could be hours (or even days, during RC) between when a change is made to the API, and when the tests are run.

I'd be more inclined to to add them to the API repo along with an appropriate phpunit.xml, so phpunit can easily be run before any changes are committed. (Having a pre-commit hook feel a bit excessive, but may be worth investigating.)

#17 @obenland
20 months ago

I don't feel strongly either way. Do you want to take point on this? What do you think about the tests themselves?

This ticket was mentioned in Slack in #meta by obenland. View the logs.


20 months ago

This ticket was mentioned in Slack in #meta by obenland. View the logs.


20 months ago

#20 @obenland
20 months ago

  • Milestone changed from Plugin Directory v3 - M1 to Plugin Directory v3 - M3

This ticket was mentioned in Slack in #meta by obenland. View the logs.


19 months ago

#22 @obenland
19 months ago

  • Milestone changed from Plugin Directory v3 - M3 to Plugin Directory v3 - M4

#23 @pento
19 months ago

  • Resolution set to fixed
  • Status changed from assigned to closed

In 3211:

Plugin Directory: Add unit test scaffolding, and API unit tests.

Props enej, obenland, pento.

Fixes #1614.

#24 @pento
19 months ago

Note: These tests will run in a VVV install of WordPress, not on a WordPress.org sandbox.

They can be easily extended to work on Travis, if we choose to do that in the future.

#25 @dd32
19 months ago

In 3227:

Plugin Directory: Tests: Remove group from the supported fields, we never actually implemented it. Remove the hot_categories method too, as once again, it was never actually implemented.

See #1614.

#26 @samuelsidler
9 months ago

  • Milestone Plugin Directory v3 - M4 deleted

Milestone Plugin Directory v3 - M4 deleted

Note: See TracTickets for help on using tickets.