WordPress.org

Making WordPress.org

Opened 5 months ago

Closed 4 months ago

Last modified 4 months ago

#2981 closed enhancement (fixed)

Proper destination for codebase logging WP PHPUnit test run results?

Reported by: danielbachhuber Owned by: pento
Milestone: Priority: normal
Component: General Keywords:
Cc:

Description

Some of us have been working on a WP PHPUnit test suite runner to run the test suite against arbitrary web host environments. This runner then will report its results to a central WordPress instance, where results will be displayed in aggregate.

What's the best place for this code to live? Should it be kept isolated, or could we create another site on the WordPress.org network (or potentially reuse the make/hosting site)?

The reporter instance is pretty much a REST API endpoint with custom post type.

cc @octalmage @mikeschroder @boogah

Change History (17)

This ticket was mentioned in Slack in #hosting-community by danielbachhuber. View the logs.


5 months ago

#2 @mikeschroder
5 months ago

I think make/hosting is a fine place to start out (especially if this lets us test things/iterate more easily)

I'd love to have it somewhere more central with other automated testing pass/failures eventually.

#3 follow-up: @pento
5 months ago

I agree, make/hosting is a good place to start with this. It's not going to be super high volume, so it probably doesn't need to be ported over to api.wordpress.org - the final place will be a WordPress site somewhere on w.org.

A few things to note:

  • The server repo definitely needs to be moved to the WordPress GitHub org, and I think the client repo probably should be, too.
  • I think o2 is fine with custom post types (see the handbook, for example), but it might be a bit quirky. Be prepared.
  • I'm not wild about the server using edit_posts as the auth capability. A custom capability would be better, so that hosts can link it to a host-controlled w.org account, that just needs to be a user on make/hosting.

Finally, it looks like the server front end is being built as a theme? Is that still the plan, or will it be a plugin, instead? I think I'd prefer a plugin, that'd be easier to integrate with w.org's existing UI.

#4 in reply to: ↑ 3 ; follow-up: @danielbachhuber
5 months ago

Replying to pento:

  • The server repo definitely needs to be moved to the WordPress GitHub org, and I think the client repo probably should be, too.

I agree. What's the best way to go about this? If we add you as an admin on the repos, can you transfer them in and then create a team for us to manage contributors?

  • I think o2 is fine with custom post types (see the handbook, for example), but it might be a bit quirky. Be prepared.

Lovely :)

  • I'm not wild about the server using edit_posts as the auth capability. A custom capability would be better, so that hosts can link it to a host-controlled w.org account, that just needs to be a user on make/hosting.

Good suggestion, we'll take care of it.

Finally, it looks like the server front end is being built as a theme? Is that still the plan, or will it be a plugin, instead? I think I'd prefer a plugin, that'd be easier to integrate with w.org's existing UI.

We're doing a bit of refactoring at the moment. The high-level overview to the refactor is here: https://paper.dropbox.com/doc/WP-Unit-Test-Reporter-tssvu4ACKCQPcY3jHnaiX

tl;dr: Yes, it will just be a plugin.

#5 in reply to: ↑ 4 ; follow-up: @pento
5 months ago

Replying to danielbachhuber:

I agree. What's the best way to go about this? If we add you as an admin on the repos, can you transfer them in and then create a team for us to manage contributors?

I'm not entirely sure if admins can transfer user owned repos, but lets find out!

I'll also need a list of GitHub usernames to add to the team.

#6 in reply to: ↑ 5 @danielbachhuber
5 months ago

Replying to pento:

I'm not entirely sure if admins can transfer user owned repos, but lets find out!

Great, I've asked Jason to add you.

I'll also need a list of GitHub usernames to add to the team.

danielbachhuber and octalmage should be sufficient to start.

#7 @pento
4 months ago

  • Owner set to pento
  • Status changed from new to assigned

Repos have been transferred to the WordPress org, and the team now exists.

https://github.com/WordPress/phpunit-test-runner
https://github.com/WordPress/phpunit-test-reporter
https://github.com/orgs/WordPress/teams/automated-testing/

Need to consider the authentication issue before the reporter plugin can be added to W.org.

#8 @danielbachhuber
4 months ago

@pento I'm going to do a full code review too and add more tests.

#9 @pento
4 months ago

Noice.

#10 @danielbachhuber
4 months ago

In 5804:

phpunit-test-reporter: Create empty folder with SVN ignore

See #2981

#11 @danielbachhuber
4 months ago

In 5805:

phpunit-test-reporter: Initial commit

See #2981

#12 @danielbachhuber
4 months ago

@pento I committed the plugin to meta SVN so it's ready for deploy + activation whenever you're around. I'll also need admin access on make/hosting to manage these bot users.

#13 @pento
4 months ago

  • Resolution set to fixed
  • Status changed from assigned to closed

Plugin is deployed and activated, @danielbachhuber is now an admin on make/hosting.

Have fun, y'all. :-)

#14 @danielbachhuber
4 months ago

In 5806:

phpunit-test-reporter: Misc display changes

  • Use rewrite rule hackery for consistent breadcrumbs
  • Display test result timing when present
  • Drop empty column from table view

See #2981

#15 @danielbachhuber
4 months ago

In 5807:

phpunit-test-reporter: Only filter content for queried object

See #2981

#16 @danielbachhuber
4 months ago

In 5808:

wporg-breathe: Properly set up $post

Otherwise, any filters using the global post will be using the wrong post.

See #2981

Note: See TracTickets for help on using tickets.