WordPress.org

Making WordPress.org

Opened 2 weeks ago

Last modified 2 weeks ago

#5681 new enhancement

Add a test template to Trac tickets

Reported by: francina Owned by:
Milestone: Priority: normal
Component: Trac Keywords:
Cc:

Description

During the WordPress 5.6 release cycle, the squad had a test lead and amongst the different tasks, we started actively looking into improving the quality of everything released in WordPress.

To make sure all tickets can get properly tested, developers need to help the testing team to test their submissions.

This means that developers:

  • Need to add test cases/steps in all tickets so testers know exactly what to test;
  • If a ticket has priority, notify #core-test in Slack

There is a #core-test channel in Slack where we’re happy to help out any developer or tester with testing or the scenarios that need to be covered.

Possible fix
We cannot expect every developer to exactly know what to write to explain to the test team what needs to be tested and how. This means the Trac template needs to be updated with testing information.

The issue is in this: inserting comments as on GitHub can be done, but are not very user friendly, as a Trac comment will look like this:

#! Comment
Note to: ....

This kind of comment in Trac might scare off (new) developers and this is something we absolutely do not wish.

We’d like to propose the following:
When creating a Trac ticket, you can submit various details, such as Keywords, Priority, and Description. Extra steps with details on how to test will be added here. Perhaps it’s wise to automatically add an URL after it with a short ‘how-to’ on how to write testing descriptions. If one is unsure of how to write testing descriptions, the developer will then also learn where to reach out for help.

We should make the fix as clear as possible, but not without forgetting the goal. While we want to make sure that developers will continue to develop and provide the best testing scenarios possible, the ultimate goal is to test more and deliver more quality products.

Template:

  • List the exact steps necessary to reproduce the problem.
  • Specify the environment details where you reproduced the issue, e.g. WordPress version, PHP version, browser & version, operating system & version, plugins installed, and their versions.
  • Paste any relevant error messages.
  • Are there any workarounds for this issue?
  • How can this issue be tested to ensure it’s fixed?
  • Are there any testing dependencies, such as a plugin or script?
  • What is the expected behavior after applying the patch?

Thank @monikarao, @carolinegeven, @hellofromtonya, and @ma0sm for brainstorming and review

Change History (3)

This ticket was mentioned in Slack in #core-test by francina. View the logs.


2 weeks ago

#2 @dd32
2 weeks ago

It's not clear to me what changes are actually being proposed here, but I don't think it should be discussed here - It feels like something that needs taking to the #core team directly to figure out a solution.

From what I can gather, there's a few things being suggested:

  • A template for new tickets (Possible)
  • An extra field for test directions (Possible, but not doable really - we've been down that road before for a different request)
  • Additional Slack notifications (Possible, but probably not how you're thinking)

On the slack front, we could pipe Trac comments that add a keyword like needs-testing to the channel if the team wanted.. it'll be noisy and not what you want though.

#3 @francina
2 weeks ago

Ha! Because there is a sentence missing 🤦‍♀️
I'll write a post in the Core blog and then come back here if we reach a conclusion. Thanks!

Note: See TracTickets for help on using tickets.