Add a test template to Trac tickets

[francina]

During the WordPress 5.6 release cycle, the squad had a test lead and amongst the different tasks, we started actively looking into improving the quality of everything released in WordPress.

To make sure all tickets can get properly tested, developers need to help the testing team to test their submissions.

This means that developers:

• Need to add test cases/steps in all tickets so testers know exactly what to test;
• If a ticket has priority, notify #core-test in Slack

There is a #core-test channel in Slack where we’re happy to help out any developer or tester with testing or the scenarios that need to be covered.

Possible fix
We cannot expect every developer to exactly know what to write to explain to the test team what needs to be tested and how. This means the Trac template needs to be updated with testing information.

The issue is in this: inserting comments as on GitHub can be done, but are not very user friendly, as a Trac comment will look like this:

#! Comment
Note to: ....

This kind of comment in Trac might scare off (new) developers and this is something we absolutely do not wish.

We’d like to propose the following:
When creating a Trac ticket, you can submit various details, such as Keywords, Priority, and Description. Extra steps with details on how to test will be added here. Perhaps it’s wise to automatically add an URL after it with a short ‘how-to’ on how to write testing descriptions. If one is unsure of how to write testing descriptions, the developer will then also learn where to reach out for help.

We should make the fix as clear as possible, but not without forgetting the goal. While we want to make sure that developers will continue to develop and provide the best testing scenarios possible, the ultimate goal is to test more and deliver more quality products.

Template:

• List the exact steps necessary to reproduce the problem.
• Specify the environment details where you reproduced the issue, e.g. WordPress version, PHP version, browser & version, operating system & version, plugins installed, and their versions.
• Paste any relevant error messages.
• Are there any workarounds for this issue?
• How can this issue be tested to ensure it’s fixed?
• Are there any testing dependencies, such as a plugin or script?
• What is the expected behavior after applying the patch?

Thank @monikarao, @carolinegeven, @hellofromtonya, and @ma0sm for brainstorming and review

[dd32]

It's not clear to me what changes are actually being proposed here, but I don't think it should be discussed here - It feels like something that needs taking to the #core team directly to figure out a solution.

From what I can gather, there's a few things being suggested:

• A template for new tickets (Possible)
• An extra field for test directions (Possible, but not doable really - we've been down that road before for a different request)
• Additional Slack notifications (Possible, but probably not how you're thinking)

On the slack front, we could pipe Trac comments that add a keyword like needs-testing to the channel if the team wanted.. it'll be noisy and not what you want though.

[francina]

Ha! Because there is a sentence missing 🤦‍♀️
I'll write a post in the Core blog and then come back here if we reach a conclusion. Thanks!

[dd32]

Since this ticket came up in #meta, I think just having the description field be something like this would do the trick

Replace this text with a brief summary of the issue

** How to reproduce **
Replace this text with steps on how to reproduce

** What happens **
Replace this text with the experience you encounter

** What I expected to happen **
Replace this text with what you expected to happen.

** Environment Details **
WordPress: Replace this text with your WordPress version
Plugins: List any plugins you're running that you think might be relevant
Themes: List the theme you're running if you think it might be relevant

If core wants to use HTML comments, such as <!-- This is a template --> we can always strip those out / replace them with Trac comments upon save via JS too. But I don't think HTML comments are necessarily any more user friendly to the majority of people, except developers, who can probably follow some basic directions :)

Another alternative that would need a developer to take charge of adding, is to create a wizard flow for creating tickets, ​similar to the GitHub Issue creation flow which is just a JS overlay on the ticket creation screen for new users that ultimately just fills out the ticket details for them.

[mai21]

Nice idea @dd32.

I'd recommend having a different template for each ticket type (bug, enhancement, new feature).
Also, having mandatory and optional fields.

Why?
Facilitate reproducing, fixing, and testing the ticket by any team member.

Example:

• For Bug:

As a reporter, I want to have a template to add bug details.
Acceptance criteria:
1- The following fields exist: Title or brief description, precondition, steps, expected o/p, actual o/p, screenshot/ video, affected module, environment ( browser, plugin, theme, OS, version(s)), severity
2- Mandatory fields are: Title, steps, expected o/p, actual o/p, environment
3- Optional fields that aren't filled, won't appear on the ticket
4- Definition or sample data exist for fields when needed i.e Title, steps, precondition..etc

• For Enhancement or new feature:

As a reporter, I want to have a template to add enhancement/ new feature details.
Acceptance criteria:
1- The following fields exist: Title or brief description, the value (why do we need this), precondition, steps, expected o/p, module, technical details
2- Mandatory fields are: Title, value, steps, expected o/p
3- Optional fields that aren't filled, won't appear on the ticket
4- Definition or sample data exist for fields when needed i.e Title, steps, technical details..etc

Note for comments on the ticket:
Recommend the Add Comment Btn be close to the comment text area if doable.

Why?
For better usability.