Plugin submission: Show errors with low severity to users

[davidperez]

We've finished in PCP the ​PR related to showing ERRORS with low severity (less than 7) as a new category of errors ERRORS_LOW_SEVERITY. We want not to block plugins submissions with less than 7 severity errors, but show them to the users so they can work on them after they get the manual review. These errors could have false positives, so it's important to say to the user that review them.

It will help to the Team, as we will review fewer issues in every plugin.

Adding this flag: --include-low-severity-errors in the WP CLI command will give new ERRORS as ERRORS_LOW_SEVERITY.

What we need is:

• Add this new flag in plugin submission.
• Show the ERRORS_LOW_SEVERITY with a disclaimer.

[frantorres]

I suggest the disclaimer to be something like
"Note that automated tools can give false positives, or may miss issues. If you believe there are false positives, do not worry, a reviewer will check it during the review."

[dd32]

Replying to frantorres:

I suggest the disclaimer to be something like
"Note that automated tools can give false positives, or may miss issues. If you believe there are false positives, do not worry, a reviewer will check it during the review."

Almost every item that Plugin Check ever reports should have that disclaimer though right?

The existing line after the output is:

Note: While the automated plugin scan is based on the Plugin Review Guidelines, it is not a complete review. A successful result from the scan does not guarantee that the plugin will be approved, only that it is sufficient to be reviewed. All submitted plugins are checked manually to ensure they meet security and guideline standards before approval.

Perhaps that could be clarified that listed issues may be false-positives and should just be raised with the reviewer.

[dd32]

In 14220:

Plugin Directory: Plugin Check: Run the plugin-check process through proc_open() so we can extract the STDERR output.

See #7840.

[dd32]

In 14222:

Plugin Directory: Better handling for plugin-check output.

See #7840.

[dd32]

In 14223:

Plugin Directory: Plugin Check: Fix a variable typo that slipped into [14222].

See #7840.

[frantorres]

Replying to dd32:

Almost every item that Plugin Check ever reports should have that disclaimer though right?

Perhaps that could be clarified that listed issues may be false-positives and should just be raised with the reviewer.

Yes, maybe that disclaimer could be completed with something along the lines of "The warnings displayed may contain false positives, if you think this is the case don't worry, a reviewer will check it during the review."

[davidperez]

Yes, errors are correct as we are not seeing any false positives. In warnings it happens like we are saying. It could contain false positives.

[dd32]

In 14239:

Plugin Directory: Plugin Check: Include low-severity issues in the output of plugin-check on plugin submission.

Fixes #7840.

[davidperez]

Thanks! It seems great to me!