GitHub integration: Showing "All checks pass" on failing PR

[ryelle]

The issue was seen on ​https://core.trac.wordpress.org/ticket/53246 — the action checks failed, but the UI in trac reported the PR as "All checks pass".

I haven't investigated to see if this is working on other tickets.

[dd32]

(This is a note for myself, and any others who look at it, mostly thinking out loud)

Before opening the trac ticket up, I looked at the cached data:

"mergeable_state":"unknown","check_runs":[],"reviews":[],

After loading the ticket, I saw

✅ All checks pass, GitHub Actions running

And I can now see the cached data is updated to be

"mergeable_state":"blocked","check_runs":{"GitHub Actions":"in_progress"},"reviews":[],

At a guess, I'm thinking an API request to GitHub may have timed out, resulting in no check_runs being cached, resulting in an all-checks-pass?

Having no check_runs is expected for some PRs, for example to WordPress/wordpress.org, but not for core trac. But having All checks pass without check_runs doesn't make sense, until you consider that for meta it shows whether it can be merged or not (even tho we don't merge PRs on GitHub)

[johnbillion]

I just saw this same problem on ​https://core.trac.wordpress.org/ticket/47280. PR 2829 has test failures but is showing as "All checks pass" on Trac.