Require source and map files for JS

[kkmuffme]

One of the core values of WP is being an open ecosystem.

With more and more plugins using WP blocks or other JS (react), the amount of plugins/themes shipped without core JS files and maps increases.

These plugins/themes are therefore a closed box - any console errors are unactionable (since it's from minified JS without a map) and the plugin can also not be extended easily (since the source code is obfuscated in the minified JS)

Therefore I suggest that:

• plugins/themes listed in the directory must contain .map files for minified/obfuscated JS
• plugins/themes listed in the directory must contain unminified/unobfuscated source JS (or coffee/ts/...) files
• plugins/themes listed in the directory must not contain obfuscated PHP (I think that's the case already now?)

This should obviously also apply for WP core.

[rixeo]

Most plugins will host the uncompressed and un-minified versions in a public repo like Github or Bitbucket to reduce the overall plugin size.
I agree that php code should not be obfuscated

[kkmuffme]

Some will, but many won't. And of those that won't, many will not even provide it when you email the author to get them.

In any way, .map files must be included with the plugins, as otherwise error tracking will be impossible (you'd have to get all the .map files from the repo and upload it to your error handling tool manually, which is impossible)

[rixeo]

I agree, but this can only be enforced during the initial plugin or theme review process. Some developers do not know how to generate map files. Its good practice but something like this will take sometime and planning.

Do you have any suggestions on how best to handle this for existing themes and plugins (as there are thousands)?

Some will, but many won't. And of those that won't, many will not even provide it when you email the author to get them.

In any way, .map files must be included with the plugins, as otherwise error tracking will be impossible (you'd have to get all the .map files from the repo and upload it to your error handling tool manually, which is impossible)

[kkmuffme]

If the developer knows how to minify their JS, they will certainly know how to include the .map in their build process (if your plugin only contains unminified JS, you don't need a map anyway)

Atm it's even missing in WP core in a few cases, but there's a discussion to fix this with the next release.

Do you have any suggestions on how best to handle this for existing themes and plugins (as there are thousands)?

I am not too familiar with the plugin directory, but I assume this shouldn't be too hard to do as a first step: automatically find all plugins that contain a .min.js but no .map and .js with same name, send message/email to the plugin author that this will be required going forward (or: just give an error when a new plugin version is added, that is missing either file?)

--

For future automation of this, in addition to the above check, one could simply remove all code comments from the JS file and if the remaining code of the file is less than e.g. 5 lines, one can be pretty certain that this is minified JS (even when it doesn't have a .min.js in name), to avoid things being missed (e.g. when the .js is generated from .coffee/.ts/whatever)

[rixeo]

Not all developers use build tools to minify their js files :)

If the developer knows how to minify their JS, they will certainly know how to include the .map in their build process (if your plugin only contains unminified JS, you don't need a map anyway)

You can find WordPress source code here ​https://github.com/WordPress/wordpress-develop As mentioned, some developers would rather store this in Github or any other version control systems

Atm it's even missing in WP core in a few cases, but there's a discussion to fix this with the next release.

Not all plugin and theme developers use tools to build, and for most they prefer to host these on Github. I don't this this is something WordPress should enforce as different plugin and theme authors use different methods/means to build their products. This is just my opinion though and I do not currently see a valid reason or a good way to enforce this without increasing the size of themes or plugins

[kkmuffme]

Not all plugin and theme developers use tools to build

How do these people create their .min.js files then?

[rixeo]

Not ll plugin authors minify their files. For example Akismet ​https://plugins.trac.wordpress.org/browser/akismet/trunk/_inc/akismet.js . I can't speak for all developers and creators, but its a broad area to assume that most or all themes and plugins use build tools to generate js and css.

How do these people create their .min.js files then?

[kkmuffme]

Not sure I get your point. You only need a .map if your files are minified or compiled (and this whole request is purely about where we have compiled/minified JS)
So nothing would change for people who just have regular .js in their plugins/themes.

[Otto42]

This is already covered in the plugin guidelines by guideline 4.

​https://developer.wordpress.org/plugins/wordpress-org/detailed-plugin-guidelines/#4-code-must-be-mostly-human-readable

There is no plan to require any specific type of file, be it a map file or anything else. The providing of the source depends on the source material. Additionally, these files can be provided separately from the material of the "compiled" plugin files itself. If a plugin does not provide source, then they can be reported to the plugins team, and they will handle it.

[kkmuffme]

This should be updated to require sourcemaps for compiled/minified JS too

[dd32]

This should be updated to require sourcemaps for compiled/minified JS too

I'm not against adding a requirement for sourcemaps personally, but it's not as easy as just "require a source map"..

Sourcemaps as generated by the majority of tools expect that the source is bundled with the minified files, but as you've seen with WordPress.org plugins, they're often not.. but sourcemaps DO allow for the source files to be hosted at a different remote location, via a GitHub repo for example.

What we'd probably need is published build steps on how to generate source-maps for plugins that refer to a GitHub repo (or other public location of the files), as an optional thing for quite some time before requiring it, if we could even require it.

That would allow for plugins guideline 4 to be pragmatically verified, which is something we can't really currently do, it has to be reviewed by a human.

But, hosting the source-files remotely and referencing that location through the sourcemap might make some fall afoul of guidelines 7/8 inadvertantly with plugins not being allowed to offload loading of JS/CSS/image files.. although I guess it's not really offloading, as the minified files would stil be in use, it's just developer tools inspecting that would use them.

After saying all that.. I disagree that we should require it, but I think we should educate authors on how to include a sourcemap that points to the source files AND that if minified files are included the source location MUST be included in the readme (which it often isn't)

[kkmuffme]

@dd32

expect that the source is bundled with the minified files, but as you've seen with WordPress.org plugins, they're often not

Which already is not allowed: Rule 4 technically requires the source code to be included:
​https://developer.wordpress.org/plugins/wordpress-org/detailed-plugin-guidelines/#4-code-must-be-mostly-human-readable

Again, automating this would be incredibly simple:
if we have a .min.js/.min.css file (or for some super-smarts a file with line lengths exceeding a specific character count), it will require a .js/.css without min and same name, and for js additionally a .map file
There are some edge cases, but those could be ironed out once we look at actual examples, as that would probably cover 98% of cases already.

[dd32]

Replying to kkmuffme:

Which already is not allowed: Rule 4 technically requires the source code to be included:
​https://developer.wordpress.org/plugins/wordpress-org/detailed-plugin-guidelines/#4-code-must-be-mostly-human-readable

Minified code doesn't conflict with that rule, and source files do not need to be included. As long as they document a public location of the source non-minified build repo, it's fine. Only one of the bullet points in that rule needs to be adhered to.

Again, automating this would be incredibly simple:

Yes, and most build tools already do that. But since the source files will rarely be included with the deployed plugin, there's no point in their build tools including the source map files in the distribution package.

To be clear; This isn't going to happen, plugin authors are NOT going to comply, it's a waste of the plugin reviewers time attempting to enforce a rule that benefits very few people and is only going to cause authors to debate the underlying reason for it.

[kkmuffme]

Why are they not going to comply? WP core and tons of plugins already have it.

going to cause authors to debate the underlying reason for it.

Why? The underlying reason is very clear:
Make the web more accessible.

The benefit of a map file is that errors get more accessible for every plugin user, for end users and for the plugin authors themselves, since they will receive more actionable errors.

It's only hearsay atm why this shouldn't be introduced, I didn't really get any compelling arguments of the drawbacks of enforcing this.

[dd32]

Why are they not going to comply? WP core and tons of plugins already have it.

You just have to look at why they've removed the source files in the first place.

Why? The underlying reason is very clear:
Make the web more accessible.

Unfortunately that's not something that's enough to push most developers along.

This rule isn't going to change, this ticket isn't going to go anywhere, you can complain directly to plugin authors if you wish, and if any don't publish where the source files are located, report it to the plugin team.

[kkmuffme]

You just have to look at why they've removed the source files in the first place.

Which is why?
To make the zips smaller? - this is a complete non-issue nowadays, e.g. WP gutenberg/react is so bloated, that file size/bundling unnecessary stuff is not something anyone is concerned with anymore.

Unfortunately that's not something that's enough to push most developers along.

Exactly, which is why WP should enforce this.
This is just like how Google now enforces accessibility by including it in Lighthouse reports and including it for SEO ranking.

[rixeo]

Hey @kkmuffme,

I think as its a notable recommendation, I do not see it being enforced at the moment as some developers will ignore this while others already have public facing repositories with the source files.
Plugin file sizes do matter as not everyone has access to fast and reliable internet to update/download plugin zip files.
This would be more of a developer focused feature used in debugging as opposed to a normal end user

[kkmuffme]

@rixeo

Plugin file sizes do matter as not everyone has access to fast and reliable internet to update/download plugin zip files.

The average plugin has less than 5 JS files with a total average size of less than 250kb. When it's zipped for upload, it's 50-100kb - if you the slowest of the slow, a 56k modem, this will only add 15 seconds to your upload time. (btw loading ANY WordPress admin page would take you literally minutes at that speed with all the assets loaded by it)

This isn't an argument against it requiring non-minified JS.

This would be more of a developer focused feature used in debugging as opposed to a normal end user

Nope, not at all. This will be incredibly useful to a normal end user for 2 reasons:
1) the end user will have an actionable error message that can be reported to the website owner, which will ensure that the website will be working correctly faster
2) the website owner will have actionable errors - this means a) he can either self-fix it or b) can report it to the plugin owner, to fix it

Without the .map, ALL errors the plugin JS produces are useless. Nobody can use them for anything and these issues can also not be fixed in many cases.

[rixeo]

Hey @kkmuffme,

For now its more of a nice-to-have feature, but not something that can be enforced. Plugins have a support page and a rating page that can be used to help push this on the affected plugin