Automated release packages should support including custom plugins in the package

[miyauchi]

Japanese package is including custom plugin wp-multibyte-patch, so Japanese translation team has to release new version manually.

I guess some localized package want to add fonts or other customizing via including custom plugin like Japanese package.

e.g, ​https://core.trac.wordpress.org/ticket/29664#comment:1

Can automated release packages include custom plugins like wp-multibyte-patch?

Related:

• ​https://make.wordpress.org/polyglots/handbook/translating/packaging-localized-wordpress/automated-release-packages/
• ​https://core.trac.wordpress.org/ticket/19603
• ​https://wordpress.org/plugins/wp-multibyte-patch/

We had a conversation about this ticket on slack.
​https://wordpress.slack.com/archives/C02RP50LK/p1506322215000131

This ticket was moved from following.
​https://core.trac.wordpress.org/ticket/41975

[atachibana]

This is important ticket from the security view, too.

Usually, manual package requires a few days for it, and that means vulnerability risk exists for zero day attack.
Even though WordPress provides auto update function, it will install en_US version during those period, and later re-install ja version. It is too complicated.

[Nao]

As a Japanese GTE/locale manager, I'd support this change.

We are manually packaging JA releases only because we want to make sure all new installs have the WP Multybite Patch plugin included. Other than that, JA package only has minor modifications (translated readme & wp-config-sample.php).

As @atachibana said the manual work can delay the release process even after the translation is 100% completed and I would much rather move away from it if we can.

[ocean90]

Are there any unit tests? Is there a Travis CI setup for testing? Something like that must be available so we can ensure that we don't ship broken packages.

[dd32]

I'd love to see more of the multibyte patch merged into core, unfortunately the source is sparsely documented, and to a non-Japanese user it's not immediately easy to understand if most of it is still needed (For example, the theme-specific stuff can probably be removed, I kind of hate that we allow the wplink.php hack to be shipped in the plugin, a bunch of other things look like we may have made some efforts in core to fix it, etc)

I'd love to see another round of ​https://core.trac.wordpress.org/ticket/19603 with a focus on the multibyte plugin, eliminating as much of it as possible, etc.
Seems like a much better use of developer time than continuing to maintain the plugin and/or change the locale package release process.

[miyauchi]

Replying to ocean90:

Are there any unit tests? Is there a Travis CI setup for testing? Something like that must be available so we can ensure that we don't ship broken packages.

I will write phpunit for wp-multibyte-patch within a few days.
Also, I want to run tests in the WordPress trunk to see what does this plugin change WordPress's behavior.
I will publish those tests on GitHub + Travis CI.

Replying to dd32:

I'd love to see another round of ​https://core.trac.wordpress.org/ticket/19603 with a focus on the multibyte plugin, eliminating as much of it as possible, etc.

I think so too. :)
But I want release package to be able to included custom plugins as soon as possible for now.

Thanks :)

[SergeyBiryukov]

Replying to dd32:

I'd love to see another round of ​https://core.trac.wordpress.org/ticket/19603 with a focus on the multibyte plugin, eliminating as much of it as possible, etc.

I'd love to see that as well, for Japanese and perhaps other "complex" locales. That's what I suggested in #polyglots, instead of changing the package release process.

[ShinichiN]

Replying to SergeyBiryukov:

Replying to dd32:

I'd love to see another round of ​https://core.trac.wordpress.org/ticket/19603 with a focus on the multibyte plugin, eliminating as much of it as possible, etc.

I'd love to see that as well, for Japanese and perhaps other "complex" locales. That's what I suggested in #polyglots, instead of changing the package release process.

I'd also love to see that happen, but I think the inclusion of custom plugin should happen sooner. I'd imagine that merging multibyte patches functions into core could take a long time.

While Japanese package is not ready after security releases, security issues in old minor version remain until the ja version gets manual tests, build, and shipping. I want the Japanese packages built as quick as the other languages can be.

Can we separate these two issues for now?

[SergeyBiryukov]

Replying to ShinichiN:

I'd also love to see that happen, but I think the inclusion of custom plugin should happen sooner. I'd imagine that merging multibyte patches functions into core could take a long time.

Depending on the changes, it might actually happen sooner than changing the build process :) I'd suggest starting with creating a better inline documentation for the Multibyte Patch plugin, re-evaluating its features to see which ones are still necessary, and creating a Core Trac ticket to merge them.

[mayukojpn]

Is it possible to support all package which build manually right now? As mentioned in ​https://core.trac.wordpress.org/ticket/29664#comment:1 some of them rewrites core files.

Here is the list which I heard from @ocean90 who use manual build and I checked and added where they changed:

Locale	SVN Repository	Type
my_MM	​https://i18n.svn.wordpress.org/my_MM/	- Several css/js file replacement in WP core - Additional css/php files in /wp-content/languages/
ja	​https://i18n.svn.wordpress.org/ja/	single plugin
fa_IR	​https://i18n.svn.wordpress.org/fa_IR/	single plugin
sr_RS	​https://i18n.svn.wordpress.org/sr_RS/	single plugin
hu_HU	​https://i18n.svn.wordpress.org/hu_HU/	single plugin
zh_CN	​https://i18n.svn.wordpress.org/zh_CN/	Additional css/js/php files in /wp-content/languages/

Replying to SergeyBiryukov:

Depending on the changes, it might actually happen sooner than changing the build process :) I'd suggest starting with creating a better inline documentation for the Multibyte Patch plugin, re-evaluating its features to see which ones are still necessary, and creating a Core Trac ticket to merge them.

Though the author of the multibyte plugin is busy right now that I saw on Japanese mailing list, he pointed out at least those ticket should be fixed before we start talking about if we merge plugin or not.

• ​https://core.trac.wordpress.org/ticket/27733
• ​https://core.trac.wordpress.org/ticket/22692

I think we should slow down and wait his idea of solution since there are very few people who understand deeply about multibyte issues. I saw his talk ​https://wordpress.tv/2015/11/23/seisuke-kuraishi-wordpress-locale-mechanism/ but it was too difficult for me! 😭

[SergeyBiryukov]

Replying to mayukojpn:

Though the author of the multibyte plugin is busy right now that I saw on Japanese mailing list, he pointed out at least those ticket should be fixed before we start talking about if we merge plugin or not.

• ​https://core.trac.wordpress.org/ticket/27733
• ​https://core.trac.wordpress.org/ticket/22692

Thanks! ​#WP27733 should be looked into, maybe early in 5.0. ​#WP22692 was fixed four years ago :)

[mayukojpn]

Replying to SergeyBiryukov:

Thanks! ​#WP27733 should be looked into, maybe early in 5.0. ​#WP22692 was fixed four years ago :)

It looks ​#WP22692 has a lot of derived tickets about bugs around PCRE which are still open.

I'm not sure if here is right place to continue talking about that - Should we continue using ​#WP22692 as reference of PCRE bugs or make another list in somewhere in a trac or outside? What do you think @SergeyBiryukov ?

[SergeyBiryukov]

Replying to mayukojpn:

I'm not sure if here is right place to continue talking about that - Should we continue using ​#WP22692 as reference of PCRE bugs or make another list in somewhere in a trac or outside? What do you think @SergeyBiryukov ?

I see four tickets linked in ​https://core.trac.wordpress.org/ticket/22692#comment:91, of which three are fixed and one is still open: ​#WP26842.

If there are any other tickets besides ​#WP27733 and ​#WP26842 that need to be fixed before merging other parts of the Multibyte Patch plugin, I'd suggest listing them here for now, and then we can find a way to track them :)

[miyauchi]

This is a quick update on we started to write unit testing with the WordPress and WP Multibyte Patch.
​https://github.com/jawordpressorg/automated-testing-for-wordpress-ja

[miyauchi]

We wrote a unit test for WP Multibyte Patch.
​https://github.com/jawordpressorg/automated-testing-for-wordpress-ja

It is running on Travis CI once a day by Travis CI's cron.

However, I think that this should be triggered with the commit of the WP Multibyte Patch as CI.
I ask Japanese Translation Team to move WP Multibyte Patch into GitHub for enabling CI.
But I couldn't get their agreement.

There are some ticket to solve problems for Japanese users as a first step.

• ​https://core.trac.wordpress.org/ticket/43829
• ​https://core.trac.wordpress.org/ticket/44296

I am going to open some tickets too.

[netweb]

Related: ​#core44541

[miyauchi]

Related: ​#core44548

[miyauchi]

Related: ​#core15955

[miyauchi]

Related: ​#core44662

[miyauchi]

I saw an announcement WP Multibyte Patch will be removed on WordPess 5.0 without any improvements.
​https://ja.wordpress.org/2018/11/04/things-to-know-before-using-wordpress-5-0/

Actually, I know tickets related on WP Multibyte Patch were moved to milestone 5.1.
​https://docs.google.com/spreadsheets/d/13oGbc7AqEN6OUvmze-JDCKXDdruFqsxqSAdI7-b6Lho/edit#gid=0

I can't understand why Japanese team decides to remove this pugin from Japanese package at the next major release.

I am thinking the plugin should be removed in the near future.
But I believe we should keep compatibility for Japanese package, for example, excerpt length or so.

I am hoping that there are chance to have discussion about it.
And I want Japanese team to explain why WP Multibyte Patch should be removed at WordPress 5.0 without any improvements?

[nukaga]

I think that it is best to stop bundling after solving the tickets related to WP Multibyte Patch, which had a milestone moved to WordPress 5.1.
But, the Japanese team announced that it will stop bundling in WordPress 5.0. They may think that changing it makes confusion. Also, the author said that Japanese GTEs team no plan to change this decision in Japanese Slack. They said that even if they are not bundled WP Multibyte Patch in 5.0, there are no big problems. So, the considering the priority, I cannot stop that WP Multibyte Patch will be removed on WordPress 5.0.
The automatic release will advance.

And, I think that the tickets related to WP Multibyte Patch should be continued to avoid various troubles occurring in multibyte of the "complex" locales.

[miyauchi]

​https://wordpress.slack.com/archives/C02RP50LK/p1544127205104000

We can close this ticket because the custom plugin has been removed from Japanese package from 5.0.
Thanks!