Making WordPress.org

Opened 2 years ago

Last modified 4 months ago

#6188 new enhancement

Language Pack generation: trigger on plugin update

Reported by: adamsilverstein's profile adamsilverstein Owned by:
Milestone: Priority: normal
Component: Translate Site & Plugins Keywords:
Cc:

Description

Feature request: always generate language packs on plugin update (or enable a way to trigger that behavior).

For the Site Kit plugin, we have been receiving support requests stating (JavaScript) strings were not translated. As far as we can tell, this happens because these users aren't getting updated translation packs when we update the plugin. I imagine other plugins using JavaScript translations could also run into similar issues.

  • Site Kit uses hashed names for our JavaScript files, we found that without this users would get JavaScript errors after upgrades due to cached JS in their browsers (despite versioning the script tag).
  • Because WordPress i18n keys on the script file names, users need a new language pack for each Site Kit release, regardless of whether new translations were added (since our JavaScript file names change with each release).

As I understand it currently language packs are only regenerated when someone updates a translation string. Is that correct? Would it be possible to have translations packs also regenerated for each plugin version update?

If this would consume excessive resources as a default, maybe it could be an opt-in behavior, with a header option for "auto-regeneration", or perhaps a "translation version" string that when changed would trigger an update?

cc: @ocean90 @swissspidy

Change History (11)

#1 @flixos90
2 years ago

To add to that, while I can see how our usage may be a bit obscure with using file hashes, what about the following sccenario which is more common: A plugin introduces a new JS file including translation strings which already existed before in the plugin somewhere else (e.g. another JS file or PHP). As far as I understand, such plugins would also not get new language packs generated after the plugin update including the new file, which would result in those strings not showing up in the correct locale for end users.

#2 @dd32
2 years ago

Would it be possible to have translations packs also regenerated for each plugin version update?

My understanding is that this already happens, but that it doesn't happen immediately at plugin update.

For example, I see that Site Kit 1.70.0 was released at 2022-03-14 19:23, and the language packs were generated at 2022-03-14 20:08.
For Site Kit 1.69.0, released at 2022-02-28 19:33 first language packs built at 2022-02-28 20:18.

We may be able to decrease this delay for tagged releases, as it's primarily delayed (AFAIK) for plugins which make multiple commits to trunk in a row / commit to trunk and then commit to tags a few minutes later. Ideally we'd delay the plugin update until the language pack is built, but that's not always a viable solution.

There's a delay between the language pack being built, and when it's installed on the site as well, which could be upwards of 24hrs depending on the site, or even delayed until the next time they manually install a plugin if the site doesn't have write access to the language files.

If this would consume excessive resources as a default, maybe it could be an opt-in behavior, with a header option for "auto-regeneration", or perhaps a "translation version" string that when changed would trigger an update?

Not required, We can detect this based on the string references changing, but instead we simply rebuild them for each plugin version it appears.

Site Kit uses hashed names for our JavaScript files, we found that without this users would get JavaScript errors after upgrades due to cached JS in their browsers (despite versioning the script tag).

While this makes zero sense.. it sounds like an issue related to filesystem caching, This sounds like a hosting issue rather than a plugin issue.. as WordPress would be running into the same issue with every plugin and core CSS/JS file.
Nevertheless, this is a choice for the plugin to make, and we don't require you to keep file references consistent.

#3 @ocean90
2 years ago

I can confirm that each version gets its own language pack if there were string changes or the latest theme/plugin version has no language pack yet. But there are indeed some delays:

The issue with not building new language packs if only the references have changed is true and was also reported by me here for Traduttore. A simple regeneration like suggested here won't work at all because language packs are updated by the plugin/theme version and the date of the last edited translation.
But this should only apply to authors who either only use trunk or are changing files in a tagged version.

#4 @adamsilverstein
2 years ago

While this makes zero sense

Given the scale of our install base, we run into every edge case. We found a subset of users regularly missed critical JS updates unless we performed this per version naming change. Unfortunately it looks like this breaks translations so we may need to investigate an alternate approach to solving this issue.

I'm curious: why do translations key off of file names instead of the script handle?

I can confirm that each version gets its own language pack if there were string changes or the latest theme/plugin version has no language pack yet. But there are indeed some delays:

Sounds like close to an hour between plugin update and new language packs generated.

There's a delay between the language pack being built, and when it's installed on the site as well, which could be upwards of 24hrs depending on the site, or even delayed until the next time they manually install a plugin if the site doesn't have write access to the language files.

Could we improve this by scheduling a language pack update for ~ 1hr after a plugin is updated - when we would expect new language packs to be available for the plugin?

#5 @aaemnnosttv
2 years ago

@adamsilverstein if language packs are generated on tagging a new release, can we not delay the update of the stable tag until the translations are ready? Right now, we push the tag with the update of the stable tag at the same time. If all we need to do is delay that update (to prevent people from starting to get the new version) until translations are ready, that seems like the easiest solution here. As long as language packs will be generated on tag (even if not the stable version yet) and we can see when the language packs are ready/complete for a new tag, then I think this should work, no?

cc: @ocean90

#6 @adamsilverstein
23 months ago

If all we need to do is delay that update (to prevent people from starting to get the new version) until translations are ready,

We gave this a try and it failed. The language packs were not generated until we pushed the plugin version update (the tagging didn't trigger language pack regeneration).

@ocean90 or @dd32 - would it possible to trigger language pack regeneration when a new tag is added - that way we could trigger the update before pushing the plugin update itself.

Note that this issue isn't specific to our approach (although it is clearly exaggerated) - any plugin that has a JS filename change or adds a new JS file will be in the same boat - JS translations won't work for people who upgrade - until the new pack is built and their WordPress downloads it (maybe 24 hours at worst?).

A more complex/complete solution would be when a plugin is updated with new JS strings/files, delay releasing on .org until the language packs have been regenerated. This would solve for all cases automatically. Is something like that even feasible?

#7 @dd32
23 months ago

We gave this a try and it failed. The language packs were not generated until we pushed the plugin version update (the tagging didn't trigger language pack regeneration).

Correct, the way translate.wordpress.org currently works, is that there's a Stable project, and a Develop project.

The stable is where language packs come from, and there isn't a way to have the stable project show the upcoming releases strings until all of the processes are run (See Ocean's comment above)

A more complex/complete solution would be when a plugin is updated with new JS strings/files, delay releasing on .org until the language packs have been regenerated. This would solve for all cases automatically. Is something like that even feasible?

This is definitely something that could be made to happen, but there's other short-comings in the system, we don't really have anything that allows for an "upcoming release" in a way which would work with our translation system process. Release Confirmation exists, but it's still not an "Upcoming release" / Beta release type versioning thing that we'd need for this.

Delaying the plugin release (max 1hr perhaps) until translation packages are built is a reasonable option, and one that I would probably be in favour of (At least, for plugins who currently have at least one language with >90% translation status).

We might not be able to delay the release of the plugin, but we might be able to delay the update notifications for a plugin, to allow time for the packages to be built..

There isn't exactly a straight forward easy option here with the way everything interacts, and how most of it is triggered by a SVN commit (and we have to wait, because often committers follow it up with another commit 15 minutes later as part of it...), that's why it's not just been an easy-fix..

#8 follow-up: @adamsilverstein
23 months ago

Delaying the plugin release (max 1hr perhaps) until translation packages are built is a reasonable option, and one that I would probably be in favour of (At least, for plugins who currently have at least one language with >90% translation status).

Also, only for plugins with JavaScript translations, PHP translations shouldn't be an issue.

that's why it's not just been an easy-fix

Understand it is complex an appreciate any work to improve!

#9 in reply to: ↑ 8 @dd32
23 months ago

Replying to adamsilverstein:

Also, only for plugins with JavaScript translations, PHP translations shouldn't be an issue.

PHP translations are affected, just not to the same extent.

For PHP translations, any newly added strings, even if translated at the time of plugin release, won't be available in a language pack. Not as bad as the JS situation, but still not ideal.

#10 @adamsilverstein
22 months ago

We might not be able to delay the release of the plugin, but we might be able to delay the update notifications for a plugin, to allow time for the packages to be built..

I'll try to take a look thru the code to see if we could make this work; maybe this is something we could work on at contributor day.

#11 @aaemnnosttv
4 months ago

Howdy friends 👋 Any update on this one? It seems like this continues to be an issue without a workaround.

Note: See TracTickets for help on using tickets.