WordPress.org

Making WordPress.org

Opened 6 years ago

Closed 5 years ago

#340 closed enhancement (fixed)

Add i18n Team Crawler Plugin to Make Polyglots

Reported by: netweb Owned by: nacin
Milestone: Priority: normal
Component: International Sites (Rosetta) Keywords:
Cc:

Description

Source: https://github.com/markoheijnen/wp-i18n-team-crawler

Per #wordpress-polyglots IRC Chat here

Extend plugin to include 'dormant' translations, we're at 3.8, anything 3.6 and below is dormant.

Attachments (7)

340.diff (5.1 KB) - added by markoheijnen 6 years ago.
340.2.diff (8.2 KB) - added by iandunn 5 years ago.
Refactor 340.diff to access data directly instead of using API
340.3.diff (11.0 KB) - added by iandunn 5 years ago.
Refactored 340.2.diff to work on make.wordpress.org
340.4.diff (17.7 KB) - added by iandunn 5 years ago.
340.5.diff (18.3 KB) - added by iandunn 5 years ago.
Minor fixes
340.6.diff (3.5 KB) - added by ocean90 5 years ago.
Add filter for locales with 100% translated strings
340.6.2.diff (9.7 KB) - added by ocean90 5 years ago.

Download all attachments as: .zip

Change History (39)

#1 @markoheijnen
6 years ago

  • Cc info@… added

#2 @vanillalounge
6 years ago

  • Cc jfontainhas@… added

@markoheijnen
6 years ago

#3 @markoheijnen
6 years ago

I just uploaded my plugin I'm using on wpcentral.io. I'm sure getting the data can be smarter. Also I was curious what to do when there are no validators. Do we have a form where people could claim responsibility?

Currently the sites are requested by using the locales file of GlotPress. This is now an SVN external.

#4 @Otto42
6 years ago

Can somebody explain what the purpose of this plugin is?

It appears to be making remote requests to the api server for something, which means it makes no sense to put this on a make site as-is, or indeed to use a transient for caching it. The website itself already has access to the same data, but directly.

A good description of what the code is trying to do and display might mean that we can create a much simpler solution, such as simply writing code to pull and display the desired data.

#5 @nacin
6 years ago

  • Component changed from make.wordpress.org / P2 to International Sites (Rosetta)
  • Owner set to yoavf
  • Status changed from new to assigned

The code is pretty straightforward as to what it's trying to do. Most Rosetta code isn't open-sourced yet (in progress) so this is what was written. Obviously, when adapted for WP.org we'd drop the API calls.

yoavf is gonna implement this; he can handle the transition to the internal calls.

#6 @samuelsidler
5 years ago

  • Owner changed from yoavf to nacin

@iandunn
5 years ago

Refactor 340.diff to access data directly instead of using API

#7 @iandunn
5 years ago

340.2.diff refactors 340.diff to run locally on WordPress.org rather than using the API.

  • I started with the GitHub repo instead of 340.diff, because it was more recent, but I used f38bc24 instead of HEAD because the commits after that were caching bits that'd be removed anyway.
  • I assumed that Nacin changing the component to International Sites implied that the shortcode will run on global.wordpress.org instead of make.wordpress.org, but I can adjust it to work on Make if that was a bad assumption.
  • The stats in the header vary slightly from wpcentral.io, but it looks like they're correct. I'm guessing the difference is caused by the caching in the wpcentral.io version.
  • The translators/validators bits were removed because they weren't being used by anything, but I kept the existing $locale_data structure so they can be added back in the future.

Nacin, can you confirm where you intended this to live? After that, I think it's ready to commit, unless anyone has any feedback.

#8 @markoheijnen
5 years ago

I broke caching stuff till a few days ago but I did fixed that now.

Nacin did told me that the get_locale_url() method could be replaced by a WordPress.org function. I'm wondering why the plugin is not inside a folder but the root directory.

We need to look into translators/validators and how to display that data.

#9 @samuelsidler
5 years ago

It'll run on make.wordpress.org/polyglots/, specifically, not on global.wordpress.org. I think nacin moved it there because that's where a lot of the work is being done.

As Marko said, we'll want translators/validators back, but we'll need a good way to display that. It's okay to skip it for now.

#10 @iandunn
5 years ago

Nacin did told me that the get_locale_url() method could be replaced by a WordPress.org function.

Sound good. I'll take a second look and see what would be better.

I'm wondering why the plugin is not inside a folder but the root directory.

I just goofed it up when copying it from my sandbox to the Meta repo checkout. I'll fix that when I make the other changes.

It'll run on make.wordpress.org/polyglots/, specifically, not on global.wordpress.org. I think nacin moved it there because that's where a lot of the work is being done.

Sounds good, I'll make the necessary adjustments.

@iandunn
5 years ago

Refactored 340.2.diff to work on make.wordpress.org

#11 @iandunn
5 years ago

340.3.diff is updated to work on make.wordpress.org instead of global.wordpress.org. but get_locale_url() still needs work. I found some code in international_forums() that looks like it could be used as a canonical source for those URLs, but it will need to be abstracted out.

This patch also adds translator/validator data back in, but I created a [wp-locale-contributors] shortcode for displaying it instead of using custom post types, since the CPT in the GitHub repo was a caching layer for wpcentral.io.

#12 @valeriosza
5 years ago

  • Cc valeriosza@… added

#13 @sadeghr
5 years ago

  • Cc web.webmaster20@… added
  • Component changed from International Sites (Rosetta) to buddypress.org
  • Priority changed from normal to high

I'm at the translations.
I got these numbers translate to
https://translate.wordpress.org/projects/buddypress/dev/fa/default
Thank you
Username:
sadeghr

#14 @samuelsidler
5 years ago

  • Component changed from buddypress.org to International Sites (Rosetta)
  • Priority changed from high to normal

@sadeghr: Your comment doesn't belong here. If you need help with translating, try here: https://make.wordpress.org/polyglots/

This ticket was mentioned in Slack in #meta by iandunn. View the logs.


5 years ago

@iandunn
5 years ago

#16 follow-up: @iandunn
5 years ago

There's still more to do here, but 340.4.diff gets us a lot closer. Here are the highlights:

  • Locale subdomains and latest releases pulled from MySQL instead of being hardcoded and scraped from the filesystem, respectively.
  • Front end always built from cached data, which gets updated via cron every 5 minutes.
  • Merged latest commits from the GitHub repo (through ab3f685), with a few exceptions. See below.
  • Refactored HTML output into external views instead of being echo'd by PHP, to improve readability.
  • Modularized the locale status counts (e.g., minor-behind, no-site, etc) for better caching and maintainability.
  • Merged inc/crawler.php into the main class, since it only had 2 methods left in it after pulling more data from MySQL.
  • Removed references to "crawler" since the wporg version isn't actually crawling anything.
  • Removed the secondary shortcode that was introduced in 340.3.diff, so that both views are generated by the main shortcode.
  • Lots of minor tweaks.

I didn't merge the flag image on the locale details page, because it was being pulled from a remote server that we can't trust, and which doesn't support HTTPS, which would lead to mixed-content warnings. It also seems wrong, since locales don't map cleanly to countries.

I also didn't merge the Core translations column, since I couldn't find a good way to access the data.

  • Calling GP_Route_Locale::single() directly in PHP would require include()'ing all of GlotPress alongside WP, which seems like a recipe for disaster.
  • Requesting https://translate.wordpress.org/api/languages/{locale} would require a separate call for each locale.
  • Pulling directly from the database would mean duplicating a lot of the logic from the GlotPress API.

Any ideas for a better approach? Maybe we can create a GlotPress plugin to add the % complete to the https://translate.wordpress.org/api/languages endpoint?

#17 in reply to: ↑ 16 @iandunn
5 years ago

Replying to iandunn:

I didn't merge the flag image [...] because [...] locales don't map cleanly to countries.

Er, nevermind, I was thinking of languages, not locales. So, the flag might be fine, but we'll need to find an open-source image set that we can bundle, rather than relying on a remote service.

#18 @vanillalounge
5 years ago

the flag might be fine

Very sorry to barge in this late, but I'm not sure I follow the 'flag' logic. The way I see it, it is simply not needed, nor is it even relevant. The country-to-language relationship (and vice-versa) is n-to-n, so that a flag adds nothing to the list.

#19 follow-up: @ocean90
5 years ago

Looking at 340.4.diff:
The $local_data array shouldn't be key'ed by wp_locale. We stopped with this in core and on API site to be future-proof in terms for variantions of the same locale (formal/informal). I think it should also use the same sorting as we have for for the language dropdown on install, see https://meta.trac.wordpress.org/browser/sites/trunk/api.wordpress.org/public_html/translations/lib.php#L77.

#20 in reply to: ↑ 19 @iandunn
5 years ago

Replying to ocean90:

The $local_data array shouldn't be key'ed by wp_locale. We stopped with this in core and on API site to be future-proof in terms for variantions of the same locale (formal/informal).

Nice catch, thanks for the heads up :)

@iandunn
5 years ago

Minor fixes

#21 @iandunn
5 years ago

340.5.diff only has a few small fixes over 340.4.diff, but it may be awhile before I have time to circle back to this again, so I wanted to make sure the latest changes were here in case anyone else wants to work on it.

#22 @nacin
5 years ago

In 1061:

First pass at http://make.wordpress.org/polyglots/teams/.

props markoheijnen, iandunn.
see #340.

#23 @nacin
5 years ago

There's a lot more I want to improve/fix here, but [1061] is my first pass after going through the patches here.

#24 @markoheijnen
5 years ago

Great work! After compared some data in general it looks fine but somehow nl_NL is showing it seems everyone who translated for Dutch but the API does return only a handful.

#25 @ocean90
5 years ago

In 1068:

I18N Teams: Hide download links for locales which have never released a version.

see #340.

#26 @ocean90
5 years ago

In 1139:

i18n Teams: Improve translation counts.

  • Fix division by zero warning.
  • Show 0% for locales which have the projects, but without translations.

see #340.

#27 @ocean90
5 years ago

In 1140:

i18n Teams: On details page use english name of the locale.

fixes #795.
see #340.

#28 @ocean90
5 years ago

In 1157:

GlotPress templates: Add link to i18n teams in header.

see #340.

@ocean90
5 years ago

Add filter for locales with 100% translated strings

@ocean90
5 years ago

#29 @ocean90
5 years ago

In 1163:

i18n Teams: Add filters for translation status.

see #340.

#30 @ocean90
5 years ago

In 1164:

i18n Teams: Bump version for assets.

see #340.

#31 @ocean90
5 years ago

In 1190:

WP i18n Teams: Avoid notices if locale has no site yet.

see #340.

#32 @samuelsidler
5 years ago

  • Resolution set to fixed
  • Status changed from assigned to closed

Fixed a while ago. :)

Note: See TracTickets for help on using tickets.