Explore Crowdin Context Harvester for translation of product strings

• Crowdin Context Harvester CLI: https://store.crowdin.com/crowdin-context-harvester-cli
• Video: https://www.youtube.com/watch?v=m73j7m4F1zU

[source: Crowdin YouTube channel, and LinkedIn post: https://www.linkedin.com/posts/crowdin_crowdin-context-harvester-cli-activity-7200825991286444033-JJ62/ ]

added to epic &24

changed the description

Ideas for what we can do:

• Test how it works, gather feedback
• Use the tool to simulate manual research, build exception scenarios
• Saturate strings that are synced to Crowdin for translation with context info, such as MR or issue/epic links
• Inject context info directly to .pot file as comments / attributes, to be readable by Crowdin 
• Agentize it to use Blame for context harvesting. Create a series of prompts, e.g. find this string reference in code, look for fragments, look for adjacent strings, explain this code, etc. - use ideas from this experiment: https://gitlab.com/gitlab-com/localization/localization-team/-/issues/130+
• etc.

One interesting possibility could be if there's an option to integrate to the gitlab pipeline, which might be able to automatically generate context based on merge requests. The assumption is, the newly added string has the relevant code snippet in the same merge requests (for most cases).

Even if this isn't an option, this can still be beneficial if it can be plugged into the localization pipeline something similar to this:

Gitlab repo <--> Argo <--> Context generation AI <--> Machine Translation (using generated context) <--> Post-Human.

Thanks for your thoughts., @rasamhossain !

Feel free to engage with @mchrastek-ext to collaborate. cc @jan.bares.argos

Additional helpful background info:

Argo as a request management system (or any of its integrations - see linked epics here) is not involved in the localization process of product strings. As well as there is no MT involved, and no plans for MT (yet). Localizability would need to be addressed first, before we are able get the most value out of machine-translating product strings.

Product translations workflows are enabled via the Crowdin integration with GitLab: Unable to restore the Crowdin Integration (#49 - closed)

Product translation process is described here: https://docs.gitlab.com/ee/development/i18n/

The wider community is translating GitLab product UI. Vendor-supported continuous translation cycles for select languages are managed (thus far) here: https://gitlab.com/gitlab-com/localization/issue-tracker/-/issues/30+

This chart https://gitlab.com/gitlab-com/localization/localization-team/-/issues/25#note_1946708644 visualizes the process of how the wider community is still involved in DE, FR and JA, too, and what vendor's role is.

MRs that the Import and Integrate team is merging translations with are: https://gitlab.com/gitlab-org/gitlab/-/merge_requests?scope=all&state=all&author_username=gitlab-crowdin-bot

Hello @rasamhossain I had a quick look at the Crowdin AI Context Harvester and I believe that it's taking opportunity of scanning any available surrounding information around the source code. Would you have example of the merge request to test on how much data does it contain compared to the whole resource file and what is the difference in AI output? I started to do test on few Vue components that are available ion the open source repo. Is this a representative MR commit? gitlab-org/gitlab!110295 (dc209e72)

Hi there @jan.bares.argos, you've asked a very good question - are you looking for a specific Merge Request that contains all the source code changes and the relevant string resource changes (often times the source code and the related string resource content is in the same MR only when a new string resource content is added), or is your question is more related just the string contents/resource file(s) only?

@jan.bares.argos , @rasamhossain , @mchrastek-ext , here are some examples:

• MR with UI text changes and changes to surrounding code logic: gitlab-org/gitlab!148112 (diffs)
• MR with changes of a namespace only, that affects both haml and .pot: gitlab-org/gitlab!150724 (diffs)
• MR with only UI text changes: (I am not sure these exist. They usually come with other relevant changes to non-.pot files), like the !150724 one above)
• MR with UI text changes to .pot, haml, and also to other .md files in the docs folder that are affected by same UI change: gitlab-org/gitlab!159277 (diffs). + screenshots in one of this MR's related issues.
• Another MR that's as complex/loaded with context and docs .md changes as the one above: gitlab-org/gitlab!157585 (diffs)

(The examples of MRs above that affect docs .md may be helpful for when we work on How to update tech docs when product UI strings... (#190 - moved) as we localize our tech docs)

Also, this filter gives examples of MRs where feature code changes, UI text changes and docs changes are made in the same MR: https://gitlab.com/groups/gitlab-org/-/merge_requests?label_name%5B%5D=Technical+Writing&label_name%5B%5D=UI+text&label_name%5B%5D=documentation&page=2&scope=all&state=merged  [source: #190 (comment 2008010076)\\\]

Hope this is what you were looking for, @jan.bares.argos !

@jan.bares.argos , here is another latest MR with visual context: gitlab-org/gitlab!164920 (comment 2088590676) cc @mchrastek-ext @wojciech.froelich

@jan.bares.argos @wojciech.froelich @mchrastek-ext - here is another most recent MR with visual context in it: gitlab-org/gitlab!163237 (merged)

The string in question is:

msgid "BranchRules|Edit branch protections, approval rules, and status checks from a single page. %{docs_link_start} How to use branch rules?%{docs_link_end}"

@jan.bares.argos @wojciech.froelich @mchrastek-ext , here is another MR with visual context:

User mapping - FE - Add UI for sorting placehol... (gitlab-org/gitlab!164152 - merged).

The interesting thing is, this MR did not contain the actual string msgid "UserMapping|Source name" on the screenshots. Here is the screenshot I received, after reaching out for DM chat with the developer:

And here is the clarification from the developer on what is the meaning of the term "source" is in this context:

Question:

What does Source mean in this case? a system (code repo, issue tracking) from which something is imported to GitLab? 

Answer:

Source...means the originating user. This is within the context of imports.... User mapping is the process where when you transfer or import a group to a different GitLab instance, you need to make sure the contributions of Susan in the old instance are assigned correctly to Susan in the new instance. Susan in the old instance would be the 'source user'.

I have uploaded the screenshot to Crowdin and tagged the string on it:

https://crowdin.com/project/gitlab-ee/screenshots

https://crowdin.com/editor/gitlab-ee/16/en-de?view=comfortable&filter=basic&value=2#q=UserMapping%7CSource%20name

I also added context info by typing it in the comments section on the right:

We have not been uploading screenshots as context to Crowdin. We have not been adding context as comments in Crowdin either. This is a manual process I have tried only now, and it will not scale.

Would love to explore how we could leverage context harvester for that, if at all possible.

@opysaryuk I checked the mentioned MR and it seems that it actually did contain the string "UserMapping|Source name" in the app/assets/javascripts/members/placeholders/components/app.vue file on line 160. Could you please confirm?

@mchrastek-ext it does, it also contains the same string in the .pot file: gitlab-org/gitlab!164152 (diffs) . My note was rather, this the MR did not contain the actual string msgid "UserMapping|Source name" on the screenshots

@opysaryuk Ah, sorry, I misread your original comment. My mistake

marked this issue as related to #13 (closed)

assigned to @mchrastek-ext and @rasamhossain

made the issue confidential

assigned to @jan.bares.argos

mentioned in merge request gitlab-org/gitlab!164920 (merged)

made the issue visible to everyone

mentioned in issue #13 (closed)

mentioned in issue #252 (closed)

marked this issue as blocked by #252 (closed)

@mchrastek-ext , for any experiments and work on the Context Harvester that uses AI features, let's use GitLab's AI capabilities: https://docs.gitlab.com/ee/user/gitlab_duo/

mentioned in merge request gitlab-org/gitlab!166898 (closed)

mentioned in issue #259

added L10nProduct label

mentioned in issue #268 (closed)

added localization#81 as parent epic

unassigned @rasamhossain

unassigned @jan.bares.argos

After a discussion with @mchrastek-ext , I realize we are not sing the Context Harvester as it is originally intended to be used by Crowdin. Will let Martin add more notes on the proposed direction with the context-enhancement of the translation process, before closing this issue.

removed the relation with #252 (closed)

mentioned in issue #276 (closed)

Even though two weeks ago Crowdin has added support for more AI providers (including Google Vertext AI), I still believe it would be best to have a custom code for harvesting the context from the codebase.

This would bring the following benefits:

• Not include a library of which we would not use 90%.
• Have direct control over how files and other context gets passed to an LLM including potentially fetching context from other sources like MRs/commits etc.
• If we use the approach to extract the code occurrences of the strings that I suggested in this comment &81 (comment 2172176810) (enabling include_reference_comment in @po_format_options), we can get faster and more accurate search for code occurrences (and therefore more accurate context) than if we use Crowdin Context Harvester due to escaped characters, multiline strings etc.
• Stay within our current Python development bubble as the scripts that help with automating Crowdin are written in Python whereas the Crowdin Context Harvester is in JavaScript.

Sounds good, @mchrastek-ext

Adding @wortschi for any feedback

closed

marked this issue as related to #282

mentioned in issue #282