The context of this abstract is Moodle translation quality assurance (QA). There are over 100 language packs available for Moodle, although the number of strings translated varies and the quality of translations is unknown, since translations are crowdsourced so that anyone in the Moodle community may contribute. Specifically, the Norwegian Directorate for Education and Training saw a need for improved translations of the Norwegian language pack for its Moodle site digilaer.no. Many strings are not of satisfactory quality, from simple typos to sentences that do not make sense. We have therefore developed a proof of concept (POC) for an AI-assisted translation QA tool.
We would like to present our work with this tool and its potential uses. We have conducted a test where we connected with the API of ChatGPT 3.5, processed approximately 42 000 strings, and received this output for each string:
Passed: Yes or no? I.e. is this a valid translation?
Quality rating on a scale 1-100.
A suggested alternative translation.
Ca. 450 strings had Passed=No and Quality<=30. These strings, with suggested translations, would be the natural starting point for manual review as a resource when revising the language pack.
We may run further tests with new (versions of) AI tools before MoodleMoot, and are also looking into ways to automate and improve the POC that we can present. We will also outline alternatives as regards how it can become a product useable for the community. It could for example be a contributed plugin that any admin can install and use on their site for QA locally, or even be implemented into AMOS on lang.moodle.org for AI-assisted QA of language packs there. As Moodle develops and new plugins are being created, the need for good translations will continue to be great.