In a July 30, 2024 research paper, Otso Haavisto and Robin Welsch from Aalto University presented a web application designed to simplify the process of adapting questionnaires for different languages and cultures.
This tool aims to assist researchers conducting cross-cultural studies, enhancing the quality and efficiency of questionnaire adaptation, while promoting equitable research practices.
Haavisto and Welsch highlighted that translating questionnaires is often costly and “resource-intensive,” requiring multiple independent translators and extensive validation processes. According to the authors, this complexity has led to inequalities in research, particularly in non-English-speaking and low-income regions where access to quality questionnaires is limited.
In questionnaire translation, maintaining semantic similarity is crucial to ensure that the translated version retains the same meaning as the original. The authors noted that “semantic similarity is more important than word-by-word match.” According to the authors, cultural nuances and colloquial expressions can further complicate this process, making it difficult to achieve accurate translations.
To address these challenges, they developed a web application that allows users to translate questionnaires, edit translations, backtranslate to the source language for comparisons against the original, and receive evaluations of translation quality generated by a large language model (LLM).
The tool integrates Translation Tools for initial translations and AI for evaluating and suggesting improvements. The decision to use translation tools was based on their “reliable output and promising results in translating scientific text,” which the authors said was essential for the accuracy of research questionnaires.
Haavisto and Welsch tested the tool’s effectiveness through two online studies: one involving 10 participants testing the English-German language pair and another involving 20 participants testing the English-Portuguese language pair. Both studies showed “promising results regarding LLM adoption in the questionnaire translation process,” according to the authors.
The study’s findings indicated that when supplemented by AI-generated quality scores, machine translation leads to translation quality and semantic similarity comparable to traditional translation. Participants also found the AI-generated suggestions “moderately helpful” and accurate in representing translation quality.
Haavisto and Welsch also noted that LLM-generated translation quality evaluations can assist researchers in identifying and addressing context-specific issues in their translations, highlighting that “this is the first step towards more equitable questionnaire-based research, powered by AI.”
