At the intersection of technology and linguistic diversity, recent advances highlight the importance of incorporating low-resourced languages such as Yoruba, spoken by around 47 million people across Nigeria, Benin, and Togo. A group of linguists has embarked on a groundbreaking project named YORULECT, which aims to develop comprehensive speech and text resources for four Yoruba regional dialects.
Presented at the AfricaNLP workshop on the sidelines of the prestigious International Conference on Learning Representation (ICLR) held in Vienna, 2024, YORULECT promises to fill the gap left by the predominance of standard dialects, which have historically received more focus from researchers.
Aremu Anuoluwapo, the computational linguist leading this initiative and currently pursuing his masters degree at the University of Trento, Italy, spoke exclusively with Global Voices about the project and its significance.
"The challenge lies not only in standardizing the language but also addressing the nuances found within its diverse dialects," Aremu explained. The dialects they have focused on include Ìjẹ̀bú, Ifè, Ilaje, and Standard Yoruba. Each dialect boasts its distinct features and challenges for natural language processing (NLP) systems.
Prior to developing YORULECT, Anuoluwapo's interest was sparked by discussions with Oreva Ahia, his colleague and PhD student at the University of Washington. Together, they contemplated the need for adequate representation of Yoruba dialects, having recognized the knowledge gaps during Aremu's studies on dialectology.
"We wanted to help those communities who still communicate through these dialects by making sure they are also represented in todays technological frameworks," he noted.
To construct quality datasets, Anuoluwapo and his team approached the issue by first gathering speech data from native speakers before recruiting them to transcribe the findings. This method ensured the authenticity and accuracy of the dialects, overcoming challenges posed by limited written records.
When asked about the challenges they faced, Anuoluwapo stated, "Training the models was particularly difficult. While some dialects aligned closely to Standard Yoruba, others faced hurdles due to syntactic differences and linguistic distinctiveness." For example, the Ilaje dialect exhibited characters and sentence arrangements not found within the standard version.
The long-term vision for YORULECT is ambitious. Anuoluwapo aims to redefine the awareness around low-resourced languages and their dialects within the NLP community. "People naturally focus on the standard dialects, but why cant we build tools for those who speak other dialects? They too deserve representation," he asserted.
Meanwhile, across the globe, another domain of linguistic learning is undergoing transformation through the use of AI tools. Corin Cesaric, Flex Editor at CNET, recently explored the potential of chatbots to assist language learners, focusing her efforts on Croatian.
Cesaric highlighted the challenges posed by learning Croatian, classified as a category four language by the State Department's Foreign Service Institute, requiring near 1,100 hours of study for proficiency. She began her linguistic quest by utilizing generative AI chatbots like ChatGPT and Microsoft's Copilot.
"The allure of Croatia, my familys roots, and its stunning beaches motivated me to learn its language," Cesaric shared. Despite using AI tools, she discovered their limitations, noting the importance of accuracy when it came to translations.
A language expert from Southern Methodist University remarked on the reliability of chatbots, acknowledging they produce largely accurate translations but can nonetheless make pronunciation errors. This remains key to achieving fluency, highlighting the tool's role as supplementary rather than primary instruction.
Cesaric detailed how using chatbots for interactive scenarios permitted her to practice conversational skills. She also actively sought real-time interactions with native speakers to bolster her learning.
"I certainly appreciated the benefits of using AI as an adjunct to lessons, especially when I could double-check grammar or structure with my tutor," she explained. Yet, she observed it was still necessary to learn traditionally alongside AI tools for comprehensive knowledge retention.
Variations of successful AI utilization have been showcased through other avenues, such as Tufa Labs, introducing the innovative framework known as LADDER (Learning through Autonomous Difficulty-Driven Example Recursion). This advanced mechanism enables Large Language Models (LLMs) to self-improve by facetiously generating simpler forms of complex problems.
The methodology drastically enhances LLM abilities, aiding models like Llama 3.2 to transition from near-zero proficiency to remarkable rates such as 82%. Ongoing research indicates LADDER trains models without necessarily relying on human input, lightening resource burdens.
Key findings suggest LADDER retains strong applicability across various disciplines, including but not limited to competitive programming and theorem proving.
The duality of these initiatives—advancing Yoruba dialect resources through YORULECT and exploring AI for language learning through chatbots—demonstrates the pressing necessity for inclusivity and support for diverse languages. Through continued efforts, both local and global communities stand to benefit from the tools developed to bridge gaps across language barriers.
Research and innovation, such as these, stress the centrality of inclusive technology as societies step forward, ensuring no voice is lost amid the rapid progression of language and communication.