South African startup Vambo AI has released MORENA, an open weight 1.5-billion-parameter language model trained from scratch for 12 African languages, as well as English and French.
Founded by Chido Dzinotyiwei and Isheanesu Misi in April 2023, Vambo AI builds AI that “understands the languages of people in the rest of the world”. Individual users can write, search, translate and transcribe, while businesses and innovators can use the startup’s API and developer tools to build solutions for people across multiple geographies, and localise solutions that “have solved pertinent problems in other regions”. The platform already understood 11 languages, including Arabic, KiSwahili, IsiZulu and French, but it has now released MORENA, which Misi claims is the largest open African AI model trained from scratch.
MORENA understands 12 African languages – ChiShona, Kiswahili, Hausa, Yorùbá, Igbo, isiZulu, isiXhosa, Kinyarwanda, Setswana, Afrikaans, isiNdebele and Nigerian Pidgin – as well as English and French. Misi said it beats a Google model eight times its size on African text modelling. “It tokenises African languages more efficiently so it’s both cheaper to run in African contexts and cheaper to adapt, which means startups in Nigeria, or researchers in Kenya, will spend less to make their own models and use them in real world settings, with fine-tuning being our recommended way to use this,” he said.
“It’s over three times the size of the second largest African language model, which gives other researchers a a new point of reference. It also proved that other African initiatives from scratch do not necessarily need billion dollar budgets.”