Meta's New AI: Bridging Language Barriers

In 2023, Meta's AI researchers faced a challenge: understanding what non-English speakers in the US really needed from AI translators. After speaking with Spanish and Mandarin speakers, they found that people wanted a Star Trek-like universal translator that could preserve voice, tone, and emotions while translating speech in real-time across multiple languages. So, Meta put together a team to make this happen.What did the team come up with? A cutting-edge system called Seamless, which can translate speech among 36 different languages. This system is cleverly designed to overcome language data issues. Traditionally, AI translators focus on text because there's a vast amount of it available. Institutions like the United Nations provide enormous databases of formal documents, which has led to AI translators using very serious language, even for comedy. Plus, there's a lack of audio data for training. So, Meta included less formal sources like books and Wikipedia to tackle the formality problem. However, the scarcity of audio data remains, especially for low-resource languages like Icelandic or Zulu, making the challenge even bigger.