Panjaya, a startup focusing on generative AI, is revolutionizing video dubbing by creating hyperrealistic translations. This AI-driven tool aims to recreate a person’s original voice speaking in a new language while also adjusting their physical movements to align with the new speech patterns, offering a more natural viewing experience.
The company, which had been operating in stealth mode for three years, is launching its first product, BodyTalk, along with a $9.5 million funding round. Panjaya was founded by Hilik Shani and Ariel Shalom, deep learning experts with extensive backgrounds in Israeli government projects. Guy Piekarz, who joined the company in 2022 as CEO, has a notable career in the tech industry, having sold his startup, Matcha, to Apple in 2013. His previous experience with Apple’s TV and streaming strategies is valuable for Panjaya’s growth.
BodyTalk integrates several cutting-edge technologies, starting with audio-based translation in 29 languages. The tool not only translates the spoken content but also modifies the video to sync with the new speech patterns, adjusting the speaker’s lips and other movements. Users upload their videos to the platform, which processes them automatically. Although the process is near real-time, Panjaya plans to further enhance the speed and offer an API for easier integration.
The company has developed its own AI models, such as its custom lip-sync engine, to improve the quality of the dubbing. The current focus is on B2B clients like TED and JFrog, but Panjaya plans to expand into industries like sports, education, marketing, and healthcare.
While the technology behind BodyTalk is similar to deepfake techniques, Piekarz emphasizes that Panjaya avoids the negative connotations associated with the term “deepfake.” Instead, he refers to their work as part of the “deep real” category, aiming for authenticity. By targeting the B2B market and carefully controlling access, Panjaya intends to prevent misuse and is working on tools to detect synthetic media.
The company faces competition from other AI-based video translation services, including big players like Vimeo and ElevenLabs. However, subtitles and captions have become a standard in video consumption, especially on platforms like TikTok, where captions are enabled by default. Despite this, there remains significant demand for dubbed content, especially in non-English-speaking regions. Panjaya believes that more natural-sounding, native-language dubbing could result in better engagement, as seen with TED, where translated talks have experienced a 115% increase in views and higher completion rates.
Panjaya’s technology represents a significant step forward in video dubbing, combining AI innovation with practical applications to meet the growing demand for high-quality translations.
