- A new Sony patent discusses a lip-sync system for game NPCs to match various translations easily with mouth movements.
- The evaluation and fixing system can detect a similarity score for developers to ensure the character animations match all languages.
- The system can also fix the duration of translated sounds to align them with the original dialogue’s timing, resolving sync issues.
Gaming studios are competing to secure the wildest patent ideas after huge innovations in AI and technology over the last decade. Now, a new Sony patent may revolutionize game development as we know it.
We have discovered a new Sony patent that will shorten the game development time by automating a highly tedious process for developers. It discusses a next-gen lip-sync system for game NPCs to synchronize translations easily with mouth movements.
The patent discusses a system that can evaluate how well NPCs’ mouth movements hold up across all the available languages and pass a similarity score to developers. It also includes automated solutions to fix synchronization problems.
Based on the shape of the character’s mouth corresponding to each translated phoneme included in the translated phoneme sequence, a similarity [Score] indicating the similarity of the character’s mouth movements may be generated.
Why it matters: Sony’s new patent could dramatically speed up game development while making NPC animations look natural in any language.
The patent, ‘TRANSLATION LANGUAGE EVALUATION DEVICE’ mentions that the system can adjust the duration of translated speech and the movement of NPCs’ mouths to better align with the original dialogue’s timing, which can automatically fix the syncing issues.
Sony argues that when NPCs’ mouths don’t match the translated dialogue, it turns off gamers by hurting immersion. Manually fixing these redundant issues can take a long time in game development.
[When] the sound of the translated text does not correspond to the movement of the character’s mouth, [Then] the player of the game or the viewer of the video work may feel uncomfortable.
Overall, if Sony ever implements the system outside the patent, it might significantly shorten game development time.
Sony has secured many innovative ideas over the years, including one to create solar-powered controllers that charge in the presence of light and a unique custom recap system that creates highlight reels of missed gameplay events if the player stops paying attention.
Do you think Sony will utilize this lip sync evaluation and fixing system to shorten the time it takes to synchronize NPCs’ mouth movements? Let us know your thoughts in the comments below, or join the discussion on the Tech4Gamers forum.
Thank you! Please share your positive feedback. 🔋
How could we improve this post? Please Help us. 😔
Shameer Sarfaraz has previously worked for eXputer as a Senior News Writer for several years. Now with Tech4Gamers, he loves to devoutly keep up with the latest gaming and entertainment industries. He has a Bachelor’s Degree in Computer Science and years of experience reporting on games. Besides his passion for breaking news stories, Shahmeer loves spending his leisure time farming away in Stardew Valley. VGC, IGN, GameSpot, Game Rant, TheGamer, GamingBolt, The Verge, NME, Metro, Dot Esports, GameByte, Kotaku Australia, PC Gamer, and more have cited his articles.
Join Our Community
