- A new Sony patent proposes that players control games solely through lip movements.
- Users can chat with others using silent lip movements. The system automatically produces speech.
- The patent also discusses a real-time translation feature that translates speech into other players’ languages.
PlayStation and Xbox have seen great innovations over the last decade, but the voice chat feature has remained practically unchanged. A new Sony patent filing proposes a next-gen audio system that enables real-time game controls via silent lip movements.
We have found a new Sony patent that describes a next-gen chat-and-voice control system. It uses advanced AI and sensors to let players control single and multiplayer games with lip movements, without producing any sounds.
So, a user can issue voice commands to control gameplay and other mechanics, such as pausing and opening the menu. It is pitched as a great accessibility feature for gamers with stutters or tremors. But there’s much more to this patent.
A form of ‘silent’ voice control may be implemented based on the determined intended speech input of the first user. This provides a new form of input for users, and can further improve accessibility of the game by allowing speech impaired users to control the game without providing manual inputs.
Why it matters: Sony could revolutionize gaming with this next-gen voice system, allowing players to control games and chat with others solely through lip movements. Those who stutter or are unable to speak loudly for any reason can greatly benefit.
The patent “AUDIO PROCESSING METHOD AND SYSTEM” will also allow players to chat with others using only lip movements in multiplayer games. The mouthing would be interpreted using sensors (camera, depth, infrared, electromyography, etc.) and machine learning to capture lip, tongue, and facial movements.
The system would generate natural, context-aware speech that can match the user’s real voice or a preferred alternative. It may create urgent whispers if the gameplay is stealth-based while health is low, for example.
Sony also discusses adding a real-time translation feature that converts your speech into the language of other players.
In addition, users may find communication over voice chat difficult due to speech issues, cognitive issues, not being able to speak fluently in the language being used to communicate, or the like.
Sony says speaking while gaming can be difficult. Making a lip-sync voice system can be a useful accessibility and convenience feature. So, we can expect the publisher to bring the discussed features into its upcoming single and multiplayer titles.
Sony has published a motley of unique patents in the past, such as one discussing a method to simplify console game development without using devkits and another about the PS6 featuring an innovative dustproof design.
Do you think the Sony patent will open up PlayStation to a whole new audience of disabled users? Let us know your thoughts in the comments below, or join the discussion on the Tech4Gamers forum.
Thank you! Please share your positive feedback. 🔋
How could we improve this post? Please Help us. 😔
Shameer Sarfaraz has previously worked for eXputer as a Senior News Writer for several years. Now with Tech4Gamers, he loves to devoutly keep up with the latest gaming and entertainment industries. He has a Bachelor’s Degree in Computer Science and years of experience reporting on games. Besides his passion for breaking news stories, Shahmeer loves spending his leisure time farming away in Stardew Valley. VGC, IGN, GameSpot, Game Rant, TheGamer, GamingBolt, The Verge, NME, Metro, Dot Esports, GameByte, Kotaku Australia, PC Gamer, and more have cited his articles.
Join Our Community
