Sony Patents Revolutionary In-Game Sign Language Tool

Sony's groundbreaking patent aims to revolutionize accessibility for deaf gamers. The technology proposes a real-time in-game sign language translator, bridging communication gaps between players using different sign languages.

Sony Patents ASL to JSL Translator for Video Games

Leveraging VR and Cloud Gaming for Seamless Translation

This innovative patent, titled "TRANSLATION OF SIGN LANGUAGE IN A VIRTUAL ENVIRONMENT," details a system enabling real-time translation between sign languages, such as American Sign Language (ASL) and Japanese Sign Language (JSL). Sony envisions a system that facilitates in-game communication for deaf players by instantly translating sign language gestures. On-screen avatars or indicators would visually represent the translated sign language in real time. The process involves a three-step translation: sign language to text, text translation between languages, and finally, text to the target sign language.

As Sony explains in the patent: "Implementations of the present disclosure relate to methods and systems for capturing sign language of one user (e.g., Japanese), and translating the sign language to another user (e.g., English). Because sign languages vary depending on geographical origins, sign language is not universal. This provides a need for appropriately capturing the sign language of one user, understanding the native language, and generating new sign language as output for another user in their native sign language."

Sony suggests utilizing VR headsets or head-mounted displays (HMDs) for implementation. These HMDs would connect to a user's device (PC, game console, etc.), providing immersive visuals within the virtual game environment. Sony details: "In some implementations, the HMD connects via a wired or wireless connection to a user device, such as a personal computer, game console, or other computing device. In some implementations, the user device renders graphics for display through the HMD that provide immersive viewing of the virtual environment for the user."

Furthermore, Sony proposes inter-device communication via a game server, enabling seamless interaction between players in a shared virtual environment. The patent highlights the potential for cloud gaming integration: "In some implementations, the game server executes a shared session of a video game, maintaining the canonical state of the video game and its virtual environment, and to which the user devices are synchronized regarding the state of the virtual environment." This cloud-based approach allows for efficient video rendering and streaming between user devices, enhancing the overall gaming experience.