Loading...

See-Through Captions: Real-Time Captioning on Transparent Display for Deaf & Hard-of-Hearing People

arXiv Preprint 2021

Real-time captioning is a useful technique for deaf and hard-of-hearing (DHH) people to talk to hearing people. With the improvement in device performance and the accuracy of automatic speech recognition (ASR), real-time captioning is becoming an important tool for helping DHH people in their daily lives. To realize higher-quality communication and overcome the limitations of mobile and augmented-reality devices, real-time captioning that can be used comfortably while maintaining nonverbal communication and preventing incorrect recognition is required. Therefore, we propose a real-time captioning system that uses a transparent display. In this system, the captions are presented on both sides of the display to address the problem of incorrect ASR, and the highly transparent display makes it possible to see both the body language and the captions.

Credits

Author
Kenta Yamamoto
University of Tsukuba
Ippei Suzuki
University of Tsukuba
Akihisa Shitara
University of Tsukuba
Yoichi Ochiai
University of Tsukuba
Three authors contributed equally to this research.
Director
Cinematographer
Editor
Ippei Suzuki
University of Tsukuba
System Operation
Camera Assistant
Kenta Yamamoto
University of Tsukuba
Cast
Maya Torii
University of Tsukuba
Akihisa Shitara
University of Tsukuba
Equipment
Transparent Display
Japan Display Inc.

Related Materials