Full Program »
Helping Deaf or Hard of Hearing (DHH) Students Participate in Group Work Setting
Certain colleges or universities provide only expensive or impractical assistive technology solutions to address specific needs of students with disabilities. This limitation demonstrates the lack of cost-effective and practical options to assist deaf or hard of hearing (DHH) students in participating in group discussions. We propose a solution to remedy this problem in order to allow a DHH student to participate in a group discussion with a near-natural conversational flow.
Approximately 20,000 DHH students currently attend college or university, where quite often they are required to work in a group of their peers in order to complete an assignment. However, this can be problematic for such students for two primary reasons. First, the DHH student can struggle to understand what is being said in group communication. Second, they may struggle to, or be uncomfortable, responding verbally, depending on the severity of their impairment. Thus, without a natural communication solution, a DHH student may be severely limited in effectively participating with their peers in a group work setting. Some solutions currently in use include either having a live interpreter or a remote transcription service. For the latter, Skype is used to transmit audio to a remote transcriber, who then sends the transcript of the conversation back, after a substantial time delay. This solution costs an estimated $42,000 per student per academic year. The use of a live interpreter is not cost-effective either. Neither solution is apt for a DHH student in a group work setting.
Our project for senior design focuses on developing and testing a cost-effective and efficient solution that a DHH student can utilize to realize near-natural communication with peers in a group discussion. The technology used in our solution includes Bluetooth enabled headsets (ear pieces and microphone), a mobile phone app, and a desktop software (using a laptop computer). The overall scheme of our solution begins with all students participating within a group choosing a common group name, which is used as a key to allow the different devices to recognize signals sent among themselves. The non-DHH members speak (natural conversation style) into their headset’s microphone which would send audio to their respective mobile devices, where the developed (by our team) mobile app transcribes audio to text using Google’s Voice-to-Text software. Subsequently, the transcribed text (from each of the non-DHH participants) is sent to the DHH student’s laptop computer, where the developed (by our team) software displays the text’s using multiple panes, i.e., one pane for each non-DHH student. The DHH student has the ability to type a response to either the entire group or to an individual student. This typed response, would then be synthesized into speech using a C# speech synthesis library. Finally, the speech is transmitted to the non-DHH students either via the DHH student’s computer’s speakers or via Bluetooth to the non-DHH students’ headsets. Our solution will be a deliverable prototype to our university’s Disabilities Office for effectiveness live-testing by DHH students.