The Deaf Translator Student Project at the Sikorsky Challenge Innovation Project Festival was highly praised by the international jury and placed first in the Best Project Idea category.
The editorial office of "KP" addressed to the senior lecturer L. A. Lyushenko, who is responsible for student startup projects on FPM, with the question: how did the idea of the project come about?
"In Ukraine, more and more attention is being paid to integrating people with special needs into active life," said Lesya Anatoliivna. - Modern society is trying to provide people with disabilities with equal opportunities to communicate, access and use technology, automation and "smart" things. One of the important tasks is to provide them with opportunities for communication, including through special techniques and software tools. A team of three students of the Faculty of Applied Mathematics: Oleksandr Peni, Mykhaylo Ivashchenko, Denys Okhrymchuk is working on a computer program that allows recognizing sign language on video from the camera of the device. Students are studying in the first year of master's training at the science department. "
Therefore, the Deaf Translator project team developed a prototype software platform for automated sign language recognition using computer vision and artificial neural networks, which provides its detection and translation functions, as well as native natural language translation and its transformation in a sign language. Implementing the platform on mobile devices, personal computers, extensions to other communications and devices such as smart glasses or automated terminals will allow people with speech and/or hearing impairments to communicate easily and conveniently with those who do not know sign language, and vice versa, others will be able to understand and communicate with people who do not hear and even learn sign language.
The Deaf Translator project has:
- simple stand-alone means of two-way communication;
- possibility to work with different natural and sign languages;
- modular structure to provide a wide range of technology applications;
- a common way of communicating with both parties.
The developers told that the proposed technology can be used in mobile applications to help people with hearing impaired communication in everyday situations, and will be useful for people who have to communicate with those who do not hear: salespeople, doctors, employees, etc. It is also capable of enhancing the capabilities of existing means of communication such as Skype or Viber, smart interactive embedded systems.
The project is currently undergoing research and development. The plans for the near future include the creation of a scalable and stable working prototype for the Ukrainian tactile sign language to identify optimal technologies and effective technical solutions.
We wish the founders a successful implementation of their startup. After all, Igor Sikorsky Kyiv Polytechnic Institute actively helps to create and implement student designs.