Detection and Recognition of Real-Time Hand Gestures Using YOLO and AlexNet Techniques

Abstract

At least three thousand five hundred million people in the world cannot hear or speak, they are called the deaf and dumb. Often this segment of society is partially isolated from the rest of community due to the difficulty of dealing, communicating and understanding. To address this problem, many solutions have been proposed that try bridging this gap between this segment and the rest of society and using the technical development of devices, especially computers and mobile phones. The science of image processing with artificial intelligence has been used to generate programs for converting natural speech into a sign language that enables the people with disabilities (the deaf and dumb) understand it. Initially, a set of sign dictionaries were made in the deaf and dumb language, including the Indian Sign Language (ISL), the American Sign Language (ASL), the European Dictionary and so on, and the main reason for this is to simplify the understanding of the sign language. These dictionaries depend mainly on the movement of hands, and one or both hands can be used to form special characters for this conversation. These dictionaries can be used after training this segment of people. Because of technological progress, this process can be automated using a computer and various programming languages in addition to secondary connections (cameras and microphones) and advanced algorithms for artificial intelligence. This goal can be reached by building a special program for communication between people with disabilities (deaf and dumb) and healthy people, or both directions. The research results show that the use of neural networks, especially convolutional neural networks, is very suitable in terms of accuracy, speed of performance and generality in processing the previously unused input data.

Keywords: the deaf and dumb, Indian sign language, American sign language, hand gesture, artificial intelligence, hand detection, convolutional neural networks.