IJRIT International Journal of Research in Information Technology, Volume 3, Issue 2, February 2015, Pg. 120-126

International Journal of Research in Information Technology (IJRIT)

www.ijrit.com

ISSN 2001-5569

Artificial Intelligence Based Robot Control Using Face and Hand Gesture Recognition: A Review Piyush Anjankar 1, Shubhangi Borkar 2 1

2

M.E Scholar, Department of CSE, Nagpur Institute of Technology, RTMNU, Nagpur Nagpur, Maharashtra, India [email protected]

Asst. Professor, Department of CSE, Nagpur Institute of Technology, RTMNU, Nagpur Nagpur, Maharashtra, India [email protected]

Abstract In this Paper, we are presenting a review for the interaction to robot for control its operation with the help of Artificial intelligence techniques. We are gone through the many research papers and article for review, our many focus for the robot control mechanism with the help of hand gesture recognition and artificial intelligence technique. We focus on HCI with artificial intelligence technique where robot control authentication process is done with the help of face recognition and movement of robot is control with the help of hand gestures. While dealing with this issue we are came with some useful output, for both face recognition and hand gesture with artificial intelligence technique.

Keywords: Human Computer Interaction (HCI), Artificial Intelligence, face recognition, Hand gestures, Neural Network

1. Introduction The crucial aim of building hand gesture recognition system is to build a traditional communication between human and computer where the predictable gestures can be used for controlling a robot or conveying expressive information [2]. By what means to form the resulted hand gestures to be understood and well understood by the computer considered as the problem of gesture interaction. In recent years, the arena of computer visualization matured quickly and the efforts have been made to apply studied results in the practical scenarios. When realizing scholar’s findings, hardware budget becomes a crucial subject [1].Exploiting computers had all time plead the demand of interfacing. The methodologies through which human has been work together with computers has went a long way. The ride still remain continues and novel strategies of technologies and structures appear further every day and the study in this area has been emergent very fast in the previous few years [3]. Human-computer interaction (HCI) get up as a field from tangled origins in computer graphics, operating systems, human factors, ergonomics, industrial engineering, cognitive psychology, and the systems part of computer science. The goal for the exploration comprises, artificial intelligence which is human alike intelligence using Hardware or software. The artificial intelligence has application in numerous disciplines such as Neural Network, natural language processing, pattern matching fuzzy system etc. As well this approaches are used for gesture detection. Piyush Anjankar,

IJRIT- 120

IJRIT International Journal of Research in Information Technology, Volume 3, Issue 2, February 2015, Pg. 120-126

One of the utmost incredible capability of human visualization is face recognition. The study of face recognition comprise several method such as template matching and facial feature detection. Geometric feature established corresponding contains recognition of individual eyes, nose and mouth etc. In template matching, the image is characterized as a dimensional arrangement of intensity values and are equated by means of some appropriate matric like Euclidean distance, with single template representing the whole face [5]. Hand gesture recognition structure acknowledged excessive attention in the last few years since of its numerous applications and the capability to work together with device efficiently via human computer interface[2]. In hand gesture appreciation comprise , Best of the scholars classified gesticulation acknowledgment system into mostly three stages afterward obtaining the input image from nominated source- Removal or extraction Method, features valuation and removal, and classification or recognition. The paper organization is as follows: the following section explains Human computer interaction, artificial intelligence and face recognition system and hand gesture recognition and finally the summary. Section 2 provided a review of Human computer interaction. Section 3 contributes the small assessment of artificial intelligence and section 4 and 5 gives the review of face and hand recognition on systems. The concluding section i.e. Section 6 gives the summary of this reviews for our forthcoming progress in this area.

2. Human Computer interaction The utmost and well-known definition of “Human Computer Interaction” is- “Human-computer interaction is a discipline concerned with the design, evaluation and implementation of interactive computing systems for human use and with the study of major curiosities surrounding them” [4]. Human-computer interaction is the study of in what way individual design, implement, and usage interactive computer systems and how computers touch individuals, organizations, and humanity. This incorporates not only easiness of usage but also innovative interaction procedures for accompanying user tasks, providing improved access to data, and structure additional foremost forms of communication. HCI in the huge is an interdisciplinary region. it is emerging as a forte concern surrounded by numerous disciplines, each with different importance: computer science -application design and engineering of human interfaces, psychology -the application of philosophies of intellectual procedures and the realistic scrutiny of user behavior, sociology and anthropology communications among technology, work, and organization, and engineering design interactive products [6]. Human-computer interaction is anxious with the combined performance of responsibilities by individuals and machines, the configuration of communication among human and machine.

2.1 Objective of HCI The objectives of HCI are to produce operational and anodyne system, as well as real-world systems. In demand to produce computer system with good usability and operational, designer of HCI must try to [6]: •

Recognize the aspects that regulates in what way people uses technology.



Improve tools and technique to facilitate for building appropriate system.



Attain well-organized, actual and harmless interaction.



Place people first as concern.

Following are the different configurations and strategies upon which an interface is created in HCI [3]. Multimodal HCI Systems Each of the dissimilar autonomous solitary channels of input and output is called a modality. The term multimodal mentions to amalgamation of multiple modalities. Uni-modal HCI Systems Piyush Anjankar,

IJRIT- 121

IJRIT International Journal of Research in Information Technology, Volume 3, Issue 2, February 2015, Pg. 120-126

A system which are built on single modality is called Uni-modal. Depend on the environment of different modalities, they can be alienated into three categories: 1. Visual-Dependent 2.

Audio- Dependent

3. Sensor- Dependent The visual depended human computer interaction is perhaps the paramount, and well-known area in HCI exploration, some of the key research areas in this division are as follow: •

Facial Appearance or Expressions Analysis



Body Movement Tracking (Large-scale)



Gesticulation or Gesture Recognition



Gaze Detection (Eyes Movement Tracking)

The audio depended interface between a computer and a human is another remarkable zone of HCI systems. This zone pacts with statistics attained by different audio signals. Study areas in this segment can be divided to the following parts: •

Speech Recognition



Auditory Emotion Analysis



Human-Made Noise/Sign Detections (Gasp, Sigh, Laugh, Cry, etc.)



Musical Interaction

The sensor depended communication is a intermingling of diversity of areas with a extensive variety of applications. The unity of these dissimilar areas is that at least one physical sensor is used among user and machine to offer the communication. These sensors as named underneath can be particular primitive or very classy. •

Pen-Based Interaction



Mouse & Keyboard



Joysticks etc.

3. Artificial Intelligence Artificial intelligence (AI) is the human alike intelligence shown by machines or software. Major AI scholars and manuals defined this field as "the study and design of intellectual agents" where an intelligent agent is a system that perceives its surroundings and takes appropriate actions and that actions make the most of its chances of success. Artificial intelligence has been the subject of optimism and research, but has too agonized obstacles and, nowadays, has become an essential aspect of almost all the technology industry, providing the considerable stimulating for several of the most tough problems in computer science. Neural networks and artificial intelligence are frequently studied collectively meanwhile as the practice of neural networks is one of the way by which artificial intelligence programming can be succeeded. While this is not the only way or method, or may not the best method in all occasions, for making programs for artificial intelligence, but the practice of neural networks has turn into increasingly popular approach. Artificial neural networks are Piyush Anjankar,

IJRIT- 122

IJRIT International Journal of Research in Information Technology, Volume 3, Issue 2, February 2015, Pg. 120-126

basically based on the approach in which the human brain functions and takes actions depend on the surroundings, through the formation of networks that interrelate and communicate with each other to complete various allocated tasks. This type of network stereotypically lookalikes such as organic neural networks, nevertheless neural networks and artificial intelligence have not yet been used to make a program that matches human thought progressions. Artificial intelligence is basically the idea of a computer program or system that might equivalent or exceed the human brain in terms of processing supremacy and intellectual dimensions. There are numerous dissimilar methods in which this sort of artificial intelligence is being established, and neural networks are amongst the furthermost popular forms of artificial intelligence. Neural networks and artificial intelligence are work well in composed manner ever since both systems naturally use the human brain as the standard for processing capability. Artificial neural networks (ANN) were used mainly in the current years in the arenas of image processing (compression, recognition and encryption) and pattern recognition. Several studies in this area used various ANN architecture and prototypes for face detection and recognition to attain improved performance and efficiency gain of recognition.

4. Face Recognition Gesture recognition is an area in computer science and language technology with the objective of understanding human gestures via accurate algorithms. Gestures can initiate from any bodily gesture or state however usually initiate from the face or hand. A principal objective of gesture recognition exploration is to build a system which know how to recognize unambiguous human gestures and use them to carry factor for device control and work promptly. We emphasis on face detection, head position approximation and hand gesture recognition, various scholars’ works which are restricted to identifying straight, frontal faces, many studies only permit one degree of liberty, which is not appropriate for the estimation of head incline positions [1]. In latest years, face recognition has fascinated much responsiveness and its study has quickly extended by not only engineers but also neuroscientists, since it has many prospective applications in computer hallucination communication and programmed or automatic access control system. Particularly, face detection is an essential part of face recognition as the first stage of involuntary face recognition. Though, face detection is not straight forward mechanism for detection, for the reason that it has lots of variations of image presence, such as pose disparity (front, non-front), image orientation, enlightening condition and facial expression and appearance. [5]. A lot of pioneering methods have been recommended to resolve each disparity listed above. For example, the template-matching approach are used for face localization and detection by calculating the correlation of an input image to a standard face pattern. The feature invariant methods are used for feature detection of eyes, mouth, ears, nose, etc[7] [8].Face recognition is used for two principal works [9]:

4.1 Verification (one-to-one matching): When offered through a face image of an indefinite individual along with a right of identity, determining whether the indefinite individual is who he/she privileges to be as he/she claimed. 4.2

Identification (one-to-many matching): Given an image of an unidentified individual, decisive that person’s uniqueness by matching (possibly after encoding) that image with a databank or database of images of (possibly encoded) images of identified individuals. In the rotation invariant centered face detection assessment interconnected study we found that, for identifying the face in an image general technique to create template-based face detector rotation in variant centered Piyush Anjankar,

IJRIT- 123

IJRIT International Journal of Research in Information Technology, Volume 3, Issue 2, February 2015, Pg. 120-126

neural network-algorithm in gray-scale images is used. The system offered in Rowley et al used two different network system for face detection viz. Router network for manipulating the head tilt angle and detector network, which derotated the image given by router network and identify the face within the image. To eliminate any further irregularities in the detection of face the heuristic approach is used. This system is able to detect about 70% of two large test set. The main shortcoming of this system is the necessity of very large database for template matching, which comprise of both negative and positive image. The positive image are those images which comprise the face, and negative are the one who does not comprise any faces [10]. In the head angle approximation associated exploration, scholar’s presented a vigorous approach to evaluation of the 3D pose of human heads by means of a single image. Their process only makes use of the statistics about the skin and hair region of the heads. Chen et al gives the robustness and the consistency of facial feature tracking. This system uses two global evidence as area and center of skin and hair region. This system uses a perceptually unchanging color space to define the color information of images. It uses two global tables to describe the skin color likeness and the hair color likeness of all visible colors. These tables are named as SCDM (Skin Color Distribution Model) and HCDM (Hair Color Distribution Model). It first convert the color information of each pixel (R,G,B) to the chromaticity in perceptually unchanging color space (uf,vf) and the luminance (y): (R, G, B)→(y,uf,vf) The skin region and the hair region in an image are then removed by estimating the skin color likeness and the hair color likeness for each pixel with SCDM and HCDM. SCSM (p) =SCDM (uf (p)), vf (p)) HCSM (p) =HCDM(y (p), uf (p), vf (p)) Where SCSM and HCSM are the skin color likeness and the hair color likeness of pixel p. y (p), uf(p) and vf(p) are the luminance and the chromaticity of pixel p. The face detection is carried out by comparing every rectangle region of SCSM and HCSM with several prebuilt head shape model. After finding the skin and hair color likeness region, this system find the area, center and axis of skin/ hair region and then center for face and hair region. This system also calculate the depth in three dimension in X-axis, Yaxis and Z-axis [11]

5. Hand Gesture Recognition Hand gesture recognition system can be used for interacting between computer and human using hand gesture. The prime goal of gesture recognition study is to form a system which can recognize specific human gestures and use them to convey information for device like robots to control. The methodologies for hand gesture recognition can be generally divided into Data Glove Based and Vision Based approaches. The Data Glove based methods use sensor devices for digitizing hand and finger gestures into multi-parametric data. The additional sensors make it casual to gather hand configuration and movement [12].Conversely, the devices are to a certain extent expensive and bring much clumsy experience to the users. In contrast, the Vision Based methods needs only a camera, thus comprehending a normal interaction between humans and computers without the use of any additional device[13]. The earlier scholars explored the hand gestures with four processes -perceiving the hand in bimanual movements, excruciating of a meaningful gesture area from an image stream, removing the required features and recognizing the gesture. The user requires to wear long sleeve clothing, revealing only palms and hands in order to permit the hand gesture recognition to function appropriately and accurately.

Piyush Anjankar,

IJRIT- 124

IJRIT International Journal of Research in Information Technology, Volume 3, Issue 2, February 2015, Pg. 120-126

Fig 1: Hand Gesture recognition system steps [2]

4.1 Extraction Method and image pre-processing Segmentation method is the first method for recognizing hand gestures. It is the process of separating the input image (in this case hand gesture image) into areas alienated by boundaries. The segmentation process be subject to on the nature of gesture, if it is dynamic gesture then the hand gesture essential to be placed and tracked, if it is static or stationary gesture (posture) the input image have to be segmented only. The hand should be positioned firstly, usually a bounding box is used to identify the liable on the skin color and secondly, the hand have to be tracked. The common useful indication used for segmenting the hand is the skin color, ever since it is easy and invariant to scale, translation, and rotation changes [2].

4.2 Features Extraction Good segmentation method lead to faultless features removal process and the later play an significant role in a successful recognition process Features vector of the segmented image can be mined in different methods according to specific application. Numerous methods have been useful for signifying the features that can be extracted. Some approaches used the shape of the hand such as hand outline and shape while others make use of fingertips location, palm center, etc.[2].

4.3. Gestures Classification After exhibiting and exploration of the input hand image, gesture grouping method is used to recognize the gesture. Recognition method affected with the appropriate assortment of features constraints and appropriate classification algorithm. Neural network has remained far and extensively applied in the arena of extraction of the hand shape, and for hand gesture recognition. Additional spineless computing tool are effective in this field as well, such as Fuzzy C -Means clustering (FCM) , and Genetic Algorithms [2].The Wachs et al describes the hand gesture recognition associated research, recommended a methodology via a neighborhood-search algorithm for change the system parameters. They instructed the problem of simultaneous standardization of the parameters of the processing/fuzzy C-means (FCM) components of a hand gesture recognition system. This system is restricted if it is applied as a part of HCI systems because it is not accomplished for detecting the hand gesture positions in the image automatically. The user must limit the hand gestures in a convinced area [14].

4.4 Application areas of hand gesture system Hand gestures recognition system has been functional for different applications on various domains, including; sign language translation, virtual environments ,medical systems, smart surveillance, robot control, etc. Since the sign language is used for understanding and enlightenments of a definite subject during the conversation, it has acknowledged special attention in recent year. A numerous of systems have been recommended to recognize gestures using different kinds of sign languages For example recognized American Sign Language ASL using boundary histogram. Controlling the robot via gestures considered as one of the fascinating applications in this field for example the some proposed system that uses the numbering to count the five fingers for controlling a robot using hand pose signs. Also the condition where the human operator cannot work only because of extreme conditions, in such environment the robot will be most eminent device to be operated [2] Piyush Anjankar,

IJRIT- 125

IJRIT International Journal of Research in Information Technology, Volume 3, Issue 2, February 2015, Pg. 120-126

5. Summary and Future Work In this paper we proposed various aspects of human computer interaction, artificial intelligence, face and hand gesture recognition. While discussing this issue we are provide the some useful facts and gives the better approach of study this key factor together to build the comprehensive interactive system between human and computer. The future work includes to build a system to for interacting the human with robot, where the authentication of user to robot is done with the help of face recognition and the control movement of robot is done with the hand gestures. The application of this system building is may be anything that is describe in section 5. The robot can be built to act as per the instruction given by the authorized person to control its activity.

6. References [1] Yo-Jen Tu, Chung-Chieh Kao, Huei-Yung Lin,” Human Computer Interaction Using Face and Gesture Recognition” published by IEEE for signal and information processing Association Annual Summit and Conference(APSIPA),2013 AsiaPacific, pp11-8,2013. [2] Rafiqul Zaman Khan and Noor Adnan Ibraheem,” hand gesture recognition: a literature review”, International Journal of Artificial Intelligence & Applications (IJAIA), Vol.3, No.4, July 2012. [3] Fakhreddine Karray, Milad Alemzadeh, Jamil Abou Saleh and Mo Nours Arab, “Human-Computer Interaction: Overview on State of the Art”, international journal on smart sensing and intelligent system, ppa 137-158 Vol. 1, No. 1, March 2008. [4] Book Reference, Desney Tan and Anton Nijholt, Chapter No. 1 Brain-Computer Interfaces and Human Computer Interaction [5] R.brunelli,T poggio,” face recognition: Feature versus Template”,IEEE transaction on pattern analysis and machine intelligence, vol.15,No.10,October 1993. [6]

Human Computer Interaction. Dhiren Parmar, IIT KGP,

[7] Omaima N. A. AL-Allaf,” Review of face detection system based artificial neural network algorithm”, The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014. [8] Article Reference, Face detection, Inseong Kim, Joon Hyung Shim, and Jinkyu Yang. [9] Rabia Jafri ,Hamid R. Arabnia, “A Survey of Face Recognition Techniques”, Journal of Information Processing Systems, ppa 41-68 Vol.5, No.2, June 2009. [10] Henry A. Rowley, Shumeet Baluja and Takeo Kanade, “Rotation Invariant Detection”,IEEE Conference on Computer Vision and Pattern Recognition, pp. 38-44, 1998.

Neural

Network-Based Face

[11] Qian Chen, Haiyuan We, Takeshi Fukumoto and Masahiko Yachida, “3D Head Pose Estimation without Feature Tracking”, IEEE International Coference on Automatic Face and Gesture Recognition, pp. 88-93, 1998. [12] Yikai Fang , Kongqiao Wang , Jian Cheng and Hanqing Lu ,” a real-time hand gesture recognition method”, published by IEEE for ICME 2007. [13] Pragati Garg, Naveen Aggarwal and Sanjeev Sofat.” Vision Based Hand Gesture Recognition”, World Academy of Science, Engineering and Technology 49 2009. [14] Juan P. Wachs, Helman Stern,and Yael Edan,” Cluster Labeling and Parameter Estimation for the Automated Setup of a Hand-Gesture Recognition System”, IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 35, NO. 6, NOVEMBER 2005.

Piyush Anjankar,

IJRIT- 126

Artificial Intelligence Based Robot Control Using Face and ... - IJRIT

Human-computer interaction (HCI) get up as a field from tangled origins in computer graphics, ... is a discipline concerned with the design, evaluation and implementation of interactive computing systems for ... method, or may not the best method in all occasions, for making programs for artificial intelligence, but the practice ...

131KB Sizes 0 Downloads 282 Views

Recommend Documents

Artificial Intelligence Based Robot Control Using Face and ... - IJRIT
artificial intelligence technique where robot control authentication process is done ... intelligence and face recognition system and hand gesture recognition and ...

RFID Based Face Attendance System RFID Based Face ... - IJRIT
ability to uniquely identify each person based on their RFID tag type of ID card make .... Fortunately, Intel developed an open source library devoted to easing the.

RFID Based Face Attendance System RFID Based Face ... - IJRIT
This paper describes the development of a employee attendance system .... One can increase both the false alarm rate and positive hit rate by decreasing the.

Robot and control system
Dec 8, 1981 - illustrative of problems faced by the prior art in provid ... such conventions for the purpose of illustration, and despite the fact that the motor ...

Human-Level Artificial Intelligence: One for the ... - IJRIT
IJRIT International Journal of Research in Information Technology, Volume 2, Issue 9, September 2014, Pg. 373-379 ... the solution of specific problems. Others ...

Human-Level Artificial Intelligence: One for the ... - IJRIT
When access to digital computers became possible in the middle 1950s, ... in Information Technology, Volume 2, Issue 9, September 2014, Pg. 373-379.

Authorization of Face Recognition Technique Based On Eigen ... - IJRIT
IJRIT International Journal of Research in Information Technology, Volume 2, ..... computationally expensive but require a high degree of correlation between the ...

Enforcing Message Privacy Using Attribute Based ... - IJRIT
IJRIT International Journal of Research in Information Technology, Volume 2, Issue 3, .... j ∈ Ai, Ai chooses ri ∈ Zp and a random ki − 1 degree polynomial. 4.

Enforcing Message Privacy Using Attribute Based ... - IJRIT
When making decision on use of cloud computing, consumers must have a clear ... identifier (GID) to bind a user's access ability at all authorities by using an ...

Prediction of Software Defects Based on Artificial Neural ... - IJRIT
IJRIT International Journal of Research in Information Technology, Volume 2, Issue .... Software quality is the degree to which software possesses attributes like ...

Authorization of Face Recognition Technique Based On Eigen ... - IJRIT
IJRIT International Journal of Research in Information Technology, Volume 2, ..... computationally expensive but require a high degree of correlation between the ...

using hard and soft artificial intelligence algorithms to ...
Abstract: We describe the development of a Go playing computer program that combines the use of ... and it is shown that the use of hard AI enhances the performance of the soft AI system. ...... shows the most recent move, the blue circles.

Wall Follower Robot Using Fuzzy Logic: A Review - IJRIT
and to enter open areas of the rooms [1,2]. In attempts .... Computer simulations are carried out to compare between a Knowledge Based Fuzzy Logic Controller.

Prediction of Software Defects Based on Artificial Neural ... - IJRIT
studied the neural network based software defect prediction model. ... Neural Networks models have significant advantage over analytical models because they ...

Wall Follower Robot Using Fuzzy Logic: A Review - IJRIT
system that enables a mobile robot in moving through a corridor or following a .... The gain scheduling controller will be used before the FLC to control the error signal ... 2) computing the path winding number, 3) learning a combinatorial map,.

ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING.pdf ...
ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING.pdf. ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING.pdf. Open. Extract. Open with. Sign In.

Mobile Robot Indoor Localization Using Artificial ...
to validate our approach several ANN topologies have been evaluated in experimental ... to accomplish several tasks in autonomous mobile robotic area. Also, knowledge about ... the wireless network are Received Signal Strength Indication. (RSSI) and

Artificial Intelligence - GitHub
Dec 21, 2011 - based on the average of your six best homework assignments (30%), a midterm examination (30%) ... confer a Stanford degree or a certificate.

A Survey on Artificial Intelligence-Based Modeling ... - IEEE Xplore
Jun 18, 2015 - using experimental data, thermomechanical analysis, statistical or artificial intelligence (AI) models. Moreover, increasing demands for more ...

Using Artificial Intelligence in E-Tourism
Offering tourist services on the Internet has become a great business over the .... Currently LBTS applications are being used by mobile phones, iPods, PDAs ...