Graduate School of Informatics /Game Technologies 

In partial fulfillment of the requirements for the degree of Master of Science Umut Demirel will defend his thesis.

Title: CREATING A GENERIC HAND AND FINGER GESTURE RECOGNIZER BY USING FOREARM MUSCLE ACTIVITY SIGNALS

Date: 01th August 2017

Time: 13:30 PM

Place: A-212

Thesis Abstract : Hand and finger gestures are the most natural way of communication without speaking. Therefore using hand and finger gestures as inputs is widely used in human-computer interaction applications. There are variety of applications using gestures as inputs such as: sign language recognition, robot controlling, medical device controlling and video game controlling. This thesis deals with creating a generic hand and finger gesture recognizer by processing muscle activity signals. For data collection, MYO from Thalmic Labs, a portable armband having 8 channel EMG sensors and IMU, is used. Normally, EMG sensors should directly be placed on muscles for each user. However, because MYO is a generic device for anyone to use, EMG sensor placements will differ from user to user. As a result, the recognizer will be calibration based, meaning that it is person and session dependent. Gestures are chosen from the set of expressive gestures used by classical orchestra conductors.13 different hand and finger gestures and 1 rest gesture is performed 5 times, each for 3 seconds by 20 subjects in separate sessions. While preprocessing the collected data, not only the time domain is used by applying filters to raw data, but also the frequency domain is used by representing each gesture with circular harmonic coefficients. Neural Network is trained, validated and tested to recognize 14 different gesture patterns. Cross validation method is performed with 5 data sets for each test subjects. 3 sets are used for training, 1 set is used for validation and 1 set is used for testing. 20 combinations of data sets are fed to 20 different neural network for the cross validation. The proposed generic gesture recognizer system is tested with one of the best Turkish classical orchestra conductor, Orhun Orhon. Before the performance, his forearm EMG data is collected while he performs his own gestures. During the performance, data is continued to be collected and real-time accuracy is tested.