Odysseus

This project will be part of a larger team-effort, whose aim is to research and develop a computerbased Sign language translation system which can recognise Irish Sign Language (ISL) and translate it to English text. Our system will be developed and implemented on a standard personal computer connected to a colour video camera. Hand, face and body gestures will be recognised using image processing techniques based on Principal Component Analysis (PCA) and multi-scale hierarchical search. The variability in gestures will be modelled using Hidden Markov Models. The system will have a vocabulary of over 1000 gestures and will incorporate an ISL grammar and syntax. The system will be trained and tested by native ISL signers.

In order to train our system we need a good database containing examples of gestures from ISL. We propose to employ an intern student to help create this database. We hope that this intern student will be a native ISL signer, in order that his/her signs will be authenic and fluent. The database will consist of both video sequences and dataglove data and these will be manually segmented and labelled by the experienced signer. If the signer performs the same gesture many times, we can build up a distribution of hand-shapes within that gesture. If the signer performs a number of sentences in which the same gesture occurs preceded and followed by other gestures we can build a model of how one gesture modifies another. We can also use datagloves to get a full 3D description of how the hand changes in between gestures and this will allow us to construct a physiological model.

Facial expressions are also an important part of ISL and we will build a model of which facial expressions occur and how they modify the meaning of the gesture.

Relevance of Project to the Host Laboratories:

Our group already contains a Marie Curie Fellow, and a post-doctoral researcher funded by an SFI Research Frontiers grant, an IRCSET funded postgraduate and two School-funded postgraduates. All are working on various aspects of Sign Language recognition. The work in this project will provide accurate and authentic data about ISL, which is necessary to train and test our system.

Supervisors:

Dr. Alistair Sutherland (Computing, DCU)

Keywords:

Sign Language Recognition, Human-computer Interaction, Machine Translation

Links:

http://www.cse.unsw.edu.au/~waleed/gsl-rec/