Odysseus

This project will be part of a larger team-effort, whose aim is to research and develop a computerbased Sign language translation system which can recognise Irish Sign Language (ISL) and translate it to English text. Our system will be developed and implemented on a standard personal computer connected to a colour video camera. Hand, face and body gestures will be recognised using image processing techniques. The system will have a vocabulary of over 1000 gestures and will incorporate an ISL grammar and syntax. The system will be trained and tested by native ISL signers.

A vital part of this system will be a module for recognising hand shapes. The hand can be segmented from the image using skin colour, but the human hand is a highly deformable threedimensional object and can present a vast range of different shapes, when projected into a twodimensional image. The problem which this project attempts to solve is to search through the vast space of possible shapes to find the closest match to the current image.

We intend to compare multi-scale techniques for shape-matching. Work has already been done using Fourier Descriptors and Moments. We intend to extend this by convolving the data using Gaussian kernels of varying widths and applying an eigenvalue decomposition. This should express the shape data as a tree where successive levels represent different resolutions. This should speed up the matching process and make the algorithm more robust.

We will train and test the system on hand images derived from videos of native ISL signers but the technique can be applied to shape data from many other sources. The technique could be useful for other projects within the CDVP such as object recognition for video retrieval.

Relevance of Project to the Host Laboratories:

Our group already contains a Marie Curie Fellow, and a post-doctoral researcher funded by an SFI Research Frontiers grant, an IRCSET funded postgraduate and two School-funded postgraduates. All are working on various aspects of Sign Language recognition. The work in this project will form an important component in that team-effort by providing accurate shape descriptions to other parts of the system, such as the motion tracking and linguistic and grammatical modules. The work can also be extended to other problems such as face recognition, and human body tracking which are also vital parts of the system - and which also occur in other projects within the CDVP.

Supervisors:

Dr. Alistair Sutherland (Computing, DCU).

Keywords:

Shape Recognition, Multi-scale Analysis, Sign Language Recognition, Human-computer Interaction

Links:

http://www.cse.unsw.edu.au/~waleed/gsl-rec/