Using Multiple Sensors to determine Posture

Hosting University

Dublin City University

Overview

Using many of the resources within the Adaptive Information Cluster it is now possible to use a range of off-the-shelf and specialist sensors to monitor and record a person's normal daily activity for subsequent analysis. For example, within our group we have used the following:

* BodyMedia device: worn on the armband to record galvanic skin response, heat flux and accelerometer values in 3 axes, with data sampled and stored on-board for subsequent wireless upload to a PC. This is a specialist device which can use its sensed data to roughly categorise a person's activities based on movement, GSR, etc..
* SenseCam: is a personal camera worn around the neck and developed by Microsoft Research in Cambridge, which records up to 3,000 images per day of a person's daily life. We have developed software to automatically structure and organise these photos into ``events''. This is a specialist device, not available except through MSR.
* GPS: we have used personal hand-held GPS recorders as well as within-vehicle GPS trackers, to record a person's physical location (outdoors) to 5m accuracy;
* Audio Events: in conjunction with Columbia University we used audio recordings of a person's day, taken from a hand-held MP3 recorded, to segment a full day into ``events'' based on ambient sound and based on speaker analysis.

Additionally, AIC in conjunction with the UCD Department of Physiotherapy has developed a wearable bend sensor that can be incorporated into standard apparel, resulting in a very wearable, very comfortable sensing modality. This has been evaluated in a spinal posture-sensing application, but in theory can be easily applied to any joint bend.

This portfolio of sensing technologies can now be used to truly ``instrument" a person with a view to harvesting a huge amount of raw data generated as the go about their daily activities. However, translating all of this raw data into an accurate activity profile for the wearer and then using that activity profile in applications such as memory aids is a challenging task and one that has not been addressed in a comprehensive manner. Understanding how different combination of sensors behave in relation to different sequences of activities is a critical step in being able to reliably classify and track the wearer's actions. No one sensing technology will provide the answer. Instead combining different sensor technologies is likely to offer significant advantages. In this project we take a number of important first-steps when it comes to understanding how we might combine these different sensor technologies in order to construct an accurate picture of a person's daily activities and then use this in aiding human memory. In terms of how this project relates to the theme of ODCSSS, the project will enable cleaner and more accurate activity classification and recording. In turn this means that we will be able to create a high level log or record of the wearer's major activities which can be reviewed or searched.

Relevance to Host Laboratory

The SenseCam along with the innovative posture-sensing torso garment is one of several projects within the SFI-funded Adaptive Information Cluster which aim to capture a person?s life experiences.

This ODCSSS project incorporates the work of two students. One student will work on developing and characterizing the bend sensor for use in detection of other joint movement, concentrating primarily on modeling the sensor?s performance in order to extract useful data. This will be accomplished by first defining the hysteresis and other irregularities of the sensor, and then using filtering techniques and sensor placement to minimize or model the sensor error. The second student will explore activity recognition for a user wearing a combination of sensing technologies, concentrating on determining the minimum number and combination of activity monitoring devices from the list above to allow approximate identification of the wearer's activity. We will address this by over-instrumenting a vet wearer for some period of time, collecting and recording as much data as we can using the devices mentioned above and then process the data to determine what is the minimum combination of devices needed.

Supervisor

Prof. Alan F. Smeaton     

Students who have worked on this project:

See also: