Capture and Analysis of Biometric Data For Memory Augmentation with a SenseCam

Hosting University

Dublin City University

Overview

The SenseCam is a small wearable proactive personal camera developed by Microsoft Research in Cambridge, UK. It incorporates a digital camera and various sensors including a light sensor, an accelerometer, an ambient thermometer, and a passive infra red sensor. It is worn as a pendant around the user's neck, and incorporates a fish eye lens which captures what the wearer is currently viewing. The SenseCam automatically takes pictures when its sensors determine that a potentially interesting scene is in view, or by default every 50 seconds. Such scenes from might contain a person, or change of scenery. This results in the automatic capture of up to 3,000 images per day. The resulting captured data can be thought of as a visual diary of a wearer's activities. In addition to the photos the SenseCam also stores data from its sensors aligned with the captured SenseCam images. Reviewing of SenseCam images has been shown to improve short-term memory yet when reviewing. However, it is clear that intelligent techniques are needed to support effective searching and browsing of a SenseCam archive, since the continuous passive capture of images results in collections that make it difficult to locate important images by manual browsing.

In addition to the SenseCam, the emergence of highly portable highly portable, cost effective, pervasive, ubiquitous computing devices mena that is now possible to automatically capture context information associated with events in ones life. For example, biometric and physiological sensors can capture Galvanic Skin Response (a measure of skin conductivity which is affected by sweat from physical activity and emotional stimuli), Heat Flux (a measure of the heat being dissipated by the body), Skin Temperature (a measure of the body's core temperature), Acceleromater (a measure of stillness and movement) and Heart Rate monitors.

In this project large volumes of SenseCam images will be annotated with biometric and physiological data captured automatically with these portable devices. The aim of the project is to explore methods of using this biometric context data in conjunction with analysis of the SenseCam images to identify interesting events from an individual's visual daily. If biometric patterns associated with significant events can be identified, a SenseCam user can be directed to important images. By showing the user a summary of such important images they can be reminded of important scenes from their daily life. Existing research suggests that showing the user a summary of significant events will stimulate them to develop improved long term memory of important events in their life. The final objective of this project is explore the extent to which SenseCam images can be used as part of a successful memory augmentation process.

Relevance to Host Laboratory

The SenseCam is one of several projects within the SFI-funded Adaptive Information Cluster which aim to capture a person?s life experiences. Other project include using a range of biometric signal monitoring devices as well as several ``media'' projects to record all the TV, movies, etc. one listens to or views during the day. The project proposed here is important because it can potentially form an important component in exploiting very large archives of images captured automatically using the SenseCam.

Supervisor

Gareth Jones     

Students who have worked on this project: