Interactive Browsing for a Visual Diary

Hosting University

Dublin City University

Overview

SenseCam is a small wearable digital camera that incorporates a number of sensors including light/infrared sensors, ambient thermometer and accelerometer, to automatically trigger photo capture throughout the day without the wearer having to consciously take the photos. As a result, the device captures about 3,000 photos in an average day, in effect making it an automatic visual chronicling tool to store the various things that happen throughout a day. Clearly, the potential benefits of having a visual record of your day are enormous - from the rather trivial example of helping you remember where you left your keys, to more serious applications with Alzheimer sufferers. However, the amount of photos taken during a particular day means that it is difficult to easily review those photos or quickly find a particular photo where the wearer remembers that something interesting happened, and this problem becomes far greater when multiple days' SenseCam images are to be browsed.

Within the Centre for Digital Video Processing (CDVP) at DCU, we are developing content-based photo analysis tools to automatically structure SenseCam photos. Our tools can identify boundaries of discrete events among the photos (e.g. a meeting, chatting with a friend, commuting in a bus, having dinner, etc.), select most representative photo within each of the events, and determine which events are more unique than others (thus being able to highlight or emphasise this to the user reviewing the photos). Taking full use of this processing, we are developing the SenseCam Interactive Browser to automatically compose an efficient and interesting visual summary of SenseCam photos for the user. The browser takes representative photos from each event, then places each of the photos on the browser by re-sizing it based on how unique the event is, and locating it on the panel in such a way that there will not be wasted space on the screen. While having different sizes of photos and conforming to chronological order of the photos, this automatic layout of different-sized photos on a screen requires a packing algorithm to make full use of the available screen area. The purpose of the proposed internship is to support the development of this browser, especially in the programming of the automatic layout of the photos. While we have a number of research student/staff working on the SenseCam project, we need student support in order to be able to quickly implement different versions of the browser and modify these as our design ideas evolve. This is a project covering aspects of image processing, interface design and optimisation approaches.

Relevance to Host Laboratory

The SenseCam is one of several projects within the SFI-funded Adaptive Information Cluster which aim to capture a person?s life experiences. Others include using a range of biometric signal monitoring devices as well as several ?media? projects to record all the TV, movies, etc. one listens to or views during the day. The project proposed here is important because it will enable us to manage and visually present very large archives of images taken using SenseCam to the SenseCam wearer and potentially aid human memory.

Supervisor

Dr. Hyowon Lee     

Students who have worked on this project: