Edinburgh Napier University

  This short paper introduces the research of the 14 partner,
EU Framework 6 project, COMPANIONS. It focuses on
the development of PhotoPal, a multimodal system
harnessing the cutting edge of speech recognition, natural
language understanding and dialogue management,
alongside advances in touch, multi-touch and gesture driven
interactions. PhotoPal allows for a far more natural
engagement with a user’s digital photo collection, the tasks
associated with it and the tools enabling those tasks to be
achieved. Removing the mediation of a keyboard and
mouse paradigm and introducing a speech, touch and
gesture driven interaction experience enables the intuitive
familiarity of dealing with analogue photographs whilst
retaining the huge storage, metadata addition, editing,
sorting and distribution advantage of the digital format.
Utilising the power of natural language understanding has
the considerable addition of enabling a user to establish not
only a hugely rich and sophisticated pool of metadata, but
also a very personal and unique audio narrative. The
combination of this image specific metadata (such as
content, time, location) alongside more abstract elements
(such as why a photo was taken or what emotion it
provokes) gives rise to the potential of establishing rich life
narratives, or Digital Echo.
Furthermore, the conversational nature of the interaction
experience positively encourages and facilitates a social
experience, both with the PhotoPal system itself (and it’s
representative avatar) and with family, friends and beyond,
both locally and remotely. This experience not only goes
beyond locale, but also time and potentially life time