Research Output

Motion correlation: selecting objects by matching their movement

  Selection is a canonical task in user interfaces, commonly supported by presenting objects for acquisition by pointing. In this article, we consider motion correlation as an alternative for selection. The principle is to represent available objects by motion in the interface, have users identify a target by mimicking its specific motion, and use the correlation between the system’s output with the user’s input to determine the selection. The resulting interaction has compelling properties, as users are guided by motion feedback, and only need to copy a presented motion. Motion correlation has been explored in earlier work but only recently begun to feature in holistic interface designs. We provide a first comprehensive review of the principle, and present an analysis of five previously published works, in which motion correlation underpinned the design of novel gaze and gesture interfaces for diverse application contexts. We derive guidelines for motion correlation algorithms, motion feedback, choice of modalities, overall design of motion correlation interfaces, and identify opportunities and challenges identified for future research and design.

  • Type:


  • Date:

    01 May 2017

  • Publication Status:


  • DOI:


  • ISSN:


  • Library of Congress:

    QA75 Electronic computers. Computer science

  • Dewey Decimal Classification:

    005.437 User interfaces

  • Funders:

    Google through a Faculty Research Award; Microsoft Research Centre for Social Natural User Interfaces (SocialNUI); Victorian State Government and Microsoft


Velloso, E., Carter, M., Newn, J., Esteves, A., Gellersen, H., & Clarke, C. (2017). Motion correlation: selecting objects by matching their movement. ACM transactions on computer-human interaction : a publication of the Association for Computing Machinery, 24(3), 1-35.



Motion correlation, interaction techniques, gaze interaction, gesture interfaces, eye tracking, motion tracking, natural user interfaces

Monthly Views:

Available Documents

  • pdf

    Motion Correlation: Selecting Objects by Matching their Movement

    Number of Downloads in the past year: 3

    © ACM, 2017. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Computer-Human Interaction, {VOL.24, ISS 4,01/05/2017}

  • Downloadable citations