Research Output

IRIDiuM: immersive rendered interactive deep media

  Compelling virtual reality experiences require high quality imagery as well as head motion with six degrees of freedom. Most existing systems limit the motion of the viewer (prerecorded fixed position 360 video panoramas), or are limited in realism, e.g. video game quality graphics rendered in real-time on low powered devices. We propose a solution for presenting movie quality graphics to the user while still allowing the sense of presence afforded by free viewpoint head motion. By transforming offline rendered movie content into a novel immersive deep media representation, we display the content in real-time according to the tracked head pose. For each frame, we generate a set of 360-degree images (colors and depths) using cameras placed in selected locations within a small view volume surrounding a central viewing position. We employ a parallax masking technique which minimizes the rendering work required for the additionally visible surfaces in viewing locations around the main viewpoint. At run-time, a decompression and rendering algorithm fetches the appropriate surface data in real-time and projects them to the eye positions as the user moves within the tracked view volume. To further illustrate this ability for interactivity and embodiment within VR movies, we track the full upper body using our sparse sensor motion capture solver allowing users to see themselves in the virtual world. Here, both head and upper body are tracked in realtime using data from IMU (Inertial Measurement Unit) and EMG (Electromyogram) sensors. Our real-time solver, Triduna Live uses a physics-based approach to robustly estimate pose from a few sensors. Hand gesture and object grasping motions are detected from the EMG data and combined with the tracked body position to control gameplay seamlessly integrated within the deep media environment.

  • Date:

    24 July 2016

  • Publication Status:

    Published

  • Publisher

    ACM Press

  • DOI:

    10.1145/2929490.2929496

  • Library of Congress:

    QA75 Electronic computers. Computer science

  • Dewey Decimal Classification:

    006 Special Computer Methods

  • Funders:

    Innovate UK

Citation

Koniaris, B., Israr, A., Mitchell, K., Huerta, I., Kosek, M., Darragh, K., …Moon, B. (2016). IRIDiuM: immersive rendered interactive deep media. https://doi.org/10.1145/2929490.2929496

Authors

Keywords

real-time rendering, VR, viewer tracking, interaction

Monthly Views:

Available Documents

  • pdf

    IRIDiuM: immersive rendered interactive deep media

    16MB
    Number of Downloads in the past year: 1

    Permission to make digital or hard copies of part or all of this work for
    personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for thirdparty components of this work must be honored. For all other uses, contact the owner/author(s).
    c 2016 Copyright held by the owner/author(s).

  • Downloadable citations

    HTML BIB RTF