Research Output

Deep Precomputed Radiance Transfer for Deformable Objects

  We propose, DeepPRT, a deep convolutional neural network to compactly encapsulate the radiance transfer of a freely deformable object for rasterization in real-time.
With pre-computation of radiance transfer (PRT) we can store complex light interactions appropriate to the shape of a given object at each surface point for subsequent real-time rendering via fast linear algebra evaluation against the viewing direction and distant light environment. However, performing light transport projection into an efficient basis representation, such as Spherical Harmonics (SH), requires a numerical Monte Carlo integration computation, limiting usage to rigid only objects or highly constrained deformation sequences. The bottleneck, when considering freely deformable objects, is the heavy memory requirement to wield all pre-computations in rendering with global illumination results.
We present a compact representation of PRT for deformable objects with fixed memory consumption, which solves diverse non-linear deformations and is shown to be effective beyond the input training set. Specifically, a U-Net is trained to predict the coefficients of the transfer function (SH coefficients in this case), for a given animation's shape query each frame in real-time.
We contribute deep learning of PRT within a parametric surface space representation via geometry images using harmonic mapping with a texture space filling energy minimization variant. This surface representation facilitates the learning procedure, removing irrelevant, deformation invariant information; and supports standard convolution operations. Finally, comparisons with ground truth and a recent linear morphable-model method is provided.

  • Type:

    Conference Paper

  • Date:

    02 April 2019

  • Publication Status:

    Published

  • DOI:

    10.1145/3320284

  • ISSN:

    2577-6193

  • Library of Congress:

    QA76 Computer software

  • Dewey Decimal Classification:

    006.6 Computer graphics

  • Funders:

    EC European Commission

Citation

Li, Y., Wiedemann, P., & Mitchell, K. (2019). Deep Precomputed Radiance Transfer for Deformable Objects. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 2(1), 1-16. https://doi.org/10.1145/3320284

Authors

Keywords

Real-time rendering, global illumination, pre-computed radiance transfer, spherical harmonics, deep learning

Monthly Views:

Available Documents

  • pdf

    Deep Precomputed Radiance Transfer For Deformable Objects

    46MB
    Number of Downloads in the past year: 68

    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without feeprovided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and thefull citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored.Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requiresprior specific permission and/or a fee. Request permissions from permissions@acm.org.©2019 Copyright held by the owner/author(s). Publication rights licensed to ACM.2577-6193/2019/5-ART3 $15.00https://doi.org/10.1145/3320284Proc. ACM Comput. Graph. Interact. Tech., Vol. 2, No. 1, Article 3. Publication date: May 2019.

  • Downloadable citations

    HTML BIB RTF