A Generalized Ray Formulation For Wave-Optical Light Transport

Shlomi Steinberg; Ravi Ramamoorthi; Benedikt Bitterli; Eugene d'Eon; Ling-Qi Yan; Matt Pharr

doi:10.1145/3687902

Abstract

Under ray-optical light transport, the classical ray serves as a linear and local “point query” of light’s behaviour. Linearity and locality are crucial to the formulation of sophisticated path tracing and sampling techniques, that enable efficient solutions to light transport problems in complex, real-world settings and environments. However, such formulations are firmly confined to the realm of ray optics, while many applications of interest—in computer graphics and computational optics—demand a more precise understanding of light: as waves. We rigorously formulate the generalized ray, which enables linear and weakly-local queries of arbitrary wave-optical distributions of light. Generalized rays arise from photodetection states, and therefore allow performing backward (sensor-to-source) wave-optical light transport. Our formulations are accurate and highly general: they facilitate the application of modern path tracing techniques for wave-optical rendering, with light of any state of coherence and any spectral properties. We improve upon the state-of-the-art in terms of the generality and accuracy of the formalism, ease of application, as well as performance. As a consequence, we are able to render large, complex scenes, as in Fig. 1, and even do interactive wave-optical light transport, none of which is possible with any existing method. We numerically validate our formalism, and make connection to partially-coherent light transport.

**From ray optics to wave optics.** In this paper we present the generalized ray: an extension of the classical ray to wave optics. The generalized ray retains the defining characteristics of the ray-optical ray: locality and linearity. These properties allow the generalized ray to serve as a ``point query'' of light's behaviour---the same purpose that the classical ray fulfils in rendering. By using such generalized rays, we enable the rendering of complex scenes, like the ones shown, under rigorous wave-optical light transport. Materials admitting diffractive optical phenomena are visible: e.g., a diffraction grated Aluminium strip dispersing light; a Bornite ore with a layer of copper oxide causing interference; a Brazilian Rainbow Boa, whose scales are biological diffraction grated surfaces; and, a Chrysomelidae beetle, whose colour arises due to naturally-occurring multilayered interference reflectors in its elytron. Our formalism serves as a link between path tracing techniques and wave optics, and admits a highly general validity domain. Therefore, we are able to apply sophisticated sampling techniques, and achieve performance that surpasses the state-of-the-art by orders-of-magnitude. We indicate resolution and samples-per-pixel (spp) count in all figures rendered using our method. While these figures showcase converged (high spp) results, our implementation also allows interactive rendering of all these scenes at 1 spp. Frame times (at 1 spp) for interactive rendering are indicated. Implementation, as well as additional renderings and videos are available in our supplemental material.

Press

Two-minute papers episode

Overview

The purpose of this paper is not the reproduction of a particular appearance, or the modelling of a specific material. The purpose of this paper is to enable sampling, that is backward light transport, of arbitrary wave-optical distributions of light in complex, real-life scenes. The rendered images and materials in this work may not appear photorealistic, for example the iridescent Rainbow Boa’s scales are modelled as a simple two-dimensional diffraction grating, not entirely faithful to the real-world material. Better modelling of a material can be done, but is not the point. What these rendering do demonstrate is our ability to reproduce general wave-interference and diffraction effects, accurately, when doing backwards light transport (from the eye or camera), and applying a variety of sophisticated rendering and path tracing tools. This was not achieved by previous work.

In other words, the core contribution and motivation for this work is enabling the application of a much wider range of rendering and light transport tools to wave optics, and not so much the reproduction of some appearance or particular visual effect.

Wave Optics: Locality and linearity

Simulating the behaviour of light under rigorous wave optics, in complex scenes, is often difficult, because wave optics admit no clear linearity or locality.

Linearity is lost because waves interfere: when “adding up” light packets, they might annihilate each other (destructive interference) or add up to an intensity that exceeds their sum (constructive interference), anyhow the superposition is not linear.
Locality is lost as wave optics is a global process: there is, in general, no clear way to decompose light into local, linear packets that propagate into a narrow set of directions. It is not trivial to “cut” waves into smaller waves.

Classical rendering techniques rely on a linear rendering equation to (Monte-Carlo) integrate; they also require propagating local packets of energy (for example, a ray) in order to: (i) make use of spatial-subdivision accelerating structures, crucial to making path tracing an $\mathcal{O}(\log n)$ algorithm instead of linear; and, (ii) simplify the formulation of light-matter interaction, which now happen at a localized point, and not over a region containing multiple, mutually-interfering scatterers. If you forgo linearity and locality, you need to consider the mutual interference of the entire scene with itself, for example via highly-impractical wave solving.

Previous work that targets wave-optical rendering, for example A Generic Framework for PLT and Towards Practical Physical-Optics Rendering, regains a degree of locality and linearity by propagating forward partially-coherent light. Because such a decomposition depends on the wave properties of light, these properties need to be known in order to propagate light and evaluate its interaction with matter. This can be quite limiting in the kind of rendering tools we may use. While these previous work are able, at least in theory, to render any scene that you see in this paper, when the light transport gets complex, as in the snake enclosure scene, previous work will require a practically-impossible number of samples and runtime.

To achieve a local and linear description of light we take a novel approach: Instead of dealing with the light field directly and finding means to decompose it into local and linear packets, this work considers the eigenstates of photoelectric detection (i.e., our generalized rays). This change of physics—from quantifying light to quantifying its observable artefacts—is what enables an accurate decomposition into simple, mutually-incoherent (i.e. linear) constructs, which are used to sample and do backward light transport, thereby simulate (the observable response of) wave optics in complex scenes. Unlike previous work, this decomposition is exact.

Fast Forward

Renders

Wave-optical rendering, additional scene 1

Wave-optical rendering of a snake enclosure

Wave-optical rendering, additional scene 2

Wave-optical rendering, additional scene 3

Wave-optical rendering, additional scene 4

Wave-optical rendering, additional scene 5