Importance Sampling for Production Rendering Mark Colbert

Google (formerly Disney’s ImageMovers Digital)

Simon Premoze Guillaume Francois

Weta Digital (formerly Moving Picture Company)

Problem

Problem • Focus on efficiently evaluating the illumination integral

Problem • Focus on efficiently evaluating the illumination integral

• Looking predominately at direct lighting

Illumination Integral

Illumination Integral

Illumination Integral

Illumination Integral

Illumination Integral

Illumination Integral

Illumination Integral ωo ωi

Lo (ωo , x) =

!

Ω

Li (ωi , x)f (ωo , ωi ) cos θi dωi

Illumination Integral •

Li - incoming light

ωo Li ωi

Lo (ωo , x) =

!

Ω

Li (ωi , x)f (ωo , ωi ) cos θi dωi

Illumination Integral ωo Li ωi

• •

Li - incoming light f - Bidirectional Reflectance Distribution Function (BRDF)

f

Lo (ωo , x) =

!

Ω

Li (ωi , x)f (ωo , ωi ) cos θi dωi

Illumination Integral ωo Li ωi

• •

Li - incoming light

•

Cosine for projection

f - Bidirectional Reflectance Distribution Function (BRDF)

f

Lo (ωo , x) =

!

Ω

Li (ωi , x)f (ωo , ωi ) cos θi dωi

Illumination Integral

Illumination Integral • Evaluate and sum all incoming directions

Illumination Integral • Evaluate and sum all incoming directions

• Computationally Expensive

Importance Sampling • Re-orient the sample directions to point in directions that provide better estimates with fewer samples

Importance Sampling • Re-orient the sample directions to point in directions that provide better estimates with fewer samples

Overview • BRDF Importance Sampling • Filtered Importance Sampling (FIS) • FIS for Area Light • Multiple Importance Sampling Sampling at MPC • Importance Guillaume Francois Simon Premoze

Goal • With the presentation and course notes,

the goal is to provide you with the tools to implement importance sampling in your production pipeline

Integration • Discretize integral into smaller solid angles N ! 2π ≈ Li (ωi , x)f (ωi , ωo ) cos θi N i=1 2

Integration • Deterministic Sampling • Break into even sized pieces (e.g. Trapezoidal Rule)

• Stochastic Sampling • Randomly choose sample directions • Monte Carlo Integration

Importance Sampling •

Sample according to a Probability Density Function (PDF)

• •

Effectively an educated guess

•

PDF

Map from a uniform sample space to a nonuniform space

• • •

Peaks represent important regions Non-negative Integrates to one

Importance Sampling • Need to adjust for not evenly spacing the samples

Importance Sampling • Need to adjust for not evenly spacing the samples

Importance Sampling • Need to adjust for not evenly spacing the samples

Importance Sampling • Need to adjust for not evenly spacing the samples

Importance Sampling • Need to adjust for not evenly spacing the samples

Importance Sampling • Need to adjust for not evenly spacing the samples

N ! 1 Li (ωi , x)f (ωi , ωo ) cos θi ≈ N i=1 p(ωi )

What do we sample from?

What do we sample from? • BRDF

What do we sample from? • BRDF • Environment

What do we sample from? • BRDF • Environment • Both?

BRDF Sampling • Sampling depends on choice of BRDF • Common BRDFs [1982] • Cook-Torrance (similar to specular in RenderMan) • Phong [1975]

Phong BRDF • Exponentiated cosine lobe around the specular reflection direction

!

n

Li (ωi , x) cos θs cos θi dωi

Ω

θs

Phong Sampling • Convert to spherical coordinates centered around the specular direction

p(θs , φs ) ∝ cosn θs sin θs

• Normalize cosn θs sin θs

p(θs , φs ) = ! π/2 ! 2π 0

0

cosn θs sin θs dφs dθs

(n + 1) n cos θs sin θs = 2π

Inverse Transform Sampling • Discrete case

S1

S2

S3

S4

S5

Inverse Transform Sampling • Stack the transforms on top of each other and sample uniformly

Inverse Transform Sampling • Stack the transforms on top of each other and sample uniformly

S1

S2

S3

S4

S5

Inverse Transform Sampling • Stack the transforms on top of each other Uniform Sampling

and sample uniformly

S1

S2

S3

S4

S5

Inverse Transform Sampling • Stack the transforms on top of each other Uniform Sampling

and sample uniformly

S1

S2

S3

S4

S5

Inverse Transform Sampling • Analytical equivalent is to integrate over the PDF

P (Θ) =

!

Θ

p(θ)dθ

0

• P is called

Cumulative Distribution Function (CDF)

• Invert and solve for the sample direction P (Θ) = ξ P −1 (ξ) = Θ

Phong Sampling • Often better to sample over each

dimension (depends on sampling pattern)

• Two dimensional equation • Create marginal and conditional

probabilities for each dimension and sample independently

Marginal Density • Integrate over entire domain of a dimension

Marginal Density • Integrate over entire domain of a dimension

Marginal Density • Integrate over entire domain of a dimension

Marginal Density • Integrate over entire domain of a dimension

Marginal Density • Integrate over entire domain of a dimension

p(θs ) =

!

2π

p(θs , φs )dφs

0

= (n + 1) cos θs sin θs n

Conditional Density • Using Bayes law, we can find the conditional distribution

Conditional Density • Using Bayes law, we can find the conditional distribution

Conditional Density • Using Bayes law, we can find the conditional distribution

Conditional Density • Using Bayes law, we can find the conditional distribution

p(θs , φs ) p(φs |θs ) = p(θs )

Conditional Density • Using Bayes law, we can find the conditional distribution

p(θs , φs ) 1 p(φs |θs ) = = p(θs ) 2π

Putting it all together • Integrate each to find the CDF • Invert to find the sample mapping: !

1 n+1

θs = arccos ξ1 φs = 2πξ2

"

Randomness • Given the mappings for importance

sampling integration we just to generate the uniformly random samples

• So what? Call rand().

Randomness

Randomness • rand() causes clumping

Randomness • rand() causes clumping • Blue-noise property:

Each sample point has a radius in which no other sample point can exist [Dunbar 2006]

Randomness • rand() causes clumping • Blue-noise property:

Each sample point has a radius in which no other sample point can exist [Dunbar 2006]

Low-Discrepancy Sequences • Quasi-random numbers (Deterministic)

• Maximal distance from each sample point (Stronger than blue noise property)

• Ideal coverage that looks non-regular to minimize visible aliasing 0,0

0,0

1,1

1,1

Folded Hammersley Sequence (ξ1 , ξ2 ) =

!

" i + 0.5 , Φ2 (i) N

Φ2 (i) ! 1 2 3 4

Binary! 1 10 11 100

Inverse Binary 0.1 0.01 0.11 0.001

Value 0.5 0.25 0.75 0.125

Let’s see some pictures... • Using importance sampling we can quickly produce okay results

40 Samples Per Pixel

Let’s see some pictures... • Using importance sampling we can quickly produce okay results

40 Samples Per Pixel

Deterministic Sampling

• Replace noise with aliasing

40 Samples Per Pixel

Deterministic Sampling

• Replace noise with aliasing

40 Samples Per Pixel

Deterministic Sampling

• Replace noise with aliasing

40 Samples Per Pixel

Filtered Importance Sampling

• Replace noise/aliasing with a smooth result

40 Samples Per Pixel

Filtered Importance Sampling

• Replace noise/aliasing with a smooth result

40 Samples Per Pixel

Filtered Importance Sampling

• Replace noise/aliasing with a smooth result

40 Samples Per Pixel

FIS Foundation • Warp the environment proportionally to the BRDF

FIS Foundation • Warp the environment proportionally to the BRDF

FIS Foundation • Warp the environment proportionally to the BRDF

FIS Foundation • Observing the environment proportional to the BRDF we have evenly spaced samples

Environment ! BRDF ! cos !

FIS Foundation • Observing the environment proportional to the BRDF we have evenly spaced samples

Environment ! BRDF ! cos !

FIS Foundation • Undersampling causes aliasing

FIS Foundation • Suppress the aliasing by pre-filtering

FIS Foundation

FIS Foundation • Ideal filter region

defined by going between the neighboring points

FIS Foundation • Ideal filter region

defined by going between the neighboring points

FIS Foundation • Ideal filter region

defined by going between the neighboring points

• Gives the solid angle: 1 Ωs = N · p(ωi )

FIS with Environment Maps • Must convert solid angle to a filter region in the image-based environment

• MIP-Map Filtering • Ptex Filtering (RenderMan)

MIP-Map Filtering • Pyramid of filtered images,

where each level is 1/4 the size of the previous

• Commonly available in all

renderers (including GPUs)

MIP-Map Filtering • Convert the solid angle Ωp = d(ωi ) to pixels for some mapping

• Use ratio of solid angle for a pixel to the total solid angle !

1 Ωs l = max log2 ,0 2 Ωp

"

w·h

MIP-Map Filtering • Which parameterization is ideal? • Most filtering engines treat each map of an environment independently

• Need a mapping that has overlap between other maps

MIP-Map Filtering • One solution, scaled dual paraboloid

Can we do better?

Can we do better? • •

YES! Ptex-based filtering [Burley 2008]

•

RenderMan

Cube map that filters across edges of faces giving seamless filters

• •

Use environment call Must convert solid angle into blur parameter:

blur =

!

3 Ωs 2π

Area Light • Environment lighting still a relatively newer construct for production

• Soft lighting for female characters in ACC

IMD’s Area Light • • • •

Decoupled model (wrong, but controllable) Visibility

•

Done independently and used to attenuate other components

Diffuse

•

Used analytical form factor computation [Baum et al. 1989]

Glossy reflections

•

Filtered Importance Sampling

Area Light • Still importance sample the BRDF • Send out samples to intersect area light rectangle

• Augment the rays with ray differentials to

project the filter region onto the area light

Spherical Cap • How wide do we make the ray differentials? Ωs !

Ωs θs = arccos 1 − 2π

"

θs

Ray differentials • Ideally 2 rays to capture the shape of the projected filter region

• Requires orthogonal basis vectors to the sample direction

• This requires a smooth coordinate frame

Area Light Filtering • •

Intersect

• •

Axis aligned clipping

Compute the area around the central intersection ...and filter!

Zi

Zi (a)

Zi (b)

(c)

Area Filtering

Area Light

Camera

MIP-Map FIS Results

n = 17 100 Samples

n=3 5 Samples

Environment Sampling

Reference

Filtered Sampling

MIP-Map FIS Results • Extreme anisotropy does not behave as well as isotropic BRDFs

"x = 0.01 "y = 0.01

"x = 0.01 "y = 0.08 Ward BRDF

"x = 0.01 "y = 0.29

MIP-Map FIS Results

50 Samples

200 Samples

Reference

50 Samples

5 Samples

Summary • Importance Sampling provides a good framework for numerically solving Illumination Integral

• FIS provides a production proven method of integrating glossy surface reflection

• FIS replaces alias or noise with smooth reflections ideal for production scenes

Acknowledgements • Google • ImageMovers Digital • Doug Epps • Davy Wentworth • Christophe Hery • Jaroslav Krivanek