INFORMATIK

Hardware-Accelerated Visual Hull Reconstruction and Rendering June 11, 2003 Ming -Peter Seidel Magnor Hans Ming Li, Li, Marcus Marcus Magnor, Magnor,, HansHans-Peter Seidel MPI MPI Informatik Informatik Saarbrü ücken, Saarbr cken, Germany Saarbrücken, Germany GI‘2003

Ming Li

Talk outline

INFORMATIK

Introduction Introduction Previous Previous Work Work Hardware -Accelerated VH Hardware-Accelerated VH Reconstruction Reconstruction Hardware -Accelerated VH Hardware-Accelerated VH Rendering Rendering System System Performance Performance & & Demo Demo Video Video Conclusions Conclusions & & Future Future Work Work GI‘2003

Ming Li

1

Introduction

INFORMATIK

Visual Visual hull hull •• An An approximate approximate shape shape representation representation –– Envelope Envelope of of true true geometry geometry •• Fast Fast acquisition acquisition and and visualization visualization of of real-world objects real real-world objects –– 3D 3D interactive interactive TV, TV, computer computer games, games, tele-communication, etc. tele tele-communication, etc.

Principle Principle of of visual visual hull hull reconstruction reconstruction •• Reconstructed -view silhouette multi Reconstructed from from multimulti-view silhouette images images –– Generate Generate silhouette silhouette cones cones from from multiple multiple silhouettes silhouettes –– Find Find intersection intersection of of these these cones cones GI‘2003

Ming Li

Previous Work

INFORMATIK

Visual Visual hull hull reconstruction reconstruction •• Voxel-based Voxel Voxel-based –– Discretize Discretize volume volume into into voxels voxels –– Discard Discard voxels voxels outside outside silhouette silhouette cones cones Rapid octree construction from image sequences. [Szeliski [Szeliski 1993]

•• Polyhedron-based Polyhedron Polyhedron-based –– Generate Generate boundary boundary representations representations of of silhouette silhouette cones cones from from silhouette silhouette polygons polygons –– Intersect Intersect polyhedral polyhedral silhouette silhouette cones cones Polyhedral visual hulls for realreal-time rendering [Matusik [Matusik et al. 2001]

GI‘2003

Ming Li

2

Previous Work

INFORMATIK

Visual Visual hull hull rendering rendering with with implicit implicit reconstruction reconstruction •• Render Render desired desired view view directly directly from from reference reference views views –– Reconstruction Reconstruction implicitly implicitly performed performed during during rendering rendering ImageImage-Based Visual Hull [Matusik [Matusik et al. 2000] Online Model Reconstruction for Interactive Virtual Environments [Lok 2001]

Our Our approach approach •• Render Render silhouette silhouette cones cones with with multiple multiple projective projective silhouette silhouette textures textures –– Reconstruction Reconstruction and and rendering rendering in in one one single single pass pass –– Fully Fully accelerated accelerated by by graphics graphics hardware hardware GI‘2003

Ming Li

Talk Outline

INFORMATIK

Introduction Introduction Previous Previous Work Work Hardware -Accelerated VH Hardware-Accelerated VH Reconstruction Reconstruction Hardware -Accelerated VH Hardware-Accelerated VH Rendering Rendering System System Performance Performance & & Demo Demo Video Video Conclusions Conclusions & & Future Future Work Work GI‘2003

Ming Li

3

Classification of Intersections

INFORMATIK

Definition Definition •• Silhouette Silhouette face face

Face -Cone intersection Face-Cone intersection •• The The intersection intersection between between aa silhouette silhouette face face and and aa silhouette silhouette cone cone from from another another view view •• Result Result :: A A polygon polygon on on the the silhouette silhouette face face •• Iterate Iterate such such intersection intersection for for other other silhouette silhouette cones cones GI‘2003

Classification of Intersections

Ming Li

INFORMATIK

Polygon -Polygon Polygon-Polygon intersection intersection •• The The intersection intersection of of the the set set of of polygons polygons obtained obtained from from Face-Cone intersections Face Face-Cone intersections •• Result: Result: A A visual visual hull hull face face •• Iterate Iterate such such intersection intersection for for each each silhouette silhouette face face of of each silhouette cone each silhouette cone GI‘2003

Ming Li

4

Hardware-Accelerated Visual Hull Reconstruction

INFORMATIK

Key Key idea idea •• Perform Perform 3D 3D intersections intersections on on rasterized rasterized silhouette silhouette faces faces

Accelerate -Cone Intersections Accelerate Face Face-Cone Intersections •• Silhouette Silhouette mask mask encoded encoded in in the the texture’s texture’s alpha alpha channel channel •• Project Project the the texture texture onto onto aa silhouette silhouette face face •• Multiple Multiple texture texture units units make make this this parallel parallel

GI‘2003

Ming Li

Hardware-Accelerated Visual Hull Reconstruction

INFORMATIK

Accelerate -Polygon Intersections Accelerate Polygon Polygon-Polygon Intersections •• Modulate -fragment per Modulate projected projected alpha alpha texels texels perper-fragment –– Fragment Fragment alpha alpha value value is is 11 only only ifif the the alpha alpha values values projected from all views except view k are 1 projected from all views except view k are 1 N

A = ∏ Ak k =1 k ≠ kˆ

view view kk :: the the view view which which generates generates the the silhouette silhouette face face currently currently being being rendered rendered

•• Enable Enable the the alpha alpha test test –– Pass Pass ifif (alpha (alpha value value >> 0) 0) GI‘2003

Ming Li

5

Talk Outline

INFORMATIK

Introduction Introduction Previous Previous Work Work Hardware -Accelerated VH Hardware-Accelerated VH Reconstruction Reconstruction Hardware -Accelerated VH Hardware-Accelerated VH Rendering Rendering System System Performance Performance & & Demo Demo Video Video Conclusions Conclusions & & Future Future Work Work GI‘2003

Ming Li

Textured Visual Hull Rendering INFORMATIK View -dependent texture View-dependent texture mapping mapping •• Overall -dependent weight view Overall viewview-dependent weight of of face face ff w.r.t w.r.t view view kk Wk , f = Vk , f ∗ Wk

•• Per-face visibility Per Per-face visibility function function  1, v • n < 0 Vk , f =   0, otherwise

–– Valid Valid for for convex convex objects objects •• Viewing Viewing angle angle deviation deviation weight weight –– Depend Depend on on novel novel viewing viewing directions directions Wk = 1 / cos −1 (d k • d t ) GI‘2003

Ming Li

6

Textured Visual Hull Rendering INFORMATIK Per -fragment color Per-fragment color computation computation •• Weighted Weighted average average C f = ∑ Wk , f * Tk  N

 k =1



N

∑W k =1

k, f

Cf : Output fragment color for face f. Tk: Texel color from view k N

∑W

•• Pre-normalization to Pre Pre-normalization to avoid avoid division division Wˆ k , f = Wk , f

k =1

k, f

•• Target Target function function to to be be evaluated evaluated on on graphics graphics hardware hardware N

C f = ∑ Wˆk , f * Tk k =1

–– NV_register_combiners, shader,, ATI_fragment_ NV_register_combiners, ATI_fragment_shader ATI_fragment_shader, ARB_fragment_program ARB_fragment_program GI‘2003

Ming Li

Textured Visual Hull Rendering Algorithm

INFORMATIK

Rendering Rendering Configurations Configurations •• Load -view silhouette multi Load multimulti-view silhouette color color images images and and silhouette silhouette masks masks as as RGBA RGBA textures textures •• Setup Setup corresponding corresponding projective projective texture texture matrices matrices •• Set Set alpha alpha test test to to “GREATER “GREATER than than 0” 0” •• Configure -fragment programmable per Configure perper-fragment programmable hardware hardware –– Compute Compute weighted weighted average average in in the the color color channel channel –– Compute Compute modulation modulation in in the the alpha alpha channel channel GI‘2003

Ming Li

7

Textured Visual Hull Rendering Algorithm

INFORMATIK

Rasterize Rasterize silhouette silhouette faces faces •• For For each each silhouette silhouette face face of of each each silhouette silhouette cone cone –– Compute -normalized view-dependent pre view Compute prepre-normalized view-dependent weighting weighting factors factors –– Encode Encode weighting weighting factors factors into into vertex vertex colors colors of of the the silhouette silhouette face face –– Render Render the the silhouette silhouette face face with with projective projective textures textures

GI‘2003

System Configuration

Ming Li

INFORMATIK

•• Clients Clients — — Four Four AMD AMD Athlon Athlon 1.1GHz 1.1GHz machines machines •• Server -processor machine dual machine Server — — Intel Intel P4 P4 1.7GHz 1.7GHz dualdual-processor •• Eight -calibrated pre Eight prepre-calibrated firewire firewire video video cameras cameras arranged along arranged along aa circle circle •• Sync Sync via via external external trigger trigger at at about about 15 15 fps fps •• Video resolution Video resolution: resolution:: 320x240 320x240 pixels pixels GI‘2003

Ming Li

8

Implementation & Limitations

INFORMATIK

Implementation Implementation •• OpenCV OpenCV for for image image processing processing •• OpenGL OpenGL for for rendering rendering

Limitations Limitations •• The The number number of of reference reference views views limited limited by by the the number number of of available available texture texture units units –– Can -pass rendering multi Can be be lifted lifted by by multimulti-pass rendering •• Discretization Discretization artifacts artifacts while while applying applying projective projective texture texture mapping mapping –– Can Can be be alleviated alleviated by by increasing increasing texture texture resolution resolution GI‘2003

Ming Li

Experiments & Performances

INFORMATIK

•• Rendering Rendering window window size size :: 640 640 xx 480 480 pixels pixels •• Each -120 silhouette 100 Each silhouette silhouette cone cone has has 100100-120 silhouette faces faces Graphics hardware

GeForce 3

Radeon 9700 Pro

Reference view number

4

8

Frame rates(fps)

84

44

Previous fastest system performance for 4 views : 30 fps [Matusik et al. 2001] GI‘2003

Ming Li

9

Video — Dancer in Uffizi

INFORMATIK

88 reference reference views, views, 40 40 fps fps GI‘2003

Conclusion & Future Work

Ming Li

INFORMATIK

Conclusions Conclusions •• Proposed Proposed aa new new method method to to reconstruct reconstruct and and render render visual visual hulls hulls from from multiple multiple video video streams streams •• Significantly Significantly faster faster compared compared to to previous previous similar similar systems systems due due to to exploiting exploiting graphics graphics hardware hardware

Future Future work work •• Visibility Visibility computation computation for for general general objects objects using using shadow shadow maps maps •• Non-Photorealistic Rendering Non Non-Photorealistic Rendering GI‘2003

Ming Li

10

Hardware-Accelerated Visual Hull Reconstruction and Rendering Talk ...

Jun 11, 2003 - Talk outline. Introduction. Previous ... Talk Outline. Introduction ... Key idea. • Perform 3D intersections on rasterized silhouette faces. Accelerate ...

450KB Sizes 1 Downloads 145 Views

Recommend Documents

Hardware-Accelerated Visual Hull Reconstruction and ...
Jun 11, 2003 - System Performance & Demo Video. Conclusions & Future Work. Introduction ... 3D interactive TV, computer games, tele-communication, etc.

Combining Stereo and Visual Hull-MMSP
Disparity range: two disparity maps, upper and lower bound. Disparity range map from VH. • Use OpenGL to render VH into depth buffer. • Convert the depth ...

Online Accelerated Rendering of Visual Hulls in Real ...
Feb 7, 2003 - This paper presents an online system which is capable of reconstructing and rendering dynamic objects in real .... model are textured by different number of textures, we ..... Constructing virtual worlds using dense stereo.

Online Accelerated Rendering Visual Hulls in Real ...
Feb 6, 2003 - Online Accelerated Rendering of. Visual Hulls ... communication and sports analysis etc. Visual hull .... INFORMATIK. Ming Li. WSCG'03. Video ...

Saliency Detection via Foreground Rendering and Background ...
Saliency Detection via Foreground Rendering and Background Exclusion.pdf. Saliency Detection via Foreground Rendering and Background Exclusion.pdf.

Talk, Talk, Talk Student Example.pdf
... below to open or edit this item. Talk, Talk, Talk Student Example.pdf. Talk, Talk, Talk Student Example.pdf. Open. Extract. Open with. Sign In. Main menu.

Dynamic Stability of Flared and Tumblehome Hull ...
Leigh McCue, Aerospace and Ocean Engineering, Virginia Tech. ABSTRACT ... waves, and assess variations in ultimate stability for changes in the center of gravity (KG) for two ..... experimental and numerical data: techniques of nonlinear.

mobilizing capacity for reconstruction and development - Human ...
4.9 Paying the Price of Conflict: A Strategic Challenge ...... which deals with governance, democracy and the rule ..... money monthly to a member of the club.

convex hull best one.pdf
pn. },. the convex hull of P is the smallest. convex set C such that P⊂C. p1. p2. pn. C. Examples. Two Dimensions: The convex hull of P={p1. ,...,pn. } is a set. of line segments with endpoints in P. p1. p2. pn. C. Examples. Three Dimensions: The c

mobilizing capacity for reconstruction and development - Human ...
Population with Access to Education (%) .... broken promises of security and human development. ... access to knowledge, better nutrition and health services,.

Palaeohydrology and climate variability reconstruction ...
In the Xistral Mountains and using geochemical data, Martínez Cortizas et al. (1999) inferred the .... software package C2 version 1.5 (Juggins 2007). A Principal Component ..... The recovery of the native forest continued during ~1905-. 15 ...

RTTI reconstruction - GitHub
Mobile. Consumer. Cmd. Consumer. Munch. Sniffer. FileFinder. FileCollect. Driller ... o Custom data types: ✓ wrappers ... Identify Custom Type Operations ...

Replacement Hull Measurement Form.pdf
Sign in. Page. 1. /. 1. Loading… Page 1 of 1. Page 1 of 1. Replacement Hull Measurement Form.pdf. Replacement Hull Measurement Form.pdf. Open. Extract.

volume rendering pdf
Page 1 of 1. File: Volume rendering pdf. Download now. Click here if your download doesn't start automatically. Page 1 of 1. volume rendering pdf. volume ...

RENDERING GERRYMANDERING IMPOTENT
Specifically, the Democrats. (3.6) min dn. [Fd (dn)(dn − (−1)) + (1 − Fd (dn)) (rn − (−1))] ,. 8I'll discuss alternative obejective functions when examining the reform. 9Perhaps such candidates have trouble amassing the support needed to ma

pdf rendering engine
... was a problem previewing this document. Retrying... Download. Connect more apps... Try one of the apps below to open or edit this item. pdf rendering engine.

Schematic Surface Reconstruction - Semantic Scholar
multiple swept surfaces, of which the transport curves lie in horizontal planes. This section will introduce the basic reconstruction framework that initializes a set ...