Affine Layer Segmentation and Adjacency Graphs for Vortex Detection Shravan Heroor, Isaac Cohen Institute for Robotics and Intelligent Systems University of Southern California {heroor|icohen}@usc.edu Abstract In this paper we review and present different methods for the detection and characterization of vortices. Our algorithm works on the segmentation of the image into affine layers. These layers are computed using a parametric tensor voting and encoded in an adjacency graph. Paths are computed from the adjacency graph and are used for characterizing paths’ properties such as: critical points and vortices. We illustrate the proposed approach to a satellite image sequence of water vapor in the atmosphere.

1. Introduction Oceanographic and atmospheric images obtained from satellite platforms present a challenge to computer vision and pattern recognition. For example sea surface temperature (SST) data can be used to characterize vortex structures in the ocean and study sea surface streams. These sequences of images characterize the short-range evolutions of atmospheric and oceanographic processes. The evolutions of such processes are studied through the characterization of the changes appearing in the image sequences. Hence the first step would be to estimate dense motion fields and the salient features of these flow fields in order to estimate vortices and other critical points. The flow fields and their salient features are estimated through simultaneous segmentation using tensor voting and modeling the motion as affine. The underlying physical properties of meteorological phenomena like cloudy structures are complex. Our system does not take into account any underlying model of the meteorological phenomena, instead we use the information obtained from segmented image based on motion estimation. After the motion estimation there are a few methods we applied in order to characterize the location of the vortices. We discuss these approaches, their advantages and disadvantages in the following sections. In [2], the authors proposed an automatic algorithm for Geometric verification of swirling features in flow fields. Their method is based on the geometry of streamlines and choosing candidate vortex cores. Their work was tested on datasets of known vortex models like the ranking vortex, whereas our method works without assuming any prior information about the vortices present. Their work also has the disadvantage that the right vortex cores have to be chosen and good estimates of the initial seeds for streamlines are needed. This is not so in our method. Vollmers [3] presented a paper on various approaches to

vortex detection and characterization from twodimensional instantaneous vector fields like those obtained by means of the particle image velocimetry technique. Several methods were proposed and chosen depending on the available data and their inherent information content. Our work employs a uniform method for all types of flow fields. Cohen et al [4] proposed a non-quadratic regularization technique for solving the optical flow constraint equation and applied it to oceanographic images. The regularization problem was solved by the finite element method. They used a multi-resolution approach for coarse to fine grid generation depending on the region of interest. A phase portrait model for characterizing salient flow features was used. The first part of the paper deals with the motion estimation and affine layer segmentation. The second section explains the theory behind vortex characterization and then the implementation follows. The next section will comprise of our results. The final section of the paper will be conclusion and a discussion.

2. Affine Layer Segmentation We have considered the method proposed by Kang et al [1] for estimating the affine motion layers. The local motion regions between consecutive satellite images can be modeled by local affine motion. A parametric approach is used for region segmentation. Especially, the affine motion model is used since local motion regions between consecutive images can be modeled by affine motion. The approach combines tensor voting and parametric models, hence taking advantage of the representation of parametric affine motion model for the voting and region clustering. Hence multiple motion regions are segmented simultaneously based on tensor voting. Tensor voting allows inference of likelihood information, which identifies a smooth clustering criterion and the salient pixels belonging to a smooth motion group. In each set of consecutive images the regions are detected and segmented. Then, the affine parameters for each region are calculated and, based on these affine parameters region merging is done. This represents the overall view of the approach. First the dense correspondences between the two consecutive images are calculated using the Lukas and Kanade pyramidal algorithm to estimate the optical flow [6] . These dense correspondences are used to detect and segment regions. Affine motion is defined by six parameters. An affine joint image is defined by the 4D space (x,y,x’,y’) where

Proceedings of the 17th International Conference on Pattern Recognition (ICPR’04) 1051-4651/04 $ 20.00 IEEE

(x,y) and (x’,y’) are an image correspondence. The affine transform can be expressed in this representation as

(q

T

)

1 PT

q   1  

= 0 where

q = ( x, y, x ' , y ' ,1) and

s s s ' d s = e11 x s + e12 y s + e13 x s and i i i ' d i = e11 x i + e12 y i + e13 xi

In this equation ( x s , y s , x s' ) and ( xi , yi , xi' ) are the locations T of the two points. Each (e11 , e12 , e13 ) represents its normal tx  a b −1 0  . P =  direction. If the motion difference is smaller than a pixel, c d 0 − 1 t y   the two points get clustered together. For each clustered By defining p x = (a, b,−1, t x ) T and p y = (c, d ,−1, t y ) T we region, affine parameters are estimated by analyzing correlation matrix of clustered points in each space. The get two separate joint spaces and hence for the affine parameters are characterized by the eigenvector associated transform we use the equations defining planes in the with the smallest eigenvalue of the correlation matrix. decoupled space. These equations are given by: The next step is to combine the two joint spaces are p Tx q x = ax + by − x '+t x = 0 merged and then region merging is done using the same criteria in the combined space. p T q = cx + dy − y '+t = 0 y

y

y

Therefore, in decoupled joint space, each point defined by 3. Computation of Displacement Vectors q x and q y lies on a plane parameterized by p x and p y . The first step in characterization of vortices is determining the displacement vectors between the two consecutive Consequently, the correspondences constrained by an frames. These vectors help in characterizing the structure affine transformation are lying on a plane. Hence grouping of motion between the two frames. After the motion points belonging to the same plane is a good way to segmentation and parameter estimation we have the perform region segmentation and parameter estimation. correspondences between the two images and the However image correspondences contain many parameters of the affine patches. We also know which mismatches. Hence we use tensor voting to infer salient point belongs to which affine patch. So the displaced point plane features within noisy correspondences. The point for each point in the segmented images is calculated based coordinates, and its associated tangent or normal gives a on the affine patch it belongs to and its parameters. This is local description of the curve. The point coordinates and given by the equation: the surface normal at that point represent local information  x '   a b  x   t x  of a surface. To capture first order differential geometry  =   +    y '   c d  y   t  information and its singularities we use a second order    y     tensor. Such a tensor can be illustrated by an ellipse in 2D Here, (x,y) are the points in the segmented image and or as an ellipsoid in 3D. Intuitively, the shape of the tensor (x’,y’) are the end points after the motion. From (x,y) and defines the type of information captured (point, curve, (x’,y’) we can calculate the displacement vectors (u,v)=(x’surface), and the associated size represents the saliency. x,y’-y). A second order tensor can be expressed as a linear combination of a stick, a plate and a ball tensor. At each 4. Characterization of Vortices voting location, the estimate of each of the 3 types of Vorticity of a vector field is defined by the curl operator information and their associated saliency is captured. and vortices correspond to locations of zero curl. However, These local geometric properties are then propagated the detection of zero-curl pixels in the image is not within a domain of influence depending on the principal straightforward, as it depends on the resolution used for computing the curl operator. Often, zero curl pixels are not orientation. There are two voting stages. During the first voting, present in the image but rotational motion can be observed. each point collects votes cast by its neighbors and infers a One common way to describe vortices is by characterizing principal direction defining a plane. During the second them to be swirling kind of patterns occurring in the flow voting, voting is performed with a planar voting field fields or as a central core (or region) around which there is defined by the normal direction derived during the first swirling motion of the fluid. Most vortex detection step. At this step only highly salient points are allowed to algorithms have the disadvantage that they assume some model for the vortex. A number of vortex detection vote. Clustering points from the most salient points after the algorithms are discussed in Vollmers [3]. The main second voting performs the actual region segmentation. disadvantage with most of the methods described is that This is like a seeded region growing but does not require they are dependent on placement of the initial point and the number of regions to be known beforehand nor is the also to a certain extent on the scale factor. If a dense field seed random. Each neighboring point is compared to the is to be evaluated correctly more than one initial point has seed while clustering. The decision for clustering is made to be used and this leads to further complexity. In the next by the following distance equation: dist (d s , d i ) < 1

Proceedings of the 17th International Conference on Pattern Recognition (ICPR’04) 1051-4651/04 $ 20.00 IEEE

section we describe the different approaches we have approximate representative. The weighted average of the incorporated and their advantages and disadvantages. displacement vectors for that region form the displacement vector at the centroid. Hence initially each labeled region 4.1. Streamlines Streamlines are defined to be the path traced by a mass less forms a node in our graph. Now for each node, we find its particle as it moves with the flow in a velocity or adjacent nodes. This is based on connected neighbors. displacement field. The streamline equations are given as: Hence any region that shares a common boundary with the corresponding region is adjacent to it. So the adjacency list dy dx of the node in question is updated with the node = u ( x , y ), = v( x, y) dt dt corresponding to the region that shares a boundary with it. By eliminating the time variable we get a stream function This process is carried out for all the nodes. as: The next step is the extraction of paths. Paths are dx dy extracted from each node by doing a depth first search = using the node and its adjacent nodes. The depth first u ( x, y ) v( x, y ) search is stopped as soon as a node already present in the which, after integration becomes an equation of the path is encountered. This process is repeated for all the form: f ( x, y) = C . Each value of the integration constant nodes. Hence we are getting a complete representation of describes a streamline since the value off is constant. The all possible combinations of flow through the image. biggest problem with the streamline method is the correct Critical points can be characterized from flow fields by estimate of the initial position. We can choose an initial analyzing the paths through the image. For example, seed through visual inspection and estimation of the vortices can be characterized by paths, which satisfy the position of the vortex or other critical points. This would condition: however defeat the purpose of automatic detection. We where, dθ = atan(v / u ) , and (u, v) propose a method based on adjacency graphs. This is  ∫ dθ ≥ 2π described in the next section and this has given promising represents the displacement vector at the position of each results. node. The main disadvantages of the streamline method are not found here. There is no use of any initial point 4.2. Adjacency graph Method The streamline method does not really utilize the since we are considering all possible paths. However the information contained in the affine patches. It only relies main disadvantage is that it is computationally expensive on the velocity field. We propose in the following a method and also we are not taking into account the information that makes use of the information from the affine patches given by the displacement vectors to help construct the and region segmentation i.e. all the pixels in each affine path. To reduce the overall complexity we introduce a region have very similar motion and hence can be constraint while constructing the paths. For each path we represented by a single point or can be assigned as a node add those nodes lying in the half plane defined by the in a graph. This idea is the central motivation behind the displacement vectors at the position of its parent node. adjacency graph approach and has none of the disadvantages of the streamline method. So now the main Hence instead of considering all possible paths we are now idea is to represent the whole segmented image through an using the vector information to reduce the number of paths. Another method for reducing the numerical adjacency graph with each affine region forming a node of complexity is to reduce the number of nodes itself. The the graph. original segmented image after labeling is subject to a An adjacency graph is a graph in which each node has region merging, where regions with angles between them an adjacency list depending on whether the neighbors of less than or equal to 30° are merged and the weighted the node in question are adjacent to the particular node. average of the vectors is calculated. This step reduced the Here the term adjacent depends on the application and in number of nodes by almost half. Using the relabeled image our work adjacent means the surrounding nodes satisfying we construct the paths as stated earlier with the half plane a certain criteria. The initial step to the adjacency graph construction is constraint. This method performed better and gave very labeling of the image. Construction of a good graph good results. We can observe the paths forming closed depends on proper labeling. The labeling is done based on loops or paths indicating a central core with lot of swirling pixel color and regions belonging to the same affine patch motion, indicating the presence of a vortex. The total have the same color and each closed region is assigned a angles of these paths also satisfy the vortex condition different label. So there might be many regions with the stated earlier. same affine characteristics and hence similar motion The overall algorithm is given as: Find the affine motion patches of consecutive frames of models but with different labels. Then we represent each labeled region with a point, which is representative of the the dataset using tensor voting.

region. The centroid of the region can be considered as an

Proceedings of the 17th International Conference on Pattern Recognition (ICPR’04) 1051-4651/04 $ 20.00 IEEE

Calculate the displacement vectors for each pixel in the segmented image Label the segmented image of the affine patches. Re-label the image using the angle between the neighboring regions. Merge regions if angle less than 35°. Assign a node for each labeled region. For each node, find the neighboring nodes which share a common boundary with the node and update the adjacency list of the corresponding node with the indices of the adjacent nodes Construct the depth first search tree for each node with the constraint that a node is added to the path only if it lies in the half plane defined by the vectors of its parent node. Analyze paths with summation of angles at each node Figure 2: Streamlines computed using the adjacency graph. The gray circle corresponds to the starting point greater than or equal to 360°. of the streamlines. The figure illustrates the evolution of 5. Experimental Results: the vortex over time. We consider a sequence of METEOSAT images acquired during a single day. The total set of 48 images taken at an 6. Conclusion interval of 30 minutes (see Figure 1). 10 pairs of We have presented in this paper a new approach for consecutive images in the water vapor channel are used. detecting vortices in environmental image sequences. The method is based on a segmentation of the image into affine motion layers representing these layers in an adjacency graph. This graph permits us to infer streamlines from the graph and characterize the presence of vortices in the image. Future work will focus on a multi-resolution study of the proposed method to localize vortices present in the image at various scales.

Acknowledgments The authors would like to thank the AIR project at INRIA for the use of the data. This research was partially funded by the NSF US-France cooperation (INRIA) research grant.

References [1] E. Kang, I. Cohen and G. Medioni. “Non-Iterative Approach to Multiple 2D Motion Estimation”, ICPR’04. Cambridge, United Kingdom. August 2004. Figure 1: Original water vapor images, segmented [2] M. Jiang, R. Machiraju, and D. Thompson, “Geometric affine regions and computed streamlines. Verification of Swirling Features in Flow Fields”. Proc. of Visualization '02. pp 307-314 The first two images above are the original frames from the METEOSAT sequences, the corresponding affine layer [3] H. Vollmers, “Detection of Vortices and quantitative evaluation of their main parameters from experimental segmentation and the computed streamlines. As one can velocity data”, Institute of physics publishing, 2001,pp. observe, the detection of vortices form streamlines is not 1199. obvious due to the sampling and the selected seed points. [4] I. Cohen, I. Herlin, “Non Uniform Multiresolution Method In Figure 2, we show the evolution of vortex streamlines for Optical Flow and Phase Portrait Models: Environmental detected using the proposed adjacency graph in subsequent Applications”, IJCV, 33(1), 1999, pp.29-49. frames [5] Q. Yang, B. Parvin, and A. Mariano, “Detection of Vortices and Saddle points in SST Data”. Geophysical Research Letters, vol. 28, Issue 2, p.331, 2001. [6] Lucas, B. D. and Kanade, T. “An iterative image registration technique with an application to stereo vision”, IJCAI’81 Vancouver, pp. 674--679

Proceedings of the 17th International Conference on Pattern Recognition (ICPR’04) 1051-4651/04 $ 20.00 IEEE

Affine layer segmentation and adjacency graphs for ...

We illustrate the proposed approach to a satellite image sequence of water vapor in ... affine motion model for the voting and region clustering. Hence multiple ...

291KB Sizes 1 Downloads 195 Views

Recommend Documents

Affine layer segmentation and adjacency graphs for vortex ... - USC IRIS
modeling the motion as affine. The underlying physical properties of meteorological phenomena like cloudy structures are complex. Our system does not take into account any underlying model of the meteorological phenomena, instead we use the informati

Polynomial algorithm for graphs isomorphism's
Polynomial algorithm for graphs isomorphism's i. Author: Mohamed MIMOUNI. 20 Street kadissia Oujda 60000 Morocco. Email1 : mimouni.mohamed@gmail.

A NEW AFFINE INVARIANT FOR POLYTOPES AND ...
where u·y denotes the standard inner product of u and y. The projection body,. ΠK, of K can be defined as the convex body whose support function, for u ∈ Sn−1 ...

A new affine invariant for polytopes and Schneider's ...
New affine invariant functionals for convex polytopes are introduced. Some sharp ... presented applications of such results in stochastic geometry. However, a ...

Endogenous Market Segmentation for Lemons
Sellers may send some messages to buyers (explicit). Or, they .... Stango (2004) for an online used computer market. 5Although many ... economy where the informational free-riding problem is so severe that a socially efficient technology .... Sellers

Power and subcarrier allocation for physical-layer security in ...
I. INTRODUCTION. S ECURITY is a crucial issue in wireless systems due to ... The as- sociate editor coordinating the review of this manuscript and approving it for .... _in_OFDMA_Based_Broadband_wireless_networks.pdf ... .pdf. Page 1 of 14.

Power and subcarrier allocation for physical-layer ...
Page 3 of 10. Page 3 of 10. Wang_Xiaowei_TIFS11_Power_and_subcarrier_alloca ... _in_OFDMA_Based_Broadband_wireless_networks.pdf.

Cross-Layer Routing and Rate Control Strategies for ...
Los Angeles (UCLA), USA. Email: {jlhsu, rubin}@ee.ucla.edu. we characterize the performance behavior experienced by packets and flows traversing this node ...

Joint Cross-Layer Scheduling and Spectrum Sensing for ... - IEEE Xplore
secondary system sharing the spectrum with primary users using cognitive radio technology. We shall rely on the joint design framework to optimize a system ...

Capping layer for EUV optical elements
Jun 28, 2000 - post-exposure bake WEB), development, a hard bake and measurement/ .... design program TFCalc (SoftWare Spectra Inc.) and veri?ed.

Cross-Layer Routing and Multiple-Access Protocol for ...
packet flows or real-time streams, we are able to provide quality of service guarantees for selected group of flows. Transmission power control in wireless ...

Cross-Layer Routing and Multiple-Access Protocol for ... - CiteSeerX
Requests are considered for store-and-forward service by allocating slots for ..... [21] S. M. Selkow, The Independence Number of Graphs in. Terms of Degrees ...

Combined cross-layer design and HARQ for multiuser systems with ...
design framework for cross-layer scheduling and HARQ design in multiuser systems with slow ... obtain useful design insights such as the role of HARQ in the.

Robust Speaker segmentation and clustering for ...
cast News segmentation systems([7]), in order to be able to index the data. 2 ... to the meeting or be videoconferencing. Two main ... news clustering technology.

Mo_Jianhua_CL12_Relay Placement for Physical Layer Security A ...
Sign in. Page. 1. /. 4. Loading… .... PDF (d)=1 − dα. sedα. re. (dα. rd + dα .... In Fig. 2, we plot. PDF (d) and PRF (d) as functions of the relay position. We. find that ...

using graphs and random walks for discovering latent ...
ROUGE-1 scores for different MEAD policies on DUC 2003 and 2004 data. . . . . 43. 3.4. Summary of official ROUGE scores for DUC 2003 Task 2. ...... against using force against Iraq, which will destroy, according to him, seven years of difficult diplo

The Language of Function and Graphs - Shell Centre for Mathematical ...
Many pupils are well acquainted with graphs, tables of numbers, and ...... To optimise the profit made by a small business which assembles and sells two .... outward speed of 350 km/h? How can you show a return speed of 250 km/h? 800. ~.

Capping layer for EUV optical elements
Jun 28, 2000 - alternative apparatusiwhich is commonly referred to as a step-and-scan .... energy-sensitive material to said second object table; irradiating said mask and ..... (e) is an optimized RhiMo/Be stock similar to example. 40 above;.

Mo_Jianhua_CL12_Relay Placement for Physical Layer Security A ...
Mo_Jianhua_CL12_Relay Placement for Physical Layer Security A Secure Connection Perspective.pdf. Mo_Jianhua_CL12_Relay Placement for Physical ...

Endogenous Market Segmentation for Lemons
Information asymmetry between sellers and buyers often prevents socially desirable ..... where the informational free-riding problem is so severe that a socially efficient technology cannot ... a unit of the good, whose quality is either high or low.

ACTIVITY-BASED TEMPORAL SEGMENTATION FOR VIDEOS ... - Irisa
The typical structure for content-based video analysis re- ... tion method based on the definition of scenarios and relying ... defined by {(ut,k,vt,k)}t∈[1;nk] with:.

Validation Tools for Image Segmentation
A large variety of image analysis tasks require the segmentation of various regions in an image. ... In this section, we first describe the data of the experiments.

ACTIVITY-BASED TEMPORAL SEGMENTATION FOR VIDEOS ... - Irisa
mobile object's trajectories) that may be helpful for semanti- cal analysis of videos. ... ary detection and, in a second stage, shot classification and characterization by ..... [2] http://vision.fe.uni-lj.si/cvbase06/downloads.html. [3] H. Denman,