The
nd 2
order local-image-structure solid Lewis D Griffin Computer Science, University College London, UK
[email protected]
0. Motivation The possible varieties of pure 1st order and pure 2nd order local structure are well understood. How are we to make sense of the range of possibilities of mixed 1st and 2nd order structure?
c00 mainly 1st order
=
1st & 2nd order
1. Abstract
c10 c20
mainly 2nd order
Gaussian derivative filters are a good model of V1 simple cells and their action can be interpreted in the framework of differential geometry. Six such filters can together probe local structure up to 2nd order. We have factorized the 6-D space of their joint outputs by the action of a group of transformations that leave intrinsic image structure invariant. The group is generated by: spatial translation, rotation and reflection, and increasing linear-transformations of intensity. The resulting factored space is a 3-D bounded orbifold. The orbifold has non-flat intrinsic curvature, but we have found a volume-preserving, mildly distorting (mean 20%) embedding of it into Euclidean 3-space. We call the embedded orbifold the 2nd order local-image-structure solid. It is shaped like a lemon, half-flattened so that it has two creased edges running between two sharp points. The two points correspond to umbilic extremum. One crease corresponds to the varieties of pure second order structure (extrema, saddles and ridges), the other to mixtures of an umbilic extremum and a plane. The solid can be used in the study of image statistics. For example, the histogram of local 2nd order forms for natural images shows a clustering of density around effectively 1-D local forms.
c01 c11
c02
Image Patch Gaussian Derivative filters
2. Local image structure is probed by V1 simple cells, the RFs of which are well-modelled as derivatives of Gaussians. Six filters are needed to measure up to 2nd order.
3. By factoring out luminance, contrast, orientation & handedness we arrive at a bounded 3-D orbifold of local forms. This can be embedded (with preservation of infinitesimal volumes but moderate distortion) into 3-D to give the solid shown to the right. The Maths These parameters (l, b, a), calculated from the derivative measurements cij and scale σ, are invariant to (i) affine transformations of the image intensity, (ii) rotation of the image plane, and (iii) reflection of the image plane. They are a coordinate system for the orbifold.
This is the metric tensor of the orbifold (induced by a metric on jet space) relative to the lba-system, expressed in line element form.
This is the embedding of the orbifold into Euclidean space:
σ ( c20 + c02 ) l = tan −1 2 2 4 c102 + c01 + σ 2 ( c20 − c02 ) + 4 c112
(
−1 b = tan σ a=
1 2
(
)
)
+ 4 c112 2 4 c102 + c01
( c20 − c02 )
2
(
)
2 c01 − c102 c11 + c10 c01 ( c20 − c02 ) tan 2 2 c01 − c102 ( c02 − c20 ) + 4c10 c01 c11 −1
( (
) )
2 2sin 2b 2 2 2 2 2 ds = dl + cos l db + da 5 − 3cos2b
b − π4 ) cosl ( l cosl sin2b χ b = ( a − π4 ) 2 5 − 3cos2b a −l xyz
white noise
solid centre
1/f noise
6. Breaking the solid into natural pieces defines a plausible set of feature categories.
4. The distribution of local forms for random-phase noise can be computed analytically. This shows that 1/f noise is more ‘edgy’ than white noise.
7. Key References
5. Natural images (unlike noise) show a bias towards local structure that is effectively 1-D (pink & brown curves)
Griffin LD (in press) The 2nd order local-image-structure solid. IEEE Trans. Patt. Anal. Mach. Intell. Koenderink JJ & van Doorn AJ (2003) Local structure of gaussian texture. IEICE Trans. Inf & Sys, IE86-D(7):1165-1171 Lee AB, Pedersen KS & Mumford D (2003) The nonlinear statistics of high-contrast patches in natural images. Int J Comp Vis 54(1-2):83-103
8. Acknowledgements natural images
1/f noise
Funded as part of EPSRC-funded project ‘Basic Image Features’ EP/D030978/1.