Draft. Not for circulation.

UNSUPERVISED CHANGE DETECTION USING RANSAC Bishwajit Sharma, Ish Rishabh, Subrata Rakshit Centre for Artificial Intelligence and Robotics, CV Raman Nagar, Phase III, Bangalore-560093, India {b_sharma, ish, subrata}@cair.res.in

Keywords: Change detection, RANSAC.

Abstract Detection of changes in images is a much discussed problem in a variety of disciplines, such as remote sensing, surveillance, medicine, civil infrastructure, etc. Fundamentally, two images captured at different time instances differ not only in the subject, but also in the conditions when the images were captured, namely, illumination, atmospheric absorption, sensor characteristics, noise, etc. A change detection algorithm must be tolerant enough to classify these changes as no-change, while keeping track of changes in the subject itself. The subject may appear, disappear, move, change its shape, or change its brightness or colour. In this work we model the spectral signals received from these surfaces at two different times as a linear function, resulting in amplification or attenuation of the dynamic range of the image. We estimate this shift in the dynamic range of the images using the Random Sample Consensus algorithm, and classify pixels not satisfying this shift as changes.

1 Introduction Multi-temporal images have been used extensively to detect changes for monitoring changes in satellite imagery. Change detection has important roles to play in medical diagnosis, video surveillance, and civil infrastructure, etc. Outputs from a change detection system can be utilised in change understanding, which is application dependent, e.g., tracking humans in surveillance video, or detecting changes in concrete structures in aerial imagery. Contrary to the understanding problem, change detection is fairly universal in approach and design. Two images of the same scene taken at different times may differ not only in content, but also in the recording conditions. Factors such as camera displacement or orientation change can be compensated via image registration techniques. Radiometric factors, such as illumination change, gain settings of the sensor, etc., which affect the dynamic range of the captured image need to be modelled and compensated efficiently in order to correctly identify changes. We assume that the images have been registered appropriately before being input to the change detection algorithm. The radiometric changes are dealt in accordance with classical model proposed by [7].

In this paper, we assume that the spectral signals received from a surface being imaged at two different times can be modelled approximately as a linear function. Various methods have been proposed to detect changes based on this model. Intensity normalisation [8] method is one of the most primitive, but is still used in studies [1]. Shading models use homomorphic filtering to separate the illumination component [5] and the reflectance component. For surfaces that do not follow the Lambertian model, local illumination changes are modelled as a piece wise polynomial function [6] or generic model suggested by [3]. These models have been used in applying Principle Component Analysis (PCA) to identify nochange along the principal component, while the other components identify changes [4,9]. We propose a RANSAC [2] based approach to determine the linear relationship between the radiometric variations in multi-temporal images. This method has the direct advantage of faster estimation, irrespective of magnitude of change, as opposed to the PCA based approach. The PCA based approach uses all pixels to determine the linear relationship, while RANSAC uses randomly selected pixels to determine the same, and corrects it iteratively. Since PCA uses all pixels, the method is directly dependent on the image size. RANSAC can determine this relationship using only a few pixels and is roughly independent of the image size. We explore the change detection problem and the linear relation between the spectral signals in the first section. The RANSAC based approach to determine this relation is discussed in section 2. Results are discussed and compared with PCA based change detection. Further improvements are explained in section 3, and the implementation details and results are presented in section 4.

2 The Change Detection Problem The basic aim of a change detection algorithm is to generate a binary mask, M: Rl -> [0, 1], differentiating between the change and no-change areas as M(x) = 1 if the pixel x has changed significantly (1) = 0 if the change is not significant. where x Є Rp is the pixel location, p = 2 for images, while for volumetric data, p = 3.Typically, l = 1 for greyscale images, l = 3 for colour images, or higher for multi-spectral and hyperspectral images.

Draft. Not for circulation.

The changes are considered significant if they pertain to motion, change in shape, size, colour, or the like. Changes in the noise pattern, illumination, and mis-registration error should not be identified as a change. We assume that the radiometric variation between two images of a Lambertian surface can be modelled approximately as a linear function. Thus, the pixel intensity in the second image is linearly related to the intensity of the first image as (2) I2(x) = s I1(x) + c where s and c are constants indicating the scale and offset of the second image with respect to the first. For surfaces that do not follow the Lambertian assumption, s and c can be modelled as piece-wise smooth functions [3]. Once the scale and offset are determined, the pixels that do not follow this relationship are classified as changes. The scale and offset can be determined using statistical approaches of intensity normalisation [1,7], or more robust approaches such as PCA [9]. All these methods use all the available pixels to determine the scale and offset. It should be noted that the methodology includes the changed pixels though they do not necessarily follow any relationship. Rejecting these change-pixels would result in better estimation of the scale and offset parameters. Iterative schemes take advantage of this fact and eliminate changed pixels in subsequent intensity normalisation calculations. These iterative schemes, though robust, are slow, especially when the images are large in size. Since the constants s and c can be determined, ideally, by using only two pixels having different intensities, the inclusion of all pixels in their determination is clearly an overhead. But since unchanged pixels satisfy the Lambertian assumption only to an approximation, it is better to achieve an average or general solution of s and c, which fits all the pixels to the model within an error bound ±δ. In this work we follow the Lambertian assumption. We determine the scale and offset parameters for radiometric change in the images and classify pixels that do not satisfy this relationship as change. Further, the deviation of a pixel's intensity, d, from this model determines the amount of change taken place at that pixel. (3) d(x) = |I2(x) - (s I1(x) + c)| /√(1+s2) We associate a Gaussian model of change at each pixel where the probability of a pixels being classified as no-change, pN is described by a function of d, (4) pN(x) = exp{-d(x)2/σG2} where pN(x) is the probability that the pixel at x suffered no change, d(x) is the amount of deviation measured as the distance from axis with slope s and offset c, and σG is chosen suitably according to the standard deviation of pixel intensities from the Lambertian approximation. The probability of a pixels suffering change can, therefore, be represented as (5) pC(x) = 1 – pN(x) where pC(x) is the probability that the pixel at x suffered change. The probabilities can be used to determine the change mask after directly applying a suitable threshold, or can be first pre-processed to eliminate noisy pixels and misregistration errors using spatial correlation.

The algorithm used in determining the scale and offset is based on a popular algorithm for model fitting, the Random Sample Consensus, and is discussed in the next section. The method [2] and its variants have been popular in the image processing community in the areas of scene analysis, automated cartography, and location determination problem.

3 Random Sample Consensus: use in Change Detection Fitting data to predefined model is a classical problem with solutions like least squares fitting, etc. These techniques tend to fit all of the presented data to the model, including outliers, which can deform the solution potentially. The Random Sample Consensus (RANSAC) algorithm iteratively selects a random subset of this data, fits it to the model, and evaluates the parameters at all other data points. Iterations stop when a set of size greater than a threshold fits the model. If such a set is not found, the largest consensus set is declared 'fit' to the model, while the rest are discarded as outliers. In the context of a change detection problem, the equation 2 describes a model and the set of intensities [I1, I2] form the complete data set. This data set consists of pixels that are in consensus with the model, implying no-change pixels, and pixels that do not fit the model, implying changed pixels. The RANSAC scheme is used to fit pixels in the two images to the model described in equation 2. Those pixels that do not fit the model are marked as changed. The RANSAC algorithm finds a solution and evaluates it for all the data points. However, in the change detection problem, testing a solution (s, c) on all the pixels turns out to be much more expensive. A modified approach is to choose N (>>2) randomly selected pixels, and use n out of these N pixels to find a solution, and evaluate it at the remaining pixels in the set of N. This ensures a fair evaluation of the solution, while limiting the computations to a much smaller subset. In the subsequent iterations, fresh samples of N pixels are chosen. To determine the solution (s, c), only two pixels are required, given the intensity at these pixels are different, hence, n is set to be 2. A larger value could be used with a least-square solution but it turns out to be computation intensive. The number of pixels in the sample, N, should be more than n. Typically, we choose N as 1000 for 640x480 images and 4000 for satellite images, with 500 iterations in each case. A direct advantage of this method over other statistical methods is seen when the change cluster in the scattergram of the input images is located far from the no-change cluster. PCA based iterative scheme, [9], uses all pixels to estimate the parameters (s, c), and refines them through further iterations. Pixels that are detected as change in the first iteration are discarded in subsequent iterations. In case of large deviations, a statistical estimate of (s, c) discards pixels from the no-change cluster as well, thus affecting the final solution of (s, c). Figure 1 gives a conceptual explanation. The axis AX1 is chosen when the change cluster is located as C1. Subsequent iterations discard pixels from the change cluster, and the final axis is along PAX. On the contrary, if

Draft. Not for circulation.

the change cluster is located at C2, the initial guess is along AX2, thus discarding pixels with no-change.

Figure 1: Diagrammatic representation of a scattergram. Iterative schemes start with axis AX1 and converge to PAX, declaring pixels in NC as no change, and pixels at C1 as change. If change pixels are clustered at C2, the initial axis shifts to AX2, causing final alignment away from PAX. The RANSAC based scheme, however, iterates without discarding pixels. It, therefore, has a higher probability of picking the correct parameters, which fit majority of pixels in the image. We present an experimental study to compare the behaviour of these two algorithms under the large deviation case.

Figure 2(b): The artificially introduced change block with a higher intensity (225). The PCA based iterative scheme (bottom-left) fails, aligning its axis with the constant intensity block. Bottom-right image shows change magnitudes captured using RANSAC. Consider two temporal images of the same location. We introduce an artificial block of uniform intensity representing change in one of the images. The cluster in their scattergram moves away from the no-change cluster as the intensity in the block is increased. In our experiment, the PCA based iterative scheme failed when the intensity in the artificial block was increased beyond a threshold, while a plain RANSAC based scheme worked regardless of the intensity. Figure 2 shows the experimental results. The artificially introduced block has a lower intensity in figure 2(a) than in figure 2(b). PCA based iterative algorithm detected changes in the image as shown. For higher intensities, the iterative algorithm failed, while RANSAC was successful irrespective of the intensity. It must be noted that the RANSAC scheme is fairly independent of the intensity of the change. This scheme is, therefore, more reliable.

4 Robust Change Detection

Figure 2(a): Top row shows two temporal images with artificially introduced change block of constant intensity (200). Change magnitudes (distance from no-change axis) using PCA based iterative scheme (bottom-left). Change magnitudes using RANSAC (bottom-right). Note that the actual changes in the images are captured as well.

Various techniques have been used to increase the algorithm efficiency, such as tweaking the RANSAC, and improving performance in noisy images. The basic change detection problem can be improved to counter noise effects and misregistration errors. The RANSAC algorithm can be further fortified to exclude implausible results for the slope. In this section, we discuss these techniques which make the complete algorithm robust. Noisy pixels not only disturb the convergence of RANSAC, but also allow impulses to be detected as false changes.

Draft. Not for circulation.

Spatial correlation is used to improve the performance in presence of noisy pixels. The probability of a pixel to depict a change is encouraged depending on the probabilities of its neighbouring pixels. Simple image smoothing operation achieves this spatial correlation, thus reducing the effect of noisy pixels.

were first registered using texture features, and then changes were detected. Each column in figure 4(a) show three bands. Seasonal variations are visible in addition to change in the overall intensity levels. First columns in figure 4(b) shows detected changes, which, after application of an appropriate threshold, are shown in the second column.

Further, mis-registration errors result in edge-like features in the change mask. Square root of the local variance of pixels is used weigh the changes. Since pixels in homogeneous neighbourhood have larger weights than those at edges, misregistration errors get suppressed automatically. (6) σi2(x) = ∑x'ЄŊ(x)(Ii (x')' –µi)2/(k2-1) where Ii (x') is a pixel in the k×k neighbourhood, Ŋ(x), of pixel x, having neighbourhood mean µi, and variance σi2(x), for the i'th image. Note that the current pixel is not included in the calculations. We choose the weight of change at that pixel as (7) w(x) = (√{∑i σi2(x)} + к)-1 where an additional bias, к, avoids indeterminate values. Further, it can be noticed that the factor s can never be negative, since a negative scale indicates that one of the images is negative of the other. In practice, depending on the pixel size, the factor s can have values in a range [1/2b 2b], where b is the size of each pixel in bits. Applying this constrain on the slopes determined in each iteration of the RANSAC algorithm considerably reduces the chances of errors, while ensuring faster convergence.

Figure 3: Two indoor images (top row) with illumination changes and changes on the screen area. Change magnitudes (bottom-left) include reflection from non-Lambertian surfaces. Changes detected after applying threshold (bottomright).

5 Implementation and Results

6 Conclusions

The images were first low-pass filtered to reduce the effect of spurious noise in subsequent processing. A RANSAC based algorithm then calculates the scale s, and offset, c, iteratively. A local variance map is computed on both images separately and then added, as shown in equation 6. The square-root of the inverse of these variances forms the weighing factor at each pixel of the change image, which is calculated according to equation 7.

A robust unsupervised change detection algorithm is presented. Solution of the Lambertian assumption is attempted via the use of RANSAC. It was noted that the RANSAC based scheme is independent of the magnitude of change. Further improvements were made to counter misregistration error and noise. A probabilistic model of change based on magnitude of deviation is used. Results for both indoor imagery and satellite images have been produced. It is noted that surfaces that do not follow the Lambertian assumption are classified as changes under different illumination conditions. The actual changes are detected successfully. The resulting change mask can be utilised in specific problems of change understanding.

Assuming a Gaussian model, change is measured as deviation of intensity from the intensity dictated by computed scale and offset, as shown in equation 3. This change is weighed as explained above, thus removing effects of mis-registration error. Finally, a suitable threshold can be applied on the change image to obtain the change mask. The algorithm was tested on indoor scenes, as well as multitemporal satellite images. The results are presented in this section. Figure 3 shows an indoor scene taken at two different times under different lighting conditions. Note the changes on the screen, in addition to illumination variation. The changes that are detected show actual changes on the screen, and changes due to illumination variation on surfaces that violate the Lambertian assumption. Figure 4 shows seasonal changes detected in each of the three bands of muti-temporal images of the same area. The images

Acknowledgements The authors wish to thank the Director, CAIR, for his continued support, and for granting permission to present this work. We are also thankful to the Computer Vision Group, CAIR, for their help.

References [1] X. Dai, S. Khorram. “The effects of image misregistration on the accuracy of remotely sensed change detection”, IEEE Trans. Geoscience and Remote Sensing, 36, pp. 1566-1577, (1998).

Draft. Not for circulation.

[2] M. A. Fischler, R. C. Bolles. “Random Sample Consensus: A paradigm for model fitting with application to image analysis and automated cartography”, Commun. ACM, 24, pp. 381-395, (1981). [3] S. Negahdaripour. “Revised definition of optical flow: integration of radiometric and geometric cues for dynamic scene analysis”, IEEE Trans. on Pattern Anal. Machine Intell, 20, pp. 961-979, (1998). [4] I. Niemeyer, M. Canty, and D. Klaus. “Unsupervised change detection techniques using multispectral satellite images”, in Proc. IEEE International Geoscience and Remote Sensing Symposium, pp. 327-329, (1999). [5] B. Phong. “Illumination for computer generated pictures”, Commun. ACM, 18, pp. 311-317, (1975).

Figure 4(a): The First Column shows three bands of satellite images of some location. The second column shows images of the same location with seasonal differences, and contrast variation probably due to drift in sensor settings. [6] R. J. Radke, S. Andra, o. Al-Kofahi, B. Roysam. “Image change detection algorithms: A systematic survey”, IEEE Transaction on Image Processing, 14, pp. 294307, (2005).

[7] A. Singh. “Digital change detection techniques using remotely sensed data”, International Journal of Remote Sensing, 10, pp. 989-1003, (1989). [8] M. S. Ulstad. “An algorithm for estimating small scale differences between two digital images”, Pattern Recognition, 5, pp. 323-333, (1973). [9] R. Wiemker, A. Speck, D. Kulbach, H. Spitzer, J. Bieniein. “Unsupervised robust change detection on multispectral imagery using spectral and spatial features”, Proc. Third International Airborne Remote Sensing Conference and Exhibition, 1, pp. 640-647, (1997).

Figure 4(b): Normalised distance (magnitude alone) of each pixel from the no-change axis in each of the bands (first column) and change mask after applying a suitable threshold. Only the seasonal variations are picked up, while the illumination change is discarded.

unsupervised change detection using ransac

the noise pattern, illumination, and mis-registration error should not be identified ... Fitting data to predefined model is a classical problem with solutions like least ...

1MB Sizes 1 Downloads 335 Views

Recommend Documents

Unsupervised Change Detection with Synthetic ...
False alarm rate no SRAD. 0.05%. 0.02%. 0.1. 0.21%. 0.01%. 0.5. 82.30%. 0%. 1.0. 80.05%. 0%. Alessandria. Λ (SRAD). Detection accuracy. False alarm rate.

Semantic-Shift for Unsupervised Object Detection - CiteSeerX
notated images for constructing a supervised image under- standing system. .... the same way as in learning, but now keeping the factors. P(wj|zk) ... sponds to the foreground object as zFG and call it the fore- ..... In European Conference on.

A New Shot Change Detection Method Using ...
Department of Electronic Engineering, Shanghai Jiao Tong University ..... Consumer Electronics, vol. 49, pp. ... Circuits and Systems for Video Technology, vol.

Automatic Affine Structure Recovery Using RANSAC
sented by a 6 degrees of freedom 4 4 matrix,. 3 for rotation and 3 for .... a good idea to normalize the 3D points so that .... Notes in Computer Science, 2018:144–.

Unsupervised Motion Artifact Detection in Wrist ...
MAs using machine learning (Taylor et al., 2015). – Trained supervised machine learning algorithms on a small. EDA data set collected in a lab ... Page 7 ...

Unsupervised Spatial Event Detection in Targeted ... - Semantic Scholar
Oct 28, 2014 - built with the expanded query, we first derive an optimization ..... and the keyword feature were chosen for its best performance. ..... materials/analysis tools: LZ TH JD. ... applications to biological deep web data integration.

Unsupervised Feature Selection for Outlier Detection by ...
v of feature f are represented by a three-dimensional tuple. VC = (f,δ(·),η(·, ·)) , ..... DSFS 2, ENFW, FPOF and MarP are implemented in JAVA in WEKA [29].

Unsupervised Morphological Disambiguation using ...
Below you can see three possible morphological parses for the Turkish word “masalı” .... We used the Good-Turing and the Kneser-Ney smoothing techniques to ...

Unsupervised Feature Selection Using Nonnegative ...
trix A, ai means the i-th row vector of A, Aij denotes the. (i, j)-th entry of A, ∥A∥F is ..... 4http://www.cs.nyu.edu/∼roweis/data.html. Table 1: Dataset Description.

Unsupervised Spatial Event Detection in Targeted Domains ... - People
Oct 28, 2014 - New York, United States of America, 3 Google, New York City, New York, United States of America. Abstract. Twitter has become a popular data source as a surrogate for monitoring and detecting events. Targeted domains such as crime, ele

Improving Part based Object Detection by Unsupervised, Online ...
based on online boosting to improve the performance on ... based image/video retrieval, etc. Recently, the boosting .... number of sequences from the CAVIAR video corpus [19], and a number of ..... In the Fifth International Conference on.

Unsupervised Spatial Event Detection in Targeted ... - Semantic Scholar
Oct 28, 2014 - 1 Department of Computer Science, Virginia Tech, Falls Church, Virginia, ... Funding: This work is supported by the Intelligence Advanced Research ...... 365. Brazil. 57. O Globo; O Estado de Sa˜o Paulo; Jornal do Brasil. 451.

Credit Card Fraud Detection Using Neural Network
some of the techniques used for creating false and counterfeit cards. ..... The illustration merges ... Neural network is a latest technique that is being used in.

Fire Detection Using Image Processing - IJRIT
These techniques can be used to reduce false alarms along with fire detection methods . ... Fire detection system sensors are used to detect occurrence of fire and to make ... A fire is an image can be described by using its color properties.

Protein Word Detection using Text Segmentation Techniques
Aug 4, 2017 - They call the short consequent sequences (SCS) present in ..... In Proceedings of the Joint Conference of the 47th ... ACM SIGMOBILE Mobile.

Fire Detection Using Image Processing - IJRIT
Keywords: Fire detection, Video processing, Edge detection, Color detection, Gray cycle pixel, Fire pixel spreading. 1. Introduction. Fire detection system sensors ...

Masquerade Detection Using IA Network
lenge to the computer security, where an illegitimate entity poses as (and assumes the identity of) a legitimate entity. The illegitimate user, called masquerader ...

Performance Evaluation of RANSAC Family
checking degeneracy. .... MLESAC takes into account the magnitude of error, while RANSAC has constant .... International Journal of Computer Vision, 6.