Variance WIE based infrared images processing L. Yang, Y. Zhou, J. Yang and L. Chen As a robust criterion to evaluate the complex degree of infrared (IR) backgrounds, the variance weighted information entropy (WIE) is applied to preprocess the IR small target images and detect the sea–sky line in sea–sky IR images. Experimental results show that the local variance WIE filter can evidently increase the signal-to-noise ratio of preprocessed images. In addition, the validity of the variance WIE based sea–sky line detection method is verified by experiments.

a

b

Introduction: In real infrared (IR) images, objects with different IR radiation normally appear in the form of different changes of greyvalues. To describe the change of grey-values on different objects more clearly, a concept ‘complex degree’ was presented [1, 2]. To measure the complex degree of different IR images quantitatively, the weighted information entropy (WIE) is introduced to describe different IR backgrounds [1]. Subsequently, the variance WIE, an improved criterion, was presented in [2]. Let S denote the set of grey-values in an IR image with 256 grey levels, ps be the probability of grey-value s occurring in the set S, s¯ is the mean of grey-values in the IR image, the variance WIE of the image can be formulated as

e

f

HðSÞ ¼ 

255 P

ðs  s Þ2  ps log ps ;

c

d

g

h

Fig. 1 Local variance WIE filtering samples a–d Original IR small target images e–h Corresponding local variance WIE filtering results

In Fig. 1, four representative IR small target images with 128128 pixels are selected to confirm the validity of our method. We use the following two metrics that are similar to the metrics presented in [1, 5] to compare the different preprocessing methods. Signal-to-noise ratio gain : SNR gain ¼

when ps ¼ 0; let ps log ps ¼ 0

s¼0

ðS=CÞout ðS=CÞin

ð3Þ

ð1Þ As a robust criterion to describe different IR backgrounds, the variance WIE can be applied to many domains of IR images processing. Here we discuss how to use the variance WIE to preprocess IR small target images and detect the sea–sky line in IR images.

Variance WIE based IR small target images preprocessing: Image preprocessing is an important step to detect or track small targets in IR images. Filtering in the frequency domain is a direct and effective approach for image preprocessing. For example, an adaptive Butterworth highpass filter was presented to preprocess IR images [1]. This method does not have good real-time performance in actual applications. Alternatively, filtering in the spatial domain is more practical for preprocessing IR images. Peng and Zhou designed a 55 spatial highpass filter template to preprocess IR images [3]. Through calculating the local standard derivation of each pixel in an IR image, a more effective preprocessing method, called the local standard derivation filter, was presented in [4]. Actually, the complex degree can be regarded as a concept that corresponds to the distribution of frequency components in an IR image. For instance, the regions with little change of grey-values in an image (low complex degree), such as mild sky, mainly consist of the low frequency components in the image. The regions with small targets usually have drastic change of grey-values (high complex degree) and thus mainly consist of the high frequency components in the image. Therefore, it naturally gives us an opportunity to design a spatial highpass filter for preprocessing IR small target images. For a pixel x in an IR image, if there are m kinds of grey-values s1, s2, . . . , sm in the neighbourhood M of the pixel, and the probabilities of each grey-value in the neighbourhood are ps1, ps2, . . . , psm, respectively, the local variance WIE value of the pixel x can be defined as

Background suppression factor : BSF ¼

Cin Cout

ð4Þ

where S is the signal amplitude, and C the noise standard deviation in the image. Experimental data are listed in Table 1. Here we select a 55 window as the neighbourhood M to do the local standard derivation filter and the local variance WIE filter. From the preprocessed images in Fig. 1 we see that the intensity of every target in each individual image is increased greatly. From Table 1, we can see that the SNR gain of the local variance WIE filter is dramatically larger than that of others. Although the BSF of the median filter is usually larger than the BSF of the local variance WIE filter, it makes more sense to increase the SNR of the preprocessed images in real applications such as IR small target detection and tracking in complex backgrounds.

Table 1: Comparison of several IR images preprocessing methods Filtering methods Metrics

55 highpass template SNR gain

BSF

Median SNR gain

BSF

Local standard deviation SNR gain

BSF

Local variance WIE SNR gain

BSF

a

1.0163 3.3432 3.2800 4.8271 4.6404 4.0142 10.8480 4.3390

b

1.5146 1.7731 1.2801 2.8257 1.7154 1.9094

3.4261

2.1608

c

1.4161 1.2321 2.3359 2.8850 2.2871 1.7128

3.8747

1.8374

d

1.0828 1.9538 1.1525 3.0147 3.1411 2.1201

6.7803

2.8873

a–d Original IR images shown in Fig. 1

ð2Þ

Variance WIE based sea–sky line detection in sea–sky IR images: The sea–sky line can be utilised to separate the IR sky backgrounds from the IR sea backgrounds, thus it is a significant condition to categorise and recognise the two kinds of targets. In general, the sea–sky line can be approximately regarded as a line with a slope that corresponds to the large gradient between the sky and the sea. The common algorithm for sea–sky line fitting is to find the maximal gradient in images [6, 7]. For an IR image F(x, y) with M  N pixels, the common algorithm can be briefly described as the following steps.

where s¯ (x) is the mean of grey-values in the neighbourhood M of the pixel. By calculating the local variance WIE value of each pixel in the IR image, a local variance WIE image will be obtained. To simplify the following discussion, the preprocessing method proposed here is called the local variance WIE filter.

Step 1: Evenly break down the original image into K sub-images along the row direction, so that the size of each sub-image is M  P (P ¼ N=K) pixels. Step 2: Compute the column vector with the means of grey-values in each row of every sub-image, respectively. For example, for the kth (k ¼ 1, 2, . . . , K ) sub-image, the means of grey-values of the jth P(k (P)j ¼ 1, 2, . . . , M ) row can be calculated as s¯ ( j, k) ¼ 1=P i¼(k1)  Pþ1F( j, i). Therefore, a column vector with the mean of

HðxÞ ¼ 

m P

ðsi  s ðxÞÞ2  psi log psi

i¼1

ELECTRONICS LETTERS 20th July 2006 Vol. 42 No. 15

grey-values in each row of this sub-image can be expressed as S¯ k ¼ (s¯ (1, k), s¯ (2, k), . . . , s¯ (M, k))T. Step 3: Calculate the gradient vector of each column vector and obtain the candidate points for sea–sky line fitting. For example, for the kth sub-image, the gradient vector of the column vector can be presented as G¯k ¼ (g(1, k), g(2, k), . . . , g(M  1, k))T, where g(h, k) ¼ js¯ (h, k)  s¯ (h þ 1, k)j (h ¼ 1, 2, . . . , M  1). Suppose that the maximal element in vector G¯k is g(vk, k), thus the co-ordinate of the candidate point for sea–sky line fitting corresponding to this sub-image can be calculated by xk ¼ vk and yk ¼ (k  1)  P þ (P=2). Step 4: Sea–sky line fitting with candidate points. Suppose that the coordinates of candidate points that correspond to each sub-image are (x1, y1), (x2, y2), . . . , (xk, yk), respectively, and the formula of sea–sky line is y ¼ Ax þ B. According to the principle of least-square error, the parameters of the fitted sea–sky line can be computed by

a

b

e

f

c

g

d

h

Fig. 2 Comparison of common algorithm with variance WIE algorithm for sea–sky line detection in IR sea–sky images

K A¼

K P

xi yi 

i¼1

K

K P i¼1

K P

xi

i¼1

x2i

K P i¼1

 K 2 P  xi i¼1

K P

yi B¼

i¼1

x2i K

K P

yi 

i¼1 K P

i¼1

x2i 

K P

xi

i¼1

K P

xi yi

i¼1

 K 2 P xi i¼1

The precondition of this algorithm is that the sea–sky line is corresponding to the maximal gradient value in images. However, influenced by sea clutter, the gradient values of the mean of grey-values in sea clutter are occasionally larger than that of the sea–sky line. A false sea– sky line will be fitted in the common algorithm (Figs. 2a–d). To detect the sea–sky line more robustly, a variance WIE based algorithm is proposed as follows. Step 1: Evenly break down the original image F(x, y) into T sub-images f1, f2, . . . , fT along the column direction, so that the size of each subimage is Q  N (Q ¼ M=T) pixels. Step 2: Calculate the variance WIE value of each sub-image using the formula (1), respectively. A column vector with the variance WIE values can be obtained as H¯ ¼ (H( f1), H( f2), . . . , H( fT))T. Step 3: Calculate the gradient vector of the column vector H¯ , that is R¯ ¼ (r(1), r(2), . . . , r(T  1))T, where r(l) ¼ jH( f1)  H( flþ1)j (l ¼ 1, 2, . . . , T  1). Suppose that the maximal value in vector R¯ is, r(n), the sea–sky line existing region W(x0, y0) that consists of the sub-images fn1, fn, fnþ1 and fnþ2 can be obtained (it can avoid that the sea–sky line is cut off artificially). Step 4: Use the common algorithm we reduced above to fit the sea–sky line in the sea–sky line existing region. Note that here the IR sea–sky image F(x, y) discussed in the common algorithm should be replaced by the sea–sky line existing region W(x0, y0), and the corresponding size of image will be M  N ¼ 4Q  N. In Fig. 2 four samples are selected to compare these two algorithms. Here we let K ¼ 16 to test the common algorithm and let T ¼ 16, K ¼ 16 to test the variance WIE based algorithm, respectively. Through roughly estimating the sea–sky line existing region of a sea–sky image, the influence of sea clutter with large change of grey-values can be decreased greatly. Thus the variance WIE based algorithm shows better performance than the common algorithm (Figs. 2e–h).

a–d Detection results with common algorithm e–h Corresponding detection results with variance WIE algorithm

Conclusion: By calculating the local variance WIE values of each pixel in IR images, the local variance WIE filter, a new IR small target images preprocessing method is proposed. In addition, a variance WIE based sea–sky line detection method is proposed. Experimental results show the validity of our improvements. # The Institution of Engineering and Technology 2006 18 March 2006 Electronics Letters online no: 20060827 doi: 10.1049/el:20060827 L. Yang, Y. Zhou, J. Yang and L. Chen (Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, No. 800 Dongchuan Road, Shanghai 200240, People’s Republic of China) E-mail: [email protected] References 1 Yang, L., Yang, J., and Yang, K.: ‘Adaptive detection for infrared small target under sea–sky complex background’, Electron. Lett., 2004, 40, (17), pp. 1083–1085 2 Yang, L., Yang, J., and Ling, J.: ‘New criterion to evaluate the complex degree of sea–sky infrared backgrounds’, Opt. Eng., 2005, 44, (12), pp. 126401–126406 3 Peng, J.-X., and Zhou, W.-L.: ‘Infrared background suppression for segmenting and detecting small target’, Acta Electron. Sin., 1999, 27, (12), pp. 47–51 4 Yilmaz, A., Shafique, K., and Shah, M.: ‘Target tracking in airborne forward looking infrared imagery’, Image Vis. Comput., 2003, 21, pp. 623–635 5 Hilliard, C.I.: ‘Selection of a clutter rejection algorithm for real-time target detection from an airborne platform’, Proc. SPIE., 2000, 4048, pp. 74–84 6 Liu, S., et al.: ‘Research on locating the sea–sky line tal region of ship target’, Laser Infrared, 2003, 33, (1), pp. 51–53 7 Wei, Y., et al.: ‘An automatic target detection algorithm based on wavelet analysis for infrared image small target in background of sea and sky’, Proc. SPIE., 2003, 5082, pp. 123–131

ELECTRONICS LETTERS 20th July 2006 Vol. 42 No. 15