1

Image interpolation by blending kernels Luming Liang

Abstract A new convolution-based image interpolation method is presented, whose kernel function is designed via blending some well-known kernels. The new kernel is a better approximation to Sinc function both in the space domain and the frequency domain. Comparative experiments with several polynomial spline type algorithms indicate that our approach exhibits a significant improvement in image quality. Index Terms Image interpolation, kernel function, blending, convolution.

I. I NTRODUCTION Served as a basic brick of many higher level image operation, the interpolation is widely used in the field of digital image processing [1], [2]. For example, in some basic geometrical manipulations, such as zoom-in, zoom-out ,rotation and even sub-pixel panning to images, interpolations are inevitable [3], [4]. Moreover, the image interpolation has played a very important role in actual imaging applications, including online videos, medical image processing [5], mobile photos, etc. However, digital images cannot be magnified too much without a degeneration of their visual qualities. How to alleviate this degeneration is the key task in designing a new image interpolation algorithm. In literature, there are various methods [6]–[13] of different kinds can achieve this target to some extends. Actually, the image interpolation is a signal recovery problem. To a band-limited signal, if the uniform sampling is done above the Nyquist frequency, this signal can be perfectly reconstructed by choosing the sinc function, i.e. f (x) = sinπxπx , as the kernel function, and this interpolation process is the so-called ideal interpolation. However, using the ideal interpolation kernel function cannot yield high quality high resolution(HR for short) images in practice, since this function in the space domain does not have a compact support. Thus it must be clipped to a certain interval before being used to interpolate the low resolution(LR for short) image. The main problem is that if we clip this sinc function into a window version, its shape in the frequency domain will change dramatically, therefore the interpolation result will be no longer ”ideal”(See Fig. 1). sinc window sinc

1

0.8

0.6

0.4

0.2

0

0

5

10

(a)

15

(b)

(c)

Fig. 1. The interpolation effect by using the window sinc. (a) is the comparison of the ideal kernel in the frequency domain, (c) is the magnified image of (b) when the window sinc is chosen as kernel. Luming Liang is with the Center for Wave Phenomena and the Department of Mathematical and Computer Science, Colorado School of Mines, part of this work was done in Central South University, China. E-mail: [email protected] Manuscript received ; revised

IEEE SIGNAL PROCESSING LETTER, VOL. , NO. , 2008

2

So, how can we solve this problem? One strategy is to find some functions which have compact support in the space domain and also possess a similar shape in the frequency domain compared to the sinc function. This thinking gives birth to a kind of image interpolation method, i.e. convolution-based methods. Among algorithms of this kinds, the simplest one is the nearest neighbor interpolation [14], [15], which has a quite low time complexity and a relatively easy implementation. However, the HR images produced by this method are full of blocking artifacts. In the purpose of alleviating the blocking effects, the bilinear interpolation [7], [8] is introduced, which uses the linear interpolation model to calculate unknown pixels. Results show that edges become blurred after being magnified. Another traditional method is called the bicubic interpolation [6], which interpolates a signal by a weighted sum of values at control points to a spline function. This method is also used in practice currently. The disadvantage of this method is that the high frequency part of the original image will be filtered due to the over smoothness of the cubic spline kernel. Blending is a technique widely used in the field of computer aided geometric design, which can produce curves and surfaces of new types by weighted averaging known curves or surfaces for some specific uses [16], [17]. With the development of the approximation theory and the computational methods, lots of new kernels are designed, such as o-Moms’ functions [18], Key’s functions [19], adaptive osculatory rational functions [12]. Based on these kernels, we will introduce a new convolution-based image interpolation method, whose kernel is a blending of those known ones. For simplicity, we assume here that the amplification factor k is the same both horizontally and vertically, which means that our algorithm will produce an (k × m) × (k × n) image from an m × n input image. The organization of this paper is as follows. In section 2, we will present our new interpolation kernel and then compare it to some others both in the space domain and the frequency domain. In section 3, some experimental results are given. Section 4 is devoted to conclusions. II. T HE DESIGN OF NEW KERNEL There are some well-known kernels used extensively in image interpolation, we first list them here for clarity. The third-degree oMoms’ function [18]: |x|3 |x| 13 − |x|2 + + , 2 14 21

oM om3 (x) = |x|3 85 29 2 − + |x| + |x| + , 6 42 21 0, Key’s functions [19]: Key(x) =

3 2 (a + 2)|x| − (a + 3)|x| + 1,

3

2

a|x| − 5a|x| + 8a|x| − 4a, 0,

0≤x<1 1≤x<2 2 ≤ x. 0≤x<1 1≤x<2 2 ≤ x.

Traditionally, a is set as -0.5. Adaptive osculatory rational interpolation kernel function [12]:

osc(x) =

1.0808 − 0.168|x|2 − 0.9129|x| , |x|2 − 0.8319|x| + 1.0808

0≤x<1

0.3905 + 0.1953|x|2 − 0.5858|x| , |x|2 − 2.4402|x| + 1.7676

1≤x<2

0,

2 ≤ x.

IEEE SIGNAL PROCESSING LETTER, VOL. , NO. , 2008

3

We define our new kernel function by blending two known kernels. I(x) = (1 − w)I1 (x) + w ∗ I2 (x),

(4)

where weight w ∈ (0, 1), and I1 and I2 can be arbitrary kernels mentioned above, we call them parents kernels. Based on our observations, we choose Key’s function and adaptive osculatory rational interpolation functions as parents kernels, i.e. ( I1 = Key(x) (5) I2 = osc(x) Blending kernels will also have compact supports in the space domain, when their parents possess this advantage. Various kernels including traditional ones(nearest neighbor, linear, cubic spline, see Fig. 2) and also recently developed ones listed above(See Fig. 3) are plotted both in the space domain and the frequency domain for comparisons with our new kernel defined by equation (4) and (5). From the Fig. 2 and Fig. 1 sinc nearest neighbor bilinear bicubic blending kernel

0.8

sinc nearest neighbor bilinear bicubic blending kernel

1

0.8

0.6

0.4

0.6

0.2

0.4 0

0.2 −0.2

−0.4 −2

−1.5

−1

−0.5

0

0.5

1

1.5

0

2

0

5

(a) space domain Fig. 2.

10

15

(b) frequency domain

Comparisons with traditional kernels, here ω = 0.5.

1 sinc oMom osculatory rational key blending kernel

0.8

sinc oMom osculatory rational key blending kernel

1

0.8

0.6

0.4

0.6

0.2

0.4 0

0.2 −0.2

−0.4 −2

−1.5

−1

−0.5

0

0.5

1

1.5

2

0

0

(a) space domain Fig. 3.

5

10

15

(b) frequency domain

Comparisons with some recently developed kernels, here ω = 0.5.

3, we can find out that our new kernel(represented in red lines) has better stop band and pass band. More details can be found in Fig. (4), which are magnified parts of sub-figure (b) in Fig. 3. III. E XPERIMENTS In this section, we will illustrate the ability of our new image interpolation kernel mentioned in the previous section by some comparing experiments. For convenience, it will be assumed herein that the amplitude factor is the same both horizontally and vertically. In our first experiment, we compare our methods with bicubic, oMom’s, Key’s and adaptive osculatory rational interpolations. We here amplify an ”A”-like grey image and an ”eye” photo non-integer times. We also use the sobel edge detector to extract the edge of our magnified images, and plot them next to corresponding result images. From Fig. 7 and Fig. 8, we can find out that, images interpolated by our methods have less jaggies then Key’s method, especially in the ”A”-like example, and also preserve more edge information when

IEEE SIGNAL PROCESSING LETTER, VOL. , NO. , 2008

4

sinc oMom osculatory rational key blending kernel

1.02

sinc oMom osculatory rational key blending kernel

0.1

0.08

1

0.06

0.98 0.04

0.96 0.02

0.94 0

0.92

−0.02

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2

2.5

(a) pass zone Fig. 4.

3.5

4

4.5

5

5.5

Details of the comparing results in Fig. 3.

(a) Fig. 5.

3

(b) stop zone

(b)

Original images

compared to adaptive osculatory rational interpolation, especially in the ”eye”-photo. That is to say, the blending kernel takes both advantages from its two parents. The second example is to evaluate the value of parameter w by calculating the PSNR (Peak Signal to Noise Ratio). We first down-sample an image, then up-sample it by our methods with different ws, and finally get an image which has a same size as the original one. Here, we choose bilinear down-sampling method to zoom in the test image, the classical Lena image. For two m × n images, the PSNR value is defined as follows: 2552 P SN R = 10log10 , (6) M SE 1 Pm Pn ˆ 2 where, M SE = m×n j=1 (Xij − Xij ) . i=1 For comparing, we also interpolate the LR image using two parents methods: Key’s interpolation [19] and osculatory adaptive rational interpolation [12]. Results can be found in TABLE I and Fig. 6. TABLE I T HE PSNR VALUES k

1.5

2

2.5

3

3.5

Key

28.1419

26.1344

24.7105

23.2954

22.0914

osc

28.0811

26.1023

24.7167

23.2677

22.0447

w=0.7

28.1388

26.1381

24.7410

23.2964

22.0762

w=0.6

28.1491

26.1456

24.7448

23.3020

22.0831

w=0.5

28.1569

26.1521

24.7455

23.3053

22.0885

w=0.4

28.1618

26.1517

24.7418

23.3076

22.0919

w=0.3

28.1620

26.1508

24.7379

23.3075

22.0937

w=0.2

28.1594

26.1472

24.7313

23.3041

22.0949

Here, we can see that performances of those blending kernels are not a simply average of that of their parents, however, they have a significant improvement. Also, we can find out that when the amplitude factor is fallen in different zones, different w should be chosen. For example, when amplitude factor k ∈ [1.5, 2), w should be 0.4, when k ∈ [2, 2.5), w should be 0.5. IV. C ONCLUSION A new design method of the image interpolation kernel is proposed, where new kernels are produced by blending some known kernels. Comparing results have shown that our methods can improve the visual quality to some extends as well as the PSNR values.

IEEE SIGNAL PROCESSING LETTER, VOL. , NO. , 2008

5

1.002

1.001

Key w=0.2 w=0.3 w=0.4 w=0.5 w=0.6 w=0.7

1.0005

1.0015 1

1.001 psnr/Key’s psnr

psnr/Key’s psnr

0.9995

0.999

1.0005

0.9985

1 0.998

Key Osculatory Adaptive rational blending kernels when w=0.5

0.9975

0.997 1.5

0.9995

2

2.5 amplitude factor

3

3.5

0.999 1.5

2

(a)

2.5 amplitude factor

3

3.5

(b)

Fig. 6. PSNR comparisons. We here divide all psnr values with corresponding psnrs of Key’s interpolation. (a) is a comparison with two parent kernels, (b) is a comparison between different values of parameter w.

(a) bicubic interpolation

(b) oMoms’ interpolation

(c) Key’s interpolation

(d) Osculatory adaptive rational interpolation

(e) Blending kernel’s interpolation Fig. 7.

Interpolation results of the ”A”-like image.

R EFERENCES [1] P. Thvenaz, T. Blu, and M. Unser, Image Interpolation and Resampling, Handbook of medical imaging. Orlando, FL, USA: Academic Press, Inc., 2000. [2] D. F. Watson, Contouring: A Guide to the Analysis and Display of Spatial Data. New York: Pergamon Press, 1992.

IEEE SIGNAL PROCESSING LETTER, VOL. , NO. , 2008

6

(a) bicubic interpolation

(b) oMoms’ interpolation

(c) Key’s interpolation

(d) Osculatory adaptive rational interpolation

(e) Blending kernel’s interpolation Fig. 8.

Interpolation results of the ”eye” photo.

[3] M. R. Smith and S. T. Nichols, “Efficient algorithms for generating interpolated (zoomed) mr images,” Magnetic Resonance in Medicine, vol. 1, pp. 156–171, 1988. [4] M. Unser, A. Aldroubi, and M. Eden, “Enlargement or reduction of digital images with minimum loss of information,” IEEE Transactions on Image Processing, vol. 4, pp. 247–258, March 1995. [5] T. M. Lehmann, C. Gonner, and K. Spitzer, “Survey: Interpolation methods in medical image processing,” IEEE Transaction on medical imaging, vol. 18, pp. 1049–1075, November 1999. [6] H. Hou and H. Andrew, “Cubic splines for image interpolation and digital filtering,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 26, pp. 508–517, 1978. [7] K. Jensen and D. Anastassiou, “Subpixel edge localization and the interpolation of still images,” IEEE Transactions on Image Processing, vol. 4, pp. 285–295, 1995. [8] W. Y. V. Leung, P. J. Bones, and R. G. Lane, “Statistical interpolation of sampled images,” Optical Engineering, vol. 40, pp. 547–553, 2001. [9] X. Li and M. Orchard, “New edge directed interpolation,” Proc. IEEE Int. Conf. Image Processing, vol. 2, pp. 311–314, 2000. [10] X. Yu, B. Morse, and T. Sederberg, “Image reconstruction using data-dependent triangulation,” IEEE Computer Graphics and Applications, vol. 21, no. 3, pp. 62–68, 2001. [11] D. Su and P. Willis, “Image interpolation by pixel level data-dependent triangulation,” Computer Graphics Forum, vol. 23, no. 2, pp. 189–202, 2004. [12] M. Hu and J. q. Tan, “Adaptive osculatory rational interpolation for image processing,” Journal of Computational and Applied Mathematics, vol. 195, pp. 46–53, 2006. [13] S. H. Hong, R. H. Park, S. j. Yang, and J. Y. Kim, “Image interpolation using interpolative classified vector quantization,” Image Vis. Comput., vol. doi, p. 10.1016, 2007. [14] V. Caselles, J. M. Morel, and C. Sbert, “An axiomatic approach to image interpolation,” IEEE Transactions on Image Processing, vol. 7, pp. 376–386, 1998. [15] E. Meijering, K. J. Zuiderveld, and M. A. Viergever, “Image reconstruction by convolution with symmetrical piecewise nth order polynomial kernels,” IEEE Transactions on Image Processing, vol. 8, pp. 192–201, 1999. [16] H.-J. Wenz, “Interpolation of curve data by blended generalized circles,” Computer Aided Geometric Design. [17] M. Szilvasi-Nagy and T. Vendel, “Generating curves and swept surfaces by blended circles,” Computer Aided Geometric Design. [18] T. Blu, P. Thvenaz, and M. Unser, “Minimum support interpolators with optimum approximation properties,” in Proc. IEEE International Conference on Image Processing, Chicago, Illinois, U.S.A.,. [19] R. G. Keys, “Cubic convolution interpolation for digital image processing,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. ASSP, pp. 1153–1160, 1981.