IJRIT International Journal Of Research In Information Technology, Volume 2, Issue 5, May 2014, Pg: 571-576

International Journal of Research in Information Technology (IJRIT)

www.ijrit.com

ISSN 2001-5569

Improving convergence time of BPNN Image Compression / Decompression System Dr. V.P. Vishwakarma1, Sanjay Bhardwaj2 1

Associate Professor, U.S.I.C.T, G.G.S.I.P. University New Delhi, Delhi, India [email protected] 2 Student, U.S.I.C.T, G.G.S.I.P. University New Delhi, Delhi, India [email protected]

Abstract Image data comprises large size into memory. The transport of image data via communication path is a costly process. With image compression, we can reduce the number of bits in transmission. This helps in reducing the amount of data transfer and also helps in reducing the cost of transmission. In this paper, a three layered Backpropagation neural network (BPNN) was designed for constructing image compression and decompression system. The BPNN algorithm was used for training the designed network. We used the modified algorithm for training to reduce the training time. The different parameters and architecture were used to improve the MBP for training. At the end, we compared the results of MBP and BP. The performance of the system was compared using results like Compression Ratio (CR), Signal to noise ratio (SNR), Peak Signal to noise ratio (PSNR) and Normalized Mean Square Error (NMSE).

Key Words: Artificial Neural Network (ANN), Backpropagation Neural Network (BPNN), Backpropagation algorithm (BPA), Modified Backpropagation (MBP),Compression Ratio (CR),Signal to noise ratio (SNR),Peak signal to noise ratio(PSNR),Normalized Mean Square Error (NMSE), Mean Square Error (MSE)

1. Introduction Neural networks are inherent adaptive systems and are useful for managing nonstationaries in image data. ANNs are being extensively used for image compression problems as they are superior to traditional methods. ANNs are most suitable for image compression because of the ability to preprocess input patterns to produce patterns with smaller number components. The compressed information contains the complete information from the external environment. The artificial neural networks not only provides adequate compression rate but also provide security of data. [4] ANNs are comprised of various interconnected neurons that are connected together via weights. [5]. The BPNN is currently the most widely used architecture. The backpropagation algorithm (BP) is most commonly used in the training of the BPNN. We can apply BP to any BPNN which uses activation function and supervised learning. [6] The compression can be achieved by using backpropagation (BP) in the ANNs by training the network in a way such that weights scale the input vector of N-dimension into a narrow channel of P-dimension, (where P < N) and produce the output value of N-dimension. [1, 2] The main drawback of BPNN architecture for image compression is slow convergence. Many previous studies focused on improving the speed of convergence, but all ended in large convergence time. [3, 7, 9] Therefore in this paper we used the modified backpropagation algorithm (MBP) to train the network to reduce the convergence time. Dr. V.P. Vishwakarma,IJRIT

571

IJRIT International Journal Of Research In Information Technology, Volume 2, Issue 5, May 2014, Pg: 571-576

The BPNN architecture was designed on the basis of previous research. [8] Also, the processes for image normalization, segmentation and preparation of BPNN training set are same as described in the previous research. [10] 2.

Program Steps for BP

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.

Initialize learning rate, weights and threshold value. Initialize iterations to zero. Randomly select the training image array. Prepare non-overlapping blocks from the array. Rasterize the blocks of image array Normalize the resultant array i.e convert the coordinates [0-255] to coordinates [-1 - +1] Get one column vector from the resultant array and feed it to the input neurons. The target vector is same as the input vector, so initialize the input vector to target vector Compute the output of hidden layer Compute the output of the output layer Compute the error as error = target value – output value Sum the square of error for one image array Compute the local gradient of output layer units and then update the weights between output and hidden layer units. Compute the local gradients of the hidden layer units and update the weights between the hidden unit and input layer units. If there are more column vectors in the resultant array go to step 5 After processing one image array, compute the MSE and check if MSE is less than or equal to the threshold value If MSE < = threshold value then stop otherwise go to step 2

13. 14. 15. 16.

3. Program Steps for MBP The MBP algorithm is similar to BP algorithm. The only difference between these two is the rule for calculating the local gradient of the output layer units. [8] The rule for computing local gradient is modified as follows. [10] δj = β(tj – Oj ) + ( 1 – β)tanh(tj – Oj )

(1)

Where β is calculated by equation [10] β = exp (-µ / E2)

(2)

The value of β reduces from one to zero during the training

4. Implementation of MBP In this paper, the programs to implement MBP and BP were built using MATLAB 1. 2. 3. 4. 5. 6. 7.

Initialize β to 1 and initialize the value of µ in the range [ 0.1 to 10] Use hyperbolic tangent function for hidden layer and output layer neurons Normalize the image array by converting the coordinates [ 0-255] to the coordinates [-1 - +1] Initialize the value of learning rate parameter η in the range [0.01 – 0.1] Initialize momentum factor to zero Use the modified rule of Eq.(1) for calculating the local gradients of the output layer neurons.[10] Compute the value of β in each iteration

Dr. V.P. Vishwakarma,IJRIT

572

IJRIT International Journal Of Research In Information Technology, Volume 2, Issue 5, May 2014, Pg: 571-576

5. Compression / Decompression Process The compression process of BPNN is as follows: 1. 2. 3. 4. 5. 6.

Normalize the image array by converting it from [0-255] to [-1 - +1] Divide the image array into non-overlapping blocks Convert each block into one dimensional vector Feed this vector to the input layer Calculate the output of the hidden layer units Store the output of the hidden layer units into an array.

The Decompression process of BPNN is as follows: 1. 2. 3. 4.

Compute the outputs of the output layer neurons using output of the hidden layer neurons and weight matrix Derasterize the outputs of the output layer Convert each vector of the array into non-overlapping blocks Return the blocks to their original locations

6. Results We compared the MBP and BP on the basis of number of iterations to achieve the same value of MSE. We used both algorithms to train the same BPNN with same weights and the same threshold value (0.005). The following sections show the results of changing the values of η and µ and Nh (no. of hidden layer neurons) on the number of iterations. 6.1 Learning Rate vs Number of Iterations

Algorithm BP(η=0.01) MBP(η=0.01,µ=0.1 BP(η=0.02) MBP(η=0.02,µ=0.1

Iterations 98304 27648 98304 27648

Algorithm BP(η=0.01) MBP(η=0.01,µ=0.1 BP(η=0.02) MBP(η=0.02,µ=0.1

Iterations 21504 12288 24576 12288

Table 1 (Nh=8) SNR(dB) 41.53 45.54 41.50 48.98 Table 2 (Nh=16) SNR(dB) 45.16 40.13 45.11 42.19

PSNR(dB) 45.37 43.20 45.82 41.14

NMSE 1.9009 3.1342 1.7129 5.0336

PSNR(dB) 42.91 46.15 43.01 45.01

NMSE 3.3527 1.5880 3.2765 2.0649

PSNR(dB) 44.65 42.25 42.28 42.17 42.01

NMSE 2.2428 3.9001 3.8705 3.9683 4.1236

6.2 µ vs number of iterations, Nh = 8 Algorithm BP(η=0.01) MBP(η=0.01,µ=2) MBP(η=0.01,µ=4) MBP(η=0.01,µ=6) MBP(η=0.01,µ=8)

Dr. V.P. Vishwakarma,IJRIT

Iterations 98304 15360 9216 6144 6144

Table 3 (Nh=8) SNR(dB) 42.63 46.88 46.86 47.34 47.74

573

IJRIT International Journal Of Research In Information Technology, Volume 2, Issue 5, May 2014, Pg: 571-576

6.3 Effectiveness of Hidden layer neurons on CR, SNR,PSNR and NMSE (η = 0.01, µ = 4) Trained Image or not trained image (Nh=8) Trained Image Not Trained

SNR(dB)

PSNR(dB)

47.39 46.68

42.03 42.34

Trained Image or not trained image (Nh=16) Trained Image Not Trained

SNR(dB)

PSNR(dB)

45.50 45.17

42.94 43.00

Trained Image or not trained image (Nh=32) Trained Image Not Trained

SNR(dB)

PSNR(dB)

43.63 44.49

43.85 43.27

Dr. V.P. Vishwakarma,IJRIT

Table 4 NMSE

4.0996 3.8208 Table 5 NMSE

CR

Image Name

8:1 8:1

Lena.ppm Car.jpg

CR

Image Name

3.3262 3.2831 Table 6 NMSE

4:1 4:1

Lena.ppm Car.jpg

CR

Image Name

2.6991 3.0803

2:1 2:1

Lena.ppm Car.jpg

574

IJRIT International Journal Of Research In Information Technology, Volume 2, Issue 5, May 2014, Pg: 571-576

Fig. 1 Source Images for BPNN training

Fig. 2 Decoded Image (Trained Image)

Fig. 3 Decoded Image (untrained Image)

Fig. 4 Source Image (car.jpg)

7. Conclusion All programs were developed on single processor using MATLAB. The methods we used to speed up the training time are limited only to few patterns. Performance of BPNN was tested on various images of size 256 X 256 using blocks of 8 x 8 and 16 x 16. It was also tested by varying the size of hidden layer. The values of CR, SNR, PSNR and NMSE were calculated in different conditions and compared for both BP and MBP. From the observations we can conclude that the performance of the BPNN can be enhanced by making changes in the structure itself. We also observed that the ability of network to compress the untrained image is not same as for trained images. Dr. V.P. Vishwakarma,IJRIT

575

IJRIT International Journal Of Research In Information Technology, Volume 2, Issue 5, May 2014, Pg: 571-576

References [1] S. Marsi, G. Ramponi and G.L. Sicuranza, “Improved neural structures for image compression”,Proc IEEE Intl Conf on Acoustic Speech and signal Processing, 1991,pp 281-284 [2] Ian Vilovic, “An Experience in image Compression using Neural Networks”, 48th International symposium ELMAR, 2006,pp. 95-98 [3] V.V. Joseph Rajapandian and N. Gunaseeli. “Modified Standard Backpropagation Algorithm with Optimum Initialization for Feedforward Neural Networks”, International Journal of Imaging Science and Engineering (IJISE), vol.1, no.3, july 2007, GA, USA, ISSN: 1934-9955 [4] Simon Haykin, “Neural Networks and Learning Machines”, PHI Learning Private Limited, New Delhi, 2012 [5] K.P. Lippmann, “An Introduction to Computing with Neural Nets”,IEEE ASSP Magazine, vol 4, no.2, April 1987,pp4-22. [6] P.D. Wasserman. Neural Computing: Theory and Practice. Van Nostrand Reinhold Co. New York, NY, USA, 1989, ISBN, 0-442. [7] T. Xianghong and L. Yang. “An Image Compression Algorithm Based on Classified Blocks with BP Neural Netwworks”, Proc. Of the international conference on computer science and software engineering. IEEE Computer Society, Wuhan Hubei, Vol. 4, Dec 2008, pp. 819-822, ISBN: 978-0-7695-3336-0,DOI: 10.1109/CSSE. 2008.1357. [8] O.N.A Al-Allaf. “Fast Back Propagation Neural Network Algorithm for reducing Convergence Time of BPNN Image Compression” Proceedings of 5th International Conference on IT and Multimedia at UNITEN(ICIMU 2011) Malaysia [9] Fatima B. Ibrahim. “Image Compression using Multilayer Feed Forward Artificial Neural Network and DCT”, Journal of Applied Sciences Research, Vol.6 no.10 2010,pp.1554-1560, INSInet Publication. [10] B.S. Mahmood and O.N. Al-Allaf, “ Fast Backpropagation Neural Network for VQ-Image Compression”, Raf. J. of Comp. Sc. And Math’s, vol.1, no.1, 2004

Dr. V.P. Vishwakarma,IJRIT

576