Signal Processing: Image Communication 26 (2011) 427–437

Contents lists available at ScienceDirect

Signal Processing: Image Communication journal homepage: www.elsevier.com/locate/image

An integrated visual saliency-based watermarking approach for synchronous image authentication and copyright protection Lihua Tian n, Nanning Zheng, Jianru Xue, Ce Li, Xiaofeng Wang Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an 710049, China

a r t i c l e i n f o

abstract

Article history: Received 24 June 2010 Accepted 2 June 2011 Available online 13 June 2011

This paper proposes an integrated visual saliency-based watermarking approach, which can be used for both synchronous image authentication and copyright protection. Firstly, regions of interest (ROIs), which are not in a fixed size and can present the most important information of one image, would be extracted automatically using the protoobject based saliency attention model. Secondly, to resist common signal processing attacks, for each ROI, an improved quantization method is employed to embed the copyright information into its DCT coefficients. Finally, the edge map of one ROI is chosen as the fragile watermark, and is then embedded into the DWT domain of the watermarked image to further resist the tampering attacks. Using ROI-based visual saliency as a bridge, this proposed method can achieve image authentication and copyright protection synchronously, and it can also preserve much more robust information. Experimental results on standard benchmark demonstrate that compared with the state-of-the-art watermarking scheme, the proposed method is more robust to white noise, filtering and JPEG compression attacks. Furthermore, it also shows that the proposed method can effectively detect tamper and locate forgery. & 2011 Elsevier B.V. All rights reserved.

Keywords: Watermarking Image authentication Visual saliency Tamper detection and localization

1. Introduction Over the past few years, digital watermarking has attracted much attention from researchers, since it protects the copyright or verifies the integrality of media by embedding a digital signal into the media. In the practical application, media data may be processed by many operations such as low pass filtering, cropping, scaling and compression in the usage and transmission. Thus the embedded watermark should be robust against these possible attacks. Cox et al. [8] defined the robustness as the ‘‘ability to detect the watermark after common signal processing operations’’. On the other hand, besides the

n Corresponding author. Tel.: þ86 29 82668802x8016; fax: þ 86 29 83237910. E-mail addresses: [email protected] (L. Tian), [email protected] (N. Zheng), [email protected] (J. Xue), [email protected] (C. Li), [email protected] (X. Wang).

0923-5965/$ - see front matter & 2011 Elsevier B.V. All rights reserved. doi:10.1016/j.image.2011.06.001

above common operations, malicious attacks and forgery exist. In this case, the embedded watermark should also be sensitive to the tampering. Many watermarking schemes have been proposed. They could be divided into two groups—robust schemes and fragile schemes. Robust watermarking schemes [1–3] show robustness for image processing operations, while fragile watermarking schemes [4–7] could be fragile to image malicious tamper. However, whether they are robust or fragile, most of watermarking methods are based on transform domain to implement the watermark embedding and extraction. Many research efforts have been made in the discrete cosine transform (DCT), discrete wavelet transform (DWT) and discrete Fourier transform (DFT). Cox et al. [1] used spread spectrum techniques to embed the watermark in the DCT domain. Langelaar and Biemond [2] proposed an algorithm to embed a bit sequence in the digital image by selective removing DCT coefficients but not modifying DCT coefficients. In order to

428

L. Tian et al. / Signal Processing: Image Communication 26 (2011) 427–437

maintain the high fidelity of the image, Jiang et al. [3] presented a blind watermarking scheme in the DCT domain, which exploits the characteristics of human visual system (HVS) to generate high visual quality watermarked images. Some fragile watermarking schemes for the authentication of JPEG images were proposed in [4–7]. In some applications wavelet-based watermarking schemes show superiority over DCT-based approaches. A multiresolution watermarking technique for watermarking video and images was presented by Zhu et al. [9]. Kaewkamnerd and Rao [10,11] further improved this technique by adding the HVS factor. Zhao et al. [12] presented a dual domain watermarking technique for image authentication and image compression. In [13], multiple watermarks were embedded into both low and high frequency bands of DWT. Some researchers incorporated mathematical tools such as singular value decomposition (SVD) [14] and classifier [15] into the watermarking methods to increase the robustness. Wavelet-based fragile watermarking scheme for image authentication was presented in [16,17]. Furthermore, there are also some scholars using DFT for watermarking [18,19]. However, these techniques neglect the characteristics of image regions. In reality, people are attracted by some regions in one image, which present its most important information and thus attract more attention from readers. Some researchers have realized the importance of the interesting region of images and have proposed some approaches to protect the important regions of image. Fan et al. [20] presented a new ROI-based watermarking method to protect the copyrights of images, which embedded the watermark into the host image based on the characteristics of the ROIs. However, this scheme integrated the watermarking process into the JPEG 2000 compression procedure, which is not suitable for images with JPEG format. Ni and Ruan [21] proposed a method based on fractal dimension to protect the information of ROI, but the ROI in [21] is determined by the user and it needs people’s interaction. Mohanty et al. [22,23] described a color image watermarking method on DCT domain; however, extracting the watermark needed the original image. In [24], a ROI-based medical image watermarking method using DWT was put forward. It embedded the watermark with ROI information into non-ROI area. Nevertheless, this method can only resist JPEG compression attacks and it is a non-blind watermark scheme. Chu et al. [25] proposed an image watermarking authentication method based on ROIs, which embedded watermark in the interior of ROI, which only performs well for authenticating the ROI of an image. Although the importance of ROI in image watermarking has been realized, most of the current watermarking methods cannot obtain the ROI automatically. For example, the ROI of image was determined by users in [20,21]. In [24], the central area of image was selected directly as the ROI. The Poisson matting [26] was adopted to extract foreground as the ROI of the image in [25], but the Poisson matting method still needs user to specify the boundary in an interactive manner. In [22,23], the ROI of image was determined by several influencing characteristics of HVS such as intensity, contrast, edginess, texture and location.

To determine the ROI, the host image was divided into 8  8 blocks and a sliding square window containing NB of pffiffiffiffiffiffi pffiffiffiffiffiffi such blocks (the ROI of size NB  NB blocks) was considered. The sliding window moved across the image and computed the influencing characteristics of HVS mentioned above to locate the ROI. However the size of ROI in the paper was determined by the parameter NB. Once NB was determined, the size of sub-image was the same for different images, which is inappropriate because the most perceptually important sub-image may have different sizes in different images. Furthermore in daily life, an image may suffer the common signal processing operations as well as the tampering attack. In this case, the robust watermarking method cannot detect the tamper while the fragile watermarking is too sensitive to the common signal processing. As far as we know, there are few methods, which can realize the two functions (authentication and copyright protection) at the same time. Our purpose is to solve this problem, using the ROI as the technical bridge to achieve image authentication and copyright protection synchronously. The watermark would be embedded into these ROIs to protect copyright so long as the interesting regions are not removed or damaged. If the interesting region or the remainder regions of image is removed or tampered, the other watermark should be embedded to ensure the integrality. In this paper, we propose an integrated visual saliencybased watermarking approach for synchronous image authentication and copyright protection. To solve the issue of ROI selection mentioned above, we extract several salient regions from an image based on a protoobject visual attention model. The regions can be extracted automatically and be in different sizes for different images, and the extracted regions are regarded as ROIs of the images. Then the copyright information is embedded into ROIs as the robust watermark, and the edge map of the most salient ROI is embedded into the watermarked image again as the fragile watermark. Hence, the proposed method is not only robust to attacks but also fragile to tampering. This new proposed algorithm has been tested and the experimental results demonstrate its efficiency to solve the image protection and authentication problems synchronously. This paper is organized as follows. In Section 2, an overview of the proposed integrated watermarking approach is presented. In Section 3, the ROI extracting algorithm using proto-object based visual attention model is stated. Following that is Sections 4 and 5 in which the ways to embed and extract the robust and fragile watermark are given. Experimental results are shown in Section 6 and a conclusion is finally drawn in Section 7. 2. The proposed integrated watermarking scheme In this paper, an integrated visual saliency-based watermarking approach is proposed, which can prevent attacks and detect tampering synchronously. Generally speaking, an image may experience some signal processing operations as well as tampering attack in daily life.

L. Tian et al. / Signal Processing: Image Communication 26 (2011) 427–437

Original Image

Proto-Object Region

429

Edge of ROI as Fragile Watermark

ROI of Image

XJTU Watermarked Image

Watermark Watermark Insertion Watermarked Image

Watermark Insertion LL Subband of Watermarked Image

Fig. 1. An overview of the proposed integrated visual saliency-based method.

The existing robust watermarking method or fragile watermarking method can hardly resist the dual attacks at the same time. Fortunately, it is known that in most images there are observational focuses, which could easily attract people’s great attention. Therefore watermark can be embedded into these regions to protect copyright as long as the interesting regions are not removed or damaged. Furthermore, if some interesting regions or the remainder region of image were removed or tampered, the whole image should be protected to ensure the integrality. So the fragile watermark needs to be embedded into the image. In this paper, we embed double watermarks into image using interesting regions to achieve the dual-purpose of copyright protection and integrity authentication. In order to obtain the interesting regions in image automatically, a proto-object visual attention model based on multi-scale image features is used. Then the copyright information is embedded into several extracted ROIs and the edge map of the most salient ROI is embedded into the watermarked image. Fig. 1 provides a schematic overview of the proposed integrated visual saliency-based watermarking method. The overall algorithm can be summarized as follows: Stage 1. Gives an image, and extracts salient regions automatically based on the proto-object based visual attention model. These regions are not fixed and can present the most important information of the image. Stage 2. Embed the copyright information into the DCT coefficients of each ROI extracted to prevent common signal processing attacks. Stage 3. Decompose the watermarked image by wavelet transform, and then embed the edge map of one of the ROIs as the fragile watermark into the LL sub-band of the watermarked image to detect and locate tampering.

3. ROI extraction Generally, ROIs present the most important information of the image. How to extract ROIs is very important for the proposed dual watermark embedding. Most of the previous watermarking schemes based on ROIs cannot

obtain the ROIs automatically. For example, the ROIs are determined by users in some methods. While in some other methods, the center of image is even selected directly as the ROI. Most recently, Mohanty et.al [22,23] determined the ROI by several influencing characteristics of HVS, which is better than the other methods. But in those papers the size of sub-image is determined by the parameter NB (number of blocks). Once NB is determined, the size of sub-image will be the same for different images. In fact, the size of ROI is generally different for different images. In order to obtain the ROI automatically and properly, we adopt the thought in [27], in which the bottom-up saliency information is obtained from the image to select the most potential interesting points in the image. Highly salient points in the image can be selected in this method by combining multi-scale image features automatically. And the size of ROIs is determined by the image itself. The procedure of the method is summarized below. Firstly, salient points are identified by computing the center–surround features. It decomposes the input image into seven channels: image intensity contrast, red/green and blue/yellow double opponent channels and four orientation contrasts. From these channels, the feature maps are yielded by center–surround operation, which denotes the across-scale difference between two maps at the center and the surround levels of the respective feature pyramids. The feature maps are normalized and summed over the center–surround combinations using across-scale addition to yield ‘‘conspicuity maps’’ for the general features intensity, color and orientation. Then three conspicuity maps are normalized and summed into the final saliency map. Although Itti et al.’s [27] model successfully identifies the saliency map of the image; it cannot explicitly explain the extent of the attended object or object part at this location. In order to obtain the potential object in the image, Walther and Koch’s method [28] is adopted here to estimate the size and shape of the region based on the maps and salient locations computed so far. From the conspicuity maps, one that contributes the most to the activity at the most salient location will be found, and the feature maps that give rise to the conspicuity map

430

L. Tian et al. / Signal Processing: Image Communication 26 (2011) 427–437

will be examined; thus the one that contributes most to its activity at the winning location would be determined. The ‘‘winning’’ feature map is segmented around (xw, yw) by the method of the 4-connected neighborhood of active pixels. For this operation, a binary version of the map (B) is obtained by threshold with 1/10 of its pixel value at the attended location. Then the proto-object is estimated by the 4-connected neighborhood of active pixels in B. We can obtain several potential salient regions in the image automatically after the operations mentioned above. However, the area we got is not a region in regular size. In this paper, for the convenience of operation in the following steps, we preprocessed ROIs to be regular region with the same aspect ratio with the original image. 4. Robust watermark embedding and extracting In this section, in order to resist the common signal processing attacks, a new robust watermark algorithm based on ROI is proposed. 4.1. Watermark embedding After obtaining the ROIs in the image, the copyright information is embedded into these regions. In order to increase the security of watermarking, the watermark data are preprocessed by operations such as encryption or permutation. Suppose W and K are the original watermark and the key respectively, and G is the function to generate the new watermark w, and then the function works: wl ¼ GðW,KÞ,

l ¼ 1,. . .,n

ð1Þ

Here Logistic mapping can be selected as G function to permute the original watermark. In order to realize the semi-blind watermark detection, a quantization method is employed to embed the watermark. The steps of watermark embedding are as follows: Step 1. Divide each ROI into 8  8 sub blocks, Rm,n, where m, n denote the position of the block in ROI. Then obtain DCT coefficients xRm,n(i,j) for the individual blocks of ROI. Step 2. After DCT transformation, the energy of ROI concentrates in several low frequency DCT coefficients and high frequency DCT coefficients reflect the detail information of the ROI. In order to improve the robustness of the watermarking, we select the lowmiddle frequency DCT coefficients as the candidate embedding position. Here the coefficients xRm,n(i,j), iA{2.3.4}, jA{2.3.4} are selected. Step 3. For the selected coefficient xRm,n(i,j), according to the watermark wl, the watermarked coefficient is defined as if wl ¼ 1

   xRm,n ði,jÞ if mod ,2 ¼ 0 D ( xRm,n ði,jÞ þ D, x^ Rm,n ði,jÞ ¼ xRm,n ði,jÞD,

if ðxRm,n ði,jÞ Z0Þ if ðxRm,n ði,jÞ o0Þ

ð2Þ

if wl ¼ 0    xRm,n ði,jÞ ,2 ¼ 7 1 if mod D ( xRm,n ði,jÞ þ D, if ðxRm,n ði,jÞ Z0Þ x^ Rm,n ði,jÞ ¼ xRm,n ði,jÞD, if ðxRm,n ði,jÞ o0Þ

ð3Þ

D is the quantization step, which determines the robust degree of the embedded watermark. It means that the change of the coefficients in the range of D is accepted. The larger the step D is, the more robust the watermark is. However, the quality of image will be decreased if the D is too large. Therefore obviously there is a tradeoff between the robustness and the quality. The capacity of robust watermark is affected by the ROIs since robust watermark is embedded into the ROIs. The ROI in a larger size will have a stronger watermark capacity. 4.2. Watermark extracting The host image is not needed to extract the embedded watermark. For watermarked image, the watermark can be extracted directly in DCT domain from the ROIs. If the watermarked image is not attacked, the ROIs can still be extracted using the proto-object based visual attention model, otherwise the reserved information is needed. For the watermarked image, the watermark is extracted using the following steps: Step 1. Obtain the ROIs of the image according to the reserved information; Step 2. Divide each ROI into 8  8 sub blocks and obtain the DCT coefficients x^ Rm,n ði,jÞ. For each block, the watermark detection is defined as j ^ k  8 xRm,n ði,jÞ > ,2 ¼ 0 < 0, if mod D j ^ k  ð4Þ w0I ¼ ði,jÞ x R m,n > : 1, if mod ,2 ¼ 1 D where iA{2.3.4}, jA{2.3.4}. Step 3. Decrypt the extracted watermark with the correct key and obtain the original watermark. The visual aspect and the correlation detection algorithm are used to determine the robust of the extracted watermark. The correlation is defined as Pn i ¼ 1 ðwi wÞðwei we Þ g¼ P ð5Þ P 1=2 n f i ¼ 1 ðwi wÞ2 ni¼ 1 ðwei we Þ2 g where w and we are the original and extracted watermarks, and w and we , are their pixel mean values, respectively. The subscript i of w or we denotes the index of an individual pixel of the corresponding watermark. It is a semi-blind watermark detection method because the detection process does not need the original image but needs other extra information. Here only a very small amount of extra information is required. And more than one ROI may be obtained from the image. The robustness of watermark can be increased by embedding the same watermark into multiple ROIs. We can obtain

L. Tian et al. / Signal Processing: Image Communication 26 (2011) 427–437

the final watermark according to these watermarks extracted from the multiple ROIs of the image. 5. Fragile watermark embedding and extracting In this section, the fragile watermark method based on ROI is introduced to detect and locate the tampering in the image. 5.1. Watermark embedding After embedding the copyright information into the ROIs of image, we embed the feature of ROI as the fragile watermark into the watermarked image. Generally speaking, the ROI is often the region with obvious profile. So we select the edge map as the feature of ROI and embed it into the LL wavelet sub-band of the image. In order to completely embed the fragile watermark, the size of LL sub-band should be larger than or at least equal to that of the ROIs. Moreover, in order to locate the tampering accurately, the fragile watermark should be proportional to the original image. Then we can locate the position of tampering in the image by scaling the fragile watermark to the original image size. So in this study, the ROIs are preprocessed to be in the same aspect ratio with the original image. The level, k, of wavelet transform should be determined according to the size of ROI so as to make the LLk sub-band in the same size with the ROI. In our study, Canny operator is employed to detect the edge of ROI. And for color image, the detected edge of ROI is embedded into each color channel of the image. Quantization method is adopted too to embed the watermark. The steps of watermark embedding are as follows: Step 1. Choose one of the ROIs for the edge detection, and the extracted edge map of that ROI is regarded as the fragile watermark wedge(i,j) to be embedded; Step 2. Determine the level k of wavelet transform according to the size of ROI, make LLk sub-band after transformation in the same size with ROI, and decompose the watermarked image by wavelet transform with level k; Step 3. Detect the edge map of ROI and embed it into the coefficients vi,j in LLk sub-band of the image using the formula: if wedge(i,j) ¼0 8 if vi,j mod2 ¼ 0 v > < i,j ð6Þ v^ i,j ¼ vi,j þ 1 if vi,j mod2 ¼ 1 > : v 1 if v mod2 ¼ 1 i,j i,j if wedge(i,j) ¼1 8 if vi,j mod2 ¼ 71 v > < i,j v^ i,j ¼ vi,j þ 1 if ðvi,j mod2 ¼ 0Þ and ðvi,j 4 0Þ > : v 1 if ðv mod2 ¼ 0Þ and ðv o 0Þ i,j i,j i,j

ð7Þ

After embedding, the maximum change of the selected coefficients in LLk sub-band is only 1. When the image is altered by some operations, the alteration can be

431

detected. The capacity of fragile watermark is affected by ROIs since the edge of ROI is embedded into the LL subband of DWT as fragile watermark. The ROI in a larger size will have stronger watermark capacity. For each watermarked image, the key K, the quantization step, the level k of DWT, the location and size of ROIs (as the fragile watermark) and the image size should all be reserved for watermark extraction. The key K for Logistic mapping is a decimal number between 0 and 1. The quantization step is less than 100 in our method. If we assumed that the size of an image is no more than 10,000  10,000, and then the length of the message would be 31bytes at most. It is such a short message that it can be encrypted by a public key method such as RSA. After that the encrypted message can be saved and transmitted. 5.2. Watermark extracting For the watermarked image, the watermark is extracted in the following steps: Step 1. Obtain all the ROIs of the image and their edge information wedge(i,j); Step 2. Decompose the watermarked image by wavelet transform with level k; Step 3. Extract the watermark in the LLk sub-band of the watermarked image as follows: ( 0 if v^ i,j mod2 ¼ 0 w0edge ði,jÞ ¼ ð8Þ 1 if v^ i,j mod2 ¼ 7 1 Step 4. Compare the extracted watermark w0edge ði,jÞ with the edge detection result wedge(i,j) to confirm whether there is a forgery in the image. Furthermore, if there is tampering; the site of forgery can be located. The original image is not needed in this proposed method. Moreover, by comparing the extracted watermark and the detected edge of ROI, the tampered parts in the image can be detected and located. By embedding the feature of ROI as the fragile watermark into the watermarked image, the copyright protection and integrity verification can be realized simultaneously. 6. Experimental results To verify the effectiveness of our presented method, a series of experiments has been conducted with a set of standard test images. Furthermore, to prove the robustness of the proposed method, we compare it with the method in [23]. All experiments are implemented by Matlab 7.0, and tested on PC with Genuine Intel CPU, 1.6 GHz and 1 G RAM. 6.1. ROI extraction In this part, the automaticity and efficiency of our method for ROIs extraction are tested. With a set of standard test images, the proto-object based visual attention model is applied to extract several salient regions of each image. And these salient regions are preprocessed to

432

L. Tian et al. / Signal Processing: Image Communication 26 (2011) 427–437

be in the regular size (in the same aspect ratio with the original image). Some experimental results are shown in Fig. 2 and Table 1. Fig. 2 shows the results of proto-objects and several detected ROIs of image with the method mentioned in Section 3. The original images are shown in Fig. 2(a), (e), (i). Fig. 2(b), (f), (j) illustrate the proto-objects obtained from the original image by considering the visual features. Fig. 2(c), (d), (g), (h), (k), (l) and (m) are several extracted ROIs of the correlative images. From Fig. 2, we can see that the ROIs we obtained are indeed the saliency object in the image from the visual senses. In order to detect and locate the tamper in the whole image, the size of ROI should be in the same aspect ratio with the original image after the pre- processing. Then we can select a suitable level k of DWT to make LL sub-band have the same size with one of the ROIs. Table 1 shows the size of each image and its ROIs respectively as well as the ratio of ROI to image and the corresponding level k of DWT transform.

For example, two ROIs are extracted from the image shown in Fig. 2(a), and their sizes are both 128  96. The robust watermark is embedded into the two ROIs, and the edge of one ROI is embedded into the LL sub-band as the fragile watermark. 2 levels DWT should be selected to keep LL sub-band (128  96) in the same size with the ROI. 6.2. Watermark generation and embedding We implement the dual watermark embedding based on the detected ROIs. Some results of the double watermark, the watermarked image and the quality of the watermarked image are presented in Fig. 3 and Table 2. Fig. 3 describes the process of watermark generation and insertion. Fig. 3(a), (e) and (i) shows the original image. The robust watermarks are shown in Fig. 3(b), (f) and (j). It is embedded into the detected ROIs (shown in Fig. 2). Then the edge map of one suitable ROI is selected

Fig. 2. ROIs extraction. (a) original image, (b) proto-object, (c) ROI 1, (d) ROI 2, (e) original image, (f) proto-object, (g) ROI 1, (h) ROI 2, (i) original image, (j) proto-object, (k) ROI 1, (l) ROI 2 and (m) ROI 3.

Table 1 Size of ROIs and DWT level. Image

Image Size

ROI1 Size

ROI2 Size

ROI 3 Size

ROI1/Image

DWT level k

Itti.bmp Airplane.bmp Balloons.bmp

512  384 512  512 384  256

128  96 104  104 96  64

128  96 104  104 152  104

– – 96  64

1/16 1/16 1/16

2 2 2

L. Tian et al. / Signal Processing: Image Communication 26 (2011) 427–437

433

Fig. 3. Dual water mark and watermarked image. (a) original image, (b) robust watermark, (c) edge map of ROI, (d) watermarked image, (e) original image, (f) robust watermark, (g) edge map of ROI, (h) watermarked image, (i) original image, (j) robust watermark, (k) edge map of ROI and (l) watermarked image.

6.3. Watermark extracting

Table 2 Quality of watermarked Image (PSNR). Image

Itti.bmp

Airplane.bmp

Balloons.bmp

PSNR

59.65(R) 59.74(G) 59.69(B)

55.32(R) 55.29(G) 55.42(B)

42.29(R) 42.16(G) 42.47(B)

as the candidate fragile watermark. For example, we select ROI 1 for the first and the third test images, and ROI 2 for the second image in Fig. 2. The results of edge detection for the selected ROIs are shown in Fig. 3(c), (g) and (k). The edge map is embedded into the LLk (where the value of k is shown in Table 1’s DWT level) sub-band of the watermarked image as the fragile watermark. In order to show the edge map of ROI in a better way, the ROI edge maps are shown in inverted color. Fig. 3(d), (h) and (l) illustrates the watermarked images, from which we can see that the impact to the quality of image is very small. There is no visible distinction between them. Table 2 presents the difference between the original image and the watermarked image measured by peak signal to noise ratio (PSNR), the data in which shows the efficiency of the proposed method.

The watermark can be extracted from the watermarked image. If the watermarked image does not suffer any attacks, the ROIs still can be extracted exactly using the proto-object based visual attention model, because the watermark embedding process has little impact on the quality of watermarked image. If the watermarked image is attacked, the proto-objects obtained from the image will change since the underlying data has changed. Hence the reserved information is needed to obtain the ROIs and to extract the relevant watermark as mentioned in Section 5.1.

6.4. Robustness against image processing operation The following experiments are conducted to test our algorithm’s robustness and efficiency to protect the copyright of image. When the watermarked image is attacked by white noise, filter or compression operation, the watermark can be extracted according to the reserved data. The experiments for all the attacks provided in Stirmark are performed. The extracted watermarks are shown in Fig. 4. Furthermore, in order to prove the

434

L. Tian et al. / Signal Processing: Image Communication 26 (2011) 427–437

Fig. 4. Dual watermark extracted after common signal processing attacks. (a) No attack, (b) median filter, (c) white noise, (d) Gaussian blurred, (e) Jpeg (Q ¼60), (f) no attack, (g) median filter, (h) white noise, (i) Gaussian blurred and (j) Jpeg (Q¼ 60).

Table 3 Quality and correlation of extracted robust watermark after attacks. Image

No attack Median filter White noise Gaussian blurred Jpeg compression

Our method

Mohanty’s method [23]

Extracted watermark’s PSNR

Correlation g

Extracted watermark’s PSNR

Correlation g

36.08 28.63 28.50 26.57 26.18

0.9984 0.9287 0.9342 0.8354 0.8124

35.17 25.58 27.63 30.43 24.69

0.9947 0.8373 0.9286 0.9856 0.7930

robustness of our method, the compared results with Mohanty’s method [23] are given in Table 3. With the image of airplane as an example, Fig. 4 shows the visual results of the dual watermark extracted after common signal processing attacks. The proposed method is shown to be robust to white noise and median filter operation. In these cases, the watermark is clear in visual inspection, although it does not make very good performance for blur and JPEG compression attacks, which also shows the sensitivity of fragile watermark to the common signal processing. Results of quantitative analysis are summarized in Table 3, which indicate the correlation between the embedded robust watermark and the extracted watermark for corresponding attacks. In Table 3, we also compare our method with Mohanty’s method [23], which shows the advantage of our method over [23] when resisting the white noise, median filter and the JPEG compression attacks, but worse than [23] for Gaussian blur attack. 6.5. Sensitivity and tamper detection of fragile watermark In this part, the fragileness and efficiency to detect and locate the tampering of our method are tested. If an image is not attacked, the extracted fragile watermark is the edge of its ROI. When the watermarked image is tampered, the extracted fragile watermark will change. The tampered position can be detected and located by comparing the extracted fragile watermark and the edge of ROI in the watermarked image. The fragile watermark will also change when the watermarked image suffers the common

signal processing operations. The experiment results are shown in Figs. 5–7 to illustrate this phenomenon. Fig. 5 shows the visual results of the fragile watermark extracted after common signal processing attacks. It illustrates that our proposed method is sensitive to the common signal processing. Even by adding the white noise it would differ from the original fragile watermark. Fig. 6 shows the sensitivity of the fragile watermark to modification in ROIs. As shown in Fig. 6(b), (g) and (l), three images are tampered. The elephant balloon (one of the ROIs) in the top right corner of the image is erased from the image in the first row. The embedded fragile watermark is the edge of the leftmost balloon. The extracted edge and the extracted watermark are shown, respectively, in Fig. 6(c) and (d). We can see a tamper appears in the top right corner of the image. In the second row, the leftmost balloon (one of the ROIs) is replaced by the top middle balloon. And its edge is the embedded fragile watermark. The extracted edge and the extracted watermark are shown in Fig. 6(h) and (i), respectively. The extracted edge appears totally different from the extracted watermark. We can make a conclusion that the ROI is tampered. In the third row, the road sign (one of the ROIs) is covered with the surround trees and its edge is the embedded fragile watermark. The extracted edge and the extracted watermark are shown in Fig. 6(m) and (n), respectively. Therefore the ROI is also tampered because they are apparently different. The extracted robust watermarks are shown in Fig. 6(e), (j) and (o). The results reveal that the extracted robust watermark is still clear after the tampering attack. These experiments demonstrate that the proposed fragile watermark scheme is sensitive to the modification in the ROIs.

L. Tian et al. / Signal Processing: Image Communication 26 (2011) 427–437

435

Fig. 5. Sensitivity of fragile watermark to common signal processing. (a) No attack, (b) median filter, (c) white noise, (d) Gaussian blurred, (e) Jpeg(Q¼ 60), (f) no attack, (g) median filter, (h) white noise, (i) Gaussian blurred and (j) Jpeg(Q¼ 60).

Fig. 6. Sensitivity of fragile watermark to modification in ROIs. (a) Original image, (b) tempered image, (c) edge, (d) extracted watermark, (e) robust watermark, (f) original image, (g) tempered image, (h) edge, (i) extracted watermark, (j) robust watermark, (k) original image, (l) tempered image, (m) edge, (n) extracted watermark and (o) robust watermark.

The sensitivity of fragile watermark to modification in other regions of image is shown in Fig. 7. In the first row, an airplane is added into the top right corner of the image. The extracted edge and the extracted watermark are shown in Fig. 7(b) and (c), respectively. The difference between them is shown in Fig. 7(d). It can be seen that the tamper is in the top right corner of the image. In the second row, a red car is added on the road. From Fig. 3(f) and (g), a tamper can be found in the image. The location is in the lower left corner of the image as shown in Fig. 3(h). The extracted robust watermarks are shown in Fig. 7(e) and (j). The result reveals that the extracted robust watermark is still clear after the tampering attack; while the extracted fragile watermark has obvious

difference with the edge feature of ROI shown in Fig. 7 at the tampering location. We can also locate the position of tampering in the image with the proportional of the ROI and the whole image. From the fragile watermark shown in Fig. 7(c) and (h), we can find that tampering occurred in the top right corner in the first image and in the bottom left corner in the other image. The results illustrate that the proposed fragile watermark scheme is sensitive to the modification in the other region of the image, which means that the proposed method can realize the integrity verification of image. Fig. 8 shows the detection results when the watermarked image suffers the attack and tampering simultaneously. In this experiment, the tampered airplane image in

436

L. Tian et al. / Signal Processing: Image Communication 26 (2011) 427–437

Fig. 7. Sensitivity of fragile watermark to modification in the other region of image. (a) Tampered image, (b) edge, (c) extracted watermark, (d) difference, (e) robust watermark, (f) tampered image, (g) edge, (h) extracted watermark, (i) difference and (j) robust watermark. (For interpretation of the references to color in this figure, the reader is referred to the web version of this article.)

Fig. 8. Dual watermark extracted after attacks and tampering. (a) Only tampering, (b) tamperingþ noise, (c) tamperingþmedian filter, (d) tamperingþ blurred, (e) tamperingþjpeg, (f) only tampering, (g) tamperingþnoise, (h) tamperingþmedian filter, (i) tamperingþ blurred and (j) tamperingþjpeg.

Fig. 7(a) is selected as an example to suffer attacked. In this case, the extracted robust watermark can be identified, and the extracted fragile watermark indicates the tamper in the top right corner of the image. The experimental results explain that the proposed method can protect the copyright and detect the tampering synchronously. 7. Conclusion In this study, an integrated visual saliency-based watermarking method is presented to protect copyright and detect tampering synchronously for a given image. The ROIs of image are extracted automatically based on the visual features of the image. Then the robust watermark is embedded into the DCT coefficients of the ROIs, and the fragile watermark is embedded into the LLk subband of the watermarked image. Therefore, if the image suffers the double attacks simultaneously, the proposed method can protect copyright and detect tampering

synchronously. The experimental results demonstrate that the proposed algorithm is robust to the common signal processing attacks and sensitive to the tampering. The main features of the proposed method include the following aspects: (1) It can extract the ROIs from the image automatically for embedding the robust watermark, while the existing watermark schemes based on ROI cannot; (2) The embedded watermark is robust since it is embedded into several ROIs; (3) It makes the tampering localization easily with the fact that it embeds the edge map of ROI into LL wavelet sub-band of watermarked image; (4) It can protect the copyright and authenticate of the image synchronously. Though progress has been made in this paper, there is still a plenty of work remained to improve the proposed

L. Tian et al. / Signal Processing: Image Communication 26 (2011) 427–437

method. Our future work will focus on studying video watermarking algorithm to realize the copyright protection and authentication. Moreover, an attempt will also be made in the future to study the image authentication issue without embedding the watermarks.

Acknowledgment This work was supported in part by the National Natural Science Foundation of China under Grants 60875008, 61075007 and the National Basic Research Program of China under Grant No. 2010CB327902. References [1] I.J. Cox, J. Kilian, F.T. Leighton, T. Shamoon, Secure spread spectrum watermarking of images, audio and video, in: Proceedings of the IEEE International Conference on Image Processing, vol. 3, pp. 243–246, 1996. [2] R.L.G. Langelaar, J. Biemond, Watermarking by DCT coefficient removal: statistical approach to optimal parameter settings, in: Proceedings of the SPIE IS&T/SPIE’s Eleventh Annual Symposium on Electronic Imaging: Security and Watermarking of Multimedia Contents, vol. 3657, pp. 2–13, 1999. [3] Gangyi Jiang, Mei Yu, Shoudong Shi , Xiao Liu, Yong-Deak Kim, New blind image watermarking in DCT domain, in: Proceedings of the Sixth International Conference on Signal Processing, vol. 2, pp. 1580–1583, 2002. [4] Chang-Tsun Li, Digital Fragile Watermarking Scheme For Authentication Of JPEG Images, IEE Proceedings—Vision Image and Signal Processing 151 (2004) 460–466. [5] H.X. Wang, K. Ding, C.X. Liao, Chaotic watermarking scheme for authentication of JPEG images, in: Proceedings of the International Symposium on Biometrics and Security Technologies, pp. 1–4, 2008. [6] C.-Y. Lin, S.F. Chang, Semi-fragile watermarking for authenticating JPEG visual content, in: SPIE Proceedings Series: Security and Watermarking of Multimedia Contents, vol. 3971, pp. 140–151, 2000. [7] C.-H. Lin, T.-S. Su, W.-S. Hsieh, Semi-fragile Watermarking Scheme for Authentication of JPEG Images, Tamkang Journal of Science and Engineering 10 (1) (2007) 57–66. [8] I.J. Cox, M.L. Miller, J.A. Bloom, Digital Watermarking, Morgan Kaufmann Publisher, San Francisco, CA, USA, 2002. [9] W. Zhu, Z. Xiong, Y.Q. Zhang, Multiresolution watermarking for images and video, in: Proceedings of the IEEE Transactions on Circuit and System for Video Technology, vol. 9, no. 4, pp. 545–550, 1999. [10] N. Kaewkamnerd, K.R. Rao, Wavelet based image adaptive watermarking scheme, in: Proceedings of the IEE Electronics Letters, vol. 36, pp. 312–313, 2000. [11] N. Kaewkamnerd, K.R. Rao, Multiresolution based image adaptive watermarking scheme, in: Proceedings of the EUSIPCO, Tampere, Finland, Sept. 2000.

437

[12] Y. Zhao, P. Campisi, D. Kundur, Dual domain watermarking for authentication and compression of cultural heritage images, in: Proceedings of the IEEE Transactions on Image Processing, vol. 13, no. 3, pp. 430–448, 2004. [13] M.S. Raval, P.P. Rege, Discrete wavelet transform based multiple watermarking scheme, in: Proceedings of the Conference on Convergent Technologies for Asia-Pacific Region, TENCON, vol. 3, pp. 935–938, 2003. [14] E. Ganic, A.M. Eskicioglu, Robust digital watermarking: robust DWT-SVD domain image watermarking: embedding data in all frequencies, in: Proceedings of the Multimedia and Security Workshop on Multimedia and Security, pp. 166–174, 2004. [15] W.-H. Lin, Y.-R. Wang, S.J. Horng, A Wavelet-tree-based watermarking method using distance vector of binary cluster, Expert Systems with Applications 36 (6) (2009) 9869–9878. [16] H. He, J. Zhang, H.-M. Tai, A wavelet-based fragile watermarking scheme for secure image authentication, in: Proceedings of the Fifth International Workshop on Digital Watermarking. Lecture Notes in Computer Science, vol. 4283, pp. 422–432, 2006. [17] X. Zhang, S. Wang, Statistical fragile watermarking capable of locating individual tampered pixels,, IEEE Signal Processing Letters 14 (10) (2007) 727–731. [18] A. De Rosa, M. Barni, F. Bartolini, V. Cappellini, A. Piva, Optimum decoding of non-additive full frame DFT watermarks, in: Proceedings of the Third Workshop of Information Hiding, pp. 159–171, 1999. [19] M. Ramkumar, A.N. Akansu, A.A. Alatan, A robust data hiding scheme for digital images using DFT, in: Proceedings of the IEEE ICIP, vol. 2, pp. 211–215, 1999. [20] Y.C. Fan, A. Chiang, J.H. Shen, ROI-based watermarking scheme for JPEG 2000, Circuits, Systems, and Signal Processing 27 (5) (2008) 763–774. [21] R.R. Ni, Q.Q. Ruan, Region of interest watermarking based on fractal dimension, in: Proceedings of the Eighteenth International Conference on Pattern Recognition, vol. 3, pp. 934–937, 2006. [22] S.P. Mohanty, P. Guturu, E. Kougianos, N. Pati, A novel invisible color image watermarking scheme using image adaptive watermark creation and robust insertion-extraction, in: Proceedings of the Eighth IEEE International Symposium on Multimedia, pp. 153–160, 2006. [23] S.P. Mohanty, B.K. Bhargava, Invisible watermarking based on creation and robust insertion–extraction of image adaptive watermarks, ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP) 5 (2) (2008) 12:1–12:22. [24] H.-K. Lee, H.-J. Lee, K.K. Ryong, K.J.-K. Lee, ROI medical image watermarking using DWT and bit-plane, Asia-Pacific Conference on Communications, pp. 512–515, 2005. [25] Yiping Chu, Yin Zhang, Sanyuan Zhang, Xiuzi Ye, Region of interest fragile watermarking for image authentication, in: Proceedings of the First International Multi-Symposiums on Computer and Computational Sciences, vol. 1, pp. 726–731, 2006. [26] J. Sun, J. Jia, C. Tang, H. Shum, Poisson Matting, in: Proceeding of the ACM SIGGRAPH, pp. 315–321, 2004. [27] L. Itti, C. Koch, E. Niebur, A model of saliency-based visual attention for rapid scene analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (11) (1998) 1254–1259. [28] D. Walther, C. Koch, Modeling attention to salient proto-objects, Neural Networks 19 (2006) 1395–1407.