IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 9, SEPTEMBER 2012

4117

Oriented Modulation for Watermarking in Direct Binary Search Halftone Images Jing-Ming Guo, Senior Member, IEEE, Chang-Cheng Su, Yun-Fu Liu, Student Member, IEEE, Hua Lee, Fellow, IEEE, and Jiann-Der Lee, Senior Member, IEEE

Abstract— In this paper, a halftoning-based watermarking method is presented. This method enables high pixel-depth watermark embedding, while maintaining high image quality. This technique is capable of embedding watermarks with pixel depths up to 3 bits without causing prominent degradation to the image quality. To achieve high image quality, the parallel oriented high-efficient direct binary search (DBS) halftoning is selected to be integrated with the proposed orientation modulation (OM) method. The OM method utilizes different halftone texture orientations to carry different watermark data. In the decoder, the least-mean-square-trained filters are applied for feature extraction from watermarked images in the frequency domain, and the naïve Bayes classifier is used to analyze the extracted features and ultimately to decode the watermark data. Experimental results show that the DBS-based OM encoding method maintains a high degree of image quality and realizes the processing efficiency and robustness to be adapted in printing applications. Index Terms— Halftone image classification, halftoning, least mean square (LMS), naïve Bayes classifier.

I. I NTRODUCTION

D

IGITAL halftoning [1] is a technique to display two-tone binary images by simulating continuous tone image. The halftone images are perceived as continuous tone images when viewed at a distance due to the low-pass effect in the human visual system (HVS). Halftoning is commonly used in printed materials such as books, magazines, and newspapers, since the printing devices can normally render at most two tones, black and white (with and without ink). Many different halftoning methods have been developed, including direct binary search (DBS) [2]–[5], ordered dithering (OD) [6], error diffusion (ED) [7]–[9], dot diffusion (DD) [10]–[11]. Among these,

Manuscript received August 14, 2011; revised April 23, 2012; accepted April 26, 2012. Date of publication May 8, 2012; date of current version August 22, 2012. This work was supported in part by the National Science Council, Taiwan, under Contract NSC 100-2631-H-011-001 and Contract NSC 100-2221-E-182-047-MY3. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Bulent Sankur. J.-M. Guo, C.-C. Su, and Y.-F. Liu are with the Department of Electrical Engineering, National Taiwan University of Science and Technology, Taipei 10607, Taiwan (e-mail: [email protected]; [email protected]; [email protected]). H. Lee is with the Department of Electrical and Computer Engineering, University of California, Santa Barbara, CA 93106 USA (e-mail: [email protected]). J.-D. Lee (corresponding author) is with the Department of Electrical Engineering, Chang Gung University, Taoyuan 333, Taiwan (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIP.2012.2198221

DBS offers the best image quality, however, it has the highest computational complexity. Digital watermarking has many applications, such as protecting the ownership rights of an image, tracking unauthorized uses of a work as in fingerprinting, or authenticating the originality of an image, that it has not been tampered with. Since the halftoning is widely used, halftone-based watermarking methods have been studied in recent years. Typical practical applications of halftone-based watermarking are printing security documents such as ID cards, currency banknotes, or confidential documents, and to control their illegal duplication and forgery following their conversion to digital form. The watermarking schemes can be separated into two categories according to the decoding manner as discussed below. For the methods in the first category, the watermark can be retrieved by scanning and applying the corresponding extraction algorithms. In this category, the methods normally require higher computational complexity and produce higher image quality. These methods include using the concept of vector quantization to embed watermark into the most or least significant bit of an error-diffused image [12]. The advantage is that a low bit-depth halftone can be directly obtained from a higher bit-depth halftone. In [13], different dither cells were exploited to create a threshold pattern in the halftoning process, in which each dither cell represents the corresponding information bit of the watermark. In this paper, a specific statistical model of input images is analyzed, leading to an optimal decoding algorithm in terms of rate distortion. In [14], the modified data-hiding error diffusion method was employed to embed data into an error-diffused image. The amount of hidden data is relatively simple to manage. The security relies on the key, and not on the system itself. In [15], the DBS was employed to achieve halftoning and watermarking simultaneously. This method requires three steps to achieve the watermarking. First, the HVS-based error metric is adopted in DBS. Then, the algorithm defines a corresponding detection measurement. And, subsequently, search traverse strategy is applied to the space of halftones. Conversely, the proposed scheme adopts the platform of the DBS and the features of various texture angles to embed watermarks. Moreover, the least mean square (LMS) algorithm and the naïve Bayes classifier are catered for decoding watermarks. Between the two methods, the proposed method uses fewer training parameters than that in [15], and the proposed method is able to embed watermarks of various pixeldepths. In [16] and [17], the approach is to modulate the

1057–7149/$31.00 © 2012 IEEE

4118

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 9, SEPTEMBER 2012

dot orientation to symbolize different watermark bits. Because the two former techniques involved OD-based method and adjustment of the dot angles, the marked dots provide high robustness against attacks [6]. However, this method also sacrifices the representative ability to render the details of images, and is normally employed for laser printer. Its high dpi induces the serious dot gain problem, and as a result, the green-noise halftoning, such as the clustered-dot OD, is the preferred technique. Conversely, the DBS provides much better image quality than that of the prior work, and the blue-noise halftoning is normally employed for ink-jet printers. Although currently the ED dominates the ink-jet printer market, the fast DBS and the advance of CPU computational capability have reduced the processing speed gap. For the methods in the second category, the hidden watermark can be perceived directly when the marked halftone images are overlaid with each other. The methods in this category typically have lower computational complexity, and the embedded watermarks are usually no more than 1bit depth. Sharma and Wang [18] controlled the phase shift in clustereddot halftones and then printed on duplex printed documents, in which the hidden watermark was revealed when the sheet was viewed with a light source from behind. Knox [19] employed stochastic screen patterns to embed a watermark into ordered dither images. Fu and Au [20] presented the stochastic ED to embed a watermark into halftone images. The marked halftone image is a phase-shifted version of the unmarked halftone image. However, there exist some drawbacks such as low contrast of the overlaid visual pattern, low marked image quality, and low robustness. Hence, the same group of authors proposed the conjugate ED [21] to resolve these problems. Other basic watermarking concepts can also be found in [22] and [23]. In this paper, a watermarking for halftone images, namely DBS-based orientation modulation (OM), which is classified to the first category, is proposed. The proposed scheme is developed based on the concept of DBS halftoning and the features of various texture angles to embed a watermark. The LMS algorithm and the naïve Bayes classifier are combined for decoding the watermark. According to the experiment results, this method is capable of achieving high image quality, excellent correct decode rate (CDR), and good robustness. In addition, because of the feasibility of the parallel structure, the method can be implemented for high-speed processing. Section II provides the definitions of quality assessment methods for a marked image and a decoded watermark. Sections III and IV introduce the DBS-based OM encoder and decoder with the LMS-trained filter and naïve Bayes classifier. The experimental results are documented in Section V, and Section VI draws the conclusions. II. P ERFORMANCE E VALUATIONS The HVS-based peak signal-to-noise ratio (HPSNR) is employed in this paper for image quality estimation. This criterion is different from the traditional PSNR which does not consider the nature of HVS. Suppose a test halftone image of

Fig. 1. scheme.

Conceptual flowchart of the proposed DBS-based OM encoding

size P × Q is examined, the HPSNR metric is defined as HPSNR = 10 log10

 P Q i=1



j =1

P×Q×2552 2 m,n∈R qm,n (gi+m, j +n −h i+m, j +n )

 (1)

where the variables gi, j and h i, j denote the pixel values of an original grayscale image and its corresponding halftone image, respectively, the value 255 denotes the maximum value of an 8-bit digital image, variable qm,n denotes the coefficients of a 2-D Gaussian filter, where the support region (R) size is set at 7 × 7. The exact qm,n is derived as follows: 

qm,n = e

− 12

2 m2 + n2 σx2 σy



(2)

where the variables σx and σ y are both set at 1.3 [24], denoting the standard deviations along the two perpendicular directions. In this paper, the summation of the derived Gaussian coefficients is normalized to 1 before it is used. The quality of a decoded watermark of size M × N is estimated with the following criterion, namely CDR: M  N wm,n dwm,n × 100% (3) CDR = m=1 n=1 M×N where wm,n and dwm,n denote an original watermark and a decoded watermark, respectively, operator  denotes the exclusive NOR operation. Notably, the watermarks used in this paper can involve various pixel depths, and thus the “correct decoding” happens only when wm,n = dwm,n . III. P ROPOSED DBS-BASED OM E NCODING S CHEME In this section, the DBS-based OM DBS-based OM encoding method is described. The flowchart of this procedure is given in Fig. 1. First, the P × Q grayscale host image is partitioned into nonoverlapped subimages of size (P/M) × (Q/N), and each of the subimages hides a pixel value of the pseudorandom permutated watermark of size M × N. Then the watermark pixel is embedded through the DBS-based OM by controlling the direction of the point-spread function, which is employed by the DBS to perform the toggle and swap. Subsequently, the marked subimage is given a direction characteristic. In this analysis, the bit depth of a watermark ranges from 1 to 3 bits. Each subimage can be processed in parallel through the DBS-based OM for the improvement of computation efficiency. Finally, the output of this encoding method is the marked image of size P × Q in halftone fashion.

GUO et al.: ORIENTED MODULATION FOR WATERMARKING

4119

The traditional DBS halftoning [2] has lower processing speed as a consequence of its high computational complexity due to the inherent iterative process. The complexity arises from the difference calculation between the original grayscale image and its temporary halftone result. To alleviate this problem, the required number of calculations was significantly reduced by the efficient DBS in [3]. In this paper, this efficient DBS method is further improved by incorporating the proposed OM method to achieve an additional watermarking advantage. In the sequel, the watermarking function is illustrated and the details of the efficient DBS are presented. While the proposed method is amenable to parallelism, it suffices to describe only the embedding process of a single subimage. Suppose a grayscale subimage g[m, n] of size 32 × 32 and a temporary randomly initialized halftone image h[m, n] are provided for processing. The values of these images are normalized to be bounded within the interval from 0 to 1, corresponding to the black and white values, respectively. We should point out that the appearance of the temporary initialized halftone image can slightly affect the required processing time and the final marked halftone quality. To limit the impact of this effect, the temporary halftone subimage is initialized and set to the black value throughout this paper. The iteration process of the temporary halftone subimage is presented in detail in this section. As it converges, the result is the marked halftone subimage. In the following subsections, the efficient DBS method is first briefly reviewed and this proposed modified scheme is then described in details to embody the proposed watermarking.

Fig. 2. Toggle and swap operations used to create various appearances of halftone patterns in traditional DBS [2].

The two variables c p˜ p˜ [m, n] and ce˜ p˜ [m, n] are then employed for the estimation of the error between the original grayscale subimage and the current temporary halftone subimage. Step 2: To generate more patterns, toggle and swap are utilized. Fig. 2 shows the examples of these two operations, in which each square denotes a pixel, and the shaded squares denote the affected pixels. The gray and white squares denote black and white pixels, respectively. Suppose the centers of these 3 × 3 blocks denote the current processing position. The toggle operation changes the value of the current processing position [m 0 , n 0 ] to its converse color (black to white). The swap exchanges the value at position [m 0 , n 0 ] with its eight neighbors at position [m 1 , n 1 ]. Thus, the exchanged perceived halftone and error images can be represented as ˜ y) + a0 p(x ˜ − m 0 X, y − n 0 Y ) h˜  (x, y) = h(x, +a1 p(x ˜ − m 1 X, y − n 1 Y ) ˜ − m 0 X, y − n 0 Y ) e˜ (x, y) = e˜(x, y) + a0 p(x +a1 p(x ˜ − m 1 X, y − n 1 Y )

A. Efficient DBS The procedure of the efficient DBS is structured in the following steps. Step 1: Two resources are first initialized, and both of them are related with the point-spread function p(s, ˜ t) which represents the nature of HVS generated by Nasanen’s contrast sensitivity function (CSF) in the spatial domain. The perceived grayscale (g), halftone (h), and error (e) images are defined as  g[m, n] p(x ˜ − m X, y − nY ) (4) g(x, ˜ y) = m,n

˜ h(x, y) =



h[m, n] p(x ˜ − m X, y − nY )

(5)

e[m, n] p(x ˜ − m X, y − nY )

(6)

m,n

e(x, ˜ y) =

 m,n

where e[m, n] = h[m, n]−g[m, n], and X and Y are constants denoting the bases for the lattice of printer-addressable dots in units of inch/dot. The autocorrelation function of p(s, ˜ t) is defined as   p(s, ˜ t) p(s ˜ + m X, t + nY )dsdt. (7) c p˜ p˜ [m, n] = To estimate the cross-correlation ce˜ p˜ [m, n] between p(s, ˜ t) and the perceived error image e(x, ˜ y) is formulated as  e[u, v]c p˜ p˜ [u − m, v − n]. (8) ce˜ p˜ [m, n] = u,v

(9)



(10)

where the variables a0 and a1 are changed according to whether toggle or swap is estimated. In the toggle operation −1, if h[m 0 , n 0 ] changed from 1 to 0 a0 = (11) 1, if h[m 0 , n 0 ] changed from 0 to 1 and a1 = 0. In the swap operation, the definition of the variable a0 is same as (8), while the variable a1 = −a0 . In total, nine different halftone patterns are generated, and the pattern which achieves the best image quality is reserved. To produce an objective estimation of image quality, the squared perceived error ε = x,y |˜e(x, y)|2 dxdy is employed, and the error difference is defined as ε = ε − ε = (a02 + a12 )2 c p˜ p˜ [0, 0] +2a0 a1 c p˜ p˜ [m 1 −m 0 , n 1 − n 0 ] +2a0 ce˜ p˜ [m 0 , n 0 ]+2a1ce˜ p˜ [m 1 , n 1 ].

(12)

Step 3: The pattern with the minimal ε is selected to replace the pixels of the corresponding position of the temporary halftone subimage. For example, if each of eight swaps achieves the minimal error, the pixels at positions [m 0 , n 0 ] (the current processing position) and [m 1 , n 1 ] (each of the eight neighbors) on the temporary halftone subimage are modified according to the formula h k+1 [m, n] = h k [m, n] + a0 δ[m − m 0 , n − n 0 ] +a1 δ[m − m 1 , n − n 1 ]

4120

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 9, SEPTEMBER 2012



where δ[m, n] =

1, if [m, n] = [0, 0] 0, otherwise

(13)

and variable k denotes the iteration index. The modifications may also affect the perceived image quality from other neighboring positions. Thus, the corresponding ce˜ p˜ [m, n] has to be updated for the next iteration as k cek+1 ˜ p˜ [m, n] = ce˜ p˜ [m, n] + a0 c p˜ p˜ [m − m 0 , n − n 0 ]

+ a1 c p˜ p˜ [m − m 1 , n − n 1 ].



(15)

where the parameter η denotes the quality factor, and variables a, b, and c are defined as sin 2 θ cos2 θ + 2σs2 2σt2 sin 2θ sin 2θ + b=− 4σs2 4σt2 2 sin θ cos2 θ c= + 2σs2 2σt2

Fast Fourier Transform (FFT)

Classify the angle with naïve Bayes classifier

Prob. data

Conceptual flowchart of the proposed decoding scheme.

(14)

The point-spread function p(s, ˜ t) given in (4) is employed and modified to represent different watermark values. The CSF used to derive p(s, ˜ t) is replaced with the modified 2-D Gaussian distribution which can be used to generate halftone textures with various directions

a=

LMStrained filters

cek˜ p˜ [m, n]

B. Oriented Modulation

2 +η bst +ct 2 )

Feature extraction

decoded watermark of a pixel (1x1)

Fig. 3.

Step 4: The system does not converge, when =  k−1 ce˜ p˜ [m, n]. If so, we repeat Steps 2 and 3. Otherwise, the temporary halftone image is considered the final converged halftone image. As shown in Fig. 1, the method provides a good parallel characteristic from its block-based processing strategy. To fully utilize the property, each of the subimages of size (P/M) × (Q/N) is processed independently and simultaneously. This implies that the modifications include the calculation of squared perceived error of (12), and the updating procedure of (14) does not affect other subimages.

p(s, ˜ t) = e−(as

marked sub-image (P/MxQ/N)

(16)

(a)

(b)

(c)

(d)

Fig. 4. Four marked subimages (left) and the corresponding patterns in the frequency domains (right), where the subimages are generated by the proposed DBS-based OM and the embedded watermark is with 2-bit (four kinds of colors) pixel depth, the parameter of the quality factor η is set at 2. These subimages are marked with (a) 0°, (b) 45°, (c) 90°, and (d) 135°, respectively.

IV. P ROPOSED D ECODING S CHEME W ITH LMS F ILTERS AND NAÏVE BAYES C LASSIFIERS The marked image of size P × Q generated by the DBSbased OM can be considered as the combination of many independent marked subimages of size (P/M) × (Q/N), and each subimage embeds one pixel of the watermark. This property enables the corresponding block-independent decoding algorithm as illustrated in Fig. 3. First, the marked subimage is transformed into frequency domain by the fast Fourier transform (FFT) to extract the features for classification. Then the LMS-trained filter is applied to enhance and distinguish various directions by amplifying the differences among features and converting each feature into a numerical index. Subsequently, these extracted features along with the naïve Bayes classifier and the corresponding probability data are adopted to decode the embedded watermark.

(17) A. Features Extraction (18)

where the two empirical parameters σs and σt define the standard deviations which are set to 1 and 2, respectively, for the ellipse distribution. The variable θ ∈ (0°, 180°) denotes the angle, which is governed by the embedded watermark values. It should be noted that this watermarking method can embed watermarks of various bit depths. An N-bit case contains colors, where N = 1, 2, or 3, in this paper. And in this analysis, since each angle represents a specific color, to improve the distinguish ability of each halftone texture angle, the difference between each pair of consecutive angles is defined as 180° (19) θ = N 2 where N denotes the number of representative colors. For instance, suppose a watermark of 2-bit pixel depth (N = 2) is embedded in a halftone image. The four angles 0°, 45°, 90°, and 135° can be used to represent of the 22 watermark colors.

With various texture orientations in the subimages, embedding can introduce a slightly different spatial dot distribution density, which is easily overlooked by human vision. Yet, it can be apparent in the frequency domain. To illustrate this effect, Fig. 4 shows that the pattern on the right-hand side of each pair is transformed by the FFT. To distinguish the orientation characteristics, L LMS-trained filters associated with L orientations are trained, and the procedure is outlined here  u k (m, n) × f dc (m, n) (20) vˆ c = m,n∈R



e = (v − vˆ ) , where v = 2

c 2

∂e2 = −2e f dc (m, n) ∂u k (m, n)  u k+1 (m, n) = u k (m, n) +

0, if c = tc 255, otherwise

(21) (22)

γ e f dc (m, n), if −γ e f dc (m, n),

if

∂e2 ∂u k (m,n) ∂e2 ∂u k (m,n)

<0 >0 (23)

GUO et al.: ORIENTED MODULATION FOR WATERMARKING

0°

22.5°

4121

32 × 32, and each of the filters is trained with 6144 halftoned images in frequency domain, and each is generated with a specific angle. These halftone images of size 32 × 32 are randomly cropped from the USC-SIPI dataset [25]. It can be seen that the obtained filters exhibit two features. The first is that the absolute value of each filter is uniquely different. And secondly, the positive coefficients clearly define the angles. The feature values governing the orientations of the halftone patterns in the frequency domain can be extracted with the LMS-trained filters. Each extracted feature also indicates the similarity level with an angular value. The feature is calculated as  u c (m, n) × H (m, n) (24) xc = m,n∈R

45°

67.5°

where H (m, n) denotes the frequency domain result of a marked halftone pattern h(m, n)in the spatial domain, and the variable u c (m, n) denotes the coefficient of the LMS-trained filter with angle c. B. Naïve Bayes Classifier

Fig. 5. angles.

90°

112.5°

135°

157.5°

LMS-trained filters of size 32 × 32 obtained with eight different

where f dc ∈ F = { f 1c , f 2c , . . . , f Dc } denotes the training halftone patterns in the frequency domain with a fixed size. The halftone image is generated with a specific angle c ∈ {1, 2, . . . , L}, and the constant D denotes the number of training patterns in the frequency domain. The variable u k (m, n) denotes the kth trained filter with a support region R, of size identical to that of the pattern training in the frequency domain. The summation of the trained filter is set to unity to normalize the total energy level. The variable u k (m, n) can be used to separate the target class (tc) and the rest of the classes (tc). In addition, tc ∈ c, and the size is identical to the pattern in the frequency domain. We also note that the target values are set to the theoretical maximum (255) and minimum (0) of a digital image for improved discrimination results. In this training algorithm, the convergence rate (γ ) is set to 10−10 . The iteration is terminated when the increment of the total weights is below the threshold of 10−5 . Fig. 5 shows trained filters of size

Fig. 6 shows the normalized feature distributions. The color line on each subfigure is the average from 25 600 halftone patterns in the frequency domain. The distribution of each subfigure is obtained by one LMS-trained filter corresponding to a specific angle. Based on the observations, two properties are identified. First, the feature values with the same class of the employed LMS-trained filters are higher than that of other classes. Secondly, except for the class that belongs to the employed LMS-trained filter, other feature distributions also show the possibility for classification, because they are distinguishable and offer specific relationships to the angle of interest. For instance, in the case with 157.5° shown in Fig. 6(a), the distributions of 0° and 135° provide the second highest features. These two observed characteristics are fully utilized to further improve the performance of the correct classification, and the naïve Bayes classifier which is derived from the Bayesian theorem is later proved as a powerful tool p(H |I ) =

p(H ) p(I |H ) p(I )

(25)

where I denotes information, and H denotes hypothesis. This formula gives a relationship between the observed information and the future hypothesis. Equation (25) can be rewritten for a practical application with L number of information and K number of hypotheses p(h k |i 1 , i 2 , . . . , i L ) =

p(h k ) p(i 1 , i 2 , . . . , i L |h k ) p(i 1 , i 2 , . . . , i L )

(26)

where 1 ≤ k ≤ K . The term p(i 1 , i 2 , . . . , i L |h k ) can be assumed as independent and will not affect the performance of the classification significantly [26]. Hence, (26) can be further rewritten as L p(h k ) b=1 p(i b |h k ) (27) p(h k |i 1 , i 2 , . . . , i L ) =  K L a=1 p(h a ) b=1 p(i b |h a ) where the denominator is obtained by the law of total probability. To apply it to the halftoning classification application,

4122

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 9, SEPTEMBER 2012

where x n denotes the feature extracted by the LMS-trained filter of class n, and ck denotes the orientation k. The probability of p(ck ) is set to be uniform because the occurrences of different orientations are assumed to have equal probability. The term p(x b |ck ) can be obtained by conducting the feature statistics under the condition when the subhalftone image is obtained by ck , which is shown in Fig. 6(a). Based on the formula, the probabilities of the N angles can be obtained. Subsequently, the maximum a posteriori (MAP) method can be employed for identifying the class with the highest probability. As a result, the final class of a tested halftone image can be determined cˆMAP (x 1 , x 2 , . . . , x 4 ) = arg max p(c|x 1, x 2 , . . . , x 4 ) c

(29)

where the term p(c|x 1 , x 2 , . . . , x 4 ) is identical to that in (28). The denominator of p(c|x 1, x 2 , . . . , x 4 ) can be neglected since it is identical to the halftoning scheme. The probability p(c) can be neglected as well since it is uniform. Thus, the MAP can be replaced with the maximum likelihood format

L p(x b |c). (30) cˆ M L (x 1 , x 2 , . . . , x 4 ) = arg max c

b=1

By doing this, the computational complexity can be significantly simplified. This approach, namely nonparametric decision, utilizes the statistic information, as shown in Fig. 6(a), and requires significant memory capacity for the storage of the feature distribution. To overcome this problem, the parametric decision method is selected. Each of the feature distributions, shown in Fig. 6(a), is modeled as a 1-D Gaussian distribution with the corresponding mean and standard deviation. Fig. 6(b) shows an example for the 0° feature distribution. By doing this, the memory consumption can be significantly reduced. V. E XPERIMENTAL R ESULTS

Fig. 6. Feature distributions obtained with the eight LMS-trained filters as shown in Fig. 5. (a) Each of the feature distributions is constructed by 25 600 halftone patterns in the frequency domain. (b) 0° feature distribution is modeled by 1-D Gaussian distributions.

under the assumption that the orientation is divided by a factor of four, (27) becomes p(ck ) 4b=1 p(x b |ck ) (28) p(ck |x 1 , x 2 , . . . , x 4 ) =  K 4 a=1 p(ca ) b=1 p(x b |ca )

In this section, the performance of the proposed DBS-based OM is evaluated from various perspectives. Fig. 7 shows the CDRs of the subimages of sizes 16 × 16 and 32 × 32, where the CDR is averaged from the 25 test images [27] of size 512 × 512, and the watermarks involve the pixel depths, up to 3 bits. Three different decoders are tested in these experiments. 1) LMS: Because the calculated features obtained from (24) indicate the similarity with the angle of the LMStrained filter, the angle with the maximum feature value is considered as the classified result. 2) Nonparametric Decision: Classification with the naïve Bayes classifier and the feature distribution are constructed by the statistics such as the results in Fig. 6(a). 3) Parametric Decision: The feature distributions used by the naïve Bayes classifier are modeled by 1-D Gaussian distributions constructed by the corresponding means and standard deviations of the distributions from the nonparametric statistics. According to the results, the size of a subimage is directly proportional to the CDR. In addition, the quality factor (η) in (15) is also directly proportional to the CDR, because

GUO et al.: ORIENTED MODULATION FOR WATERMARKING

CDR vs. Quality factor

100 CDR (%)

4123

LMS Parametric decision Nonparametric decision

80 60 2

1.75

1.5 1.25 1 Quality factor

0.25

Fig. 8.

0.75

0.5

2.5

CDR (%)

Processing time (millisecond)

CDR vs. Quality factor

2

1.5

LMS Parametric decision Nonparametric decision

87.5 75

0.75

1

1.25

1.5

1.75

2

Eight different quality factors with identical 45°.

0.25

(a)

100

0.5

Processing time vs. Various sizes of sub-image Encoder: DBS-based OM Decoder: Parametric decision method Decoder: Nonparametric decision method

1

0.5 0

2

1.75

1.5

1.25 1 0.75 Quality factor

0.5

0.25

16x16 32x32 Various sizes of sub-image

(b)

60 2

1.75

1.5

1.25 1 0.75 Quality factor

0.5

0.25

(c)

CDR (%)

CDR vs. Quality factor 90

2

1.75

1.5

1.25 1 0.75 Quality factor

100 80 60 HPSNR_1-bit HPSNR_3-bit CDR_2-bit 2

LMS Parametric decision Nonparametric decision

70

36 35 34 33 32 31 30 29

0.5

0.25

1.75

1.5

40

HPSNR_2-bit CDR_1-bit CDR_3-bit

1.25 1 0.75 Quality factor

0.5

CDR (%)

LMS Parametric decision Nonparametric decision

80

HPSNR

100 CDR (%)

Fig. 9. Processing time comparisons with various subimage sizes and decoding methods.

CDR vs. Quality factor

20 0 0.25

Fig. 10. Relationships among quality factor (η), image quality (HPSNR), and decoded rate (CDR).

CDR (%)

(d)

CDR vs. Quality factor

90 70 50 30 10

LMS Parametric decision Nonparametric decision 2

1.75

1.5

1.25 1 0.75 Quality factor

0.5

0.25

(e)

CDR vs. Quality factor CDR (%)

90 LMS Parametric decision Nonparametric decision

40 2

1.75

1.5

1.25 1 0.75 Quality factor

0.5

0.25

(f)

Fig. 7. Averaged CDR comparisons under various sizes of subimages and various numbers of angles. (a) and (b) Two angles. (c) and (d) Four angles. (e) and (f) Eight angles. (a) 16 × 16. (b) 32 × 32. (c) 16 × 16. (d) 32 × 32. (e) 16 × 16. (f) 32 × 32.

this parameter can control the orientation shape of a halftone pattern in the frequency domain. We also note that the circular shape (η = 0) shows no apparent direction, and thus η is expected to be greater to provide a better discrimination capability, as it can be seen in Fig. 8. In comparison, the

nonparametric and parametric decision methods, using the naïve Bayes classifier, give superior performance to that of solely using LMS. In addition, the nonparametric decision method achieves the best CDR, because it provides more precise analytical capability of the feature distribution than the parametric decision. However, in return, the consumed memory of the nonparametric method is higher than the parametric method by a factor of 300. The additional cost of the parametric method is due to the calculation of the probability of each feature. Thus the computation speed is slightly inferior to that of the nonparametric, as demonstrated in Fig. 9, in which the processing speed of the proposed naïve DBS-based OM encoding scheme is also compared. Since the DBS-based OM can be implemented in parallel form, the processing speed is approximately 422 f/s with images of size 512 × 512. According to the experiments, quality results can be yielded when the subimage size is set to 32 × 32, using the parametric decision method. Consequently, the following experiments are performed accordingly. Fig. 10 shows the relationships among the HPSNR, CDR, and quality factor η, and these results are averaged from the 25 test natural images. Because the proposed method can embed watermarks of various pixel depths, three different pixel depths are included in the experiments. According to the results, a watermark with a lower pixel depth can yield better performance in terms of HPSNR and CDR. In addition, when the quality factor is lower than 0.5, the marked image

4124

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 9, SEPTEMBER 2012

CDR vs. Cropped size

CDR (%)

100 90

1-bit 2-bit 3-bit

80 70 32x32

Grayscale host images

64x64 128x128 Cropped size

256x256

Fig. 12. CDRs under cropping attack with various cropped sizes, including 32 × 32 to 256 × 256. The size of a host image is 512 × 512.

one bit

two bits three bits

CDR vs. Scan dpi

100 CDR (%)

one bit

two bits three bits (a)

80 Printed at 150 dpi Printed at 300 dpi Printed at 600 dpi

60 40 150

300Scan dpi600

1200

HPSNR=31.5 dB

CDR=100% HPSNR=31.6 dB CDR=100%

CDR (%)

(a)

(b)

CDR vs. Scan dpi

100 80 60 40 20

Printed at 150 dpi Printed at 300 dpi Printed at 600 dpi 150

300Scan dpi600

1200

HPSNR=31.7 dB CDR=100%

HPSNR=31.9 dB

CDR=100%

(c)

CDR (%)

(b)

CDR vs. Scan dpi

80 60 40 20 0

Printed at 150 dpi Printed at 300 dpi Printed at 600 dpi 150

300Scan dpi600

1200

(c) Fig. 13. Averaged CDRs of the decoded watermarks that undergone print and scan distortion. Watermarks of various pixel depths are adopted, including (a) 1 bit, (b) 2 bits, and (c) 3 bits.

HPSNR=31.9 dB CDR=99.4%

HPSNR=32 dB

CDR=98.8%

(d) Fig. 11. Practical marked results. (a) Original host images of size 2048×2048 and watermarks with various pixel depths of size 64 ×64. The marked images and the decoded watermarks with (b) 1 bit, (c) 2 bits, and (d) 3 bits, and the enlarged parts are of size 64×64. (Host/marked images are printed at 600 dpi, and watermarks are printed at 72 dpi).

quality degrades rapidly and CDRs become saturated. Thus, the quality factor is recommended to be set at 0.5 in this paper. Fig. 11 shows some practical marked results and the corresponding decoded watermarks. It can be seen that when the pixel depth of a watermark is higher than 2 bits, the CDR cannot be maintained at 100% level. Nonetheless, the decoded results remain to be of good quality with CDR around 99%, as shown in Fig. 11(d). Furthermore, the enlarged parts as shown at the top-right corner of Fig. 11(b)–(d), which indicates that,

although the proposed encoder is processed independently, the marked images do not exhibit the blocking effect. Some distortions, such as cropping and print and scan, may occur in the practical applications. Fig. 12 shows the CDRs under cropping distortion with various cropped sizes, where each datum is averaged by 25 different test images, and watermarks of three different pixel depths are also involved in this experiment. Herein, each host image is of size 512 × 512. Among these, when the cropped sizes are less than or equal to 128, the CDRs can be maintained above 90% level, and thus higher cropped sizes should be avoided. Fig. 13 shows the CDRs of decoded watermarks extracted from the marked images through the print and scan distortion. Normally, the print and scan channel involves zooming, shifting, rotation, and dot gain darkening effect, and which cause severe distortions.

GUO et al.: ORIENTED MODULATION FOR WATERMARKING

4125

TABLE I P ERFORMANCE C OMPARISON A MONG VARIOUS WATERMARKING S CHEMES (H OST I MAGE I S OF S IZE 2048 × 2048) CDR under various attacks (%) Schemes

Knox [19]

(dB)

Cropping (1024 × 1024 is cropped)

Additive noise (15% area are noised)

Print and Scan (printed 150 dpi, scanned 300 dpi)

26.87

72.93

73.01

70.08

2048 × 1024 2048 × 2048/2 (one watermark embedded into two host images)

Image quality

Data capacity (bits)

Fu and Au [2]

18.89

64.48

61.72

55.57

Pei et al. [28]

31.29

95.64

95.98

90.32

64 × 64

Guo and Liu [29]

30.88

84.24

84.78

79.28

2048 × 2048 (one watermark embedded into two host images, while pixel depth = 2)

Proposed method

35.07

87.22

95.94

98.00

64 × 64 × 3

It can be seen that the resolution of printing/scanning is inversely/directly proportional to the CDR. According to the results of the experiments, the proposed method is sufficiently robust for practical applications. Table I shows the comparisons among former related methods and the proposed method, in which three attacks, including the cropping of the one-fourth area of the host image, 15% additive noise (the ratio between the noise-adding area and the entire image size), and print and scan (printed at 150 dpi and scanned at 300 dpi) are considered. Ideally, a fair comparison should fix one parameter, such as embedding capacity, yet each method has its limitation on the adjustable data capacity according to its algorithm. For instance, the methods of Fu and Au [20] and Guo and Liu [29] employ two host images to embed one watermark of the same size of each host image. Then the two embedded images are overlaid to visually decode the hidden watermark. Consequently, the size of the watermark is considered as half of the overall host images’ sizes. Nonetheless, Guo and Liu’s method can embed more than one pixel depth. In this experiment, the pixel depth is fixed at 2. Thus, the capacity is 2048 × 2048. It seems, even the capacity of [20] and [29] is reduced, we are unable to improve the image quality and robustness. Although the capacity of [20] and [29] seems higher than that of this method, the embedded data can only be partially decoded by human vision after the two embedded images are overlaid. The fact that the capacity of [20] and [29] is higher than the proposed scheme is not conclusive. According to the overall indices from the experimental results, it is apparent that the proposed scheme performs well against the other two methods. Knox’s method [19] is a self-decoding method, and the maximum data capacity is up to half of the host image size, Pei et al.’s method [28] embeds a watermark in bitleaving domain, and the capacity is adjustable. Although the robustness of the proposed method is slightly lower than that of Pei’s method under the cropping and noise attacks, the image quality and embedding capacity of this method are much higher. Moreover, Knox [19] and Pei et al. [28] are OD-based schemes, while the proposed method is DBS-based scheme. It is not difficult to examine why the proposed method can yield superior image quality than that of the two existing

schemes. Based on the observations, the proposed method has the advantage in terms of image quality as well as the highest pixel depth to render the original watermark more accurately. Although Guo and Liu’s [29] method can embed multitone watermark, the robustness is significantly degraded as the pixel depth is increased. In addition, the proposed method is capable of providing the highest robustness under the print and scan attack than other prior works. VI. C ONCLUSION In this paper, the DBS-based OM method was presented. The main objective of this technique is the embedding of multibit watermarks into halftone images. DBS is a halftoning scheme, known for the high-quality halftone patterns and deficiency in processing speed. This proposed DBS-based OM method utilized parallel architecture for high-speed processing while retaining the feature of high-quality display. To decode an embedded watermark, the LMS-trained filters and the naïve Bayes classifier were integrated to achieve superior decoding rates. In addition, the proposed parametric decision strategy significantly reduced the memory requirements. As documented in the experimental results, the proposed method provided excellent image quality, high processing efficiency, variable watermark pixel depth, and sufficient robustness to guard against distortions introduced by the cropping and print and scan operations in halftone printing. R EFERENCES [1] R. Ulichney, Digital Halftoning. Cambridge, MA: MIT Press, 1987. [2] M. Analoui and J. P. Allebach, “Model based halftoning using direct binary search,” Proc. SPIE, Human Vis. Visual Digit. Display III, vol. 1666, pp. 96–108, Feb. 1992. [3] D. J. Lieberman and J. P. Allebach, “Efficient model based halftoning using direct binary search,” in Proc. IEEE Int. Conf. Image Process., vol. 1. Oct. 1997, pp. 755–778. [4] D. J. Lieberman and J. P. Allebach, “A dual interpretation for direct binary search and its implications for tone reproduction and texture quality,” IEEE Trans. Image Process., vol. 9, no. 11, pp. 1950–1963, Nov. 2000. [5] S. H. Kim and J. P. Allebach, “Impact of HVS models on model-based halftoning,” IEEE Trans. Image Process., vol. 11, no. 3, pp. 258–269, Mar. 2002. [6] R. A. Ulichney, “The void-and-cluster method for dither array generation,” Proc. SPIE, Human Vis. Visual Process., Digit. Displays IV, vol. 1913, pp. 332–343, Feb. 1993.

4126

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 9, SEPTEMBER 2012

[7] J. F. Jarvis, C. N. Judice, and W. H. Ninke, “A survey of techniques for the display of continuous-tone pictures on bilevel displays,” Comput. Graph. Image Process., vol. 5, no. 1, pp. 13–40, Mar. 1976. [8] P. Stucki, “MECCA-a multiple-error correcting computation algorithm for bilevel image hardcopy reproduction,” IBM Research Laboratory, Zurich, Switzerland, Res. Rep. RZ1060, 1981. [9] R. W. Floyd and L. Steinberg, “An adaptive algorithm for spatial gray scale,” in Proc. SID Dig. Soc. Inf. Display, 1975, pp. 36–37. [10] D. E. Knuth, “Digital halftones by dot diffusion,” ACM Trans. Graph., vol. 6, no. 4, pp. 245–273, Oct. 1987. [11] M. Mese and P. P. Vaidyanathan, “Optimized halftoning using dot diffusion and methods for inverse halftoning,” IEEE Trans. Image Process., vol. 9, no. 4, pp. 691–709, Apr. 2000. [12] J. R. Goldschneider, E. A. Riskin, and P. W. Wong, “Embedded color error diffusion,” in Proc. IEEE Int. Conf. Image Process., vol. 1. Sep. 1996, pp. 565–568. [13] Z. Baharav and D. Shaked, “Watermarking of dither halftoned images,” in Proc. Int. Conf. Security Watermark. Multimedia Content, May 1999, pp. 307–316. [14] M. S. Fu and O. C. Au, “Hiding data in halftone image using modified data hiding error diffusion,” Proc. SPIE Vis. Commun. Image Process., vol. 4067, pp. 1671–1680, May 2000. [15] D. Kacker and J. P. Allebach, “Joint halftoning and watermarking,” IEEE Trans. Signal Process., vol. 51, no. 4, pp. 1054–1068, Apr. 2003. [16] O. Bulan, G. Sharma, and V. Monga, “Orientation modulation for data hiding in clustered-dot halftone prints,” IEEE Trans. Image Process., vol. 19, no. 8, pp. 2070–2084, Aug. 2010. [17] O. Bulan, V. Monga, G. Sharma, and B. Oztan, “Data embedding in hardcopy images via halftone-dot orientation modulation,” in Proc. Int. Soc. Opt. Eng., 2008, pp. 1–12. [18] G. Sharma and S. G. Wang, “Show-through watermarking of duplex printed documents,” Proc. SPIE, vol. 5306, pp. 19–22, Jan. 2004. [19] K. T. Knox, “Digital watermarking using stochastic screen patterns,” U.S. Patent 5 734 752, Mar. 31, 1998. [20] M. S. Fu and O. C. Au, “Data hiding in halftone images by stochastic error diffusion,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., May 2001, pp. 1965–1968. [21] M. S. Fu and O. C. Au, “Steganography in halftone images: Conjugate error diffusion,” Signal Process., vol. 83, no. 10, pp. 2171–2178, Oct. 2003. [22] M. Barni, “What is the future for watermarking? (Part I),” IEEE Signal Process. Mag., vol. 20, no. 5, pp. 55–60, Sep. 2003. [23] M. Barni, “What is the future for watermarking? (Part II),” IEEE Signal Process. Mag., vol. 20, no. 6, pp. 53–59, Nov. 2003. [24] J. M. Guo and Y. F. Liu, “Joint compression/watermarking scheme using majority-parity guidance and halftone-based block truncation coding,” IEEE Trans. Image Process., vol. 19, no. 8, pp. 2056–2069, Aug. 2010. [25] USC-SIPI Database. (2011, Jun. 1) [Online]. Available: http://sipi.usc.edu/database/ [26] H. Zhang, “The optimality of naïve Bayes,” in Proc. 7th Int. Florida Artif. Intell. Res. Soc. Conf., 2004, pp. 562–567. [27] Image Database. (2011, Jul. 1) [Online]. Available: http://msp.ee.ntust.edu.tw/public% 20file/CCSu/25photos.rar [28] S. C. Pei, J. M. Guo, and H. Lee, “Novel robust watermarking technique in dithering halftone images,” IEEE Signal Process. Lett., vol. 12, no. 4, pp. 333–336, Apr. 2005. [29] J. M. Guo and Y. F. Liu, “Hiding multitone watermarks in halftone images,” IEEE Multimedia, vol. 17, no. 1, pp. 34–43, Jan. 2010.

Jing-Ming Guo (M’06–SM’10) was born in Kaohsiung, Taiwan, on November 19, 1972. He received the B.S.E.E. and M.S.E.E. degrees from National Central University, Taoyuan, Taiwan, in 1995 and 1997, respectively, and the Ph.D. degree from the Institute of Communication Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan, in 2004. He was an Information Technique Officer with the Chinese Army, from 1998 to 1999. From 2003 to 2004, he was granted the National Science Council scholarship for advanced research from the Department of Electrical and Computer Engineering, University of California, Santa Barbara.

He is currently a Professor with the Department of Electrical Engineering, National Taiwan University of Science and Technology, Taipei. His current research interests include multimedia signal processing, multimedia security, computer vision, and digital halftoning. Dr. Guo is a senior member of the IEEE Signal Processing Society. He was a recipient of the Outstanding Youth Electrical Engineer Award from the Chinese Institute of Electrical Engineering in 2011, the Outstanding Young Investigator Award from the Institute of System Engineering in 2011, the Best Paper Award at the IEEE International Conference on System Science and Engineering in 2011, the Excellence in Teaching Award in 2009, the Research Excellence Award in 2008, the Acer Dragon Thesis Award in 2005, the Outstanding Paper Awards at the Institute for Public Policy Research Conference on Computer Vision and Graphic Image Processing in 2005 and 2006, and the Outstanding Faculty Award in 2002 and 2003.

Chang-Cheng Su received the M.S.E.E. degree from the Department of Electrical Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan, in 2011. His current research interests include watermarking and digital halftoning techniques.

Yun-Fu Liu (S’09) was born in Hualien, Taiwan, on October 30, 1984. He received the M.S.E.E. degree from the Department of Electrical Engineering, Chang Gung University, Taoyuan, Taiwan, in 2009. He is currently pursuing the Ph.D. degree with the Department of Electrical Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan. He is a Visiting Researcher with the Department of Electrical and Computer Engineering, University of California, Santa Barbara. His current research interests include digital halftoning, steganography, image compression, object tracking, and pattern recognition. Mr. Liu was a recipient of the Special Jury Award from Chimei Innolux Corporation in 2009 and the third Masters Thesis Award from Fuzzy Society, Taiwan, in 2009.

Hua Lee (S’78–M’80–SM’83–F’92) received the Ph.D. degree from University of California, Santa Barbara (UCSB), in 1980. He was a Faculty Member with the University of Illinois at Urbana-Champaign, Urbana. His research laboratory was the first to produce the holographic and tomographic reconstructions from a scanning laser acoustic microscope, and his research team is also known as the leader in pulse-echo microwave nondestructive evaluation of civil structures and materials. His current research interests include the areas of imaging system optimization, high-performance image formation algorithms, synthetic-aperture radar and sonar systems, acoustic microscopy, microwave nondestructive evaluation, and dynamic vision systems. Dr. Lee was an Associate Editor of the IEEE T RANSACTIONS ON C IRCUITS AND S YSTEMS FOR V IDEO T ECHNOLOGY from 1992 to 1995 and the IEEE T RANSACTIONS ON I MAGE P ROCESSING from 1994 to 1998. From 1988 to 1994, he was the Editor of the International Journal of Imaging Systems and Technology. He was the Chairman of the 18th International Symposium on Acoustical Imaging in 1989 and the 13th International Workshop on Maximum Entropy and Bayesian Methods in 1993.

GUO et al.: ORIENTED MODULATION FOR WATERMARKING

Jiann-Der Lee (M’98–SM’11) was born in Tainan, Taiwan, in 1961. He received the B.S., M.S., and Ph.D. degrees from the Department of Electrical Engineering, National Cheng Kung University, Tainan, in 1984, 1988, and 1992, respectively. He is currently a Full Professor with the Department of Electrical Engineering, Chang Gung University, Taoyuan, Taiwan. His current research interests include image processing, pattern recognition, computer vision, consumer electronics, and VLSI computer-aided designs.

4127

Dr. Lee is a member of the International Association for Pattern Recognition. He is listed in Who’s Who in the World and Who’s Who in Finance and Industry. He was a recipient of a number of best investigator awards (e.g., from the National Science Council, Taiwan, and from Acer Foundations, Taiwan), and the Excellent Teacher Award in 2002 and the Excellent Researcher Award in 2003 from the Chang Gung University.