1

Multiterminal Secure Source Coding for a Common Secret Source Chung Chan

Abstract—A multiterminal secure source coding problem is proposed where multiple users discuss in public until they can recover a particular source as securely as possible. The model provides a unified framework that combines and generalizes the problems of multiterminal secret key agreement and secure computation. Bounds on the achievable discussion rate, level of secrecy and reliability are derived.

I. I NTRODUCTION Consider a set of users who observe privately some correlated random sources. They want to discuss in public so that each user can learn some part of the sources observed by others while keeping some part of his source secret to some other users and a wiretapper who listens to the discussion. The problem of multiterminal secure source coding is to find a public discussion scheme that allows the users to exchange and hide the intended portions of the random sources. In this work, we consider the special case where the users observe sequences of some correlated discrete memoryless sources. A subset of the users called the active users want to recover a particular portion of the sources referred to as the secret source. In order to achieve this goal, each user including the non-active users has to encode some information about their sources and communicate it to the active users. They can do this by means of a public discussion, where users broadcast one-by-one some messages about their source over a noiseless public channel. However, there is a wiretapper who try to learn the secret source by listening to the entire public discussion. The users must therefore carefully design their public messsages to minimize the amount of information leaked to the wiretapper. The secrecy capacity of the system is defined as the maximum rate of information about the secret source that can be hidden from the wiretapper, asymptotically as the block length goes to infinity. This problem can be viewed alternatively as a generalization of the multiterminal secret key agreement problem proposed in [3]. Since the secret source has to be recovered by all active users, the portion of the source that can be hidden from the wiretapper is a valid secret key shared among the active users. Thus, the capacity of the system is the maximum rate of a secret key that is chosen as a function of the secret source. In the special case when the secret source is the entire source Chung Chan ([email protected], [email protected]) is with the Institute of Network Coding, the Chinese University of Hong Kong. This work is partially supported by a grant from the University Grants Committee of the Hong Kong Special Administrative Region, China (Project No. AoE/E-02/08). The manuscript is available online in [1] at http://goo.gl/slRFN, covering the related work in [2].

observed by all users, the problem becomes the one proposed in [3]. If instead the secret source is the source observed by one of the active users, the problem is the one considered in [4], which showed that the capacity is not diminished by such restriction. In the general case, the capacity can become strictly smaller and so it is interesting to characterize it in terms of the statistics of the random sources. It is also important to characterize some performance metrics other than the capacity. The error exponent and the secrecy exponent, for instance, refer to the exponential rates at which the error probability in recovering the secret source and the amount of leaked information of the secret key decay to zero respectively. It is desirable to have these exponents large and the public discussion rate small because the computational complexity and the overall delay of the system are both increasing in the block length and the public discussion rate. While [3] showed that any secret key rate below capacity could be achieved with positive error and secrecy exponents, the bounding technique does not yield a good secrecy exponent. In particular, for the case in [4] where the key is restricted to be a function of an active user, it was not known whether all rates below capacity are strongly achievable with positive secrecy exponents. It is therefore interesting to characterize the optimal tradeoff between the achievable exponents and rates, and to show whether strong achievability is possible. We find that this can be done indeed by extending the privacy amplification theorem in [5], [6] to the multiterminal case. If the secret source can be completely hidden from the wiretapper, its entropy rate must equal the capacity of the system because the entire randomness in the secret source can be used as a secret key. This is possible in some cases. A trivial example is when the secret source is part of the source observed by each active user. The active user can obtain the secret source perfectly without any public discussion because they all observe it directly. The problem of deciding whether a function is securely computable was proposed and solved in [7]. A sufficient and a slightly different necessary condition were derived with the help of the aided secret key agreement problem, also proposed in [7]. The necessary condition is that the entropy rate of the secret source is no larger than the maximum rate of a secret key that all users, including the non-active users, can agree on if a genie reveals the secret source to the non-active users after the entire public discussion session. We will see that this condition indeed coincides with the necessary condition that directly compares the entropy rate with the capacity of the system described earlier. The sufficient condition in [7] is obtained by a public discussion scheme that is required to solve the aided secret key agreement problem

2

in addition to the original secure computation problem. The bounding technique extends that of [3], which does not give a good secrecy exponent. This motivates us to consider the more direct approach using the privacy amplification theorem, which turns out to give good exponents. In the sequel, we will formulate the problem in §II, derive an inner bound to the achievable rates and exponents in §III, give the converse results in §IV, and extend the privacy amplification theorem to the multiterminal case in §V. Finally, we will briefly describe some possible extensions in §VI. More details can be found in the paper in [1] at http://goo.gl/slRFN. II. S YSTEM MODEL Let V denote the finite non-empty set of users. User i ∈ V observes the discrete memoryless source Zi taking values from the finite set Zi . The entire source is denoted by ZV ∶= (Zi ∶ i ∈ V ) which is correlated according to the joint distribution PZV . A possibly empty subset D ⊆ V of the users are untrusted in the sense that all their observations including the source ZD are revealed to the wiretapper. A non-empty subset A of the trusted users Dc ∶= V ∖ D are called the active users. They want to recover a secret source G which is a deterministic function of ZV characterized by the joint distribution PGZV . The remaining users in Dc ∖ A, if any, are the helpers. For the active users to learn the secret source, every user discuss in public over a noiseless authenticated channel as in [3], [8], which allow interaction and randomization. More precisely, the users have access to a public randomization U0 independent of ZV . Then, each user i ∈ V generates a private randomization Ui conditionally independent over i ∈ V given U0 and independent of ZV . i.e. PU0 UV ∣ZV = PU0 ∏ PUi ∣U0

(1)

i∈V

(U0 , UV ) can be continuous random variables as in [8] and so the above distributions are the probablity density functions. After the randomization, each user i ∈ V observes an nsequence Zni ∶= (Zit ∶ t ∈ [n]) of his private source where [n] ∶= {1, . . . , n} for any positive integer block length n. Then, ˜1 , F ˜ 2 , . . . ) of public messages they reveal a sequence F ∶= (F ˜ j is chosen by some one-by-one, where the j-th message F user ij ∈ V as a function of his accumulated observations, ˜ j ′ for j ′ < j. For convenience, the namely U0 , Uij , Znij , and F collection of messages from user i ∈ V is denoted by Fi taking values from a finite set Fi . The message rate is defined as ri ∶= lim sup n→∞

1 log∣Fi ∣ n

(2)

where all logarithms are taken with base 2 unless stated otherwise. The collection of public message rates is rV ∶= (ri ∶ i ∈ V ) and the total discussion rate is r(V ) ∶= ∑i∈V ri . After the public discussion, each active user i ∈ A computes ˆ i of the the secret source Gn from his accumuan estimate G lated observations U0 , Uij , Znij , and F. The error probability is denoted as εin ∶= Pr {Gˆi ≠ Gn } (3)

The system should be designed such that the error probability decreases to zero asymptotically as n goes to infinity. The convergence rate is captured by the error exponent defined as 1 Ei ∶= lim inf − log εin (4) n→∞ n The collection of error exponents is EA ∶= (Ei ∶ i ∈ A). There is a wiretapper who attempts to learn the secret source from the public discussion F and his side information about the private observations of the untrusted users, namely W ∶= (U0 , UD , ZnD ). The secrecy level of the system is measured by the secrecy index for G defined as 1 ςnG ∶= I(Gn ∧ U0 UD F∣ZD ) n 1 (5) = I(Gn ∧ FDc ∣W) n where I denotes the mutual information [9]. The last equality follows from (1) and the fact that FD is determined by W and FDc . Note that secrecy index is 0 iff Gn is conditionally independent of (U0 UD F) given ZnD . It is not meaningful to include the information leaked through ZnD here because that is fixed and cannot be diminished by the design of the system. The secrecy exponent is defined as 1 SG ∶= lim inf − log ςnG (6) n→∞ n If ςnG and εin for i ∈ V go to zero asymptotically in n, then the secret source is said to be securely computable (except for the part that depends on ZD ). In this case, (rV , EA , SG ) is said to be achievable. It is strongly achievable if the exponents are positive, i.e. SG > 0 and Ei > 0 for all i ∈ A. While it is clear that the error probabilities can decay to zero by choosing the message rates sufficiently large [9], it may not be possible to have the secrecy index for G go to zero simultaneously. This happens, for instance, when the secret source is observed only by one helper while the private sources for other users are deterministic. The helper has to reveal information about the source in public or the active user cannot recover the source reliably. In such cases, it is natural to go for a weaker secrecy requirement of hiding as much information about the secret source as possible. More precisely, choose some deterministic function K of the secret source Gn taking values over a finite set K. The rate is 1 R ∶= lim inf log∣K∣ (7) n→∞ n The secrecy index for K is defined as 1 ςnK ∶= [log∣K∣ − H(K∣FW)] (8) n which is zero iff K is uniformly distributed independent of F and W. R should be chosen sufficiently small so that the secrecy index for K goes to zero in n. The convergence rate is captured by the secrecy exponent 1 SK ∶= lim inf − log ςnK (9) n→∞ n (R, rV , EA , SK ) is said to be achievable if ςnK and εin for i ∈ V go to zero asymptotically in n. It is strongly achievable if all the exponents are positive. K is referred to as a the secret key and the secrecy capacity is the maximum achievable key rate.

3

III. ACHIEVABLE RATES AND EXPONENTS As usual, the inner bound to the achievable rates and exponents can be obtained using a random coding argument. For i ∈ V , let {θli ∶ l ∈ Li } be a family of functions θli ∶ Zin ↦ Fi indexed by Li . A function is uniformly randomly chosen from this family to generate the public message Fi from user i ∈ V . i.e. Fi ∶= θLi i (Zni ) where LV ∶= (Li ∶ i ∈ V ) is uniformly random independent of the source ZnV . LV is known to everyone including the wiretapper. As in [5], the families of functions are further restricted to be 2-universal in the sense that for all i ∈ V 1 ˜ i ∈ Zin Pr {θLi i (z i ) = θLi i (˜ z i )} ≤ ∀z i ≠ z ∣Fi ∣ This means that two different source realizations are likely to map to two different realizations of the public message. In other words, the probability of a collision is small. It can be shown easily that the random coding approach covers the well-known random binning and random linear coding with the family of functions chosen to be the set of all functions and the set of all linear functions respectively. The resulting code is expected to be good because a small collision probability tends to make the public messages uniformly distributed independent of the secret source, while simultaneously help distinguish the source for the active users, who possess additional side information from their private sources. Consider the precise error analysis first. For each active user i ∈ A, the error probability (3) in recovering the secret source Gn is upper bounded by the error probability in recovering the entire source ZnV since Gn is a function of ZnV . Thus, for any given message rate tuple rV ≥ 0, the error exponent (4) for the error probability averaged over LV can be lower bounded by the random coding exponent [9] +

E i (rV ) ∶= min D(QZV ∥PZV ) + ∣ min ΥQZV (B)∣ QZV ∈P(ZV )

(10)

B⊆V ∶i/∈B≠∅

+

with ∣⋅∣ ∶= max{0, ⋅} and ΥQZV (B) ∶= r(B) − H(QZB ∣ZBc ∣QZBc )

(11)

where QZV is a distribution chosen from the simplex P(ZV ) of all distributions on ZV , and D(⋅∥⋅) and H(⋅∣⋅) are the information divergence and conditional entropy respectively [9]. Since D(QZV ∥PZV ) ≥ 0 with equality iff QZV = PZV , the exponents for all i ∈ A are positive if ΥPZV (B) > 0 for all B ⊆ V ∶ B ⊉ A. i.e. r(B) > H(ZB ∣ZB c )

∀B ⊆ V ∶ ∅ ≠ B ⊉ A

(12)

To obtain the achievable secrecy exponent (6) for G, we first bound the secrecy index (5) as follows. ςnG ≤ log∣FDc ∣ − H(FDc ∣ZnD Gn LV ) For any given rV ≥ 0, the following lower bound on the secrecy exponent can be obtained by extending the privacy amplification in [5], [6] to the multiterminal case as to be shown in Theorem 8 in §V. +

S G (rV ) ∶= min D(QGZV ∥PGZV ) + ∣ mincΓQGZV (B)∣ QGZV ∈P(GZV )

B⊆D ∶B≠∅

(13)

where ΓQGZV (B) ∶= D(QZB ∣ZD G ∥PZB ∣ZD G ∣QGZD ) + H(QZB ∣ZD G ∣QGZD ) − r(B)

(14)

More precisely, we have applied Theorem 8 with FV replaced by FDc and G replaced by (G, ZD ) for this particular case. The above exponent is positive if r(B) < H(ZB ∣ZD G)

∀B ⊆ Dc ∶ B ≠ ∅

(15)

In summary, we have the following inner bound to the achievable rates and exponents. Theorem 1 (rV , EA , SG ) is strongly achievable if 0 < Ei ≤ E i (rV ) defined in (10) for all i ∈ A and 0 < SG ≤ S G (rV ) defined in (13). This is possible if rV satisfies (12) and (15).2 The secret source is securely computable if the linear system in rV defined by (12) and (15) is feasible. This sufficient condition turns out to be equivalent to that in [7] as shown in the following theorem. It is also possible to simplify the system by removing some constraints without affecting the feasibility of the system. Theorem 2 The secret source is securely computable with positive error and secrecy exponents if rDc exists satisfying r(B) > H(ZB ∣ZB c ) r(B) < H(ZB ∣ZD G)

∀B ⊆ Dc ∶ ∅ ≠ B ⊉ A (16a) c ∀B ⊆ D ∶ B ≠ ∅ = B ∩ A (16b)

r(Dc ) < H(ZDc ∣ZD G)

(16c)

which, in turn, is feasible if (and only if) H(G∣ZD ) < H(ZDc ∣ZD ) − min r′ (Dc ) ′

where

(17a)

∀B ⊆ Dc ∶ B ⊉ A ∀B ⊊ Dc ∶ B ⊇ A

(17b) (17c)

rD c

r′ (B) ≥ H(ZB ∣ZB c ) r′ (B) ≥ H(ZB ∣ZB c G)

In particular, the secret source can be securely computed with public message rates rV for any rDc in the closure of the solutions to (16) and rD chosen sufficiently large. 2 P ROOF Consider proving the first implication. Suppose rDc is a solution to (16). (16a) implies (12) with ri chosen sufficiently large for i ∈ D. (16b) implies part of (15). The remaining part where B ⊆ Dc ∶ B ∩ A ≠ ∅ is also satisfied as r(B) equals (a)

r(Dc )−r(Dc ∖ B) < H(ZDc ∣ZD G) − H(ZDc ∖B ∣ZB∪D ) (18) = H(ZB ∣ZD G) where, in (a), we have used (16c) to upper bound r(Dc ) and (16a) to lower bound r(Dc ∖ B) with the fact that Dc ∖ B ⊉ A and the convention that H(Z∅ ∣ZD ) = 0. Hence, there exists rV satisfying both (12) and (15), implying that the secret source is securely computable by Theorem 1. Indeed, the above argument can be reversed to show that (16) is equivalent to the linear system with (12) and (15). ′ Consider proving the second implication. Suppose rD c is an optimal solution satisfying (17). By (17a), there exists δ > 0 satisfying H(G∣ZD ) < H(ZDc ∣ZD ) − r′ (Dc ) − ∣Dc ∣δ or equivalently r′ (Dc ) + ∣Dc ∣δ < H(ZDc ∣ZD G). With ri ∶= ri′ + δ for i ∈ Dc , the above inequality implies (16c). Furthermore,

4

(17b) implies (16a). (17c) is also satisfied because (18) holds for all B ⊆ Dc ∶ B ≠ ∅ ≠ B ∩ A. More precisely, the inequality (a) in (18) can be obtained by upper bounding r(Dc ) using (16c) argued earlier and lower bounding r(Dc ∖ B) first by r′ (Dc ∖ B) and then by (17c). Hence, (16) is feasible if (17) is. Indeed, the converse is also true. Given a solution rDc to (16), we can define for some δ > 0 and some active user j ∈ A that ri′ ∶= ri − δ for all i ∈ Dc ∖ {j} and ′ rj′ ∶= H(ZDc ∣ZD G) − r(Dc ∖ {j}). It can be shown that rD c is a feasible solution to (17) with δ sufficiently small. ∎

replaced by ZD . S K (rV ) ∶=

min

QGZV ∈P(GZV )

+ ∣min {

D(QGZV ∥PGZV )

B⊆D ∶B≠∅

Γ1QZ (B), minc Γ2QGZ (B)}∣ V

B⊆D

V

where Γ1QZ (B) ∶= D(QZB ∣ZD ∥PZB ∣ZD ∣QZD ) V

+ H(QZB ∣ZD ∣QZD ) − r(B) Γ2QGZ (B) ∶= D(QGZB ∣ZD ∥PGZB ∣ZD ∣QZD ) V

(17) is the sufficient condition obtained in [7]. The R.H.S. of (17a) is the secrecy capacity of the aided secret key agreement problem considered in [7] for the non-trivial case ∣V ∣ ≥ 2. With D = ∅, the optimal solution rV′ to (17a) gives the minimum discussion rate r′ (V ) for the achieving scheme in [7]. However, this tends to be larger than necessary as illustrated by the following simple example. Example 1 Consider the 2-user case with V = {1, 2}, A = {1}, D = ∅, Z1 = (Y1 , Y2 , Y3 ), Z2 = Y2 and G = Y1 where Y1 , Y2 and Y3 are independent uniformly random bits. Since there is only one active user who observes the secret source as part of his private source, no public discussion is required. Choose r1 = r2 > 0 arbitrarily small. (16a) is satisfied since H(Z2 ∣Z1 ) = 0 < r2 . (16b) is also satisfied since H(Z2 ∣G) = 1 > r2 . Finally, H(Z1 Z2 ∣G) = 2 > r1 + r2 which implies (16c). Thus, (16) is feasible with r1 + r2 arbitrarily small. For (17), however, it is not possible to achieve arbitrarily small sum rate because r1′ ≥ H(Z1 ∣Z2 G) = 1 by (17c). The optimal solution is (r1′ , r2′ ) = (1, 0) and so (17a) is also satisfied with the R.H.S equal to 2. This corresponds to the capacity of the aided secret key agreement problem [7] that user 1 and 2 can share two secret key bits (Y1 , Y2 ) if G were revealed to the non-active user, namely user 2. If Y3 = 0 deterministically instead, both (16) and (17) are feasible with r(Dc ) and r′ (Dc ) arbitrarily close to 0. Thus, the discussion rate can be zero in both cases. If Y2 = Y3 = 0 deterministically instead, it can be shown that (16) and (17) are not feasible. However, the secret source is still securely computable because it is directly observed by the active user. This means that the sufficient condition given here may not be necessary. Indeed, the sufficient condition can be strengthened in this case using the network coding approach in [10]. 2 To compute the achievable secrecy exponent (9) and rate (7) for the secret key K, we again consider a random coding scheme of assigning K ∶= θL (Gn ) where {θl ∶ l ∈ L} is a 2-universal family of functions θl ∶ Gn ↦ K, and L is a uniformly random index independent of (ZV , LV ) and known by everyone. We first bound the secrecy index (8) as follows. ςnK ≤ log∣K∣∣FDc ∣ − H(KFDc ∣ZnD LV L) = log∣K∣∣FDc ∣ − H(KFDc ∣ZnD LDc L) because LD is independent of (K, ZV , L, LDc ) and therefore FDc , which is a function of ZDc and LDc . The following lower bound on the secrecy exponent can be obtained for any rV ≥ 0 by applying Theorem 8 with FV replaced by (K, FDc ) and G

(19)

+

min c

+ H(QGZB ∣ZD ∣QZD ) − r(B) − R

(20a) (20b)

The exponent is positive if r(B) < H(ZB ∣ZD ) R + r(B) < H(GZB ∣ZD )

∀B ⊆ Dc ∶ B ≠ ∅ ∀B ⊆ Dc

(21a) (21b)

Hence, we have the following inner bound to the achievable rates and exponents for the secret key. Theorem 3 (R, rV , EA , SK ) is strongly achievable if 0 < Ei ≤ E i (rV ) in (10) for all i ∈ A and 0 < SK ≤ S K (rV ) in (19). This is possible if rV satisfies (12) and (21). 2 It follows that the key rate R is strongly achievable if the linear system in rV defined by (12) and (21) is feasible. Similar to Theorem 2, the system can be simplified as follows. Theorem 4 The key rate R ≥ 0 is strongly achievable if rDc exists satisfying r(B) > H(ZB ∣ZB c ) r(B) < H(ZB ∣ZD ) R + r(B) < H(GZB ∣ZD ) R +r(Dc )< H(ZDc ∣ZD )

∀B ⊆ Dc ∶ ∅ ≠ B ⊉ A

(22a)

∀B ⊆ D ∶ B ≠ ∅ = B ∩ A (22b) ∀B ⊆ Dc ∶ B ∩ A = ∅ (22c) (22d) c

which, in turn, is feasible for all R ≤ H(G∣ZD ) if (17) is.

2

P ROOF The first implication can be proved in the same way as in the proof of Theorem 2. Consider proving the second implication. Note that R ≤ H(G∣ZD ) is the case of interest because the secret key should only depend on the portion of the secret source Gn that is independent of the wiretapper’s observation ZnD . Suppose rDc is a solution to (16). Then, (16b) implies (22c) under the assumption that R ≤ H(G∣ZD ). (16a) implies (22a) trivially and (16b) implies (22b) since H(ZB ∣ZD G) ≤ H(ZB ∣ZD ). In other words, rDc solves (22).∎ It will be shown that the achievable key rates given by (22) attains the secrecy capacity when there is no helper, i.e. A = Dc . If there are helpers, however, the capacity may not be attained as illustrated by the following example. Example 2 Consider the two-user case defined in Example 1 but with Y3 = 0 deterministically. It is clear that the secrecy capacity is 1 because the active user observe the secret source G = Y1 directly. (22) in this case is r2 > H(Z2 ∣Z1 ) = H(Y2 ) r2 < H(Z2 ) = H(Y2 ) R < H(G) = H(Y1 ) R + r1 + r2 < H(GZ1 Z2 ) = H(Y1 Y2 )

(22a), B = {2} (22b), B = {2} (22c), B = ∅ (22d), B = {1, 2}

5

It is solved by (R, r1 , r2 ) = (1 − δ, 0, δ) with δ > 0 arbitrarily small. The secrecy capacity is therefore achievable asymptotically with no public discussion rate. Consider Y2 = Y3 = 0 deterministically while Y1 remains uniformly random. The capacity is still equal to 1. However, the first two constraints from (22) requires that r2 > 0 and r2 < 0 respectively. This is infeasible and so (22) does not give any achievable key rate. 2 Fortunately, there is an alternative argument that can enlarge the set of achievable key rates. Imagine there is a virtual user who observes the secret source and the entire public discussion. The other users first discuss in public until all active users learn the entire source reliably. This can be done with message rates rV satisfying the Slepian-Wolf constraints [9] r(B) ≥ H(ZB ∣ZB c )

∀B ⊆ V ∶ B ⊉ A

(23a)

Now, one of the active users, say j ∈ A, sends out a virtual public message at rate u ≥ 0 such that the virtual user can recover the entire source reliably. This can be done with the additional Slepian-Wolf constraints that u + r(B) ≥ H(ZB ∣ZB c G)

∀B ⊆ V ∶ B ⊇ A

(23b)

The other required constraints are implied by (23a), i.e.

P ROOF (24) can be derived from (23) using the same argument in the proof of Theorem 2. Note that (24) is different from (17) because of the additional constraint (24c) for B = Dc . Indeed, the R.H.S. of (24a) is the desired secrecy capacity by the converse in Theorem 6 in the next section. It is therefore non-negative. Consider proving the last implication. Suppose the R.H.S. of (24a) is strictly positive, A = Dc and (R′ , rDc ) is a solution to (24) with R replaced by R′ . It suffices to show that R ∶= R′ − 2∣Dc ∣δ solves (22) with some choice of rDc for δ > 0 arbitrarily small. This would imply the desired result that R′ is a limit point of the set of feasible solutions R to (24). Let ri ∶= ri′ + δ. (24a) with R replaced by R′ implies that R′ − 2∣Dc ∣δ ≤ H(ZDc ∣ZD ) − r′ (Dc ) − ∣Dc ∣δ (a)

R < H(ZDc ∣ZD ) − r(Dc ) (b)

< H(ZDc ∣ZD ) − H(ZDc ∣ZD G) = H(G∣ZD )

where (a) is because δ > 0, and (b) is because (24c) implies that H(ZDc ∣ZD G) ≤ r′ (Dc ) < r(Dc ). (a) and (b) imply (22d) and (22c) respectively. (24b) implies (22a). (22b) trivally holds since it is imposed on an empty set by the assumption that A = Dc . This completes the proof. ∎

u + r(B) ≥ r(B) ≥ H(ZB ∣ZB c ) ≥ H(ZB ∣ZB c G)

IV. C ONVERSE RESULTS

for all B ⊆ V ∶ B ⊉ A. The additional requirement that the virtual user can recover ZnV from Gn and F means that H(Gn ∣FZnD ) is approximately H(ZnV ∣FZnD ). In other words, the randomness in Gn hidden from the wiretapper is essentially the entire randomness in ZnV hidden from the wiretapper. Using the same argument as in [4], this randomness can be extracted as a secret key chosen purely as a function of Gn at rate

We first give the converse proof for the maximum achievable rate in Theorem 5, using Shearer-type lemmas as in [4], [8].

0 ≤ R ≤ H(ZDc ∣ZD ) − r(Dc ) − u

(23c)

since n1 H(ZnV ∣FZnD ) = H(ZDc ∣ZD ) − n1 H(F∣ZnD ) and the last term is at most r(Dc ) + u. The details can be found in [4]. A rather subtle observation is that the total public discussion rate can be r(V ) instead of r(V ) + u because the virtual message needs not be revealed. By construction, all active users in A can recover the entire source and therefore the secret source before the virtual message is sent. The virtual user and message are only introduced to affect how the other public messages should be chosen. They need not appear in the actual system. In summary, we have the following achievable rate region for the secret key. Theorem 5 (R, rV ) is achievable if (23) is satisfied for some u ≥ 0. It follows that R is achievable if rDc exists with 0 ≤ R ≤ H(ZDc ∣ZD ) − min r′ (Dc ) ′

where

(24a)

∀B ⊆ Dc ∶ B ⊉ A ∀B ⊆ Dc ∶ B ⊇ A

(24b) (24c)

rDc

r′ (B) ≥ H(ZB ∣ZB c ) r′ (B) ≥ H(ZB ∣ZB c G)

The R.H.S. of (24a) is always non-negative. If it is strictly positive and there is no helper, i.e. A = Dc , the set of feasible R for (24) is in the closure of the set of feasible R for (22). n.b. strong achievability is also proved in [11]. 2

Theorem 6 The R.H.S. of (24a) is the secrecy capacity. If there is no helper, i.e. A = Dc , any rate below the capacity is strongly achievable with the achievable rates and exponents given by Theorem 3. 2 P ROOF The result follows from Theorem 5 if we can prove that any rate above the R.H.S. of (24a) cannot be achieved. To do so, we consider the general model given in §II. By (8), the secret key rate n1 log∣K∣ is upper bounded by n1 H(K∣FW)+ςnK . Since ςnK is required to go to zero, the key rate is asymptotically upper bounded by n1 H(K∣FW). H(K∣FW) = H(K∣FZnV UV ) + I(K ∧ UDc ZnDc ∣FW) = H(UDc ZnDc ∣FW) − H(UDc ZnDc ∣KFW) where we have expanded I(K ∧ UDc ZnDc ∣FW) and removed H(K∣FZnV UV ) since ZnV determines Gn and therefore K. Consider some fractional partition λ ∶= (λB ∶ B ⊆ Dc ) where λB ≥ 0 and ∑B∋i λB = 1 for all i ∈ Dc . Then, by Shearer-type lemma [8, Lemma A.1, (A.10a)], H(UDc ZnDc ∣KFW) ≥ ∑ λB H(UB ZnB ∣KFZnB c UB c U0 ) B

For B ⊉ A, the conditional in the last entropy term contains the complete knowledge of at least one active user, i.e. (F, Zni , Ui , U0 ) for some i ∈ A. Thus, K can be removed from the conditional without changing the entropy too much because K is determined by G, which can be recovered almost surely by every active user. i.e. there exists δn → 0 such that H(UB ZnB ∣KFZnB c UB c U0 ) ≥ H(UB ZnB ∣FZnB c UB c U0 ) − δn

6

by Fano’s inequality. For the remaining B ⊇ A, we can bound the entropy simply by replacing K with Gn , i.e. H(UB ZnB ∣KFZnB c UB c U0 ) ≥ H(UB ZnB ∣Gn FZnB c UB c U0 ) This holds again because K is determined by Gn . Applying these bounds on H(K∣FW) with the expansion H(UB ZnB ∣FZnB c UB c U0 ) = H(UV ZnV ∣U0 ) − H(UB c ZnB c F∣U0 ) we have the following upper bound on the key rate (a)

1 n

log∣K∣

(b)

³¹¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹· ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ µ ³¹¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ · ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ µ 1 [ ∑ λB H(UB c ZnB c F∣U0 ) + ∑ λB H(UB c Gn ZnB c F∣U0 ) n B⊉A B⊇A − H(UD ZnD F∣U0 ) −(∑B λB − 1) H(UV ZnV ∣U0 ) ] + δn′ ´¹¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¸¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¶ ´¹¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹¸ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¶ (d)

(c)

for some δn′ → 0. The entropies can be expanded casually as (a) = =

H(UB c ∣U0 ) + H(ZnB c ∣UB c U0 ) + H(F∣ZnB c UB c U0 ) H(UB c ∣U0 ) + nH(ZB c ) + H(F∣ZnB c UB c U0 )

by the independence constraint (1). Similarly, (b) = H(UB c ∣U0 ) + nH(GZB c ) + H(F∣ZnB c Gn UB c U0 ) ≤ H(UB c ∣U0 ) + nH(GZB c ) + H(F∣ZnB c UB c U0 ) (c) = H(UD ∣U0 ) + nH(ZD ) + H(F∣ZnD UD U0 ) (d) = H(UV ∣U0 ) + nH(ZV ) By Shearer-type lemma [8, Lemma A.1, (A.10b)] and the independence constraint (1), the following sum is 0. ∑ λB H(UB c ∣U0 ) − H(UD ∣U0 ) − (∑B λB − 1)H(UV ∣U0 ) = 0 B

Again by Shearer-type lemma [8, Lemma A.1, (A.10c)], H(F∣ZnD UD U0 ) ≥ ∑ λB H(F∣ZnB c UB c U0 )

can be hidden from the wiretapper. More precisely, since K is a function of G, the secrecy index (5) for G satisfies 1 1 ςnG = I(Gn ∧ FDc ∣W) ≥ I(K ∧ F∣W) n n G implying that H(K∣FW) + nςn ≥ H(K∣W) = H(K∣ZnD ) by (1). The secrecy index (8) for K therefore satisfies 1 1 ςnK = [log∣K∣ − H(K∣FW)] ≤ [log∣K∣ − H(K∣ZnD )] + ςnG n n Suppose G is securely computable, i.e. ςnG → 0 for some public discussion scheme. Then, ςnK → 0 if log∣K∣ − H(K∣ZnD ) → 0. This is possible for any key rate R ≤ H(G∣ZD ) by the privacy amplification theorem [6] as desired. Hence, we have established the desired necessary condition. Indeed, it is equivalent to the one given in [7] using the secrecy capacity (17a) for the aided secret key agreement problem. Theorem 7 The secret source is computable only if the secrecy capacity given by the R.H.S. of (24a) equals H(G∣ZD ). This is satisfied (if and) only if the R.H.S. of (17a) holds with non-strict inequality. 2 P ROOF Consider proving the second implication. Suppose ′ rD is an optimal solution to (24) with H(G∣ZD ) = c H(ZDc ∣ZD ) − r′ (Dc ). (24b) and (24c) imply (17b) and (17c) respectively. Thus, the R.H.S. of (17a) is at least H(ZDc ∣ZD )− r′ (Dc ), which is at least H(G∣ZD ) as desired. ′ To prove the converse, suppose rD c is an optimal solution to (17) but with the non-strict inequality H(G∣ZD ) ≤ H(ZDc ∣ZD ) − r′ (Dc ) in place of (17a). Increase rj′ for some j ∈ A until the inequality is satisfied with equality, which implies (24c) for B = Dc . The other constraints in (24b) and ′ (24c) are also satisfied by (17b) and (17c). Thus, rD c is a feasible solution to (24) with H(G∣ZD ) = H(ZDc ∣ZD ) − r′ (Dc ). Indeed, r′ (Dc ) is minimal by (24c) with B = Dc as desired.∎

B

Substituting these bounds into the terms (a) to (d), we have the following upper bound on lim supn→∞ n1 log∣K∣. ⎧ ⎪ B⊉A ⎪H(ZB ∣ZB c ) min H(ZDc ∣ZD ) − ∑ λB ⎨ ⎪ λ c H(Z ∣Z G) B⊇A c ⎪ B B B⊆D ⎩ where the minimization is over the fractional partition λ. By the strong duality theorem [12], this linear program equals the R.H.S. of (24a) as desired. ∎ The above theorem summarizes and extends some results in [4]. Consider the case when G = Zj for some active user j ∈ A. (24c) holds iff for all B ⊆ Dc ∶ j ∈/ B ⊇ A, rj′ + r′ (B) ≥ H(ZB∪{j} ∣ZB c ∖{j} G) = H(ZB ∣ZB c ) where the last equality is because G = Zj . This, in turn, is implied by (24b) and so (24c) is redundant. When ∣A∣ ≥ 2, the capacity given here is identical to the secrecy capacity [3] when the secret key is not restricted to be a function of Zj . This fact was pointed out in [4] but the fact that any rates below the capacity can be strongly achievable was not known. If the secret source is securely computable, the secrecy capacity should be H(G∣ZD ) since the entire secret source

V. M ULTITERMINAL PRIVACY AMPLIFICATION In this section, we will extend the privacy amplification theorem in [5], [6] to the multiterminal case. Given two random variables Z and G distributed over the finite sets Z and G respectively, the purpose of privacy amplification is to find a function F of Z that is nearly independent of G. A trivial solution is to have F deterministic but this is not very useful. It is desirable to have F as random as possible while maintaining a good level of secrecy. If we measure the secrecy by the index log∣F ∣ − H(F∣G), then there is a tradeoff between secrecy and ∣F ∣. In [5], a random coding scheme is used to derive an achievable tradeoff as follows. Let {θl ∶ l ∈ L} be a 2-universal family of functions θl ∶ Z ↦ F satisfying 1 Pr {θL (z) = θL (˜ z )} ≤ ∀z ≠ z˜ ∈ Z (25) ∣F ∣ where L is a uniformly random index independent of (Z, G). F is then computed from a function uniformly randomly chosen from this family, i.e. F ∶= θL (Z). The choice L of the function is known to everyone and so the secrecy is measured by ς ∶= log∣F ∣−H(F∣GL) = E [log (∣F ∣PF∣GL (F∣G, L))] ≤ E [log (∣F ∣ E [PF∣GL (F∣G, L)∣G, Z])]

7

where the last inequality is by Jensen’s inequality. The inner expectation is over L only, while the outer expectation is over G and Z. For any (z, g, l) ∈ Z × G × L, the conditional probability PF∣GL (θl (z)∣g, l) in the inner expectation is PZ∣G (z∣g) +

z ∣g)χ{θl (z) = θl (˜ z )} ∑ PZ∣G (˜

Averaging this over l and applying (25), 1 ∣F ∣

which gives the following upper bound.

[5] further bounds this using R´enyi entropy of order 2 and [6] improves the bound using R´enyi entropy of order optimized over (1, 2]. We will bound this more directly using the inequality that for all x > 0 log(1 + x) ≤ min{1, x} max{log e, log(1 + x)}

(26)

where e is the natural number. This follows from the fact that log(1 + x) ≤ x log e for all x ≥ 0 and log(1 + x) ≤ log ex for x ≥ 1. Applying (26) to the bound on ς with the fact that ∣F ∣ ≥ 1, we have

This turns out to give the same exponent as in [6] for the case with discrete memoryless sources. More precisely, replace Z, G and F ∶= θL (Z) by the vector case Zn , Gn and F ∶= θL (Zn ) with rate R ∶= lim supn→∞ n1 log∣F ∣. . n ς ∶= log∣F ∣ − H(F∣Gn L) ⩽ E [min{1, ∣F ∣PZ∣G (Zn ∣Gn )}] . where ∶= ∏t∈[n] PZ∣G (Zt ∣Gt ) and a(n) ⩽ b(n) denotes the inequality in the exponents that n1 log a(n) ≤ 0. b(n) We can compute the exponent using the method of types [9] as follows. For every n-sequence (g, z) in the type class TQnGZ for some empirical distribution QGZ ∈ Pn (GZ) on G × Z, n PZ∣G (Zn ∣Gn )

=

∣TQnGZ ∣

−n[D(QGZ ∥PGZ )+H(QGZ )]

=2

. Denote equality in the exponents with a ≐ b, i.e. a ⩽ b and . n −nD(QGZ ∥PGZ ) n nH(QGZ ) b ⩽ a, we have PGZ (TQGZ ) ≐ 2 , ∣TQGZ ∣ ≐ 2 but ∣Pn (GZ)∣ ≐ 1. The desired bound on ς is . n ς⩽ PGZ (g, z) min{1, ∣F ∣PZ∣G (z∣g)} ∑ ∑ n QGZ ∈Pn (GZ) (g,z)∈TQ

QGZ

GZ

∑

n (g,z)∈TQ

n PGZ (g, z) min{1, ∣F ∣PZ∣G (z∣g)}

GZ

−n[

≐2

zV ∑ PZV ∣G (˜ z˜V ∈ZV ∶ z˜i =zi iff i∈B

∣g) ∏ χ{θlii (z) = θlii (˜ z )} i∈B c

E [PFV ∣GLV (fV ∣g, LV )] ≤

PZB ∣G (zB ∣g) 1 + ∑ ∣FV ∣ B⊆V ∶B≠∅ ∣FB c ∣

which gives the following upper bound. ς ≤ E [log (1 +

∑

B⊆V ∶B≠∅

PZB ∣G (zB ∣g)∣FB ∣)]

As before, consider the vector case by replacing G, Zi and Fi ∶= θLi i (Zi ) for i ∈ V by GnV , Zni and Fi = θLi i (Zni ) with lim supn→∞ n1 log∣Fi ∣ = ri . By the previous arguments, ∑

B⊆V ∶B≠∅

PZnB ∣G (ZnB ∣Gn )∣FB ∣}]

Finally, the secrecy exponent can be obtained as follows by the method of types. Theorem 8 For the multiterminal privacy amplification problem above, the exponent 1 log [log∣FV ∣ − H(FV ∣GLV )] n of the secrecy index is at least − lim inf n→∞

min

D(QGZV ∥PGZV )

QGZV ∈P(GZV )

+ ∣ min D(QZB ∣G ∥PZB ∣G ∣QG )+H(QZB ∣G ∣QG ) − r(B)∣

+

B⊆V ∶B≠∅

Thus, any non-negative exponents below this is achievable with some deterministic choice for LV . 2

n PZ∣G (z∣g) = 2−n[D(QZ∣G ∥PZ∣G ∣QG )+H(QZ∣G ∣QG )]

. ⩽ max

∑ B⊆V

. ς ⩽ E [min {1,

ς ≤ E [min{1, ∣F ∣PZ∣G (Z∣G)}] log(e∣F ∣)

PGZ (TQnGZ )

≤ E [log (∣FV ∣ E [PFV ∣GLV (FV ∣G, LV )∣G, ZV ])]

Averaging over li for each i ∈ V and applying the 2-universal property of the families of functions,

ς ≤ E [log (1 + (∣F ∣ − 1)PZ∣G (Z∣G))]

n PGZ (g, z)

ς ∶= log∣FV ∣ − H(FV ∣GLV ) For any (zV , g, lV ) ∈ ZV × G × LV , and fi ∶= θlii (zi ) for i ∈ V , the conditional probability PFV ∣GLV (fV ∣g, lV ) in the inner expectation is

z˜∈Z∶˜ z ≠z

E [PF∣GL (θl (z)∣g, L)] ≤ PZ∣G (z∣g) + (1 − PZ∣G (z∣g))

functions θli ∶ Zi ↦ Fi and Li is a uniformly random index independent of (ZV , G, LV ∖{i} ). The secrecy index is

+

min D(QGZ ∥PGZ )+∣D(QZ∣G ∥PZ∣G ∣QG )+H(QZ∣G ∣QG )−R∣ ]

QGZ ∈P(GZ)

+

where we have applied min{1, 2−x } = 2− max{0,x} = 2−∣x∣ . For the multiterminal case, consider ZV , G and Fi ∶= θLi i (Zi ) for all i ∈ V where {θli ∶ l ∈ Li } is a 2-universal family of

VI. E XTENSIONS In this work, we have proposed a framework for the problem of multiterminal secure source coding for a common secret source. It consists of a group of users who want to recover a secret source from their private observations by exchanging information in public. Ideally, the secret source should be computed securely without leaking information in public but this is not always possible. Instead of giving up entirely, the users try to hide as much information about the secret source as possible. We proposed a way to measure the level of imperfect secrecy using the rate and secrecy exponent tradeoff of a secret key that can be extracted from the secret source. The problem can then be viewed as the more general secret key agreement problem with a restricted choice of the key function [11],

8

unifying the previous model on secret key agreement [3] and secure computation [7]. We have derived bounds on the achievable rates, error and secrecy exponents, extending the previous results in [3], [4], [7]. Good secrecy exponents are obtained using a more direct approach of privacy amplification in [5], [6], which is extended here to cover the multiterminal case. When there is no helper, we have shown that any rate below the secrecy capacity can be achieved strongly with a key chosen purely as a function of the private source of any active user. This fact was not known in [4]. A sufficient condition and a slightly different necessary condition for the secret source to be securely computable are also derived and shown to match the ones in [7]. The necessary condition uses the secrecy capacity for the secret key agreement problem with restricted key, which appears to be more natural than that of the aided secret key agreement problem in [7]. Finding good outer bounds on the achievable secrecy exponents is an interesting but challenging problem. The inner bounds derived here can be loose because they include an unnecessary component that measures how non-uniform the public messages are distributed. In fact, we can improve the secrecy exponent for the secret key by allowing the public messages to be non-uniformly distributed as long as they do not reveal the key to the wiretapper. This is done in [11]. It shows as a corollary that any rates below the secrecy capacity can be achieved strongly even if there are helpers. The result is not covered by the current approach that uses the technique in [3] to prove Theorem 5. Although the bounding technique in [3] may not give a good secrecy exponent, it can be used to show existence of a scheme that is universal to a large class of observation models for the wiretapper. In particular, it can strengthen the universality of the secure network code considered in [13], which is done in [14]. It would be ideal to have a practical code for privacy amplification that gives perfect secrecy for a finite block length and that is universal to a large class of possible models for the wiretapper. This is possible for the special finite linear source model considered in [10], [15] using techniques from combinatorial optimization and network coding. It is also possible to apply the information identity in [16] to relate the secure source coding problem to certain undirected network coding problem. In particular, the capacity can be computed efficiently using the submodularity of entropy [17] as pointed out in [16]. R EFERENCES [1] C. Chan, publications. http://chungc.net63.net/pub, http://goo.gl/4YZLT. [2] ——, “Multiterminal secure source coding,” accepted by Allerton, 2011, see [1]. [3] I. Csisz´ar and P. Narayan, “Secrecy capacities for multiple terminals,” IEEE Transactions on Information Theory, vol. 50, no. 12, Dec 2004. [4] ——, “Secrecy capacities for multiterminal channel models,” IEEE Transactions on Information Theory, vol. 54, no. 6, pp. 2437–2452, June 2008. [5] C. H. Bennett, G. Brassard, C. Cr´epeau, and U. M. Maurer, “Generalized privacy amplification,” IEEE Transactions on Information Theory, vol. 41, no. 6, pp. 1915–1923, Nov 1995. [6] M. Hayashi, “Exponential decreasing rate of leaked information in universal random privacy amplification,” Information Theory, IEEE Transactions on, vol. 57, no. 6, pp. 3989 –4001, june 2011.

[7] H. Tyagi, P. Narayan, and P. Gupta, “When is a function securely computable?” CoRR, vol. abs/1007.2945, 2010. [8] C. Chan, “Generating secret in a network,” Ph.D. dissertation, Massachusetts Institute of Technology, 2010, see [1]. [9] I. Csisz´ar and J. K¨orner, Information Theory: Coding Theorems for Discrete Memoryless Systems. Akad´emiai Kiad´o, Budapest, 1981. [10] C. Chan, “Linear perfect secret key agreement,” in 2011 IEEE Information Theory Workshop Proceedings (ITW2011), Paraty, Brazil, Oct. 2011, see [1]. [11] ——, “Agreement of a restricted secret key,” see [1]. [12] G. B. Dantzig and M. N. Thapa, Linear Programming. 1: Introduction. Springer-Verlag New York, 1997-2003. [13] R. Matsumoto and M. Hayashi, “Secure multiplex network coding,” CoRR, vol. abs/1102.3002, 2011. [14] C. Chan, “Universal secure network coding by secret key agreement,” see [1]. [15] ——, “Delay of linear perfect secret key agreement,” accepted by Allerton, 2011, see [1]. [16] ——, “The hidden flow of information,” in 2011 IEEE International Symposium on Information Theory Proceedings (ISIT2011), St. Petersburg, Russia, Jul. 2011, see [1]. [17] R. W. Yeung, Information Theory and Network Coding. Springer, 2008.