1

Improved Tangential Sphere Bound on the ML Decoding Error Probability of Linear Binary Block Codes in AWGN Interference Amaan Mehrabian and Shahram Yousefi

Abstract— The error probability of Maximum-Likelihood (ML) decoded binary block codes rarely accepts nice closed forms. Hence, one usually has to resort to bounding techniques for such error probabilities. Tangential Sphere Bound (TSB) of Poltyrev is one of the tightest upper bounds on ML decoding of binary block codes up to date. TSB uses the so-called Gallager First Bounding Technique (GFBT) together with the well-known union bound. In this paper, we modify the TSB by applying a second-order Bonferroni-type inequality instead of union bound. This results in a very tight but complex upper bound as it needs the global geometrical properties of the code and only the distance spectrum will not suffice. Subsequently, we propose an upper bound which requires only the distance spectrum of the code. This simple version of the bound is the tightest upper bound on the ML performance of binary block codes. Index Terms— Additive White Gaussian Noise (AWGN) channel, decoding error probability, probability of error, Tangential Sphere Bound (TSB), upper bound.

I.

INTRODUCTION

R

ECENTLY, with the evolution of such coding techniques as Turbo codes and Low Density Parity Check (LDPC) codes, more attention has been drawn to performance evaluation methods for linear binary block codes. Particularly, the problem of performance evaluation of linear binary block codes with soft-decision Maximum Likelihood (ML) decoding in Additive White Gaussian Noise (AWGN) environment has long been a central problem in information and coding theory as usually there is no closed-form expression for the corresponding error probabilities. Hence, we often resort to tractable bounding techniques which are relatively easier to implement and faster than Monte-Carlo simulation. The most well-known upper bound in the literature of communication theory is the Union Bound which is a Bonferronitype inequality [1] in the probability theory. Bonferroni-type inequalities are a family of inequalities which are true regardless of the underlying probability space and for all choices of the basic events. Union bound is very easy to implement as it only needs the distance spectrum of the code and is very tight in high SNR range but performs poorly in low and medium SNR ranges making it inappropriate for the applications dealing with low SNR range. Another reason for the recent overwhelming attention given to bounding techniques, is the introduction of some nearShannon-limit performing codes. Turbo codes, invented by Berrou et al. [2] in 1993, Repeat-Accumulate (RA) codes of Divsalar et al. [3], and Low Density Parity Check (LDPC) codes of Gallager [4] re-introduced by Mackay et al. [5] This work was supported in part by Natural Sciences and Engineering Research Council of Canada (NSERC). The authors are with the Department of Electrical and Computer Engineering, Queen’s University, Kingston, ON, Canada K7L 3N6 (email:mehrabia, [email protected]).

in 1996, are the best examples. Utilizing union bound for analyzing these codes results in very loose measures of performance which again highlights the importance of addressing the bounding techniques. One of the tightest bounds known, is the Tangential Sphere Bound (TSB) of Poltyrev [6], which uses a general bounding technique developed by Gallager [4]. Divsalar [7] refers to this method as Gallager First Bounding Technique (GFBT). For long, TSB was actually the tightest upper bound on ML decoding error probability of linear binary block codes [8], [9]. In the development of TSB, one has to utilize the union bound which is known to be loose in low SNR range. In this paper, we propose another Bonferroni-type inequality to replace the union bound and hence improve the conventional TSB bound. It is also shown that this method yields further improvements when the underlying code spectrum is sparse, making it a suitable bound for the well-known LDPC codes. The rest of the paper is organized as follows: In section II, some preliminaries and the problem formulation are given. The conventional TSB is explained in section III. The so-called Improved TSB (ITSB) is proposed in section IV. Comparison of the two methods and conclusions are given in section V. II. P RELIMINARIES Consider an [n, k] linear binary block code C = {c0 , c1 , ..., c2k −1 } to be used with Binary Phase-Shift Keying (BPSK) modulation. The resulting signal set will be S= {s0 , s1 , ..., s2k −1 }

where si √= m(ci ) = (m(ci1 ), m(ci2 ), ..., m(cin )), and m(α) = Es (2α − 1), α ∈ {0, 1} and Es is the symbol energy1. We denote the Hamming distance between two codewords ci and cj by dij . We assume an AWGN channel, hence the received signal can be expressed as: r = si + n

(1)

where n is an n-dimensional vector whose elements are independent zero-mean Gaussian random variables with a variance of σ 2 . The probability of word error will be Pw (E) =

k 2X −1

P (E|si )P (si ) = P (E|si )

i=0

1 Without

loss of generality, Es will be chosen to be unity.

(2)

2

Gallager region (cone)

where the second equality is valid for the case of equiprobable and geometrically uniform signal constellations and si can be any signal point. We assume that s0 , signal corresponding to the all-zero codeword, c0 , has been transmitted. transmitted.

δj0 /2 z2

III. TANGENTIAL S PHERE B OUND (TSB) A class of tight upper bounds use a general bounding technique developed by Gallager” [4] . The main idea of this technique is to bisect the error probability to the joint probability of error and noise residing in a region R plus the joint probability of error and noise residing in the complement region of R. Divsalar [7] refers to this technique as Gallager First Bounding Technique (GFBT). Mathematically: Pw (E) = P (E, r ∈ R) + P (E, r 6∈ R) = P (E, r ∈ R) + P (E|r 6∈ R)P (r 6∈ R) ≤ P (E, r ∈ R) + P (r 6∈ R).

Z

βj (z1 ) Sj z1

r(z1 )

√

O

n

(3)

R, which is a region around the transmitted codeword, is referred to as Gallager region. In order to develop the TSB, At the first step, we separate the radial component of the noise, z1 , which is merely the component along the − s0→ o direction (See Fig. 1). It is clear that we have: P (E) =

S0

Fig. 1. √ Geometry of the TSB; s0 and sj are both on a hyper-sphere with radius n.

+∞

P (E|z1 )fz1 (z1 )dz1 .

(4)

and

√ n − z1 q βk (z1 ) = n dk − 1

−∞

Now applying GFBT to P (E|z1 ) will yield : P (E|z1 ) ≤ P (E|z1 , r ∈ R) + P (r 6∈ R).

(5)

Considering the geometry of the problem in Fig. 1, it can be seen that the hyper-cone (which is the chosen Gallager region suggested by poltyrev [6]) has azimuthal symmetry along the radial axis, hence the cross sections of the Gallager region along this axis will be hyper-spheres in (n − 1)-dimensional space. Therefore, the condition
Pn (r ∈ R) will be simplified to y ≤ r2 (z1 ) where y = i=2 zi2 , and r(z1 ) is the radius of the sphere cross section of the Gallager region at z1 . Hence (5) can be rewritten as P (E|z1 ) ≤ P (E|z1 , y ≤ r2 (z1 )) + P (y > r2 (z1 ))

(10)

is the projection of the perpendicular bisector hyper-plane of the line joining s0 and
sk onto the z1 −z2 plane. The condition k : βk (z1 ) < |r(z1 )| is actually indicating that union bound is applied only to those codewords residing inside the Gallager region since we have already separated the complement of Gallager region. Now, by further separating the tangential component of the noise z2 , we have P (Ek |z1 , y ≤ r2 (z1 )) = P (βk (z1 ) < z2 < |r(z1 )|, y1 ≤ r2 (z1 ) − z22 )

(6)

(11)

where y is a Chi-square random variable with (n − 1) degrees of freedom, i.e., −y n−1 1 · e 2σ2 y 2 −1 U (y) fy (y) = n−1 n−1 (7) n−1 2 2 Γ( 2 )σ

where y1 is a Chi-square random variable with (n−2) degrees of freedom, i.e., n−2 −y1 1 −1 · e 2σ2 y1 2 U (y1 ). (12) fy1 (y1 ) = n−2 n−2 n−2 2 2 Γ( 2 )σ

where Γ(x) and U (x) are Gamma and unit-step functions, respectively.

Thus, the overall bound can be written as (13) on the top of the next page. Since the Gallager region is chosen to be a cone, the radius of the sphere cross-sections can be written as [10] √ r(z1 ) = r0 ( n − z1 ) (14)

Furthermore, P (E|z1 , y ≤ r2 (z1 )) can be upper bounded using the union bound yielding

X

P (E|z1 ) ≤ 2

k:βk (z1 )<|r(z1 )|

2

Ak P (Ek |z1 , y ≤ r (z1 )) + P (y > r (z1 )) (8)

where the pairwise error event Ek is the event that the received vector r is closer to sk than to the transmitted codeword s0 , i.e., Ek = {kr − sk k ≤ kr − s0 k |s0 }

(9)

where r0 is obtained from the following equation [6]: √ Z θk X πΓ( n−2 n−3 2 ) sin θdθ = Ak . n−1 Γ( 2 ) 0 r2 n k:dk ≤⌊ 0 2 1+r 0

(15)

⌋

and −1

θk = cos

s

dk . − dk )

r02 (n

(16)

3

Z

+∞

 

−∞

Pω (E) ≤

X

Ak

Z

|r(z1 )|

fz2 (z2 )

r 2 (z1 )−z22

fy1 (y1 )dy1 .dz2

0

βk (z1 )

k:βk (z1 )<|r(z1 )|

Z

IV. I MPROVED TSB (ITSB) TSB is one of tightest upper bounds ever-known. This is mainly due to selection of hyper-cone as the Gallager region which is the closest to Voronoi region in geographical sense [11]. On the other hand, the poor performance of union bound at low and medium SNR ranges, degrades the performance of the overall TSB at these ranges. This provides some room for further improvement of TSB by using a tighter Bonferronitype inequality instead of the union bound. In this paper, a second-order Bonferroni-type inequality is used:

P(

M [

i=1

Ei ) ≤ P (E1 ) +

or equivalently: P(

M [

i=1

Ei ) ≤

M X i=1

M X

P (Ei , Eˆic )

(17)

i=2

P (Ei ) −

M X

P (Ei , Eˆi ).

(18)

i=2

SM

This is a second-order upper bound for i=1 Ei (also referred to as Hunter bound [14]), which is based on both first and second-order probabilities. However, the value of the bound (in particular the second term) depends not only on the arbitrary choice of ˆi but also on indexing of the events. Hunter suggests a graph theory approach to this problem [14]: consider each of the random events, Ei , as a node of a graph, and the edges, Ei Ej , as branches joining nodes Ei and Ej . A spanning tree of the nodes Ei , i = 1, ..., M , is a connected graph with M − 1 branches such that at least one branch is incident on each of the M nodes. This definition implies that each node is joined to every other node through a set of branches in a spanning tree and that there is only one such set for any pair of nodes (i.e., a spanning tree has no cycles). Introducing these concepts, Hunter proves the following theorem: Theorem: For some assignment of subscripts and some arbitrary choices ˆi, a set of (M − 1) intersections may be used in the second term of (17) if and only if it forms a spanning tree of the nodes {Ei }M i=1 (The proof is given in [14]). Hence, tightening (17), requires minimization of the second term of (17) over all possible spanning trees.2 So if we denote any of the M ! possible permutations of the indices of the error events E1 to EM by π{1, 2, ..., M } = {π1 , π2 , ..., πM }, the tightest Hunter bound will be

P(

M [

i=1

Ei ) ≤ min π,Λ

(

M X i=1

P (Eπi ) −

M X i=2

)

P (Eπi , Eπˆi )

(19)

where Λ = {πˆ2 , πˆ3 , ..., πˆM }, in which πˆj ∈ {π2 , π3 , ..., πM }, j = 2, 3, ..., M , is the index of the event that minimizes the corresponding pairwise probability. 2 It is known that there are k k−2 spanning trees in a totally connected graph with k nodes [14].

!

+

Z

+∞

r 2 (z1 )



(13)

fy (y)dy  fz1 (z1 )dz1 .

The solution to this problem, known as Minimal Spanning tree, has been proposed by Kruskal [15]. Now we apply the aforementioned second-order Bonferronitype bound, i.e. Hunter bound, to TSB. Substituting (17) in (6) and denoting P (E|z1 ) by Pz1 (E), we will have:   Pz1 (E) ≤ Pz1 E1 , y ≤ r2 (z1 ) + M X i=2

    Pz1 Ei , Eˆic , y ≤ r2 (z1 ) + P y > r2 (z1 )

(20)

where Ei s can be any arbitrary error event (i.e., no indexing has been done so far).   The first term, Pz1 E1 , y ≤ r2 (z1 ) , in similarity with TSB can be written as   Pz1 E1 , y ≤ r2 (z1 ) =   (21) P β1 (z1 ) < z2 < |r(z1 )|, y1 ≤ r2 (z1 ) − z22 .

For the probabilities of the form Pz1 (Ei , Eˆic , y ≤ r2 (z1 )) encountered in (20), we use the same approach introduced in [10] which results in the following triple-event probability corresponding to the second term of the Hunter bound:   Pz1 Ei , Ejc , y ≤ r2 (z1 ) =  P βi (z1 ) ≤ z2 ≤ r(z1 ),

−r(z1 ) ≤ z3 ≤ l(z1 , z2 ), y2 ≤ r2 (z1 ) − z22 − z33 ′

(22) 

where z2 and z3 are independent zero-mean gaussian random variables with variance σ 2 , l(z1 , z2 ) =

βj (z1 ) − ρz2 p 1 − ρ2

(23)

and ρ = cos φ is the correlation coefficient between z2 and ′ z3 .3 y2 is a Chi-square random variable with (n − 3) degrees of freedom, i.e., n−3 −y2 1 −1 · e 2σ2 y2 2 U (y2 ). (24) fy2 (y2 ) = n−3 n−3 n−3 2 2 Γ( 2 )σ Plugging (22) and (21) in (20) will result in the overall bound as in (25) where βmin (z1 ) is simply the βk (z1 ) introduced in equation (10) with dk = dmin and τ is the set of all possible spanning trees of a graph with Ei ’s as its node. The above bound is too complex to compute. Firstly as it depends on the correlation coefficient ρ which means the bound needs the global geometrical properties of the code and only the distance spectrum will not suffice. Furthermore the optimization over τ is prohibitively complex. In order to avoid the necessity of calculating the correlation coefficients, we try to replace the probabilities of the form 3 Refer

to [10] for more details.

4

Pw (E) ≤ min τ

X

j>1:βj (z1 )
Z

r(z1 )

βj (z1 )

Z

l(z1 ,z2 )

nZ

+∞

−∞

hZ

r(z1 )

fz2 (z2 )

βmin (z1 )

fz2 ,z3 (z2 , z3 )

Z

M [

i=1

fy1 (y1 )dy1 .dz2 +

r 2 (z1 )−z22 −z32



fy2 (y2 )dy2 dz3 dz2 +

0

−r(z1 )

Ei ) ≤ P (Edmin ) +

r 2 (z1 )−z22

0

  Pz1 Ei , Ejc , y ≤ r2 (z1 ) (denoted by Pij c from now on) with an upper bound which depends only on correlation between the two layers of code with hamming weights di = w(si ) and dj = w(sj ), rather than all individual codewords with hamming weight di and dj . In order to do so, we first prove that, as long as di > dj , the probabilities Pij c are monotonically decreasing functions of correlation coefficient between si and si , i.e., ρij (Appendix 1). This means that we can upper bound Pij c by using the minimum correlation coefficient between si and sj . The very first consequence of the above, is that we have an extra condition in the minimal spanning tree problem. In other words, now, for each i, j < i should be chosen in such a way that di > dj . Hunter [14] proves that any spanning tree of the k vertices of the events can be used in (17) and hence also in our (20) and (25). Therefore, we are looking for some spanning trees which satisfy the above condition. One of the ways to satisfy this extra condition is to use star structure for the underlying spanning tree with the node corresponding to minimum Hamming weight codeword as the center of the star. This topology has two advantages, first, it is simple to analyze specially for calculating the minimum correlation coefficient between different layers of the code as we only have to calculate this coefficient between all layers within the Gallager region and the layer corresponding to minimum Hamming distance. Second, intuitively speaking, for a fixed i the probabilities Pij c are increasing with dj so choosing dj = dmin makes the upper bound on Pij c tighter, resulting in an overall tighter Hunter bound. If we denote the center node of the spanning tree by Edmin , then (17) can be rewritten as P(

Z

M X

P (Ei , Edcmin )

(26)

i=2

Now that we have satisfied the condition, we can replace the probabilities P (Ei , Edcmin ) by their upper bounds. Another important point in bounding the probabilities P (Ei , Edcmin ) is that, we just look at the layers of the code-book (i.e. all the codewords with equal hamming weight) rather than all codewords within a layer. This makes the minimal spanning tree problem more tractable and less complex as we are dealing with at most n layers of the code as opposed to 2k codewords. So now, the problem of replacing the probabilities P (Ei , Edcmin ), will simplify to find the minimum correlation coefficient between the two layers of the code with Hamming weights dmin and di , so that the P (Ei , Edcmin ) is upper bounded. In this paper, we use the minimum correlation coefficient introduced in [10]. After all, the overall ITSB can be written as (27) in the next page where lmax (z1 , z2 ) is obtained by plugging the minimum correlation coefficient between the two layers of code with Hamming weight dmin and di , into (23).

Z

+∞

r 2 (z1 )

i o fy (y)dy fz1 (z1 )dz1

(25)

If we use the second form of the Hunter bound in (18), we will have the equivalent bound in the form of (28) in the next page. Comparing (28) with TSB (13), it can be easily seen that ITSB is indeed tighter than TSB. It is important to note that for ITSB we have chosen r(z1 ) to be the same as that of TSB. One may argue that applying the second-order Bonferroni-type inequality inside the Gallager region, provides more room to increase r(z1 ). However, numerical results show that the improvements are negligible as opposed to the complexity of the new r(z1 ) optimization process. The above bound, depends only on the spectrum, making it easy to calculate. Also comparing it to TSB, it has only one more nested integral which keeps the complexity fair. And finally, it does not involve any optimization making it straight forward. V. R ESULTS AND C ONCLUSIONS In this section we provide simulation results to show the improvement of ITSB over TSB. For this purpose, we choose the well-known BCH codes of length 63. These codes are long enough to render the ML decoding prohibitively complex while they have a spectrum which is very close to average spectrum. Fig. 2 shows the performance of TSB, ITSB and Shannon lower bound (SLB) for BCH [63, 39]. The well-known Shannon lower bound is the tightest lower bound at low SNR range. It is based on matching the shape of voronoi region of the constellation. SLB on the performance of ML decoding of Slepian codes is based on upper bounding the voronoi region of the transmitted codeword by a spherical n-cone with a solid angle equal to the solid angle of a n-sphere divided by the number of codewords and for an [n, k] binary code is given by [16]: P (E) ≥ Z

0

∞

xn−2 . exp(−x2 /2).Q

1 n−3 2

2 r

Γ( n−1 2 )

.

! 2nREb − x cot(θSLB ) du N0 (29)

where the optimal θSLB is obtained from: √ Z θSLB n π Γ((n + 1)/2) sinn−2 u.du = . (n − 1)2k Γ((n + 2)/2) 0

(30)

At high SNR, both TSB and ITSB coincide with union bound. At low SNR, TSB coincides with SLB which shows that TSB is a very tight bound in this range. For example, for BCH [63,36] (Fig. 3), at an error rate of 10−1 , the difference between SLB and TSB is 0.6 dB while we report an improvement of 0.1 dB over TSB (more than 15%).

5

Pw (E) ≤ X

j>1:βj (z1 )
r(z1 )

Z

βj (z1 )

Pω (E) ≤ −

Z

Z

lmax (z1 ,z2 )

nZ

+∞

−∞

hZ

fz2 ,z3 (z2 , z3 )

−∞

X

h

X

k:βk (z1 )<|r(z1 )|

j>1:βj (z1 )
Z

r(z1 )

βj (z1 )

Z

fz2 (z2 )

βmin (z1 )

−r(z1 )

+∞

r(z1 )

Z

 Z Ak

fy1 (y1 )dy1 .dz2 +

0

r 2 (z1 )−z22 −z32



fy2 (y2 )dy2 dz3 dz2 + 0

|r(z1 )|

fz2 (z2 )

βk (z1 )

r(z1 )

fz2 ,z3 (z2 , z3 )

Z

r 2 (z1 )

i o fy (y)dy fz1 (z1 )dz1



0 r 2 (z1 )−z22 −z32

Z

(27)

+∞

fy (y)dy

r 2 (z1 )

(28)

i fy2 (y2 )dy2 dz3 dz2 fz1 (z1 )dz1

Performance Bounds for BCH [63,36]

0

10

+∞

fy1 (y1 )dy1 .dz2 +

0

lmax (z1 ,z2 )

Z

r 2 (z1 )−z22

Z

Performance Bounds for BCH [63,39]

0

r 2 (z1 )−z22

Z

10

TSB ITSB SLB

TSB ITSB SLB

−1

10 Decoding Error Probability

Decoding Error Probability

−1

10

−2

10

−2

10

−3

10

−3

10

−4

−1

Fig. 2.

−0.5

0

0.5

1 SNR=Eb/N0

1.5

2

2.5

3

Performance Bounds for BCH[63,39]

The proposed bound is very efficient in the sense that it only needs the spectrum of the code making it easy to implement and applicable to cases where we can not ask for more than the distance spectrum. Both TSB and ITSB apply only to spherical constellations and are not limited to BPSK modulated signals.

ON

THE BEHAVIOR

A PPENDIX  I  OF THE Pz1 Ei , Ejc , y ≤ r2 (z1 )

  In section IV, we claimed that Pz1 Ei , Ejc , y ≤ r2 (z1 ) is a monotonically decreasing function of ρij . In order to prove this, we expand Pij c using the geometry of Fig.4 :   Pz1 Ei , Ejc , y ≤ r2 (z1 ) =   ′ Pz1 βi (z1 ) < z2 , z3 < βj (z1 ), y ≤ r2 (z1 ) = P (r ∈ A) (31) where r is the received vector from the channel and A is the crosshatched region in Fig.4. We can rewrite (33) in terms

10

1

Fig. 3.

1.2

1.4

1.6

1.8

2 SNR=Eb/N0

2.2

2.4

2.6

2.8

3

Performance Bounds for BCH[63,36]

of the two orthogonal noise components z2 and z3 :   Pz1 Ei , Ejc , y ≤ r2 (z1 ) =  Pz1 βi (z1 ) < z2 < r(z1 ),

(32)

−r(z1 ) < z3 < l(z1 , z2 ), y2 ≤ r2 (z1 ) − z22 − z32



where l(z1 , z2 ) =

βj (z1 ) − ρz2 p 1 − ρ2

(33) ′

represents the straight line perpendicular to z3 as show in Fig.4, z2 and z3 are independent zero-mean Pn gaussian random variables with variance σ 2 and y2 = i=4 zi2 is a chi-square random variable with (n − 3) degrees of freedom, i.e.,: fy2 (y2 ) =

1 2

n−3 2

n−3 Γ( n−3 2 )σ

−y2

n−3

· e 2σ2 y2 2

−1

U (y2 ) .

(34)

6

R l(z1 ,z2 ) + fz3 (z1 , z2 , z3 )dz3 of ρ, it would mean that the integral −r(z 1)   2 c and hence the overall Pz1 Ei , Ej , y ≤ r (z1 ) are monotonically decreasing with ρ. Differentiating l(z1 , z2 ) with respect to ρ

z3

′

z3 φ

l(z1 , z2 )

111 000 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 A 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111

z2

Fig. 4.

ρβj (z1 ) − z2 ∂ l(z1 , z2 ) = ∂ρ (1 − ρ2 )3/2

βj (z1 )

βi (z1 )

and setting the above derivative negative, will result in: z2 ρ< . (40) βj (z1 )

r(z1 )

The Geometry of the probabilities Pz1 (Ei , Eˆc , y ≤ r 2 (z1 )). i

Particularly, in the aforementioned probabilities in (22) and (35), z2 is varying in the range of [βi (z1 ), r(z1 )], q so the mind (n−d ) βi (z1 ) imum value of the right side of (40) is βj (z1 ) = dij (n−dji ) . It can be seen that if di > dj , the minimum ρ will be greater than one and the condition (40) will be always satisfied and this finishes the proof.  We conclude that, as long as di > dj , Pz1 Ei , Ejc , y ≤  r2 (z1 ) is a monotonically decreasing function of ρ.4 R EFERENCES

So, (32) can be expanded as   Z Pz1 Ei , Ejc , y ≤ r2 (z1 ) =

r(z1 )

fz2 (z2 )

βi (z1 )

Z

l(z1 ,z2 )

fz3 (z3 )

−r(z1 )

Z

r 2 (z1 )−z22 −z32

(35)

fy2 (y2 )dy2 dz3 dz2 .

0

In order to analyze  the behavior of the function c 2 Pz1 Ei , Ej , y ≤ r (z1 ) , we first define: fz+3 (z1 , z2 , z3 )

, fz3 (z3 )

Z

r 2 (z1 )−z22 −z32

fy2 (y2 )dy2 .

(36)

0

It is obvious that fz+3 (z1 , z2 , z3 ) is an always-positive function of z3 : fy2 (y2 ) and fz3 (z3 ) are positive Probability Density Functions (pdf). So (35) is reduced to:   Pz1 Ei , Ejc , y ≤ r2 (z1 ) = Z r(z1 ) Z l(z1 ,z2 ) (37) fz2 (z2 ) fz+3 (z1 , z2 , z3 )dz3 dz2 . βi (z1 )

(39)

−r(z1 )

Next, we notice that: Z l(z1 ,z2 ) Z  ∂  r(z1 ) fz+3 (z1 , z2 , z3 )dz3 dz2 = fz2 (z2 ) ∂ρ βi (z1 ) −r(z1 ) Z r(z1 ) Z  ∂  l(z1 ,z2 ) + fz2 (z2 ) fz3 (z1 , z2 , z3 )dz3 dz2 ∂ρ −r(z1 ) βi (z1 ) (38) which shows that the behavior of the above function (with respect to ρ) depends only on the behavior of the second integral Z l(z1 ,z2 ) fz+3 (z1 , z2 , z3 )dz3 .

[1] J. Galambos and I. Simonelli, Bonferroni-type Inequalities with Applications, New York, NY: Springer, 1996. [2] C. Berrou, A. Glavieux, and P. Thitimajshima, Near Shannon limit error correcting coding and decoding: Turbo codes, in Proc. 1993 IEEE Int. Conf. on Comm., Geneva, Switzerland, pp. 1064-1070, 1993. [3] D. Divsalar, H. Jin, and R. J. McEliece, “Coding theorems for Turbo-like codes”, 1998 Allerton Conference, Sept. 23-25, 1998. [4] R. G. Gallager, Low Density Parity Check Codes. Cambridge, MA: MIT Press, 1963. [5] D. J. C. MacKay and R. M. Neal, “Near Shannon limit performance of low density parity check codes”, Electron. Lett., vol. 32, pp. 1645-1646, Aug. 1996. [6] G. Poltyrev, “Bounds on the decoding error probability of binary linear codes via their spectra”, IEEE Trans. Inform. Theory, vol IT-40, no. 4, pp. 1284-1292, July 1994. [7] D. Divsalar, “A simple tight bound on error probability of block codes with application to Turbo codes”, TMO Progress Report 42-139, NASA, JPL, Pasadena, CA, USA, 1999. [8] I. Sason and S. Shamai(Shitz),“Variations on the Gallager bounds, connections and applications”, IEEE Trans. Inform. Theory, vol. IT-48, no. 12, pp. 3029-3051, Dec. 2002. [9] S. Yousefi, “Bounds on the performance of maximum-liklihood decoded binary block codes in AWGN interference”, PhD Disseration, Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario, Canada, Sept. 2002. [10] S. Yousefi and A. Khandani, “A new upper bound on the ML decoding error probability of linear binary block codes in AWGN interference”, IEEE Trans. Inform. Theory, Vol. IT-50 , no. 12 , pp. 3026 - 3036, Dec. 2004, [11] S. Yousefi and A. Khandani, “Generalized tangential sphere bound on the ML decoding error probability of linear binary block codes in AWGN interference”, IEEE Trans. Inform. Theory, Vol. IT-50 , no. 11 , pp. 2810 - 2815, Nov. 2004, [12] G. D. Forney, Jr., “Geometrically uniform codes”, IEEE Trans. Inform. Theory, vol. IT-37, no. 5, pp. 1241-1260, Sept. 1991. [13] I. Sason and S. Shamai, “Improved upper bounds on the decoding error probability of parralel and serial concatenated turbo codes via their ensamble distance spectrum”, IEEE Trans. Inform. Theory, vol. IT-46, no. 1, pp. 1-23, JAN. 2000. [14] D. Hunter, “An upper bound for the probability of a union”, J. Applied Probability, vol 13, pp. 597-603, 1976. [15] J.B. Kruskal, “On the shortest spanning tree of a graph and the travelling salesman problem”, Proc. of Amer. Math soc., no. 7, pp. 48-50, Nov. 1956. [16] G.E. S`eguin, “A lower bound on the error probability for signals in white Gaussian noise”, IEEE Trans. Inform. Theory, vol IT-44, no. 7, pp. 3168-3175, Nov. 1998.

−r(z1 )

Noting that the integrand is always positive, we can say the integral is an increasing function of l(z1 , z2 ). Therefore, if we can show that l(z1 , z2 ) is a monotonically decreasing function

4 The

condition di > dj is sufficient but not necessary.