GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM FOR SECTIONS OF CONVEX BODIES

Boris Rubin∗ and Gaoyong Zhang† Abstract. We present generalizations of the Busemann-Petty problem for dual volumes of intermediate central sections of symmetric convex bodies. It is proved that the answer is negative when the dimension of the sections is greater than or equal to 4. For 2- and 3-dimensional sections, both negative and positive answers are given depending on the orders of dual volumes involved, and certain cases remain open. For bodies of revolution, a complete solution is obtained in all dimensions.

0. Introduction Let M be a compact convex set of dimension i in Rn that contains the origin in its relative interior. When i = n, M is called a convex body. For 1 ≤ k ≤ i, when a subspace η ⊂ Rn has dimension n − i + k, the intersection M ∩ η is a k-dimensional compact convex set in general. The k-th dual volume V˜k (M ) is defined as the average of the k-dimensional volume volk (M ∩ η) about η, Z (0.1)

V˜k (M ) =

volk (M ∩ η)dη, Gn,n−i+k

where volk (·) denotes the k-dimensional volume functional, and dη is the invariant probability measure on the Grassmannian Gn,n−i+k . Note that V˜i (M ) = voli (M ). Dual volumes are important geometric invariants of sections of convex sets; see Remark 2.2 for further discussion. One of the subjects for sections of convex sets is to study how dual volumes behave. In this paper, we investigate inequalities of dual volumes of origin-symmetric convex bodies. We consider the following problem: 1991 Mathematics Subject Classification. Primary 52A38; Secondary 44A12. Key words and phrases. Convex bodies, dual volumes, the Busemann-Petty problem, the spherical Radon transform. ∗ Supported in part by the Edmund Landau Center for Research in Mathematical Analysis and Related Areas, sponsored by the Minerva Foundation (Germany); † Supported in part by NSF Grant DMS-0104363. Typeset by AMS-TEX

1

2

B. RUBIN AND G. ZHANG

Problem A. Let K and L be origin-symmetric convex bodies in Rn , 1 ≤ k ≤ i, and k < l ≤ n. If V˜k (K ∩ ξ) ≤ V˜k (L ∩ ξ) for any ξ ∈ Gn,i , does it follow that V˜l (K) ≤ V˜l (L)? The special case, k = i = n − 1, l = n, is the well-known Busemann-Petty problem [BP]: If K and L are origin-symmetric convex bodies in Rn , is the volume of K smaller than that of L when voln−1 (K ∩ ξ) ≤ voln−1 (L ∩ ξ) for every hyperplane ξ through the origin? This problem has been resolved by a series of works, see [LR], [Lu], [B], [Bo], [Gi], [P], [G1], [G2], [K1], [Z2], [GKS]. Simplified proofs were presented in [BFM], [K3], [R3]. The Busemann-Petty problem has a positive solution in R3 and in R4 , and a negative solution in higher dimensions. The case k = i, l = n, of Problem A is known as the generalized Busemann-Petty problem [Z1]. It generalizes the above-mentioned problem of Busemann and Petty to the case of i-dimensional plane sections where i may be less then n − 1. It was shown in [BZh] that the generalized Busemann-Petty problem has a negative solution when i ≥ 4. A different proof of this result was given in [K2]. For i = 2, or 3, the answer is still open. For the special case, when K is a body of revolution, the answer for i = 2 and 3 is positive [GrZ], [Z1]. An alternative proof of this fact is given below in the framework of a more general consideration. Problem A can be reformulated in an equivalent form which is more convenient to handle and has generalizations. We denote by S n−1 the unit sphere in Rn endowed with the rotation-invariant probability measure du. Let K be a convex body in Rn that contains the origin in its interior. The radial function of K is defined by ρK (u) = sup{λ ≥ 0 : λu ∈ K},

u ∈ S n−1 .

One can show (see Lemma 2.1) that Z V˜k (K ∩ ξ) = κk

S n−1 ∩ξ

ρK (u)k dξ u

where κk is the volume of the k-dimensional unit ball, and dξ u denotes the normalized Lebesgue measure on S n−1 ∩ ξ. In view of this, we introduce the following functionals: Z Z k (0.2) Ik (K, ξ) = ρK (u) dξ u, Jl (K) = ρK (u)l du. S n−1 ∩ξ

S n−1

Problem B. Let K and L be origin-symmetric convex bodies in Rn , 0 < k < l < ∞. If Ik (K, ξ) ≤ Ik (L, ξ) for any ξ ∈ Gn,i , does it follow that Jl (K) ≤ Jl (L)?

GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM

3

In Problem B, k and l are any positive numbers. If they are integers satisfying 1 ≤ k ≤ i and k < l ≤ n, then Problem B is a reformulation of Problem A. Particular cases of Problem B were considered by Hadwiger [H] (n = 3, l − k ≤ 1) for bodies of revolution, and Koldobsky [K2, Theorem 8] (k = n − 1, i = 3) for any origin-symmetric convex bodies. For these cases they gave affirmative answers. Problem B has trivially an affirmative answer when i = 1. So it is always assumed that 2 ≤ i ≤ n − 1. Main results. (a) Problem B (and therefore Problem A) has a negative answer in the cases (i) i ≥ 4 ; (ii) l − k > n − i for i = 2 or 3. (Theorem 6.1) (b) Problem B has a positive answer when l = k + 1 for i = 2 or 3. (Theorem 5.2) (c) Other cases of i = 2 or 3 remain open in general, but have positive answers when K is a body of revolution. (Corollary 5.4) These results are generalizations of the full solution to the Busemann-Petty problem. The main tools we use are totally geodesic Radon transforms and harmonic analysis on the sphere. We also apply some techniques of fractional calculus [R1], [SKM], in particular, the Erdelyi-Kober fractional integrals which arise in our problem in a natural way. The curvature of a convex body plays an important role in the proof of negative results. The paper is almost self-contained. Section 1 contains auxiliary facts related to the spherical Radon transform and the corresponding dual transform. In Section 2 we define a class of star bodies called (i, p)-intersection bodies and establish connection between these bodies and Problems A and B. Section 3 includes a few elementary geometric lemmas. In Section 4 we derive important formulas that represent the radial function of a convex body in terms of volumes of parallel sections via the dual spherical Radon transform. These are generalizations of the corresponding formulas in [G1], [Z2] and [GKS]. In Section 5 we justify positive answers for Problems A and B. Negative answers are obtained in Section 6. All results for Problem A in the case of bodies of revolution are summarized in Table 1 at the end of Section 6. As we have already mentioned, a negative answer to the generalized Busemann-Petty problem for i ≥ 4 was first given in [BZh]. However, the proof of one of the lemmas in [BZh] has certain gap. We will give another proof of that lemma in Appendix. Notation. Denote by σp−1 = 2π p/2 /Γ(p/2), p > 0, and κp = σp−1 /p. Then σn−1 is the surface area of the unit sphere S n−1 in Rn , and κn is the volume of the n-dimensional unit ball B n . Let Gn,i , 1 ≤ i ≤ n − 1, be the Grassmann manifold of i-dimensional subspaces of Rn . For u ∈ S n−1 and ξ ∈ Gn,i , we denote by du and dξ the corresponding SO(n)-invariant measures with total mass 1. A convex body in Rn is a compact convex set with nonempty interior. The class of all convex bodies in Rn containing the origin o in its interior will be denoted by Kn . Denote by Ken the subclass of origin-symmetric convex bodies. A star body L in Rn is a star shaped set about the origin that has continuous radial function ρL (u) = sup{λ ≥ 0 : λu ∈ L}, u ∈ S n−1 . The class of star bodies is denoted by S n . The subclass of origin-symmetric star bodies is denoted by Sen . Denote by N the set of positive integers.

4

B. RUBIN AND G. ZHANG

1. The spherical Radon transform Various problems about sections of centrally symmetric bodies are intimately connected with the totally geodesic Radon transform on the unit sphere S n−1 and the corresponding dual transform. In this section we present some auxiliary lemmas about these transforms. More information can be found in [He], [R2], [R3], and other sources. For continuous functions f (u) on S n−1 and ϕ(ξ) on Gn,i , the totally geodesic Radon transform Ri f (which is also called the spherical Radon transform of f ) and its dual transform R∗i ϕ are defined by Z (1.1)

(Ri f )(ξ) =

Z S n−1 ∩ξ

(R∗i ϕ)(u)

f (u)dξ u,

ϕ(ξ)du ξ,

= ξ3u

where dξ u and du ξ denote the induced normalized measures on the corresponding manifolds S n−1 ∩ ξ and {ξ ∈ Gn,i : ξ 3 u}. These manifolds can be identified with S i−1 and Gn−1,i−1 , respectively. The duality between Ri and R∗i is expressed by Z

Z

(1.2)

(Ri f )(ξ)ϕ(ξ)dξ = Gn,i

S n−1

f (u)(R∗i ϕ)(u)du.

This allows to define Ri µ and R∗i ν for arbitrary finite Borel measures µ on S n−1 and ν on Gn,i . Let e1 , e2 , . . . , en be coordinate unit vectors, and x · y the usual inner product in Rn . Given u ∈ S n−1 and ξ ∈ Gn,i , we write d(u, ξ) for the geodesic distance between u and S n−1 ∩ ξ, and denote by |Pξ u| the length of the orthogonal projection of u onto ξ. It is easily seen that (1.3)

sin d(u, ξ) = |Pξ⊥ u|,

cos d(u, ξ) = |Pξ u|.

A function on the unit sphere S n−1 or on the Grassmannian Gn,i is called a zonal function if it is invariant under the group SO(n − 1) of rotations preserving the north pole en . Lemma 1.1. (i) A function f (u) on S n−1 is zonal if and only if there is a function f0 (t) on [−1, 1], such that f (u) = f0 (cos ω), ω = d(en , u). If f (u) is zonal then f (u) = f (e1 sin ω + en cos ω). (ii) A function ϕ(ξ) on Gn,i is zonal if and only if there is a function ϕ0 (s), s ∈ [0, 1], such that ϕ(ξ) = ϕ0 (sin θ), θ = d(en , ξ). If ϕ(ξ) is zonal then ϕ(ξ) = ϕ(ξθ ) where ξθ = span(e1 , . . . , ei−1 , ei sin θ + en cos θ). The statement (i) and the “if” part in (ii) are obvious. The “only if” part in (ii) can be understood by geometric reasoning and proved analytically following more general Lemma 2.7 from [GrR]. The next lemma provides Abel type representations for the Radon transforms (1.1) of zonal functions.

GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM

5

Lemma 1.2. For u ∈ S n−1 and ξ ∈ Gn,i , we denote ω = d(en , u),

t = cos ω,

θ = d(en , ξ),

s = sin θ.

If f (u) = f0 (t) and ϕ(ξ) = ϕ0 (s) then (1.4) (1.5)

c1 (Ri f )(ξ) = cosi−2 θ (R∗i ϕ)(u) =

sin

c2 n−3

c1 =

Z cos θ −cos θ sin ω

(cos2 θ − t2 )(i−3)/2 f0 (t) dt,

Z

ω

(sin2 ω − s2 )(i−3)/2 sn−i−1 ϕ0 (s) ds,

0

σi−2 , σi−1

c2 =

σi−2 σn−i−1 , σn−2

provided that the corresponding integrals exist in the Lebesgue sense. If f is an even function then (1.4) becomes (1.6)

2c1 (Ri f )(ξ) = cosi−2 θ

Z cos θ

(cos2 θ − t2 )(i−3)/2 f0 (t) dt.

0

Proof. The formula (1.4) can be found in many sources in different forms. Both formulas were presented in [R2], Lemma 2.4. For the sake of completeness we give another proof which is more geometric. Let Pξ en = (cos θ)u0 , ψ be the angle between u0 and u. Using spherical trigonometry on the triangle (en , u, u0 ), we get u · en = cos ω = cos ψ cos θ, and Z (Ri f )(ξ) = f0 (u · en ) du =

S n−1 ∩ξ Z σi−2 π

σi−1

f0 (cos ψ cos θ) sini−2 ψ dψ.

0

This coincides with (1.4). In order to prove (1.5) we fix a subspace ξ0 ∈ Gn,i that contains en and u, and denote by Gu the group of rotations in u⊥ . For α ∈ Gu , let αen = v sin ω + u cos ω, v ∈ S n−1 ∩ u⊥ . Then |Pξ0⊥ (αen )| = |Pξ0⊥ v| sin ω, and by (1.3), Z (R∗i a2 )(u)

= Z

ξ3u

Z

Gu

Z

Gu

= =

a(|Pξ⊥ en |)dξ

a(|Pα−1 ξ0⊥ en |)dα a(|Pξ0⊥ (αen )|)dα

= S n−1 ∩u⊥

a(|Pξ0⊥ v| sin ω)dv.

6

B. RUBIN AND G. ZHANG

Using the bi-spherical coordinates [VK, pp. 12, 22] v = x cos ψ + y sin ψ, x ∈ S i−2 ⊂ ξ0 ∩ u⊥ ,

dv = c2 sinn−i−1 ψ cosi−2 ψ dψdxdy, y ∈ S n−i−1 ⊂ ξ0⊥ ,

0 < ψ < π/2,

we have |Pξ0⊥ v| = sin ψ, and Z (R∗i a2 )(u) This gives (1.5).

π/2

= c2

a(sin ω sin ψ) sinn−i−1 ψ cosi−2 ψdψ.

0

¤

Let Ci be the class of C ∞ zonal functions on the Grassmannian Gn,i . Note that C1 is the class of even C ∞ -functions on S n−1 . Lemma 1.3. The Radon transform Ri and the dual transform R∗i are bijective mappings from C1 onto Ci and from Ci onto C1 , respectively. Proof. It is known (see [He], Proposition 2.4, p. 60) that Ri and R∗i act from C ∞ (S n−1 ) into C ∞ (Gn,i ) and from C ∞ (Gn,i ) into C ∞ (S n−1 ), respectively. Since both transforms commute with rotations, then Ri (C1 ) ⊂ Ci and R∗i (Ci ) ⊂ C1 . Injectivity of Ri on C1 and R∗i on Ci follows from Lemma 1.1 and uniqueness of inversion of Abel type integrals (1.6), (1.5) [SKM]. It was also proved in [GZ] (Lemma 2.2) using harmonic analysis on ∞ Grassmannians. Note that Ri is injective on Ceven (S n−1 ) [He, p. 99], [R2], whereas R∗i ∞ is not injective on C (Gn,i ). In order to prove surjectivity of Ri : C1 → Ci and R∗i : Ci → C1 we introduce the spherical Riesz potential (or the generalized sine transform) [R2] (1.7)

Z σn−1 Γ((n − 1 − α)/2) (Q f )(u) = (1 − |u · v|2 )(α−n+1)/2 f (v)dv 2π (n−1)/2 Γ(α/2) n−1 S Z σn−1 Γ((n − 1 − α)/2) = (sin d(u, v))α−n+1 f (v)dv, (n−1)/2 2π Γ(α/2) S n−1 α

α > 0,

α − n + 1 6= 0, 2, 4, . . . .

Here d(u, v) is the geodesic distance between the points u, v ∈ S n−1 . The normalizing coefficient in (1.7) is chosen so that if Yj is a spherical harmonic of degree j, then by the Funk-Hecke formula and [PBM, 2.21.2(3)], Qα (Yj ) = qα (j)Yj where (1.8)

qα (j) =

Γ((j + n − 1 − α)/2) Γ((j + 1)/2) Γ((j + α + 1)/2) Γ((j + n − 1)/2)

(∼ (j/2)−α

as

j → ∞)

for j even, and qα (j) = 0 for j odd. Since qα (j) is finite for α−n+1 6= 0, 2, 4, . . . , different ∞ from zero for j even, and has a power behavior as j → ∞, then Qα maps Ceven (S n−1 ) ∞ n−1 ∞ n−1 into Ceven (S ) and is injective. By the same reason, for any g ∈ Ceven (S ), if

GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM

7

P P ∞ g= Yj , then f = j even qα (j)−1 Yj belongs to Ceven (S n−1 ) and Qα f = g. Thus Qα ∞ is an automorphism of Ceven (S n−1 ). It is known [He, p. 94], [R2, Theorem 1.1], that R∗i Ri f = c Qi−1 f,

(1.9)

c=

Γ((n − 1)/2)Γ(i/2) . Γ((n − i)/2)Γ(1/2)

Let us show that each function ϕ ∈ Ci is represented by the Radon transform ϕ = Ri f for some f ∈ C1 . We have R∗i ϕ ∈ C1 . Hence there exists f ∈ C1 so that R∗i ϕ = cQi−1 f . By (1.9), R∗i ϕ = R∗i Ri f. Since ϕ and Ri f are zonal and R∗i has a trivial kernel in Ci , this implies ϕ = Ri f . Now let us show that each function f ∈ C1 is represented by the dual Radon transform f = R∗i ϕ for some ϕ ∈ Ci . By (1.9), f = Qi−1 (Qi−1 )−1 f = R∗i ϕ,

ϕ = c−1 Ri (Qi−1 )−1 f.

Since Ri and (Qi−1 )−1 preserve smoothness and zonality, we are done.

¤

2. Dual volumes and intersection bodies The following lemma contains various representations of the dual volume functional ˜ Vk . For ξ ∈ Gn,i , we denote by Gi,k (ξ) the Grassmann manifold of all k-dimensional subspaces η in ξ. Lemma 2.1. Let K ∈ S n , ξ ∈ Gn,i , 1 ≤ k ≤ i ≤ n. Then the dual volume Z V˜k (K ∩ ξ) =

(2.1)

volk (K ∩ ξ ∩ η)dη Gn,n−i+k

has the following representations: Z V˜k (K ∩ ξ) =

(2.2)

volk (K ∩ ζ)dζ Gi,k (ξ)

Z

(2.3)

= κk

S n−1 ∩ξ

ρK (u)k dξ u

= κk (Ri ρkK )(ξ) Z σk−1 = |x|k−i dx. σi−1 K∩ξ

(2.4) (2.5)

Proof. Let us prove (2.2). For almost all η ∈ Gn,n−i+k , the intersection ζ = ξ ∩ η is an element of Gi,k (ξ). Given ζ ∈ Gi,k (ξ), denote by dζ η the invariant probability measure on the homogeneous manifold {η ∈ Gn,n−i+k : η ⊃ ζ}. By the formulas (14.40) and (14.42) in [Sa], for any f ∈ L1 (Gn,n−i+k ) we have Z (2.6)

Z

Z

f (η)dη = Gn,n−i+k

dζ Gi,k (ξ)

f (η) ∆(η)dζ η {η∈Gn,n−i+k : η⊃ζ}

8

B. RUBIN AND G. ZHANG

where the factor ∆(η) satisfies Z ∆(η) dζ η = c ≡ const. {η∈Gn,n−i+k :η⊃ζ}

The value of c depends on normalization of measures involved. If we set f (η) = volk (K ∩ ξ ∩ η) then for η ⊃ ζ we get f (η) = f (ζ), and (2.6) yields Z Z (2.7) volk (K ∩ ξ ∩ η)dη = c volk (K ∩ ζ)dζ. Gn,n−i+k

Gi,k (ξ)

If K is a unit ball then (2.7) gives c = 1 and (2.2) follows. Let us prove (2.3) and (2.4), which are equivalent. By passing to polar coordinates, we have volk (K ∩ ζ) = κk (Rk ρkK )(ζ), and therefore Z Z Z k volk (K ∩ ζ)dζ = κk (Rk ρK )(ζ)dζ = κk ρK (u)k dξ u. Gi,k (ξ)

S n−1 ∩ξ

Gi,k (ξ)

The last step is clear in view of duality (1.2) (set ϕ ≡ 1). The equivalence of (2.5) and (2.3) follows if we write (2.5) in polar coordinates. ¤ Remark 2.2. (i) If M is a compact convex set of dimension i in Rn that contains the origin in its relative interior, and M ⊂ ξ ∈ Gn,i , then V˜k (M ∩ ξ) = V˜k (M ), cf. (0.1). By (2.3), the dual volume V˜k (M ) is independent of the dimension n of the ambient space, and can be called the kth dual intrinsic volume of M . This notion is dual in a sense to the intrinsic volume Vk (M ) introduced by McMullen in [Mc1]; see also [Mc2], [S, p. 210]. Whereas Vk (M ) corresponds to projections of convex bodies, V˜k (M ) is appropriate to studying central sections. The representation (2.5) shows that V˜k (M ) are k-homogeneous rotation invariant valuations on Kn ; see [A], [Kl]. (ii) The integrals in (2.3) and (2.4) are meaningful for all positive k. We will keep the same notation V˜k (K ∩ ξ) and V˜k (K) for these cases. If i = n, then (2.3) becomes Z ˜ (2.8) Vk (K) = κk ρK (u)k du. S n−1

For k ≤ n, k ∈ N, (2.8) is the re-normalized version of the dual quermassintegral (2.9)

κk ˜ Wn−k (K). V˜k (K) = κn

Dual volumes of star bodies were introduced by Lutwak; see [Lu] and [BZ, p. 158]. The definition here differs by a normalization constant factor, which was resulted from a discussion with R.J. Gardner. (iii) In the case i = n, k ≤ n, k ∈ N, (2.2) yields Z (2.10) V˜k (K) = volk (K ∩ ζ)dζ. Gn,k (ξ)

GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM

9

This quantity measures the mean volume of k-dimensional central sections of K. (iv) From (0.2) and (2.3), we have (2.11)

V˜k (K ∩ ξ) = κk Ik (K, ξ),

V˜k (K) = κk Jk (K).

These equalities give the connection between Problems A and B. Definition 2.3. For 1 ≤ i ≤ n − 1, p ∈ R, an origin-symmetric star body K ∈ Sen is called an (i, p)-intersection body if there exists a non-negative measure µ on the Grassmannian Gn,i such that ρpK = R∗i µ.

(2.12)

We denote by Ii, p the class of all (i, p)-intersection bodies in Rn . An (n − 1, 1)-intersection body is simply called intersection body. This notion was introduced by Lutwak [Lu]. The (i, n − i)-intersection bodies were studied in [GrZ] and [Z1]. The connection of Problems A and B with (i, p)-intersection bodies is given by the following two lemmas. The basic idea comes from Lutwak [Lu]. Lemma 2.4. Let 0 < k < l, and K, L ∈ S n . If K ∈ Ii, l−k , and V˜k (K ∩ ξ) ≤ V˜k (L ∩ ξ) for any ξ ∈ Gn,i , then

V˜l (K) ≤ V˜l (L).

Proof. Since K ∈ Ii, l−k , there exists a non-negative measure µ on Gn,i such that ∗ ρl−k older inequality, we have K = Ri µ. By (2.8), (1.2), and the H¨ Z V˜l (K) = κl ρK (u)l du n−1 ZS = κl (R∗i µ)(u)ρK (u)k du n−1 ZS = κl (Ri ρkK )(ξ)dµ(ξ) Gn,i

=

κl κk

κl ≤ κk = κl

Z

V˜k (K ∩ ξ)dµ(ξ) Z

Gn,i

V˜k (L ∩ ξ)dµ(ξ) Z

Gn,i

S n−1

ρK (u)l−k ρL (u)k du

≤ V˜l (K)1−k/l V˜l (L)k/l . This gives the inequality V˜l (K) ≤ V˜l (L). ¤

10

B. RUBIN AND G. ZHANG

Lemma 2.5. Let 0 < k < l, and let L be a C ∞ origin-symmetric convex body of revolution that has positive curvature. If L ∈ / Ii,l−k then there exists another C ∞ origin-symmetric convex body of revolution K having positive curvature so that V˜k (K ∩ ξ) < V˜k (L ∩ ξ) for any ξ ∈ Gn,i , but V˜l (K) > V˜l (L). Proof. By Lemma 1.3, there is a unique C ∞ zonal function f on Gn,i so that ρl−k = R∗i f. L Since L ∈ / Ii,l−k , f is negative somewhere. Therefore, there is a C ∞ zonal function g > 0 on Gn,i such that Z f g < 0. Gn,i

By Lemma 1.3, g = Ri g1 for some g1 ∈ C1 , and we can define an origin-symmetric body of revolution K by ρkK = ρkL − εg1 where ε > 0 is sufficiently small. Using the argument from [G2, p. 439], one can show that K is convex and its boundary has positive curvature. It follows that Ri ρkK = Ri ρkL − εg. By (2.4), this gives V˜k (K ∩ ξ) < V˜k (L ∩ ξ) for any ξ ∈ Gn,i . But Z

Z

S n−1

k ρl−k L (ρL



ρkK )

Z

=ε S n−1

(R∗i f )g1



f g < 0. S n−1

Thus, the H¨older inequality gives Z

µZ

Z S n−1

ρlL

< S n−1

k ρl−k L ρK

This yields V˜l (K) > V˜l (L). ¤

≤ S n−1

¶1−k/l µZ ρlL

¶k/l S n−1

ρlK

.

GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM

11

3. Some elementary geometric lemmas Given a point z ∈ int(K) (the interior of K), define the extended radial function of K with respect to z, ρ(z, v) = sup{λ > 0 : z + λv ∈ K},

(z, v) ∈ Ω = int(K) × S n−1 .

Lemma 3.1. If a convex body K ∈ Kn has C m boundary ∂K, 1 ≤ m ≤ ∞, then the extended radial function ρ(z, v) is C m in Ω. Proof. Consider the function v = g(z, x) =

x−z , |x − z|

z ∈ int(K), x ∈ ∂K.

Since ∂K is C m , g(z, x) is a C m function in int(K) × ∂K. When z is fixed, g(z, ·) is a C m diffeomorphism from ∂K to S n−1 . By the implicit function theorem, x = f (z, v) is a C m function in Ω. Thus, ρ(z, v) = |x − z| = |f (z, v) − z| is a C m function in Ω. ¤ Lemma 3.2. For K ∈ Ken , there exist origin-symmetric convex bodies Kj ⊂ K of positive curvature such that ρKj (u) ∈ C ∞ (S n−1 ) and ρKj (u) → ρK (u) uniformly on S n−1 as j → ∞. Proof. Without loss of generality, assume that ρK (u) ≥ 1. It is well known that every origin-symmetric convex body can be approximated in the Hausdorff metric by originsymmetric convex bodies having C ∞ -boundary of positive curvature; see [S], pp. 158160. Therefore, there exist C ∞ origin-symmetric convex bodies of positive curvature Kj0 such that 1 |ρKj0 (u) − ρK (u)| < ∀u ∈ S n−1 . j+1 Let Kj =

j 0 j+1 Kj .

Then, obviously, ρKj (u) → ρK (u) uniformly on S n−1 as j → ∞, and ρK j =

j 1 j ρKj0 < (ρK + ) ≤ ρK . j+1 j+1 j+1

Thus, Kj ⊂ K. ¤ Lemma 3.3. If K is a C ∞ convex body of revolution with the radial function ρK , then the star body Kε , ε > 0, defined by p+1 ρpKε = ρpK − εpρK ,

is a C



p > 0,

convex body that has positive curvature when ε > 0 is small enough.

Proof. We only need to prove that the boundary of Kε has positive curvature. Note that ρKε = ρK − ερ2K + O(ε2 ). Let ρ = ρK − ερ2K . Abusing notation, one can write ρ(u) ≡ ρ(θ), θ being the angle 2 between u and the hyperplane xn = 0. Since ρ2K + 2ρ0K − ρK ρ00K ≥ 0, an elementary calculation gives 2

2

ρ2 + 2ρ0 − ρρ00 = (1 − 3ερK )(ρ2K + 2ρ0K − ρK ρ00K ) + ερ3K + O(ε2 ) > 0 when ε is small enough. This gives the desired result.

¤

12

B. RUBIN AND G. ZHANG

Lemma 3.4. Let C be a C 2 closed convex curve in the plane that encloses the origin. If (ρ, θ) are the polar coordinates of a point on the curve and κ(θ) is the curvature, then Z



2

1

κ(θ)(ρ(θ)2 + ρ0 (θ) ) 2 dθ = 2π.

0

Proof. Let s be the parameter of arc length of C, and let ϕ be the angle between the tangent line and the x-axis. Then ds 2 1 = (ρ(θ)2 + ρ0 (θ) ) 2 , dθ Thus

Z

Z



2π =

dϕ = 0

Z

`



κds = 0

dϕ = κ. ds

2

1

κ(θ)(ρ(θ)2 + ρ0 (θ) ) 2 dθ,

0

where ` is the perimeter of the curve. ¤ 4. Dual Radon transform formulas for radial functions of convex bodies It was discovered in [G1] and [Z2] that positive solutions to the Busemann-Petty problem in R3 and in R4 are intimately connected with the volume of parallel hyperplane sections of convex bodies. For i = n − 1, this connection evolves through appropriate representation of the inverse spherical Radon transform of the radial function ρK (u). The corresponding formulas were obtained in [G1] (n=3), [Z2] (n=4), and [GKS] (all n ≥ 3). A different proof of the formulas was given in [BFM]. In this section, we generalize these formulas to i-dimensional sections for all 2 ≤ i ≤ n − 1. For each convex body K ∈ Kn , we define the following functions: Z (4.1)

Ai (t, ξ) =

S n−1 ∩ξ ⊥

voli (K ∩ {tu + ξ})du,

ξ ∈ Gn,i ,

t ∈ R,

Z (4.2)

a(t, v) = S n−1 ∩v ⊥

vol1 (K ∩ {tu + Rv})du,

v ∈ S n−1 ,

t ∈ R.

The function (4.1) averages volumes of all i-dimensional sections of K parallel to ξ at distance |t| from the origin. The function (4.2) is the mean length of chords parallel to v at distance |t| from the origin. Note that (4.3)

a(0, v) = ρK (v).

GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM

Lemma 4.1. Let K ∈ Kn , 2 ≤ i < n. Then Z ∞ i−3 ∗ (4.4) Ri Ai (t, ·)(v) = σi−2 a(r, v)(r2 − t2 ) 2 rdr,

13

t ≥ 0.

t

Proof. This equality was established in [BFM] for i = n − 1. In the general case the proof is as follows. We denote gi (t, v) = R∗i Ai (t, ·)(v),

v ∈ S n−1 , x ∈ Rn .

b(x, v) = vol1 (K ∩ {x + Rv}),

Fix a unit vector in v1 ∈ v ⊥ and let ξ0 be a subspace of dimension i that contains v and v1 . Let SO(n − 1) be the group of rotations about v. Then Z gi (t, v) = Ai (t, αξ0 )dα SO(n−1) Z Z Z du0 dx dα = SO(n−1)

Z =

Z

S n−1 ∩ξ0⊥

dα SO(n−1)

Z =

Z

S n−1 ∩ξ0⊥

dα S n−1 ∩ξ0⊥ Z ∞

SO(n−1)

Z

= σi−2

dα Z

= σi−2 which gives (4.4).

SO(n−1)

x∈K∩{αξ0 +tαu0 }

Z du0

y∈ξ0 ∩v ⊥

Z du0

y∈ξ0 ∩v ⊥

b(α(y + tu0 ), v)dy p b( |y|2 + t2 αv1 , v)dy

p b( s2 + t2 αv1 , v)si−2 ds

0

Z



du S n−1 ∩v ⊥

b(ru, v)(r2 − t2 )

i−3 2

rdr,

t

¤

Let (4.5)

rK = sup{t > 0 : tB ⊂ K}

be the radius of inscribed ball in K. Lemma 4.2. If a convex body K ∈ Kn is C m , 1 ≤ m ≤ ∞, then the derivatives µ ¶j d (j) Ai (t, ξ) = Ai (t, ξ), 1 ≤ j ≤ m, dt are continuous in (−rK , rK ) × Gn,i . Proof. Let z = tu, |t| < rK , u ∈ S n−1 ∩ ξ ⊥ . Then Z σi−1 (4.6) voli (K ∩ {ξ + tu}) = ρ(tu, v)i dv, i n−1 S ∩ξ and by (4.1) we have σi−1 Ai (t, ξ) = i

Z

Z ρ(tu, v)i dv.

du S n−1 ∩ξ ⊥

The lemma follows from Lemma 3.1.

¤

S n−1 ∩ξ

14

B. RUBIN AND G. ZHANG

Theorem 4.3. If a convex body K ∈ Ken is C 2 , then Z 1 ∗ ∞ A2 (0, ·) − A2 (t, ·) (4.7) ρK = R dt, 2π 2 0 t2 1 (4.8) ρK = − R∗3 A003 (0, ·). 4π Proof. When u and v are fixed, the function r → b(ru, v) = vol1 (K ∩ {ru + Rv}) has compact support and is continuously differentiable except on the boundary of the support. Owing to (4.4) and the Fubini theorem, Z ∞ Z 2 2 − 21 g2 (t, v) = σi−2 (r − t ) rdr b(ru, v)du t S n−1 ∩v ⊥ Z Z ∞ 1 = −σi−2 du (r2 − t2 ) 2 b0 (ru, v)dr. S n−1 ∩v ⊥

By changing variable, we get 1 (g2 (t, v) − g2 (0, v)) t2 Z ³ Z du − = σi−2 S n−1 ∩v ⊥

Z = σi−2

S n−1 ∩v ⊥

³ Z du −



t

Z 1 2

2

0

(s − 1) b (tsu, v)ds + 1



´ sb0 (tsu, v)ds

0 ∞

2

1 2

Z

1

0

(s − (s − 1) )b (tsu, v)ds + 1

´ sb0 (tsu, v)ds .

0

Thus, by the Fubini theorem, Z ∞ Z ∞ A2 (t, ·) − A2 (0, ·) g2 (t, v) − g2 (0, v) ∗ R2 dt = dt t2 t2 0 0 Z Z ∞ 1 ´ ³ s − (s2 − 1) 2 ds − b(0, v) = σi−2 du − b(0, v) s S n−1 ∩v ⊥ 1 = −2πρK (v). This yields (4.7). To show (4.8), we differentiate (4.4) and obtain d ∗ R A3 (t, ·)(v) = −2πta(t, v). dt 3 Hence

¯ d2 ∗ ¯ R3 A3 (t, ·)(v)¯ = −2πa(0, v) = −4πρK (v). 2 dt t=0 By Lemma 4.2, one can differentiate under the sign of R∗3 that gives (4.8).

¤

The special case n = 3 of the formula (4.7) was obtained by Gardner [G1] in a slightly different form. The special case n = 4 of the formula (4.8) was proved by Zhang [Z2]. Both cases were given different proofs by Gardner, Koldobsky and Schlumprecht [GKS], and later by Barthe, Fradelizi and Maurey [BFM], Koldobsky [K3], and Rubin [R3]. The proof of formula (4.7) requires that the convex body is C 1 . In fact, (4.7) holds for any origin-symmetric convex body.

GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM

15

Theorem 4.4. If K ∈ Ken then the function 1 ϕ2 (ξ) = 2π

(4.9)

Z

∞ 0

A2 (0, ξ) − A2 (t, ξ) dt, t2

is well defined for almost all ξ ∈ Gn,2 , non-negative, and integrable on Gn,2 . Furthermore, for all v ∈ S n−1 , ρK (v) = (R∗2 ϕ2 )(v).

(4.10)

Proof. We fix v ∈ S n−1 , and set g(t) = R∗2 A2 (t, ·)(v), a(r) = a(r, v). By (4.4), Z ∞ hZ ∞ i g(0) − g(t) = 2t a(ts)ds − a(ts)(s2 − 1)−1/2 sds 0 1 Z ∞ = 2t a(ts)k(s)ds 0

where k(s) = 1 if s < 1 and k(s) = 1 − s(s2 − 1)−1/2 if s > 1. Then for ε > 0, 1 2

Z



g(0) − g(t) dt = t2

Z



dt t

Z



a(ts)k(s)ds Z ∞ Z ∞ ¡ y ¢ dt = a(y)dy k t t2 0 ε Z ∞ Z dy y/ε = a(y) k(s)ds. y 0 0

ε

ε

0

This gives 1 2

(4.11)

Z



ε

1 λ(y) = y

Z

Z

g(0) − g(t) dt = t2 y

0



a(εy)λ(y) dy, 0 −1/2

y − (y 2 − 1)+ k(s)ds = y

∈ L1 (0, ∞),

−1/2

where (y 2 − 1)+ = (y 2 − 1)−1/2 if y > 1 and 0 otherwise. Since a(r) is bounded and continuous at r = 0, the Lebesgue theorem on dominated convergence yields (4.12)

1 lim 2 ε→0

Z ε



g(0) − g(t) dt = a(0) t2

Z



λ(y)dy = 0

π a(0) = πρK . 2

To finish the proof we denote 1 ϕε (ξ) = 2π

Z ε



A2 (0, ξ) − A2 (t, ξ) dt. t2

16

B. RUBIN AND G. ZHANG

It is clear that ϕε is bounded on Gn,2 for each ε > 0. Moreover, since K is convex and origin-symmetric, then A2 (0, ξ) − A2 (t, ξ) ≥ 0 ∀t, ξ, and therefore ϕε (ξ) represents a sequence of non-decreasing (in ε) non-negative functions. The integrals R∗2 ϕε

1 = 2π

Z



ε

g(0) − g(t) dt t2

R∞ are uniformly bounded in ε because by (4.11), R∗2 ϕε ≤ π −1 ||a||∞ 0 λ(y)dy. Applying the Beppo Levi theorem [KF, p. 58] and using (4.12), we conclude that the limit 1 ϕ2 (ξ) = lim ϕε (ξ) = ε→0 2π

Z

∞ 0

A2 (0, ξ) − A2 (t, ξ) dt t2

exists a.e. on Gn,2 , ϕ2 is integrable on Gn,2 , and R∗2 ϕ2 = lim R∗2 ϕε = ρK . ε→0

¤

The next theorem extends (4.7) and (4.8) to the case of sections of arbitrary dimension 1 ≤ i < n. It generalizes known formulas for i = n − 1 which were obtained in [GKS] using Fourier transform techniques. Our proof is based on another idea, and might be useful in different occurrences. Theorem 4.5. If K ∈ Ken is C ∞ , then the radial function ρK can be represented by the dual Radon transform ρK = R∗i ϕi where (4.13) Z

1−i

π 2 ϕi (ξ) = ¡ 1−i ¢ Γ 2



t1−i

h

0

i−2

2 i X t2j (2j) Ai (t, ξ) − Ai (0, ξ) dt, (2j)! j=0

i even,

(4.14) i−1

i

(−1) 2 π 1− 2 (i−1) ¡ ¢ Ai ϕi (ξ) = (0, ξ), 2i Γ 2i

i odd.

Proof. Consider the following analytic family of functions associated to the body K: (4.15)

1 gλ (ξ) = Γ(λ/2)

Z K

|Pξ⊥ x|λ+i−n dx,

ξ ∈ Gn,i ,

0 < Reλ < n − i,

where |Pξ⊥ x| denotes the length of the orthogonal projection of x on ξ ⊥ . Integration in (4.15) over slices parallel to ξ gives 1 gλ (ξ) = Γ(λ/2) (4.16)

σn−i−1 = Γ(λ/2)

Z ξ⊥

Z

0

|y|λ+i−n voli (K ∩ {ξ + y})dy



tλ−1 Ai (t, ξ)dt.

GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM

17

On the other hand, by passing to polar coordinates we have Z σn−i−1 (4.17) gλ (ξ) = |P ⊥ u|λ+i−n ρλ+i K (u)du. (λ + i) Γ(λ/2) S n−1 ξ ∞ For f ∈ Ceven (S n−1 ), consider the generalized cosine transform [R2] Z σn−1 Γ((n − i − λ)/2) λ (4.18) (Ri f )(ξ) = f (u)|Pξ⊥ u|λ+i−n du. 2π (n−1)/2 Γ(λ/2) n−1 S

By Theorem 1.1 from [R2], (4.19)

Γ((n − i)/2) ∗ λ R R f = Qλ+i−1 f, Γ((n − 1)/2) i i

where Qλ+i−1 f is the spherical Riesz potential defined by (1.7). For f = ρλ+i K , combining (4.19), (4.18), and (4.17), we obtain (4.20)

∗ Qλ+i−1 ρλ+i K = cλ Ri gλ ,

cλ =

(λ + i) Γ((n − i)/2) Γ((n − i − λ)/2) . 2π (n−1)/2 Γ((n − 1)/2)

Owing to (1.8), analytic family Qα includes the identity operator (for α = 0). Hence analytic continuation (a.c.) of (4.20) at λ = 1 − i reads (4.21)

¯ £ ¤ ρK = c R∗i a.c. gλ ¯λ=1−i ,

c=

Γ((n − i)/2) . 2π (n−1)/2

We evaluate analytic continuation in the square brackets using (4.16). By the wellknown formula from [GS, Chapter 1, Sec. 3], for −` < Re λ < −` + 1, ` ∈ N, we have Z ∞ Z ∞ `−1 j h i X t (j) λ−1 λ−1 (4.22) a.c. t Ai (t, ξ)dt = t Ai (t, ξ) − Ai (0, ξ) dt. j! 0 0 j=0 Since all derivatives of Ai (t, ξ) of odd order are zero at t = 0, then for ` odd, the sum P`−1 P` j=0 can be replaced by j=0 , and (4.22) holds for −` − 1 < Re λ < −` + 1. It follows that for i even, one can set ` = i − 1 in (4.22) and obtain (4.13). On the other hand, the duplication formula for Γ-functions yields h 1 Z ∞ i 2λ−1 π 1/2 σn−i−1 tλ−1 Ai (t, ξ)dt , gλ (ξ) = (λ + i) cos(λπ/2) Γ((1 − λ)/2) Γ(λ) 0 and therefore [GS, Chapter 1], ¯ (−1)(i−1)/2 π 1/2 σn−i−1 (i−1) Ai a.c. gλ (ξ)¯λ=1−i = (0, ξ). 2i Γ(i/2) This equality together with (4.21) imply (4.14).

¤

18

B. RUBIN AND G. ZHANG

5. Positive answers The following theorem gives a partial solution to Problem B. Theorem 5.1. Let i = 2, or 3, k > 0, and K, L ∈ Ken . If Ik (K, ξ) ≤ Ik (L, ξ) for any ξ ∈ Gn,i , then Jk+1 (K) ≤ Jk+1 (L). Proof. Let K have a C ∞ boundary. Then t 7→ [voli (K ∩ {ξ + tu})]1/i ,

u ∈ ξ⊥,

is a concave function and has maximum at t = 0. It follows that A00i (0, ·) ≤ 0 (the derivative exists by Lemma 4.2), and A2 (0, ξ) − A2 (t, ξ) ≥ 0 ∀ξ. Hence Theorem 4.3 implies that K ∈ Ii,1 when i = 2, or 3. The desired result now follows by Lemma 2.4. If K is not smooth, one can pick a sequence {Kj } specified by Lemma 3.2. Since Kj ⊂ K, then Ik (Kj , ξ) ≤ Ik (K, ξ) ≤ Ik (L, ξ), and, by above, Jk+1 (Kj ) ≤ Jk+1 (L) for all j. It remains to pass to the limit in the last inequality, by taking into account that Jk+1 (Kj ) → Jk+1 (K) as ρKj (u) → ρK (u) uniformly on S n−1 . ¤ The following theorem gives a partial solution to Problem A and generalizes positive solutions to the Busemann-Petty problem in R3 and R4 ; cf. [G1], [GKS], [Z2]. Theorem 5.2. Let 1 ≤ k ≤ i, i = 2, or 3, K, L ∈ Sen . If K is convex, and V˜k (K ∩ ξ) ≤ V˜k (L ∩ ξ) for all ξ ∈ Gn,i , then V˜k+1 (K) ≤ V˜k+1 (L). Theorem 5.1 resolves Problems A and B only for l = k + 1. If we wish to extend it to any l > k then, by Lemma 2.4, we need representation (5.1)

∗ ρl−k K = Ri ϕ,

ϕ ≥ 0.

Theorem 4.5 does not provide it, but the proof of this theorem shows that one can arrive at (5.1) and express ϕ in the same manner as in (4.13), if we change the definition (4.1) of Ai (t, ξ). Let us sketch the basic idea [R4]. The expression voli (K ∩ {ξ + tu}) is the Euclidean Radon i-plane transform of the characteristic function χK (x) of the body K [He]. If we replace Ai (t, ξ) by the i-plane transform of the more general function |x|λ χK (x) with appropriate λ and proceed as in the proof of Theorem 4.5 we arrive at (5.1). This way leads to another class of star bodies rather than just convex bodies. Bodies of revolution. Theorems 5.1 and 5.2 can be essentially strengthened for bodies of revolution. To this end we use representation (1.5) the right hand side of which resembles the so-called Erdelyi-Kober fractional integral. Such integrals are well known in fractional calculus and arise in numerous applications; see [SKM, Section 18(1)] and references therein. We recall some basic facts.

GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM

19

For α > 0 and η ≥ −1/2, the Erdelyi-Kober fractional integral is defined by Z 2r−2(α+η) r 2 α (5.2) Iη f (r) = (r − s2 )α−1 s2η+1 f (s)ds. Γ(α) 0 For α = 0, we set Iηα f = f . Operators Iηα enjoy the composition law β Iηα Iη+α = Iηα+β .

(5.3)

This can be easily checked by changing the order of integration. The left inverse of Iηα has the form µ ¶m 1 d m−α α −1 −2η (5.4) (Iη ) f (r) = r r2η+2m Iη+α f (r), ∀m ≥ α, m ∈ N. 2r dr Owing to (1.5), we have R∗i ϕ(u) = c Iηα ϕ0 (r),

(5.5) c=

π (i−1)/2 σn−i−1 , σn−2

α=

i−1 , 2

η=

n−i − 1, 2

r = sin d(en , u).

Formulae (5.2)-(5.5) will be repeatedly used in the following. We recall that a body K is axially convex (with respect to the xn -axis) if any segment [A, B] parallel to the xn -axis lies in K provided that A, B ∈ K. Theorem 5.3. Let i = 2 or 3, k > 0, and 0 < p ≤ n − i. If K, L ∈ Sen and K is an axially convex body of revolution satisfying Ik (K, ξ) ≤ Ik (L, ξ) for any ξ ∈ Gn,i , then Jk+p (K) ≤ Jk+p (L). Proof. It is enough to prove the result when ρK ∈ C ∞ (S n−1 ). In this case ρpK ∈ C1 , and, by Lemma 1.3, ρpK = R∗i ϕ, ϕ ∈ Ci . According to Lemma 1.1, we set ρK (u) ≡ ρ(r), ϕ(ξ) ≡ ϕ0 (s), where r = sin d(en , u), s = sin d(en , ξ). By (5.5), (5.6)

ρ(r)p = c Iηα ϕ0 (r).

If i = 3, then α = 1, 2η = n − 5, and (5.4) (with m = 1) yields (5.7)

c ϕ0 (r) =

1 4−n n−3 r (r ρ(r)p )0 . 2

This expression is non-negative for 0 ≤ p ≤ n−3 because rρ(r) is non-decreasing thanks to axial convexity of K. Therefore, K ∈ I3,p . If i = 2, then α = 1/2, 2η = n − 4, and by (5.4) we have ´ r3−n d ³ n−2 1/2 p c ϕ0 (r) = r I(n−3)/2 ρ (r) 2 dr Z r r3−n d (5.8) = 1/2 (r2 − s2 )−1/2 sn−2 ρ(s)p ds dr 0 π Z 1 r3−n d (5.9) = 1/2 (1 − t2 )−1/2 (rt)n−2 ρ(rt)p dt. dr 0 π

20

B. RUBIN AND G. ZHANG

If 0 ≤ p ≤ n − 2, then rn−2 ρ(r)p = rn−2−p (rρ(r))p is non-decreasing since rρ(r) is non-decreasing. It follows that the integral in (5.9) is a non-decreasing function of r. Hence ϕ0 ≥ 0, and therefore, K ∈ I2,p . The desired result now follows from Lemma 2.4. ¤ For n = 3, i = 2, Theorem 5.3 was proved by Hadwiger [H]. In the context of Problem A we have the following Corollary 5.4. Let i = 2 or 3, 1 ≤ k ≤ i, k < l ≤ n + k − i. If K, L ∈ Sen and K is an axially convex body of revolution satisfying V˜k (K ∩ ξ) ≤ V˜k (L ∩ ξ) for any ξ ∈ Gn,i , then

V˜l (K) ≤ V˜l (L). 6. Negative answers

To prove negative results, we need counterexamples. Our counterexamples are perturbations of a cylinder, and constructed as follows: We take a rectangle centered at the origin in the two-plane (x1 , xn ): {(x1 , xn ) : |x1 | ≤ 1, |xn | ≤ h}, h > 0. Let us smoothen each corner of the rectangle to get a closed convex curve C which is symmetric about the axes. Specifically, for small ε > 0, around the corner (1, h), we connect the points (1 − ε, h) and (1, h − ε) by a C ∞ convex curve that is C ∞ tangent to the rectangle at both points, and do the same for other corners. Then the curve C is C ∞ and contains four line segments. As an explicit example, one can divide the closed convex curve 2 2 2 e−1/x1 + e−1/xn = e−1/ε into four symmetric parts and put them at the four corners of the rectangle. Rotating the curve C about the xn -axis, we get a C ∞ convex body of revolution Lε . Theorem 6.1. For all k > 0, in the cases (a)

1 < i ≤ 3, p > n − i,

and

(b)

i ≥ 4, p > 0,

there exist origin-symmetric convex bodies of revolution K and L in Rn such that Ik (K, ξ) ≤ Ik (L, ξ) for any ξ ∈ Gn,i , but Jk+p (K) > Jk+p (L). Proof. By Lemma 2.5, it suffices to show that there exists a C ∞ origin-symmetric convex body of revolution with positive curvature that is not in Ii,p . By Lemma 3.3, we only need to construct a C ∞ origin-symmetric convex body of revolution L ∈ / Ii,p . That is, if (6.1)

ρpL = R∗i ϕ,

GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM

21

then ϕ is negative somewhere. Let L be the body Lε constructed above. Note that for some ωε > 0, (6.2)

ρL (u) =

1 , sin ω

ωε ≤ ω ≤

π , 2

where ω is the geodesic distance between u and the north pole. Let ρ(r) = ρL (u), r = sin ω, rε = sin ωε . Then ρ(r) = 1/r ∀r ∈ [rε , 1]. Let us consider all cases step by step. 1◦ . Let i = 2, p > n − 2. We choose a long smoothened cylinder Lε (h is large). As in (5.8), we have cπ where

1/2 n−3

r

d ϕ0 (s) = ds d g1 (s) = ds

Z

s

(s2 − r2 )−1/2 rn−2 ρ(r)p dr = g1 (s) + g2 (s)

0

Z



(s2 − r2 )−1/2 rn−2 ρ(r)p dr < 0,

0

and d g2 (s) = ds

Z

s

(s2 − r2 )−1/2 rn−2−p dr



Z i d h n−2−p 1 2 −1/2 n−2−p = s (1 − t ) t dt . ds rε /s

An elementary calculation gives g2 (s) = s−2 rεn−1−p [a(rε /s) + b(rε /s)], Z p+1−n

1

a(x) = (n − 2 − p)x

(1 − t2 )−1/2 tn−2−p dt,

b(x) = (1 − x2 )−1/2 ,

x

x = rε /s ∈ [rε , 1]. Note that rε is very small when the height of Lε is large. Therefore, the statement will be proved if we show that (6.4)

lim [a(x) + b(x)] < 0.

x→+0

Indeed, in this case there exists xε ∈ (0, 1) such that a(xε ) + b(xε ) < 0. If we choose ωε in (6.2) so that sin ωε = xε , then g2 (1) = xn−1−p [a(xε ) + b(xε )] < 0, and we are done. ε Let us check (6.4). If n − 1 − p ≥ 0 then a(x) → −∞, b(x) → 1, and (6.4) follows. In the case n − 1 − p < 0 we have lim [a(x) + b(x)] =

x→+0

1 n−2−p +1= < 0. p+1−n n−1−p

22

B. RUBIN AND G. ZHANG

2◦ . Let i = 3, p > n − 3. Then for r ∈ [rε , 1], (5.7) yields 2cϕ0 (r) = r4−n (rn−3−p )0 = (n − 3 − p)r−p < 0. 3◦ . Let i = 4. Then α = 32 , and η = By (5.6),

n 2

− 3. We choose the example Lε so that h = 1. 3

ρ(r)p = cI n2 −3 ϕ0 (r), 2

and therefore (5.4) and (5.2) yield ¡ 1 d ¢2 n−2 ¡ 12 ¢ r I n−3 ρ(·)p (r) 2r dr 2 Z 1 2r6−n ¡ 1 d ¢2 r 2 = √ (r − s2 )− 2 sn−2 ρ(s)p ds. π 2r dr 0

cϕ0 (r) = r6−n

Using integration by parts and differentiation twice, we have r6−n cϕ0 (r) = √ 2 π

Z

r

1

(r2 − s2 )− 2 [(sn−3 ρp )0 /s]0 ds.

0

Now we set s = sin θ and get ¡ 1 ¢ 2 n2 −4 cϕ0 √ = √ π 2

Z

π 4

0

g(θ)

¡1 ¢− 1 − sin2 θ 2 dθ, 2

where d (ρp sinn−3 θ) ´ d ³ dθ = a(θ) − b(θ)D(θ)sinn−4 θ, dθ sin θ cosθ ¡ 1 ¢ p−1 dρ a(θ) = (n−3)(n−5)ρp sinn−6θ cosθ + sinn−5θ 2n−8 + pρ cos2 θ dθ ¡ dρ ¢2 ¢ sinn−4 θ p−2 ¡ 2 + pρ ρ + (p + 1) , cosθ dθ pρp−2 ¡ 2 ¡ dρ ¢2 ¢ b(θ) = ρ + , cosθ dθ ¡ ¡ dρ ¢2 ¢ 12 D(θ) = κ(θ) ρ2 + , dθ ¡ ¢2 2 ρ2 + 2 dρ − ρ ddθρ2 dθ κ(θ) = £ ¡ ¢2 ¤3/2 . ρ2 + dρ dθ

g(θ) =

(6.5)

Since ρ and M so that

dρ dθ

are uniformly bounded for any 0 < ε < 14 , there are constants b0 and |a(θ)| < M,

b(θ) > b0 > 0

GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM

for 0 ≤ θ ≤

π 4.

Let M1 = M

R

π 4

¡1

0

2

− sin2 θ

¡ 1 ¢ cϕ0 √ < M1 − b0 2

(6.6)

¢− 12

Z

π 4

0

23

dθ. Then,

D(θ) ¡

sinn−4 θ 1 2

− sin2 θ

¢ 12 dθ.

Since the curve C converges to a square as ε → 0, the integral in (6.6) can be arbitrarily large. Indeed, one can choose δ > 0 so that ¡1 2

sinn−4 θ

8M1 , 1 > ¢ πb0 − sin2 θ 2

∀θ ∈

¡π π¢ − δ, . 4 4

Then we choose ε > 0 of Lε such that κ(θ) = 0 for 0 ≤ θ ≤ π4 − δ. By Lemma 3.4, we obtain Z π4 Z π4 sinn−4 θ 8M1 2M1 D(θ) ¡ D(θ)dθ = . 1 dθ > ¢ 2 πb b 1 2 0 0 0 0 2 − sin θ ¡ 1 ¢ Therefore, by (6.6), ϕ0 √2 < −M1 /c < 0 when ε is small enough. 4◦ . Let i ≥ 5. Again, we choose the example Lε so that h = 1. By (5.6) and (5.3), α−2 ρ(r)p = cIηα ϕ0 = cIη2 Iη+2 ϕ0 = Iη2 ψ, α−2 where ψ = cIη+2 ϕ0 . By (5.4),

ψ(r) = ri+2−n ¯

dρ ¯ dφ φ= π 4

Let r = sin φ. Note that

¡ 1 ¢ 1 ψ √ = 2 2

¡ 1 d ¢2 n−i+2 r ρ(r)p . 2r dr

= 0. An elementary calculation gives

µ ¶ 2 p p−1 d ρ (n − i + 2)(n − i)ρ + pρ . dφ2 φ= π 4

Since the curvature κ of the curve C at φ = π4 tends to +∞ as ε → 0, (6.5) implies that ¡ 1 ¢ d2 ρ √ tends to −∞ as ε → 0. Therefore, ψ < 0 when ε is small enough. It follows 2 dφ 2 that ϕ0 is negative somewhere when ε is sufficiently small. ¤ Remark 6.2. In the case i ≥ 4, p ≥ n − i, one can give a simpler proof of Theorem 6.1. Namely, by using notation of Lemma 1.2, from (1.5), (6.1) and (6.2), we have Z (6.7)

r

n−i−p

= c2 0

r

µ ¶(i−3)/2 s2 1− 2 sn−i−1 ϕ0 (s) ds, r

∀r ∈ [rε , 1],

where r = sin ω, ϕ0 (sin d(en , ξ)) = ϕ(ξ). If ϕ0 ≥ 0 and p ≥ n − i then the left hand side of (6.7) does not increase on [rε , 1], whereas the right hand side is increasing because its derivative is positive. This contradiction shows that ϕ0 is negative somewhere. For p = n − k, Theorem 6.1 (or Remark 6.2) implies the following

24

B. RUBIN AND G. ZHANG

Corollary 6.3. If 1 ≤ k ≤ i and 4 ≤ i < n, then there exist origin-symmetric convex bodies of revolution K and L in Rn such that Ik (K, ξ) ≤ Ik (L, ξ) for any ξ ∈ Gn,i , but voln (K) > voln (L). If k = i this corollary gives the known negative answer to the generalized BusemannPetty problem for i-dimensional sections of origin-symmetric convex bodies in Rn in the case 4 ≤ i < n. This result was established in [BZh] and [K2] by different methods. The following table is a summary of solutions to Problem A when K is a body of revolution. Table 1 i k 2 1 2 1 2 2 3 1≤k≤3 3 1≤k≤3 i≥4 1≤k
l Answer n No Theorem 6.1 1
The following theorem was proved in [BZh]. Theorem A.1. (see [BZh], Theorem 1.3). For 3 < i < n, there exist origin-symmetric convex bodies of revolution K and L in Rn so that voli (K ∩ ξ) < voli (L ∩ ξ),

∀ξ ∈ Gn,i ,

but voln (K) > voln (L). The proof of this theorem used the following Lemma A.2. (see Lemma 3.2 in [BZh]). For 3 < i < n, there exist a C ∞ convex body K and a C ∞ function g so that ¡

Ri ρi−4 K g

¢

Z < 0, S n−1

ρn−4 K g > 0.

The proof of this lemma given in [BZh] has certain gap. We give a correct proof here. Proof of Lemma A.2. We shall seek a C ∞ convex body K and a C ∞ function g that are SO(n − 1)-invariant. One can take K to be a smoothened cylinder Lε described in the beginning of Section 6. By Lemma 1.3, (A.1)

ρn−i = R∗i ϕ, K

GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM

25

and (1.5) yields Z (A.2)

r

µ

1 = c2 0

s2 1− 2 r

¶(i−3)/2 sn−i−1 ϕ0 (s) ds,

∀r ∈ [rε , 1],

where r = sin ω, ϕ0 (sin d(en , ξ)) = ϕ(ξ). If ϕ0 ≥ 0 then the right hand side of (A.2) is increasing because its derivative is positive. This contradiction shows that the function ϕ(ξ) in (A.1) R is negative somewhere. Then we construct g1 ∈ Ci so that g1 (ξ) < 0 ∀ξ ∈ Gn,i, and Gn,i ϕg1 > 0. To this end we take the absolute value of g1 sufficiently small where ϕ ≥ 0 and sufficiently large where ϕ < 0. By Lemma 1.3, there is g ∈ C1 satisfying g1 = Ri (ρi−4 K g). Now by (A.1) and (1.2), we have Z

Z S n−1

ρn−4 K g

Z

= S n−1

R∗i ϕ ρi−4 K g

=

ϕg1 > 0, Gn,i

and the lemma is proved. ¤ Acknowledgments. The paper was finished when the first-named author was visiting Polytechnic University of New York in the Spring of 2003. He is very grateful to Professors Erwin Lutwak and Gaoyong Zhang for their hospitality. References [A]

S. Alesker, Continuous rotation invariant valuations on convex sets, Annals of Math. 149 (1999), 977–1005. [B] K. Ball, Some remarks on the geometry of convex sets, in: Geometric aspects of Functional Analysis —1986-1987 (J. Lindenstrauss and V. Milman. Eds), Lecture Notes in Math., vol. 1317, Springer-Verlag, Berlin, 1988, 224-231. [BFM] F. Barthe, M. Fradelizi and B. Maurey, A short solution to the Busemann-Petty problem, Positivity 3 (1999), 95–100. [Bo] J. Bourgain, On the Busemann-Petty problem for perturbations of the ball, Geom. Funct. Anal. 1 (1991), 1–13. [BZh] J. Bourgain and G. Zhang, On a generalization of the Busemann-Petty problem, Convex Geometric Analysis, MSRI Publications 34 (1998), (K. Ball and V. Milman, Eds), Cambridge University Press, New York, 1998, 65–76. [BZ] Yu. D. Burago, and V. A. Zalgaller, Geometric inequalities, Springer-Verlag, Berlin, 1988. [BP] H. Busemann and C. M. Petty, Problems on convex bodies, Math. Scand. 4 (1956), 88–94. [G1] R. J. Gardner, A positive answer to the Busemann-Petty problem in three dimensions, Annals of Math. 140 (1994), 435–447. [G2] , Intersection bodies and the Busemann-Petty problem, Trans. Amer. Math. Soc. 342 (1994), 435-445. [G3] , Geometric tomography, Cambridge University Press, New York, 1995. [GKS] R. J. Gardner, A. Koldobsky, and T. Schlumprecht, An analytic solution to the Busemann-Petty problem on sections of convex bodies, Annals of Math. 149 (1999), 691–703. [GS] I. M. Gelfand, and G. E. Shilov, Generalized functions, vol. 1, Properties and Operations,, Academic Press, New York, 1964. [Gi] A. Giannopoulos, A note on a problem of H. Busemann and C. M. Petty concerning sections of symmetric convex bodies, Mathematika 37 (1990), 239–244. [GZ] P. Goodey and G. Zhang, Inequalities between projection functions of convex bodies, Amer. J. Math. 120 (1998), 345–367.

26

B. RUBIN AND G. ZHANG

[GrR] E. Grinberg and B. Rubin, Radon inversion on Grassmannians via G˚ arding-Gindikin fractional integrals, Annals of Math., (in press). [GrZ] E. Grinberg and G. Zhang, Convolutions, transforms, and convex bodies, Proceedings of London Math Soc. (3)78 (1999), 77-115. [H] H. Hadwiger, Radialpotenzintegrale zentralsymmetrischer Rotations-k¨ orper und Ungleichheitsaussagen Busemannscher Art, Math. Scand. 23 (1968), 193–200. [He] S. Helgason, The Radon transform, Birkh¨ auser, Boston, Second edition, 1999. D.A. Klain, Invariant valuations on star-shaped sets, Advances in Math. 125 (1997), 95–113. [Kl] [K1] A. Koldobsky, Intersection bodies, positive definite distributions and the Busemann-Petty problem, Amer. J. Math. 120 (1998), 827–840. [K2] , A functional analytic approach to intersection bodies, Geom. Funct. Anal. 10 (2000), 1507–1526. [K3] , The Busemann-Petty problem via spherical harmonics, Preprint (2002). [KF] A.N. Kolmogorov, and S.V. Fomin, Elements of the theory of functions and functional analysis, volume 2, Graylock Press, Albany, N.Y., 1961. [LR] D. G. Larman and C. A. Rogers, The existence of a centrally symmetric convex body with central cross-sections that are unexpectedly small, Mathematika 22 (1975), 164–175. E. Lutwak, Intersection bodies and dual mixed volumes, Adv. Math. 71 (1988), 232–261. [Lu] [Mc1] P. McMullen, Non-linear angle-sum relations for polyhedral cones and polytopes, Math. Proc. Cambridge Phil. Soc. 78 (1975), 247–261. [Mc2] , Inequalities between intrinsic volumes, Mh. Math. 111 (1991), 47–53. [M] C. M¨ uller, Spherical harmonics, Springer, Berlin-Heidelberg-New York, 1966. [Ne] U. Neri, Singular integrals, Springer, Berlin, 1971. [P] M. Papadimitrakis, On the Busemann-Petty problem about convex, centrally symmetric bodies in Rn , Mathematika 39 (1992), 258–266. [PBM] A.P. Prudnikov, Y.A. Brychkov and O.I. Marichev, Integrals and series: Special Functions, Nauka, Moscow, 1983 (Russian). [R1] B. Rubin, Fractional integrals and potentials, Pitman Monographs and Surveys in Pure and Applied Mathematics, 82, Longman, Harlow, 1996. [R2] , Inversion formulas for the spherical Radon transform and the generalized cosine transform, Advances in Appl. Math., 29 (2002), 471-497. , Notes on Radon transforms in integral geometry, Fractional Calculus and Applied [R3] Analysis, 6 (2003), 25–72. [R4] , Analytic families associated to Radon transforms in integral geometry, Lecture delivered at the PIMS, Vancouver, July 1-5, 2002. [SKM] S.G. Samko, A.A. Kilbas, and O.I. Marichev, Fractional integrals and derivatives. Theory and applications, Gordon and Breach Sc. Publ., New York, 1993. [Sa] L.A. Santalo, Integral geometry and geometric probability, Addison-Wesley Publ. Comp., 1976. [S] R. Schneider, Convex Bodies: The Brunn-Minkowski Theory, Cambridge University Press, 1993. [VK] N. Ja. Vilenkin and A.V. Klimyk, Representations of Lie groups and special functions, Vol. 2, Kluwer Academic publishers, Dordrecht, 1993. [Z1] G. Zhang, Sections of convex bodies, Amer. J. Math. 118 (1996), 319–340. [Z2] , A positive solution to the Busemann-Petty problem in R4 , Ann. of Math. 149 (1999), 535–543.

Boris Rubin Institute of Mathematics Hebrew University Jerusalem 91904, ISRAEL E-mail: [email protected]

GENERALIZATIONS OF THE BUSEMANN-PETTY PROBLEM

Gaoyong Zhang Department of Mathematics Polytechnic University 6 Metrotech Center Brooklyn, NY 11201, U.S.A. E-mail: [email protected]

27

Generalizations of the Busemann-Petty problem for ...

Let s be the parameter of arc length of C, and let ϕ be the angle between the tangent line and the x-axis. Then ...... 91904, ISRAEL. E-mail: [email protected].il ...

269KB Sizes 1 Downloads 165 Views

Recommend Documents

The quest for generalizations over consonants
ing vowels altogether, a trend toward generalization over consonants emerges. ... Consonants and vowels differ in a number only within phonological ..... in isolation, using a Sony ECM microphone connected to a laptop computer via an ...

generalizations of the dennis-moré theorem1
in Rn containing ¯x, a zero of f, the Jacobian mapping ∇f is continuous at ¯x and ... x to a set C. We employ the following property of set-valued mappings:.

The Lemoine Cubic and Its Generalizations
May 10, 2002 - and efficient help. Without them, this paper would never have been the ..... APa, BPb, CPc. The tangents at P to the cubic are tangent to both ...

INTERVAL MV-ALGEBRAS AND GENERALIZATIONS ...
Oct 7, 2009 - algebra, triangularization, interval functor, interval Lukasiewicz logic, t-norm, ... many classes of ordered algebras related with logical systems.

English-Language Spelling Pattern Generalizations - TPRI
sign, sit, master, loss. The letter s is almost always doubled when it comes at the end of a one-syllable word and is preceded by one short vowel (FLOSS rule). ce.

Limits of performance for the model reduction problem ...
bounds on the norm of the difference between a hidden Markov model of order n and any lower order approximant of order n

the-problem-of-ordination.pdf
layingAonAofAhands!in!the!New!Testament,!therefore,!it!is!clear!that!the!rite!had!its!roots!in!the!anA. cient!Hebrew!practices.! Like!in!the!Old!Testament,!the!New!

The Problem of Social Cost
Page 1 .... be solved in a completely satisfactory manner: when the damaging business has. *Coase. ...... the timber. Then it would have been the wall-builder ...

Problem of the Week
Power Trip. The power 67 equals 279 936 when expressed as an integer and its last two digits are 36. When 62015 is expressed as an integer, what are its last ...

Simulating History: The Problem of Contingency - CiteSeerX
outcome measures that affect the security and prosperity of the world community. The problem of contingency for testing hypotheses about the causes of certain.

Simulating History: The Problem of Contingency - CiteSeerX
Analyses of Social Issues and Public Policy, Vol. 3, No. 1, 2003. Simulating History: The Problem of Contingency. David R. Mandel∗. University of Victoria.

pdf-12101\strategies-for-creative-problem-solving-gs1140-problem ...
... the apps below to open or edit this item. pdf-12101\strategies-for-creative-problem-solving-gs1 ... stitute-custom-edition-from-usa-pearson-custom-ed.pdf.

pdf-12101\strategies-for-creative-problem-solving-gs1140-problem ...
... the apps below to open or edit this item. pdf-12101\strategies-for-creative-problem-solving-gs1 ... stitute-custom-edition-from-usa-pearson-custom-ed.pdf.

The Pendulum Problem
3 They will say it is wrong to be married and wrong ... Renounces making creation into an idol. Renounces ... ever-living God, they worshiped idols made to look.

The Pendulum Problem
tiles. 24 So God abandoned them to do whatever shame- ful things their hearts desired. As a result, they did vile and degrading things with each otherʼs bodies. 25 They traded the truth about God for a lie. So they worshiped and served the things Go

Border Security: The Rising Problem for the United ...
Border Security: The Rising Problem For The United States. 1. Border Security: The Rising ... always been a controlled open border. Meaning that it was possible ...

The Pendulum Problem
Page 1 ... worshiped and served the things God created in- stead of the Creator himself, who is worthy of eternal praise! Amen.” Continued study :: other helpful passages. Passage dealing with the freedom to use Godʼs good creation. Colossians 2:6

SHORT COmmuniCaTiOnS The Problem of understanding of nature in ...
Feb 6, 2014 - Vol. 2, no. 2 (autumn 2014). The nature of understanding (comprehending). Knowledge of something is not understanding if it comprises ignorance. Understanding is a process, a human movement toward wisdom, toward seeing the unity of dive

SHORT COmmuniCaTiOnS The Problem of understanding of nature in ...
Feb 6, 2014 - human beings are not conscious beings; they are incapable of empathy, incapable of making their own decisions based on free will. The fact that exact science sees only 'laws of nature'—that is, physical or scientific laws—does not m