A Note on Strong Duality and Complementary Slackness Chee-Khian Sim Last updated: 27 August 2015 Abstract In this note, we consider the equivalence between strong duality and complementary slackness by studying the class of linear conic optimization problems. Keywords. Lagrangian function; Primal problem; Dual problem; Strong duality; Complementary slackness.

1

Lagrangian Function and Primal-Dual Problem

We start off by considering a function, which we called the Lagrangian function, Φ : X × Y → <, where X ⊆ H1 and Y ⊆ H2 , H1 , H2 are two Hilbert spaces. The primal problem (P) associated with this Lagrangian function is given by   min sup Φ(x, y) , x∈X

while the dual problem (D) associated with this Lagrangian function is given by   max inf Φ(x, y) . y∈Y

(1)

y∈Y

x∈X

l Example 1.1 For x ∈ X ⊆
Φ(x, u, v) = f (x) +

m X

ui gi (x) +

i=1

l X

vi hi (x).

i=1

Then the primal problem (P) associated with Φ(x, u, v) is given by min f (x)

x∈
1

(2)

subject to gi (x) ≤ 0, i = 1, . . . , m, hi (x) = 0, i = 1, . . . , l, x ∈ X, since for x ∈ X,  sup Φ(x, u, v) = (u,v)∈Y

f (x) , if gi (x) ≤ 0, i = 1, . . . , m, hi (x) = 0, i = 1, . . . , l, +∞ , otherwise,

while the dual problem (D) associated with Φ(x, u, v) is given by max

u∈
θ(u, v)

subject to u ≥ 0, Pl P where θ(u, v) = inf{f (x) + m i=1 vi hi (x) ; x ∈ X}. See [1]. i=1 ui gi (x) + Now, it is clear the weak duality holds for the above Problem (1) and Problem (2), that is,     min sup Φ(x, y) ≥ max inf Φ(x, y) . x∈X

y∈Y

y∈Y

x∈X

Strong duality holds if the optimal values of both problems are equal, that is,     min sup Φ(x, y) = max inf Φ(x, y) . x∈X

y∈Y

y∈Y

x∈X

Given x ∈ X and y ∈ Y , we say that strong duality holds for (x, y) if sup Φ(x, y) = inf Φ(x, y). y∈Y

x∈X

Remark 1.1 It is easy to see that strong duality holds for (x, y) if and only if x an optimal solution for Problem (1) and y an optimal solution for Problem (2), with common optimal value. As a side note, we now relate strong duality with the Lagrangian function Φ(x, y). A solution (x, y) is called a saddle point of the Lagrangian function Φ(x, y) if x ∈ X, y ∈ Y , and Φ(x, y) ≤ Φ(x, y) ≤ Φ(x, y), for all x ∈ X, y ∈ Y. We have the following claim relating the saddle point of a Lagrangian function with the optimal solutions to its primal and dual problems, and which also relates strong duality with the function: 2

Claim 1.1 (x, y) is a saddle point of Φ(x, y) if and only if x is an optimal solution to (P) and y is an optimal solution to (D), and the primal and dual optimal values are equal (that is, strong duality holds). Furthermore, the common optimal value is given by Φ(x, y). Proof: (⇒) Note that Φ(x, y) = maxy∈Y Φ(x, y) and Φ(x, y) = minx∈X Φ(x, y). The conclusion then follows by Φ(x, y) = max Φ(x, y) y∈Y

≥ inf max Φ(x, y) x∈X y∈Y

≥ inf Φ(x, y) x∈X

= Φ(x, y) = max Φ(x, y) y∈Y

≥ max inf Φ(x, y) y∈Y x∈X

≥ inf Φ(x, y) x∈X

= Φ(x, y). (⇐) It follows from Φ(x, y) ≤ sup Φ(x, y) y∈Y

= min sup Φ(x, y) x∈X y∈Y

= Φ(x, y) = max inf Φ(x, y) y∈Y x∈X

=

inf Φ(x, y)

x∈X

≤ Φ(x, y), for all x ∈ X, y ∈ Y. 2

2

Linear Conic Optimization Problem - A Consideration

We consider the following linear conic optimization problem: min hc, xiH1

x∈H1

subject to Ax = b, Bx − d ∈ K, 3

(3)

where x, c ∈ H1 , b ∈ H2 , d ∈ H3 , A is a linear map from H1 to H2 , B is a linear map from H1 to H3 , K ⊆ H3 is a nonempty closed convex cone. Here, H1 , H2 , H3 are finite dimensional Hilbert spaces. For x ∈ H1 , u ∈ H2 , v ∈ −K ∗ , we formed the Lagrangian function that arises from the above linear conic optimization problem as Φ(x, u, v) = hc, xiH1 + hu, Ax − biH2 + hv, Bx − diH3 , where K ∗ ⊆ H3 is the dual cone of K and is defined as K ∗ = {y ∈ H3 ; hy, wiH3 ≥ 0 ∀w ∈ K}. Note that we have K ∗∗ = K, since K is a nonempty closed convex cone (see for example [3]). The primal problem (P) associated to the above Lagrangian function is exactly the above linear conic optimization problem, since it is easy to see that sup

Φ(x, u, v) =

u∈H2 ,−v∈K ∗

sup u∈H2 ,−v∈K ∗

 =

hc, xiH1 + hu, Ax − biH2 + hv, Bx − diH3

hc, xiH1 , if Ax = b, Bx − d ∈ K ∗∗ = K, +∞ , otherwise.

Let us now find the dual problem (D) associated to this Lagrangian function. Observe that for u ∈ H2 and v ∈ −K ∗ , inf Φ(x, u, v) =

x∈H1

inf hc, xiH1 + hu, Ax − biH2 + hv, Bx − diH3

x∈H1

inf hc + A∗ u + B ∗ v, xiH1 − hu, biH2 − hv, diH3  −hu, biH2 − hv, diH3 , if c + A∗ u + B ∗ v = 0, = −∞ , otherwise. =

x∈H1

where A∗ , B ∗ are the adjoint of A, B respectively. Hence, the dual problem (D) is given by: max

u∈H2 ,v∈H3

−hu, biH2 − hv, diH3

(4)

subject to c + A∗ u + B ∗ v = 0, v ∈ −K ∗ . Let us equate the objective function values of Problem (3) and Problem (4). Upon algebraic manipulations using primal and dual feasibilities, the following holds: −hu, biH2 − hv, diH3 = hc, xiH1 ⇔ −hu, AxiH2 − hv, diH3 = h−A∗ u − B ∗ v, xiH1 ⇔ −hA∗ u, xiH1 − hv, diH3 = −hA∗ u, xiH1 − hv, BxiH3 ⇔ hv, Bx − diH3 = 0. 4

We called hv, Bx − diH3 = 0 the complementary slackness condition for Problems (3) and (4), and we say complementary slackness condition holds for (x, u, v). Remark 2.1 As a rule of thumb, the complementary slackness condition can be obtained easily for a given primal-dual linear conic program pair by forming “inner product” of the primal cone constraint “left-hand side” with the corresponding dual cone constraint “left-hand side”, and setting the “inner product” equal to zero. The main result for this note then follows immediately: Claim 2.1 Given x feasible to Problem (3), and (u, v) feasible to Problem (4), complementary slackness condition holds for (x, u, v) if and only if strong duality holds for (x, u, v). Remark 2.2 In practice, to find primal and dual optimal solutions, we only need to find (x, u, v) that is primal and dual feasible and also satisfies the complementary slackness condition. Then Claim 2.1 holds, and it then follows from Remark 1.1 that x is an optimal solution to Problem (3) and (u, v) is an optimal solution to Problem (4), with common optimal value. Example 2.1 (Linear Program) H1 = H3 =
x∈
subject to Ax = b, x ≥ 0, and Problem (4) becomes max

u∈
−bT u

subject to c + AT u + v = 0, v ≤ 0. The complementary slackness condition is then the usual complementary slackness condition, xT v = 0, that is considered in linear programming. Analogous semi-definite program example to the above linear program example can be obtained in a similar manner. A natural question to ask next is: “Under what conditions do we have strong duality?”. A reason we ask this is that if we have strong duality, then we can for example use 5

primal, dual feasibility conditions and complementary slackness condition to attempt to “solve” for primal and dual optimal solutions. It is well-known that with certain constraint qualifications, such as the existence of x ∈ H1 with Ax = b and Bx − d ∈ int(K), that is, Problem (3) has a strictly feasible solution, then strong duality holds and there exists an optimal solution to Problem (4) (see for example [2]) . We will consider this aspect of strong duality (or equivalently complementary slackness) in future.

References [1] M. S. Bazaraa, H. D. Sherali & C. M. Shetty, Nonlinear Programming: Theory and Algorithms, 2nd Edition, John Wiley & Sons, Inc., 1993. [2] S. Boyd & L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004. [3] R. T. Rockafellar, Convex Analysis, Princeton Landmarks in Mathematics, Princeton University Press, 1970.

6

A Note on Strong Duality and Complementary Slackness

Aug 27, 2015 - where x, c ∈ H1, b ∈ H2, d ∈ H3, A is a linear map from H1 to H2, B is a .... Convex Analysis, Princeton Landmarks in Mathematics, Princeton.

209KB Sizes 1 Downloads 314 Views

Recommend Documents

Note on Drafting a speech.pdf
Page 1 of 1. Speech is supposed to be an oral presentation. But ,since you have speech as a discourse ,it is desirable. that we must learn the techniques of writing a speech.While presenting a speech on a stage, the speaker has a. lot of advantages .

A note on Kandori-Matsushima
Sep 16, 2004 - Social Science Center, London, ON, N6A 5C2, Tel: 519-661-2111 ext. ... equilibria, where private information is revealed every T-periods, as δ ...

A Note on Quasi-Presuppositions and Focus
Jan 31, 2011 - If John came late, the party was a disaster implies: ..... The above data seem to show that indeed the inference triggered by modifiers seems.

A Note on Quality Disclosure and Competition
I analyze how a change in the degree of horizontal product differentiation affects the incentives of duopolists to disclose quality information. If disclosure is costly, ...

A note on Minimal Unanimity and Ordinally Bayesian ...
E-mail address: [email protected]. 0165-4896/$ - see front matter © 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.mathsocsci.2006.12.

A Note on -Permutations
We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Mathematical Association of America is collaborating with JSTOR to digitize,

A note on for-phrases and derived scales
phrase set to restrict its domain; Bale (2008) takes a gradable adjective to determine ... by a New Initiatives Funding grant from Centre for Research on Language, Mind and Brain in ..... (30) Mia: “Buy me a hat that costs more than $5!” (31) Mia

A Note on Discrete- and Continuous-time Optimal ...
i.e. taking into account that next period's policy-maker will choose policy in the same way as today's policy-maker: gt+1 = G(kt+1), kt+2 = K(kt+1), ct+1 = f(kt+1,1) ...

A note on the upward and downward intruder ... - Springer Link
From the analytic solution of the segregation velocity we can analyze the transition from the upward to downward intruder's movement. The understanding of the ...

A note on extremality and completeness in financial ...
σ(X, Y ), and we apply it to the space L∞ with the topology σ(L∞,Lp) for p ≥ 1. .... Now, we want to apply Theorem 3 to the special case X = L∞(µ) equipped with ...

A Note on Discrete Convexity and Local Optimality
... (81)-45-339-3531. Fax: (81)-45-339-3574. ..... It is easy to check that a separable convex function satisfies the above condition and thus it is semistrictly quasi ...

briefing note on - Services
systems. In the light of these conclusions, a series of meetings in Africa, including the Foresight. Africa workshop in Entebbe, the AU meeting of Directors for Livestock Development in. Kigali 2004, the Congress ... OIE meeting of SADC Chief Veterin

A note on juncture homomorphisms.pdf - Steve Borgatti
A full network homomorphism f: N -+ N' is a regular network homomorphism if for each R E [w fi( a) f2( R) fi( b) * 3 c, d E P such that fi(u) = fi( c), fi( b) = fi( d), cRb and uRd for all a, b E P. In a network N the bundle of relations B,, from a t

A Note on Heterogeneity and Aggregation Using Data ...
Abstract: Using data from Indonesia, I show that both household- and ... (i) the Pareto being more appropriate than the exponential distribution for Yiv and Riv, ...

TANNAKA DUALITY AND STABLE INFINITY ...
Then by the semi-simplicity of representations of G, the pair (GG,H) is a flat descent structure in the sense of [10]. Consequently, there exists a combinatorial ...

A Duality Involving Borel Spaces
A Duality Involving Borel Spaces. Dharmanand Baboolal & Partha Pratim Ghosh. School of Mathematical Sciences. University of KwaZulu Natal. Westville Campus. Private Bag X54001. Durban 4041. South Africa. The purpose of the talk is to exhibit a dual e

A computational exploration of complementary ... - Semantic Scholar
Dec 8, 2015 - field (Petkov & Kruizinga, 1997), r controls the number of such periods inside .... were generated using FaceGen 3D face modeling software (an.

Duality, Geometry, and Support Vector Regression
Hard and soft ϵ-tubes are constructed by separating the convex or reduced convex hulls ... With duality analysis, the existence of a hard ϵ-tube depends on.

TANNAKA DUALITY AND STABLE INFINITY ...
The theory of Tannakian categories from Grothendieck-Saavedra [49], ..... the axiom of ZFC together with the axiom of Grothendieck universes (i.e., every Grothendieck universe is ... ∞-category S (resp. an edge) an object (resp. a morphism).

TANNAKA DUALITY AND STABLE INFINITY ...
given symmetric monoidal stable ∞-category) does not have a Tannakian category or the like as its full subcategory in ... One is to think of C⊗ as the category of sheaves on a geometric object (or the representation category .... A quasi- categor

Note on commented games - GitHub
The starting point for debate upon a classic joseki. 4. An other ... At the start of this game, White made grave errors. ..... 3: At move 28, Black cannot start a ko.

Issues of duality on noncommutative manifolds: The ...
Oct 27, 2004 - We study issues of duality and dual equivalence in noncommutative manifolds. In particular, the ... has no propagating degree of freedom but carries the ... gauge invariant master action or the Lagrange multiplier imposing ...

086JCR-INTEGRATIVE & COMPLEMENTARY MEDICINE.pdf ...
086JCR-INTEGRATIVE & COMPLEMENTARY MEDICINE.pdf. 086JCR-INTEGRATIVE & COMPLEMENTARY MEDICINE.pdf. Open. Extract. Open with. Sign In.