EN.550.665: Convex Optimization Instructor: Daniel P. Robinson Homework Assignment #4 Any starred exercises require the use of Matlab. Exercise 4.1: Let f : Rn → R be a proper convex function. Prove that a linear minorant with slope s ∈ Rn exists if and only if f ∗ (s) < ∞, where f ∗ is the conjugate function of f . Exercise 4.2: Let f : Rn → R be a proper convex function. Prove that ∂f (x) ⊆ dom(f ∗ ), where f ∗ is the conjugate function of f . Exercise 4.3∗ : Implement the following three algorithms: 1. The SubGradient Descent (SGD) method (Algorithm 1 in Lecture04) with diminishing step size. 2. The Averaged SubGradient Descent (ASGD) method (Algorithm 2 in Lecture04). 3. The Simple Dual Averaging (SDA) Method (Algorithm 3 in Lecture04). To test your implementations, you should first download the file 665 logistic.zip from the course website. Once the file is downloaded, you need to unzip the file, which will produce two directories and two files. You should start with the file ReadMe and then proceed to understand the file demo.m. Specifically, you should solve the following optimization problem: minimize f (θ) := L(θ) + λkθk1 n θ∈R

(1)

for some choice of weighting parameter λ > 0. The function L and its gradient ∇L can be evaluated as shown in the file demo.m. Using this information and the fact that we have already shown how to compute the subdifferential for λkθk1 , allows us to compute the subdifferential for f . To successfully complete this homework exercise, you should do the following: (i) Solve problem (1) using the three algorithms SGD, ASGD, and SDA for the three different data sets breast cancer.mat, ijcnn1.mat, and rcv1.mat; these data sets will be formed once you unzip the file 665 logistic.zip described above. Try this for various values of λ > 0 and report your experience. (ii) What happens to the solution to problem (1) as the weighting parameter λ → ∞? Show that your claim is true via numerical examples. (iii) Choose a value for λ > 0 for which the solution to problem breast cancer.mat has approximately 50% of its entries equal to zero. What is the value of λ > 0 that makes this the case? For that value of λ, perform the following tests for problem breast cancer.mat. (a) Examine how sensitive SGD is to the choice of the parameter c > 0 used in the diminishing step length choice αk = c/k. Exhibit this via numerical experiments. (b) Examine how sensitive ASGD √ is to the choice of the parameter c > 0 used in the diminishing step length choice αk = c/ k. Exhibit this via numerical experiments. (c) Compare the performance of SDA to both SGD and ASGD.

1

EN.550.665: Convex Optimization - MOBILPASAR.COM

You should start with the file ReadMe and then proceed to understand the file demo.m. Specifically, you should solve the following optimization problem: minimize θ∈Rn f(θ) := L(θ) + λθ1. (1) for some choice of weighting parameter λ > 0. The function L and its gradient ∇L can be evaluated as shown in the file demo.m.

118KB Sizes 0 Downloads 237 Views

Recommend Documents

Convex Optimization
Mar 15, 1999 - 5.1 Unconstrained minimization and extensions . ..... It is (as the name implies) a convex cone. Example. ..... and lies in the domain of f (i.e., c. T.

Guaranteed Non-convex Optimization: Submodular ...
Submodular continuous functions naturally find applications in various real-world settings, including influence and revenue maximization with continuous assign- ments, sensor energy management, multi-resolution data summarization, facility location,

boyd convex optimization pdf
Download. Connect more apps... Try one of the apps below to open or edit this item. boyd convex optimization pdf. boyd convex optimization pdf. Open. Extract.

An Efficient Convex Optimization Approach to 3D ...
evolution, which allows a large time step-size to accelerate the speed of .... (8) is dual to the convex relaxation problem (7), by means of similar analytical .... the high anisotropy of the sampled 3D prostate MRI data does interfere achieving.

Distributed Dual Averaging for Convex Optimization ...
The issue is not however essential and we prove that a simple correction term ...... Illustration of the effect of fixed edge delays on distributed dual averaging.

A Study on Convex Optimization Approaches to Image ...
ful applications include image denoising [22,8,30], image decomposition [2,17] ..... + ui/c)}) . (30). Interestingly, our experiments show that just one single step of ... experiments are computed by a Ubuntu desktop with AMD Athalon 64 X2 5600.

LNCS 7510 - A Fast Convex Optimization Approach to ...
scar tissue solely from 3D cardiac delayed-enhancement MR images (DE-. MRI). ..... Foundation for Innovation (CFI) grant #20994, Canadian Institutes of Health.