JOURNAL OF INDUSTRIAL AND MANAGEMENT OPTIMIZATION Volume 1, Number 4, November 2005

Website: http://AIMsciences.org pp. 465–476

OPTIMAL PARAMETER SELECTION IN SUPPORT VECTOR MACHINES K. Schittkowski Department of Computer Science University of Bayreuth, 95440 Bayreuth, Germany

(Communicated by Kok Lay Teo) Abstract. The purpose of the paper is to apply a nonlinear programming algorithm for computing kernel and related parameters of a support vector machine (SVM) by a two-level approach. Available training data are split into two groups, one set for formulating a quadratic SVM with L2 -soft margin and another one for minimizing the generalization error, where the optimal SVM variables are inserted. Subsequently, the total generalization error is evaluated for a separate set of test data. Derivatives of functions by which the optimization problem is defined, are evaluated in an analytical way, where an existing Cholesky decomposition needed for solving the quadratic SVM, is exploited. The approach is implemented and tested on a couple of standard data sets with up to 4,800 patterns. The results show a significant reduction of the generalization error, an increase of the margin, and a reduction of the number of support vectors in all cases where the data sets are sufficiently large. By a second set of test runs, kernel parameters are assigned to individual features. Redundant attributes are identified and suitable relative weighting factors are computed.

1. Introduction. During the last ten years, support vector machines (SVM) became an important alternative to neural networks for machine learning. Meanwhile there is a large number of applications, see e.g. http://www.kernel-machines.org/ Especially from the viewpoint of mathematical programming, SVMs are extremely interesting and touch a large variety of important topics, for example duality theory, convex optimization, large scale linear and quadratic optimization, semi-definite optimization, least squares optimization, L1 -optimization, and interior point methods. In a series of papers, Mangasarian and co-workers investigate support vector machines from the viewpoint of mathematical programming, see Bradley et al. [3], Fung and Mangasarian [9], Mangasarian and Musicant [11], and Mangasarian [12, 13]. As shown below in more detail, a support vector machine is identified by a kernel function to determine a certain similarity measure, a method compute a classification error, and an approach to handle outliers. Although being a well established 2000 Mathematics Subject Classification. 68T10, 90C90, 65K05. Key words and phrases. machine learning; support vector machine; SVM; binary classification; Gaussian kernel; kernel parameter; sequential quadratic programming; SQP; nonlinear programming. Sponsored by EU Network of Excellence PASCAL (Pattern Analysis, Statistical Modelling and Computational Learning) under contract no IST 2002-506778.

465

OPTIMAL PARAMETER SELECTION IN SUPPORT ...

Website: http://AIMsciences.org ... K. Schittkowski. Department of Computer Science ... algorithm for computing kernel and related parameters of a support vector.

52KB Sizes 3 Downloads 290 Views

Recommend Documents

OPTIMAL PARAMETER SELECTION IN SUPPORT ...
Abstract. The purpose of the paper is to apply a nonlinear programming ... convex optimization, large scale linear and quadratic optimization, semi-definite op-.

Parameter selection for support vector machines
Nov 10, 2003 - learning method for both regression and classification ... problem that for support vector learning is: %'& (0)21436587@ 9 ABC. D. 5. C бг. #.

3 Our Parameter Selection Algorithms
Email: {rahulv,riskin}@ee.washington.edu, [email protected] ... In this paper, we use the x264 encoder, an open source implementation of the H.264 ...

Parameter Extraction and Support-Loss in MEMS Resonators - Comsol
to the position x. Even if the resonator is not moving, a stray capacitance Cw=Aactε0/g across the gap is present across the actuation gap. Equation (7) was derived for a MEMS resonator with a ... Solutions of equation (9) have a time dependence giv

Optimal Interventions in Markets with Adverse Selection
Mar 8, 2010 - ‡Leonard Stern School of Business, Kaufman Management Center, 44 West .... they are significantly more complicated to set up and administer ...

Optimal Interventions in Markets with Adverse Selection
part in a government program carries a stigma, and outside options are .... show that bailouts can be designed so as not to distort ex-ante lending incentives.

Multi-Constrained Optimal Path Selection
Marwan Krunz. Department of Electrical & Computer Engineering ... HE current Internet has been designed to support connectiv- ity based ...... 365–374, 1999.

A Bayesian approach to optimal monetary policy with parameter and ...
This paper undertakes a Bayesian analysis of optimal monetary policy for the United Kingdom. ... to participants in the JEDC conference and the Norges Bank conference, ... uncertainty that confront monetary policy in a systematic way. ...... 2 call f

Optimal Mobile Actuation Policy for Parameter ...
Optimal Mobile. Actuation Policy for. Parameter Estimation of. Distributed Parameter. Systems. ∗. Christophe Tricaud and YangQuan Chen†. 1 Introduction.

Time-optimal Active Portfolio Selection
Time-optimal Active Portfolio Selection. Thomas Balzer [email protected]. November 27, 2001. Abstract. In a complete financial market model where the prices of the assets are modeled as Ito diffusion processes, we consider portfolio problems wh

working fluid selection through parameter estimation
WORKING FLUID SELECTION THROUGH PARAMETER ESTIMATION. Laura A. Schaefer. Mechanical Engineering Department, University of Pittsburgh [email protected]. Samuel V. Shelton. Woodruff School of ...... [23] Wilding, W., Giles, N., and Wilson, L., 1996, “P

Optimal Chirp Parameter Determination for Soliton ...
The project is about determining the optimal chirp parameter in the soliton pulse pression, which serve ..... To achieve a more accurate solution, we apply Runge-Kutta Method of order 5. Eventually, the solution to the ODE ...... [5] Michael Frosz an

Optimal ࣇ-SVM Parameter Estimation using Multi ...
May 2, 2010 - of using the Pareto optimality is of course the flexibility to choose any ... The authors are with the Dept. of Electrical and Computer Engineering.

Optimal ࣇ-SVM Parameter Estimation using Multi ...
May 2, 2010 - quadratic programming to solve for the support vectors, but regularization .... the inherent parallelism in the algorithm [5]. The NSGA-II algorithm ...

A Bayesian approach to optimal monetary policy with parameter and ...
more useful communication tools. .... instance, we compare micro-founded and non micro-founded models, RE vs. non-RE models, .... comparison with the others. ...... Kimball, M S (1995), 'The quantitative analytics of the basic neomonetarist ...

Model Selection for Support Vector Machines
New functionals for parameter (model) selection of Support Vector Ma- chines are introduced ... tionals, one can both predict the best choice of parameters of the model and the relative quality of ..... Computer Science, Vol. 1327. [6] V. Vapnik.

On efficient k-optimal-location-selection query ...
a College of Computer Science, Zhejiang University, Hangzhou, China ... (kOLS) query returns top-k optimal locations in DB that are located outside R. Note that ...

Optimal Orbit Selection and Design for Airborne Relay ...
A software with interactive GUI is designed and ... planning software and discuss results. ... on Bidirectional Analytic Ray Tracing and Radiative Transfer).

disutility, optimal retirement, and portfolio selection
DISUTILITY, OPTIMAL RETIREMENT, AND PORTFOLIO SELECTION. KYOUNG JIN CHOI. School of Computational Sciences, Korea Institute for Advanced Study,. Seoul, Korea. GYOOCHEOL SHIM. Graduate School of Management, Korea Advanced Institute of Science and Tech

An Efficient Genetic Algorithm Based Optimal Route Selection ... - IJRIT
Wireless sensor Network (WSN) is getting popular especially for applications where installation of the network infrastructure is not possible, such as.

An Efficient Genetic Algorithm Based Optimal Route Selection ... - IJRIT
infrastructure, but imposes some drawbacks and limitations (mainly on .... Networks”, http://www.monarch.cs.rice.edu/monarch-papers/dsr-chapter00.pdf.

Optimal Training Data Selection for Rule-based Data ...
affair employing domain experts, and hence only small .... free rules. A diverse set of difficult textual records are given to set of people making sure that each record is given to a ..... writes when presented with the 100 chosen patterns. A.

Speculative Markov Blanket Discovery for Optimal Feature Selection
the remaining attributes in the domain. Koller and Sahami. [4] first showed that the Markov blanket of a given target at- tribute is the theoretically optimal set of ...

On efficient k-optimal-location-selection query ... - Semantic Scholar
Dec 3, 2014 - c School of Information Systems, Singapore Management University, ..... It is worth noting that, all the above works are different from ours in that (i) .... develop DBSimJoin, a physical similarity join database operator for ...