Online Kernel SVM Alekh Agarwal Microsoft Research

Alekh AgarwalMicrosoft Research

KSVM

Online Kernel SVM

Implemented the LASVM algorithm of Bordes et al. (2005) Similar to their paper, but without the bias term Variant of Dual Coordinate Ascent Given a dual variable, fully minimize with respect to it No bias term means this can be done in closed form

Alekh AgarwalMicrosoft Research

KSVM

LASVM

Consists of two steps Process step: Take a new example, update its dual variable

Alekh AgarwalMicrosoft Research

KSVM

LASVM

Consists of two steps Process step: Take a new example, update its dual variable Reprocess step: Take an existing support vector, update its dual variable Reprocess heuristics: Can pick a random support vector Easy to maintain suboptimality estimates of each SV, pick greedily Usually best to pick at least one greedy and one random

Alekh AgarwalMicrosoft Research

KSVM

LASVM

Consists of two steps Process step: Take a new example, update its dual variable Reprocess step: Take an existing support vector, update its dual variable Reprocess heuristics: Can pick a random support vector Easy to maintain suboptimality estimates of each SV, pick greedily Usually best to pick at least one greedy and one random

Online algorithm: Receive new example, perform process and then reprocess a fixed number of times

Alekh AgarwalMicrosoft Research

KSVM

Convergence properties

Guaranteed to converge to batch solution with enough reprocess steps and multiple passes over data Typically works quite well in just one pass and 1-2 reprocess steps Can use active learning for faster convergence

Alekh AgarwalMicrosoft Research

KSVM

Other implementation details

Maximum change in one update capped at 1 for stability Cache of kernel evaluations for efficiency, maxcache parameter set to 230

Alekh AgarwalMicrosoft Research

KSVM

Command line flags

Enabled by --ksvm option to VW Number of reprocess steps through --reprocess (default 1)

Alekh AgarwalMicrosoft Research

KSVM

Command line flags

Enabled by --ksvm option to VW Number of reprocess steps through --reprocess (default 1) Kernel type through --kernel. Supported types: Linear: specified as --kernel linear Polynomial: specified as --kernel poly. Additionally takes --degree d (default 2) RBF: specified as --kernel rbf. Additionally takes --bandwidth b (default 1.0)

Alekh AgarwalMicrosoft Research

KSVM

Command line flags

Enabled by --ksvm option to VW Number of reprocess steps through --reprocess (default 1) Kernel type through --kernel. Supported types: Linear: specified as --kernel linear Polynomial: specified as --kernel poly. Additionally takes --degree d (default 2) RBF: specified as --kernel rbf. Additionally takes --bandwidth b (default 1.0)

Do not forget to specify regularization through --l2

Alekh AgarwalMicrosoft Research

KSVM

Online Kernel SVM - GitHub

Usually best to pick at least one greedy and one random. Alekh AgarwalMicrosoft Research. KSVM ... Additionally takes --degree d (default 2). RBF: specified as ...

74KB Sizes 8 Downloads 250 Views

Recommend Documents

Linux Kernel Development - GitHub
Page 10 .... Android's “life of a patch” flowchart. Gerrit is only one tiny part in the middle. Replace that one part with email, and everything still works, and goes ...

Fast Algorithms for Linear and Kernel SVM+
Sep 1, 2016 - i-th entry being one, the subproblem for solving d can be formulated as min d f(β + dδ(i)) s.t. β + dδ(i) ≥ 0,. (10) which has an analytic solution ...

MeqTree Kernel Design Overview (PSS4) - GitHub
Nodes are implemented as C++ objects, subclassed from the abstract Meq::Node class2. ..... Glish array indices are 1-based, while C++ indices are 0-based. This ...... This notation conveniently hides a lot of messy processing: real Vells are ...

svm timeline.pdf
It is pointed out publicly that the hosting company for this domain has a. primary focus on anonymous hosting. • Metal Augmentor publishes its own analysis of ...

Project Triforce: AFL + QEMU + kernel = CVEs! (or) How to ... - GitHub
Page 10 ...... This opens up the fuzzing of Android devices and other things built on. ARM*. • You just ... Or, port our existing work to the Android QEMU emulator.

MULTI-SOURCE SVM FUSION FOR ENVIRONMENTAL ...
decision trees and support vector machines (SVM), showing that SVM ..... 352-365, 2005. ... Computer Science & Information Engineering, National Taiwan.

Stanford ONLINE - GitHub
Apr 1, 2014 - PLEASE NOTE: SOME ONLINE COURSES MAY DRAW ON MATERIAL FROM COURSES TAUGHT ON-CAMPUS BUT THEY ARE.

Linux Kernel - The Series
fs include init ipc kernel lib mm net samples scripts security sound tools usr virt ..... then the system can get severely damaged, files can be deleted or corrupted, ...

SVM-KNN: Discriminative Nearest Neighbor ...
Michael Maire. Jitendra Malik. Computer Science Division, EECS Department. Univ. of California, Berkeley, CA 94720 ... yields excellent results in practice. The basic idea is to find close neighbors to a query sample and ... rather by similarity to p

SVM. Support Vector Machines. Francisco Javier Cervigon ...
Page 4 of 8. SVM. Support Vector Machines. Francisco Javier Cervigon Ruckauer.pdf. SVM. Support Vector Machines. Francisco Javier Cervigon Ruckauer.pdf.

SVM Optimization for Lattice Kernels - Semantic Scholar
gorithms such as support vector machines (SVMs) [3, 8, 25] or other .... labels of a weighted transducer U results in a weighted au- tomaton A which is said to be ...

iOS Kernel Exploitation - Media.blackhat.com…
break out of sandbox. • disable codesigning and RWX protection for easier infection. • must be implemented in 100% ROP untethering exploits. • kernel exploit ...

SVM Optimization for Lattice Kernels - Semantic Scholar
[email protected]. ABSTRACT. This paper presents general techniques for speeding up large- scale SVM training when using sequence kernels. Our tech-.

SVM Selective Fusion (SELF) for Multi-Source ...
improved by inputting multi-sensory and GIS data since complex vegetation type identification benefits from .... imagery and GIS data (elevation, slope, distance to water body and land cover/land use map) to map 14 ... algorithms mainly focuses on si

Normalized Online Learning Tutorial - GitHub
Normalized Online Learning Tutorial. Paul Mineiro joint work with Stephane Ross & John Langford. December 9th, 2013. Paul Mineiro. Normalized Online ...