A Conscience On-line Learning Approach for Kernel-Based Clustering Chang-Dong Wang1 Jian-Huang Lai1 Jun-Yong Zhu2 1 School
of Information Science and Technology Sun Yat-sen University, P. R. China.
2 School
of Mathematics and Computational Science Sun Yat-sen University, P. R. China.
The 10th IEEE International Conference on Data Mining Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
1 / 24
Outline
1
Motivation Kernel Clustering and Problem Previous Work
2
Conscience On-Line Learning The Model Computation
3
Applications Digit Clustering Video Clustering
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
2 / 24
Kernel Clustering Kernel clustering is one of major methods for partitioning nonlinearly separable dataset. 1.5 1 0.5 0 −0.5 −1 −1.5
Wang et al. (Sun Yat-sen University)
−1
−0.5
0
0.5
1
1.5
COLL for Kernel-Based Clustering
2
2.5
ICDM 2010
3 / 24
Kernel Clustering Given X = {x1 , . . . , xn }, a kernel mapping φ, and the number of clusters c, find an assignment function ν ν : X → {1, . . . , c} to minimize the distortion error n X
k φ(xi ) − µνi k2 ,
i=1
where νi is short for ν(xi ), and µk =
1 |ν −1 (k )|
X
φ(xi ),
νi = arg min k φ(xi ) − µk k2
φ(xi )∈ν −1 (k )
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
k =1,...,c
ICDM 2010
4 / 24
Problem
k -means is used to solve min 1
Pn
i=1 k
φ(xi ) − µνi k2 via
Assigning the samples: νi ← arg min k φ(xi ) − µk k2 , ∀i = 1, . . . , N, k =1,...,c
2
Recomputing the prototypes: µk ←
1 −1 |ν (k )|
X φ(xi
φ(xi ), ∀k = 1, . . . , c.
)∈ν −1 (k )
However, ill-initialization would cause degenerate result.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
5 / 24
Problem
k -means is used to solve min 1
Pn
i=1 k
φ(xi ) − µνi k2 via
Assigning the samples: νi ← arg min k φ(xi ) − µk k2 , ∀i = 1, . . . , N, k =1,...,c
2
Recomputing the prototypes: µk ←
1 −1 |ν (k )|
X φ(xi
φ(xi ), ∀k = 1, . . . , c.
)∈ν −1 (k )
However, ill-initialization would cause degenerate result.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
5 / 24
Problem
k -means is used to solve min 1
Pn
i=1 k
φ(xi ) − µνi k2 via
Assigning the samples: νi ← arg min k φ(xi ) − µk k2 , ∀i = 1, . . . , N, k =1,...,c
2
Recomputing the prototypes: µk ←
1 −1 |ν (k )|
X φ(xi
φ(xi ), ∀k = 1, . . . , c.
)∈ν −1 (k )
However, ill-initialization would cause degenerate result.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
5 / 24
Problem
k -means is used to solve min 1
Pn
i=1 k
φ(xi ) − µνi k2 via
Assigning the samples: νi ← arg min k φ(xi ) − µk k2 , ∀i = 1, . . . , N, k =1,...,c
2
Recomputing the prototypes: µk ←
1 −1 |ν (k )|
X φ(xi
φ(xi ), ∀k = 1, . . . , c.
)∈ν −1 (k )
However, ill-initialization would cause degenerate result.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
5 / 24
Ill-initialization and Degenerate Clustering
μ1 μ1
μ2: degenerate
μ2: ill−initialized μ3
(a) Ill-initialization
μ3
(b) Degenerate results by k -means
Figure: Ill-initialization and degenerate result by k -means.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
6 / 24
Previous Work
1
Methods refining the initial prototypes. e.g., Bradley and Fayyad, ’98, Khan and Ahmad, ’04.
2
Lower bound approach. e.g., Zhang et al., ’06.
3
Evolutionary algorithm. e.g., Krishna and Murty, ’99, Abolhassani et al., ’04.
4
Global search strategy, i.e., global k -means. e.g., Likas et al., ’03, Tzortzis and Likas, ’09.
Computationally expensive.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
7 / 24
Previous Work
1
Methods refining the initial prototypes. e.g., Bradley and Fayyad, ’98, Khan and Ahmad, ’04.
2
Lower bound approach. e.g., Zhang et al., ’06.
3
Evolutionary algorithm. e.g., Krishna and Murty, ’99, Abolhassani et al., ’04.
4
Global search strategy, i.e., global k -means. e.g., Likas et al., ’03, Tzortzis and Likas, ’09.
Computationally expensive.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
7 / 24
Previous Work
1
Methods refining the initial prototypes. e.g., Bradley and Fayyad, ’98, Khan and Ahmad, ’04.
2
Lower bound approach. e.g., Zhang et al., ’06.
3
Evolutionary algorithm. e.g., Krishna and Murty, ’99, Abolhassani et al., ’04.
4
Global search strategy, i.e., global k -means. e.g., Likas et al., ’03, Tzortzis and Likas, ’09.
Computationally expensive.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
7 / 24
Previous Work
1
Methods refining the initial prototypes. e.g., Bradley and Fayyad, ’98, Khan and Ahmad, ’04.
2
Lower bound approach. e.g., Zhang et al., ’06.
3
Evolutionary algorithm. e.g., Krishna and Murty, ’99, Abolhassani et al., ’04.
4
Global search strategy, i.e., global k -means. e.g., Likas et al., ’03, Tzortzis and Likas, ’09.
Computationally expensive.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
7 / 24
Previous Work
1
Methods refining the initial prototypes. e.g., Bradley and Fayyad, ’98, Khan and Ahmad, ’04.
2
Lower bound approach. e.g., Zhang et al., ’06.
3
Evolutionary algorithm. e.g., Krishna and Murty, ’99, Abolhassani et al., ’04.
4
Global search strategy, i.e., global k -means. e.g., Likas et al., ’03, Tzortzis and Likas, ’09.
Computationally expensive.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
7 / 24
Conscience On-Line Learning (COLL) For each randomly taken data point φ(xi ): 1
Select the winner based on conscience mechanism: νi = arg min {fk k φ(xi ) − µk k2 }, k =1,...,c
2
Update the winner: µνi ← µνi + ηt φ(xi ) − µνi ,
3
Update the winning frequencies {fk , k = 1, . . . , c}: c X nl , ∀k = 1, . . . , c. nνi ← nνi + 1, fk = nk / l=1
So as to Bring all prototypes available into the solution quickly. Allow all prototypes to win the competition fairly. Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
8 / 24
Conscience On-Line Learning (COLL) For each randomly taken data point φ(xi ): 1
Select the winner based on conscience mechanism: νi = arg min {fk k φ(xi ) − µk k2 }, k =1,...,c
2
Update the winner: µνi ← µνi + ηt φ(xi ) − µνi ,
3
Update the winning frequencies {fk , k = 1, . . . , c}: c X nl , ∀k = 1, . . . , c. nνi ← nνi + 1, fk = nk / l=1
So as to Bring all prototypes available into the solution quickly. Allow all prototypes to win the competition fairly. Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
8 / 24
Conscience On-Line Learning (COLL) For each randomly taken data point φ(xi ): 1
Select the winner based on conscience mechanism: νi = arg min {fk k φ(xi ) − µk k2 }, k =1,...,c
2
Update the winner: µνi ← µνi + ηt φ(xi ) − µνi ,
3
Update the winning frequencies {fk , k = 1, . . . , c}: c X nl , ∀k = 1, . . . , c. nνi ← nνi + 1, fk = nk / l=1
So as to Bring all prototypes available into the solution quickly. Allow all prototypes to win the competition fairly. Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
8 / 24
Conscience On-Line Learning (COLL) For each randomly taken data point φ(xi ): 1
Select the winner based on conscience mechanism: νi = arg min {fk k φ(xi ) − µk k2 }, k =1,...,c
2
Update the winner: µνi ← µνi + ηt φ(xi ) − µνi ,
3
Update the winning frequencies {fk , k = 1, . . . , c}: c X nl , ∀k = 1, . . . , c. nνi ← nνi + 1, fk = nk / l=1
So as to Bring all prototypes available into the solution quickly. Allow all prototypes to win the competition fairly. Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
8 / 24
Conscience On-Line Learning (COLL) For each randomly taken data point φ(xi ): 1
Select the winner based on conscience mechanism: νi = arg min {fk k φ(xi ) − µk k2 }, k =1,...,c
2
Update the winner: µνi ← µνi + ηt φ(xi ) − µνi ,
3
Update the winning frequencies {fk , k = 1, . . . , c}: c X nl , ∀k = 1, . . . , c. nνi ← nνi + 1, fk = nk / l=1
So as to Bring all prototypes available into the solution quickly. Allow all prototypes to win the competition fairly. Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
8 / 24
Conscience On-Line Learning (COLL) For each randomly taken data point φ(xi ): 1
Select the winner based on conscience mechanism: νi = arg min {fk k φ(xi ) − µk k2 }, k =1,...,c
2
Update the winner: µνi ← µνi + ηt φ(xi ) − µνi ,
3
Update the winning frequencies {fk , k = 1, . . . , c}: c X nl , ∀k = 1, . . . , c. nνi ← nνi + 1, fk = nk / l=1
So as to Bring all prototypes available into the solution quickly. Allow all prototypes to win the competition fairly. Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
8 / 24
Conscience On-Line Learning
μ1 μ1
μ2: ill−initialized
μ2
μ3 μ
3
(a) Ill-initialization
(b) Satisfactory result by COLL
Figure: Ill-initialization and satisfactory result by COLL.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
9 / 24
Conscience On-Line Learning 1 0.9
Winning frequency
0.8 0.7 0.6 0.5 0.4 0.3 0.2 Prototype 1 Prototype 2 Prototype 3
0.1 0 0
20
40
60 80 100 120 140 Competition index over six iterations
160
180
200
Figure: The winning frequencies {f1 , f2 , f3 } of three prototypes as a function of competition index. Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
10 / 24
Kernel Matrix
The kernel mapping φ is often unknown or hard to obtain. The feature space Y is characterized by the kernel function κ(x, z) = hφ(x), φ(z)i and corresponding kernel matrix Ki,j = κ(xi , xj ). The distance between two points in the kernel space is computed via kernel trick. For instance, the distance between point φ(xi ) and prototype µk is computed as P P 2 j∈ν −1 (k ) Ki,j h,l∈ν −1 (k ) Kh,l 2 − k φ(xi ) − µk k = Ki,i + |ν −1 (k )|2 |ν −1 (k )| The COLL model must work with only the kernel matrix K .
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
11 / 24
Kernel Matrix
The kernel mapping φ is often unknown or hard to obtain. The feature space Y is characterized by the kernel function κ(x, z) = hφ(x), φ(z)i and corresponding kernel matrix Ki,j = κ(xi , xj ). The distance between two points in the kernel space is computed via kernel trick. For instance, the distance between point φ(xi ) and prototype µk is computed as P P 2 j∈ν −1 (k ) Ki,j h,l∈ν −1 (k ) Kh,l 2 − k φ(xi ) − µk k = Ki,i + |ν −1 (k )|2 |ν −1 (k )| The COLL model must work with only the kernel matrix K .
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
11 / 24
Kernel Matrix
The kernel mapping φ is often unknown or hard to obtain. The feature space Y is characterized by the kernel function κ(x, z) = hφ(x), φ(z)i and corresponding kernel matrix Ki,j = κ(xi , xj ). The distance between two points in the kernel space is computed via kernel trick. For instance, the distance between point φ(xi ) and prototype µk is computed as P P 2 j∈ν −1 (k ) Ki,j h,l∈ν −1 (k ) Kh,l 2 − k φ(xi ) − µk k = Ki,i + |ν −1 (k )|2 |ν −1 (k )| The COLL model must work with only the kernel matrix K .
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
11 / 24
Prototype Descriptor To this end, we develop an efficient framework for computation of the COLL model based on the prototype descriptor.
Definition (Prototype descriptor) A prototype descriptor is a matrix W φ ∈ Rc×(n+1) , such that the k -th row represents prototype µk by Wkφ,i = hµk , φ(xi )i, ∀i = 1, . . . , n, Wkφ,n+1 = hµk , µk i, i.e., Wφ =
hµ1 , φ(x1 )i . . . hµ2 , φ(x1 )i . . . .. .. . . hµc , φ(x1 )i . . .
Wang et al. (Sun Yat-sen University)
hµ1 , φ(xn )i hµ1 , µ1 i hµ2 , φ(xn )i hµ2 , µ2 i .. .. . . hµc , φ(xn )i hµc , µc i
COLL for Kernel-Based Clustering
.
ICDM 2010
12 / 24
Prototype Descriptor To this end, we develop an efficient framework for computation of the COLL model based on the prototype descriptor.
Definition (Prototype descriptor) A prototype descriptor is a matrix W φ ∈ Rc×(n+1) , such that the k -th row represents prototype µk by Wkφ,i = hµk , φ(xi )i, ∀i = 1, . . . , n, Wkφ,n+1 = hµk , µk i, i.e., Wφ =
hµ1 , φ(x1 )i . . . hµ2 , φ(x1 )i . . . .. .. . . hµc , φ(x1 )i . . .
Wang et al. (Sun Yat-sen University)
hµ1 , φ(xn )i hµ1 , µ1 i hµ2 , φ(xn )i hµ2 , µ2 i .. .. . . hµc , φ(xn )i hµc , µc i
COLL for Kernel-Based Clustering
.
ICDM 2010
12 / 24
Initialization
Theorem (Initialization) The random initialization of prototype descriptor can be realized by φ W:,1:n = AK ,
φ W:,n+1 = diag(AKA> )
c×n where the matrix A = [Ak ,i ]c×n ∈ R+ reflects the initial assignment ν
( Ak ,i =
Wang et al. (Sun Yat-sen University)
1 |ν −1 (k )|
if i ∈ ν −1 (k )
0
otherwise.
COLL for Kernel-Based Clustering
ICDM 2010
13 / 24
Winner Selection & Updating Theorem (Conscience based winner selection) The winning prototype can be selected based on the conscience mechanism as νi = arg min {fk · (Ki,i + Wkφ,n+1 − 2Wkφ,i )}. k =1,...,c
Theorem (On-line winner updating) The winner can be updated in the way of φ (1 − ηt )Wνi ,j + ηt Ki,j Wνφi ,j ← (1 − ηt )2 Wνφi ,j + ηt2 Ki,i + 2(1 − ηt )ηt Wνφi ,i Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
j = 1, . . . , n, j = n + 1.
ICDM 2010
14 / 24
Winner Selection & Updating Theorem (Conscience based winner selection) The winning prototype can be selected based on the conscience mechanism as νi = arg min {fk · (Ki,i + Wkφ,n+1 − 2Wkφ,i )}. k =1,...,c
Theorem (On-line winner updating) The winner can be updated in the way of φ (1 − ηt )Wνi ,j + ηt Ki,j Wνφi ,j ← (1 − ηt )2 Wνφi ,j + ηt2 Ki,i + 2(1 − ηt )ηt Wνφi ,i Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
j = 1, . . . , n, j = n + 1.
ICDM 2010
14 / 24
Iteration Stopping Criterion
Theorem (Iteration stopping criterion) If eφ < or t > tmax , stop the iteration. φ
e
! 2 mk X mk c X Kπk ,πk X 1 φ 2 h l = 1− W + η t k ,n+1 (1 − ηt )mk (1 − ηt )h+l k =1 k =1 h=1 l=1 φ m c k Wk ,πk X X 1 l . 1− +2ηt (1 − ηt )mk (1 − ηt )l c X
k =1
l=1
Here, the array π k stores the indices of mk ordered points assigned to the k-th prototype in one iteration.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
15 / 24
Demonstration of COLL Winner Updating
φ (ο )
Nonlinear separator x ο
ο
ο W1,: ο
x
x Wˆv ,:
x
ο
x
ο
Wv ,:
φ (ο )
x′
φ
φ (ο )
W1,:φ
φ (ο ) φ (ο )
x
φ (ο ) Linear separator
φ (x) φ (x) φ (x′)
φ (x) Wˆ φ v ,:
φ (x) Wvφ,: φ (x) φ (x)
Figure: The mapping φ embeds the data into a feature space where the nonlinear pattern becomes linear. Then COLL is performed in this feature space. The linear separator in feature space is actually nonlinear in input space, so is the update of the winner.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
16 / 24
Digit Clustering Table: Summary of four digit datasets.
Dataset Pendigits Mfeat USPS MNIST
n 10992 2000 11000 5000
c 10 10 10 10
d 16 649 256 784
Balanced × √ √ √
σ 60.53 809.34 1286.70 2018.30
Figure: Some samples of MNIST.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
17 / 24
Comparing Convergence Rate 800
500
Kernel k−means The proposed COLL
700 600
φ
Log(eφ)
500 Log(e )
Kernel k−means The proposed COLL
400
400 300 200
300
200
100
100 0 0
10
20
30
40
50 60 Iteration step
70
80
90
0 0
100
10
20
30
(a) Pendigits 800
70
80
90
100
500
500 Log(eφ)
600
400
400
300
300
200
200
100
Kernel k−means The proposed COLL
700
600
φ
Log(e )
50 60 Iteration step
800
Kernel k−means The proposed COLL
700
0 0
40
(b) Mfeat
100 10
20
30
40
50 60 Iteration step
70
80
90
100
0 0
10
20
(c) USPS
30
40
50 60 Iteration step
70
80
90
100
(d) MNIST
Figure: Comparing the convergence rate in terms of log(eφ ) as a function of iteration step. Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
18 / 24
Comparing Distortion Error Kernel k−means Global kernel k−means The proposed COLL
7200
Distortion error
Distortion error
1700
7000 6900 6800
1600 1500 1400
6700
1300
6600 6500 2
Kernel k−means Global kernel k−means The proposed COLL
1800
7100
4
6
8
10 12 Number of clusters
14
16
18
1200 2
20
4
6
(a) Pendigits Kernel k−means Global kernel k−means The proposed COLL
8200 8100
10 12 Number of clusters
14
16
18
20
Kernel k−means Global kernel k−means The proposed COLL
3150 3100
8000
3050 Distortion error
Distortion error
8
(b) Mfeat
7900 7800 7700
3000 2950 2900
7600 2850 7500 2800 7400 2
4
6
8
10 12 Number of clusters
14
16
18
20
2750 2
4
6
(c) USPS
8
10 12 Number of clusters
14
16
18
20
(d) MNIST
Figure: Comparing the distortion error as a function of cluster number.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
19 / 24
Comparing using Internal & External Measurement Table: Average distortion error (DE).
Dataset Kk -means Gkk -means Pendigits 6704.0 6664.9 Mfeat 1405.5 1363.3 USPS 8004.7 7782.0 MNIST 3034.3 2894.1
COLL 6619.3 1324.2 7561.0 2754.9
Table: Average normalized mutual information (NMI).
Dataset Kk -means Gkk -means Pendigits 0.715 0.736 Mfeat 0.533 0.542 USPS 0.354 0.369 MNIST 0.441 0.472 Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
COLL 0.753 0.604 0.461 0.520 ICDM 2010
20 / 24
Video Clustering Cluster 1: 0001-0061
Cluster 2: 0062-0163
Cluster 3: 0164-0289
Cluster 4: 0290-0400
Cluster 5: 0401-0504
Cluster 6:0505-699
Cluster 7: 0700-0827
Cluster 8: 0828-1022
Cluster 9: 1023-1083
Cluster 10: 1084-1238
Cluster 11: 1239-1778
Cluster 12: 1779-1983
Cluster 13: 1984-2119
Cluster 14: 2120-2243
Cluster 15: 2244-2400
Cluster 16: 2401-2492
Figure: Clustering frames of ANNI002 into 16 scenes using COLL. Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
21 / 24
Comparison Results
Table: The means of NMI and computational time in seconds on the 11 video sequences. Video (#frames) ANNI001 (914) ANNI002 (2492) ANNI003 (4265) ANNI004 (3897) ANNI005 (11361) ANNI006 (16588) ANNI007 (1588) ANNI008 (2773) ANNI009 (12304) ANNI010 (30363) ANNI011 (1987)
Wang et al. (Sun Yat-sen University)
kk -means NMI Time 0.781 72.2 0.705 94.7 0.712 102.2 0.731 98.3 0.645 152.2 0.622 193.0 0.727 81.1 0.749 95.9 0.727 167.0 0.661 257.2 0.738 85.4
gkk -means NMI Time 0.801 94.0 0.721 126.4 0.739 139.2 0.750 121.6 0.656 173.3 0.638 255.5 0.740 136.7 0.771 119.0 0.763 184.4 0.709 426.4 0.749 142.7
COLL for Kernel-Based Clustering
Proposed COLL NMI Time 0.851 70.4 0.741 89.0 0.762 99.5 0.759 93.6 0.680 141.2 0.642 182.3 0.770 79.1 0.794 81.5 0.781 160.4 0.734 249.0 0.785 83.7
ICDM 2010
22 / 24
Summary
1
In this paper, we have shown that, existing method for solving kernel clustering may suffer the ill-initialization problem.
2
We present a conscience on-line learning (COLL) approach for addressing the ill-initialization.
3
In the future, we may explore the on-line learning framework with other mechanisms, such as rival penalization, to realize automatic cluster number selection in the kernel clustering.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
23 / 24
Summary
1
In this paper, we have shown that, existing method for solving kernel clustering may suffer the ill-initialization problem.
2
We present a conscience on-line learning (COLL) approach for addressing the ill-initialization.
3
In the future, we may explore the on-line learning framework with other mechanisms, such as rival penalization, to realize automatic cluster number selection in the kernel clustering.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
23 / 24
Summary
1
In this paper, we have shown that, existing method for solving kernel clustering may suffer the ill-initialization problem.
2
We present a conscience on-line learning (COLL) approach for addressing the ill-initialization.
3
In the future, we may explore the on-line learning framework with other mechanisms, such as rival penalization, to realize automatic cluster number selection in the kernel clustering.
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
23 / 24
Thank you very much! Q&A
Wang et al. (Sun Yat-sen University)
COLL for Kernel-Based Clustering
ICDM 2010
24 / 24
