2007 International Conference on Artificial Intelligence and Pattern Recognition (AIPR-07)
A New Point Pattern Matching Method for Palmprint Jiyi Li, Guangshun Shi, Qiu Feng, Hongwu Wan Institute of Machine Intelligence, College of Information Technical Science Nankai University, Tianjin, China, 300071 Email:
[email protected];
[email protected]
Abstract— Point pattern matching is an important topic in the field of computer vision and pattern recognition. Currently most palmprints automatic recognition methods are only suitable for the online palmprints or full palmprints. In the practical applications, it have to use high-quality matching method to deal with the full and part palmprints, online and off-line palmprints, different quality of images, and massive samples. To achieve the accurate matching and meet the requirements, this paper proposes a multi-phases point pattern matching method based on both of the local structure and global feature. This method designs a reasonable score mechanism to distinguish the reliability of the candidates. The test on the speed and accuracy shows that the performance of this robust method which can meet the requirements of the practical applications is satisfactory. Index Terms— Palmprint, Point Pattern Matching, MultiPhases, Local Structure, Global Feature.
I. I NTRODUCTION As an important biometric characteristic, the palmprint is featured by its large amount of information, stable structure and low-cost acquisition. With its promising prospects, the technology of palmprint recognition has been widely used in many practical applications, gaining more and more attention from the research groups of biometric recognition. With different applications, the capacity requirement of palmprint recognition system also varies. In the civilian fields, it generally only needs to make a verification(1:1) on the full palmprint; while in the criminal investigation field it has to make a identification(1:N,N:N) on the massive palmprint data(in millions grade), including full palmprints collected by various approachs and part palmprints collected from the crime scenes. According to the mode of sample acquisition, the palmprint automatic recognition could be divided into categories of online and offline. The online mode deals with the real-time acquisition and real-time processing on the full palmprints with a stable and high quality, which usually can not be provided in offline mode. Thus the feature extraction and matching on the later mode are much more difficult. In pratcial applications, a good approach of palmprint matching should meet several requirements as follows: a). Be independent on the center of the palmprint image. b). Be unaffected by the geometric transformation of translation and rotation. c). Be tolerant with the nonlinear distortion of palmprint in a certain extent.
d). Be tolerant with existence of false features and loss of true features in a certain extent. e). Be able to provide a candidate queue sorted by matching similarity for the references of decision-making. f). Be efficient, accurate and reliable. Point pattern matching plays an important part in computer vision and pattern recognition. In recent years, many methods have been proposed, but a complete theoretical system has not been established. There are many point pattern matching algorithms based on images, some of which has been applied in the fingerprint recognition. Compared with the fingerprints, the palmprints is far more complex and the amount of data is much larger. a). Methods Based on Mathematical Transformation: Ranade etc.[1] proposes a iterative algorithm, which uses a pair of corresponding points to define a relative transformation between the two point sets. It computes the matching reliability with the matching extent of the other points, and finds out a pair of points which make the two sets to have the largest number of points pairs with a stable relationship between the transformation. The method doesn’t use the structural information of the points, and the iterative calculation speed is very slow. Stockman etc.[2] proposes a method based on the Hough transform, which converts the point pattern matching into the peak detection in the Hough space for the transform parameters. This approach has the disadvantage that the data accumulated in the Hough space is not enough to ensure a reliable matching when the points are few. This algorithm has a high time complexity. b). Methods Based on the Information of the Structure: Hrechak etc.[3] proposes a matcing method based on the structural information. This method extracts eight types of minutiaes first, and construct the eigenvectors. Then it selects a representative form of the eigenvector, according to the appearance times of in the image. This method uses the experience and expert knowledge in a certain degree. Since there are few varieties in the form of the eigenvector, it applies only to a small number of samples. The noises and distortions in the image make it difficult for practical applications. Wahab etc.[4], [5], [6] proposes a multi-phases fingerprints matching methods based on the local structure and global feature. It makes the use of topological structure, and could deal with the transformation, rotation and distortion. The
302
2007 International Conference on Artificial Intelligence and Pattern Recognition (AIPR-07)
multi-phases model reduces the use of the time. But the local structure eigenvector is too simple, and the algorithm is easily interfered by the local errors. Zhang etc.[7] proposes a local structure matching method based on the center point, which executes the location first and then the matching. It depends on the accuracy of its reference point and is not universally applicable. Kovacs-Vajna etc.[8] proposes a triangular matching algorithm, which constructs a vector triangle for each three feature points. It searches the congruent or almost congruent triangles between the two point sets, and statistic the similarity scores as matching criteria. c). Methods by Artificial Intelligence and Machine Learning: Starink etc.[9] describes the point pattern matching from the perspective of the energy minimization. The approach uses the simulated annealing algorithm and it needs large amount of computation. Ansari etc.[10] proposes a method based on the genetic algorithm, the main disadvantage is the need to repeat coordinate transformations on the point sets for the computation of the matching extent, the amount of the computation is large and the speed of the algorithm is low. Those methods, each having its own advantages and disadvantages, could only solve some aspects of the problems. None of those can singly meet all of the requirements as above. This paper proposes a multi-phases point pattern matching method. Combining the local structure with the global feature, this method uses a composite mode score mechanism with multiple conditions. The method could handle the online and off-line palmprints, the full and part palmprints, different quality of images, and massive samples. All of the above problems are solved in a good performance. II. PALMPRINT M ATCHING M ETHOD A. Feature Selection Several types of feature information could be used for palmprint recognition, including geometric characteristics, ridges, minutiaes and so on. Compared to other types of features, minutiaes have the following characteristics. It has invariance and uniqueness. It is able to be extracted on the high-resolution images and obtain more information for the identification from the injured or part palmprint, and the matching method based on it is not dependent on the location. These characteristics meet the aforementioned application requirements which are complex and rigorous. For these reasons, this paper identifies the palmprints by the minutiaes and proposes a palmprint matching method based on point pattern. The types of the minutiaes that are commonly used include termination, bifurcation, center point, triangular point, etc. Given a practical application, different methods select some types of the features used for the identification according to the requirements. The feature definition in this paper is now the more commonly used Minutiae Model which is proposed by FBI. It uses ridge terminations and ridge bifurcations to identify palmprints. The direction of the termination is the flow
direction along the ridge and the direction of the bifurcation is the angle bisector direction of the angle which is the smallest in the three angle. B. Point Pattern Matching In the point pattern matching on the palm print recognition, the number of minutiaes in the palmprint is large. The size of the minutiaes set, get by the feature extraction and postprocessing, with possible false minutiaes in it, ranges from several hundred to several thousand. To meet the practical application, the matching method has good performance on both of speed and accuracy. Considering the advantages and disadvantages of the existing method, this paper proposes a multi-phases matching method. The method combines the local structure with the global feature, so that it would consider both of the local similarity and global similarity between the two minutiaes sets. At first it is the initial matching based on the local structure of the minutiaes, and then it is the second matching based on the global feature. The method describes the final results with a similarity score, output the top n (n = 50 in this paper) candidate samples sorted by the score, and confirm the result by manual review. 1) The Construction of the Local Structure: For each minutiae p (x, y, θ) (x is the abscissa of the minutiae in the image, y is the ordinate, and θ is the direction) in the sample, set it as the origin, and its direction as the axis direction, building a local polar coordinate. The palmprint image center has nothing to do with the local structure. The use of the polar coordinate could describe the nonlinear distortion better, and it is easier to handle the translation and rotation of the image. Searching other minutiaes within the radius of R, we can find out the n-nearest neighborhood minutiaes according to the polar radius of the minutiaes to the origin (the value n is different to the full and part palmprint). then we construct a local eigenvector, as showed in figure 1.
Fig. 1.
The Structure of the Neighborhood Minutiaes
Use the following formula to transform the minutiaes into polar coordinates. p (xi − x)2+ (yi − y)2 ri ϕi = tan−1 xyii −y −x αi θi − θ After the transformation, ri is the polar radius, φi is the polar angle, αi is the difference of the direction. To each minutiae in the sample, construct a 3 (n + 1)-dimensional eigenvector (x, y, θ, r1 , φ1 , α1 , r2 , φ2 , α2 , · · ·, rn , φn , αn ). In the eigenvector, the neighborhood minutiaes are sorted according to the order of distances from near to far.
303
2007 International Conference on Artificial Intelligence and Pattern Recognition (AIPR-07)
2) The Initial Matching Based on the Local Structure: The initial matching evaluates the association of the two palmprints on the local structure. Let Ai (t) be the tth neighborhood information of the ith local structure eigenvector in the compared palmprint. Let Bj (s) be the sth neighborhood information of the jth local structure eigenvector in the template palmprint. The matching score is the number of matching minutiaes in its neighborhood. The following formula shows the conditions which is used to judge whether two neighborhood minutiaes match each other. ∆Dif Distance < δ1 ∆Dif angle < ε1 true if Ai (t)matchBj (s) = ∆Relative dir < η1 f alse otherwise Where ∆Dif Distance is the difference of the polar radius, ∆Dif angle is the difference of the polar angle, and ∆Relative dir is the difference of relative direction, which is the direction difference between the neighborhood minutiae and the center minutiae. When all of them are smaller than the predefined thresholds, the algorithm identify the matching of the two neighborhood minutiaes. Normally, the smaller the differences are, the higher the matching extent of the two minutiaes in the neighborhood is. We use the Difference Total (∆Distance + µ1 ∆Relative angle + µ2 ∆Dif angle) to measure their matching extent, where µ1 and µ2 are the corresponding weight according to the three different levels of contribution to the matching. Due to the existence of the false minutiae, the minutiae with the highest score is not certain to be the truely matching minutiae. If we simply use the success or failure to describe each pair of the matching of the local eigenvector, the accuracy of the algorithm could not be guaranteed and the antifalse minutiae capability is not good enough. In addition, in the pattern of part-to-full, the minutiae number of the full palmprint is far greater than the minutiae number of the part palmprint, and it is quite easy to identify a number of local structure eigenvectors which have a similar matching extent from the full palmprint. This paper presents a description of the matching minutiae queue of the local structure. To each minutiae in the compared palmprint, the method provide m candidates of the matching minutiaes by the similar extent, for further matching computation in the successor steps. 3) The Confirmation of the Global Datum Mark: It is necessary to confirm the global datum mark before carrying out the second matching based on the Global Feature. During the confirmation of the datum mark, according to the results of the initial matching, we construct the newly compared minutiae set and template minutiae set with all of the candidate pairs of matching. Then we establishment global feature eigenvectors from the minutiaes in the set to filtrate the candidates pairs of matching with the results of local structure matching and the geometric relationships between two minutiae sets.
By calculating all of the possible minutiae pairs of matching, we get a matching matrix LM ∗N . The element lij is the matching minutiae number on the new minutiae sets if regarding the ith elements of the compared minutiae set and the jth elements of the template minutiae set as the temp datum mark. To compute the matching minutiae number, ∀ hp, qi ∈ C×D (C is the new compared minutiae set, D is the new template minutiae set), we traverse all of the candidates pair hu, vi ∈ C × D. Similar with the previous calculation method, if the ∆Dif Distance (the polar radius difference of u to p and v to q), ∆Dif angle, and ∆Relative dir are smaller than the predefined threshold, we denote u ”matching” v, and increase the matching minutiae number. We select the corresponding minutiae pair of the element which has the max value in the LM ∗N . The datum mark for the coordinates calibration is the foundation of the second matching based on the global feature. The result of the initial matching is used to guide the confirmation of the datum mark. On one hand, the information of the local structure could be integrated into the process of the global matching; on the other hand, this is a relatively quick method of confirming the global datum mark. 4) The Second Matching Based on the Global Feature: The algorithm carries out the second matching on the old compared and template minutiae set. Before the matching step it makes the use of the datum mark for the global coordinate calibration to all the minutiaes of the palmprints. At the coordinates calibration, we use the global datum mark hp, qi ∈ A × B to compute three coordinate transformation parameters, including the translation factors ∆x and ∆y, the rotation factor α, which could transform the vector from p to q, then we get the matrix of coordinate transformation. To 0 0 0 0 each minutiae w(xi , yi , θi ), get the vector of w (xi , yi , θi ), and 0 get a new compared set A after the coordinate transformation. The coordinate transformation formula is follow: 0 xi xi cos α − sin α 0 ∆x 0 y sin α cos α 0 ∆y yi i0 = θ 0 0 1 α θi i 0 0 0 1 1 1 The matching method after the coordinate transformation is similar with the initial matching, but it is the matching between 0 the compared minutiae set A and the template minutiae set B in this time, using a three-dimensional feature vector (x, y, θ). 0 We determine whether the minutiae Ai match Bj by the following formula: ∆x < δ2 0 ∆y < η2 true if Ai matchBj = ∆θ < ε2 f alse otherwise The algorithm use the Difference Total (∆x+∆y +µ∆θ) to measure the extent of their matching. When the one-to-many matching appears, only the matching minutiae pair with the smallest Difference Total is chosen and obtain the final set of the matching pairs.
304
2007 International Conference on Artificial Intelligence and Pattern Recognition (AIPR-07)
5) About the Computation of Score: The computation of score goes through the whole flow of the algorithms. The basis of the computation includes the number, the difference, the local structure of the matching minutiae pairs, and so on. The algorithm get the matching extent on the local structure by the local structure matching, as the extent is specified by the matching number of the neighborhood minutiaes in the local structure, and record the similar extent of the local structure of the ith matching minutiae pair as S1i . By global feature matching, it could get the final number S2 of the matching minutiae pairs. We compute the difference of each matching pair with the coordinate transformation factor on the minutiaes sets, and record the difference of the ith matching minutiae pair as S3i = ∆x + ∆y + δ∆θ. Thereinto, ∆x and ∆y are the differences of coordinates, the ∆θ is the difference of direction, and δ is the weight parameter. The final score is calculated by a linear combination of the three, S increases as S1i or S2 increases, S decreases as S3i increases. a, b, c, d are the weight parameters of each score condition in the computation. S = a × S2 + b ×
S2 X i=0
S1i +
S2 X
(d − cS3i )
i=0
6) The Optimization Based on the Description Model of Palmprint Feature: The feature extraction methods usually give the two-valued logic judgment to the details of whether there is a minutiae on the appointed coordinate of the image. However, the quality of different positions in the palmprint is different, and the reliability of the minutiaes extracted from these positions are different. Therefore, it would distinguish the minutiaes with reliability in the palmprint matching process. The minutiae with the higher reliability should provide more contribution to the matching result. To this end, we modify the description form of the minutiaes. In addition to record geometric information of the minutiaes in the image, we also record the quality information of the minutiaes, describing the reliability of the minutiaes with numerical quantification. Those information is obtained at the stage of feature extraction and post-processing. After being extracted by the feature extraction, a minutiae is initialized by a default reliability value. Then with a series of minutiaes filter rules in the feature post-processing, and the quality of local image which is get by the image pre-processing, the reliability value is modified constantly. In the end, the final reliability value of the minutiae is confirmed. In the process of matching, the reliability information of the minutiaes would be used in the computation of the score. For each minutiae, the reliability information is taken as the weight parameters to compute a new matching score. III. T EST R ESULT Figure 2 reveals the processing flow chart of the system which is used to implement and test the method given in this paper. First the image pre-processing module transforms the input gray-scale palmprint image into the thinning ridge map
Fig. 2.
Palmprint Core Processor Flow
image, using the image processing methods including location, segmentation, enhancement, repair, filter, thinning and so on. Then the feature extraction module extracts the minutiaes. The feature post-processing module filtrates the raw minutiae set, eliminating false minutiaes as many as possible while retaining the true minutiaes. And finally the output is the corresponding minutiae set of the palmprint. We add this minutiae set into the palmprint feature database which is constructed by the minutiae sets of the palmprint images. For a specific compared minutiae set taken out from the palmprint feature database, we use the matching module, with a matching pattern according to the requirements, and take out a series of template minutiae sets from the palmprint feature database, to match to this compared minutiae set. Then the queue of candidates sorted by score is output, and the result is confirmed by manual review. The optional matching patterns include part or full palmprint compared to part or full palmprint, four kinds of patterns in all. The specifications for the use of experimental samples: 2400×2400(pixel), 500DPI, 256-level gray-scale offline palmprint image. We build the full and part palmprint sample databases respectively by 200 full palmprints and 20 part palmprints. The noises are introduced into the samples by random translation, rotation, local distortion, increase of false minutiae, deletion of true minutiae and so on. The test is based on the following hardware platform: Intel Celeon 1.0G CPU, Memory 256M. The algorithm is evaluated on the two aspects of speed and accuracy. A. The Speed Test of the Algorithms The Result of the speed test is related to the minutiae number of the palmprint and the hardware platforms which the test is running on. For each minutiae number, statistics the average value of 1,000 missions. Test the calculation speed of the local structures’ establishment and the matching speed of the minutiae sets respectively. Table 1 shows the result of the matching speed test. B. The Accuracy Test of the Algorithms In this test, the matching score is related to minutiae number of the palmprint in the matching process. To make a uniform measurement on the score difference among each candidates,
305
2007 International Conference on Artificial Intelligence and Pattern Recognition (AIPR-07) TABLE I T EST R ESULT OF M ATCHING S PEED * Neighborhood Eigenvectors Calculation
Minutiaes Number of Template Palmprint Changeless
Minutiaes Number of Compared Palmprint Changeless
Number
Speed(s)
Compared Number
Template Number
Speed(s)
Compared Number
Template Number
Speed(s)
400 300 200
0.038 0.027 0.018
436 200 50
400 400 400
19.300 4.188 0.375
25 25 25
400 300 200
0.134 0.108 0.084
*
all of the data in this table is the average value TABLE II T EST R ESULT OF M ATCHING ACCURACY Matching Pattern
Number of First Candidate
Full*to Full (15:200)
15
11
4
0
0
0
0
95.80%
Part to Full (20:200)
20
0
0
0
5
15
0
47.19%
*
Palmprint Number Distribution On the Score Difference Range ≤ 95% 80-95% 60-80% 50-60% 40-50% ≤ 40%
Average Score Difference
part: part palmprint; full: full palmprint
we evaluate the accuracy on the two aspects of First Candidate and Score Difference. The First Candidate means that the compared palmprint’s homology palmprint in the template palmprints is in the first place of the candidate queue. In the result, statistic the number of the First Candidate. As follows, it is the formula of the Score Difference DIF F S. Si is the score of the ith place in the candidate queue, and N is that the compared palmprint’s homology palmprint in the template palmprints is in the ith place of the candidate queue. In the result, statistic the distribution and the average value of the Score Difference. Ideally, the homology palmprint of the compared palmprint is the First Candidate, and its Score Difference is far greater than other candidates. ( SN −S2 × 100%, N = 1 SN DIF F S = S1 −SN − S1 × 100%, N > 1 Table 2 shows the test result of the matching accuracy. Evaluation it in the two matching patterns of full-to-full and part-to-full. The test sample set consists of 15 compared printed full palmprints, 20 compared part palmprints and 200 template palmprints. Table 2 shows that, in the matching pattern of full-to-full, as the quality of the full palmprints are higher, it could extract a greater number of minutiaes, and there is less impact caused by the minutiae distortion and the false minutiae. The algorithm proposed by this paper could achieve very good performance in this pattern. In the matching pattern of part-to-full, though the accuracy of the feature extraction is not high due to the low quality of part palmprints, the performance of the algorithm could also be satisfactory. IV. C ONCLUSION
samples. The performance of the matching speed and accuracy could meet the requirements of practical applications satisfactorily. The new point pattern matching method proposed by this paper is also able to serve as a solution or a reference idea for other topics related to the point pattern matching problems in the field of computer vision and pattern recognition. ACKNOWLEDGMENT This work is supported by TJNSFC (Natural Science Foundation of TianJin, China), Grant number: 05YFJMJC01500. R EFERENCES [1] A. Ranade and A. Rosenfeld, Point Pattern Matching by Relaxation, Pattern Recognition, 1993, 12(2): 269-275. [2] G. Stockman and S. Kopstein and S. Benett. Matching Images to Models for Registration and Object Detection via Clustering, IEEE Trans. on Pattern Analysis and Machine Intelligence, 1982, 4(3): 229-241. [3] AK. Hrechak, JA. McHugh, Automated fingerprint recognition using structural matching, Pattern Recognition, 1990, 23(8): 893-904. [4] A. Wahab and SH. Chin and EC. Tan, Novel Approach to Automated Fingerprint Recognition, IEE Proceedings Vision Image and Signal Processing, 1998, 145(3): 160-166. [5] DP. Mital and EK. Teoh, An Automated Matching Technique for Fingerprint Identification, First Internatioal Conference on Knowledge-based Intelligent Electronic Systems, 1997, 5: 21-23. [6] X. Jiang and W. Yau, Fingerprint Minutiae Matching Based on the Local and Global Structures, Proc. of 15th ICPR, 2000, 1038-1041. [7] Weiwei Zhang and Yangsheng Wang, Core-based structure matching algorithm of fingerprint verification, Pattern Recognition, 16th International Conference on Proceedings, 2002, 1: 70-74. [8] ZM. Kovacs-Vajna, A Fingerprint Verification System based on Triangular Matching and Dynamic Time Warping, IEEE Transaction on Pattern Analysis and Matching Intelligence, 2000, 22(11): 1266-1276. [9] JPP. Starink and E. Backer, Finding Point Correspondences Using Simulated Annealing, Pattern Recogniton, 1995, Vol 28, No.2, 231-240. [10] N. Ansari and MH. Chen and ESH. Hou, A Genetic Algorithm for Point Pattern Matching, Dynamic Genetic and Chaotic Programming, 1992.
The new palmprint matching method proposed by this paper is able to solve multi-type problems in the palmprint recognition, including online and offline palmprint, full and part palmprint, images with different qualities, and massive 306
