1

Content-Based Histopathology Image Retrieval Using Latent-Semantic-Kernels Jose G. Moreno

Abstract—In this paper we propose an approach to explote the visual semantic existing in visual features in a medical image retrieval system. We achieve this using Latent Semantic Kernels based on different kernel functions to generate a latent semantic space in which topics relate different visual contents. The proposed method is tested in the histopathology images. Experimental results show an improvement of the visual latent approach with respect to single visual information. Many image retrieval systems use automatic annotation techniques, that are trained using manually assigned semantic labels. However, in real world collections those annotations are not always available and instead. This work presents a strategy The proposed approach is based on the query-by-example paradigm, so an image is used to search more images. The main problem is to identify relevant images exploting the visual information. Different kernel functions are used to generate a latent semantic space with topics that relate several visual contents. Index Terms—Latent semantic kernels, Histopathology Images.

I. I NTRODUCTION nformation retrieval is area currently working for research community, such effort emerge new models focused on inproving results. Importance of the work are obtained with the information retrival of text documents has given a strong impetus for generating interest in other important areas, such as content based information retrieval. Content based information retrieval systems to extract features to establish the visual representation of the images, which is then used with different strategies to accomplish the retrieval task. In medicine, the amount of generated images is constantly growing, creating management and storage needs. Is interesting that can be found associate images with similar medical records. Image analysis is focused on different tasks like classification[10], visualization, annotation[13] and retrieval[14], among others, are being exploited for purposes that range from the academy in the medical environment to support activities related to medical diagnosis. In recent methods is a strong trend of motivated approaches developed for text information retrieval, these methods use a visual representation called visual words. This visual words are image subsets that frequently occurs in all image database. Such techniques are interesting for the possibility of applyed textual developments in images, assuming that the words in visual images behave similarly to the words in a document. Regions based techniques[1], [5] are based on the notion of the photographic composition of

I

Jose G. Moreno is student master in system engineer and computer science of the National University of Colombia. (e-mail: [email protected]). Fabio A. González is Associate Professor of System and Computer Engineering Departament of National University of Colombia and he is subdirector in Bioingenium Research Group. (e-mail: [email protected]).

an image, focus on the identification of relevant regions and the extraction of features such as color and texture of this region identified. Several papers [1] propose to use characteristic vectors representra used the multiple features and other models, which are exploring techniques used kernel functions between images to perform the same tasks, among these the most widely used methods of support vector machines (SVM). Tendency to use strategies that come from similar areas of study is strongly influenced by strategies such as bag of words, or others such as the application of latent semactic analysis in visual systems. This paper proposes the use of latent semantic kernels for the construction of a retrieval system for content. This strategy allows merging different visual characteristics to construct a kernel function that is the basis for the construction of a latent space where visual information is related to visual topics. These topics allow the reduction of the original with no loss information and are making an analogy with the strategy textual, visual concepts groupings, i.e. a reduction in similar topics. The LSK is based on singular decomposition and the results show that it is possible a significant reduction of the original space. The paper is organized as follows: Section 2 presents the previous work on histopathology image retrieval. Section 3 describes the latent semantic kernel. Section 4 presents experimental results, and finally the concluding remarks are in Section 5. II. P REVIOUS W ORK Methods like bag-of-words [5]for documents are carried to images approach. More intuitive operation of this approach is: 1. Get the whole regions of images to be processed. 2. Extract the characteristics of these regions. 3. Apply a clustering algorithm for visual words. There are different approaches that depence on the techniques used for find number of visual words . In such strategies it is still considered adequate obtensión appropriate vocabulary for the representation of images. In [8], [7], globals features are described. Image content is analised and the visual features are extrated for the all image. Similarity between the images is calculate using this global information. In the histogram based approach, an histogram is building whit this features and histograms moments. Methods using PCA has been applied in medical imaging, in [11] create a correlation matrix of mean centered data set, then this matrix is computed eigenvalues and eigenvectors. Eigenimages are the projection of images using eigenvectors. These eigenimages have a particular characteristic, it is possible to obtain the database of original images from linear combinations of eigenimages. Similarity of the queries regarding the

2

database of images is found using the relationship between the weights that allow the query image is reconstructed and the weights of the images in the database. III. V ISUAL L ATENT S EMENTIC KERNELS Visual features combination is a non-trivial task. More intuitive approaches tend to concatenate the feature vectors constructed for each feature. LSK [2] is a technique based on SVD is similar to LSI, but we do not have a direct relationship between the visual terms and images. Whit LSK, we can use |kernel functions for the image representation, or combinations thereof. With this representation is related semantic information explicit. LSK allows reducing the information to the k most relevant topics, allowing information represent different precision making it big or small the set of k selected topics. With a certain kernel function, it is possible to construct the kernel matrix of the image database. This kernel matrix is a nxn squared matrix defined as: Ki,j = k(di , dj ); j = 1, ..., n; i = 1, ..., n where k(di , dj ) is a kernel value between image i and image

IV. R ESULTS AND D ISCUSSION Experimental setup is defined as follows: • Kernel matrix construction and eigendecomposition. • Calculating image database and queries proyections in the latent space. • Calculation of the similarity between in the laten space. • Evaluation of the rankings. following describes in detail. A. Data Set A collection of medical images consist of histopathological images used for experimentation. This comprised of 1502 images annotated by experts in 18 categories. These entries correspond to concepts in the histopathologic image. It pick 76 images as queries and the rest of the coleeccion, 1426 images are used as the database of images. When you perform a query the resulting images are relevant if the concept of the histopathological image query is contained in the image result. Each image can have one or more annotations for each image it is queried with each of its. Finally, we have 151 differents queries with the 76 images.

j. Kernel matrix K is descomposed whit SVD and U matrix is used for image database projection. K = U ΣV 0 Σ and U can be calculating whit the follow equation: vi KK 0 = λi KK 0 where all vi are the columns of U and Σ is a diagonal matrix whit corresponding λi in the diagonal. This is an eigendescomposition equation. LSK is applied to the corresponding kernel matrix to obtain a matrix U with the eigenvectors v of the latent semantic space. This new representation space correlates visual and textual features in common semantic topics. The new semantic space is defined by selecting k topics, that corresponds to the k eigenvectors v associated to the k largest eigenvalues λ. Then, all images are projected to the latent semantic space using the following equation:  φ(d)Uk =

1 λi− 2

l X j=1

k (vi )j km (dj , d)

,

B. Kernel construction It has six kernels, constructed from the following visual features: • Local Binary Partition (LBP). • Color (RGB). • Tamura (TAM). • Sobel (SOB). • Invariant (INV). • Gray (GRA). With each of these images are features extraction and build a histogram. For the kernel function was used kernel intersection. Histogram intersection is a similarity function devised to calculate the common area between histograms as follows: k(A, B) =

m X

min{ai , bi }

i=0

Where A and B are the histogram of the image A and the image B respectivily. It has been originally used for image similarity search [9].

(1) C. Rank1

i=1

where φ(d)Uk is the vector whose coordinates contain the presence degree of each topic in the document d. Images database and queries can be mapped to a semantic space with the equation 1. In this space it is possible to determine the similarity between images, and in the specific task of retrieving information may be used a measure of similarity to can be establish a ranking. Then the similarity between a query and an image of the database is given by: S(di , dq ) = S(φ(di )Uk , φ(dq )Uk )

Rank1 is a measure use for ranks and this is the average position of the firts revelant image. The results show that LSK whit all features get a better results that use a single feature. Figure 1 show the results of rank1 values for diferents k values. D. Average Rank This measure evalute the average position of relevant images. Smaller values indicate better position in the ranking of the images. Results show that LSK improvement over the use of only one feature.

3

F. Selecting k value Calculation of k visual topics was conducted by calculating the map, rank1, presicion at 1 and average rank varing the k values. Table I show the better measures values obteined and the respectibily k value. The proposed method shows an improvement on the results obtained with the information in the histograms. V. C ONCLUSIONS AND FUTURE WORK

Figure 1.

Rank1 for differents k values

The proposed model does not use an explicit representation of information from the images of the finished document is sufieciente with a kernel function that takes a measure of similarity between images. But exploiting the spectral decomposition of this information. It is based on spectral decomposition that allows exploting the visual semantic relationships between different representations of the images. The results show no significant decrease of MAP, however, achieves an increase in measures such as Rank1 and Average Rank, indicating that the LSK should be used in tasks focused on the use of these measures. As future work is the use of this approach on data sets with different cross-modal representations made and the improvement self-correlation of documents associated problem with spectral descomposition. R EFERENCES

Figure 2.

Average rank for differents k values.

E. Mean Average Presicion MAP is a measure used in information retrieval to assess the accuracy taking into account the location of the relevant images. Whit the mixed kernel Map, the measurement of MAP is not improved. However, it is not significantly deteriorated.

Figure 3.

MAP for diffrenets k values

[1] C. Carson, S. Belongie, H. Greenspan, and J. Malik. Region-based image querying. In IEEE Workshop on Content-Based Access of Image and Video Libraries, 1997. Proceedings, pages 42–49, 1997. [2] Nello Cristianini, John Shawe-Taylor, and Huma Lodhi. Latent semantic kernels. Journal of Intelligent Information Systems, 18(2):127–152, March 2002. [3] Ritendra Datta, Dhiraj Joshi, Jia Li, and James Z. Wang. Image retrieval: Ideas, influences, and trends of the new age. ACM Comput. Surv., 40(2):1–60, 2008. [4] J. Jeon, V. Lavrenko, and R. Manmatha. Automatic image annotation and retrieval using cross-media relevance models. In Proc. ACM SIGIR, pages 119–126, Toronto, Canada, 2003. ACM. [5] R. W. K. Lam, H. H. S. Ip, K. K. T. Cheung, L. H. Y. Tang, and R. Hanka. A multi-window approach to classify histological features. In Pattern Recognition, 2000. Proceedings. 15th International Conference on, volume 2, 2000. [6] T. M. Lehmann, M. O. Gold, C. Thies, B. Fischer, K. Spitzer, D. Keysers, H. Ney, M. Kohnen, H. Schubert, and B. B. Wein. Content-based image retrieval in medical applications. Methods of Information in Medicine, 43(4):354–361, 2004. [7] H. MÃijller, N. Michoux, D. Bandon, and A. Geissbuhler. A review ˘ Tˇ of content-based image retrieval systems in medical applicationsâA clinical benefits and future directions. International journal of medical informatics, 73(1):1–23, 2004. [8] A. Mojsilovic and J. Gomes. Semantic based categorization, browsing and retrieval in medical image databases. In Proc. of IEEE ICIP, 2002. [9] C. L. Nutt, D. R. Mani, R. A. Betensky, P. Tamayo, J. G. Cairncross, C. Ladd, U. Pohl, C. Hartmann, M. E. McLaughlin, and T. T. Batchelor. Gene expression-based classification of malignant gliomas correlates better with survival than histological classification 1, volume 63. AACR, 2003. [10] Jia-Yu Pan, Hyung-Jeong Yang, Christos Faloutsos, and Pinar Duygulu. Automatic multimedia cross-modal correlation discovery. In Proc. ACM SIGKDD, pages 653–658, Seattle, WA, USA, 2004. ACM. [11] U. Sinha and H. Kangarloo. Principal Component Analysis for Contentbased Image Retrieval 1, volume 22. RSNA, 2002. [12] H. L. Tang, R. Hanka, and H. H. S. Ip. Histological image retrieval based on semantic content analysis. IEEE Transactions on Information Technology in Biomedicine, 7(1):26–36, 2003.

4

MAP Feature LBP RGB TAM SOB INV GRA LSK

k 150 100 100 200 50 200 100

P(n=1) Value 0.093 0.089 0.084 0.093 0.091 0.088 0.097

k 1150 450 100 700 150 150 450

g Rank

Rank1 Value 0.47 0.45 0.34 0.55 0.40 0.37 0.52

k 600 850 400 50 500 50 150

Value 18.03 11.42 9.56 11.32 12.85 12.78 7.80

k 900 850 700 600 300 750 800

Table I B EST VALUES FOR EACH MEASURE FOR DIFFERENTS FEATURE . B EST SCORE ARE IN BOLD .

[13] A. Vinokourov, D. R. Hardoon, and J. Shawe-Taylor. Learning the semantics of multimedia content with application to web image retrieval and classification. In Proc. of Fourth International Symposium on Independent Component Analysis and Blind Source Separation., 2003. [14] L. Zheng, A. W. Wetzel, J. Gilbertson, and M. J. Becich. Design and analysis of a content-based pathology image retrieval system. IEEE Transactions on Information Technology in Biomedicine, 7(4):249–255, 2003.

Value 542.16 519.52 543.88 498.29 510.17 512.96 485.29