2011 30th International Conference of the Chilean Computer Science Society

M-CBIR: A medical content-based image retrieval system using metric data-structures Herbert Chuctaya, Christian Portugal, C´esar Beltr´an Cathedra Concytec in TIC National University of San Augustin, UNSA Arequipa, Peru

Juan Guti´errez, Cristian L´opez, Yv´an T´upac Cathedra Concytec in TIC National University of San Augustin, UNSA Arequipa, Peru



development of new computer vision techniques that allow the development of tools to support medical diagnosis, taking advantage of the multimedia information processed. In this scenario, the aim of this work is to implement a content-based retrieval system of medical images that allows us to retrieve and visualize, from a database, the set of most similar by content images for a given query image. In this way, the CBIR system can result in an automatic human body atlas. This work is organized as follows, Section 2 describes briefly the pre-processing of medical images through four feature extraction techniques, Section 3 describes the MCBIR model developed in this work, Section 4 discusses the test steps and experimental setups and Section 5 shows the results of the tests, conclusions and analysis are discussed in Section 6.

Abstract—This work is focused on the modeling and development of a CBIR (Content-based image retrieval) system applied to the recovery of digital medical images of a human body, denominated M-CBIR. This model is composed on two methodologies: features extraction techniques and metric data structures. When this set of techniques is applied to the search of different human body regions, it can retrieve the most relevant similar images to a query image. A real database of medical images composed of 772 medical studies was used to compare the robustness of the extraction techniques and evaluate the performance of the system, through four different extractors. The objective of this work will result in a digital atlas of human body for medical radiological center. Finally, analysis and conclusions are also discussed. Keywords-CBIR; image processing; feature extraction; medical images;

I. I NTRODUCTION CBIR (Content-based image retrieval) systems appears in the 80’s, where one of the first implementations was the IBM QBIC (Query by Image Content). In the last decade, CBIR systems became one of the most interesting topics in computer vision [1]. Some years ago, the systems of medical information only provided textual information about patients in treatment, later this data was stored in large databases where queries were made by text searching into the records of patients. Nowadays, the rapid increment of amount of medical information and the quick accessibility to equipment for medical imaging (MRI, PET, Scanners, etc. . . ), makes relevant the manage and storage of multimedia information. Traditional searches by text were not enough to accomplish the tasks of the medical staff, furthermore the increase in the production of medical digital images and clinical data requires the development of techniques for efficient storing and methods for a robust information extraction of this kind of data. Nowadays, the technology advances in medical research and the intensive use of digital multimedia information, produce data that has become the main resources for medical diagnosis used by the experts in the area. This increase of medical information has attracted the attention of the academic community, involving a lot of research for the 1522-4902/12 $26.00 © 2012 IEEE DOI 10.1109/SCCC.2011.18

II. P RE - PROCESSING OF MEDICAL DIGITAL IMAGES We could say that Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) are different in its method of production and have different utilities of application. The noise produced by this methods is associated with the use of electrical and external influences which affects the results in the features extraction of the image. These influences are associated with random noise that alter its brightness as consequence of this variability is necessary make a digital image pre-process step to ensure that the images conforms correctly to an initial pattern, in Fig. 1 we show the preprocessing step applied to an image with noisy environment around the regions of interest.




Figure 1. Example of a pre-processing step to a digital image. (a) Original Image (b) K-means and morphological filter (c) Resulting Image.


The pre-processing initial step converts the image from RGB to gray scale, then adjust the intensity value of the pixels and finally we binarize the image employing Kmeans with two channels, resulting in regions of interest like showed in Fig. 1(b), in this way we eliminate the noisy existent in the regions of interest in the image as shown in Fig. 1(a), obtaining a resultant image as shown in Fig. 1(c). III. F EATURE EXTRACTORS A. BIC (Border/Interior classification) As was proposed in [2], the method classifies the pixels of the image as interior or border. Then two histograms for the pixels classifies as edge and interior are generated. This descriptor uses the RGB color space, quantized in 64 colors. A pixel is considered as edge if this is found in the border of the image or if one of its adjacent pixels has different color and interior if its adjacent pixels have the same color. The neighborhood criteria considers four adjacent pixels (up, down, left, right).

Figure 2. Spectrum frequency 2D with 4 states and 6 orientations.

that a pixel with gray level i is adjacent to a pixel with gray level j.   p1,1 p1,2 . . . p1,Ng  p2,1 p2,2 . . . p2,,Ng    G= .  .. ..   .. . p .

B. Gabor Transform


Gabor filter is a two-dimensional Gaussian function modulated with sinusoidal orientations at a particular frequency and direction. This technique extracts texture information from an image. In this work, the method proposed by Manjunath and Ma [3], is used. Expanding the mother wavelet Gabor forms a complete but non-orthogonal basis set. The non-orthogonality implies that there will be redundant information between different resolutions in the output data. This redundancy has been reduced by [3] with the following strategy: Lets Ut and Uh the high and low frequency of interest, S be the total numbers of scales, and K the total number of orientations (or translations) to be computed. Then the design strategy is to ensure that the half-peak magnitude support of the filter response in the spectrum of frequency of each contact or view as shown in the Fig. 2, to S = 4 and K = 6. The Gabor transform is then defined by [4] as:

pNg ,1


pNg ,Ng

This method consists of 13 features obtained from the co-occurrence matrix calculated: 1) Energy 2) Correlation 3) Inertia 4) Entropy 5) Inverse Difference Moment 6) Sum Average 7) Sum Variance 8) Sum Entropy 9) Difference Average 10) Difference Variance 11) Difference Entropy 12) Information measure of correlation 1 13) Information measure of correlation 2 D. Gray level Histogram

Z Wm,n (x, y) =

pNg ,2

I(x1 , y1 )gmn ∗ (x − x1 , y − y1 )dx1 dy1

Gray level histograms are the most common technique to describe low-level properties of an image. The histogram of a digital image with intensity levels in then range [0, L − 1], is a discrete function h(rk = Nk ) , where rk is the kth value of intensity and nk is the number of pixels in image with intensity rk . It is common to normalize the histogram to divide each of its components by the total number of pixels in the image denoted by the product M xN , where it is usual that M and N are the dimensions of the image (row and column). The normalized histogram is given by P (rk ) = Nk /M N to k = [0, 1, . . . , L − 1], which becomes a probability of occurrence of gray levels in rk , which implies that the sum of all must be 1.

where: * indicates the complex conjugate; m, n are integers, m = [1, 2, . . . S] and n = [1, 2, . . . , K]; C. Haralick Haralick [5] presents a general statistical model to extract texture information from blocks belonging to an image. This design includes the construction of a spatial co-occurrences matrix GNg ×Ng , where Ng is the number of gray levels. Each element G[i, j] is obtained verifying the amount of pixels with gray level i adjacent to pixels with gray level j. Thus, each entry G[i, j] can be considered as a probability 136


IV. T HE M-CBIR M ODEL A. State of the Art

Features Extractor Gabor Transform Haralick Gray level Histogram BIC Total

Recent works propose a possibly structure of CBIR systems [1], [6]. In this paper the work realized by [7] is considered, where the author makes a revision of 200 references about related works to the content-based retrieval, introducing the concepts of patterns of use, kind of pictures, the role of the semantic in images and the gap semantic problem, also the author presents an introduction to the storage systems and the techniques used to evaluate the CBIR systems performance. In [8] the author propose a term called pattern classification where the CBIR system try to obtain important information about related images based on patterns of similarity present in the image database, so making an automatic categorization of medical images, this categorization obtains 80 categories based on 6231 images of references from a hospital, also describe the modality and orientation of the object contained in the digital image of different body regions. This kind of categorization allows to obtain pathological information from the medical digital images based on previous records stored in the database of medical images, the categorization of medical images is very important for CBIR systems which are not restricted to a specific context, specially in applications of digital radiology as computeraided diagnosis. The images classification is very important to a subsequent recovery by content process because it would allows to select a particular context for the application of a specific algorithmic filter of features extraction[8]. Some works were focused in the resolution of gap semantic problems in order to achieve better result on the level of accuracy obtained from the system of content-based retrieval, [9] presents a new approach to the automatic segmentation based on texture using a modification of DWF (discrete wavelet frame) and the Mean Shift Algorithm, this approach was applied to a data set of real images of textures objects from a museum, this work makes an introduction to a possible integration between content-based image retrieval and texture feature extractors. In [10], the author propose a design and implementation of a IDB system (Image DataBase) that allows contentbased retrieval, focusing on a methodology for the efficient representation and retrieve of medical images based on the spatial information represented by a Attribute Relational Graphs (ARG), thus preserving the characteristics of the objects and its relations between them and its regions. The performance of the systems has been evaluated using a set of 13500 synthetic images. The objective of the this work is try to obtain a model that would allows make content-based retrieval images with a bigger set of unprocessed medical images of different body

Features Vector 48 13 256 128 435

regions, to achieve this the model use a combination of a feature extractor and a Slim-tree metric structure, then we evaluate the performance of each extractor against the same set of queries for each one. We can mention that a few published articles deals with the content-based retrieval for different body regions and a bigger amount of data, they focused on a specific characteristic ob body region and a explicit feature extractor for its solution. B. Set of selected feature extractors In this model, four different features extractors (Gabor, Haralick, Histogram and BIC), were applied to the same images set, obtaining various features vectors that were stored in a Slim-tree metric structure [11]. In addition, we used a distance function to compare the vectors during the data indexing and thus make possible the search by similarity. The Fig. 3 shows a framework of the proposed model, in this Figure we describe the images retrieval process, first an MRI (Magnetic Resonance Imaging) or CT (Computed Tomography) is converted to bitmap format is provided to the system. The image is pre-processed using the above feature extractors in order to obtaining the best image description. • Texture extractors as Gabor transform [3], Haralick[5]. • Intensity extractors as Histogram, Border - Interior Classification (BIC) [12]. The Table I shows the number of features introduced by each extractor used. The feature vectors are stored into several files, one by each technique used. This fact ensures the comparison of feature vectors obtained from the same technique during the retrieval images process. The similarity degree between feature vectors is computed using Euclidean distance metric where a similar image returns a minor distance when compared with the query image. Finally, through the indexing structure, the metric space of more similar images were found. C. Architecture of the M-CBIR model In the previous section were described several previous works in the field of Content-Based Image Retrieval, in [13] and [14], the authors researched the performance of the systems used in a hospital like HIS, RIS and PACS as well analyze the integration and availability of environments works connected to the network. Works like [15] use 137

Wavelets of Daubechies and [16], [17], [18], were focused on the study of the performance of the features extractors and its relevance in the content-based image retrieval, also analyzed the level of accuracy, then evaluated its advantages and disadvantage comparing them against others extractors. Works like [19] and [20] are very important to this model because present an analysis of different features extractors in digital medical images, highlighting the performance of the better algorithm studied under certain restrictions, an analysis of the results allowed us select between various features extractor as Gabor, Haralick, BIC and Gray Level Histogram, all of them were selected by its performance in the field of content-base image retrieval, as a drawback we found that these works made its test with a data set of preprocessed images of specific content 1 . This model, works with 4 extractors analyzed above, these were applied to a wide data set of non preprocessed images containing different body regions, the vectors features obtained for each extractor are then indexed in a Slim-tree metric structure, to achieve this indexing we use a euclidian distance function to compare the vectors throughout the indexing and the query by content. Fig. 3 illustrates the CBIR framework used in this work.

extract a Bitmap image and get access to the tag that describe the region of body contained. This repository contains images of Magnetic Resonance (MR) and Computed Tomography (CT) which are characterized by different uses of interpretation for medical staff and could be real and synthetic data (not suitable). The set of patient’s studies was previously classified into 34 sets, which can be observed in the Table II, this classification shows an abbreviation of a body region. VI. R ESULTS The Fig. 4 shows an scheme of the data diagram flow of the evaluation model of the M-CBIR, note that the model is a content-based image retrieval system that will generate Precision vs Recall Graphs for a set of queries performed for each of the extractors used. Tests were performed on a Core i5/4GB RAM computer, c using Matlab for the process of feature extraction used the extractors explained above, then we use the framework Arboretum2 to make the indexing of the vector features obtained. The user interface was made on QT, Fig. 5 shows an example of the implemented CBIR system when a query was performed and the correspondent similar images retrieved from the database. From the whole images data set, 28933 images were chosen as test images and 1067 as query images. In this work, a total of 70 queries were performed using the features extractors mentioned above, between the set of queries images there was images with CERE, ABTR, ABCO labels.

Histogram GLCM

Features Extraction

Haralick Gabor BIC

Image Query

Features Database

Features of Query Image

2 The GBDI Arboretum is a portable C++ library which implements various metric access method (MAM). http://www.gbdi.icmc.usp.br/

Indexing Technique

Database Images



Similarity Measure Distance metrics

Retrieved Images

Figure 3.


The basis CBIR framework.


Number by Class





Number by Class 190












250 158





















The database used in the experiments was provided by SEDIMED (Support Service to Medical Diagnose), a Medical Radiology Company from Arequipa. The images repository contains 772 studies realized in several patients, resulting on 28292 medical images files in DICOM format. From each DICOM file we can manipulate the patient’s data,








265 2449

1 Images

of specific content here is related to images that contains specific body regions e.g in [20] used medical images of lungs.











1805 401














Medical Images Database

Images indexing

Extractors Selection


Feature vectors

Euclidian distance


Indexed Database


Medical Images Database for Queries



Feature vectors

Similarity Images

Precision vs Recall

Figure 4.

M-CBIR model scheme. This model let us obtain Graphics Precision vs Recall for each feature extractor being selected

For each query image, a subset of 5 k nearest neighbors was applied with k = 50, 100, 150, 200, 150 values. For each of the k values was calculated its precision and recall, then for the set of 5 queries per images the average precision and recall was computed, later the values of precision and recall were ordered. The Fig. 6 shows the average precision from 70 different images, for each feature extractor.

future feature extractors. We can note that if the Gabor filter is better than the others (BIC, HISTOGRAM) these others can obtain good results for our purposes, the behavior of the extractors is almost the same except by Haralick who obtained low precision results for the same set of queries. The values of recall are between 0 and 0.2, this is because the amount of relevant images in the database is higher than the values of K on the query, e.g images with label CERE are 7204 and a query with k=50 will obtain a very small value but the values of precision are good considering the variety and amount of images present in the database. The development of this work, as part of a complete project, consists in the implementation of a framework that integrate the CBIR model with another module which analyzes the semantic of the medical diagnose and description provided for the expert in radiology. By this integration we will obtain an image and a analyzer system of medical content. ACKNOWLEDGMENT

Figure 5.

The authors gratefully acknowledges the financial support provided by FIDECOM (Fondo de Investigaci´on y Desarrollo para la Competividad) and Cathedra Concytec of Information Technology and software development at National University of San Augustin.

Example of a query and retrieval image



In this work, four features texture and intensity extractors were evaluated. The experiments were performed using a considerable real data volume and, by the outcomes observed for this data, the Gabor Transform shows the best results in similarity for queries. This results we able to evaluate our

[1] H. Muller, N. Michoux, D. Bandon, and A. Geissbuhler, “A review of content-based image retrieval systems in medical applications-clinical benefits and future directions,” International Journal of medical informatics, vol. 73, pp. 1–23, Febraury 2004. 139










BIC Gabor Haralick


Histogram 0




Figure 6.









Average precision from 70 query images for 4 feature extractors.

[2] A. X. F. a. Renato O. Stehling, Mario A. Nascimento, “A compact and efficient image retrieval approach based on border/interior pixel classification,” Proceedings of the eleventh international conference on Information and knowledge management, vol. I, pp. 102–119, 2002.

[9] M. Fauzi and P. Lewis, “Automatic texture segmentation for content-based image retrieval application,” Pattern Analysis & Applications, vol. 9, no. 4, pp. 307–323, 2006. [10] E. Petrakis, “Content-based retrieval of medical images,” International Journal of Computer Research, vol. 11, no. 2, pp. 171–182, 2002.

[3] B. S. Manjunath and W. Ma, “Texture features for browsing and retrieval of image data,” IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI - Special issue on Digital Libraries), vol. 18, no. 8, pp. 837–42, Aug 1996. [Online]. Available: http://vision.ece.ucsb.edu/publications/ 96PAMITrans.pdf

[11] C. T. Jr., A. Traina, B. Seeger, and C. Faloutsos, “Slim-trees: High performance metric trees minimizing overlap between nodes,” in Proceedings of EDBT 2000, march 2000, pp. 51– 65. [12] R. O. Stehling, M. A. Nascimento, and A. X. Falc˜ao, “A compact and efficient image retrieval approach based on border/interior pixel classification,” in Proceedings of the eleventh international conference on Information and knowledge management, ser. CIKM ’02, New York, NY, USA, 2002, pp. 102–109.

[4] W. S. H. M. W. Ahmad and M. F. A. Fauzi, “Comparison of different feature extraction techniques in content-based image retrieval for ct brain images.” in MMSP’08, 2008, pp. 503– 508. [5] R. M. Haralick and K. S. D. . Hak, “Texture features for image classification of image data,” IEEE Transactions on Systems, Man and Cybernetics, vol. 3, pp. 610––621, 1973.

[13] N. Rosa and A. Traina, “Uma abordagem pr´atica e eficiente de consultas por similaridade para suporte a diagn´ostico por imagens,” Master’s thesis, Instituto de Matem´atica e Estatistica - USP, 2002.

[6] K. Wanjale, T. Borawake, and S. Chaudhari, “Content based image retrieval for medical images techniques and storage methods-review paper,” International Journal of Computer Applications IJCA, vol. 1, no. 19, pp. 110–112, 2010.

[14] E. C. Carit´a, “Sistema de gerenciamento de imagens para ambiente hospitalar com suporte a` recuperac¸a˜ o de imagens baseado em conte´udo.” Master’s thesis, Faculdade de medicina de Ribeir˜ao Preto, 2009.

[7] A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, “Content-based image retrieval at the end of the early years,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 22, no. 12, pp. 1349–1380, 2000.

[15] C. Casta˜no´ n, “Recuperac¸a˜ o de imagens por conte´udo atrav´es de an´alise multiresoluc¸a˜ o por wavelets,” Master’s thesis, Instituto de Matem´atica e Estat´ıstica, 2003.

[8] T. Lehmann, M. G¨uld, T. Deselaers, D. Keysers, H. Schubert, K. Spitzer, H. Ney, and B. Wein, “Automatic categorization of medical images for content-based retrieval and data mining,” Computerized Medical Imaging and Graphics, vol. 29, no. 2-3, pp. 143–155, 2005.

[16] J. C. Felipe, “Desenvolvimento de m´etodos para extrac¸a˜ o, comparac¸a˜ o e an´alise de caracter´ısticas intr´ınsecas de imagens m´edicas, visando a` recuperac¸a˜ o perceptual por conte´udo,” Ph.D. dissertation, Universidade de S˜ao Paulo, 2005. 140

[17] A. G. R. Balan, “M´etodos adaptativos de segmentac¸a˜ o aplicados a` recuperac¸a˜ o de imagens por conte´udo,” Master’s thesis, Instituto de Ciˆencias Matem´aticas e de Computac¸a˜ o, 2007. [18] A. Gracioso, “Avaliac¸a˜ o do desempenho de recuperac¸a˜ o de imagens m´edicas baseada em conte´udo em redes de computadores,” Ph.D. dissertation, Universidade de S˜ao Paulo, 2008. [19] H. Razente, “Adequando consultas por similaridade para reduzir a descontinuidade semˆantica na recuperac¸a˜ o de imagens por conte´udo,” Master’s thesis, Instituto de Ciˆencias Matem´aticas e de Computac¸a˜ o - USP, 2006. [20] P. Bugatti, “An´alise da influˆencia de func¸o˜ es de distˆancia para o processamento de consultas por similaridade em recuperac¸a˜ o de imagens por conte´udo,” 2008.

141 All in-text references underlined in blue are linked to publications on ResearchGate, letting you access and read them immediately.

M-CBIR: A Medical Content-Based Image Retrieval ...

The similarity degree between feature vectors is computed using Euclidean distance ... shows an abbreviation of a body region. VI. RESULTS. The Fig. .... dos `a recuperaç˜ao de imagens por conteúdo,” Master's thesis,. Instituto de Ciências ...

644KB Sizes 2 Downloads 210 Views

Recommend Documents

ContentBased Access to Medical Image Collections
database is too large, image visualization overlapping is reduced using a ..... The technique builds the neighborhood graph using knearest neighbors and the.

Content-Based Medical Image Retrieval Using Low-Level Visual ...
the retrieval task of the ImageCLEFmed 2007 edition [3], using only visual in- ..... models that mix up textual and visual data to improve the performance of our.

Image Retrieval: Color and Texture Combining Based on Query-Image*
into account a particular query-image without interaction between system and .... groups are: City, Clouds, Coastal landscapes, Contemporary buildings, Fields,.

signed value of one is denoted by XE = {Xj1 , ..., XjE } and is called the ... XE is in practice much smaller than. XH. .... Fourteen trademark examiners in the Intel-.

Content-Based Image Retrieval Systems: A Survey
Oct 28, 2002 - (This is one of the so-called Socratic dialogues, where two persons ... Below we describe a number of content-based image retrieval systems, in alphabetical ...... Applications The database consists of 400 advertisements and 800 color

An active feedback framework for image retrieval - Semantic Scholar
Dec 15, 2007 - coupled and can be jointly optimized. ..... space when user searches for cars images. ..... Optimizing search engines using click through data.

Semantic Image Retrieval and Auto-Annotation by ...
Conventional information retrieval ...... Once we have constructed a thesaurus specific to the dataset, we ... the generated thesaurus can be seen at Table 5. 4.3.

Shape-Based Image Retrieval in Logo Databases
In recent several years, contents-based image re- trieval has been studied with more attention as huge amounts of image data accumulate in various fields,.

Robust Content Based Image Retrieval system using ... - Dipan Pal
robust classification shows that the image database can be expanded while ... algorithm which tries to capture the data modeling and processing ... input during learning of the spatial pooler is compared with .... big advantage in scalability.

Shape Indexing and Semantic Image Retrieval Based on Ontological ...
Center retrieves images, graphics and video data from online collections using color, .... ular class of image collection, and w(i,j) is semantic weight associated with a class of images to which .... Mn is defined by means of two coordinates (x;y).

Interactive and progressive image retrieval on the ...
INTERNET, we present the principle of an interactive and progressive search ... make difficult to find a precise piece of information with the use of traditional text .... images, extracted from sites of the architect and providers of building produc

Exploring Statistical Correlations For Image Retrieval
Phone: (86-10) 62617711-3175 ..... segmentation algorithms often break an object into multiple regions. ... These extracted codewords, as the blue lines.

Enhancing Image and Video Retrieval: Learning via ...
School of Computer Science and Engineering and the Center for Neural Computation. The Hebrew ... on the clustering and retrieval of surveillance data. Our.

Tattoo-ID: Automatic Tattoo Image Retrieval for Suspect ...
performance of the system is evaluated on a database of 2,157 tattoos representing 20 .... 2 Tattoo Image Database. We have ..... Intelligence, vol. 19, no. 7, pp.