Google Image Swirl: A Large-Scale Content-Based Image Browsing System Yushi Jing, Henry A. Rowley, Charles Rosenberg, Jingbin Wang, Marius Pasca, Yi Liu, Ming Zhao, Michele Covell {jing,har,chuck,jingbinw,mars,yliu,mingzhao,covell}@google.com

Google Inc., Mountain View, California, United States of America

Abstract We demonstrate the first large-scale image browsing system applied to 200,000 popular queries which utilizes image content to organize image search results. Given a query, the system extracts image content features such as color, shape, local features, face signatures and metadata from up to 1000 image results, and hierarchically clusters them to form an exemplar tree. A dynamic web-based user interface allows the user to navigate this hierarchy, allowing fast and interactive browsing. The exemplars of each cluster provide a comprehensive visual overview of the query results, and allow the user to quickly navigate to the images of interest.

1. Exemplar Tree This work organizes image search results into an exemplar tree. We begin organizing up to 1000 results of an image search query by building a pairwise similarity matrix among these images. The similarity computation is based on a combination of image features such as color, texture, local features, face signatures, and the metadata associated with the images. We then perform clustering to partition the search results into hierarchical clusters, each associated with a representative, or exemplar, image. The hierarchical clusters for each query are pre-computed for this demonstration. For more detail, see [3].

Figure 1. Radial layout of cluster hierarchy.

2. User Interface After hierarchical clustering has been performed, the results of an image search query are organized in the structure of a tree. A number of options exist for how to present such a tree to the user. Beyond the typical layered diagram

Figure 2. Expanding sub-clusters selected by user.

used to illustrate tree data data structures, there are many options in the literature, including hyperbolic geometry to better utilize space [4], and a variety of approaches based on tree-maps [1]. In this demonstration, we used radial layout in which each layer of the tree is arranged radially around its parent, see Figure 1. When the user selects a branch of the tree to explore, it is separated from the parent and expanded, while the parent is shrunk to make space, see Figure 2. The rearrangement is animated to allow the user to follow the change without getting lost.

3. Future Work We hope to expand this demonstration to provide the users with text annotations on the clusters, to allow browsing to related queries and to incorporate other media types. We also hope to incorporate a semantic ontology such as WordNet [2] into the generation of the exemplar tree.

References [1] B. Bederson, B. Shneiderman, and M. Wattenberg. Ordered and quantum treemaps: Making effective use of 2D space to display hierarchies. ACM Transactions on Graphics, 21(4):833–854, 2002. [2] C. Fellbaum, editor. WordNet: An electronic lexical database. MIT Press, 1998. [3] Y. Jing, H. A. Rowley, C. Rosenberg, J. Wang, and M. Covell. Visualizing web images via google image swirl. In NIPS Workshop on Statistical Machine Learning for Visual Analytics, 2009. [4] J. Lamping and R. Rao. Laying out and visualizing large trees using a hyperbolic space. In Proceedings of the 7th annual ACM symposium on User interface software and technology, pages 13–14. ACM New York, NY, USA, 1994.

Google Image Swirl: A Large-Scale Content ... - Research at Google

{jing,har,chuck,jingbinw,mars,yliu,mingzhao,covell}@google.com. Google Inc., Mountain View, ... 2. User Interface. After hierarchical clustering has been performed, the re- sults of an image search query are organized in the struc- ture of a tree. A number of options exist for how to present such a tree to the user. Beyond the ...

794KB Sizes 0 Downloads 97 Views

Recommend Documents

Content Fingerprinting Using Wavelets - Research at Google
Abstract. In this paper, we introduce Waveprint, a novel method for ..... The simplest way to combine evidence is a simple voting scheme that .... (from this point on, we shall call the system with these ..... Conference on Very Large Data Bases,.

Web-scale Image Annotation - Research at Google
models to explain the co-occurence relationship between image features and ... co-occurrence relationship between the two modalities. ..... screen*frontal apple.

A CAPTCHA Based On Image Orientation - Research at Google
Apr 20, 2009 - another domain for CAPTCHA generation beyond character obfuscation. ... With an increasing number of free services on the internet, we ..... 100. 200. 300. 400. 500. Figure 8: An image with large distribution of orientations.

The W3C Web Content Accessibility Guidelines - Research at Google
[2], became a W3C recommendation in December 2008. WCAG 2.0 was ... ally possible to make static HTML websites WCAG 1.0 AA conforming without.

Large-Scale Content-Based Audio Retrieval ... - Research at Google
Oct 31, 2008 - Permission to make digital or hard copies of all or part of this work for ... Text queries are also natural for retrieval of speech data, ...... bad disk x.

Video2Text: Learning to Annotate Video Content - Research at Google
learning framework for learning celebrity facial signatures from images in the .... In each iteration, we add .... reggae, soundtrack, uplifting, electronic trance. 4.

Full Resolution Image Compression with ... - Research at Google
This paper presents a set of full-resolution lossy image compression ..... Computing z1 does not require any masked convolution since the codes of the previous.