Seeing through bag-of-visual-word glasses: towards understanding quantization effects in feature extraction methods Alexander Freytag∗ , Johannes R¨uhle∗ , Paul Bodesheim∗ , Erik Rodner∗ , Joachim Denzler∗ ∗ Computer Vision Group, Friedrich Schiller University Jena

{firstname.lastname}@uni-jena.de

Summary To answer this question, we present an indepth analysis of the effect of local feature quantization on human recognition performance. Our analysis is based on recovering the visual information by inverting quantized local features and presenting these visualizations with different codebook sizes to human observers (Fig. 2). Although feature inversion techniques are around for quite a while, to the best of our knowledge, our technique is the first visualizing especially the effect of feature quantization. Thereby, we are now able to compare single steps in common image classification pipelines to human counterparts. Our results show that (i) humans perform significantly worse than machine learning approaches when being restricted to the visual information present in quantized local features rather than having access to the original input images, and (ii) that early stages of low level local feature extraction seem to be most crucial with respect to achieving human performance on original images. Finally, we demonstrate (iii) that large codebook sizes in the order of thousands of prototypes are essential not only for good machine learning performance, but more interestingly, also for human image understanding. Method Our technique is simple and in line with current trends in image reconstruction from local features. For an unseen image, we extract local features on a dense grid and follow the bag-of-words paradigm by quantizing them using a pre-computed codebook. Based on inversion techniques for local features, we can compute the most probable image patch for every prototype, i.e., we can visually inspect the quantization quality for a given codebook. Thus, for any local feature, we vector-quantize it with a codebook and draw the inverted prototype into the reconstruction

Local descriptors

Vector quantization

120

100

80

60

20

0

0

200

400

600

800

1000

1200

1400

Index of Prototype

1600

1800

2000

su O al ur iz at io n

40

vi

Frequency

140

average recognition rate [%]

Input image

90

BoW feature

Problem formulation Vector-quantized local features frequently used in bag-of-visual-words approaches are the backbone of popular visual recognition systems due to both their simplicity and their performance. Despite their success, standard bag-ofwords-histograms basically contain low-level image statistics (e.g., number of edges of different orientations). The question remains how much visual information is lost in quantization when mapping visual features to visual “words” and elements of a codebook?

Spatial pooling

80 70

ML-performance (k=2048) ML-performance (k=32)

60 50 40 30 20 10 0

Random guessing 32

128

512

2048

HOG

Orig

type of quantization / codebook size

Fig. 1: Inversion strategy and human classification result in comparison to machine performance

Fig. 2: Overview of the images presented to human observers during the experiment (15 Scenes dataset) image with position and size according to the support of the extracted local feature (Fig. 1). In contrast to previous works for feature inversion, we aim at explicitly visualizing quantization effects. For the simplicity of demonstration, we use HOG features, where code for the inversion technique is publicly available1 . Evaluation We performed human studies, where more than 3,000 images were classified by several individuals with different levels of quantization (Fig. 2). The results are given in the right plot of Fig. 1 and will be presented in detail at the poster2 . 1 http://web.mit.edu/vondrick/ihog/ 2 Source code, detailed results, and a web interface to our evaluation server are available at http://www.inf-cv.uni-jena.de/en/image representation .

towards understanding quantization effects in feature ...

Seeing through bag-of-visual-word glasses: towards understanding quantization effects in feature extraction methods. Alexander Freytag∗, Johannes Rühle∗, ...

2MB Sizes 1 Downloads 183 Views

Recommend Documents

towards resolving ambiguity in understanding arabic ... - CiteSeerX
deal strides towards developing tools for morphological and syntactic analyzers .... ﻪﺗاذ ﲎﺒﳌا ﰲ. The meeting were attended in the same building (passive voice).

Towards Lifelong Feature-Based Mapping in ... - Research at Google
Here we briefly introduce some analytical tools from sur- vival analysis; interested readers may see [24] for more detail. A distribution pT (·) over survival time T ...

Towards Lifelong Feature-Based Mapping in ... - Research at Google
there exist mature algorithms and software libraries that can solve graphical ... observations associated with a small physical area). ..... distribution pT (·) for the survival time (accounting for the ..... Springer Science+Business Media,. 2001.

Towards Re-defining Relation Understanding in Financial Domain
leveraging domain features that capture nancial terminology. We share challenge results for our submission, which performed well achieving the highest score ...

Towards an understanding of oversubscription in cloud
Cloud providers oversubscribe their data centers to lever- ... We consider an Infrastructure as a Service (IaaS) cloud ..... Then, the total utility of the system is.

Towards Machine Understanding: Some ...
mathematical abstraction for what is "Semiosis" in the real world. 1. ..... models of reality (i.e. models that fully explain this strange thing we call reality), mainly ...

Towards Feature Learning for HMM-based Offline ...
input-image and supervised methods can be applied easily, many state-of-the-art systems for the recogni- tion of handwritten text rely on a segmentation-free ap ...

towards a threshold of understanding
Online Meditation Courses and Support since 1997. • Meditation .... consistent teaching, enable the Dhamma to address individuals at different stages of spiritual .... Throughout Buddhist history, the great spiritual masters of the. Dhamma have ...

Towards Understanding Software Evolution: One-Line Changes
all changes made during the maintenance of the software under consideration ... error that cost a company 1.6 billion dollars and was the result of changing a ...

TOWARDS THE UNDERSTANDING OF HUMAN DYNAMICS ... - arXiv
mail, making telephone call, reading papers, writing articles, and so on. Generally ...... Reunolds, P. [2003] Call Center Staffing (The Call Center School Press,.

Human Factors of Automated Driving: Towards Predicting the Effects ...
Interface. Human driving behaviour. Longitudinal and lateral dynamics. Vehicle. Road and traffic flow conditions. Driver capabilities. Environmental conditions ... Congestion. Automated driving. What are the effects on traffic flow efficiency? Accide

fractional quantization
One of my favorite times in the academic year occurs in early spring when I give my .... domain walls between even-contracted regions and odd-contracted ones. ..... a shift register, the net result being to transfer one state per Landau level.

Undo Send Feature in Gmail
If you make a typo, change your mind or forget an attachment when sending an email, you can take back an email using the ​Undo Send​feature. First, make ...

Towards a characterization and understanding of ...
technology; (b) new pedagogy which abandons an “information transfer” ..... Keitel and K. Ruthven (Eds.), Learning from Computers: Mathematics Education and.

deformation and quantization
Derivations of (1) have been given using brane quantization ... CP1 of complex structures aI + bJ + cK, a2 + b2 + c2 = 1, .... Use V to introduce the Ω-deformation:.

Towards Understanding Software Evolution: One-Line ...
[email protected]. Dewayne E. Perry. ECE & UT ARISE. The University of Texas at Austin. Austin, Texas 78712-1084 [email protected]. Abstract. Understanding the impact of software changes has been a challenge since software systems were first

1-bit Compressed Quantization
[email protected]. Abstract. Compressed sensing (CS) and 1-bit CS cannot directly recover quantized signals preferred in digital systems and require time consuming recovery. In this paper, we introduce 1-bit compressed quantization (1-bit CQ) th

Quantization of Constrained Systems
4One will recall that a function's gradient is everywhere perpendicular the function's surfaces of .... The conjugate momenta for the Cartesian coordinates are.

Understanding the Welfare Effects of Unemployment ...
∗I thank seminar and conference participants at Arizona State University, Concordia University, ..... Quantitatively, the price effect is very small (note that Figure 4 is drawn in the same scale ... (2006) in the context of business cycle stabiliz

Understanding the Effects of Unanticipated Future ...
28 Dec 2014 - Analysis of U.S. data from 1967:Q1 to 2008:Q1 shows that the information structure on monetary policy ... In contrast to the information flows as above, the main information specification of this paper employs ...... RAMEY, V. A. AND M.

Understanding the Welfare Effects of Unemployment ...
where θt ≡ vt/ut represents the labor market tightness at time t. In the calibrated .... dispersion of wealth is smaller than what is seen in the data.13. Another strong ..... In calculating the welfare after the reform, I feed the equilibrium pri

Understanding the Dynamic Effects of Government ...
effects of fiscal policy on foreign trade: an increase in government spending .... move closely together at business cycle frequencies, since the stock of debt ...

Understanding the Welfare Effects of Unemployment Insurance Policy ...
economy, the welfare benefit of having access to unemployment insurance above the current ... serve Bank of New York, Federal Reserve Bank of Philadelphia, Goethe ..... There is a continuum (population 1) of competitive firms, each using an identical