The Automatic Detection of Patterns In People’s Movement Gordon Forbes and Gerhard de Jager Department of Electrical Engineering University of Cape Town Rondebosch 7701, South Africa Email: [email protected]

Abstract This paper presents a method for automatically detecting patterns that people walk. A datastream of a person’s x and y co-ordinates in a plan view of an office are preprocessed to determine individual movements. These movements are then clustered into similar movements using a furthest-neighbour clustering algorithm that uses Dynamic Time Warping as a distance measure between individual movements.

1

determine the location of a person in the office. This was done by detecting a pink hat that was worn by the person being tracked. The centroid of the pink hat gave the person’s location.

Introduction

A standard signal processing problem is that of looking for a specified template amongst one or more datastreams. Such a template might be distorted in the datastream, but can still be detected relatively easily. This work goes one step further, by attempting to automatically detect all patterns that are present in a datastream. Once these patterns have been detected, templates can be automatically generated. This is especially useful when dealing with large amounts of data as it is unfeasible to have a person generate the templates by hand. The datastream that is used in this project is the location of a person in a plan view of an office. The location is given as a set of x and y co-ordinates. Numerous samples of the person’s location are taken at a rate of approximately 10 fps (frames per second). The hypothesis is that as the person goes about their daily “office routine”, they will repeat certain patterns rather frequently. For instance, arriving in the morning, going to the toilet and making coffee. It is also expected that a large number of less obvious patterns will emerge. A wide angle lens camera was mounted in the roof of an office, looking down. Video footage was then analysed to

Figure 1: Output from the segmentation. The black square indicates the person’s location.

2 Finding Patterns Clearly, finding all of the patterns that occur within a datastream is not a simple task. For this reason it was necessary to make a number of assumptions that would constrain the problem into a more realistic domain. The patterns that are being searched for are ones that are caused by the person moving around the office. For this reason, The datastream could be easily segmented into regions in which there was movement, and regions in which there was no movement.

The most important assumption that was made was that each movement that a person makes could contain at most, one single pattern. For example, a person would walk in to the office and go to his desk. Once there, he would be stationary (at least for a little while). It is very unlikely that someone would walk to their desk, go straight to the coffee machine, make some coffee, and then go to the toilet, all without stopping at all in between. It is much more likely that they would at least stop briefly at their desk and at the coffee machine. In doing so, they would break up the one large movement into three smaller movements.

3

dissimilarity between two of the objects that are trying to be clustered [2]. Traditional distance measures include:

 Euclidean Distance

∑ x  y  i

 Mahalanobis Distance x  m T C

1



x  m

 City Block Distance ∑ xi  yi   Chebychev Distance Maxx i  yi 

Pre-Processing

The goal of preprocessing the data was to convert the stream of x and y co-ordinates into a large array of isolated movements. Once the isolated movements have been detected, unsupervised learning techniques can be used to cluster the isolated movements into groups of movements that are similar. The following pre-processing was done to the raw input data: Filtering The data was passed through a 7 point median filter to eliminate any spurious detections. Detecting Movement The variance of the data was used to detect when the subject was remaining still and when he was moving. Merging Adjacent Movements If two movements occurred close together, they were merged into one movement. This concluded the preprocessing of the data, leaving an array of the individual movements that had been made by the person in the office.

Figure 3: Two similar movements that can’t be easily compared due to their differing length. The difficulty with applying these measures directly to compare two movements is that the movements differ in length. One possible way of overcoming this problem would be to use Linear Time Normalisation (LTN). LTN can be achieved in a number of ways. These include downsampling all signals to the length of the smallest signal and matching according to the formula [3]: Tx

d X  Y  

Figure 2: Output from the preprocessing. x co-ordinates, y co-ordinates, and the areas of movement.

Dynamic Time Warping

In order to make use of standard clustering techniques, a distance measure is required that indicates the similarity /

∑ d ix

ix 1



iy 

(1)

where ix and iy satisfy the equation: iy 

4

2

i

Ty iy Tx

(2)

Since ix and iy are integers, some kind of round off rule is required. The Summation in Equation 1 could also be done from iy  1 to iy  Ty [3]. The disadvantage of using LTN is that it performs poorly when the person walking does not maintain a constant speed. A method that overcomes this problem is Dynamic Time Warping (DTW). Dynamic Time Warping has been used with a large degree of success in the speech recognition community. It makes use of Dynamic Programming techniques to perform time-normalisation between two signals [3]. In effect, the DTW warps the two signals so that

they match each other as best as possible. The distance between two signals when using the DTW is given by the formula: [3] T

dφ X  Y  

∑ d φx k φy kmk

k1







of the movement. By relaxing the endpoint constraints, the DTW is able to find the best match with the template pattern anywhere in a test pattern. This proved to be a lot more effective.

(3)

where φx and φy are two warping functions that relate i x and iy to a common time axis, k. That is:

4.3 Local Continuity Constraints

mk is a path weighting coefficient. M φ is a path normalisation factor and φ x , φy are chosen to minimise d φ [3].

Local continuity constraints are specified as the set of possible incremental path changes. Various local continuity constraints have been suggested[3]. Although these are suitable for speech recognition, they are not suitable for the recognition of movement patterns. This is because they only work for signals that are approximately the same length. Since people walk at different speeds, there can be a vast difference in the length of the signals that need to be compared. For this reason, three different local constraint paths were developed:

Figure 4: The optimal alignment of two patterns as found by the DTW

Figure 5: Local Constraints

ix  φx k

k  1 2  T

(4)

iy  φy k

k  1 2  T

(5)

The amount of warping that takes place between the two signals should be controlled if the output is to be meaningful. For this reason a number of constraints are usually made when using the DTW.

4.4 Slope Weighting

4.1

Various slope weights can be used to set difference “preference levels” for the various paths that have been defined as local continuity constraints. For this work a symmetric slope weighting was used with all values set to one.

Monotonicity Constraint

Perhaps the most fundamental of the constraints applied to the DTW is the monotonicity constraint. This constraint maintains temporal order while the match is being done by not allowing the warping path to move backwards in time.

4.5 Global Path Constraints

4.2

The local continuity constraints can exclude certain points from the set of possible solutions. Graphically, these constraints represent a parallelogram in the warping path domain. This is shown in Figure 6. Further constraints can be placed on the warping domain as desired.

Endpoint Constraints

The endpoint constraint specifies that the first and last datapoints of the two signals must match up with each other. These constraints are relaxed in order to make the matching of patterns more accurate. This is because the segmentation is not guaranteed to find the exact start and endpoints

Figure 6: Warping domain with global path constraints present

5

Clustering

Due to the nature of the data, most standard clustering techniques can not be successfully implemented due to the following reasons:

 A distance measure is not defined for all sets of two movements. This is because if one movement is greater than four times the length of the other one then the DTW will fail. Many of the standard clustering techniques require this information to achieve a good result.  Due to the relaxing of the endpoint constraints, the function DTW a b where a and b are the two movements being compared, is not necessarily the same as DTW b a.  The optimal number of clusters is not known. This information is required for a large number of clustering techniques. A modified version of the Batchelor and Wilkins’ algorithm (Maximum Distance Algorithm) [2] is used to cluster the movements. This algorithm is used as it is based on the furthest neighbour clustering technique. This means that the resulting clusters will form naturally distinct “clumps”[1]. Due to the nature of the data it is expected that the natural clusters will form compact zones in n-dimensional space. The modification to the Batchelor and Wilkins’ algorithm is the replacement of their termination criterion with one based on the minimum distance between items in a cluster. This value is determined heuristically.

Figure 7: Flow Diagram of Clustering Algorithm

6 Results A video of a person walking a variety of preset patterns was recorded and the location of the person (in terms of x and y co-ordinates) determined for each frame. This was done with a frame rate of approximately 10 fps. This “location” data was then pre-processed as described earlier. After pre-processing, the data was clustered using the modified Batchelor and Wilkins’ clustering algorithm. The output from the segmentation algorithm can be seen in Figure 8. The sets of movement patterns that have been detected after the clustering algorithm has been run is shown in Figure 9. From these figures, it is clear that the clustering algorithm has successfully grouped together movements which are similar. At the moment, not enough test sequences have been suitably analysed to be able to quantify the results. However, it can be clearly seen that the results are very promising.

7

Conclusions

After the application of the pre-processing stage, the clustering stage successfully separates the individual movements into clusters of movements that are similar to one another. The clustering technique uses the Dynamic Time Warp for a distance measure between the two movements being compared. This is necessary as they may be of a different length. Preliminary results look promising, but further testing on more data sets is required in order to quantify the results.

Acknowledgements This research was supported by the National Research Foundation and the Machine Intelligence Research Programme at Debtech, a division of De Beers Consolidated Mines Ltd.

References [1] Cluster analysis, Last Accessed 28 August 2001. http://www.statsoftinc.com/textbook/stcluan.html. [2] S. Bow. Pattern Recognition and Image Preprocessing, chapter 5. Marcel Dekker Inc., 1992. [3] Biing-Hwang Juang Lawrence Rabiner. Fundamentals of Speech Recognition, pages 201–241. Prentice-Hall Inc., 1993.

Figure 8: Output from the segmentation: A datastream of the person’s x and y co-ordinates as they walk about the office.

Figure 9: Output from the clustering algorithm. Sets of movements that have are similar to one another.

The Automatic Detection of Patterns In People's ...

Time Warping as a distance measure between individual movements. 1 Introduction .... based on the minimum distance between items in a cluster. This value is ...

738KB Sizes 9 Downloads 225 Views

Recommend Documents

Detection-Based ASR in the Automatic Speech Attribute ...
School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA1. Department of Computer ... As a result, a wide body of expert knowledge in ... Specifically, we present methods of detector design in the Auto-.

Improving Automatic Detection of Defects in Castings by ... - IEEE Xplore
internal defects in the casting. Index Terms—Castings, defects, image processing, wavelet transform, X-ray inspection. I. INTRODUCTION. NONDESTRUCTIVE ...

A Methodology For The Automatic Detection Of ...
(lengthened syllables), it should not be used as training data for our purposes. Another example is ... silence (from the auto. align.) For this manual annotation, ...

Active Contour Detection of Linear Patterns in ...
tour algorithm for the detection of linear patterns within remote sensing and vibration data. The proposed technique uses an alternative energy force, overcom-.

DETECTION OF REPETITIVE PATTERNS IN NEAR ...
of computer vision, computer graphics, and medical imaging. ... In computer graphics, Liu et al. .... The magnitude of LAp varies in proportion to the dissimilar-.

abrupt change detection in automatic disturbance ...
155. 9.6.3. Citrix®. ...... client computing etc) for the ASP implementation are discussed in details. We also discuss the utilisation of the ..... to create the software package for the automatic disturbance recognition and analy- sis. 2.6 Overview

Pattern recognition techniques for automatic detection of ... - CiteSeerX
Computer-aided diagnosis;. Machine learning. Summary We have employed two pattern recognition methods used commonly for face recognition in order to analyse digital mammograms. ..... should have values near 1.0 on the main diagonal,. i.e., for true .

AUTOMATIC PITCH ACCENT DETECTION USING ...
CRF model has the advantages of modeling the relations of the sequential labels and is able to retain the long distance dependency informa- tion. Although ..... ECS-95-001,. Bonston University, SRI International, MIT, 1995. [8] R.-E. Fan, P.-H. Chen,

Cheap 3D Modulator Automatic Synchronization Signal Detection ...
Cheap 3D Modulator Automatic Synchronization Signal ... arized 3D System Free Shipping & Wholesale Price.pdf. Cheap 3D Modulator Automatic ...

Automatic Gaze-based User-independent Detection of ...
pseudorandom prompts from an iPhone app and discovered that people ... the 2014 User Modeling, Adaptation, and Personalization conference (Bixler & D'Mello ..... 6 the differences in the tasks and methodologies. Grandchamp et al. (2014) ...

Automatic detection of learning-centered affective ...
sensitive interfaces for educational software in classroom environments are discussed. ... Learning from intelligent educational interfaces elicits frequent affective ...... with technology. Journal of ... Springer,. Berlin Heidelberg, 1994, 171–18

Automatic Gaze-based User-independent Detection of ...
in support of an eye-mind link posits that there could be a link between external ..... caught method (discussed above) while users read texts on a computer screen. ...... 364–365. International Educational. Data Mining Society (2013)Dong, ...

Automatic Detection of Bike-riders without Helmet using ...
Email: {cs11b15m000001, cs14resch11003, ckm}@iith.ac.in. Abstract—In this paper, we propose an approach for automatic detection of bike-riders without ...

Pattern recognition techniques for automatic detection of ... - CiteSeerX
Moreover, it is com- mon for a cluster of micro–calcifications to reveal morphological features that cannot be clearly clas- sified as being benign or malignant. Other important breast .... mammography will be created because of the data deluge to

Automatic Detection of Cars in Real Roads using Haar ...
feature application to a rectangular region positioned at ,x y .... Opencv [7] provides a tool for cascade performance testing. The tool applies the cascade to all ...

Automatic Detection of Region-Mura Defect in TFT-LCD
Machine vision, image segmentation, regression diagnostics, industrial inspection, visual perception. 1. Introduction. Recently, TFT-LCD (Thin Film Transistor ...

Automatic Excitement-Level Detection for Sports ...
curate speech background is necessary for good performance. Ac- curate segmentation .... an automatic fashion using WaveSurfer and in-house tools: fun- damental frequency F0 ... were used for training and two games for testing. The overall ex- .... a

Manifesto, Baku Congress of the Peoples of the East
The Congress of representatives of the Peoples of the East calls on these peoples to realise such unity, which is ... to decide who was to rule over the countries of Asia and whose slaves the peoples of the East should be. ... miserable pittance that

Manifesto, Baku Congress of the Peoples of the East
The peoples of the East have long stagnated in the darkness of ignorance under the despotic yoke of their own tyrant rulers, and under that of foreign capitalist conquerors. But the roar of the world-wide conflict, and the thunder of the Russian work

Automatic Circle Detection on Images with an ... - Ajith Abraham
algorithm is based on a recently developed swarm-intelligence technique, well ... I.2.8 [Artificial Intelligence]: Problem Solving, Control Methods, and Search ...

Towards automatic skill evaluation: Detection and ...
1Engineering Research Center for Computer-Integrated Surgical Systems and Technology and 2Center for .... sition is the streaming 72 data-points per time unit.

Automatic Circle Detection on Images with an Adaptive ...
test circle approximates the actual edge-circle, the lesser becomes the value of this ... otherwise, or republish, to post on servers or to redistribute to lists, requires prior ... performance of our ABFOA based algorithm with other evolutionary ...