Kaggle Competition Google Cloud & YouTube-8M Video Understanding Challenge 5th place solution

Deep Learning Methods for Efficient Large Scale Video Labeling M. Pękalski, X. Pan and M. Skalic CVPR'17 Workshop on YouTube-8M Large-Scale Video Understanding Honolulu, HI July 26, 2017

Agenda 1.

Team You8M




Data Augmentation and Feature Enginnering


Training Methods


Key to Success

Team You8M

Marcin Pękalski: Master’s degree in Mathematics and Master’s degree in International Economics. Currently a data scientist at Kambi, a B2B sportsbook provider. Stockholm, Sweden. Miha Skalic: Holds Master's degree in Biotechnology. Now working towards a PhD in Biomedicine at University Pompeu Fabra . Barcelona, Spain. Xingguo E. Pan: Trained as a physicist. After getting a PhD, he went into the financial industry. Chicago, USA.

Frame level Models

Bi-directional LSTM • • • •

Dynamic length RNN Two models running in oposite directions MoE with two experts applied to last layer 6 epochs took 3 days

Bi-directional GRU • • •

Similar structure as LSTM Layer sizes 625x2, 1250 Trained with 5 folds

Bi-directional LSTM model

Video level models

MoNN: Mixture of neural network experts input

MoNN3Lw 3 FC layers: • 2305x8 • 2305x1 • 2305x3

gate activations

fully connected fully connected

3 experts Features • mean_rgb/audio • std_rgb/audio • num_frames 5 epochs took 9 hours on GeForce GTX 1080ti

fully connected expert activation final predictions

# layers




single checkpoint single fold

models of same family, checkpoints and folds ensemble










GAP All models ensemble

0.8419 Best score 0.8424

Model Correlation Models MoNN

• •

All models are implemented with Tensorflow. GAP scores are from private leaderboard.









0.98 1.0

Data augmentation & Feature engineering

Video splitting: • •

Split one video into two halves training samples: 6.3 million => 18.9 million

Video Level features: • mean-rgb/audio • std-rgb/audio • 3rd moments of rgb/audio • num_frames • Moments of entire video • top/bottom 5 per feature dimension

Training Being able to see the out-of-sample GAP greatly facilitated our management of the training process and of model and feature selection. Example: Both yellow and red models can well fit the train data, but yellow model peaked around 20 K steps on test data.

Monitor out-of-sample performance while training


Dropout Truncated Labels Batch, Global and No Normalization

Training Methods that we experimented with and that helped to deepen our understanding of the data and models.

Exponential Moving Average Training on Folds Boosting Network

Key to Success

We tried a lot of different architectures (wide and narrow, deep and shallow, dropouts, ema, boosting, etc.). Combined video and frame level models. Tried different ensemble weighting techniques, utilizing individual model’s GAP score and the correlation between models. Generated different features: pre-assembled them into data for efficient training process. Data augmentation. Training on folds of data and averaging the results over folds and checkpoints.

Resources & Acknowledgement s

arXiv Paper: • https://arxiv.org/abs/1706.04572 Source Code: • https://github.com/mpekalski/Y8M

©Apache License, version 2

Acknowledgements: •

The authors would like to thank the Computational Bio-physics group at University Pompeu Fabra for letting us use their GPU computational resources. We would also like to thank Jose Jimenez for valuable discussions and feedback.

Deep Learning Methods for Efficient Large ... - Research at Google

Jul 26, 2017 - Google Cloud & YouTube-8M Video. Understanding Challenge ... GAP scores are from private leaderboard. Models. MoNN. LSTM GRU.

311KB Sizes 4 Downloads 302 Views

Recommend Documents

Cost-Efficient Dragonfly Topology for Large ... - Research at Google
Evolving technology and increasing pin-bandwidth motivate the use of high-radix .... cost comparison of the dragonfly topology to alternative topologies using a detailed cost model. .... energy (cooling) cost within the first 3 years of purchase [8].

cost-efficient dragonfly topology for large-scale ... - Research at Google
radix or degree increases, hop count and hence header ... 1. 10. 100. 1,000. 10,000. 1985 1990 1995 2000 2005 2010. Year .... IEEE CS Press, 2006, pp. 16-28.

Learning with Deep Cascades - Research at Google
based on feature monomials of degree k, or polynomial functions of degree k, ... on finding the best trade-off between computational cost and classification accu-.

Efficient Large-Scale Distributed Training of ... - Research at Google
Training conditional maximum entropy models on massive data sets requires sig- ..... where we used the convexity of Lz'm and Lzm . It is not hard to see that BW .... a large cluster of commodity machines with a local shared disk space and a.

Efficient Spatial Sampling of Large ... - Research at Google
geographical databases, spatial sampling, maps, data visu- alization ...... fairness objective is typically best used along with another objective, e.g. ...... [2] Arcgis. http://www.esri.com/software/arcgis/index.html. ... Data Mining: Concepts and.

Large Scale Distributed Deep Networks - Research at Google
second point, we trained a large neural network of more than 1 billion parameters and .... rameter server service for an updated copy of its model parameters.

ral networks, deep learning, audio indexing. 1. INTRODUCTION. More than one billion people ... recognition technology can be an attractive and useful service.

Kernel Methods for Learning Languages - Research at Google
Dec 28, 2007 - its input labels, and further optimize the result with the application of the. 21 ... for providing hosting and guidance at the Hebrew University.

Large Scale Online Learning of Image Similarity ... - Research at Google
of OASIS learned similarity show that 35% of the ten nearest neighbors of a ..... the computer vision literature (Ojala et al., 2002, Takala et al., 2005), ...... Var10: bear, skyscraper, billiards, yo-yo, minotaur, roulette-wheel, hamburger, laptop-