GRIP
Image Processing Research Group
Video Forgery Detection and Localization based on 3D PatchMatch L. D’Amiano, D. Cozzolino, G. Poggi, L. Verdoliva University Federico II of Naples, ITALY
WeMuV – Torino 29 June 2015
From Image Forgery (a long tradition) Improving a dictator image Benito Mussolini 1942
Insurance Fraud “66 on trial” La Repubblica 2012
... to Video Forgery (growing)
The Varoufakis web case 2015
A copy-move video forgery FRAME:
65
66
67
68
69
250
Original
...
Fake
...
251
252
253
https://sites.google.com/site/rewindpolimi/downloads/datasets/videocopy-move-forgeries-dataset
254
About this work We propose an effective video copy-move algorithm It extends our image copy-move forgery detector (CMFD) Dense-field approach Rotation-invariant features Fast matching and post-processing
Numerical assessment on a publicly available database A real-world case
D.Cozzolino, G.Poggi, and L.Verdoliva, “Efficient dense-field copy-move forgery detection”, IEEE Transactions on Information Forensics and Security, to appear.
Main steps in CMFD Feature Extraction
List of features
Matching
[13,11, 32, ….,103] Patch
Feature vector
PostProcessing
Keypoint-vs-dense CMFD Keypoint-based
Dense-field
Only high-entropy points
Regular grid of points
FAST Loses «occlusive» copy-moves
SLOW Catches all types of copy-moves
Main qualifying points Feature Extraction
Rotation-invariant features: • Zernike moments on a polar grid
Matching
Fast matching with smooth matching field • Modified PatchMatch
PostProcessing
Fast and accurate post-processing • Dense linear fitting
PatchMatch A stochastic iterative fast matching algorithm Alternates propagation and random search steps Quick convergence to an approximate (but accurate) and smooth NN field
C. Barnes et al., “PatchMatch: a randomized correspondence algorithm for structural image editing,” ACM Transactions on Graphics, 2009.
Propagation in PatchMatch top neighbour
current match of top neighbour
left neighbour
candidates
current match of left neighbour
Zero-order predictors are used, good for pw-constant fields
Modified PatchMatch Copy-move with rotation and resizing induce linear (not constant) offset fields Modified propagation (includes 1st-order predictors) Allows working of compact features
Propagation in modified PatchMatch top neighbours current matches of top neighbours
left neighbours
candidates
current matches of left neighbours
First-order predictors allow one to deal with pw-linear fields
Some sample results
original image
forgery
[Christlein2012]
[Cozzolino2015]
TP V. Christlein et al., “An Evaluation of Popular Copy-Move Forgery Detection Approaches,” IEEE TIFS 2012.
TN
FN
FP
1
1
0.8
0.8
0.8
0.6 0.4 proposed Christlein2012 Bravo2011 Amerini2013
0.2
0 NC 100 90 80 70 60 50 40 30 20 JPEG-compression (QF)
avr F-measure
1
avr F-measure
avr F-measure
Comparison with state of the art 0.6 0.4 0.2
proposed Christlein2012 Bravo2011 Amerini2013
0 0.5 0.650.8 0.93 1 1.071.2 1.6 Resizing (Scale)
0.6 0.4 0.2
2
0 0° 4° 10° 30° 60° 90° Rotation (Angle)
More accurate than keypoint-based methods • Faster than comparable dense-based methods •
proposed Christlein2012 Bravo2011 Amerini2013
180°
Adaptation to video Same structure as image CMFD algorithm 3D patches Include temporal predictors in PatchMatch Temporal guard interval Post-processing includes the temporal direction
PatchMatch 3D
PatchMatch 3D current match of F.1 left neighbour
F.2
current match of top neighbour
F.3
top neighbour
F.4
F.5
F.6 candidates
Past neighbour
F.7 current match of past neighbour
left neighbour
F.8
F.N
Rewind dataset Expands the University of Surrey SULFA database 10 videos, approximately 300 frames, 320x240 pixels/f. Forgeries are created by copy-moving a portion of the video Copy-moves involve from 33 to 211 frames, with copied regions going from 630 (very small) to 30680 pixels Only rigid copy-moves are performed We add copy-moves with rotation / resizing for video #3
Detection Performance Uncompressed
QP=10
QP=20
CPU
TPR*
FPR*
TPR*
FPR*
TPR*
FPR*
(s)
Proposed
9
0
9
0
9
0
526
PM - RGB
9
0
9
0
9
0
800
Bestagini
10
3
9
1
8
1
154
(*) all numbers to be divided by 10, the number of videos
M. Bleyer, C. Rhemann, and C. Rother, “Patchmatch stereo - stereo matching with slanted support windows,” in British Machine Vision Conference, 2011. P. Bestagini, S. Milani, M. Tagliasacchi, and S. Tubaro, “Local tampering detection in video sequences,” in IEEE International Workshop on Multimedia Signal Processing, 2013.
Video #7 – wrong matches FRAME:
5
6
7
8
9
37
VIDEO FAKE
...
MATCHING
...
38
39
40
41
Video #7 – correct matches FRAME:
VIDEO FAKE
MATCHING SHOULD BE
5
6
7
8
9
37
...
...
38
39
40
41
Localization Performance Uncompressed
QP=10
QP=20
CPU
TPR
FPR
TPR
FPR
TPR
FPR
(s)
Proposed
48.77
0.02
46.10
0.12
29.41
0.06
526
PM - RGB
51.81
0.00
42.88
0.02
21.09
0.03
800
Localization Performance on video #3 Uncompressed TPR
FPR
QP=10 TPR
QP=20
FPR
TPR
FPR
Rigid copy-move Proposed
44.45
0.00
43.97
0.00
42.46
0.00
PM - RGB
42.88
0.00
32.10
0.00
42.09
0.15
Copy-move with rotation Proposed
36.64
0.00
37.47
0.00
25.26
0.00
PM - RGB
1.96
0.00
3.40
0.00
0.00
0.05
Copy-move with resizing Proposed
34.69
0.00
34.82
0.00
34.12
0.00
PM - RGB
38.84
0.00
39.34
0.00
37.06
0.05
Example masks for video #3: rotated CM
TP
TN
FN
FP
A real-world case zdf-neo video
YouTube video
Comparison of videos
zdf-neo video
YouTube video
Difference They differ in only 80 frames
Results of PM-based technique
zdf-neo video
YouTube video
Differences between matched frames
Differences between aligned frames
The verdict zdf-neo video
YouTube video
YouTube video (previous frames)
A real-world case zdf-neo video
YouTube video #1
Video YouTube #2
Future work Use 3d descriptors Exploit motion information (e.g. objects tracking) speed-up search − Multi-resolution version − Non-random fast initialization
Create a richer database of copy-move video forgeries
