The Many Ways Your Monitoring Is Lying To You Sebastian Kirsch SRECon16 Europe, Dublin, July 11.-13. 2016
The Map Is Not The Territory
≠
Time-Series Based Monitoring
environment
server 1
environment
server 2
requests: 8
requests: 14
requests: server 1: … 8 12 16 server 2: … 14 18 22 server 3: … 5 9 13 server 4: … 16 20 24 sum(requests): … 43 59 75 rate(requests): … 16 16 16
monitoring
aggregator
requests: 5 environment
environment
server 3
server 4
Time-Series Based Monitoring
requests: 16
other servers
other data sources
dashboard
Lies of Omission
job:request:rate = sum(rate(task:requests:total))
server 1
requests: 0
0
1
2
3
4
1
1
1
1
rate
1 server 2
server 3
server 4
requests: 0
requests: 0
requests: 0
1
1
1
0
1
2
3
4
1
1
1
1
0
1
2
3
4
1
1
1
1
0
1
2
3
4
1
1
1
1
4
4
4
4
sum
Lies of Omission
Lies of Omission
job:request:rate = sum(rate(task:requests:total))
server 1
requests: 0
0
1
X 3
4
1
X 2
1
rate
1 server 2
server 3
server 4
requests: 0
requests: 0
requests: 0
X 2
1
0
1
2
3
4
1
1
1
1
0
1
2
3
4
1
1
1
1
0
1
2
3
4
1
1
1
1
4
3
5
4
sum
Lies of Omission
Lies of Granularity
Lies of Granularity
Lies of Granularity
Lies of Granularity
Lies of Granularity
Lies of Granularity
Lies of Perspective
monitoring
Lies of Perspective
requests: 0 server-errors: 0
requests: 1 server-errors: 1
server
client
client
server
monitoring requests: 0 errors: 0
requests: 4 errors: 0
requests: 4 10 errors: 0 3
Lies of Perspective
Lying through Alignment
job:memory:mean = sum(task:memory) / job:num_tasks_up
Lying through Alignment
Lies of Presentation
Lies of Presentation
Lies of Presentation
Lies of Presentation
Lies of Presentation
Lying through Selection
top(5, task:errors:rate)
Lies through Selection
Lies, Damn Lies, and Misuse of Statistics
Lies, Damn Lies, and Misuse of Statistics
95th percentile ≈ 130ms
Lies, Damn Lies, and Misuse of Statistics
Lies, Damn Lies, and Misuse of Statistics
95th percentile ≈ 130ms
Lies, Damn Lies, and Misuse of Statistics
95th percentile ≈ 150ms
Lies, Damn Lies, and Misuse of Statistics
95th percentile ≈ 180ms
Lies, Damn Lies, and Misuse of Statistics
95th percentile ≈ 250ms
Lies, Damn Lies, and Misuse of Statistics
Summary
Acknowledgments Alexander Jolk, Etienne Pierre, Jules Anderson, Gráinne Sheerin, Jukka Laurila, Mike Han, Pawel Stradomski, Ralf Wildenhues
Q&A
![[PDF BOOK] 12 Ways Your Phone Is Changing You Full ...](https://p.pdfkul.com/img/300x300/pdf-book-12-ways-your-phone-is-changing-you-full-_5a2cc3da1723dd4724ba72d9.jpg)
