The Feasibility of Supporting Large-Scale Live Streaming Applications with Dynamic Application End-Points
Kay Sripanidkulchai, Aditya Ganjam, Bruce Maggs*, and Hui Zhang Carnegie Mellon University
* and Akamai Technologies
Motivation • Ubiquitous Internet broadcast – Anyone can broadcast – Anyone can tune in
9/1/2004
[email protected]
2
Overlay multicast architectures
Router Source Application end-point 9/1/2004
[email protected]
3
Infrastructure-based architecture [Akamai]
+ Well-provisioned
9/1/2004
[email protected]
Router Source Application end-point Infrastructure server 4
Application end-point architecture [End System Multicast (ESM)]
+ Instantly deployable + Enables ubiquitous broadcast 9/1/2004
[email protected]
Router Source Application end-point 5
Waypoint architecture [ESM] W
+ Waypoints as insurance
9/1/2004
[email protected]
Router Source Application end-point W Waypoint 6
Sample ESM Broadcasts http://esm.cs.cmu.edu Event
9/1/2004
Duration (hours)
Unique Hosts
Peak Size
SIGCOMM ’02
25
338
83
SIGCOMM ’03
72
705
101
SOSP’03
24
401
56
DISC’03
16
30
20
Distinguished Lectures
11
400
80
AID Meeting
14
43
14
Buggy Race
24
85
44
Slashdot
24
1609
160
Grand Challenge
6
2005
280
[email protected]
7
Feasibility of supporting large-scale groups with an application end-point architecture? • Is the overlay stable enough despite dynamic participation? • Is there enough upstream bandwidth? • Are overlay structures efficient?
9/1/2004
[email protected]
8
Large-scale groups • Challenging to address these fundamental feasibility questions – Little knowledge of what large-scale live streaming is like
9/1/2004
[email protected]
9
Chicken and egg problem
Publishers with compelling content need proof that the system works.
9/1/2004
System has not attracted large-scale groups due to lack of compelling content.
[email protected]
10
The focus of this paper • Generate new insight on the feasibility of application end-point architectures for large scale broadcast • Our methodology to break the cycle – Analysis and simulation – Leverage an extensive set of real-world workloads from Akamai (infrastructure-based architecture)
9/1/2004
[email protected]
11
Talk outline • Akamai live streaming workload • With an application end-point architecture – Is the overlay stable enough despite dynamic participation? – Is there enough upstream bandwidth? – Are overlay structures efficient?
• Summary
9/1/2004
[email protected]
12
Measurements used in this study • Akamai live streaming traces – Trace format for a request [IP, Stream URL, Session start time, Session duration]
• Additional measurements collected – Hosts’ upstream bandwidth – Hosts’ GNP coordinates
9/1/2004
[email protected]
13
Akamai live streaming infrastructure
Source
A
A
A
A
A
A
Reflectors
9/1/2004
Edge servers
[email protected]
14
Extensive traces ~ 1,000,000 daily requests ~ 200,000 daily client IP addresses from over 200 countries ~ 1,000 daily streams ~ 1,000 edge servers ~ Everyday, over a 3-month period
9/1/2004
[email protected]
15
Request volume (daily)
Number of requests
Weekdays
Weekends
Missing logs
9/1/2004
[email protected]
16
Large-scale streams • Definitions – Large-scale: peak group size of over 1,000
• Characteristics of our traces [IMC ’04] – 660 large-scale streams – 71% audio vs. 7% video vs. 22 % unknown – 79% non-stop vs. 21% short duration
9/1/2004
[email protected]
17
Largest stream 75,000 x 250 kbps = 18 Gbps!
5pm 9/1/2004
6pm
7pm
8pm
9pm
[email protected]
10pm
11pm 12am 18
Talk outline • Akamai live streaming workload • With an application end-point architecture – Is the overlay stable enough despite dynamic participation? – Is there enough upstream bandwidth? – Are overlay structures efficient?
• Summary
9/1/2004
[email protected]
19
When is a tree stable? Not stable
More stable Stable nodes
X X
Interruptions
Time Ancestor leaves 9/1/2004
Less stable X nodes • Departing hosts have no descendants • Stable nodes at the top of the tree
[email protected]
20
Extreme group dynamics
15% stay longer than 30 minutes (heavy-tailed) 45% stay less than 2 minutes!
9/1/2004
[email protected]
21
Stability evaluation: simulation • Hosts construct an overlay amongst themselves using a single-tree protocol – Skeleton protocol of the one presented in the ESM Usenix ’04 paper • Findings are applicable to many protocols
– Goal: construct a stable tree • Parent selection is key
• Group dynamics from Akamai traces (join/leave) • Honor upstream bandwidth constraints – Assign degree based on bandwidth estimation
9/1/2004
[email protected]
22
Join
Join IP1 IP2 ...
9/1/2004
[email protected]
23
Probe and select parent
IP2 IP1
9/1/2004
[email protected]
IP1 IP2 ...
24
Probe and select parent
Parent selection algorithms • Oracle: pick a parent who will leave after me • Random • Minimum depth (select one out of 100 random) • Longest-first (select one out of 100 random) 9/1/2004
[email protected]
25
Parent leave
X Host leaves
9/1/2004
[email protected]
26
Parent leave
? Host leaves All descendants are disconnected
9/1/2004
[email protected]
27
Find new parent
Host leaves All descendants are disconnected All descendants probe to find new parents
9/1/2004
[email protected]
28
Stability metrics • Mean interval between ancestor change
• Number of descendants of a departing host
Interruptions
X Time Ancestor leaves
9/1/2004
[email protected]
29
Stability of largest stream
Longest-first Random Min depth Oracle: there is stability!
9/1/2004
[email protected]
30
Is longest-first giving poor predictions? Oracle, ~100% no descendants
Longest-first, 91%
Min depth, 82%
Random, 72%
9/1/2004
[email protected]
31
Percentage of sessions with interval between ancestor change < 5 minutes
Stability of 50 large-scale streams
9/1/2004
Longest-first
Random Min depth Oracle There is stability! Of the practical algorithms, min depth performs the best.
[email protected]
32
There is inherent stability • Given future knowledge, stable trees can be constructed • In many scenarios, practical algorithms can construct stable trees – Minimum depth is robust – Predicting stability (longest-first) is not always robust; when wrong, the penalty is severe
• Mechanisms to cope with interrupts are useful – Multiple trees (see paper for details)
9/1/2004
[email protected]
33
Talk outline • Akamai live streaming workload • With an application end-point architecture – Is the overlay stable enough despite dynamic participation? – Is there enough upstream bandwidth? – Are overlay structures efficient?
• Summary
9/1/2004
[email protected]
34
Is there enough upstream bandwidth to support all hosts? Video 300 kbps DSL Upstream bandwidth only 128 kbps
DSL
DSL Saturated tree
What if application end-points are all DSL?
9/1/2004
[email protected]
35
Metric: Resource index • Ratio of the supply to the demand of upstream bandwidth Resource index == 1 means the system is saturated • Resource index == 2 means the system can support two times the current members in the system
9/1/2004
Resource Index:
[email protected]
(3+5)/3 = 2.7
36
Large-scale video streams
A few streams are in trouble or close. 1/3 of the streams are in trouble.
Most streams have sufficient upstream bandwidth. 9/1/2004
[email protected]
37
Talk outline • Akamai live streaming workload • With an application end-point architecture – Is the overlay stable enough despite dynamic participation? – Is there enough upstream bandwidth? – Are overlay structures efficient?
• Summary
9/1/2004
[email protected]
38
Relative Delay Penalty (RDP) • How well does the overlay structure match the underlying network topology? RDP = Overlay distance Direct unicast distance US
US 20ms
50ms
US 50ms Europe
US50ms Europe
Results are more promising than previous studies using synthetic workloads and topologies. 9/1/2004
[email protected]
39
Summary • Indications of the feasibility of application end-point architectures – The overlay can be stable despite dynamic participation – There often is enough upstream bandwidth – Overlay structures can be efficient
• Our findings can be generalized to other protocols • Future work: Validate through real deployment – On-demand use of waypoints in End System Multicast – Attract large groups
Thank you! 9/1/2004
[email protected]
40