Network Scheduling in Software-defined Environments Tal Mizrahi*◊, Yoram Moses*

*Technion – Israel Institute of Technology ◊Marvell

SDN Research Group, IRTF IETF 96, Berlin, Germany July 2016

Outline • • • •

Background Use cases Accurate scheduling Conclusion

The TimedSDN Project

2

Outline • • • •

Background Use cases Accurate scheduling Conclusion

The TimedSDN Project

3

Software Defined & Programmable Networks • Centralized • Flexible • Dynamic

Controller / NMS

switch

switch

switch

vSwitch

switch

The TimedSDN Project

4

at a Glance Controller A protocol allowing the controller to schedule network updates

Synchronized clocks

Switches The TimedSDN Project

5

Outline • • • •

Background Use cases Accurate scheduling Conclusion

The TimedSDN Project

6

Using Time in Network-managed Environments Coordinated Configuration Update

Coordinated Snapshot Client

Client

T T Server 1 Server 2

...

T T Server 1 Server 2

T Server n

The TimedSDN Project

...

T Server n

7

Using Time to Coordinate

Updates

OpenFlow

…

I2RS

Simultaneous

Timed sequence

Switches invoke update at the same time

Switches invoke update according to a scheduled sequence The TimedSDN Project

8

Time is Optimal for Flow Swaps

[INFOCOM ‘16]

A key benefit of SDN: Dynamic path allocation based on network load

old new

o1

...

controller o2

The TimedSDN Project

9

Time is Optimal for Flow Swaps

[INFOCOM ‘16]

A key benefit of SDN: Dynamic path allocation based on network load If O1 , O2 are not updated at the same time…

old new

Temporary congestion.

o1

...

controller o2

O2 is updated first. The TimedSDN Project

10

Time is Optimal for Flow Swaps

[INFOCOM ‘16]

A key benefit of SDN: Dynamic path allocation based on network load

old new

If O1 , O2 are updated at the same time: No congestion!

o1

...

controller o2

The TimedSDN Project

11

Consistent Path Update before after

0. 1. 2. 3.

The ‘before’ configuration. Controller updates S1. X Controller updates S2. Controller updates S3.

S3

S4

S3

S4

S2

S1

S2

S1

0

wait…

S3

S4

S2

S1

1

2 The TimedSDN Project

wait…

S3

S4

S2

S1

S3

S4

S2

S1

3 12

Simultaneous Updates? En-route packets run into a ‘black hole’. Not consistent!

S3

S4

S3

S4

S3

S4

S3

S4

S2

S1

S2

S1

S2

S1

S2

S1

1

2 The TimedSDN Project

3 13

Timed Multi-phase Consistent Updates [SOSR ‘15] - The controller sends timed update messages to S1, S2, S3. - Scheduled updates occur at times T1, T2, T3.

Controller does not need to wait between steps!

S3

S4

S3

S4

S3

S4

S3

S4

S2

S1

S2

S1

S2

S1

S2

S1

T1

The TimedSDN Project

T2

T3

14

Using Timestamps instead of ingress switch

[SWFAN ‘16] egress switch

Timestamp can be used in processing of intermediate devices.

VNF

X

External Network

vSwitch

• Delay measurement.

• Passive performance monitoring.

Timestamped Domain

https://tools.ietf.org/html/draft-ietf-ippm-alt-mark

Timestamp-based marking. Timestamp is • Pushed by ingress switch. • Removed by egress switch.

• Policy / path decision criterion: “Do action A if Timestamp≥T0” The TimedSDN Project

15

Outline • • • •

Background Use cases Accurate scheduling Conclusion

The TimedSDN Project

16

Accurate Synchronization: It’s Already Here Precision Time Protocol (PTP) [IEEE 1588 2008] ~1μsec accuracy

Mobile backhaul

Power substations

Industrial automation

Financial applications

China Mobile: over 1,000,000 PTP-enabled base stations https://mailarchive.ietf.org/arch/attach/tictoc/pdfsY1ADO.pdf The TimedSDN Project

17

TIMEFLIP: Timestamp-based TCAM Lookup A Timestamp is attached to packet’s metadata by the switch

conventional

TimeFlip

Network Device

Network Device

TCAM packet’s search key

su, …, s1

T action

s, packet’s u search key

TCAM

…, s1 T³T0

action

The timestamp is used in the TCAM key – time range. COTS switches can synchronize clocks very accurately ~ 1 μsec. [Using IEEE 1588 Precision Time Protocol (PTP) or GPS] The TimedSDN Project

18

The Cost of TIMEFLIP: 1 Bit / 1 Entry Network Device

T

s, search u key

TCAM

…, s1 T³T0

Large # of bits in timestamp field? Large # of entries per TimeFlip rule?

action

No !!

• Theorem: if 𝑇𝑂𝐿 ≥ 2 𝑙𝑜𝑔2(∆) , then the every timestamp range requires as little as: – 1 bit in the timestamp field. – 1 TCAM entry. The TimedSDN Project

19

TIMEFLIP: Would it Work on a Real Switch? Yes !! Microbenchmark: TimeFlip was tested on a Marvell 98DX4251 with a sub-microsecond accuracy.

The TimedSDN Project

20

Outline • • • •

Background Use cases Accurate scheduling Conclusion

The TimedSDN Project

21

The Why do we need time in SDN? Scheduling protocols

Accurate scheduling methods SDN Clock synchronization

Project

Flow Swaps TIME4

Multi-phase updates

Data Plane Timestamping

Network-managed coordination

[INFOCOM, ‘16]

[SOSR, ‘15]

[SWFAN, ‘16]

[NOMS, ‘16]

OpenFlow Scheduled Bundles

NETCONF Time Capability

[INFOCOM ’16]

[NOMS ’16]

TIMEFLIP

ONECLOCK

[INFOCOM ‘15]

[NOMS ‘16]

REVERSEPTP [HotSDN ‘14, ISPCS ‘14] The TimedSDN Project

22

The

Scheduling protocols

Published!

Project – Practical Aspects

OpenFlow Scheduled Bundles

NETCONF Time Capability

[INFOCOM ’16]

[NOMS ’16]

OpenFlow 1.5

RFC 7758

The TimedSDN Project

23

The

Project – Practical Aspects

Open Source Prototypes OpenFlow OpenFlow Scheduled Scheduled Bundles Bundles

NETCONF Time Capability

ReversePTP

https://github.com/TimedSDN

The TimedSDN Project

24

at a Glance Controller A protocol allowing the controller to schedule network updates

Synchronized clocks

Switches The TimedSDN Project

25

The

Project – Future Directions

Timed Updates

NETCONF

Data plane timestamping

SFC

Time as a network programming abstraction Implementation and wide-scale experiments

RFC 7758 ++

https://tools.ietf.org/html/draf t-browne-sfc-nsh-timestamp-01

P4

I2RS

…

NVO3

…

…

We would be happy to collaborate with vendors / operators who are interested!

26

Thanks! The Timed

Project

http://tx.technion.ac.il/~dew/TimedSDN.html The TimedSDN Project

A Closer Look at TIMEFLIP

Further details can be found in: • TimeFlip paper (http://tx.technion.ac.il/~dew/TimeFlipINFOCOM.pdf) • TimeFlip presentation (http://tx.technion.ac.il/~dew/TimeFlipInfocomPres.pdf) • Next few slides… The TimedSDN Project

SDN Switches controller

switch

switch

switch

switch Switch

incoming packet

TCAM1

TCAM2

...

The TimedSDN Project

TCAMn

outgoing packet

29

TCAM: Ternary Content Addressable Memory • Memory for quick searching Network Device

TCAM packet’s search key

su, …, s1

action

• Top-down search: first match “wins” • Ternary: 0 / 1 / * • * = don’t care The TimedSDN Project

30

TIMEFLIP: Timestamp-based TCAM Lookup A Timestamp is attached to packet’s metadata by the switch

conventional

TimeFlip

Network Device

Network Device

TCAM packet’s search key

su, …, s1

T action

s, packet’s u search key

TCAM

…, s1 T³T0

action

The timestamp is used in the TCAM key – time range. COTS switches can synchronize clocks very accurately ~ 1 μsec. [Using IEEE 1588 Precision Time Protocol (PTP) or GPS] The TimedSDN Project

31

Time-based Updates using TIMEFLIP TimeFlip: switch accurately starts using ‘new’ at T0.

Network Device TCAM T

s , …, s1 T³T0 …, s1 * ...*

search u su, key

o1

...

‘new’ ‘old’

old new

controller o2

The TimedSDN Project

32

Example: Timestamp Format Network Time Protocol (NTP) timestamp format: Time.Sec

Time.Frac

Seconds

Second Fraction

The TimedSDN Project

33

Example: TIMEFLIP in Practice Network Device

Goal: schedule an update to be performed at 2016-07-22 11:41:11 (at the beginning of the second)

Flow Table

timestamp field2 field3 field4 … packet header

Procedure: (steps also appear in the figure) Step 1 : install a TimeFlip with ‘1’ in the lsbit of the seconds field, and ‘don’t care’ in the rest. Step 2 : the update becomes effective exactly at the turn of the second. Step 3 : make the TCAM rule permanent by writing ‘don’t care’ to the lsbit of the timestamp field. The TimedSDN Project

*…*

Time.Sec

1

action

*…*

Time.Frac

Periodic range 1 2 3 0

... timestamp value

1 second 34

References [1]

T. Mizrahi, Y. Moses, "Software Defined Networks: It's About Time", IEEE INFOCOM, 2016.

[2]

T. Mizrahi, Y. Moses, "OneClock to Rule Them All: Using Time in Networked Applications", IEEE/IFIP Network Operations and Management Symposium (NOMS), 2016.

[3]

T. Mizrahi, E. Saat, Y. Moses, "Timed Consistent Network Updates", ACM SIGCOMM Symposium on SDN Research (SOSR), 2015.

[4]

T. Mizrahi, E. Saat, Y. Moses, "Timed Consistent Network Updates in Software Defined Networks", IEEE/ACM Transactions on Networking (ToN), 2016.

[5]

T. Mizrahi, O. Rottenstreich, Y. Moses, "TimeFlip: Scheduling Network Updates with Timestamp-based TCAM Ranges", IEEE INFOCOM, 2015.

[6]

T. Mizrahi, Y. Moses, "Time-based Updates in Software Defined Networks", workshop on hot topics in software defined networks (HotSDN), 2013.

[7]

T. Mizrahi, Y. Moses, "On the Necessity of Time-based Updates in SDN", Open Networking Summit (ONS), 2014.

[8]

T. Mizrahi, Y. Moses "Using ReversePTP to Distribute Time in Software Defined Networks", International IEEE Symposium on Precision Clock Synchronization for Measurement, Control and Communication, (ISPCS), 2014.

[9]

T. Mizrahi, Y. Moses, "ReversePTP: A Software Defined Networking Approach to Clock Synchronization", workshop on hot topics in software defined networks (HotSDN), 2014.

[10]

T. Mizrahi and Y. Moses, "ReversePTP: A clock synchronization scheme for software defined networks", International Journal of Network Management (IJNM), 2016.

[11]

T. Mizrahi, Y. Moses, "Time4: Time for SDN", arXiv preprint arXiv:1505.03421, 2016.

[12]

T. Mizrahi, Y. Moses, "The Case for Data Plane Timestamping in SDN", IEEE INFOCOM Workshop on Software-Driven Flexible and Agile Networking (SWFAN), 2016.

[13]

T. Mizrahi, Y. Moses, "Time Capability in NETCONF", RFC 7758, 2016.

[14]

Open Networking Foundation, OpenFlow switch specification, Version 1.5.0, 2015.

[15]

Open Networking Foundation, OpenFlow extensions 1.3.x package 2, 2015.

The TimedSDN Project

35