The Case for Data Plane Timestamping in SDN Tal Mizrahi, Yoram Moses Technion – Israel Institute of Technology

IEEE INFOCOM Workshop on Software-Driven Flexible and Agile Networking (SWFAN) April 2016

Outline • • • • •

Background Timestamp ranges Use cases Discussion Conclusion

Data Plane Timestamping in SDN

2

Outline • • • • •

Background Timestamp ranges Use cases Discussion Conclusion

Data Plane Timestamping in SDN

3

The Timed

Project Controller

A protocol allowing the controller to schedule network updates

Synchronized clocks

Switches [INFOCOM 2016] T. Mizrahi, Y. Moses, “Software Defined Networks: It’s About Time”. Data Plane Timestamping in SDN

4

Using Timestamps instead of The advantages of timestamping are well-known, e.g., [Lamport ‘78]. Current work: Data Plane Timestamping (DPT) in SDN. ingress switch

Timestamp is • Pushed by ingress switch. • Removed by egress switch.

egress switch

VNF

X

External Network

vSwitch

Timestamp can be used in processing of intermediate devices.

Timestamped Domain

Data Plane Timestamping in SDN

5

DPT: Data Plane Timestamping

What does DPT require from SDN switches? • Push/remove timestamps. • Take decisions based on timestamps. (Timestamp is part of the match/action).

ingress switch

egress switch

VNF

External Network

vSwitch

Timestamped Domain

Data Plane Timestamping in SDN

6

DPT: Data Plane Timestamping What can we gain from DPT? • One field, can be used for many problems / applications. • Treated uniformly in the network. ingress switch

egress switch

VNF Why is DPT feasible in softwarized environments? • Network is locally administered.

vSwitch

Adding metadata to header is feasible, e.g., [NSH], [VXLAN-GPE], [Geneve].

•

External Network

Timestamped Domain

Flexible parsing and header editing [OF-PI], [P4]. Data Plane Timestamping in SDN

7

Outline • • • • •

Background Timestamp ranges Use cases Discussion Conclusion

Data Plane Timestamping in SDN

8

Timestamp Ranges SDN switch (OpenFlow / P4) matchaction procedure SDN Switch

SDN Switch DPT Processing

Flow Table

packet header

field1 field2 field3

Flow Table

…

action

packet header

timestamp field2 field3 field4 …

action

Each Eachbit bithas hasaternary ternaryvalue: value:00//11//**

Timestamp-based ranges [TimeFlip, INFOCOM ‘15] Extremal range (T≥T0)

T0 0

Periodic range

... timestamp value Data Plane Timestamping in SDN

0

... timestamp value 9

Timestamp Format Network Time Protocol (NTP) timestamp format: Time.Sec

Time.Frac

Seconds

Second Fraction

32 bits

32 bits

Data Plane Timestamping in SDN

10

Timestamp Ranges: Examples SDN Switch

SDN Switch

DPT Processing

DPT Processing

Flow Table

Flow Table

timestamp field1 field2 field3 … packet timestamp field1 field2 field3 … action

timestamp field2 field3 field4 …

header

01 1

*…* *…*

*…* *…*

Time.Sec

Time.Frac

231>T≥230 T≥231

packet header

*…*

Time.Sec

0

*…*

Time.Frac

Periodic range

Extremal range: T≥230 230

1

action

... timestamp value

0

... timestamp value

1 second Data Plane Timestamping in SDN

11

Outline • • • • •

Background Timestamp ranges Use cases Discussion Conclusion

Data Plane Timestamping in SDN

12

Use Case 1: Consistent Updates Multicast tree update • • •

Two-phase update [Reitblatt et al. ’12]. S2 attaches version tags to packets. • ‘before’ / ‘after’. • Indicate distribution tree. Guarantees per-packet consistency.

Data Plane Timestamping in SDN

S3

S4

S2

S1

before after

13

Two-Phase Consistent Updates 1. Controller sends the ‘after’ configuration to switches (subject to ‘after’ version tag). 2. Controller updates S2 to start sending ‘after’ version tag.

S3

S4

S3

S4

S3

S4

S2

S1

S2

S1

S2

S1

1 Data Plane Timestamping in SDN

2 14

DPT-based One-Phase Consistent Updates 1. Controller sends the ‘after’ configuration to switches (subject to Timestamp ≥ Tthreshold).

2. Controller updates S2 to start sending ‘after’ version tag. Switches automatically start using the ‘after’ path when Timestamp ≥ Tthreshold

The DPT is the version tag. S3

S4

S3

S4

S2

S1

S2

S1

DPT

S3

S4

S2

S1

1 Data Plane Timestamping in SDN

15

DPT-based One-Phase Consistent Updates DPT reduces the load on the controller.

DPT reduces the management overhead of per-flow version tags.

Data Plane Timestamping in SDN

16

Use Case 2: Elephant Flow Load Balancing

src

20

S3 10

S1

10 A 20 20 S2 S5 dst B 10 S 10 4

20 Gbps elephant flow. Stateless load balancing: - Equal Cost Multiple Paths (ECMP)  50% utilization. - Random Packet Spraying (RPS) [Dixit et al. ‘13]. Stateful methods, e.g., Round Robin. Data Plane Timestamping in SDN

17

DPT-based Load Balancing Push DPT to all packets.

src

20

S1

S3 10

10 A 20 20 S2 S5 dst B 10 S 10 4

20 Gbps elephant flow.

DPT-based load balancing:

...

A

B

A

B

... timestamp value

toggle interval Data Plane Timestamping in SDN

18

DPT-based Load Balancing

src

20

S3 10

S1

10 A 20 20 S2 S5 dst B 10 S 10 4

How does the controller compute the toggle interval? • L = maximal packet length. ... A B • BW = total bandwidth of the two links. • Toggle interval = L / BW Toggle interval Independent of the flow rate ! Data Plane Timestamping in SDN

A

B

... timestamp value 19

Goodput of DPT-based Load Balancing • Load balancing using 1-bit from the DPT. • DPT: higher goodput than RPS / ECMP.

A

Time.Sec

Time.Frac

Seconds

Second Fraction

B

A

B

Toggle interval = L / BW  1 msec

timestamp ... value

A B Data Plane Timestamping in SDN

20

Use Case 3: Network Telemetry • [Coloring]: All data packets from S1 to S2 include a 1-bit ‘color’. • Measurement is performed separately for each block.

S1 CS11

Using the DPT reduces the controller load compared to using a dedicated color bit.

CR11

CS01 CR01

CS12

Time

The DPT is the only necessary field for: • Delay measurement • Loss measurement

S2

[Coloring] M. Chen, L. Zheng, G. Mirsky, G. Fioccola, and T. Mizrahi, “IP Flow Performance Measurement Framework,” draft-chen-ippm-coloringData Plane Timestamping in SDN based-ipfpm-framework, work in progress, 2016.

CR12

21

Outline • • • • •

Background Timestamp ranges Use cases Discussion Conclusion

Data Plane Timestamping in SDN

22

DPT: is it Worth the Overhead? Timestamp length / flexibility tradeoff:

64 bits fully flexible

1 bit often suffices

The common overlay technologies [NSH, VXLAN-GPE, Geneve] have built-in support for in-line metadata. Data Plane Timestamping in SDN

23

DPT Header Example: NSH Encapsulation • All traffic is encapsulated with an NSH. • NSH may include a timestamp [NSH-Timestamp]. [NSH-Timestamp] is under discussion in the SFC working group of the IETF. • •

ingress switch

egress switch

VNF

External Network

vSwitch

Timestamped Domain NSH is used for Service Function Chaining (SFC). Defined by the SFC working group of the IETF.

Network Service Header (NSH) encapsulation

[NSH-Timestamp] R. Browne, A. Chilikin, B. Ryan, T. Mizrahi, and Y. Moses, “Network service header timestamping,” draft-browne-sfc-nsh-timestamp, work in progress, IETF, 2015. Data Plane Timestamping in SDN

24

Outline • • • • •

Background Timestamp ranges Use cases Discussion Conclusion

Data Plane Timestamping in SDN

25

Conclusion DPT = Data Plane Timestamping One header for multiple applications

Consistent updates • •

Low controller CPU load Eliminates management overhead of version tags

Load balancing • •

• Stateless • High goodput

The Timed

Network telemetry Low controller CPU load No need for additional control packets

Project

http://tx.technion.ac.il/~dew/TimedSDN.html Data Plane Timestamping in SDN

26

Thanks!

Evaluation Method • Emulab testbed. • 48 Linux-based OpenFlow software switches. – CPqD OFSoftSwitch. – Dpctl controller.

• Slightly modified OpenFlow switches – Push timestamp into packet’s Ethernet source address. – NTP timestamp format.

Data Plane Timestamping in SDN

28

The Why do we need TimedSDN? Scheduling protocol Accurate scheduling method SDN Clock synchronization

Project

Timed Consistent Updates

TIME4: Dynamic Path Updates

ONECLOCK to Rule Them All

Data Plane Timestamping

[SOSR, ‘15]

[INFOCOM, ‘16]

[NOMS, ‘16]

[SWFAN, ‘16]

OpenFlow Scheduled Bundles

NETCONF Time Capability

[INFOCOM ’16, OpenFlow 1.5]

[NOMS ’16, RFC 7758]

TIMEFLIP

ONECLOCK

[INFOCOM ‘15]

[NOMS ‘16]

REVERSEPTP [HotSDN ‘14, ISPCS ‘14] Data Plane Timestamping in SDN

29

Timestamping vs. Sequence Numbers What if clocks are not synchronized? == What if we used sequence numbers [Lamport]?

ingress switch

VNF

External Network

vSwitch

• Clock synchronization has benefits. • Rough accuracy (e.g. ~1 second) may suffice in many applications. Controller is synchronized to switches  controller can plan the schedule of threshold times for network updates.

egress switch

Timestamped Domain

S3

S4

S2

S1

Controller can sample the counters / timestamps at fixed (synchronized) time intervals. Data Plane Timestamping in SDN

30

Load Balancing

10

A

B 10

S1 C ...

A

C

B

S2

20 C

A

C

B

C

... timestamp value

toggle interval Data Plane Timestamping in SDN

31

Timestamps Ranges SDN switch (OpenFlow / P4) matchaction procedure SDN Switch

SDN Switch DPT Processing

Flow Table

field1 field2 field3

Flow Table

…

packet header

action

timestamp field1 field2 field3 … packet header

action

Each Eachbit bithas hasaternary ternaryvalue: value:00//11//**

Timestamp-based ranges [TimeFlip, INFOCOM ‘15] Extremal range (T≥T0)

T0 0

Periodic range

... timestamp value Data Plane Timestamping in SDN

0

... timestamp value 32

Network Telemetry Delay measurement Loss measurement

S1

S2

T1

Time

T2

One-way delay: T2-T1 Obviously DPT can be used for delay measurement. How can DPT be used for loss measurement? Data Plane Timestamping in SDN

33

Color-based Loss Measurement • [Coloring]: All data packets from S1 to S2 include a 1-bit ‘color’. • Measurement is performed separately for each block. S1 inserts 1-bit color into packet header.

S1 CS11

S1 toggles color.

S1 and S2 periodically export counters to central point, e.g., SDN controller. Controller can compute: Packet loss = (CS12-CS11)-(CR12-CR11)

CR11

CS01 CR01 CS12

Time

S1 and S2 maintain per-color counters.

S2

CR12

[Coloring] M. Chen, L. Zheng, G. Mirsky, G. Fioccola, and T. Mizrahi, “IP Flow Performance Measurement Framework,” draft-chen-ippm-coloringData Plane Timestamping in SDN based-ipfpm-framework, work in progress, 2016.

34

Color-based Delay Measurement Delay measurement:

S1

S2

T1

Time

T2

T3 T4

Controller can compute: Two-way delay = (T4-T1)-(T3-T2) Data Plane Timestamping in SDN

35

Color-based Measurement

Q: How does an SDN switch toggle the color?

A: Controller periodically updates S1 coloring policy... Overhead on the controller…

S1 CS11

S2 CR11

CS01 CR01

Time

CS12

Data Plane Timestamping in SDN

CR12

36

DPT-based Coloring • Choose 1 bit from the timestamp. • Use this bit as the color. • Each switch has two match-action rules: • Timestamp bit=0 • Timestamp bit=1

Time.Sec

Time.Frac

Seconds

Second Fraction

‘1’

• Each block: a constant time interval. • The controller is not involved in the color toggling. 

Data Plane Timestamping in SDN

color ‘0’ ‘1’

‘0’

timestamp ... value

37

DPT-based Coloring DPT allows for both delay and loss measurement. No other fields are necessary!

Time.Sec

Time.Frac

Seconds

Second Fraction

‘1’

color ‘0’ ‘1’

‘0’

timestamp ... value

A 1-bit timestamp is sufficient.

DPT-based coloring reduces the load on controller.

Data Plane Timestamping in SDN

38

References [1]

T. Mizrahi, Y. Moses, “Software Defined Networks: It’s About Time", IEEE INFOCOM, 2016. (http://tx.technion.ac.il/~dew/Time4INFOCOM.pdf)

[2]

M. Chen, L. Zheng, G. Mirsky, G. Fioccola, and T. Mizrahi, “IP Flow Performance Measurement Framework,” draft-chen-ippm-coloring-based-ipfpmframework, work in progress, 2016. (https://tools.ietf.org/html/draft-chen-ippm-coloring-based-ipfpm-framework)

[3]

R. Browne, A. Chilikin, B. Ryan, T. Mizrahi, and Y. Moses, “Network service header timestamping,” draft-browne-sfc-nsh-timestamp, work in progress, IETF, 2015. (https://tools.ietf.org/html/draft-browne-sfc-nsh-timestamp)

[4]

T. Mizrahi, O. Rottenstreich, Y. Moses, "TimeFlip: Scheduling Network Updates with Timestamp-based TCAM Ranges", IEEE INFOCOM, 2015. (http://tx.technion.ac.il/~dew/TimeFlipINFOCOM.pdf)

[5]

T. Mizrahi, E. Saat, Y. Moses, "Timed Consistent Network Updates", ACM SIGCOMM Symposium on SDN Research (SOSR), 2015. (http://tx.technion.ac.il/~dew/TimedConsistentSOSR.pdf)

[6]

T. Mizrahi, Y. Moses, "Time-based Updates in Software Defined Networks", the second workshop on hot topics in software defined networks (HotSDN), 2013. (http://tx.technion.ac.il/~dew/TimeSDN.pdf)

Data Plane Timestamping in SDN

39