www.iparchitechs.com

b o

c .i

WISP Design – Using eBGP and OSPF transit fabric for traffic engineering

o h

a s

PRESENTED BY:

KEVIN MYERS, NETWORK ARCHITECT

m o

Profile: About

Kevin Myers

Background: • • • •

19+ years in Networking Designed/Built Networks on 6 continents MikroTik Certified Trainer MikroTik, Cisco and Microsoft Certified

b o

Community Involvement:

o h

m o

c .i

Packet Pushers (Podcast Guest / Blogger)

Group Contributor (RouterOS / WISP Talk and others)

a s

Delegate/Roundtable contributor (NFD14) MT Forum (Forum Veteran – Member since 2012) Network Collective (Podcast Guest)

Profile: About

IP ArchiTechs

m o

c .i

Expert Networking

b o

Whitebox | ISP | Data Center | Enterprise

✓ ✓ ✓ ✓ ✓

o h

Global Consulting Managed Networks Monitoring Load Testing Development

a s

Locations in: US | Canada | South America

Call us at: +1 855-645-7684 E-mail: [email protected] Web: www.iparchitechs.com

Design: Why

eBGP/OSPF Transit Fabric?

m o

Goal of this presentation: When the presentation is finished, hopefully you will have walked away with a few key concepts:

c .i

• limitations in using OSPF or other non-BGP routing protocol when attempting to influence traffic paths in a WISP network

b o

• How to leverage all bandwidth between two towers using an OPSF Transit Fabric

o h

• The benefits of using eBGP and communities to build a scalable framework for identifying and managing traffic in tower networks.

a s

Design: Why

eBGP/OSPF Transit Fabric?

m o

Problem #1: Many WISPs utilize OSPF (Static routing or Bridging also) for the forwarding of traffic. As the network grows, this creates suboptimal traffic flows.

b o

750M

750M

750 Mbps

Customer

o h

500M

a s

c .i 750M

Internet 500M

500M

Least Cost Path

Example: OSPF when using default Gigabit cost (10) will select the “shortest” path – but not the fastest.

Design: Why

eBGP/OSPF Transit Fabric?

m o

Better or Worse?: We’ve modified the OSPF cost so that traffic takes the lowest cost path. How well does this scale? New Best Path

b o

Cost 5

Cost 5

Cost 5

Customer

o h

Cost 15

a s

c .i Cost 5

Internet Cost 15

Cost 15

Example: Cost has now been modified to 5 on each link for the faster path and 15 on each link for the slower path.

Design: Why

eBGP/OSPF Transit Fabric?

m o

It’s Worse!: Now we have a new tower to add. The “fast” path is getting overloaded and we need to use some of the bandwidth on the other path. Overloaded Path

b o

Cost 5

Cost 5

Cost 5

Customer

o h

Cost 15

a s Cost 10

c .i Cost 5

Internet Cost 15

Cost 15

Example: With the addition of the new tower, we need to utilize bandwidth down both paths.

Design: Why

eBGP/OSPF Transit Fabric?

m o

Sacrificing bandwidth: Now we have utilized both paths and alleviated the traffic problem yet again, but we had to sacrifice 250Mbps on the faster path to accomplish it. ECMP

b o

Cost 15

Cost 15

Cost 15

Customer

o h

Cost 20

a s Cost 10

c .i

Cost 15

Internet Cost 20

Cost 20

ECMP

Example: The OSPF costs have been set to enable ECMP so that both paths will carry traffic.

Design: Why

eBGP/OSPF Transit Fabric?

m o

Overloaded and low on options: Now we have added another tower. At this point, we have utilized both paths and have no more room on either due to ECMP. Even though there is 250Mbps of potential bandwidth, we can’t use it without static or policy routing.

b o

Overloaded (250M unusable)

o h Cost 15

Cost 10

Customer

Cost 15

a s

Cost 15

Cost 20

Cost 10

Cost 20

Overloaded

c .i

Cost 15

Internet Cost 20

Design: Why

eBGP/OSPF Transit Fabric?

m o

Problem #2: In WISP deployments, there is often more than one set of PtP radios connecting the same two towers. Traditionally, these are configured into an active/backup setup or one link is configured to handle upstream, and the other is configured for downstream traffic.

b o

500M Backup

o h

Customer

Customer

a s

750 Mbps Active

c .i

Internet

500M Upstream

Internet 750 Mbps Downstream

Design: Why

eBGP/OSPF Transit Fabric?

m o

Unused capacity: The biggest challenge with using PtP links in active/backup or upstream/downstream configurations is the unused capacity in the lower speed links.

b o

Unused Backup

a s

o h

c .i

Internet

Overloaded Active

Unused for Downstream

Internet Overloaded Downstream

Design: OSPF

Transit Fabric and eBGP overview

m o

OSPF alone doesn’t work: Looking at the previous slides, we saw a number of issues and workarounds to manage traffic using only OSPF as the routing protocol. With every new workaround, a new issue or limitation with scaling the design came up. Why is that?

b o

c .i

• Because OSPF is designed to be a reachability protocol – NOT a policy protocol. The function of a link state routing protocol is to map the available paths and speeds to prefixes.

a s

o h

• OSPF is not well suited to implementing policy as we saw in the previous slides

Design: OSPF

Transit Fabric and eBGP overview

m o

Potential Solution: Now that we’ve identified the problems and limitations associated with turning an IGP like OSPF into a policy protocol, let’s look at a potential solution.

c .i

• Current design approach is to use OSPF to advertise transit subnets and loopbacks to form an iBGP AS. This works very well but can run into limitations in policy options due to the dependence on an IGP to advertise the next hop.

o h

b o

• Combining eBGP and the OSPF Transit Fabric allows for total control of the tower path and allows for the use of all bandwidth between any two towers.

a s

• It’s not all or nothing, iBGP can still be utilized where it makes sense and eBGP/Transit Fabric deployments can be utilized at key aggregation points to make complex traffic decisions.

Design: What

is an OSPF Transit Fabric?

m o

Defining the OSPF Transit Fabric: The OSPF Transit Fabric (TF) is a design that came out a real world deployment for an IP ArchiTechs client. We needed a way to use all of the bandwidth between two towers on links of unequal speed. OSPF Area 0

b o PtP 01

750 Mbps

6 VLANs for OSPF – Cost 10

Tower 1

VLAN 2001- 100.65.0.0/29 VLAN 2002- 100.65.0.8/29 VLAN 2003- 100.65.0.16/29 VLAN 2004- 100.65.0.24/29 VLAN 2005- 100.65.0.32/29 VLAN 2006- 100.65.0.40/29

a s

o h

c .i

Traffic is split across both physical links at a 6 to 2 ratio VLAN 2007- 100.65.0.48/29 VLAN 2008- 100.65.0.56/29

2 VLANs for OSPF – Cost 10

PtP 02 150 Mbps

Tower 2

Design: What

is an OSPF Transit Fabric?

m o

Using VLANs to create unequal load balancing: The idea behind the TF is to use VLANs at a default cost to build a ratio for balancing traffic across unequal links. ECMP balances traffic by flow when there is a next hop reachable via interfaces of equal cost.

b o

OSPF Area 0

PtP 01

750 Mbps

6 VLANs for OSPF – Cost 10

Tower 1

a s

o h

VLAN 2001- 100.65.0.0/29 VLAN 2002- 100.65.0.8/29 VLAN 2003- 100.65.0.16/29 VLAN 2004- 100.65.0.24/29 VLAN 2005- 100.65.0.32/29 VLAN 2006- 100.65.0.40/29

c .i

Traffic is split across both physical links at a 6 to 2 ratio VLAN 2007- 100.65.0.48/29 VLAN 2008- 100.65.0.56/29

2 VLANs for OSPF – Cost 10

PtP 02 150 Mbps

Tower 2

Design: What

is an OSPF Transit Fabric?

m o

Using VLANs to create unequal load balancing: Because the VLAN interface on each PtP has the same cost, RouterOS sees 8 equal-cost next hops to load balance traffic onto. This forms a 6 to 2 ratio of traffic in favor of the higher speed link.

b o

OSPF Area 0

PtP 01

750 Mbps

6 VLANs for OSPF – Cost 10

Tower 1

o h

VLAN 2001- 100.65.0.0/29 VLAN 2002- 100.65.0.8/29 VLAN 2003- 100.65.0.16/29 VLAN 2004- 100.65.0.24/29 VLAN 2005- 100.65.0.32/29 VLAN 2006- 100.65.0.40/29

a s

c .i

Traffic is split across both physical links at a 6 to 2 ratio VLAN 2007- 100.65.0.48/29 VLAN 2008- 100.65.0.56/29

2 VLANs for OSPF – Cost 10

PtP 02 150 Mbps

Tower 2

Design: What

is an OSPF Transit Fabric?

m o

The end result: What the RouterOS routing table looks like with a TF.

b o

c .i

Live traffic: Here is a look at the physical and VLAN interfaces under load in a TF - 6.2 Mbps on BH-06/ 22.8 Mbps on BH-07

a s

o h

Design: What

is an OSPF Transit Fabric?

m o

Now I want an OSPF TF, but how does it scale?

• One of the challenges of implementing the OSPF TF is scaling it to multiple towers

c .i

• WISPs that are under 50 towers can utilize the OSPF TF without using another routing protocol on top of OSPF such as BGP

b o

• WISPs that are over 50 towers or that have complex traffic management requirements may want to consider coupling the OSPF TF with eBGP

a s

o h

• eBGP provides a way to limit OSPF strictly between the towers by using eBGP multihop peering and loopbacks.

Design: Utilizing

eBGP at the tower to set traffic paths

m o

eBGP as a traffic engineering mechanism: One of the major benefits of using eBGP is the ability to definitively set the next hop by matching on a number of attributes. When coupled with BGP communities, a vast array of options for traffic engineering are available.

o h

Internet

900M

TWR-1

b o

a s

TWR-2

c .i 900M

TWR-3

750M

500M

TWR-5

750M 500M

750M 500M

TWR-4

Design: Utilizing

eBGP at the tower to set traffic paths

m o

How does eBGP differ from iBGP?: eBGP or External BGP means the AS you are peering to is different than your AS. We are going to look at a network that combines eBGP and the OSPF Transit fabric to solve traffic engineering and unequal load balancing issues.

b o

Internet

o h

RIR AS INET EDGE

900M

AS 65001 TWR-1

a s

AS 65002 TWR-2

c .i 900M

750M

500M

AS 65005 TWR-5

750M 500M

750M 500M

AS 65004 TWR-4

AS 65003 TWR-3

Design: Utilizing

eBGP at the tower to set traffic paths

m o

Combining OSPF TF and eBGP: In order to build an eBGP design but still leverage the OSPF TF for capacity, we have to build a multihop eBGP peering using the OSPF TF loopbacks in blue. Then the BGP loopbacks in red can be advertised as the reachable loopback outside of that router.

b o 750M

c .i

VLAN 101 - 100.80.0.0/29 - Link VLAN 102500M - 100.80.0.8/29 VLAN 104 - 100.80.0.24/29 VLAN 103 - 100.80.0.16/29 VLAN 105 - 100.80.0.32/29

AS 65001 TWR-1

o h

eBGP multihop peering fron 100.98.1.1 to 100.98.1.5

a s

Lo0 - 100.99.1.1 Lo-TF-1-to-5 100.98.1.1

VLAN 104 - 100.80.0.24/29 VLAN 105 - 100.80.0.32/29

BGP

500M

OSPF TF

AS 65005 TWR-5

Lo0 - 100.99.1.5 Lo-TF-5-to-1 100.98.1.5

Design: Utilizing

eBGP at the tower to set traffic paths

m o

BGP Communities: BGP Communities are a field in a BGP route advertisement that can be used to take action on a prefix tagged with that community number. They can also be used for route identification. The format is 32 bits with the first 16 bits typically set to the ASN of the network and the last 16 bits are defined by the operator.

b o

BGP Communities 1111:1 1111:2 1111:3 1111:4 1111:5 1111:200 1111:201

500M - Link VLAN 104 - 100.80.0.24/29 VLAN 105 - 100.80.0.32/29

o h

c .i

Tower 1 Tower 2 Tower 3 Tower 4 Tower 5 Northern Tower Path (T3,T2,T1) Southern Tower Path (T3,T4,T5,T1)

a s

Design: Utilizing

eBGP at the tower to set traffic paths

m o

Changing the next hop: How a community is matched and weight set.

RIR AS INET EDGE AS 65001 TWR-1

b o

AS 65002 TWR-2

Internet Match 1111:200 Set weight to 32467 For BGP peer to T2

o h 900M

a s

c .i

Advertise to T2

900M

750M

500M

Match 1111:201 Set weight to 32467 For BGP peer to T5

AS 65005 TWR-5

Sector 1/T3 10.10.0.0/23 1111:3 1111:201

750M 500M

AS 65003 TWR-3

750M 500M

Advertise to T4

AS 65004 TWR-4

Sector 1/T3 10.10.0.0/23 1111:3 1111:201

Design: Utilizing

eBGP at the tower to set traffic paths

m o

Influencing traffic down one path: By tagging the subnet for Sector 1 at Tower 3 (10.10.0.0/23) into community 1111:201, we are able to force the traffic along the higher capacity but less preferred southern tower path.

b o AS 65002 TWR-2

Internet

o h

RIR AS INET EDGE AS 65001 TWR-1

900M

a s

900M

750M

500M

10.10.0.0/23 (1111:201) Traffic flows over the southern path

c .i

AS 65005 TWR-5

750M 500M 750M 500M

AS 65004 TWR-4

AS 65003 TWR-3

Design: Utilizing

eBGP at the tower to set traffic paths

m o

Influencing traffic down two paths: Now we can break the sectors up at Tower 3 and send one subnet down the Northern path and one subnet down the Southern path to utilize multiple paths to get back to one tower.

b o AS 65002 TWR-2

Internet

o h

RIR AS INET EDGE AS 65001 TWR-1

900M

a s

c .i

10.10.2.0/23 (1111:200) Traffic flows over the northern path

900M

750M

500M

10.10.0.0/23 (1111:201) Traffic flows over the southern path

AS 65005 TWR-5

750M 500M 750M 500M

AS 65004 TWR-4

AS 65003 TWR-3

Design: Utilizing

eBGP at the tower to set traffic paths

m o Now let’s ctalk . i about your b o Questions! o h a s

Design: Utilizing

eBGP at the tower to set traffic paths

m o

Live Demo : See the design in action at the IPA booth! Come by and learn more about eBGP and OSPF Transit Fabric

a s

o h

b o

c .i

Design: Utilizing

eBGP at the tower to set traffic paths

m o

BGP Party at 6:30!

o h

b o

a s

c .i

Please RSVP by going to www.facebook.com/iparchitechs

04. Traffic_Engineering_Menggunakan_BGP_sahoobi.pdf ...

Whitebox | ISP | Data Center | Enterprise. Locations in: US | Canada | South America. ✓ Global ... key concepts: • limitations in ... in tower networks. sahoobi.com.

2MB Sizes 0 Downloads 42 Views

Recommend Documents

1720-04-04
Aug 1, 2017 - effective August 12, 1986. (For history prior to August 12, 1986 see pages (iii)-(i).) Amendment filed. March 22, 1990; effective June 26, 1990.

WardsboroAct46-04-04-17.pdf
There was a problem previewing this document. Retrying... Download. Connect more apps... WardsboroAct46-04-04-17.pdf. WardsboroAct46-04-04-17.pdf.

04-04-16.pdf
ve. Whoops! There was a problem loading this page. Retrying... Whoops! There was a problem loading this page. Retrying... 04-04-16.pdf. 04-04-16.pdf. Open.

2009-04-04 3. Mitgliederversammlung.pdf
The Old Apostolic Church (Germany). Protokoll der 3. ... Andrea Schulz, Stephan Schulz,. Ruth Schulz, Ingolf ... Mitgliederversammlung.pdf. 2009-04-04 3.

04-04-17.pdf
Page 1 of 2. By Emma Scott,. Emelia Skogstad. and Grace Isaacson. An ad-room was held to. begin the day today as students. and staff prepare for standard- ized testing of juniors to begin. Wednesday. On Wednesday, juniors. will take the S.D. Science

1974-04-04 - noise ordinance .pdf
Download. Connect more apps... Try one of the apps below to open or edit this item. 1974-04-04 - noise ordinance .pdf. 1974-04-04 - noise ordinance .pdf. Open.

04-04-16 Acton TM 2016 FINAL.pdf
Whoops! There was a problem loading more pages. Retrying... 04-04-16 Acton TM 2016 FINAL.pdf. 04-04-16 Acton TM 2016 FINAL.pdf. Open. Extract.

R-04 ...
Page 1. Whoops! There was a problem loading more pages. Retrying... R-04-03_OPERACIONES_CON_POLINOMIOS_SUMA_Y_RESTA_MAc3ESO.pdf.

Accounting (9706/04)
Ordinary shares of $0.50 each, valued at issue price of $0.70 each. 350 ... Stock. 90. Trade debtors. 42. 132. Creditors: amounts falling due in less than one year.

1720-05-04
Aug 1, 2017 - and new rule filed May 27, 1986; effective August 12, 1986. ... Repeal and new ... rule file November 10, 2005; effective March 30, 2006.

04.pdf
... the culture, health, or standards of living of the county's citizen's and thus the. quality of life. As a result, nations develop regulations aimed at protecting their citizens. Czinkota, Michael R., Ilkka A. Ronkainen and Michael H. Moffett. Fun

1720-04-03
Aug 22, 2017 - report, incident report, witness statements, video or audio recordings, photographs, text messages, or phone records. (c) More than One Respondent. In cases involving more than one (1) Respondent, SCB. Hearings concerning each Responde

04.pdf
Foreign trade groups, foreign consulates and embassies in the United States, foreign .... 04.pdf. 04.pdf. Open. Extract. Open with. Sign In. Details. Comments.

2017-04 ...
recognition that financial regulation has become far too complex and too intrusive .... -04-24_financial_choice_act_of_2017_comprehensive_summary_final.pdf.

04.pdf
... the culture, health, or standards of living of the county's citizen's and thus the. quality of life. As a result, nations develop regulations aimed at protecting their citizens. Czinkota, Michael R., Ilkka A. Ronkainen and Michael H. Moffett. Fun