TM

Advanced Services

Cisco Systems Advanced Services

BGP Best Practices
Version 3.0

Cisco
Corporate Headquarters
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax:408 526-4100

Legal Notice

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS
DOCUMENT ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,
INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE
ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY
PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT
ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND
ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE
SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE
FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the
University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating
system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND
SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND
THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED,
INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF
DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL,
CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST
PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE
THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.

CCDE, CCENT, CCSI, Cisco Eos, Cisco HealthPresence, the Cisco logo, Cisco Lumin, Cisco Nexus,
Cisco Nurse Connect, Cisco Stackpower, Cisco StadiumVision, Cisco TelePresence, Cisco WebEx, DCE,
and Welcome to the Human Network are trademarks; Changing the Way We Work, Live, Play, and Learn
and Cisco Store are service marks; and Access Registrar, Aironet, AsyncOS, Bringing the Meeting To
You, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, CCVP, Cisco, the Cisco Certified
Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco
Systems logo, Cisco Unity, Collaboration Without Limitation, EtherFast, EtherSwitch, Event Center, Fast
Step, Follow Me Browsing, FormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, iQuick
Study, IronPort, the IronPort logo, LightStream, Linksys, MediaTone, MeetingPlace, MeetingPlace Chime
Sound, MGX, Networkers, Networking Academy, Network Registrar, PCNow, PIX, PowerPanels,
ProConnect, ScriptShare, SenderBase, SMARTnet, Spectrum Expert, StackWise, The Fastest Way to
Increase Your Internet Quotient, TransPath, WebEx, and the WebEx logo are registered trademarks of
Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.

All other trademarks mentioned in this document or website are the property of their respective owners.
The use of the word partner does not imply a partnership relationship between Cisco and any other
company.

Copyright © 2009 Cisco Systems, Inc. All rights reserved.

BGP Best Practice Version 3.0 2

Contents

Contents 2

Tables 7

Figures 8

About This Design Document 9

Document Purpose 9
Intended Audience 9
Scope 9

Dynamic Update Peer-Groups and Peer-Templates 11

Background 11
Benefits 11
Guidelines 12
Risks and Limitations 12

Resource Allocation and Convergence 14

Background 14
Benefit 14
Guidelines 14
Details 15
Improving BGP Convergence 15

Soft Reconfiguration and Route Refresh 19

Background 19
Benefits 19
Guidelines 19
Risks and Limitations 19
Details 20

BGP Support for Local-AS 21

Background 21
Guidelines 21
Details 21

BGP Support for Dual-AS Configuration 22

Background 22
Guidelines 22

BGP Best Practice Version 3.0 3

0 4 . Details 23 BGP Infrastructure Security 26 BGP TTL Security Hack (BTSH) 26 Background 26 Benefits 26 Guideline 26 Risks and Limitations 26 BGP Authentication 27 Background 27 Benefits 27 Guideline 27 Risks and Limitations 27 BGP Maximum-Prefix 27 Background 27 Benefits 28 Guideline 28 Route Flap Dampening 29 Background 29 Guidelines 29 Risks and Limitations 29 Details 30 Route Reflectors 31 Background 31 Benefits 31 Guidelines 31 Risks and Limitations 32 Confederations 33 Background 33 Guidelines 33 Details 33 Route Propagation in Confederation 33 BGP-IGP Redistribution Policies 35 Background 35 Benefits 35 Guidelines 35 Risks and Limitations 35 BGP Next-hop 36 Background 36 Guidelines 36 Community Attribute 38 BGP Best Practice Version 3.

0 5 . single homed 50 Scenario 2: Stub network multi-homed to single SP with single border router 51 Scenario 3: Stub network multi-homed to single SP with multiple border routers 52 Scenario 4: Standard Network Multi-homed to Multiple ISPs with Multiple Border Routers 53 Load Balancing 54 Background 54 Guidelines 54 Inbound Load Balancing 54 Outbound Load Balancing 54 Scenario 1 54 Scenario 2 55 Scenario 3 55 Scenario 4 (iBGP Multipath) 55 Scenario 5 (Link Bandwidth) 56 Risks and Limitations 56 Details 57 Using Advertise Map for Inbound Load Balancing 57 Using EBGP Multipath for Outbound Load Balancing 58 Using BGP Link Bandwidth for Outbound Load Balancing 59 Route Aggregation 61 Background 61 Benefits 61 Guidelines 61 Risks and Limitations 62 Details 62 Configuration Example Aggregation using the network command 63 BGP Best Practice Version 3. Background 38 Benefits 38 Guidelines 38 Details 39 Extended BGP Communities 41 Synchronization Rules 44 Background 44 Benefits 44 Guidelines 44 Multi-homing 45 Background 45 Benefits 45 Multi-homing Scenarios 45 Stub Network Single-homed 46 Stub Network Multi-homed: Single Border Router 46 Stub Network Multi-homed: Multiple Border Routers 46 Standard Multi-Homed Network: Single border router / Multiple ISP’s 47 Standard Multi-Homed Network: Multiple border routers 49 Multi-homing Scenario Examples 50 Scenario 1: Enterprise stub network.

Other Miscellaneous Areas 64 Prefix Lists 64 Background 64 Conditional Route Injection 65 Background 65 Benefits 66 Guidelines 66 BGP Deterministic-med 67 Background 67 Benefits 67 Guidelines 70 BGP Router Identifier 71 Background 71 Benefits 71 Guidelines 71 Risks and Limitations 71 BGP Log Neighbor Changes 71 Background 71 Benefits 71 Guidelines 72 BGP Best Practice Version 3.0 6 .

Tables Table 1 BGP Community Design 41 BGP Best Practice Version 3.0 7 .

0 8 .Figures Figure 1 BGP Support for Dual-AS example 23 Figure 2 Single-homed Enterprise Stub Network with Border Router 50 Figure 3 Stub Network Multi-homed to Single SP and Border Router 51 Figure 4 Stub Network Multi-homed to Single SP with Multiple Border Routers 52 Figure 6 Use of advertise-map to influence inbound traffic 57 Figure 7 Using EBGP Multipath for Outbound Load Balancing 58 Figure 8 Using BGP Link Bandwidth for Outbound Load Balancing 59 Figure 9 BGP Deterministic-med Example 67 BGP Best Practice Version 3.

In other words. Intended Audience • This document is for NCEs in Advanced Services. Scope This document includes BGP Best Practices for Design. BGP Support for Dual-AS Configuration BGP Best Practice Version 3. Resource Allocation and Convergence 3. About This Design Document Routing Protocols design and implementation is quite mature. The following areas are covered: 1. Planning. Routing VT started working to capture these best practices as well as provide a dynamic way to constantly update and add. Optimization. although some examples are provided to help understand the recommendations in question. Furthermore. It is expected that the reader is familiar with basic BGP routing and is experienced in configuring Cisco routers. other Cisco Technical Teams. Migration. there are many initiatives currently underway to automate the exception reports based on violation of best practices rules. and Customers. recommendations are highlighted yellow to clearly separate them from explanations. provide a platform for converting AS expertise to Cisco Intellectual Property. In an effort to address all these issues. Document Purpose • To provide comprehensive industry best practices for BGP used on Cisco Routers. Cisco Partners.0 9 . No attempt is made to explain and clarify fundamental functionality. Throughout the document. Case Studies. This document captures best practices for implementing BGP. It was also felt that there is no central place where we can store and use all the Design Reviews and Case studies that are based on best practices in specific situations. New Features. This document focuses on the industry best practices and other related information for BGP implementation. and updated. Dynamic Update Peer-Groups and Peer-Templates 2. • To provide a medium to track and continuously update the technology best practices to ensure that the collective technical experience of Advanced Services Routing VT is recorded. Still. reviewed. our experience with Routing VT pointed to the fact that what Cisco or Advanced Services is recommending with regards to routing protocol implementation as Best Practices is not being captured. and a good amount of documentation is available within Cisco as well as on other external sites. • This document is also for the Advanced Services Tools Team for automation of Best Practices Compliance. Implementation. and Caveats. Soft Reconfiguration and Route Refresh 4. BGP Support for Local-AS 5.

The user must add information. that use BGP is beyond the scope of this document. they should be aware of hazards involved with electrical circuitry and be familiar with standard practices for preventing accidents. etc. BGP Infrastructure Security 7. The text that accompanies this symbol provides the user with a useful tip. Route Reflectors 9. The user might save time by performing or being aware of the action/information described in the paragraph. in this document they have the following meaning: This symbol means warning. Load Balancing 16. MP-BGP.0 10 . In this situation. Other Miscellaneous Topics Information regarding other related technologies. Route Aggregation 17. MBGP. the user might do something that could result in equipment damage or loss of data.6. BGP-IGP Redistribution Policies 11. Synchronization Rules 14. This symbol means timesaver. such as MPLS. This symbol means tip. The user may be in a situation that could cause bodily injury. Confederations 10. written or typed. Where side symbols are included. Community Attribute 13.. Multi-homing 15. Route Flap Dampening 8. This symbol means caution. BGP Next-hop 12. BGP Best Practice Version 3. Before the user works on the equipment. to the document during the implementation work or that the user must take note of the information presented. This symbol means note.

such as: . Hence.0(24) S]. it separates update-group replication from peer-group configuration. Peer Session Template: Peer session templates are used to group and apply the configuration of general session commands that are common to all address family and Network Layer Reachability Information (NLRI) configuration modes. A peer template is a configuration pattern that can be applied to neighbors that share common policies. To address the limitations of peer groups. The Dynamic Update Peer-Groups implementation can automatically calculate. These neighbors automatically fall under the same update-group. BGP update messages were grouped together based on peer-group configurations. With Dynamic Update Peer-Group feature. This method of grouping updates limited outbound policies and specific-session configurations. and update-groups can belong to different address families. In IOS version prior to 12. All neighbors have to belong to the same peer group and address family. All neighbors that share the same peer group configuration also has to share the same outbound routing policies . The BGP Configuration using Peer Templates feature improves the flexibility of BGP neighbor configuration through the introduction of peer-policy and peer-session configuration templates. this feature does not require any manual configuration. In previous versions of Cisco IOS Software [prior to 12. as to which neighbors can share updates. based on the configuration of the neighbors. BGP Best Practice Version 3. eliminating the need for depending on peer-group configuration. Dynamic Update Peer-Groups and Peer- Templates Background The BGP Dynamic Update Peer-Groups feature introduces a new algorithm that dynamically calculates and optimizes update-groups of neighbors that share the same outbound policies and can share the same update messages. peer-groups are also used for update-grouping in addition to configuration grouping which has some limitations. which improves convergence time and flexibility of neighbor configuration. • This feature does not require any configuration by the network operator. The following are the features of the Dynamic Update Peer-Group: • Dynamically calculates BGP update-group membership based on outbound routing policies.0 11 .0(24)S. • Optimal BGP update message generation occurs automatically and independently. Benefits The BGP Dynamic Update Peer-Group feature requires no configuration and occurs automatically. The BGP Configuration Using Peer Templates feature introduces a new mechanism called the peer template. • BGP neighbor configuration is no longer restricted by outbound routing policies. the BGP Configuration Using Peer Templates feature was introduced along with the BGP Dynamic Update Peer-Groups feature. Peer-templates address these limitations.

• A BGP neighbor cannot be configured to work with both peer groups and peer templates. Peer templates also provide an alternative to Peer-Group configuration and overcome some limitations of Peer-Groups.0(24)S and integrated into 12. Configuring peer-groups will not achieve anything in terms of performance rather it may just cut down on configuration. The inheritance capability is a key component of peer template operation. • The inheritance capability of peer template eliminates the need to repeat configuration statements that are commonly reapplied to groups of neighbors because common configuration statements can be applied once and then indirectly inherited by peer templates that are applied to neighbor groups with common configurations. So.2(18)S and 12. • A peer policy template can directly or indirectly inherit up to eight peer policy templates. So there are no performance benefits with peer-groups if the code supports BGP Dynamic Update Peer-Groups feature.0(24)S and integrated into 12. and each inherited session template can also contain one indirectly inherited session template. • If you are running an IOS version that supports BGP Dynamic Update Peer-Groups feature then the default behavior is that BGP dynamically calculates and optimizes update-groups of neighbors that share the same outbound policies and can share the same update messages which improves the performance of BGP update message generation. Peer-Template feature was first introduced in 12. Inheritance expands the scalability and flexibility of neighbor configuration by allowing you to chain together peer templates configurations to create simple configurations that inherit common configuration statements or complex configurations that apply very specific configuration statements along with common inherited configurations.3(4)T and 12. A BGP neighbor can be configured to belong only to a peer group or to inherit policies only from peer templates. • Peer templates are reusable and support inheritance. a neighbor or neighbor group can be configured with only one directly applied peer session template and seven additional indirectly inherited peer session templates. Peer Policy Template: Peer policy templates are used to group and apply the configuration of commands that are applied within specific address-families and NLRI configuration modes.2(27)SBC. • Allows the network operator to define very complex configuration patterns through the capability of a peer template to inherit a configuration from another peer template. 12. Risks and Limitations The following restrictions apply to the BGP Configuration using Peer Session Templates and Peer Policy Template feature: • A peer session template can directly inherit only one session template. BGP Best Practice Version 3. Guidelines BGP Dynamic Update Peer-Group feature was first introduced in 12.3(4)T. • The BGP Dynamic Update Peer-Group feature requires no configuration and occurs automatically. • It is recommended to use Peer templates as it improves the flexibility and enhances the capability of neighbor configuration.0 12 . which allows the network operator to group and apply distinct neighbor configurations for BGP neighbors that share common policies.2(18)S .

neighbour grouping and ease of configuration. BGP Best Practice Version 3. Peer-Groups If you are running an IOS version that does not support BGP Dynamic Update Peer-Groups. multiple policies are available for each remote AS so there is no need to configure an entirely new peer-group configuration for the each different AS. For detailed information on Peer-Templates. For detailed information on Dynamic Update Peer-Groups. With Peer- Templates. click here. There is no flexibility on a per neighbor basis as available with peer-templates and inheritance. • For small networks with simple BGP update policies.0 13 . Peer-Template configuration could seem to be extensive however the benefit that is received due to inheriting peer-template and session- template policies provides the flexibility in configuration even as the network grows. click here. With peer-group configuration. If the IOS version supports Dynamic Update Peer-Groups with Peer-Templates then it is recommended to use that feature as it provides greater flexibility. multiple peer-groups are required for each remote AS. then Peer- Groups can be used as all update messages are grouped together based on peer-group configurations and also reduces configuration.

etc. Resource Allocation and Convergence Background BGP is primarily used to provide control plane connectivity between multiple administrative domains (called autonomous systems (AS)) with varying routing policies. Benefit • Faster convergence: By assessing the requirement of appropriate resources and tuning other parameters.. and tuning session and update timers. BGP will choose the shortest path through the internetwork as defined by the AS Path. and memory burdens of expanding networks. router processing and memory demands increase. there are several steps that customers can take to further reduce convergence time. BGP Best Practice Version 3. Appropriate hardware has to be considered depending on the number of peers and routes. BGP updates rely on TCP. Guidelines BGP updates take CPU cycles. These features within Cisco IOS software assist network operators with scaling their IP-BGP networks in ways that mitigate the management. this could eventually pose scalability challenges to Service Providers and enterprise networks to maintain an ever-increasing number of TCP sessions. large input queues. As individual networks grow larger. and the number of peers and routes that you want to hold takes memory. Due to its importance in the Internet. service providers and large enterprise customers are noticing an increase in the BGP convergence time. BGP chooses the best loop free path through the internetwork. Transmission Control Protocol path Maximum Transmission Unit discovery (PMTUD). In the absence of overriding policy. with “best” being defined by the local policy of each AS. These include using peer groups. In addition. In general. TCP window size. help improve converge. the following components have an effect on the number of BGP routes/peers a router can support • Router's CPU • Route Memory • IOS version The more memory a router has the more routes it can support. path MTU discovery. BGP administrators can achieve faster convergence within their networks.0 14 . memory). Several BGP enhancements have been made to improve convergence and basic scaling properties. interface input queues. • Increased scalability: BGP enhancements and configuration recommendations allow us to handle even greater numbers of routing table entries within the same convergence time frame. Besides enhancing hardware resources (CPU. optimization of router resources like memory and TCP session parameters like maximum segment size (MSS). much like how a router with a faster CPU can support larger numbers of peers. BGP is a major focus for scaling and convergence work. processing. The best-known instance of inter-domain connectivity provided using BGP is the Global Internet. As the size of the Internet routing table and number of peers grow.

it is important to understand ways to reduce memory consumption and achieve optimal routing without the need to receive the complete Internet routing table. cuts memory consumption. Using a MSS of 536 bytes ensures that the packet will not be fragmented before it gets to its destination because most links have a MTU of at least 1500 bytes. Knowledge of these parameters is required to calculate the amount of memory required to store a certain number of BGP routes. If default values do not meet the network convergence requirements. Using a small MSS value creates a large amount of TCP/IP overhead. This means TCP will take all of the data in a transmit queue and break it up into 536 byte chunks before passing packets down to the IP layer. and hold timer) at their default values. For detailed information on achieving optimal routing and memory consumption. bgp scan-timer import and bgp scan- timer. Details Improving BGP Convergence Longer convergence times are due to the increased size of the Internet table and an increase in the number of peers supported by a single BGP speaker. This is accomplished by enabling ip tcp BGP Best Practice Version 3. • Enable Path MTU Discovery • TCP MSS is recommended along with Path MTU discovery in Multi-Hop BGP Session. route dampening. However. such as the router. Memory The amount of memory required to store BGP routes depends on many factors. • Increase SPD headroom to prevent dropping BGP Sessions. Recent features such as Next-Hop-Tracker. BGP attributes. it may become necessary to carefully observe the BGP behavior and adjust following parameters • The interface input queues to be monitored and adjusted if drops are seen. Detailed below are some areas that can be tuned to improve BGP convergence. Route summarization for example. Fast Session Deactivation have aided in improved convergence without tuning the BGP timers. With sufficient available memory. as the routing information churn that takes place consumes additional memory. the number of maximum paths configured. This limit is defined as the Maximum Segment Size (MSS) and is 536 bytes by default. especially when TCP has a lot of data to transport like it does with BGP. the number of alternate paths available. and VPN configurations. in networks with lots of peering sessions and large routing tables. However. keepalive.0 15 . It is recommended to leave the BGP timers (advertisement-interval. click here. Extra memory will be needed during the initial startup for the BGP peers. community. Path MTU discovery Every TCP session has a limit in terms of how much data it can transport in a single packet. The solution is to dynamically determine how large the MSS value can be without creating packets that will need to be fragmented. BGP would utilize the Update Packing Optimization feature which would improve convergence. • TCP Receive window-size tuned on routers with large number of routes and routers in Long Fat Networks – Networks with large bandwidth and high delay. consider tuning the timers with caution.

Interface Input Queues Large numbers of interface input queue drops are a very common problem for routers with many peers and substantial number of routes.cisco. The increase in MSS from 536 to 1460 or 4430 bytes reduces TCP/IP overhead. please refer to the following URL: http://www. If it only traverses POS segments.a. PMTU Discovery will not work. the MSS will be 4430 bytes. This condition can overrun the SPD and interface input queues and result in updates being lost. then it is a good practice to set the TCP Maximum Segment Size to the lowest MTU size in the network using “ip tcp mss <value>” In some environments the firewalls or other devices may be configured (or misconfigured) to block the ICMP messages used for path MTU discovery. PMTU allows TCP to determine the smallest MTU size among all links between the ends of a TCP session. TCP will then use this MTU value. The intent of the “MinRouteAdvertisementIntervalTimer” is to reduce the overhead of the BGP routing protocol. the MSS will be 1460 bytes. the secondary path may have a smaller MTU which would result in fragmentation of the packets till the smallest MTU is found in the new path.html TCP Window Size In scenarios like high bandwidth-high delay networks (Long Fat Networks) and routers under stress. TCP Window Scaling feature can help minimize TCP retransmissions. BGP Best Practice Version 3. Please note that the PMTUD derives the smallest MTU on the best path (primary path) between endpoints and sets the MSS value for the TCP session. This rate limiting procedure applies on a per-destination basis. TCP acknowledgements are delayed.. Optimal values for interface input queue depends upon the number of peers and Processor capabilities. Default Interface Input queue size is 75. In such cases. This delay can lead to TCP retransmissions. Increasing the interface input queue depth will help reduce the queue drops. The SPD headroom can be changed using the global SPD Headroom configuration command. using the configuration command ip tcp window-size <bytes more than 65535> Minimum Route Advertisement Interval (MRAI) MRAI determines the minimum amount of time that must elapse between an advertisement and/or withdrawal of routes to a particular destination by a BGP speaker to a peer. In case of failure in the primary path.path-mtu-discovery (a. which helps BGP converge faster. there could be large number of TCP update and ACK packets. PMTU). This feature should be enabled by setting TCP window size to more than 65535 on both the neighboring peers. If this is not desired. For additional information on SPD. Use interface configuration command hold-queue <1-4096> in increase the queue size. If a TCP session only traverses Ethernet segments. although the value of “MinRouteAdvertisementIntervalTimer” is set on a per BGP peer basis.com/web/about/security/intelligence/spd. While the original intent of the implementation of the MRAI timer was to reduce the overhead of the BGP routing protocol .k. minus room for the IP and TCP headers. it also has additional benefits in can promote stability by batching routing changes and improve update packing in some scenarios. as the MSS for the session.0 16 . During network transient events. SPD headroom depth is shared by all the interfaces.

Fast Peering Session Deactivation The default behavior for BGP when the route to neighbor is lost is to wait for the expiration of the hold timer. A host route must be available for each peering session that is configured to use BGP fast session deactivation. This can also lead to substantial delays in convergence. If the router loses the route to the peer the session is deactivated. This feature can be configured using the BGP configuration command: neighbor x. Fast Peering Session Deactivation uses the Address Tracking Filter to register the route to the peer. For iBGP and PE-CE eBGP peers it is recommended the MRAI timer be set to 0. There have been actual measurements where this resulted in a single prefix withdrawal producing 41 BGP events a few hops away! Not only is the MRAI timer a potential source of problems. research has revealed that the implementation of MRAI. This may result in a multiple best path announcements to its neighbors as each update message is received and processed.x. Risks and Limitations As processing power of routers has increased. In a large BGP network this could cause significant route churn. Likewise some ASN operators may alter the default values. In a 4 router fully-meshed network the total MRAI delay for convergence would be 10-seconds for an iBGP mesh and 60-seconds for an eBGP mesh. as a transient convergence in the IGP would cause the unwanted deactivation of the peer. can be most helpful in multi-hop BGP scenarios.0 17 . These differences mean that update messages transiting different ASNs using different vendor equipment will arrive at the target router at different times.For an iBGP Peer the default MRAI is 5 seconds. so it should be confirmed with peer ASNs whether they are using route dampening prior to making this modification. This is achieved with the following command: neighbor x. and will consider each one for best path options. This feature is not recommended for BGP peers with redundant paths to neighbor.x. there is no consistency across vendor implementations. for an eBGP peer the default is 30 seconds. without waiting for the Hold Time to expire. UPDATE. One potential result of this is that a simple update message from one ASN would be seen as a multiple route flap event a few ASN hops away .x fall-over BGP Best Practice Version 3.0(32)S For eBGP peers this could cause dampening. or NOTIFICATION messages within the period specified in the Hold Time field of the OPEN message. but also differences in CPU loadings and CPU speed will result in different update times for prefixes announcements passing from router to router.x advertisement-interval 0 This was the default beginning with 12. or rather the disparate implementation of MRAI between vendors or ASN operators has actually contributed to network instability. This occurs when a peer does not receive KEEPALIVE. While the Cisco default for an eBGP peer is 30 seconds.x.when in fact there was no instability whatsoever. Fast Peering Session Deactivation’s ability to deactivate a peer immediately upon loss of the route to the peer.x. These differences will also contribute to the effects described above. This router will see these different messages.

the delay interval for next-hop tracking is configured to occur every 20 seconds under the IPv4 address family session: router bgp 65000 address-family ipv4 unicast bgp nexthop trigger delay 20 The default trigger-delay is 5 seconds. Events like internal network outages can cause some next-hop addresses to become unreachable.0 18 . No set commands or other match commands are supported.BGP Support for Next Hop Address Tracking (NHT) BGP routes do have next-hop addresses. Only match ip address and match source-protocol commands are supported in the route map. This can lead to oscillation of next-hops. Such BGP route(s) with unreachable next-hop(s) are nonfunctional but will continue to stay UP until next BGP scanner runs. Typically next-hop addresses are learned via an IGP. BGP Selective Address Tracking Undesirable routes such as aggregate address. BGP scanner runs every 60sec and will invalidate BGP paths that have unreachable next-hop. IPv4 multicast. NHT configuration is supported for IPv4 unicast. NHT will schedule BGP next-hop scan after a configurable Trigger Delay interval (default = 5sec). extending the convergence time to 60sec. When a next-hop address becomes unreachable (deleted from RIB). This task uses prefix lists and route maps to match IP addresses or source protocols and to restrict prefixes that can be considered as next-hop routes. Configuration Example In the following example. BGP Support for Next-Hop Address Tracking is enabled by default when a supporting Cisco IOS software image is installed. This feature is available in relatively newer IOS versions. The purpose of selective next-hop route filtering is to avoid using undesirable routes to the next hop. Next-Hop Address Tracking (NHT) feature monitors the reachability of next-hop addresses of BGP routes. This way NHT will invalidate the BGP route without waiting for the default BGP Scanner for 60 sec and hence will bring down convergence time from the 60 sec range to 5 sec range. next-hop address tracking is disabled under the IPv4 address family session: router bgp 65000 address-family ipv4 unicast no bgp nexthop trigger enable In the following example. BGP prefixes or a default route to the next hop can lead to invalid condition in BGP RIB. IPv4 tunnel and IPv4 VPNV4 not for IPv6. BGP Best Practice Version 3. It takes about 60 sec for alternate BGP paths to be installed in the routing table. which effects BGP convergence.

• The Route Refresh feature was introduced in Cisco IOS–12. which initiates the storage of inbound routing table updates and requires additional memory. Soft Reconfiguration and Route Refresh Background When BGP policy is changed. Guidelines Hard reset of a BGP session is disruptive to an operational network. The BGP Soft Reconfiguration feature pre-dates the Route Refresh feature and is more resource intensive. which had been advertised on the BGP session that is being reset. • There is no configuration required to enable the Route Refresh feature. • If a peer does not support the route refresh capability. If a BGP session is reset repeatedly over a short period of time due to multiple changes in BGP policy. In addition. This capability is negotiated at session initialization. then the only soft reconfiguration option is to use the neighbor soft-reconfiguration command. causing destinations to be unreachable and traffic to be black-holed. it is recommended that this feature be used in place of BGP Soft Reconfiguration to minimize memory requirements. • If BGP Soft Reconfiguration feature is configured on an active BGP session then hard reset of the session is required the first time for it to take effect. are removed from the remote peer. • The Route Refresh feature requires both peers for the session to support this feature. The resetting of a BGP session results in all prefixes received on that session being removed and withdrawn from any neighbors to which they had been advertised. even those that are not selected by the BGP Best Path process. it can result in other routers in the network dampening prefixes. • If both peers support the Route Refresh feature. The BGP Soft Reconfiguration feature and Route Refresh feature are both methods by which the route flapping effect of resetting a session can be mitigated. The BGP Soft Reconfiguration feature does not require both peers for a session to support the feature. Risks and Limitations • The primary risk with the BGP Soft Reconfiguration feature is the increased memory requirements on a BGP router that is performing inbound soft reconfiguration. potentially causing additional withdrawals to its other peers.0 19 .0(7)T onwards. • The Route Refresh feature has the advantage of not having additional memory resource requirements for storing all received prefixes. The Route Refresh feature is a direct replacement for the BGP Soft Reconfiguration feature. the BGP session needs to be reset in order for the new policy to take effect. BGP Best Practice Version 3. all prefixes. Benefits • The benefit of BGP Soft Reconfiguration and Route Refresh features is that both methods initiate routing policy changes without resetting the BGP session.

The configuration command to enable this feature is: router bgp xxxxx neighbor <Address or Peer Group> soft-reconfiguration To soft clear a BGP neighbor. in some cases drastically. even if they are denied by the inbound policy. This reduces the resource requirements for memory. If the soft reconfiguration feature using stored routing table updates is configured for a neighbor. Details Outbound soft reconfiguration Performing outbound soft reconfiguration does not require any special configuration or have any additional memory requirements. When a session is soft cleared a new update is created. The inbound soft reconfiguration feature enables a BGP router to retain all inbound prefixes. The delta between the old update previously sent to the peer and the new update once the session was soft cleared is sent out to peers in the form of advertisements and withdrawals.0 20 . Route refresh capability advertisement is elaborated on in RFC2918. the following command shows the capabilities supported for the session: show ip bgp neighbor <Peer Address> Neighbor capabilities: Route refresh: advertised and received (old & new) If you configure Inbound Soft Reconfiguration (to verify which updates has been denied for example) on a peer where Route Refresh was negotiated. the Route Refresh feature will not be used. the Route Refresh feature is not used. Route-refresh The Route Refresh feature allows a BGP speaker to request routing update from the peer when a session is soft cleared. Inbound Soft reconfiguration When a BGP router receives updates that are denied by inbound policy. The BGP Inbound Soft Reconfiguration must be explicitly enabled on each peer or peer group. BGP Best Practice Version 3. These updates are not used in the BGP best path selection. • The BGP Soft Reconfiguration feature and the Route Refresh feature are mutually exclusive. all of the BGP updates from the remote peer must be reprocessed. the following exec command is used: clear ip bgp <Peer Address> soft <in|out> If the BGP Soft Reconfiguration feature is configured. this peer will send a Route Refresh request to fill its adjacency RIB. To verify that Route Refresh is supported for a particular BGP session. These prefixes are marked (Received and Not Used) in the BGP RIB. This allows the BGP speaker to reprocess the updates from its neighbors through its inbound policy. When the inbound policy is changed. they are not retained.

This helps when the autonomous systems are merged together and the peering configurations need to be kept intact.cisco. http://www. “support for local- AS” allows a router to appear to be a member of another autonomous system.0 21 . BGP Support for Local-AS Background BGP is by default configured with a single autonomous system. Guidelines • Local AS feature can be configured only for eBGP peers. in addition to its real autonomous system.com/en/US/tech/tk365/technologies_configuration_example09186a00800949cd.shtml BGP Best Practice Version 3. It does not work for two peers in different sub-AS in a confederation • Local-AS cannot have the local BGP protocol AS number or the AS number of the remote peer • Local-AS cannot be customized for individual peers in a peer group Details The local-AS feature is typically used when merging two autonomous systems. Detailed example of the usage of this feature is given in the following URL. A new BGP feature.

• If this feature is applied to a group of peers ( using peer-groups or peer templates). The configuration of this feature is transparent to customer peering sessions. This is an optional keyword. Customer peering sessions can be updated later during a maintenance window or during other scheduled downtime This feature comes with the following keywords: neighbor ip-address local-as [as-number [no-prepend [replace-as [dual-as]]]] local-as: The Second autonomous system configured in addition to the real autonomous system. care must be taken to make sure there are no loops in the AS PATH. BGP Support for Dual-AS Configuration Background The BGP Support for Dual AS configuration feature extends the functionality of the “BGP support for the local-AS” feature by providing additional configuration options for customizing the autonomous system paths. Since this feature has the potential to modify or delete the AS_PATH information. as a general recommendation this feature may be used as a temporary measure only during migration. Guidelines • This feature can be configured for eBGP peers only. the peers cannot be individually customized • BGP prepends the autonomous system number from each BGP network that a route traverses to maintain network reachability information and to prevent routing loops. BGP Best Practice Version 3. • The existing local BGP protocol AS number or the AS number of the remote peer cannot be configured as Local-AS • Local-AS can be configured for individual peers or configurations applied through peer groups and peer templates. This is an optional keyword. This keyword is optional dual-as: Configures the eBGP neighbor to establish a peering session using the real autonomous-system number (from the configured BGP routing process) or by using the autonomous-system number configured with the “neighbor ip-address local-as”. no-prepend: This keyword causes the BGP process NOT to prepend the “local autonomous system” number to any prefixes received from its eBGP peer. replace-as: This keyword causes the BGP speaker to replace its real autonomous system with the configured “local autonomous system” on all prefixes advertised to its eBGP peer. allowing the provider to merge two autonomous systems without interrupting customer peering arrangements.0 22 . This feature cannot be configured for iBGP peers or between different sub-autonomous systems of a confederation. • If there is a requirement to use this feature for other purposes such as the one given in the following example.

At Customer XYZ domain: 3.3 4.5 AS 64512 AS 64512 AS 1234 Customer XYZ Customer ABC BGP Best Practice Version 3. Obviously the objective is full connectivity between the two customer networks. use local-as command to replace the AS 1234 with a private AS such as 65000 while peering with the upstream AS64521 2. it can remove the private AS’s using “remove private-as” command. Now the AS3 border router will have prefixes with only private AS numbers in its AS PATH within its domain. To achieve the full connectivity between the customers without changing the AS numbers on the whole domain.4321].1 3.4. Under normal circumstances.html In addition to the dual AS support usage. Note: Remove-private-as will only remove a private AS number if it is on the end of the AS Path. http://www.cisco. and not in the middle.3.64351. two customers (XYZ and ABC) having the same private AS numbers in their domain are merging their network. In the following example.3.com/en/US/docs/ios/12_3t/12_3t11/feature/guide/gtbgpdas.7.7. Figure 1 BGP Support for Dual-AS example 1. if you have the AS Path [1234. The following steps are required to achieve this. Since XYZ domain also has the overlapping private AS number AS64512. For instance. this feature is also used by customers on situations where same AS number is present in two different network domains.Details The following CCO document explains the BGP dual AS migration in detail.4 AS 1 AS 3 AS 64521 AS 4321 7. BGP speaker at AS64512 on one customer side would drop the prefixes received from other customer’s AS 64512 as the same AS_PATH is seen on the prefixes. this should be removed using “remove private as” command at the border router in AS4321 peering with AS1. using BGP protocol. At Customer ABC domain: 1. following can be done as one of the options Remove all the Private AS while exchanging prefixes between the two customers at the border routers of AS 1 and AS 3.5.5. This happens when large customers merge their network with same autonomous system numbers or when customers get private L3 VPN offerings from providers who also use the same private autonomous system number such as 65000.4.1.1.0 23 . On the border router at AS 1234. the full connectivity between the two customer network domains is not possible as the AS 64512 is present on both domains. remove-private-as will not remove 64351 from the AS Path. Now when AS 3 border router is sending the prefixes to the border router of AS1 in customer XYZ.7 5.

1. BGP Best Practice Version 3.1.7.5. no-prepend: This keyword causes the BGP process NOT to prepend the “local autonomous system” number 65000 to any prefixes received from its peer at AS64521.5. the following configurations would be used: On AS-1234 peering with AS-64521 router bgp 1234 neighbor 10.1.3.1.Step1: On the BGP speakers peering between AS1234 and AS64512 in ABC.7/32 coming from AS 64512 has a local-AS of 65000 and an originating AS of 64512.5/32 10. we see the route to 7.1.1.1/32 10.14 remote-as 65000 AS-64521#show ip bgp BGP table version is 6.3.5.7.4.7/32 10.1.1.26 0 1234 64521 i *> 5.1.0 24 . all the autonomous systems in the AS Path are private AS numbers.1.1. local router ID is 7.26 0 1234 64521 2 i *> 4. local router ID is 4.4.1.3/32 10.7.7.1.13 remote-as 64521 neighbor 10.1.7.7 Network Next Hop Metric LocPrf Weight Path *> 1.14 0 0 65000 i *> 7. On AS-64521 peering with AS-1234 router bgp 64521 neighbor 10.4 Network Next Hop Metric LocPrf Weight Path *> 5.1.5/32 10.13 local-as 65000 no-prepend replace-as The keywords and values: local-as 65000: AS65000 is the ‘local’ AS number configured in addition the real AS number 1234.4.7.14 0 65000 64512 i With these options configured.1.4.1. replace-as: This keyword causes the BGP speaker to replace the real autonomous system AS 1234 with the ‘local’ autonomous system AS5000 in BGP updates advertised to its neighbor.4/32 10.26 0 1234 64521 2 1 i *> 3. AS-64512#show ip bgp BGP table version is 52.26 0 0 1234 i Step 2: When the BGP speaker at the edge of AS3 receives the route from AS 64521.1.1.5.1.

7.1.5.1.1 remove-private-as BGP Best Practice Version 3.1.1.2 03i Step3: Now that the private AS’s are removed on prefixes from Customer ABC’s domain.2.1.1.7/32 10.0 0 32768 i *> 4.4.1. AS-1#show ip bgp BGP table version is 54. We can do at the AS4321 as it is closer to the originating private AS.0.0.1. all prefixes received from AS64521 show only private AS numbers in its path.2 03i *> 5.4.3/32 10.1 remote-as 1 neighbor 10.5.7.1.0 0 32768 i *> 3. local router ID is 1.1.1. AS-3#show ip bgp Network Next Hop Metric LocPrf Weight Path *> 1.1 remove-private-as The following output shows that all the prefixes received from AS3.2.1.4/32 10.1.1.1.2.5/32 10.3.3.2 03i *> 7.2 0 0 64521 i *> 5.3/32 0.4.1.1.7.1.4/32 10.1. This can be done either at the border router of AS4321 or AS1.1.1 0 01i *> 3.1 Network Next Hop Metric LocPrf Weight Path *> 1. have only Autonomous System 3 in AS_Path list.3.5/32 10. we have to remove the overlapping private AS 64512 from the prefixes originating at Customer XYZ.1.1/32 10. This is because of private AS numbers being striped off at AS3. On AS4321 peering with AS1: router bgp 4321 neighbor 10.1.1. “ remove-private-as” command can strip all the AS numbers in the entire path while sending the updates to AS 1.1/32 0.0.7.2 0 64521 65000 i *> 7.1.5.In the following output.1 remote-as 1 neighbor 10.0 25 .7/32 10.2.3.2 0 64521 65000 64512 i Since all the AS numbers in the AS Path are private AS numbers on prefixes received from AS 64521.4. router bgp 3 neighbor 10.0.2.5.2 0 03i *> 4.1.

BGP Infrastructure Security

BGP TTL Security Hack (BTSH)

Background
BGP, by default, sends packets to external neighbors with a TTL of 1 and accepts packets from external
neighbors with a TTL of 0 or higher. Because BGP will accept packets with a TTL of 0 or higher, this
makes it possible for an attacker to send packets to a BGP router from many hops away, as long as the TTL
is still greater than 0 when it arrives. Since the TCP tuple is easy to discover and an attack doesn’t need the
TCP sequence number, this presents a relatively easy attack vector.
The Generalized TTL Security Mechanism (GTSM, described in RFC 3682) protects BGP routers from
attacks sourced from devices which are not directly attached to the same segment as the BGP speaker.
When configured to use GTSM, BGP originates packets with a TTL of 255, and only accepts packets with
a TTL of 254 or higher. This TTL value is evaluated after the receiving router has decremented the TTL.

Benefits
Since forging the TTL of an IP packet is still considered not possible, the deployment of GTSM, BTSH
will protect directly connected eBGP peers from this various TCP based attacks.

Guideline
BTSH should be implemented in all eBGP peering sessions, with consideration for the fact that multihop
scenarios reduce its effectiveness dependant upon network diameter. As most eBGP peering are one hop
away, the use of BTSH can be quite beneficial.
This feature is disabled by default. If enabled it should be configured on a per-neighbor basis.

It’s important to consider the impact of GTSM on processor utilization. In many cases, deploying GTSM
will open the router to TTL based attacks, since every packet received must be punted to the process level
for TTL checking.

Risks and Limitations
While this provides robust protection in directly connected peer scenarios, the usefulness declines in multi-
hop eBGP scenarios, as this requires BTSH to be configured to accept a TTL < 254, exposing the router to
CPU-utilization attacks.

This feature is not supported for iBGP peers or peer groups. For detailed information refer to
http://www.cisco.com/en/US/docs/ios/12_3t/12_3t7/feature/guide/gt_btsh.html

BGP Best Practice Version 3.0 26

BGP Authentication

Background
BGP uses TCP as its transport mechanism which makes is susceptible to certain attacks using spoofed TCP
segments and potentially resetting BGP sessions by injecting bogus TCP resets.
BGP Authentication uses an implementation of RFC2385 which provides a TCP option for carrying a MD5
digest in a TCP segment. This digest functions as a signature for the segment as it uses information only
available to the two endpoints.

Benefits
The implementation of an MD5 digest to authenticate each TCP segment significantly increases the
difficulty in successfully implementing a spoof attack against a BGP session.

Guideline
This feature is disabled by default
BGP Authentication should be employed when feasible, taking the performance impact into consideration,
especially when BGP sessions use transport across “untrusted” network segments.

There is work currently underway to replace MD5 with alternate methods using HMAC/SHA currently in
draft states within the IETF. It is also possible as an alternative to MD5 to use IPSec to secure BGP
peering sessions.

Risks and Limitations
The performance hits associated with implementing this feature may inhibit its deployment. Testing has
shown measurable CPU impact associated with calculating the MD5 digest for BGP data segments,
additional details are available in RFC2385

BGP Maximum-Prefix

Background
The default behavior of BGP is for a router to accept unlimited number of prefixes advertised by the
neighbor.
Too many unplanned prefixes can overload BGP process and router resources which can potentially crash
the router.
This opens vulnerability in that a neighbor can send too many prefixes, causing the BGP process to
consume enough memory to impact the overall operation of the router, to potentially including crashing the
router.

BGP Best Practice Version 3.0 27

Benefits
The maximum-prefix option provides a method to limit the number of prefixes the router will accept from
a BGP peer, with an option to print a warning (log the limit being broken) or to close the session.
Additionally, there will be a Syslog alert triggered when the number of prefixes received exceeds a
specified percentage of the maximum prefixes value.

Guideline
This feature is disabled by default. When enabled, the default behavior is for peering sessions to be
disabled when the maximum prefixes value is exceeded. If the ‘restart-interval’ argument is not
configured, a disabled session will stay down after the maximum prefix value is exceeded. The default
‘threshold’ value is 75%. A Syslog alert is triggered after receiving 75% of configured value with
maximum-prefix command.
This feature should only be deployed on peering sessions where the number of prefixes being learned is
predictable and stable. It would be unwise, for example, to deploy this on a transit link in the global
Internet, as the number of prefixes being announced on the internet is continuing to grow. Keeping the
maximum number of prefixes accepted would be creating an ongoing administrative task for the network
team. Use of the restart option is not generally recommended as this could lead to session flaps.

BGP Best Practice Version 3.0 28

For example. dampening should not be applied to DNS servers.eecs. • It is advisable to apply route-dampening as close to the prefix being advertised as possible. follow the guidelines below. especially root DNS servers.umich. It was believed. For example.0 29 . The behavior of route flap dampening for routes is modified by four configurable parameters. a router will receive multiple updates pertaining to a destination network. at the time RFD was developed. Appropriate parameter selection is recommended depending on various factors like severity (frequency and duration) of flaps. and encourage network operators to clean up any routes they had which might rapidly flap. When there are multiple EBGP sessions.edu/~zmao/Papers/sig02.pdf Guidelines The RIPE and NANOG communities have agreed that RFD is not recommended anymore. This can lead to over-penalizing the network prefix. the same router will receive multiple withdraw messages.ripe. the decay half life parameter should be set to a time considerably longer than the period of the route flap it is intended to address. This should be taken into consideration when deciding dampening parameters.nanog. Route Flap Dampening Background Route flap dampening (RFD) is a mechanism developed in the 1990’s to help stabilize BGP within the Internet network. Applying dampening in such cases could result in loss of connectivity to name resolution services. This also helps ensure that after successfully repairing the problem related to prefix flapping.html http://www. or routes which changed state on a regular basis.pdf “ BGP Best Practice Version 3. dampening should be applied at the customer peering points in addition to applying it at upstream ISP peering points.org/mtg-0210/ppt/flap. Risks and Limitations The main drawback of RFD is well explained in the following study: “Route-flap Damping Exacerbates Internet Routing Congerence Sigcomm 2002 http://www. etc. prefix-length. If you still want to implement this feature. dampening parameters can be cleared on access routers. • It is recommended that certain longer prefixes that are critical for access be excluded from dampening. availability of alternate routes. When the destination network become unreachable. More recent research on the impact of RFD highlighted that this mechanism is more harmful than the issue it cures.net/ripe/docs/routeflap-damping. that widespread deployment of RFD would improve the overall convergence characteristics of the global Internet. Both RIPE and NANOG communities agreed that RFD is not a best practice anymore. • Apply dampening to inbound announcements from eBGP peers only. For example. http://www. This over penalizing can lead to suppressing the prefix unnecessarily for extended period of time. It should be noted that one time network down can be seen as multiple Downs depending on EBGP sessions.

When the timer expires. a route is assigned a penalty value of 1000 (a constant) for each flap. i.org/mtg-0706/Presentations/PhilipSmith-BGP. The accumulated is reduced every 5 seconds. The later is preferred since it is flexible and gives more granular control. By default. Basically. at a rate that the penalty is reduced by half every 15 minutes. If the value of the route’s accumulated penalties exceeds 2000. Click here for a Case Study on Route Flap Dampening. to be ignored for a period of time that depends on the duration of the flap. the route will be tagged with a history marker to indicate that it is known to be flapping. The following presentation provides also an overview of this issue: http://www. the route is suppressed until the penalty drops below 750. Generally. BGP Best Practice Version 3. Also the fact that each AS may use a different MRAI (Min Route Adv Interval) reinforces this behavior called withdrawal triggered suppression.pdf http://www. the route will be tagged with a penalty value. the route will be suppressed or dampened. When the penalty value crosses a threshold. the route will no longer be suppressed.net/ripe/docs/ripe-378.e. it shows that a single route flap (a withdrawn followed by an announcement) will make the route to be dampened further in the backbone because several updates path (an attribute change is considered as a flap) will be generated due to the next best path selection in the previous hops. This time will be extended for each additional flap that occurs.html Details The Route Flap Dampening feature of BGP causes a route that is flapping. This penalty value will be increased for each flap. Routes that flap constantly are suppressed (ignored) longer than routes that flap occasionally.0 30 . Click here for Configuring Route Dampening.ripe. The show ip bgp <route> router command will list the penalty and the state of the route.nanog. Click here for More Details on Route Flap Dampening. Route dampening parameters can either be applied to all prefixes with equal penalties or to longer prefixes with higher penalty than the short ones. when a route is withdrawn and announced in quick succession. Initially.. being announced and withdrawn constantly. The route will be tagged as suppressed when this happens and a countdown timer will be started to indicate the amount of time the route will remain in the dampened state.

If the best path was received from a client. A route reflector will only forward its best path to a client. because BGP updates now have to propagate through multiple routers (RRC-to-RRC relationships) to get from a BGP router. This produces some real scaling problems once you have more than a few BGP routers in your network. it will be reflected to all non-clients and other clients. Normal BGP speakers can coexist with route reflectors and route reflector clients. a route-reflector hierarchy serves to reduce the number of routing updates transmitted through the network.0 31 . Route reflectors and Confederations address this problem by allowing a router to advertise or reflect iBGP learned routes to other iBGP peers without requiring a full network mesh. all BGP speakers within an AS must peer directly with one another. the savings will be greater for routers lower in the network hierarchy. it is reflected to all clients if received from a non-client. which may have hundreds or even thousands of routers in their network. Typically. it follows that to have a complete set of routes. with each client physically connected to two route reflectors to ensure that BGP information continues to be reflected if there is a problem with one route reflector. This leads to less memory utilization and savings in CPU and BGP update generation. then. and to reduce the size of the BGP tables of BGP speakers within the network. one part at a time. The redundant configuration increases the number of physical TCP connections required compared to the single-route reflector configuration. with a small cost in convergence time. Route Reflectors Background To ensure any BGP router has complete routing information. help reduce the size of the iBGP mesh and the associated overhead. Guidelines • Divide the backbone into multiple clusters and use at least one route reflector and a few clients per cluster. Besides reducing the number of iBGP sessions. For larger ISPs. Because a BGP speaker will not send routes learned from one iBGP peer to another. and it will only forward the best path from all of its clients to each of its normal iBGP peers. perhaps. which allows the network engineer to reduce the size of the iBGP mesh. • Changing “next-hop” using a route-map on a route reflector can cause routing loops and it is not recommended unless absolutely required. Route reflectors. but still significantly streamlines the network topology compared with a full network mesh. A route reflector topology results in considerable savings in the number of paths stored on routers in the network. the number of peering sessions clearly becomes unmanageable and difficult to implement given the limited CPU and memory on routers. • Route reflectors can be configured in a redundant fashion. Clients can also peer with route reflectors in other clusters for redundancy. • Start by migrating small parts of the network. it is necessary for all BGP routers in an AS to have iBGP peering sessions with each other. because only the route reflector must support this feature. Clusters may be configured hierarchically–route reflectors in a cluster BGP Best Practice Version 3. Benefits Route reflectors add additional attributes to BGP updates within an AS to allow a router to reflect iBGP learned routes to other iBGP peers. This comes. Once the best path is selected.

it is important to assess whether iBGP peering should be formed over physical interface addresses or over loopback addresses. This provides a natural method to limit routing information sent to lower levels. however. particular attention must be given to the usage of same versus different cluster IDs on these redundant RRs. it is important to follow the physical topology when deciding where to place the route reflectors. all the clients were required to be fully meshed and “bgp client-to-client reflection” would need to be turned off on the route- reflector. The important drawback. This can happen in scenarios when a link failure prevents an RR from learning the same route as other RRs in the cluster. which may or may not be desirable depending upon the topology. they exchange updates through the route reflector.0 32 . • When deploying or migrating to a route-reflector topology. • Route-reflector redundancy is recommended in order to allow multiple RRs to peer with the same RR Clients. • When route-reflector clients (RRCs) peer with multiple route reflectors (RRs). When RR-RRC peering is over loopback addresses. instead. However. Configuring peer groups within such a cluster could cause a withdrawal to the source of a route on the route reflector to be sent to all clients in the cluster. Clients inside a cluster do not have direct iBGP peers. because . The benefit of configuring multiple RRs with same cluster ID (which means they are in the same cluster) is that less memory is required in the router. can be clients of route reflectors in a higher level. is that routing can be disrupted in case of link failure. if peer groups were used for clients of a route reflector. Click here for RFC 2796. • As a rule of thumb. Peering over physical interface addresses allows a physical link failure to tear down the BGP session thereby switching the RRC over to its “backup” RR. BGP Best Practice Version 3. the route reflector should be used only as a “route server” and not for switching high volumes of traffic or for other services. Risks and Limitations With earlier versions of IOS. a physical link failure may retain or re-establish the iBGP session from an RRC over to the same RR via an alternate path. because multiple updates from within the same cluster are not kept after being received.

0 33 . BGP Best Practice Version 3. Details Route Propagation in Confederation Route propagation decisions are fairly similar to Route Reflectors—special treatment of AS paths is the key distinction. • iBGP speakers in sub-AS should be fully meshed unless a route-reflector model is used. the AS numbers would remain the same. MED. • Confederations are better to use in scenarios involving AS mergers. resulting in a CONFEDERATION_SET. or outside the confederation). • By default (i. • The sub-autonomous systems within the confederation may use private AS numbers. in the absence of any configured policy). but Cisco does not implement this. and NEXT_HOP attributes are forwarded without change. • If you already have large scale deployment of confederations keep using them and use RR within sub- AS to reduce iBGP mesh. the Confederation AS. • Because the NEXT_HOP is unchanged. Confederations Background Some of the key characteristics of confederations are: • They are visible to the outside world as single AS. • Route reflectors are much easier to migrate as you don’t have to deal with sub-AS numbers.e. the LOCAL_PREF. pre-append the confederation ID to the AS-PATH attribute. we can conclude that confederations are expected to run a single IGP. Guidelines • Both confederations and route reflectors can be scaled to a good extent. • For routes learned from any external peers (sub-as. then pre-append the local sub-as number in the CONFEDERATION_SEQUENCE part of the ASPATH. send only to external peers. • For routes learned from other peers within the local sub-as. If the external peer is outside of the confederation. • The total number of neighbors is reduced by limiting the full mesh requirement to only the peers in the sub-AS. This would generally be done when migrating a single AS to a confederation. rather than public ones. remove the CONFEDERATION_SEQUENCES or CONFEDERATION_SET attributes. If the external peer is another sub-as. but when merging autonomous systems into a single AS by combining them into a confederation. send the routes to all neighbors. and forward the route. • According to the BGP RFC sub-AS can aggregate.

All BGP peers on a router must be reset if it needs to be assigned a new AS number. BGP Best Practice Version 3.Confederations require more configuration work upfront and usually require network downtime to activate. once the initial configuration work is complete. While confederations help reduce iBGP mesh themselves. However. confederations give network operators a greater degree of management flexibility. they can also be used with route reflectors.0 34 . It is difficult to transition to or from confederation because of multiple AS numbers involved.

it is highly recommended that prefix filtering be configured to limit the prefixes allowed into the IGP. only eBGP routes are redistributed into an IGP. BGP Best Practice Version 3. by default iBGP learned prefixes will not be redistributed. it is usually recommended to filter the redistribution to prevent uncontrolled prefix injection from destabilizing the entire network. Hence. the number of prefixes may be more than the IGP is able to cope with. This also includes redistributing between BGP and IGP protocols. The redistribution of iBGP learned prefixes must be explicitly enabled. in any case. • Be careful about routing loops that may occur when redistributing at multiple points. unless you’re using an IGP that doesn’t support default information originate.0 35 . The misconfiguration of a BGP to IGP redistribution point has been known to cause large- scale network outages and should be avoided as a general rule. such as EIGRP. When BGP to IGP redistribution must be performed. The redistribution of BGP into the IGP can cause network-wide instability. This is especially the case when full Internet tables are involved. Redistributing iBGP routes has a potential to create routing loops. Guidelines Avoid redistributing routes from BGP into an IGP whenever possible. • It is recommended to configure a default metric to the routes that are redistributed. The following command will enable the redistribution of iBGP learned prefixes: router bgp xxxx bgp redistribute-internal • When doing mutual redistribution. • Do not redistribute BGP routes matching on communities as it is unreliable and not supported. Even if prefix filtering is configured on the redistribution point. If BGP must be redistributed into the IGP. You should limit this only to the default route. BGP-IGP Redistribution Policies Background Cisco IOS supports redistribution between all IP routing protocols. as a protection mechanism. Typically. • Using default information originate would be the preferred mechanism. Benefits The main benefit of redistributing BGP into the IGP is to provide dynamic route advertisement of BGP learned prefixes throughout the network without requiring a pervasive BGP environment. Risks and Limitations Interior Gateway Protocols were not designed to handle the large number of prefixes that are typically found in BGP. The redistribution of BGP into an IGP is typically discouraged. ensure route propagation is fully controlled in either direction otherwise it can cause routing loops.

In cases where a prefix is learnt via eBGP and subsequently announced via iBGP the default behavior listed above creates a requirement for the external nexthop to be learnt via an IGP. If there is a requirement to modify the next hop of reflected routes or iBGP learned routes this must be accomplished with an outbound route-map. the next-hop is not modified • When a route-reflector reflects a route learnt from a router-reflector client (RRC) the next-hop is not modified. • When a route-reflector propagates a route learnt from an iBGP or eBGP peer the next-hop is not modified. associated issues and best practices to address them. BGP Best Practice Version 3. Next Hop Behavior with Route Reflection By default a Route Reflector will not modify the next-hop attribute of reflected (iBGP learnt) routes. else the prefix will not be installed as the next-hop will be unreachable.0 36 . the next hop must be reachable before a router will insert the destination prefix into its routing tables. Guidelines Some methods for ensuring next-hop reachability for externally learnt routes are: • Redistribute a static route to the eBGP peer into the local IGP • Redistribute the connected or static route which is used to reach the eBGP peer into the local IGP • Run the local IGP on the interface which is used to reach the eBGP peer advertising the route (normally the IGP would be configured not to build any neighbor adjacencies on this interface) • Set the next hop to the BGP peering address of the BGP speaker within the local AS which is receiving the routes When advertising to an iBGP peer. • When a router announces an eBGP learnt route to an iBGP peer. Cisco implementations allow a RR to modify the next hop for eBGP routes being reflected to route reflector clients by configuring the bgp next-hop-self command. Default next-hop behaviors: • When a router announces a prefix to an eBGP peer. In any peering scenario. BGP “next-hop-self” feature modifies next hop of only the eBGP learnt prefixes and not those learnt as an RR client or from iBGP peers. the next-hop is modified to the local peering interface – normally the connected interface address. Background The BGP next hop attribute is the next hop IP address to use in order to reach a destination prefix. BGP Next-hop This section describes how next-hop reachability works in BGP.

and the neighbor next-hop-self command should not be used to modify the next hop attribute for a route reflector when this feature is enabled for a route reflector client. use the option ‘allpaths’ keyword. Setting BGP attributes for a route reflector should be attempted only by an experienced network operator.0 37 . • Configure inter-AS Multi Protocol Label Switching (MPLS) Virtual Private Networks (VPNs) by not modifying the next hop attribute when advertising routes to an MP eBGP peer. To propagate the next-hop of an iBGP learnt prefix without modification. “Next-Hop-Unchanged” feature To enable an eBGP peer to propagate the next hop unchanged. routing loops. • Turn off the next hop calculation for an eBGP peer. use the neighbor next-hop-unchanged command in address family or router configuration mode.• Modifying the next hop at a Route Reflector can cause routing loops and is only advised when the network architecture requires it. which can be used with the “iBGP Multipath” Load Sharing feature to configure load balancing. Except in certain inter-AS MPLS VPN scenarios this command should not be configured on a route reflector. This command can be used to accomplish the following: • Bring the route reflector into the forwarding path. BGP Best Practice Version 3. or a loss of connectivity. Incorrectly setting BGP attributes for a route reflector can cause inconsistent routing. This feature is useful for configuring the end-to- end connection of a label-switched path.

AS path. • Assigning multiple communities to a prefix can be used to build a community “string”. In general. like attributes such as local preference. a BGP community is defined as a group of prefixes which share some common property. • The BGP Named Community Lists feature allows the network operator to assign meaningful names to community lists. Guidelines It is considered a common practice to group prefixes into communities based on classifications such as the following: • Type of customer or peering AS. or a filter based on complex regular expressions. • Building filters at an AS exit point based on the AS from which the prefixes are learned would normally involve building a large and complex AS filter list. The community scheme can significantly simplify the configuration required to control distribution of routing information. prefers or distributes to other neighbors. For example. A router has the option to add or modify a community attribute before it passes the attribute on to other peers.0 38 . This allows other routers to act based on one. those that receive full-routes versus those that receives partial (direct customer-only) routes. This feature also increases the number of community lists that can be configured by a network operator because there is no limitation on the number of named community list that can be configured. A BGP speaker may use this attribute to control which routing information it accepts. MED. some or all of the attributes. etc. BGP Best Practice Version 3. • The community attribute provides more flexibility than many other prefix attributes for managing policies because it does not form part of the BGP best-path algorithm. this filtering becomes much simpler. however. communities can be used as flags in order to mark a set of prefixes. Community Attribute Background BGP communities are used to simplify the control of routing information by providing a mechanism to group prefixes and make routing decisions based on these groupings. If all the prefixes to be filtered at a specific exit point are marked with a single community. Communities are generally leveraged by the routing policy engine to set and/or modify other attributes in order to influence best- path decisions. Benefits The biggest advantage of the BGP community attribute is it provides a scalable method of implementing routing policy. • Prefixes learned from customers. While communities themselves do not alter the BGP decision making process. • Prefixes learned from ISPs or peers. • Prefixes with identical routing policies.

it is advisable to use a prefix- list as a “backup” for the primary community filter. RFC1998 describes a way in a multi-homed environment to indicate which of the ISPs is primary and secondary for a particular set of routes. Communities are carried across AS boundaries (transitive) only if send-community has been configured for the neighbor. use the “additive” keyword. any router belongs to it. and the second two octets may be defined by that autonomous system. which contains all communities from all of the aggregated routes. community is not sent by default – you need to enable it on a per-neighbor basis. hexadecimal. Basic Configuration Below are basic configuration steps for setting BGP community attribute to receive BGP routes. Cisco implementation supports decimal value for BGP community attribute. and not send them for eBGP. • Prefixes in VPN (BGP community is fundamental to the operation of MPLS VPNs. The “well-known” communities should be understood and obeyed by all BGP4 implementations.0 39 . Router(config)#ip bgp-community new-format There are well-known BGP communities defined by RFC 1997: • no-export: Do not advertise to eBGP peers. When configuring community attribute. then the resulting aggregate should have a COMMUNITIES path attribute. To remove the existing community attribute. To use and display the newer format AA:NN. BGP Best Practice Version 3. This does nothing more than to help protect the Internet against possible mistakes. use “set community none”. internal or external. where AA is AS numbers and NN is an attribute assigned by the BGP AS administrator. where the number of prefixes in the network is small. With release 12. and then build the outgoing policy lists based on simple communities. we need to use the ip bgp new-format global configuration command. Implementation propositions are underway to have the default set to send communities for iBGP. Useful to clear any communities associate with a route.) It is better to assign prefixes into communities at the edge of the network. verify if any community tags already exist. Keep this route within an AS. • None: No community attribute. When advertising routes to the ISPs it is advisable to configure communities in agreement with the ISP.0. For historical reasons. By default. Details RFC 1997 defines a BGP community as a transitive attribute with 32-bit binary value that can be applied to BGP routes. RFC 1997 suggests the first two octets be an AS number presumably of the originating domain). and AA:NN. In smaller installations. By default. If a range of routes is to be aggregated and the resultant aggregates attribute section does not carry the ATOMIC_AGGREGATE attribute. To append to the existing community string. communities play a crucial role in identifying families of routes within VPNs. the existing community string will be OVERWRITTEN. where the first part is the AS number and the second part is a 2 byte number. if you use “set community AS:N” in a route map. • Internet: Advertise this route to the internet community. For example. • local-AS: Do not send outside local AS in confederation (special case of no-export). An enterprise network can use primary outbound filters based on communities to send to its ISP all of the routes originated in its network. BGP community can be configured in 3 different formats: decimal. • no-advertise: Do not advertise this route to any peer.

3.3.3. Community-lists A community list is a group of communities which can be used in a match clause of a route-map.0.0. and community-lists now apply policy based on the set community attribute.3.1 route-map setcommunity out ! route-map setcommunity match ip address 1 set community 200:200 ip access-list 1 permit 0.0 255.3.3.0 neighbor 3. this allows filtering or the setting of attributes based on different lists of community numbers.3 route-map check-community in ! ip bgp-community new format ! route-map check-community permit 10 match community 1 set local-preference 200 ! route-map check-community permit 20 match community 2 exact set local-preference 90 ! route-map check-community permit 30 match community 3 ! ip community-list 1 permit 100:20 ip community-list 2 permit 300:200 ip community-list 3 permit internet ! BGP Best Practice Version 3.3. Configuration Example RTR-B# ! router bgp 300 neighbor 3.0 40 .1 remote-as 300 neighbor 3. Earlier configuration that we saw set the community attribute.255.3.255 It is important to note that without neighbor x send-community command.0.3.3.3 remote-as 200 neighbor 3.1 send-community neighbor 3.10. community attribute will not be sent for those routes.255. RTR-A# ! router bgp 200 network 160.

they will not use the community-list with Internet keyword. In this case. any prefix that has 100:20 in the community attribute matches list 1. These drafts also provide a structure to the community attribute to address issues like route target and route originator indicators that help in associating routes to a L3VPN. It is important to realize what our policy is. else they will be filtered. Named community-lists do not have the limitation of 100 community groups that exist with standard and extended community lists. On the other hand. both customers and transit.2(8)T and 12. Community-lists work just like access-lists and it important for all other routes to be allowed after applying community lists.0(10)S. The keyword exact indicates that the community should only consist of 300:200 and nothing else. ISPs want tight control over routes they take into their network and in that case. Extended BGP Communities For applications which require larger communities to provide grouping of routes within BGP. BGP extended communities are 64 bits instead of the current 32 bits. and only allow routes that they chose. Since Internet community means all routes. there are multiple drafts on extended BGP communities. A named community list can be configured with regular expressions and with numbered community lists. This feature introduces a new type of community list called the named community list.0(16)ST. 12. BGP named community lists allow the network operator to assign meaningful names to community lists and increases the number of community lists that can be configured. Named BGP Community-lists With release 12.In the above example. BGP Community Design Case Study This section covers a BGP community design case study to help better explain BGP community attribute use in BGP deployments. Cisco introduced the named community-list feature. note the last community-list command. using AS 10 that services customers in single connection mode and multiple connection mode with other connections used for backup or load balancing purposes. 12.1(9)E. and has the added complexities of aggregate routes and routes for internal networks or affiliates. This ISP has many peers. For more information.0 41 . please refer to extended communities draft on IETF web site. Any prefix that has only 300:200 as community matches list 2. All rules of numbered communities apply to named community lists except that there is no limitation on the number of community attributes that can be configured for a named community list. and use appropriate implementation. Consider an ISP. The ISP has finalized the following community attributes for various route types: Table 1 BGP Community Design Route Type Community BGP Policy * ISP internal 10:50 Local Preference 100 routes Peer Routes 10:100 Local Preference 85 Preferred Peers 10:150 Local Preference 90 Customer 10:200 Local Preference 95 Routes - BGP Best Practice Version 3. this command allows all other routes to come through.

79.255. customer routes and affiliate routes.255. only aggregate should be advertised to peers. Advertised to peers Local Preference 95 ISP assigned 10:500 Aggregated towards peers.79. • If an ISP has preferred path through other ISPs for specific destinations. Specifics Customer routes 10:400 Customer routes that are . • If customer routes can be aggregated. * BGP Policy is unique to each router based on its BGP connections The BGP routing policy for the ISP has following salient points: • All core routers should have all the routes including peer routes.6 route-map set-community in neighbor 202.9.0 255. Configuration on a peering router router bgp 10 aggregate-address 129. • ISP assigned customers routes should be aggregated towards peers.9. • If customer routers are dual homed (connected to multiple ISPs) and have specific routes advertisement and AS-path prepend requirements.6 remote-as 201 neighbor 202.6 route-map peer-community out neighbor 202.33 send-community exit ! ! AS-Path ACL 71 permits all routes from peering ASs ! ip as-path access-list 71 permit ^201_ ip as-path access-list 71 permit ^651_ ! ip bgp-community new-format ip community-list 100 permit 10:50 10:200 10:400 10:500 ! BGP Best Practice Version 3.9.200.6 send-community ! neighbor 199.79.79.0 42 .0 as-set summary-only neighbor 202. • Customers with IANA assigned AS numbers and IP addresses should be advertised toward peers.200.203.168. then specific routes should be advertised to peers.168.33 route-map peer-community out neighbor 199.Aggregates aggregated.168. local customer routes preference 100. • Some customers that are single homed should get only the default route. Private AS’s and private IP addresses should not be advertised to peers.200. • Multi-homed customers have two options: either full Internet routing table or just selected ISP routes and some other critical routes for optimal routing.200. and those peers should be preferred for those destinations.9.168.168.33 remote-as 651 neighbor 199.33 route-map set-community in neighbor 199.

1 remote-as 796 neighbor 144.1 route-map customer-in in neighbor 144.70.70.1 route-customer-out out exit ! ip community-list 100 10:50 10:100 10:150 10:500 ! access-list 1 permit 144. route-map set-community permit 10 match as-path 71 set community 10:100 set local-preference 85 ! route-map peer-community permit 10 match community 100 end Configuration on a customer router The following example shows a scenario with the customer getting full Internet routing table and customer routes getting aggregated.0 43 .7. router bgp 10 neighbor 144.7.0 route-map customer-in permit 10 match ip address 1 set community 10:400 set local-preference 95 ! route-map customer-out permit 10 match community 100 end BGP Best Practice Version 3.0.70.70.7.

A prefix is synchronized in BGP if there is a matching prefix in the IGP. Synchronization should not be used to prevent the transiting of traffic through a non-transit autonomous system. The command to disable this feature is: router bgp xxxxx no synchronization Note that synchronization is off by default in recent IOS releases. Cisco recommends that BGP synchronization be disabled. Some form of route filtering should be used instead.0 44 . If it is enabled. but in more recent versions. BGP synchronization is enabled in older versions of Cisco IOS Software. BGP speakers will not advertise routes learned from an iBGP peer unless the destination described in the route is also reachable through the local IGP. it is disabled. BGP Best Practice Version 3. the prefix will not be inserted into the routing table and will not be advertised to external peers. Guidelines It is common practice in virtually every BGP network to disable this feature. The concept of a synchronized prefix is only relevant to iBGP learned prefixes. The purpose of the synchronization feature is to prevent traffic from being black-holed in networks that have non-contiguous BGP speakers. If a BGP learned prefix is not synchronized. • There are non-contiguous BGP speakers. • BGP prefix routing information is being redistributed into the local IGP. it is unlikely the BGP prefix would ever be able to become synchronized. This means that the transit path between two iBGP peers contains routers that are not running BGP. Synchronization Rules Background BGP synchronization feature refers specifically to prefix synchronization between BGP and the IGP. Benefits The BGP synchronization feature is only beneficial in very specific environments. It is seldom that a network will benefit from BGP synchronization being enabled. If no BGP prefix information is inserted into the IGP. The default setting for BGP synchronization is to be enabled on older versions of IOS. in which two requirements must be met.

Benefits There are a few primary reasons for multi-homing: • Reliability Internet connectivity has become a mission critical service in many environments. The term multi-homing has become quite common. This is commonly done through the use of different providers to provide a more diverse selection of paths to reach destinations. This is highlighted in the case of two Service Providers that may be using the same physical equipment to deliver a circuit to the Enterprise. Multi-homing (when done correctly). It is in this capacity. The performance of the Internet connectivity can be enhanced through multi-homing. This is commonly done through the use of different providers to provide a more diverse selection of paths to reach destinations. • Optimal Routing The performance of connectivity to the Internet connectivity can be enhanced through multi-homing. Customers usually prefer paths to different providers for improved redundancy reasons. Multi-homing Background Many enterprises require continuous connectivity to the global Internet to provide email. More bandwidth does not necessarily mean increasing the pipe. one of the following multi- homing scenarios can be used to provide Internet connectivity to an enterprise network. and business to business applications support. will provide the redundancy needed to ensure reliable service delivery. The use of email and the web have become tightly integrated into the world economy and therefore are critical to the way business is done. In order to support this continuous connectivity. so does the need for more bandwidth from an enterprise to the outside world.0 45 . BGP Best Practice Version 3. Keep in mind that multi-homing does not always provide physical path redundancy. There would be logical redundancy but not physical. This may be accomplished via multiple paths to a single provider. but highly redundant connectivity. So what does it mean to be multi-homed with respect to Internet connectivity? A network is multi-homed when it has more than one path to reach the Internet. It can also be achieved by increasing the number of links to the service providers and hence load share the traffic. Multi-homing Scenarios Depending on the criticality of the connectivity and bandwidth requirements. • Increased Bandwidth requirements As the traffic requirements grow. that BGP is most commonly seen in enterprise environments. The tight integration of day-to-day business communication with the Internet has made Internet connectivity a mission critical service. or multiple paths to different providers. connecting to the Internet. web browsing. The requirement for Internet connectivity is not just for connectivity. many enterprises have turned to multi-homing.

At most partial routes can be requested if needed. This is commonly achieved by using an as- path access list. • Some form of route filtering should be used to prevent the single router within the stub autonomous system from becoming a transit path for the peering AS. connectivity to the Internet will be completely lost. This type of multi-homing may be done via BGP Best Practice Version 3. the enterprise should typically receive a default route to the service provider. • The provider will configure a static route for the customers prefix.0 46 . Receiving the full Internet routing table is not necessary in a stub environment. • In the stub environment. Stub Network Multi-homed: Multiple Border Routers This is a scenario where an enterprise network connects to the Internet using multiple border routers via a single service provider to achieve load sharing and resilience. The enterprise uses a single router to achieve this connectivity. Stub Network Multi-homed: Single Border Router This is a scenario where the enterprise connects to the Internet through a single service provider and (preferably) through different physical infrastructures. multiple static routes are used. Guidelines BGP should be used for better traffic control in both directions thus achieving better load sharing across both links.Stub Network Single-homed Guidelines A single-homed stub network is one that connects to the Internet using a single connection to the service provider. However. Risks and Limitations Router failure will result in full loss of connectivity. • A single-homed stub network does not require the use of BGP. The use of multiple connections in this design assumes they are all connected to the same routers. if the Enterprise CPE or service provider Access Router fails. This is due to the single point of failure associated with this design strategy. Risks and Limitations This setup provides protection against a link failure and covers for partial loss of connectivity in the service provider network. • Private AS numbers can be used since the enterprise is connecting to a single service provider. • If multiple circuits are used between the provider router and customer router. The enterprise will configure a static default route.

This is commonly achieved by using an as-path access list. the default origination should be conditional on the circuit being up and active. Guidelines • BGP should be used for better traffic control in both directions thus achieving better load sharing across both links. Guidelines This design scenario requires the enterprise to obtain their own ASN from their regional registry. If additional prefix information is received from the upstream provider. partial routes.( Please note that there may be other disadvantages to using Tunnels on specific routers such as performance. different POPs of the same ISP and preferably through two circuits which do not share the same physical infrastructure (CO. all border routers should build iBGP adjacencies with each other through direct L2 link or through virtual interfaces such as GRE Tunnel. the traffic will not be black holed or looped. This way. fragmentation etc. Risks and Limitations In a rare event if service provider network is down completely due to a disaster. even if the transit routers do not have partial or full BGP routes. or can be received from BGP and redistributed into the IGP. The customer may accept a default route. If this is not feasible. This conditional advertisement can be based on a static default pointed at the interface. • Private AS numbers can be used because the enterprise is connecting to a single service provider. BGP Best Practice Version 3. Standard Multi-Homed Network: Single border router / Multiple ISP’s This is a scenario where an enterprise network connects to the Internet using a single border router and uses multiple links to multiple service providers. do not redistribute this into any IGP process running on the border routers. which should be investigated thoroughly before deciding to implement this solution) • The enterprise should then originate a default route from each border router. Depending on the resources available one of the following options can be considered.0 47 . Option 1 –Static Defaults The customer will use a static default route to each service provider. In order to prevent traffic from following a default route to a border router with a failed upstream circuit. • Each of the customer border routers should eBGP peer with the service provider. • Some form of route filtering should be configured to prevent the customer’s autonomous system from becoming a transit path for the service provider. or other elements).. or a full routing table from the service provider. The enterprise will also need a block of address space that is large enough to pass standard peering filters. Internet connectivity is lost. fiber. • All customer border routers and all transit routers interconnecting the border routers should speak BGP and establish an iBGP full mesh between each other. depending on the level of outbound traffic control the customer would like to achieve.

the traffic will be sent to the upstream provider whose path had the lowest ROUTER_ID. But it leads to sub-optimal routing because traffic destined for provider A or its directly connected customers may go through the other service providers. BGP Best Practice Version 3. Option 2 –ISP Provided Defaults The customer will use the default routes advertised by each service provider. This is preferred over static default routes. Option 4 – Full BGP Routes The enterprise can also receive full tables from both providers. you want to advertise a default via your IGP. Consideration has to taken with regard to how to advertise the routes within the enterprise AS. Traffic destined for other networks may still take sub optimal routing because the router uses the default route for these networks. This traffic can be balanced between the two if eBGP Multi-Path is used. This is commonly achieved by using an as-path access list.This option allows for a very even sharing of outbound traffic. Option 3 – Receive Partial Routing The next option is to receive partial routing information. If the AS_PATH is the same length. The customer should configure some form of route filtering to prevent the customer’s autonomous system from becoming a transit path between the two service providers. Page: 48 Again. This will result in the enterprise being able to correctly route traffic destined to either provider. The enterprise can request the upstream providers send only their locally originated prefixes and those prefixes for their customers. The following options require eBGP peering with the service providers. This can be resolved by asking the provider to send default + partial routes. This method results in the greatest resource requirements. This will allow the enterprise border router to send traffic to the upstream provider that is logically closest to the destination.0 48 . The customer should request a default route and a set of partial routes from the two service providers. Risks and Limitations Accepting full routing tables requires more resources (memory / cpu) on the border router. this design still poses a single-point-of-failure scenario whereby if the enterprise border router fails Internet connectivity is lost. This logical distance is derived from the AS_PATH. This option has the advantage that out going traffic from enterprise destined for the connected providers locally originated networks and their directly connected networks gets optimally routed. Typically. A default route directed at each provider would still be required to reach any destinations that are beyond the immediate upstream providers and their customers. Refrain from implementing unfiltered redistribution of BGP routes into the enterprise IGP. The customer should configure some form of route filtering to prevent the customer’s autonomous system from becoming a transit path between the two service providers.

0 49 . • All the enterprise border routers and the routers in transit between border routers should build a full iBGP mesh. • eBGP should be configured with each of the service providers. Option 2 Accept full routing updates from each upstream service provider. or work with the two service providers to obtain a block from one of the two service providers which the other service provider will agree to advertise. traffic will be directed to their network in an asymmetric way. • The customer should obtain a publicly assigned block of address from the national registry. Option 1 Request upstream service providers to send partial routes (which will include their local prefixes and their directly connected customer networks). This will prevent a layer 2 failure from impacting both paths to the global Internet. Any address block obtained should fit into the local policy for longest routable prefix length for both service providers. Risks and Limitations This option involves more complex configurations depending on the amount of level load sharing desired. • The enterprise will also need a block of address space that is large enough to pass standard peering filters as mentioned above. This option requires additional default routes on each of the border routers to reach all other destinations.Standard Multi-Homed Network: Multiple border routers In this scenario. The customer should configure a route filter of some type which prevents the customer’s autonomous system from becoming a transit network between the two service providers. If required. Depending on the resources available one of the following options can be considered. the two links to the two service providers will not share the same physical infrastructure. the attributes of the incoming updates can be altered to achieve required load sharing if the AS- PATH is the same. the enterprise network connects to the Internet using multiple border routers and uses multiple service providers for resilience and load sharing. Guidelines • This design scenario requires the enterprise to obtain their own ASN from their regional registry. This is commonly achieved by using an as-path access list. Preferably. BGP Best Practice Version 3. o The customer should be aware that if the service provider from which they have obtained an address block from aggregates the supplied addresses into a longer block. o The customer should be aware that some service providers will not advertise routes with a prefix length longer than some local policy. Static defaults can also be used in conjunction with full routes if needed depending on the eBGP connectivity.

Default static routes out each link to the service provider will do the job. The concerns in “Stub Network Multi-homed: Multiple Border Routers” for how to advertise the eBGP learned routes to the rest of the AS still apply. This situation requires that provider A also advertise a specific route along with the summary. Multi-homing Scenario Examples The following examples demonstrate the techniques described in the above section. single homed Let us assume an enterprise network with a single BGP router that connects to the Internet using a single SP. Provider A. This can lead to sub optimal inbound traffic when multi-homing to different service providers. BGP Best Practice Version 3. who owns the customer address space. These filters are used to allow only prefixes of certain length or less. When accepting full routes. sub optimal routing can occur or the enterprise may end up using only one service provider for the incoming traffic.0 50 . Service providers usually summarize the address space they assign to different connected customers and advertise only a summary to their customers and their upstream providers. Almost all of the service providers use filters at their public or private peering points. It is important to check utilization of resources in such cases since memory usage increases a lot. let us assume that there are two links from the border router to the SP router. Scenario 1: Enterprise stub network. may summarize the address space to a smaller prefix length and other providers may advertise the actual prefix (larger prefix length than the summary). If the prefix-length for the customer’s address space is not large enough to pass these filters. The illustration begins with a simple sub network with single ISP connection and builds over this setup to achieve more complex scenarios. In this scenario. additional resources on the devices are required. Load-balancing scenarios can typically be achieved when routes come from the same ISP. The service provider will also have static routes to the address space of enterprise and advertises this network to their upstream providers and its customers. In such case most of the inbound traffic to enterprise prefers provider B. Figure 2 Single-homed Enterprise Stub Network with Border Router This situation does not require the use of BGP.

Traffic is load balanced across both links. Scenario 2: Stub network multi-homed to single SP with single border router Figure 3 Stub Network Multi-homed to Single SP and Border Router This scenario assumes that the enterprise has a single border router with two connections to the same SP. Use of aggressive BGP timers instead f BFD is not recommended due to the problems witnessed by multiple customers and service providers. BGP should be configured to achieve load sharing and to achieve control of inbound and outbound traffic patterns. BFD support on the SP and Customer equipment needs to be checked. There is increasing trend to use of BFD is on Ethernet circuits between a data center and SP. By default the link failure detection can be delayed up to 3 minutes. The caveat of this design is that the single border router becomes a single point of failure. BGP Best Practice Version 3. This meets the additional bandwidth requirements and also protects against single link failure.0 51 . Bidirectional Forwarding Detection (BFD) on the circuits between the enterprise and SP may be required to cater to the needs of quicker link failure detection and BGP fallover to alternate path. The connections are terminated at the service provider in different locations.

Private ASN numbers can be used because of single SP used. BGP Best Practice Version 3.0 52 . All the enterprise border routers including the ones in the transit path should run BGP and should be fully meshed.Scenario 3: Stub network multi-homed to single SP with multiple border routers Figure 4 Stub Network Multi-homed to Single SP with Multiple Border Routers This scenario assumes that the enterprise has multiple border routers that connect to the same SP at different points of presence. Use of BFD may again be a requirement in this scenario as well subject to need and support both on SP side and enterprise side equipment. BGP should be configured to achieve traffic control and load sharing. Subnets of the enterprise space can be advertised from each border router to achieve inbound traffic load sharing. The enterprise border routers should originate a default and this default should be generated on the condition that the link to the SP is operational to avoid black holing and looping of traffic.

MPLS and Internet SP’s can be same SP as well. Enterprise should obtain its own AS number from the local registry but it can use private AS number if it is only connecting over an MPLS network to connect to remote branch offices with its datacenter and head office. Private IP addresses would suffice for internal enterprise connectivity between datacenter. This recommendation should be made subject to need and support both on SP side and enterprise side equipment. The enterprise can obtain partial internet routes or full internet routing table depending on the extent of load sharing that needs to be achieved and cost of equipment requirement.Scenario 4: Standard Network Multi-homed to Multiple ISPs with Multiple Border Routers Figure 5 Standard Network Multi-homed to Multiple SPs with Multiple Border Routers Inter-AS between AS100 and AS200 MPLS SP A Internet SP MPLS SP B AS 100 AS 500 AS 200 NAT NAT Enterprise AS 65001 This scenario is the most complex scenario in terms of connectivity. For this purpose. The more number of router. for external internet connectivity. static defaults may be required to achieve complete connectivity.0 53 . Use of BFD may again be a requirement at critical enterprise sites such as data centers as explained under scenario 2. Additionally. The large enterprise is doing multi-homing to the same SP as well as multiple SPs depending upon its geography as well as depth of network redundancy needs. higher the memory and processing power is needed at the enterprise router. BGP should be definitely used to achieve load sharing and redundancy at least at critical hub sites such as data centers. enterprise should obtain an address block from the Internet SP and use NAT at its internet gateways unless enterprise can afford to own large enough internet addressing space so that every device in the enterprise network has public IP address. etc. However. BGP Best Practice Version 3. and remote branches. head office. Customer has multiple border routers connecting to multiple SPs.

BGP will advertise a configured network to the service provider.0 54 . • If the enterprise is multi-homing to different service providers. BGP Best Practice Version 3. Perfect load balancing with any routing protocol is almost always impossible. Guidelines While optimizing traffic flows. this does not provide adequate level of redundancy. Inbound Load Balancing • If the enterprise is multi-homing to the same service provider. Each subnet can then be advertised out separate links to achieve the load balancing goal. what is really meant is managing configuration to attain optimal traffic patterns. Outbound Load Balancing Scenario 1 If the enterprise is multi-homed to a single service provider. customer can monitor a prefix and if the prefix vanishes. The strategies for load sharing inbound traffic and for load sharing outbound traffic are separate and often work independent of each other. simple default routes pointing out each of the links may achieve the goal. With an advertise-map. When load sharing is discussed. the address space can be broken down into smaller subnets. To achieve redundancy. as long as there are many distinct flows in the traffic mix. these smaller subnets can be advertised with different AS-PATH lengths to the ISP via different links. the network only controls the load sharing of outbound network traffic. Inbound load sharing can be achieved by manipulation of certain BGP metrics. AS-PATH can be pre-pended while advertising the address space to the service provider where heavy utilization is observed. Load Balancing Background Load Balancing is the ability to divide the traffic over multiple links to achieve even distribution. inbound traffic and outbound traffic should be treated separately. In reality. The goal is to achieve load sharing as close to equal distribution of traffic as possible. However. • Another way to influence inbound load balancing is to use an advertise-map. • Another option is to use communities to tag the enterprise address space prefixes in accordance with the service provider’s policies so that the provider can take further action based on their values while advertising them to their customers and upstream providers. and BGP is no exception. but this is still no guarantee of 50/50 load balancing.

This may achieve the required load sharing in some cases. with the eBGP session being sourced from a loopback instead of a physical interface. you need trial and error to decide this distribution) In each of the above cases. An eBGP session is configured between the two routers for each link. which are identical except for neighbor address from which the path was received. AS-PATH filtering can be used to achieve this. the following criteria must be met: . The eBGP sessions are tied directly to the interface addresses. it is generally a good practice to have default route along with more specific routes in order to achieve complete connectivity Scenario 3 If multiple links are used to create multiple eBGP sessions between the enterprise router and the service provider router for additional bandwidth and/or redundancy. Option 2: eBGP Multipath feature The eBGP multipath feature provides another solution to this problem.0 55 . iBGP multipath provides for load balancing iBGP traffic between iBGP neighbors. and Interior Gateway Protocol (IGP) distance. One of the following options can be used to achieve outbound load balancing in such a scenario with option (1) as a preferred solution. local preference. one for each link. . A static route to the remote loopback is configured for each interface. BGP Best Practice Version 3. Scenario 4 (iBGP Multipath) Similar to the eBGP multi-path feature. Option 1: eBGP Multi-hop feature eBGP multi-hop feature uses a single eBGP session between the two routers. All attributes must be the same. The other option is to accept full routing updates from each of the service providers and filter routes to receive mutually exclusive partial routes from each of the providers. The best paths or multi-paths are then installed in the IP routing table of the router. . For multiple paths to the same destination to be considered as multipaths. origin code.Scenario 2 If the enterprise is multi-homed to different providers. The iBGP multipath load sharing feature enables the BGP speaking router to select multiple iBGP paths as best paths to a destination. One is to accept partial routes from each of the providers and use defaults for other traffic. The attributes include weight. Multi Exit Discriminator (MED). The eBGP multipath feature will allow the router to install all paths up to the maximum-paths value configured. This provides the next-hop resolution and load balancing through recursive routing to the next-hop. The result is both routers receive multiple paths. there is a potential for using only one link because eBGP sessions are tied to the physical interface address of the link to the provider. Local-pref based on some prefix-distribution (usually. . autonomous system path (entire attribute and not just length). there are several options.

BGP load sharing over an MPLS core does not work properly in 12. This problem is not present in 12.CEF/LFIB picking an incorrect LDP label for the BGP next-hop. Load balancing almost always brings some amount of asymmetric flows with traffic for a flow exiting a particular link while traffic belonging to the same flow entering the other link so this should be accounted for while designing networks.g. Hence the distribution of traffic occurs in proportion to the egress bandwidth. The BGP Link Bandwidth feature is supported by the internal BGP (iBGP) and external BGP (eBGP) multi-path features. the process may need multiple iterations to achieve the desired load sharing. When a router receives a route from a directly connected external neighbor and advertises this route to iBGP neighbors. BGP Best Practice Version 3.0 56 . The fix suggested is to use "send-label" with BGP. The link bandwidth extended community attribute may be propagated to all iBGP peers and used with the BGP multi-path features to configure unequal cost load balancing.0S.0S. The link bandwidth extended community indicates the preference of an autonomous system exit link in terms of bandwidth. each via two outgoing interfaces as this requires in CEF two levels of recursion. . Also be aware of CSCsb52253 . At that time the whole exercise may need to be repeated.e.2S and IOS XR. which is not supported in 12. E. Risks and Limitations While achieving load balancing by multi-homing to multiple service providers. leading to a black-hole. the router may advertise the bandwidth of that link. Over time. Even if the criteria are met and multiple paths are installed. the BGP speaking router will still designate one of the multipaths as the best path and advertise only this best path to its neighbors. with two BGP next hops and two core links CEF is not taking all four paths (i. two paths to two next-hops. The next hop router for each multipath must be different. traffic flows may change due to changing applications and application connectivity requirements in the enterprise. **** Scenario 5 (Link Bandwidth) The BGP Link Bandwidth feature is used to advertise the bandwidth of an autonomous system exit link as an extended community.

The other way of doing this is to use AS-PATH pre-pending..0 57 .0/16 is advertised to ISP1 and 10.7. BGP will advertise a configured network to the service provider.. access-list 2 permit 10. Example router bgp 500 neighbor <R1> advertise-map am non-exist-map bb ! access-list 1 permit 10.0 is not learnt through ISP1. similar configuration can be done to advertise one ISP’s address space to the other and vice versa in case of a failure of either of the links. With advertise-map.15. This technique is also called conditional advertisement Figure 6 Use of advertise-map to influence inbound traffic When all links are working fine.0. BGP Best Practice Version 3. Make sure that 10. it advertises 10.15.0/24 also to ISP1.7.Details Using Advertise Map for Inbound Load Balancing A way to influence inbound load balancing is to use advertise-map.0/24 is advertised to ISP2. the enterprise router can be configured to advertise the address space of the other ISP2 to ISP1. In situations of multi-homing with two ISPs.0 !Advertise this when.10.0 !… this disappears route-map am permit 10 match ip address 1 route-map bb permit match ip address 2 This scenario achieves the purpose of inbound load sharing and in case of a failure the address space is still reachable via the other ISP.0.0.15.7. customer can monitor a prefix and if the prefix vanishes. When ISP2 link fails. Using this feature. The disadvantage of AS-PATH pre-pending is that it doubles the prefixes in the upstream providers’ BGP tables. advertise-map monitors the prefix of the link between the R2 and ISP2 and when the prefix vanishes from the routing table. 1.15.15.

1.6 remote-as 100 neighbor 10. Typically BGP chooses only one of the paths and the tie breaker is the router-id in case all other attributes happen to be the same. BGP can install up to eight equal paths in the IP routing table. BGP multipath when enabled installs both the routes in the routing tables and hence load balancing can be achieved. Figure 7 Using EBGP Multipath for Outbound Load Balancing Sample Configuration Enterprise Router router bgp 65100 no synchronization bgp log-neighbor-changes network 172.10 remote-as 100 maximum-paths 3 no auto-summary ! ISP Router router bgp 100 no synchronization bgp log-neighbor-changes BGP Best Practice Version 3. To overcome this issue.0.1.0 neighbor 10.1.2 remote-as 100 neighbor 10.Using EBGP Multipath for Outbound Load Balancing EBGP multipath can be used to load balance traffic across two links from an enterprise to the same ISP.0 58 .18. This requires that all other attributes for both the paths be the same.1. When enabled.1.1.

19.2.1. network 172. Unequal cost load balancing with BGP using link bandwidth and BGP multi-path is supported in 12.0.1 remote-as 65100 neighbor 10.0 neighbor 10. When used along with EBGP multi-path the bandwidth that is advertised to IBGP peers is the sum of DMZ link bandwidths of all EBGP multi-paths.5 remote-as 65100 neighbor 10.1. When a router receives a route from a directly connected external neighbor and advertises this route to iBGP neighbors.1.1.9 remote-as 65100 maximum-paths 3 no auto-summary ! Using BGP Link Bandwidth for Outbound Load Balancing The BGP Link Bandwidth feature is used to advertise the bandwidth of an autonomous system exit link as an extended community. The link bandwidth extended community indicates the preference of an autonomous system exit link in terms of bandwidth.0 59 .1. The BGP Link Bandwidth feature is supported by the internal BGP (iBGP) and external BGP (eBGP) multi-path features. The link bandwidth extended community attribute may be propagated to all iBGP peers and used with the BGP multi-path features to configure unequal cost load balancing. The attribute is stripped while sending to EBGP peers. Link bandwidth is propagated to IBGP peers only. the router may advertise the bandwidth of that link. Link Bandwidth is an extended community of type 0x0004.1. Figure 8 Using BGP Link Bandwidth for Outbound Load Balancing Sample Configuration R1# router bgp 100 bgp dmzlink-bw maximum-paths ibgp 6 BGP Best Practice Version 3.

10.4.1 send-community extended neighbor 172.6.1.5.1 dmzlink-bw Click here for Additional Details on BGP Link Bandwidth.6.4.1 send-community extended neighbor 172.R2# router bgp 100 bgp dmzlink-bw maximum-paths 6 neighbor 10.1 dmzlink-bw neighbor 172.1.10. BGP Best Practice Version 3.0 60 .5.1 dmzlink-bw R3# router bgp 100 neighbor 20.

The great advantage of using aggregate command is that the aggregate is dynamic (it may/may not exist and the attributes may change). if the as-set keyword is not used. To signify this. AS-PATH information from the more-specific routes is lost. For aggregates. you also need a static to null0. The command includes both net and mask. the atomic aggregate is not set. • ISPs can also use the network command to generate these summary routes. When the as-set keyword is included. Once set. on the other hand is less stable. Benefits Aggregation reduces the number of prefixes resulting in memory utilization savings since the size of the BGP and IP routing table is reduced. If you use a network statement to generate your summary routes. When a route is generated as an aggregate route using aggregate-address. • The first step in configuring route aggregation with BGP is to specify the aggregate-address command to define the aggregate prefix to be generated. so the aggregating router. Guidelines • An enterprise network’s address space should allow aggregation of its addresses to reduce the amount of extra information to be carried by each upstream provider. the router will fill in the AGGREGATOR attribute of the route with its own AS number and router ID. If you find someone is mistakenly advertising your address space as part of an aggregate. an address block with a lower mask is advertised in order to summarize the various sub-prefixes used in the autonomous system. Typically. An AS-SET includes AS-PATH information from all more specific routes that contributed to the aggregate. However. The network command will not do this. the AS sequence information associated with the contributing routes is meaningless. Aggregation of routing information also increases stability—aggregate stays even if specifics come and go. • When using the aggregate-address command. There is no need for the Internet to know about more specific routes in a particular network or AS. the AS path is still needed for loop detection. this attribute should never be removed by other routers that forward the route throughout the Internet. This can be useful for troubleshooting. • When using the network command. you can track down the AS and router that is doing it. they may choose to use the aggregate-address in order to include AS-SET and COMMUNITY summarization information in the aggregate route. Aggregation requires generating a stable summary (or supernet) route that covers all of the subnets in the network. Route Aggregation Background Aggregation is the process of combining the characteristics of several different routes in such a way that a single route can be advertised. However.0 61 . it causes the router to generate an AS-SET within the AS-PATH. Besides generating an AS-SET. This minimizes flapping of routes and eases the mandate of route dampening. Service Provider assigned address space aggregates routes from customers in order to minimize the size of the Internet route table. inserts all AS that BGP Best Practice Version 3. the route sets the ATOMIC- AGGREGATE attribute in the aggregate BGP route. the router will also include BGP community information from all of the more specific routes in the community attribute of the aggregate route.

and traffic to the other will come in through the other link (with both links providing a backup to the other). also known as proxy aggregation. IP access lists and autonomous system path access lists match clauses are supported. the summary-only keyword needs to be used. BGP Best Practice Version 3. or can be established by the network router configuration command. the provider may have to leak the "holes" of the non-provider /24’s back into it’s AS and to all its customers to ensure reach ability. • By default. can be redistributed from an IGP. Specifically. the route is suppressed. at least one valid sub-prefix must be present in the routing table for the aggregate to be considered valid. it means that it was learned from AS 60. Note that the deny logic here does not prevent the route from being advertised. For example. BGP aggregation applies to routes that exist in the BGP table. • Use a route-map to adjust other BGP attributes. rather. it prevents it from being suppressed. The way aggregates are formed and advertised and whether they carry with them more specific routes will influence traffic patterns and sizes of BGP routing tables. An aggregate can be sent if at least one more specific route of that aggregate exists in the BGP table. if a route has an AS PATH of 60 50 {10 20}. and then puts it’s own AS number. {1 2 3}. To do this. Advertising summarized information to an upstream provider could result in sub-optimal routing decisions for inbound traffic through the provider into the network. Details By default. the aggregate address command does not filter out the more specific BGP routes. the aggregate address command does not filter out the more specific BGP routes. When aggregating. but this default setting should be turned off using no auto-summary in networks relying on CIDR and VLSM since it takes control of summarization out of the hands of the engineer. if a network connects to ISPs in two different locations. into the AS SEQUENCE .0 62 . the MED attribute can be set so that traffic for one aggregate will come in through one link. Also. aggregate was generated by AS 50. When a route is permitted through the suppress map. Using the suppress-map keyword creates the aggregate route but suppresses advertisement of specified routes. This situation calls for specialized configurations to ensure connectivity to all the components of the /16 inside and outside the provider AS. For example. say 4. The suppress-map is another form of route-map that can be used to indicate the more specific routes to be suppressed or the more specific routes to be allowed. The more specific route can be injected in the BGP routing table by incoming updates from other AS areas. contribute to the aggregate route into the (unordered) AS SET say. Risks and Limitations BGP will automatically summarize on classful boundaries. • In certain instances the provider will do the aggregation for the network. Different methods of aggregation can be used depending on how the summaries and specifics need to be propagated. the summary-only keyword needs to be used. • For the aggregate address to enter the BGP table there must be at least one specific route in the BGP table. This is applicable in a scenario where the network is multi-homed to a single provider at different locations or multi-homed to different providers when the enterprise has provider independent address space. If the route is not permitted (denied). • More specific routes than the aggregate should be filtered using a prefix-list. the upstream provider must ensure that all the components of the aggregate are contained within its administrative domain. allowed. the route is not suppressed--that is. As mentioned earlier.4 {1 2 3}. and contributed to by AS 10 and 20 (since 10 and 20 are in the AS set). and generates two aggregates. You can use the match clauses of route maps to selectively suppress some more-specific routes of the aggregate and leave others unsuppressed. To do this.

the router will always prefer a more specific route first.255.0 255.0.Configuration Example Aggregation using the network command router bgp 65000 network 172.16. Additional details on Route Aggregation BGP Best Practice Version 3.0/16 into a single address. The summary-only argument at the end of the aggregate-address command tells the router to advertise the aggregate only.0 command allows a more specific route to be originated by the router so that aggregation can work.0 mask 255. as long as more specific routes are available. It is important to note that the aggregate is created only if components are learned.0 summary-only The above configuration uses the aggregate-address command to aggregate all the more specific routes of 172. However.0 ip route 172.255. A high value for the administrative distance (254) can be optionally used so that the static route gets used only as a last resort within the router. The network 172.0.0 255.0 mask 255. the aggregate-address command provides more control over propagation of summaries and specifics. Note that the static entry is pointing to null0 (bit bucket).0 63 . Aggregation using the aggregate-address command As explained.0.255.0.16.0.0 null0 254 The preceding configuration places a static instance of 172.16.16.0.255.16.255. Example: router bgp 65000 network 172.16.0/16 in the routing table.16.0.255.255.0 mask 255.50.0 aggregate-address 172.50.0.

Background The ip prefix-list command is used to configure IP prefix filtering. if the first configured sequence number is 3. providing more flexible configuration than can be configured with just the network/length argument. Prefix list information and counters are displayed in the output of the show ip prefix-list command. The prefix list is processed using an exact match when neither the ge nor le keyword is entered. 10.0/0: ip prefix-list abc deny 0. If only the le value is entered. Once a match is made that covers the network the permit or deny statement is applied to that network and the rest of the list is not evaluated The prefix list is applied to inbound or outbound updates for specific peer by entering the neighbor prefix- list command. One or the other must be entered when configuring this command.0. then subsequent entries will be 8. Prefix lists are configured to match an exact prefix length or a prefix range. Examples The following examples show how a prefix list can be used.0. The ge and le keywords are used to specify a range of the prefix lengths to match. Other Miscellaneous Areas Prefix Lists Prefix lists are used to filter IP prefixes and can match both the prefix number and the prefix length (mask). If a sequence number is entered for the first prefix list entry but not subsequent entries. An implicit deny is applied to traffic that does match any prefix-list entry. If the ge. Default sequence numbers can be suppressed by entering the no form of this command with the seq keyword. The IP address can be a classful network. then the subsequent entries will also be incremented by 5 (For example. To deny the default route 0. and onwards). 15. Prefix-list counters can be reset by entering the clear ip prefix-list command. the range falls between the values used for the ge-length and le- length arguments. If a sequence number is not entered a default sequence number of 5 is applied to the prefix list and subsequent prefix list entries will increment by 5 (for example.0 64 . 5. Compared to access-lists prefix lists provide for a significant performance improvement as it requires fewer processing cycles. If only the ge value is entered. 18. It also provides greater flexibility and supports incremental list updates. the range is the value entered for the ge ge-length argument to a full 32-bit length. ge-length and le. the range is from value entered for the network/length argument to the le le-length argument.0. a subnet.0. 13. Prefix lists are configured with permit or deny keywords to either permit or deny the prefix based on the matching condition. The bit mask is entered as a number from 1 to 32.0/0 BGP Best Practice Version 3. The following formula shows this behavior: network/length < ge ge-length < le le-length <= 32 A prefix list is configured with a name and/or sequence number. Prefix lists are evaluated starting with the lowest sequence number and continues down the list until a match is made. and onwards). or a single host route. le-length keywords and arguments are entered. A prefix list consists of an IP address and a bit mask.

cisco.0.0/16 ge 25 To permit mask lengths from 8 to 24 bits in all address space: ip prefix-list abc permit 0.0.168.168.0.0.0.html#wp1001470 Conditional Route Injection Background BGP uses conditional route injection to inject more specific prefixes into a BGP routing table over less specific prefixes that were selected through normal route aggregation.0. This feature allows more specific routes to be generated based on administrative policy or traffic engineering information in order to provide more specific control over the forwarding of packets to these more specific routes.168. The BGP Conditional Route Injection feature is enabled with the bgp inject-map exist-map command.0.0/0 le 32 More information on Prefix-list configuration can be found at.0/24 ge 25 To permit all routes with a prefix of 0/0: ip prefix-list abc permit 0. http://www.0/8 The following examples show how to specify a group of prefixes.0.0.1.0. To permit the prefix10. which are injected into the BGP routing table only if the configured conditions are met. To accept a mask length of up to 24 bits in routes with the prefix 192/16: ip prefix-list abc permit 192.0. The BGP Conditional Route Injection feature allows originating a prefix into a BGP routing table without the corresponding match. Only those prefixes that are equal to or more specific than the original prefix may be injected.0/8: ip prefix-list abc permit 10.0/16 le 24 To deny mask lengths greater than 25 bits in routes with the prefix 192/16: ip prefix-list abc deny 192. These more specific prefixes can be used to provide a finer granularity of traffic engineering or administrative control than is possible with aggregated routes.com/en/US/docs/ios/12_2/ip/configuration/guide/1cfbgp. This command uses two route BGP Best Practice Version 3.0.0/0 ge 25 To deny all routes with a prefix of 10/8: ip prefix-list abc deny 10.0/8 le 32 To deny all masks with a length greater than 25 bits routes with a prefix of 192.168.1/24: ip prefix-list abc deny 192. Enabling this feature will improve the accuracy of common route aggregation by conditionally injecting or replacing less specific prefixes with more specific prefixes.0.0/0 ge 8 le 24 To deny mask lengths greater than 25 bits in all address space: ip prefix-list abc deny 0.0 65 .0.

0 66 . Route-injection is not neighbor centric. If conditional route injection is not working properly. . The inject-map defines the prefixes that will be created and installed into the local BGP table. verify that the aggregate prefix is being received from the correct neighbor and the prefix list identifying that neighbor is a /32 match. If the aggregate prefix exists but conditional route injection does not occur. Conditional Advertisement is neighbor centric.1. The exist-map specifies the prefixes that the BGP speaker will track. Conditional Route-Injection creates new routes in the BGP table if a particular route does exist in the BGP table. Guidelines The BGP Conditional Route Injection feature is based on the injection of a more specific prefix into the BGP routing table when a less specific prefix is present. Benefits The BGP Conditional Route Injection feature allows injecting more specific prefixes into a BGP routing table over less specific prefixes that were selected through normal route aggregation. BGP Conditional Route Injection should not be confused with the existing Conditional Advertisement feature. Ensure that the inject route map is configured with the set I address command and not the match ip address command. advertising certain networks that are already in the BGP table to a given EBGP neighbor based on the existence of another particular network in the BGP table.1. Configuration Example This following configuration example configures conditional route injection for the inject-map named ORIGINATE and the exist-map named LEARNED_PATH. router bgp 109 bgp inject-map ORIGINATE exist-map LEARNED_PATH ! route-map LEARNED_PATH permit 10 match ip address prefix-list ROUTE match ip route-source prefix-list ROUTE_SOURCE ! route-map ORIGINATE permit 10 set ip address prefix-list ORIGINATED_ROUTES set community 14616:555 additive ! ip prefix-list ROUTE permit 10. maps (inject-map and exist-map) to install one (or more) more specific prefix into a BGP routing table. check the following: . There are fundamental differences between the two. These more specific prefixes can be used to provide a finer granularity of traffic engineering or administrative control than is possible with aggregated routes. . Verify that the prefix that is being injected is not outside of the scope of the aggregate prefix.0/24 BGP Best Practice Version 3.

0/25 ip prefix-list ORIGINATED_ROUTES permit 10.2.2. R2 has a loopback (BGP ID of 2. ! ip prefix-list ORIGINATED_ROUTES permit 10.1.1.128/25 ! ip prefix-list ROUTE_SOURCE permit 10. For the topology above let's look at the routing BGP Best Practice Version 3.2).1/32 BGP Deterministic-med Background As stated in RFC 1771. Enabling the bgp deterministic-med command ensures the comparison of the MED variable when choosing routes advertised by different peers in the same autonomous system.0 67 . Figure 9 BGP Deterministic-med Example In some circumstances it is possible for a router to choose different paths as the "best" path depending on the order that the route advertisements were received in.1. Benefits Using an example will illustrate the benefits of bgp deterministic-med.1. removes any dependencies of MED-based path decisions. and so on. It ensures that MED comparison is made across all routes received from the same AS and the best path is chosen to reach the routers in that AS.1. Enabling bgp deterministic-med command.1). MED is an optional non-transitive attribute that is a four octet non-negative integer. For this diagram R1 has a loopback (BGP ID of 1. The value of this attribute may be used by a BGP speaker's decision process to discriminate among multiple exit points to a neighboring autonomous system.1.2.1.

valid.3 from 3.3.4. The route from 4.4) Origin IGP.4.2. Note that because the next hop AS is not the same for these two paths.3) Origin IGP.2 from 2.4.3. table Default-IP-Routing-Table) Advertised to non peer-group peers: 2.3. external 300 400 4.0 68 .4. valid.3.3.2.2 200 400 3.0.4.4.4. internal 300 400 2.3 from 3. best #3.2 is better than the path from 3. valid.2 is external and the route through 3.0. valid. Now let's clear our 4.3.4. MED is not compared.4.3.3. R1# show ip bgp 70.4 from 4.2.3. version 4 Paths: (3 available.2.2.0. version 5 Paths: (3 available.4.4.2. best R1# With the routes in this order R1 selects the route from 4. .3.2.4 (4.4.4 and 2. internal. best #3.2.4 R1# show ip bgp 70.0/8.table from R1 for network 70.0/8 advertisement. localpref 100.0. localpref 100.0/8.3.4. metric 30.4.4.3.2.3 (3.4 as best based on the following: .3 (3.3. metric 20.0.0.3.4.3 4. metric 0.2.4) Origin IGP.4 neighbor so that the order the routes were received is changed: R1#clear ip bgp 4.3 is internal (Step 7).0 BGP routing table entry for 70. The path from 2.0.3 because the route through 2.0. Now the path from 4.4 (4.2.4 300 400 4.4.2.2.0/8.4.3. localpref 100.2 will be compared.4.4.3) Origin IGP.0 BGP routing table entry for 70. localpref 100.0. valid.4. Note that the MED values indicated on the diagram are only applied to the 70.3.3.4.0. internal 300 400 BGP Best Practice Version 3.4 from 4.2 (2. metric 0.4.0. internal 200 400 3.2) Origin IGP.4.2. metric 20. MED is compared in this case because the two paths share the same next hop AS. localpref 100.3.4 will win because it has a lower MED value (Step 6).0. table Default-IP-Routing-Table) Flag: 0x240 Advertised to non peer-group peers: 3.

2.3.2.3.2.2 (2.3.2. .3 has a lower peer address (Step 13). internal.4. MED is not compared because they are from different AS.3. Compare the best route from each sub-group in order to select the best route.2. The path from 2. metric 0.2. . For each sub-group. This inconsistency in selecting the best route can lead to routing loops in the network.3 from 3.4. Let's look at the BGP table in R1 after configuring deterministic-med: R1# show ip bgp 70.4 (4.2.3 (3. table Default-IP-Routing-Table) Advertised to non peer-group peers: 2.2 (2.3. For each sub-group.2.2.2 from 2.4. version 4 Paths: (3 available.0.3 because 2. localpref 100.4 because 3.3. external. MED is not compared because they are from different AS. Group the entries in the BGP table into sub-groups based on the next hop AS.2. order the entries within that sub-group based on MED.2.0 BGP routing table entry for 70.2 from 2. . internal 300 400 2.2.2.3. Let's take a look at why the 2. select the best route.3.2. metric 20.2 is better than the path from 3.4.0.4. valid.2) BGP Best Practice Version 3.2 route is now selected as best.4. valid.2. best #1. best 300 400 4. .3.4) Origin IGP.3.3.2) Origin IGP. localpref 100.2.3 is internal (Step 7).2.3 is better than the path from 4.3.3) Origin IGP.2.2.2 is external and the route through 3.2.2.4 from 4.2.2 200 400 3.0.0 69 .4. In order for R1 to consistently choose the same best path we need to configure deterministic-med as shown below: R1# ! router bgp 100 bgp deterministic-med ! Deterministic-med forces BGP to do the following: .0.2 route was chosen as best: .4. metric 30.0/8. best R1# This is a problem because the 2. The path from 3. localpref 100. valid.3. 2.3.

2. Configuration is shown below: router bgp 100 bgp deterministic-med BGP Best Practice Version 3. localpref 100. Another requirement when running deterministic-med is that every BGP router in the AS must also have deterministic-med configured.3.3.3.2.4 path is the best for all routes with 300 as the first hop AS.0 BGP routing table entry for 70.2.3. best #1.3.2 R1#show ip bgp 70.3 path is the best for all routes with 200 as the first hop AS. internal 300 400 2.0.3.4) Origin IGP.3.2.4.3.2 peer.0 70 . external R1# The path from 3. Now we compare the 3.4. . localpref 100.3 from 3.0.0.3 entry becomes the best path due to the following: .3. This path beats the 2. valid. internal. localpref 100.2. metric 30.4 from 4. .2.3 path wins because it has a lower peer ID (Step 13).0/8.4. This should be enough proof to convince you and your customers to ALWAYS run deterministic-med. metric 0.0.2 (2. valid. version 4 Paths: (3 available.3.4. external R1# The 3.4. Guidelines BGP deterministic-med is disabled by default and it is recommended to ALWAYS enable BGP deterministic-med.4 (4.4 path.3. best 300 400 4. valid.3.2) Origin IGP. Origin IGP. If we clear our 2.2.4. Although the above situation is rare.2 from 2. table Default-IP-Routing-Table) Advertised to non peer-group peers: 2.3) Origin IGP.4 has a lower MED (Step 6).3. it does sometimes happen.4. metric 20. valid. which would normally cause a change in what is considered the best path.3 (3.2 200 400 3.4.2.3. metric 30.3 is still the best path: R1#clear ip bgp 2. The 3.3. localpref 100. The 3.2.4. because the paths are ordered the same way each time.2. The 4.2.2.3.2. we can see that 3.2.4.3.3 path to the 4.4.3.4.3 is now considered the best every time.2 path because 4.

BGP Log Neighbor Changes Background BGP Log Neighbor Changes enables the logging of neighbor adjacency changes and/or resets. (Route with lowest BGP RID is selected.0 71 . . If the BGP router id is not manually set. . Messages are written locally to the routers logging buffer or remotely to a Syslog server. BGP ‘ORIGINATOR_ID’ and ‘CLUSTER_LIST’ path attributes are set to the BGP Router Identifier by default. . Benefits Logging changes in BGP neighbor state is useful for identifying and troubleshooting BGP peering issues. Benefits Deterministically setting the BGP Router Identifier is beneficial when considering the following scenarios. BGP Best Practice Version 3. If no loopback interface is configured the highest IP address configured on a physical interface is used. Guidelines The BGP Router Identifier is set to the highest IP address assigned to a routers loopback interface. . BGP synchronization requires that BGP and OSPF router identifiers match when redistributing routes between protocols. The ‘bgp router-id <ip address>’ command should be used to deterministically set the routers BGP Router ID to an IP address configured on a loopback interface.) .BGP Router Identifier Background The BGP Router Identifier is a 4-byte field in the BGP Open message used to identify a BGP speaker. A router running only IPv6 requires a BGP Router Identifier be manually set. changing a routers IP address can cause a BGP router id change and reset the routers BGP peers. When two equal cost BGP routes are considered by the BGP Best Path Algorithm one selection criteria is BGP Router Identifier. Risks and Limitations Changing the BGP Router ID on an active BGP peer will force the session to be reset.

Guidelines Logging of BGP neighbor adjacency changes is disabled by default. Logging of neighbor adjacency state changes should be enabled using the ‘bgp log-neighbor-changes’ command. BGP Best Practice Version 3.0 72 .