You are on page 1of 73

TM

Advanced Services

Cisco Systems Advanced Services

BGP Best Practices
Version 3.0

Cisco
Corporate Headquarters
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax:408 526-4100

Legal Notice

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS
DOCUMENT ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,
INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE
ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY
PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT
ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND
ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE
SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE
FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the
University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating
system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND
SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND
THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED,
INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF
DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL,
CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST
PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE
THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.

CCDE, CCENT, CCSI, Cisco Eos, Cisco HealthPresence, the Cisco logo, Cisco Lumin, Cisco Nexus,
Cisco Nurse Connect, Cisco Stackpower, Cisco StadiumVision, Cisco TelePresence, Cisco WebEx, DCE,
and Welcome to the Human Network are trademarks; Changing the Way We Work, Live, Play, and Learn
and Cisco Store are service marks; and Access Registrar, Aironet, AsyncOS, Bringing the Meeting To
You, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, CCVP, Cisco, the Cisco Certified
Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco
Systems logo, Cisco Unity, Collaboration Without Limitation, EtherFast, EtherSwitch, Event Center, Fast
Step, Follow Me Browsing, FormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, iQuick
Study, IronPort, the IronPort logo, LightStream, Linksys, MediaTone, MeetingPlace, MeetingPlace Chime
Sound, MGX, Networkers, Networking Academy, Network Registrar, PCNow, PIX, PowerPanels,
ProConnect, ScriptShare, SenderBase, SMARTnet, Spectrum Expert, StackWise, The Fastest Way to
Increase Your Internet Quotient, TransPath, WebEx, and the WebEx logo are registered trademarks of
Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.

All other trademarks mentioned in this document or website are the property of their respective owners.
The use of the word partner does not imply a partnership relationship between Cisco and any other
company.

Copyright © 2009 Cisco Systems, Inc. All rights reserved.

BGP Best Practice Version 3.0 2

Contents

Contents 2

Tables 7

Figures 8

About This Design Document 9

Document Purpose 9
Intended Audience 9
Scope 9

Dynamic Update Peer-Groups and Peer-Templates 11

Background 11
Benefits 11
Guidelines 12
Risks and Limitations 12

Resource Allocation and Convergence 14

Background 14
Benefit 14
Guidelines 14
Details 15
Improving BGP Convergence 15

Soft Reconfiguration and Route Refresh 19

Background 19
Benefits 19
Guidelines 19
Risks and Limitations 19
Details 20

BGP Support for Local-AS 21

Background 21
Guidelines 21
Details 21

BGP Support for Dual-AS Configuration 22

Background 22
Guidelines 22

BGP Best Practice Version 3.0 3

0 4 . Details 23 BGP Infrastructure Security 26 BGP TTL Security Hack (BTSH) 26 Background 26 Benefits 26 Guideline 26 Risks and Limitations 26 BGP Authentication 27 Background 27 Benefits 27 Guideline 27 Risks and Limitations 27 BGP Maximum-Prefix 27 Background 27 Benefits 28 Guideline 28 Route Flap Dampening 29 Background 29 Guidelines 29 Risks and Limitations 29 Details 30 Route Reflectors 31 Background 31 Benefits 31 Guidelines 31 Risks and Limitations 32 Confederations 33 Background 33 Guidelines 33 Details 33 Route Propagation in Confederation 33 BGP-IGP Redistribution Policies 35 Background 35 Benefits 35 Guidelines 35 Risks and Limitations 35 BGP Next-hop 36 Background 36 Guidelines 36 Community Attribute 38 BGP Best Practice Version 3.

single homed 50 Scenario 2: Stub network multi-homed to single SP with single border router 51 Scenario 3: Stub network multi-homed to single SP with multiple border routers 52 Scenario 4: Standard Network Multi-homed to Multiple ISPs with Multiple Border Routers 53 Load Balancing 54 Background 54 Guidelines 54 Inbound Load Balancing 54 Outbound Load Balancing 54 Scenario 1 54 Scenario 2 55 Scenario 3 55 Scenario 4 (iBGP Multipath) 55 Scenario 5 (Link Bandwidth) 56 Risks and Limitations 56 Details 57 Using Advertise Map for Inbound Load Balancing 57 Using EBGP Multipath for Outbound Load Balancing 58 Using BGP Link Bandwidth for Outbound Load Balancing 59 Route Aggregation 61 Background 61 Benefits 61 Guidelines 61 Risks and Limitations 62 Details 62 Configuration Example Aggregation using the network command 63 BGP Best Practice Version 3.0 5 . Background 38 Benefits 38 Guidelines 38 Details 39 Extended BGP Communities 41 Synchronization Rules 44 Background 44 Benefits 44 Guidelines 44 Multi-homing 45 Background 45 Benefits 45 Multi-homing Scenarios 45 Stub Network Single-homed 46 Stub Network Multi-homed: Single Border Router 46 Stub Network Multi-homed: Multiple Border Routers 46 Standard Multi-Homed Network: Single border router / Multiple ISP’s 47 Standard Multi-Homed Network: Multiple border routers 49 Multi-homing Scenario Examples 50 Scenario 1: Enterprise stub network.

0 6 .Other Miscellaneous Areas 64 Prefix Lists 64 Background 64 Conditional Route Injection 65 Background 65 Benefits 66 Guidelines 66 BGP Deterministic-med 67 Background 67 Benefits 67 Guidelines 70 BGP Router Identifier 71 Background 71 Benefits 71 Guidelines 71 Risks and Limitations 71 BGP Log Neighbor Changes 71 Background 71 Benefits 71 Guidelines 72 BGP Best Practice Version 3.

0 7 .Tables Table 1 BGP Community Design 41 BGP Best Practice Version 3.

0 8 .Figures Figure 1 BGP Support for Dual-AS example 23 Figure 2 Single-homed Enterprise Stub Network with Border Router 50 Figure 3 Stub Network Multi-homed to Single SP and Border Router 51 Figure 4 Stub Network Multi-homed to Single SP with Multiple Border Routers 52 Figure 6 Use of advertise-map to influence inbound traffic 57 Figure 7 Using EBGP Multipath for Outbound Load Balancing 58 Figure 8 Using BGP Link Bandwidth for Outbound Load Balancing 59 Figure 9 BGP Deterministic-med Example 67 BGP Best Practice Version 3.

there are many initiatives currently underway to automate the exception reports based on violation of best practices rules. Throughout the document. although some examples are provided to help understand the recommendations in question. other Cisco Technical Teams. In other words. No attempt is made to explain and clarify fundamental functionality. and Caveats. and Customers. Soft Reconfiguration and Route Refresh 4. New Features. It is expected that the reader is familiar with basic BGP routing and is experienced in configuring Cisco routers. Cisco Partners. recommendations are highlighted yellow to clearly separate them from explanations. and a good amount of documentation is available within Cisco as well as on other external sites. and updated. Case Studies. our experience with Routing VT pointed to the fact that what Cisco or Advanced Services is recommending with regards to routing protocol implementation as Best Practices is not being captured. Optimization. Intended Audience • This document is for NCEs in Advanced Services. Dynamic Update Peer-Groups and Peer-Templates 2. It was also felt that there is no central place where we can store and use all the Design Reviews and Case studies that are based on best practices in specific situations. BGP Support for Local-AS 5.0 9 . Resource Allocation and Convergence 3. The following areas are covered: 1. Migration. Document Purpose • To provide comprehensive industry best practices for BGP used on Cisco Routers. • To provide a medium to track and continuously update the technology best practices to ensure that the collective technical experience of Advanced Services Routing VT is recorded. Still. This document captures best practices for implementing BGP. Furthermore. • This document is also for the Advanced Services Tools Team for automation of Best Practices Compliance. provide a platform for converting AS expertise to Cisco Intellectual Property. reviewed. In an effort to address all these issues. About This Design Document Routing Protocols design and implementation is quite mature. This document focuses on the industry best practices and other related information for BGP implementation. Implementation. BGP Support for Dual-AS Configuration BGP Best Practice Version 3. Planning. Scope This document includes BGP Best Practices for Design. Routing VT started working to capture these best practices as well as provide a dynamic way to constantly update and add.

BGP-IGP Redistribution Policies 11. Multi-homing 15. that use BGP is beyond the scope of this document. the user might do something that could result in equipment damage or loss of data. BGP Infrastructure Security 7. The text that accompanies this symbol provides the user with a useful tip. This symbol means tip. Other Miscellaneous Topics Information regarding other related technologies. Community Attribute 13. The user may be in a situation that could cause bodily injury. in this document they have the following meaning: This symbol means warning. Where side symbols are included.. BGP Best Practice Version 3. etc. MBGP. Load Balancing 16. Route Reflectors 9. In this situation. to the document during the implementation work or that the user must take note of the information presented. MP-BGP. Synchronization Rules 14. The user must add information. Confederations 10. written or typed. The user might save time by performing or being aware of the action/information described in the paragraph. Route Flap Dampening 8. This symbol means note. BGP Next-hop 12. Route Aggregation 17. Before the user works on the equipment. such as MPLS. This symbol means timesaver. This symbol means caution.6.0 10 . they should be aware of hazards involved with electrical circuitry and be familiar with standard practices for preventing accidents.

These neighbors automatically fall under the same update-group. eliminating the need for depending on peer-group configuration. • Optimal BGP update message generation occurs automatically and independently. Peer Session Template: Peer session templates are used to group and apply the configuration of general session commands that are common to all address family and Network Layer Reachability Information (NLRI) configuration modes. With Dynamic Update Peer-Group feature. Peer-templates address these limitations. • BGP neighbor configuration is no longer restricted by outbound routing policies. the BGP Configuration Using Peer Templates feature was introduced along with the BGP Dynamic Update Peer-Groups feature. The BGP Configuration using Peer Templates feature improves the flexibility of BGP neighbor configuration through the introduction of peer-policy and peer-session configuration templates. this feature does not require any manual configuration. BGP update messages were grouped together based on peer-group configurations. such as: . and update-groups can belong to different address families. Hence. The following are the features of the Dynamic Update Peer-Group: • Dynamically calculates BGP update-group membership based on outbound routing policies. Benefits The BGP Dynamic Update Peer-Group feature requires no configuration and occurs automatically. In previous versions of Cisco IOS Software [prior to 12. This method of grouping updates limited outbound policies and specific-session configurations. • This feature does not require any configuration by the network operator.0(24) S]. it separates update-group replication from peer-group configuration. as to which neighbors can share updates. which improves convergence time and flexibility of neighbor configuration. Dynamic Update Peer-Groups and Peer- Templates Background The BGP Dynamic Update Peer-Groups feature introduces a new algorithm that dynamically calculates and optimizes update-groups of neighbors that share the same outbound policies and can share the same update messages. based on the configuration of the neighbors. The Dynamic Update Peer-Groups implementation can automatically calculate.0(24)S. The BGP Configuration Using Peer Templates feature introduces a new mechanism called the peer template. In IOS version prior to 12. All neighbors that share the same peer group configuration also has to share the same outbound routing policies . All neighbors have to belong to the same peer group and address family.0 11 . To address the limitations of peer groups. peer-groups are also used for update-grouping in addition to configuration grouping which has some limitations. BGP Best Practice Version 3. A peer template is a configuration pattern that can be applied to neighbors that share common policies.

Peer Policy Template: Peer policy templates are used to group and apply the configuration of commands that are applied within specific address-families and NLRI configuration modes. Peer templates also provide an alternative to Peer-Group configuration and overcome some limitations of Peer-Groups.2(18)S .3(4)T and 12. • The BGP Dynamic Update Peer-Group feature requires no configuration and occurs automatically. • If you are running an IOS version that supports BGP Dynamic Update Peer-Groups feature then the default behavior is that BGP dynamically calculates and optimizes update-groups of neighbors that share the same outbound policies and can share the same update messages which improves the performance of BGP update message generation. a neighbor or neighbor group can be configured with only one directly applied peer session template and seven additional indirectly inherited peer session templates. Inheritance expands the scalability and flexibility of neighbor configuration by allowing you to chain together peer templates configurations to create simple configurations that inherit common configuration statements or complex configurations that apply very specific configuration statements along with common inherited configurations. which allows the network operator to group and apply distinct neighbor configurations for BGP neighbors that share common policies. and each inherited session template can also contain one indirectly inherited session template. So. • A peer policy template can directly or indirectly inherit up to eight peer policy templates.3(4)T. Configuring peer-groups will not achieve anything in terms of performance rather it may just cut down on configuration. • Allows the network operator to define very complex configuration patterns through the capability of a peer template to inherit a configuration from another peer template. BGP Best Practice Version 3.0 12 . The inheritance capability is a key component of peer template operation. • Peer templates are reusable and support inheritance. Guidelines BGP Dynamic Update Peer-Group feature was first introduced in 12.0(24)S and integrated into 12. Risks and Limitations The following restrictions apply to the BGP Configuration using Peer Session Templates and Peer Policy Template feature: • A peer session template can directly inherit only one session template.2(18)S and 12. 12. • The inheritance capability of peer template eliminates the need to repeat configuration statements that are commonly reapplied to groups of neighbors because common configuration statements can be applied once and then indirectly inherited by peer templates that are applied to neighbor groups with common configurations. So there are no performance benefits with peer-groups if the code supports BGP Dynamic Update Peer-Groups feature. • A BGP neighbor cannot be configured to work with both peer groups and peer templates. A BGP neighbor can be configured to belong only to a peer group or to inherit policies only from peer templates. • It is recommended to use Peer templates as it improves the flexibility and enhances the capability of neighbor configuration.0(24)S and integrated into 12. Peer-Template feature was first introduced in 12.2(27)SBC.

There is no flexibility on a per neighbor basis as available with peer-templates and inheritance. For detailed information on Peer-Templates. • For small networks with simple BGP update policies. Peer-Template configuration could seem to be extensive however the benefit that is received due to inheriting peer-template and session- template policies provides the flexibility in configuration even as the network grows. With Peer- Templates. click here. With peer-group configuration. For detailed information on Dynamic Update Peer-Groups. Peer-Groups If you are running an IOS version that does not support BGP Dynamic Update Peer-Groups. click here. multiple policies are available for each remote AS so there is no need to configure an entirely new peer-group configuration for the each different AS. then Peer- Groups can be used as all update messages are grouped together based on peer-group configurations and also reduces configuration. multiple peer-groups are required for each remote AS.0 13 . If the IOS version supports Dynamic Update Peer-Groups with Peer-Templates then it is recommended to use that feature as it provides greater flexibility. neighbour grouping and ease of configuration. BGP Best Practice Version 3.

and tuning session and update timers. Several BGP enhancements have been made to improve convergence and basic scaling properties. Besides enhancing hardware resources (CPU. router processing and memory demands increase. much like how a router with a faster CPU can support larger numbers of peers. Transmission Control Protocol path Maximum Transmission Unit discovery (PMTUD). there are several steps that customers can take to further reduce convergence time. These include using peer groups. service providers and large enterprise customers are noticing an increase in the BGP convergence time. and memory burdens of expanding networks. As individual networks grow larger. processing. Resource Allocation and Convergence Background BGP is primarily used to provide control plane connectivity between multiple administrative domains (called autonomous systems (AS)) with varying routing policies. path MTU discovery. These features within Cisco IOS software assist network operators with scaling their IP-BGP networks in ways that mitigate the management. the following components have an effect on the number of BGP routes/peers a router can support • Router's CPU • Route Memory • IOS version The more memory a router has the more routes it can support. large input queues. BGP Best Practice Version 3. optimization of router resources like memory and TCP session parameters like maximum segment size (MSS). etc. BGP chooses the best loop free path through the internetwork. Appropriate hardware has to be considered depending on the number of peers and routes. interface input queues.0 14 . Benefit • Faster convergence: By assessing the requirement of appropriate resources and tuning other parameters. In addition. BGP updates rely on TCP. Guidelines BGP updates take CPU cycles. and the number of peers and routes that you want to hold takes memory. Due to its importance in the Internet. BGP will choose the shortest path through the internetwork as defined by the AS Path. BGP administrators can achieve faster convergence within their networks. help improve converge. TCP window size.. The best-known instance of inter-domain connectivity provided using BGP is the Global Internet. with “best” being defined by the local policy of each AS. In the absence of overriding policy. In general. • Increased scalability: BGP enhancements and configuration recommendations allow us to handle even greater numbers of routing table entries within the same convergence time frame. BGP is a major focus for scaling and convergence work. memory). this could eventually pose scalability challenges to Service Providers and enterprise networks to maintain an ever-increasing number of TCP sessions. As the size of the Internet routing table and number of peers grow.

cuts memory consumption. • Enable Path MTU Discovery • TCP MSS is recommended along with Path MTU discovery in Multi-Hop BGP Session. community. especially when TCP has a lot of data to transport like it does with BGP. Extra memory will be needed during the initial startup for the BGP peers. Knowledge of these parameters is required to calculate the amount of memory required to store a certain number of BGP routes. This is accomplished by enabling ip tcp BGP Best Practice Version 3. The solution is to dynamically determine how large the MSS value can be without creating packets that will need to be fragmented.0 15 . Path MTU discovery Every TCP session has a limit in terms of how much data it can transport in a single packet. route dampening. However. Fast Session Deactivation have aided in improved convergence without tuning the BGP timers. This limit is defined as the Maximum Segment Size (MSS) and is 536 bytes by default. If default values do not meet the network convergence requirements. and VPN configurations. • Increase SPD headroom to prevent dropping BGP Sessions. Memory The amount of memory required to store BGP routes depends on many factors. Using a MSS of 536 bytes ensures that the packet will not be fragmented before it gets to its destination because most links have a MTU of at least 1500 bytes. the number of alternate paths available. • TCP Receive window-size tuned on routers with large number of routes and routers in Long Fat Networks – Networks with large bandwidth and high delay. as the routing information churn that takes place consumes additional memory. consider tuning the timers with caution. click here. With sufficient available memory. such as the router. it may become necessary to carefully observe the BGP behavior and adjust following parameters • The interface input queues to be monitored and adjusted if drops are seen. This means TCP will take all of the data in a transmit queue and break it up into 536 byte chunks before passing packets down to the IP layer. it is important to understand ways to reduce memory consumption and achieve optimal routing without the need to receive the complete Internet routing table. in networks with lots of peering sessions and large routing tables. BGP would utilize the Update Packing Optimization feature which would improve convergence. Using a small MSS value creates a large amount of TCP/IP overhead. Details Improving BGP Convergence Longer convergence times are due to the increased size of the Internet table and an increase in the number of peers supported by a single BGP speaker. BGP attributes. bgp scan-timer import and bgp scan- timer. However. the number of maximum paths configured. Detailed below are some areas that can be tuned to improve BGP convergence. and hold timer) at their default values. For detailed information on achieving optimal routing and memory consumption. Route summarization for example. keepalive. Recent features such as Next-Hop-Tracker. It is recommended to leave the BGP timers (advertisement-interval.

Increasing the interface input queue depth will help reduce the queue drops. During network transient events. If it only traverses POS segments. In case of failure in the primary path. Use interface configuration command hold-queue <1-4096> in increase the queue size. the secondary path may have a smaller MTU which would result in fragmentation of the packets till the smallest MTU is found in the new path.html TCP Window Size In scenarios like high bandwidth-high delay networks (Long Fat Networks) and routers under stress. BGP Best Practice Version 3. SPD headroom depth is shared by all the interfaces. there could be large number of TCP update and ACK packets. the MSS will be 1460 bytes. PMTU Discovery will not work. Please note that the PMTUD derives the smallest MTU on the best path (primary path) between endpoints and sets the MSS value for the TCP session. If this is not desired. Default Interface Input queue size is 75. TCP acknowledgements are delayed. PMTU allows TCP to determine the smallest MTU size among all links between the ends of a TCP session. TCP will then use this MTU value. it also has additional benefits in can promote stability by batching routing changes and improve update packing in some scenarios. While the original intent of the implementation of the MRAI timer was to reduce the overhead of the BGP routing protocol . please refer to the following URL: http://www. Interface Input Queues Large numbers of interface input queue drops are a very common problem for routers with many peers and substantial number of routes. This delay can lead to TCP retransmissions. minus room for the IP and TCP headers.cisco. For additional information on SPD. This feature should be enabled by setting TCP window size to more than 65535 on both the neighboring peers.path-mtu-discovery (a. The intent of the “MinRouteAdvertisementIntervalTimer” is to reduce the overhead of the BGP routing protocol.. Optimal values for interface input queue depends upon the number of peers and Processor capabilities. which helps BGP converge faster.a. PMTU). TCP Window Scaling feature can help minimize TCP retransmissions. In such cases. The SPD headroom can be changed using the global SPD Headroom configuration command.com/web/about/security/intelligence/spd. The increase in MSS from 536 to 1460 or 4430 bytes reduces TCP/IP overhead. This rate limiting procedure applies on a per-destination basis.0 16 . as the MSS for the session. the MSS will be 4430 bytes. although the value of “MinRouteAdvertisementIntervalTimer” is set on a per BGP peer basis. If a TCP session only traverses Ethernet segments. using the configuration command ip tcp window-size <bytes more than 65535> Minimum Route Advertisement Interval (MRAI) MRAI determines the minimum amount of time that must elapse between an advertisement and/or withdrawal of routes to a particular destination by a BGP speaker to a peer.k. then it is a good practice to set the TCP Maximum Segment Size to the lowest MTU size in the network using “ip tcp mss <value>” In some environments the firewalls or other devices may be configured (or misconfigured) to block the ICMP messages used for path MTU discovery. This condition can overrun the SPD and interface input queues and result in updates being lost.

This may result in a multiple best path announcements to its neighbors as each update message is received and processed. For iBGP and PE-CE eBGP peers it is recommended the MRAI timer be set to 0. research has revealed that the implementation of MRAI. This is achieved with the following command: neighbor x. These differences mean that update messages transiting different ASNs using different vendor equipment will arrive at the target router at different times.For an iBGP Peer the default MRAI is 5 seconds. Fast Peering Session Deactivation’s ability to deactivate a peer immediately upon loss of the route to the peer. One potential result of this is that a simple update message from one ASN would be seen as a multiple route flap event a few ASN hops away . This router will see these different messages. and will consider each one for best path options. While the Cisco default for an eBGP peer is 30 seconds.x advertisement-interval 0 This was the default beginning with 12. This can also lead to substantial delays in convergence. In a large BGP network this could cause significant route churn.x.0 17 . Fast Peering Session Deactivation uses the Address Tracking Filter to register the route to the peer. Likewise some ASN operators may alter the default values. There have been actual measurements where this resulted in a single prefix withdrawal producing 41 BGP events a few hops away! Not only is the MRAI timer a potential source of problems. Fast Peering Session Deactivation The default behavior for BGP when the route to neighbor is lost is to wait for the expiration of the hold timer. can be most helpful in multi-hop BGP scenarios. This feature is not recommended for BGP peers with redundant paths to neighbor. there is no consistency across vendor implementations. or rather the disparate implementation of MRAI between vendors or ASN operators has actually contributed to network instability. UPDATE. In a 4 router fully-meshed network the total MRAI delay for convergence would be 10-seconds for an iBGP mesh and 60-seconds for an eBGP mesh.x. This feature can be configured using the BGP configuration command: neighbor x. This occurs when a peer does not receive KEEPALIVE. as a transient convergence in the IGP would cause the unwanted deactivation of the peer.0(32)S For eBGP peers this could cause dampening.x. but also differences in CPU loadings and CPU speed will result in different update times for prefixes announcements passing from router to router. or NOTIFICATION messages within the period specified in the Hold Time field of the OPEN message. without waiting for the Hold Time to expire. Risks and Limitations As processing power of routers has increased.when in fact there was no instability whatsoever.x fall-over BGP Best Practice Version 3. so it should be confirmed with peer ASNs whether they are using route dampening prior to making this modification. A host route must be available for each peering session that is configured to use BGP fast session deactivation. If the router loses the route to the peer the session is deactivated. These differences will also contribute to the effects described above. for an eBGP peer the default is 30 seconds.x.

Events like internal network outages can cause some next-hop addresses to become unreachable. This way NHT will invalidate the BGP route without waiting for the default BGP Scanner for 60 sec and hence will bring down convergence time from the 60 sec range to 5 sec range. Only match ip address and match source-protocol commands are supported in the route map. BGP Selective Address Tracking Undesirable routes such as aggregate address.BGP Support for Next Hop Address Tracking (NHT) BGP routes do have next-hop addresses. This can lead to oscillation of next-hops. This task uses prefix lists and route maps to match IP addresses or source protocols and to restrict prefixes that can be considered as next-hop routes. IPv4 multicast. Configuration Example In the following example. extending the convergence time to 60sec. next-hop address tracking is disabled under the IPv4 address family session: router bgp 65000 address-family ipv4 unicast no bgp nexthop trigger enable In the following example. BGP prefixes or a default route to the next hop can lead to invalid condition in BGP RIB. It takes about 60 sec for alternate BGP paths to be installed in the routing table. The purpose of selective next-hop route filtering is to avoid using undesirable routes to the next hop. This feature is available in relatively newer IOS versions. No set commands or other match commands are supported. BGP Support for Next-Hop Address Tracking is enabled by default when a supporting Cisco IOS software image is installed. When a next-hop address becomes unreachable (deleted from RIB). Typically next-hop addresses are learned via an IGP. IPv4 tunnel and IPv4 VPNV4 not for IPv6. NHT will schedule BGP next-hop scan after a configurable Trigger Delay interval (default = 5sec). BGP Best Practice Version 3. the delay interval for next-hop tracking is configured to occur every 20 seconds under the IPv4 address family session: router bgp 65000 address-family ipv4 unicast bgp nexthop trigger delay 20 The default trigger-delay is 5 seconds. NHT configuration is supported for IPv4 unicast. which effects BGP convergence. BGP scanner runs every 60sec and will invalidate BGP paths that have unreachable next-hop. Such BGP route(s) with unreachable next-hop(s) are nonfunctional but will continue to stay UP until next BGP scanner runs.0 18 . Next-Hop Address Tracking (NHT) feature monitors the reachability of next-hop addresses of BGP routes.

which had been advertised on the BGP session that is being reset. it can result in other routers in the network dampening prefixes. • The Route Refresh feature requires both peers for the session to support this feature. The BGP Soft Reconfiguration feature does not require both peers for a session to support the feature.0 19 . it is recommended that this feature be used in place of BGP Soft Reconfiguration to minimize memory requirements. Benefits • The benefit of BGP Soft Reconfiguration and Route Refresh features is that both methods initiate routing policy changes without resetting the BGP session. If a BGP session is reset repeatedly over a short period of time due to multiple changes in BGP policy. even those that are not selected by the BGP Best Path process. Guidelines Hard reset of a BGP session is disruptive to an operational network. • If BGP Soft Reconfiguration feature is configured on an active BGP session then hard reset of the session is required the first time for it to take effect. The Route Refresh feature is a direct replacement for the BGP Soft Reconfiguration feature. The BGP Soft Reconfiguration feature pre-dates the Route Refresh feature and is more resource intensive. causing destinations to be unreachable and traffic to be black-holed. then the only soft reconfiguration option is to use the neighbor soft-reconfiguration command. potentially causing additional withdrawals to its other peers. BGP Best Practice Version 3. • If a peer does not support the route refresh capability. • There is no configuration required to enable the Route Refresh feature. This capability is negotiated at session initialization. all prefixes. the BGP session needs to be reset in order for the new policy to take effect. are removed from the remote peer. • If both peers support the Route Refresh feature.0(7)T onwards. In addition. • The Route Refresh feature has the advantage of not having additional memory resource requirements for storing all received prefixes. Risks and Limitations • The primary risk with the BGP Soft Reconfiguration feature is the increased memory requirements on a BGP router that is performing inbound soft reconfiguration. Soft Reconfiguration and Route Refresh Background When BGP policy is changed. The BGP Soft Reconfiguration feature and Route Refresh feature are both methods by which the route flapping effect of resetting a session can be mitigated. which initiates the storage of inbound routing table updates and requires additional memory. • The Route Refresh feature was introduced in Cisco IOS–12. The resetting of a BGP session results in all prefixes received on that session being removed and withdrawn from any neighbors to which they had been advertised.

When the inbound policy is changed. When a session is soft cleared a new update is created. The configuration command to enable this feature is: router bgp xxxxx neighbor <Address or Peer Group> soft-reconfiguration To soft clear a BGP neighbor. BGP Best Practice Version 3. This allows the BGP speaker to reprocess the updates from its neighbors through its inbound policy. the following command shows the capabilities supported for the session: show ip bgp neighbor <Peer Address> Neighbor capabilities: Route refresh: advertised and received (old & new) If you configure Inbound Soft Reconfiguration (to verify which updates has been denied for example) on a peer where Route Refresh was negotiated. they are not retained. in some cases drastically. If the soft reconfiguration feature using stored routing table updates is configured for a neighbor. To verify that Route Refresh is supported for a particular BGP session.0 20 . all of the BGP updates from the remote peer must be reprocessed. • The BGP Soft Reconfiguration feature and the Route Refresh feature are mutually exclusive. The inbound soft reconfiguration feature enables a BGP router to retain all inbound prefixes. The BGP Inbound Soft Reconfiguration must be explicitly enabled on each peer or peer group. Route refresh capability advertisement is elaborated on in RFC2918. These updates are not used in the BGP best path selection. These prefixes are marked (Received and Not Used) in the BGP RIB. Route-refresh The Route Refresh feature allows a BGP speaker to request routing update from the peer when a session is soft cleared. the Route Refresh feature is not used. Inbound Soft reconfiguration When a BGP router receives updates that are denied by inbound policy. even if they are denied by the inbound policy. this peer will send a Route Refresh request to fill its adjacency RIB. the Route Refresh feature will not be used. This reduces the resource requirements for memory. The delta between the old update previously sent to the peer and the new update once the session was soft cleared is sent out to peers in the form of advertisements and withdrawals. the following exec command is used: clear ip bgp <Peer Address> soft <in|out> If the BGP Soft Reconfiguration feature is configured. Details Outbound soft reconfiguration Performing outbound soft reconfiguration does not require any special configuration or have any additional memory requirements.

cisco. “support for local- AS” allows a router to appear to be a member of another autonomous system. A new BGP feature.shtml BGP Best Practice Version 3. BGP Support for Local-AS Background BGP is by default configured with a single autonomous system.0 21 . Detailed example of the usage of this feature is given in the following URL. It does not work for two peers in different sub-AS in a confederation • Local-AS cannot have the local BGP protocol AS number or the AS number of the remote peer • Local-AS cannot be customized for individual peers in a peer group Details The local-AS feature is typically used when merging two autonomous systems. This helps when the autonomous systems are merged together and the peering configurations need to be kept intact.com/en/US/tech/tk365/technologies_configuration_example09186a00800949cd. Guidelines • Local AS feature can be configured only for eBGP peers. in addition to its real autonomous system. http://www.

replace-as: This keyword causes the BGP speaker to replace its real autonomous system with the configured “local autonomous system” on all prefixes advertised to its eBGP peer. • The existing local BGP protocol AS number or the AS number of the remote peer cannot be configured as Local-AS • Local-AS can be configured for individual peers or configurations applied through peer groups and peer templates. Guidelines • This feature can be configured for eBGP peers only. Since this feature has the potential to modify or delete the AS_PATH information. • If this feature is applied to a group of peers ( using peer-groups or peer templates). The configuration of this feature is transparent to customer peering sessions. Customer peering sessions can be updated later during a maintenance window or during other scheduled downtime This feature comes with the following keywords: neighbor ip-address local-as [as-number [no-prepend [replace-as [dual-as]]]] local-as: The Second autonomous system configured in addition to the real autonomous system. • If there is a requirement to use this feature for other purposes such as the one given in the following example.0 22 . BGP Best Practice Version 3. the peers cannot be individually customized • BGP prepends the autonomous system number from each BGP network that a route traverses to maintain network reachability information and to prevent routing loops. This keyword is optional dual-as: Configures the eBGP neighbor to establish a peering session using the real autonomous-system number (from the configured BGP routing process) or by using the autonomous-system number configured with the “neighbor ip-address local-as”. care must be taken to make sure there are no loops in the AS PATH. allowing the provider to merge two autonomous systems without interrupting customer peering arrangements. This feature cannot be configured for iBGP peers or between different sub-autonomous systems of a confederation. BGP Support for Dual-AS Configuration Background The BGP Support for Dual AS configuration feature extends the functionality of the “BGP support for the local-AS” feature by providing additional configuration options for customizing the autonomous system paths. as a general recommendation this feature may be used as a temporary measure only during migration. This is an optional keyword. This is an optional keyword. no-prepend: This keyword causes the BGP process NOT to prepend the “local autonomous system” number to any prefixes received from its eBGP peer.

Details The following CCO document explains the BGP dual AS migration in detail. In the following example.7. it can remove the private AS’s using “remove private-as” command. following can be done as one of the options Remove all the Private AS while exchanging prefixes between the two customers at the border routers of AS 1 and AS 3. the full connectivity between the two customer network domains is not possible as the AS 64512 is present on both domains. Since XYZ domain also has the overlapping private AS number AS64512.7.1 3. Obviously the objective is full connectivity between the two customer networks. this feature is also used by customers on situations where same AS number is present in two different network domains. remove-private-as will not remove 64351 from the AS Path.4. use local-as command to replace the AS 1234 with a private AS such as 65000 while peering with the upstream AS64521 2.4 AS 1 AS 3 AS 64521 AS 4321 7.4. http://www.1. Note: Remove-private-as will only remove a private AS number if it is on the end of the AS Path. Figure 1 BGP Support for Dual-AS example 1. At Customer XYZ domain: 3. two customers (XYZ and ABC) having the same private AS numbers in their domain are merging their network. Now the AS3 border router will have prefixes with only private AS numbers in its AS PATH within its domain.html In addition to the dual AS support usage. At Customer ABC domain: 1. BGP speaker at AS64512 on one customer side would drop the prefixes received from other customer’s AS 64512 as the same AS_PATH is seen on the prefixes. Under normal circumstances. For instance.64351.4321]. using BGP protocol. To achieve the full connectivity between the customers without changing the AS numbers on the whole domain.5. this should be removed using “remove private as” command at the border router in AS4321 peering with AS1. if you have the AS Path [1234.cisco.5. On the border router at AS 1234. This happens when large customers merge their network with same autonomous system numbers or when customers get private L3 VPN offerings from providers who also use the same private autonomous system number such as 65000.3. and not in the middle.7 5.3. Now when AS 3 border router is sending the prefixes to the border router of AS1 in customer XYZ.5 AS 64512 AS 64512 AS 1234 Customer XYZ Customer ABC BGP Best Practice Version 3.1.3 4. The following steps are required to achieve this.com/en/US/docs/ios/12_3t/12_3t11/feature/guide/gtbgpdas.0 23 .

Step1: On the BGP speakers peering between AS1234 and AS64512 in ABC.14 0 0 65000 i *> 7.1.1.5/32 10. replace-as: This keyword causes the BGP speaker to replace the real autonomous system AS 1234 with the ‘local’ autonomous system AS5000 in BGP updates advertised to its neighbor.1.4. local router ID is 7. local router ID is 4.7. all the autonomous systems in the AS Path are private AS numbers.3/32 10. AS-64512#show ip bgp BGP table version is 52.1.1.26 0 0 1234 i Step 2: When the BGP speaker at the edge of AS3 receives the route from AS 64521.7.5.1. On AS-64521 peering with AS-1234 router bgp 64521 neighbor 10.13 remote-as 64521 neighbor 10. BGP Best Practice Version 3.3.1/32 10.26 0 1234 64521 2 1 i *> 3.14 0 65000 64512 i With these options configured.1.1.5.1.1.0 24 .1.1.7 Network Next Hop Metric LocPrf Weight Path *> 1.5.4.4 Network Next Hop Metric LocPrf Weight Path *> 5. the following configurations would be used: On AS-1234 peering with AS-64521 router bgp 1234 neighbor 10.3.1. no-prepend: This keyword causes the BGP process NOT to prepend the “local autonomous system” number 65000 to any prefixes received from its peer at AS64521.1.5.1.1.7.7.26 0 1234 64521 i *> 5.5/32 10.7/32 coming from AS 64512 has a local-AS of 65000 and an originating AS of 64512.4.4/32 10.1.7. we see the route to 7.1.26 0 1234 64521 2 i *> 4.7.1.14 remote-as 65000 AS-64521#show ip bgp BGP table version is 6.4.1.13 local-as 65000 no-prepend replace-as The keywords and values: local-as 65000: AS65000 is the ‘local’ AS number configured in addition the real AS number 1234.7/32 10.

1 remote-as 1 neighbor 10. AS-1#show ip bgp BGP table version is 54.0 0 32768 i *> 4.7/32 10. We can do at the AS4321 as it is closer to the originating private AS.5/32 10.2.1.2.3/32 10.0 25 . AS-3#show ip bgp Network Next Hop Metric LocPrf Weight Path *> 1.7.2.2.0 0 32768 i *> 3.5.3.5.4.2 0 64521 65000 64512 i Since all the AS numbers in the AS Path are private AS numbers on prefixes received from AS 64521. “ remove-private-as” command can strip all the AS numbers in the entire path while sending the updates to AS 1.1.4.4.1.1.0.2 0 64521 65000 i *> 7.3.1 0 01i *> 3. have only Autonomous System 3 in AS_Path list.1.1 remote-as 1 neighbor 10.1.7. router bgp 3 neighbor 10.5.4/32 10.3/32 0.1. This is because of private AS numbers being striped off at AS3.5/32 10.2 03i *> 5.1.2 0 0 64521 i *> 5. On AS4321 peering with AS1: router bgp 4321 neighbor 10. local router ID is 1.1.1.1.3.In the following output. all prefixes received from AS64521 show only private AS numbers in its path.1.3.1 remove-private-as The following output shows that all the prefixes received from AS3.0.1 Network Next Hop Metric LocPrf Weight Path *> 1.4.2 0 03i *> 4.1.1.1.1.1. we have to remove the overlapping private AS 64512 from the prefixes originating at Customer XYZ.1.1. This can be done either at the border router of AS4321 or AS1.1.2.1.1.2 03i Step3: Now that the private AS’s are removed on prefixes from Customer ABC’s domain.1 remove-private-as BGP Best Practice Version 3.1.0.5.1/32 10.7/32 10.1/32 0.0.1.7.1.4/32 10.2 03i *> 7.7.

BGP Infrastructure Security

BGP TTL Security Hack (BTSH)

Background
BGP, by default, sends packets to external neighbors with a TTL of 1 and accepts packets from external
neighbors with a TTL of 0 or higher. Because BGP will accept packets with a TTL of 0 or higher, this
makes it possible for an attacker to send packets to a BGP router from many hops away, as long as the TTL
is still greater than 0 when it arrives. Since the TCP tuple is easy to discover and an attack doesn’t need the
TCP sequence number, this presents a relatively easy attack vector.
The Generalized TTL Security Mechanism (GTSM, described in RFC 3682) protects BGP routers from
attacks sourced from devices which are not directly attached to the same segment as the BGP speaker.
When configured to use GTSM, BGP originates packets with a TTL of 255, and only accepts packets with
a TTL of 254 or higher. This TTL value is evaluated after the receiving router has decremented the TTL.

Benefits
Since forging the TTL of an IP packet is still considered not possible, the deployment of GTSM, BTSH
will protect directly connected eBGP peers from this various TCP based attacks.

Guideline
BTSH should be implemented in all eBGP peering sessions, with consideration for the fact that multihop
scenarios reduce its effectiveness dependant upon network diameter. As most eBGP peering are one hop
away, the use of BTSH can be quite beneficial.
This feature is disabled by default. If enabled it should be configured on a per-neighbor basis.

It’s important to consider the impact of GTSM on processor utilization. In many cases, deploying GTSM
will open the router to TTL based attacks, since every packet received must be punted to the process level
for TTL checking.

Risks and Limitations
While this provides robust protection in directly connected peer scenarios, the usefulness declines in multi-
hop eBGP scenarios, as this requires BTSH to be configured to accept a TTL < 254, exposing the router to
CPU-utilization attacks.

This feature is not supported for iBGP peers or peer groups. For detailed information refer to
http://www.cisco.com/en/US/docs/ios/12_3t/12_3t7/feature/guide/gt_btsh.html

BGP Best Practice Version 3.0 26

BGP Authentication

Background
BGP uses TCP as its transport mechanism which makes is susceptible to certain attacks using spoofed TCP
segments and potentially resetting BGP sessions by injecting bogus TCP resets.
BGP Authentication uses an implementation of RFC2385 which provides a TCP option for carrying a MD5
digest in a TCP segment. This digest functions as a signature for the segment as it uses information only
available to the two endpoints.

Benefits
The implementation of an MD5 digest to authenticate each TCP segment significantly increases the
difficulty in successfully implementing a spoof attack against a BGP session.

Guideline
This feature is disabled by default
BGP Authentication should be employed when feasible, taking the performance impact into consideration,
especially when BGP sessions use transport across “untrusted” network segments.

There is work currently underway to replace MD5 with alternate methods using HMAC/SHA currently in
draft states within the IETF. It is also possible as an alternative to MD5 to use IPSec to secure BGP
peering sessions.

Risks and Limitations
The performance hits associated with implementing this feature may inhibit its deployment. Testing has
shown measurable CPU impact associated with calculating the MD5 digest for BGP data segments,
additional details are available in RFC2385

BGP Maximum-Prefix

Background
The default behavior of BGP is for a router to accept unlimited number of prefixes advertised by the
neighbor.
Too many unplanned prefixes can overload BGP process and router resources which can potentially crash
the router.
This opens vulnerability in that a neighbor can send too many prefixes, causing the BGP process to
consume enough memory to impact the overall operation of the router, to potentially including crashing the
router.

BGP Best Practice Version 3.0 27

Benefits
The maximum-prefix option provides a method to limit the number of prefixes the router will accept from
a BGP peer, with an option to print a warning (log the limit being broken) or to close the session.
Additionally, there will be a Syslog alert triggered when the number of prefixes received exceeds a
specified percentage of the maximum prefixes value.

Guideline
This feature is disabled by default. When enabled, the default behavior is for peering sessions to be
disabled when the maximum prefixes value is exceeded. If the ‘restart-interval’ argument is not
configured, a disabled session will stay down after the maximum prefix value is exceeded. The default
‘threshold’ value is 75%. A Syslog alert is triggered after receiving 75% of configured value with
maximum-prefix command.
This feature should only be deployed on peering sessions where the number of prefixes being learned is
predictable and stable. It would be unwise, for example, to deploy this on a transit link in the global
Internet, as the number of prefixes being announced on the internet is continuing to grow. Keeping the
maximum number of prefixes accepted would be creating an ongoing administrative task for the network
team. Use of the restart option is not generally recommended as this could lead to session flaps.

BGP Best Practice Version 3.0 28

If you still want to implement this feature. The behavior of route flap dampening for routes is modified by four configurable parameters.pdf Guidelines The RIPE and NANOG communities have agreed that RFD is not recommended anymore.pdf “ BGP Best Practice Version 3. the same router will receive multiple withdraw messages. Risks and Limitations The main drawback of RFD is well explained in the following study: “Route-flap Damping Exacerbates Internet Routing Congerence Sigcomm 2002 http://www.edu/~zmao/Papers/sig02. Applying dampening in such cases could result in loss of connectivity to name resolution services. This should be taken into consideration when deciding dampening parameters. follow the guidelines below.eecs. that widespread deployment of RFD would improve the overall convergence characteristics of the global Internet. or routes which changed state on a regular basis. a router will receive multiple updates pertaining to a destination network. dampening should not be applied to DNS servers.org/mtg-0210/ppt/flap. It was believed. Appropriate parameter selection is recommended depending on various factors like severity (frequency and duration) of flaps.nanog. For example. prefix-length. Both RIPE and NANOG communities agreed that RFD is not a best practice anymore. and encourage network operators to clean up any routes they had which might rapidly flap. For example. the decay half life parameter should be set to a time considerably longer than the period of the route flap it is intended to address. etc.html http://www. This can lead to over-penalizing the network prefix. availability of alternate routes. When the destination network become unreachable. More recent research on the impact of RFD highlighted that this mechanism is more harmful than the issue it cures. especially root DNS servers.net/ripe/docs/routeflap-damping. http://www. For example.ripe. dampening should be applied at the customer peering points in addition to applying it at upstream ISP peering points. Route Flap Dampening Background Route flap dampening (RFD) is a mechanism developed in the 1990’s to help stabilize BGP within the Internet network. It should be noted that one time network down can be seen as multiple Downs depending on EBGP sessions. • It is recommended that certain longer prefixes that are critical for access be excluded from dampening.0 29 . • It is advisable to apply route-dampening as close to the prefix being advertised as possible. This over penalizing can lead to suppressing the prefix unnecessarily for extended period of time. • Apply dampening to inbound announcements from eBGP peers only. dampening parameters can be cleared on access routers. This also helps ensure that after successfully repairing the problem related to prefix flapping. at the time RFD was developed.umich. When there are multiple EBGP sessions.

The later is preferred since it is flexible and gives more granular control.pdf http://www.nanog. Generally. The show ip bgp <route> router command will list the penalty and the state of the route. to be ignored for a period of time that depends on the duration of the flap. Also the fact that each AS may use a different MRAI (Min Route Adv Interval) reinforces this behavior called withdrawal triggered suppression. Click here for a Case Study on Route Flap Dampening. a route is assigned a penalty value of 1000 (a constant) for each flap. being announced and withdrawn constantly.ripe. By default. If the value of the route’s accumulated penalties exceeds 2000.org/mtg-0706/Presentations/PhilipSmith-BGP. Basically. Routes that flap constantly are suppressed (ignored) longer than routes that flap occasionally. i. The following presentation provides also an overview of this issue: http://www. the route will be tagged with a penalty value. Click here for More Details on Route Flap Dampening. the route will no longer be suppressed. BGP Best Practice Version 3.0 30 .e. This time will be extended for each additional flap that occurs. When the timer expires.. The accumulated is reduced every 5 seconds. when a route is withdrawn and announced in quick succession.net/ripe/docs/ripe-378. the route is suppressed until the penalty drops below 750. This penalty value will be increased for each flap.html Details The Route Flap Dampening feature of BGP causes a route that is flapping. Click here for Configuring Route Dampening. When the penalty value crosses a threshold. The route will be tagged as suppressed when this happens and a countdown timer will be started to indicate the amount of time the route will remain in the dampened state. it shows that a single route flap (a withdrawn followed by an announcement) will make the route to be dampened further in the backbone because several updates path (an attribute change is considered as a flap) will be generated due to the next best path selection in the previous hops. at a rate that the penalty is reduced by half every 15 minutes. Initially. Route dampening parameters can either be applied to all prefixes with equal penalties or to longer prefixes with higher penalty than the short ones. the route will be tagged with a history marker to indicate that it is known to be flapping. the route will be suppressed or dampened.

This leads to less memory utilization and savings in CPU and BGP update generation. with a small cost in convergence time.0 31 . Route reflectors. • Changing “next-hop” using a route-map on a route reflector can cause routing loops and it is not recommended unless absolutely required. Besides reducing the number of iBGP sessions. This comes. and it will only forward the best path from all of its clients to each of its normal iBGP peers. it follows that to have a complete set of routes. which allows the network engineer to reduce the size of the iBGP mesh. The redundant configuration increases the number of physical TCP connections required compared to the single-route reflector configuration. Route Reflectors Background To ensure any BGP router has complete routing information. help reduce the size of the iBGP mesh and the associated overhead. with each client physically connected to two route reflectors to ensure that BGP information continues to be reflected if there is a problem with one route reflector. one part at a time. and to reduce the size of the BGP tables of BGP speakers within the network. If the best path was received from a client. • Start by migrating small parts of the network. Because a BGP speaker will not send routes learned from one iBGP peer to another. Normal BGP speakers can coexist with route reflectors and route reflector clients. which may have hundreds or even thousands of routers in their network. Benefits Route reflectors add additional attributes to BGP updates within an AS to allow a router to reflect iBGP learned routes to other iBGP peers. Once the best path is selected. Clusters may be configured hierarchically–route reflectors in a cluster BGP Best Practice Version 3. perhaps. it is necessary for all BGP routers in an AS to have iBGP peering sessions with each other. it is reflected to all clients if received from a non-client. but still significantly streamlines the network topology compared with a full network mesh. the savings will be greater for routers lower in the network hierarchy. Guidelines • Divide the backbone into multiple clusters and use at least one route reflector and a few clients per cluster. because BGP updates now have to propagate through multiple routers (RRC-to-RRC relationships) to get from a BGP router. A route reflector topology results in considerable savings in the number of paths stored on routers in the network. Route reflectors and Confederations address this problem by allowing a router to advertise or reflect iBGP learned routes to other iBGP peers without requiring a full network mesh. Typically. For larger ISPs. A route reflector will only forward its best path to a client. • Route reflectors can be configured in a redundant fashion. it will be reflected to all non-clients and other clients. Clients can also peer with route reflectors in other clusters for redundancy. because only the route reflector must support this feature. the number of peering sessions clearly becomes unmanageable and difficult to implement given the limited CPU and memory on routers. all BGP speakers within an AS must peer directly with one another. a route-reflector hierarchy serves to reduce the number of routing updates transmitted through the network. then. This produces some real scaling problems once you have more than a few BGP routers in your network.

particular attention must be given to the usage of same versus different cluster IDs on these redundant RRs. • Route-reflector redundancy is recommended in order to allow multiple RRs to peer with the same RR Clients. Configuring peer groups within such a cluster could cause a withdrawal to the source of a route on the route reflector to be sent to all clients in the cluster. This can happen in scenarios when a link failure prevents an RR from learning the same route as other RRs in the cluster. Click here for RFC 2796. Risks and Limitations With earlier versions of IOS. can be clients of route reflectors in a higher level. Peering over physical interface addresses allows a physical link failure to tear down the BGP session thereby switching the RRC over to its “backup” RR. • As a rule of thumb. • When deploying or migrating to a route-reflector topology. This provides a natural method to limit routing information sent to lower levels.0 32 . which may or may not be desirable depending upon the topology. because . because multiple updates from within the same cluster are not kept after being received. all the clients were required to be fully meshed and “bgp client-to-client reflection” would need to be turned off on the route- reflector. The important drawback. they exchange updates through the route reflector. BGP Best Practice Version 3. is that routing can be disrupted in case of link failure. it is important to assess whether iBGP peering should be formed over physical interface addresses or over loopback addresses. When RR-RRC peering is over loopback addresses. it is important to follow the physical topology when deciding where to place the route reflectors. However. • When route-reflector clients (RRCs) peer with multiple route reflectors (RRs). however. the route reflector should be used only as a “route server” and not for switching high volumes of traffic or for other services. a physical link failure may retain or re-establish the iBGP session from an RRC over to the same RR via an alternate path. instead. Clients inside a cluster do not have direct iBGP peers. if peer groups were used for clients of a route reflector. The benefit of configuring multiple RRs with same cluster ID (which means they are in the same cluster) is that less memory is required in the router.

MED. the AS numbers would remain the same. If the external peer is another sub-as. the LOCAL_PREF. pre-append the confederation ID to the AS-PATH attribute. or outside the confederation). • Confederations are better to use in scenarios involving AS mergers. • The sub-autonomous systems within the confederation may use private AS numbers. and forward the route. • Route reflectors are much easier to migrate as you don’t have to deal with sub-AS numbers. rather than public ones. If the external peer is outside of the confederation. • For routes learned from any external peers (sub-as. • Because the NEXT_HOP is unchanged. remove the CONFEDERATION_SEQUENCES or CONFEDERATION_SET attributes. This would generally be done when migrating a single AS to a confederation. • For routes learned from other peers within the local sub-as. • If you already have large scale deployment of confederations keep using them and use RR within sub- AS to reduce iBGP mesh. but when merging autonomous systems into a single AS by combining them into a confederation. • The total number of neighbors is reduced by limiting the full mesh requirement to only the peers in the sub-AS. resulting in a CONFEDERATION_SET. BGP Best Practice Version 3. then pre-append the local sub-as number in the CONFEDERATION_SEQUENCE part of the ASPATH. but Cisco does not implement this. Confederations Background Some of the key characteristics of confederations are: • They are visible to the outside world as single AS. Details Route Propagation in Confederation Route propagation decisions are fairly similar to Route Reflectors—special treatment of AS paths is the key distinction.0 33 .e. send only to external peers. and NEXT_HOP attributes are forwarded without change. we can conclude that confederations are expected to run a single IGP. send the routes to all neighbors. • iBGP speakers in sub-AS should be fully meshed unless a route-reflector model is used. in the absence of any configured policy). the Confederation AS. • According to the BGP RFC sub-AS can aggregate. • By default (i. Guidelines • Both confederations and route reflectors can be scaled to a good extent.

BGP Best Practice Version 3.0 34 .Confederations require more configuration work upfront and usually require network downtime to activate. All BGP peers on a router must be reset if it needs to be assigned a new AS number. However. once the initial configuration work is complete. While confederations help reduce iBGP mesh themselves. It is difficult to transition to or from confederation because of multiple AS numbers involved. they can also be used with route reflectors. confederations give network operators a greater degree of management flexibility.

The misconfiguration of a BGP to IGP redistribution point has been known to cause large- scale network outages and should be avoided as a general rule. only eBGP routes are redistributed into an IGP. Guidelines Avoid redistributing routes from BGP into an IGP whenever possible. Typically.0 35 . The following command will enable the redistribution of iBGP learned prefixes: router bgp xxxx bgp redistribute-internal • When doing mutual redistribution. BGP-IGP Redistribution Policies Background Cisco IOS supports redistribution between all IP routing protocols. it is highly recommended that prefix filtering be configured to limit the prefixes allowed into the IGP. the number of prefixes may be more than the IGP is able to cope with. Even if prefix filtering is configured on the redistribution point. Benefits The main benefit of redistributing BGP into the IGP is to provide dynamic route advertisement of BGP learned prefixes throughout the network without requiring a pervasive BGP environment. Risks and Limitations Interior Gateway Protocols were not designed to handle the large number of prefixes that are typically found in BGP. When BGP to IGP redistribution must be performed. ensure route propagation is fully controlled in either direction otherwise it can cause routing loops. The redistribution of iBGP learned prefixes must be explicitly enabled. by default iBGP learned prefixes will not be redistributed. in any case. The redistribution of BGP into the IGP can cause network-wide instability. BGP Best Practice Version 3. This is especially the case when full Internet tables are involved. such as EIGRP. • Using default information originate would be the preferred mechanism. it is usually recommended to filter the redistribution to prevent uncontrolled prefix injection from destabilizing the entire network. as a protection mechanism. You should limit this only to the default route. If BGP must be redistributed into the IGP. unless you’re using an IGP that doesn’t support default information originate. The redistribution of BGP into an IGP is typically discouraged. • Be careful about routing loops that may occur when redistributing at multiple points. Hence. • Do not redistribute BGP routes matching on communities as it is unreliable and not supported. • It is recommended to configure a default metric to the routes that are redistributed. This also includes redistributing between BGP and IGP protocols. Redistributing iBGP routes has a potential to create routing loops.

Guidelines Some methods for ensuring next-hop reachability for externally learnt routes are: • Redistribute a static route to the eBGP peer into the local IGP • Redistribute the connected or static route which is used to reach the eBGP peer into the local IGP • Run the local IGP on the interface which is used to reach the eBGP peer advertising the route (normally the IGP would be configured not to build any neighbor adjacencies on this interface) • Set the next hop to the BGP peering address of the BGP speaker within the local AS which is receiving the routes When advertising to an iBGP peer. BGP Next-hop This section describes how next-hop reachability works in BGP. the next-hop is modified to the local peering interface – normally the connected interface address. else the prefix will not be installed as the next-hop will be unreachable. • When a route-reflector propagates a route learnt from an iBGP or eBGP peer the next-hop is not modified. BGP “next-hop-self” feature modifies next hop of only the eBGP learnt prefixes and not those learnt as an RR client or from iBGP peers. the next hop must be reachable before a router will insert the destination prefix into its routing tables. Next Hop Behavior with Route Reflection By default a Route Reflector will not modify the next-hop attribute of reflected (iBGP learnt) routes. • When a router announces an eBGP learnt route to an iBGP peer. Default next-hop behaviors: • When a router announces a prefix to an eBGP peer. If there is a requirement to modify the next hop of reflected routes or iBGP learned routes this must be accomplished with an outbound route-map. BGP Best Practice Version 3. In any peering scenario. Cisco implementations allow a RR to modify the next hop for eBGP routes being reflected to route reflector clients by configuring the bgp next-hop-self command. associated issues and best practices to address them. the next-hop is not modified • When a route-reflector reflects a route learnt from a router-reflector client (RRC) the next-hop is not modified.0 36 . In cases where a prefix is learnt via eBGP and subsequently announced via iBGP the default behavior listed above creates a requirement for the external nexthop to be learnt via an IGP. Background The BGP next hop attribute is the next hop IP address to use in order to reach a destination prefix.

BGP Best Practice Version 3. • Turn off the next hop calculation for an eBGP peer. • Configure inter-AS Multi Protocol Label Switching (MPLS) Virtual Private Networks (VPNs) by not modifying the next hop attribute when advertising routes to an MP eBGP peer. To propagate the next-hop of an iBGP learnt prefix without modification. This feature is useful for configuring the end-to- end connection of a label-switched path. Setting BGP attributes for a route reflector should be attempted only by an experienced network operator. use the option ‘allpaths’ keyword. “Next-Hop-Unchanged” feature To enable an eBGP peer to propagate the next hop unchanged. Incorrectly setting BGP attributes for a route reflector can cause inconsistent routing. routing loops. Except in certain inter-AS MPLS VPN scenarios this command should not be configured on a route reflector. and the neighbor next-hop-self command should not be used to modify the next hop attribute for a route reflector when this feature is enabled for a route reflector client.• Modifying the next hop at a Route Reflector can cause routing loops and is only advised when the network architecture requires it.0 37 . This command can be used to accomplish the following: • Bring the route reflector into the forwarding path. or a loss of connectivity. use the neighbor next-hop-unchanged command in address family or router configuration mode. which can be used with the “iBGP Multipath” Load Sharing feature to configure load balancing.

a BGP community is defined as a group of prefixes which share some common property. If all the prefixes to be filtered at a specific exit point are marked with a single community. BGP Best Practice Version 3. • Prefixes with identical routing policies. For example. some or all of the attributes. While communities themselves do not alter the BGP decision making process. A router has the option to add or modify a community attribute before it passes the attribute on to other peers. • Building filters at an AS exit point based on the AS from which the prefixes are learned would normally involve building a large and complex AS filter list. those that receive full-routes versus those that receives partial (direct customer-only) routes. • Prefixes learned from ISPs or peers. etc. Community Attribute Background BGP communities are used to simplify the control of routing information by providing a mechanism to group prefixes and make routing decisions based on these groupings. prefers or distributes to other neighbors.0 38 . • Assigning multiple communities to a prefix can be used to build a community “string”. In general. MED. AS path. • The community attribute provides more flexibility than many other prefix attributes for managing policies because it does not form part of the BGP best-path algorithm. • The BGP Named Community Lists feature allows the network operator to assign meaningful names to community lists. or a filter based on complex regular expressions. Communities are generally leveraged by the routing policy engine to set and/or modify other attributes in order to influence best- path decisions. This feature also increases the number of community lists that can be configured by a network operator because there is no limitation on the number of named community list that can be configured. • Prefixes learned from customers. The community scheme can significantly simplify the configuration required to control distribution of routing information. this filtering becomes much simpler. This allows other routers to act based on one. however. like attributes such as local preference. Guidelines It is considered a common practice to group prefixes into communities based on classifications such as the following: • Type of customer or peering AS. communities can be used as flags in order to mark a set of prefixes. A BGP speaker may use this attribute to control which routing information it accepts. Benefits The biggest advantage of the BGP community attribute is it provides a scalable method of implementing routing policy.

then the resulting aggregate should have a COMMUNITIES path attribute. To remove the existing community attribute. If a range of routes is to be aggregated and the resultant aggregates attribute section does not carry the ATOMIC_AGGREGATE attribute. By default. Implementation propositions are underway to have the default set to send communities for iBGP. use the “additive” keyword. the existing community string will be OVERWRITTEN. • local-AS: Do not send outside local AS in confederation (special case of no-export). we need to use the ip bgp new-format global configuration command. • Internet: Advertise this route to the internet community. and AA:NN. An enterprise network can use primary outbound filters based on communities to send to its ISP all of the routes originated in its network. Basic Configuration Below are basic configuration steps for setting BGP community attribute to receive BGP routes. For historical reasons. and then build the outgoing policy lists based on simple communities. To use and display the newer format AA:NN. if you use “set community AS:N” in a route map. The “well-known” communities should be understood and obeyed by all BGP4 implementations. Router(config)#ip bgp-community new-format There are well-known BGP communities defined by RFC 1997: • no-export: Do not advertise to eBGP peers. When advertising routes to the ISPs it is advisable to configure communities in agreement with the ISP. Useful to clear any communities associate with a route. • None: No community attribute. When configuring community attribute. Communities are carried across AS boundaries (transitive) only if send-community has been configured for the neighbor. which contains all communities from all of the aggregated routes. BGP community can be configured in 3 different formats: decimal. To append to the existing community string. internal or external. For example. • no-advertise: Do not advertise this route to any peer. it is advisable to use a prefix- list as a “backup” for the primary community filter. Cisco implementation supports decimal value for BGP community attribute. where AA is AS numbers and NN is an attribute assigned by the BGP AS administrator. where the number of prefixes in the network is small.) It is better to assign prefixes into communities at the edge of the network. RFC 1997 suggests the first two octets be an AS number presumably of the originating domain).0. and not send them for eBGP. RFC1998 describes a way in a multi-homed environment to indicate which of the ISPs is primary and secondary for a particular set of routes. use “set community none”. This does nothing more than to help protect the Internet against possible mistakes. hexadecimal. community is not sent by default – you need to enable it on a per-neighbor basis. With release 12. Keep this route within an AS. verify if any community tags already exist. • Prefixes in VPN (BGP community is fundamental to the operation of MPLS VPNs. communities play a crucial role in identifying families of routes within VPNs. Details RFC 1997 defines a BGP community as a transitive attribute with 32-bit binary value that can be applied to BGP routes.0 39 . and the second two octets may be defined by that autonomous system. By default. In smaller installations. BGP Best Practice Version 3. where the first part is the AS number and the second part is a 2 byte number. any router belongs to it.

1 remote-as 300 neighbor 3.0 neighbor 3.3.3.3.3.3 remote-as 200 neighbor 3.10.3.1 send-community neighbor 3.1 route-map setcommunity out ! route-map setcommunity match ip address 1 set community 200:200 ip access-list 1 permit 0. and community-lists now apply policy based on the set community attribute. Configuration Example RTR-B# ! router bgp 300 neighbor 3.3 route-map check-community in ! ip bgp-community new format ! route-map check-community permit 10 match community 1 set local-preference 200 ! route-map check-community permit 20 match community 2 exact set local-preference 90 ! route-map check-community permit 30 match community 3 ! ip community-list 1 permit 100:20 ip community-list 2 permit 300:200 ip community-list 3 permit internet ! BGP Best Practice Version 3.3.3.0 40 . RTR-A# ! router bgp 200 network 160. this allows filtering or the setting of attributes based on different lists of community numbers. Earlier configuration that we saw set the community attribute.0.0 255. community attribute will not be sent for those routes.0.3.255. Community-lists A community list is a group of communities which can be used in a match clause of a route-map.0.3.3.255 It is important to note that without neighbor x send-community command.255.

12. BGP Community Design Case Study This section covers a BGP community design case study to help better explain BGP community attribute use in BGP deployments. this command allows all other routes to come through. A named community list can be configured with regular expressions and with numbered community lists. For more information. The ISP has finalized the following community attributes for various route types: Table 1 BGP Community Design Route Type Community BGP Policy * ISP internal 10:50 Local Preference 100 routes Peer Routes 10:100 Local Preference 85 Preferred Peers 10:150 Local Preference 90 Customer 10:200 Local Preference 95 Routes - BGP Best Practice Version 3. Community-lists work just like access-lists and it important for all other routes to be allowed after applying community lists. else they will be filtered. BGP extended communities are 64 bits instead of the current 32 bits. both customers and transit. there are multiple drafts on extended BGP communities. and only allow routes that they chose. Named BGP Community-lists With release 12.0(16)ST.In the above example. It is important to realize what our policy is.0(10)S. Any prefix that has only 300:200 as community matches list 2. On the other hand. This feature introduces a new type of community list called the named community list.0 41 . The keyword exact indicates that the community should only consist of 300:200 and nothing else. and use appropriate implementation.2(8)T and 12. These drafts also provide a structure to the community attribute to address issues like route target and route originator indicators that help in associating routes to a L3VPN. In this case. Named community-lists do not have the limitation of 100 community groups that exist with standard and extended community lists. ISPs want tight control over routes they take into their network and in that case. Consider an ISP. please refer to extended communities draft on IETF web site. All rules of numbered communities apply to named community lists except that there is no limitation on the number of community attributes that can be configured for a named community list. Extended BGP Communities For applications which require larger communities to provide grouping of routes within BGP. Since Internet community means all routes. and has the added complexities of aggregate routes and routes for internal networks or affiliates. Cisco introduced the named community-list feature.1(9)E. using AS 10 that services customers in single connection mode and multiple connection mode with other connections used for backup or load balancing purposes. This ISP has many peers. any prefix that has 100:20 in the community attribute matches list 1. note the last community-list command. BGP named community lists allow the network operator to assign meaningful names to community lists and increases the number of community lists that can be configured. they will not use the community-list with Internet keyword. 12.

255.168.203. • Customers with IANA assigned AS numbers and IP addresses should be advertised toward peers. customer routes and affiliate routes.Aggregates aggregated.200.255. then specific routes should be advertised to peers.168.6 send-community ! neighbor 199.6 route-map set-community in neighbor 202. Specifics Customer routes 10:400 Customer routes that are . • ISP assigned customers routes should be aggregated towards peers. • If customer routes can be aggregated.168.0 as-set summary-only neighbor 202.79.9. Configuration on a peering router router bgp 10 aggregate-address 129. • Some customers that are single homed should get only the default route. Private AS’s and private IP addresses should not be advertised to peers. only aggregate should be advertised to peers.9.33 route-map peer-community out neighbor 199.200.9.79.79.33 send-community exit ! ! AS-Path ACL 71 permits all routes from peering ASs ! ip as-path access-list 71 permit ^201_ ip as-path access-list 71 permit ^651_ ! ip bgp-community new-format ip community-list 100 permit 10:50 10:200 10:400 10:500 ! BGP Best Practice Version 3.200.9.0 255.33 remote-as 651 neighbor 199.6 remote-as 201 neighbor 202.168. • Multi-homed customers have two options: either full Internet routing table or just selected ISP routes and some other critical routes for optimal routing.79.168.33 route-map set-community in neighbor 199. Advertised to peers Local Preference 95 ISP assigned 10:500 Aggregated towards peers.200. * BGP Policy is unique to each router based on its BGP connections The BGP routing policy for the ISP has following salient points: • All core routers should have all the routes including peer routes. • If customer routers are dual homed (connected to multiple ISPs) and have specific routes advertisement and AS-path prepend requirements. • If an ISP has preferred path through other ISPs for specific destinations.0 42 .6 route-map peer-community out neighbor 202. local customer routes preference 100. and those peers should be preferred for those destinations.

0 route-map customer-in permit 10 match ip address 1 set community 10:400 set local-preference 95 ! route-map customer-out permit 10 match community 100 end BGP Best Practice Version 3.70.0.7.7.7.70.1 remote-as 796 neighbor 144. route-map set-community permit 10 match as-path 71 set community 10:100 set local-preference 85 ! route-map peer-community permit 10 match community 100 end Configuration on a customer router The following example shows a scenario with the customer getting full Internet routing table and customer routes getting aggregated. router bgp 10 neighbor 144.70.70.0 43 .1 route-map customer-in in neighbor 144.1 route-customer-out out exit ! ip community-list 100 10:50 10:100 10:150 10:500 ! access-list 1 permit 144.

If it is enabled. If no BGP prefix information is inserted into the IGP. This means that the transit path between two iBGP peers contains routers that are not running BGP.0 44 . • There are non-contiguous BGP speakers. If a BGP learned prefix is not synchronized. A prefix is synchronized in BGP if there is a matching prefix in the IGP. Some form of route filtering should be used instead. BGP synchronization is enabled in older versions of Cisco IOS Software. the prefix will not be inserted into the routing table and will not be advertised to external peers. It is seldom that a network will benefit from BGP synchronization being enabled. Synchronization should not be used to prevent the transiting of traffic through a non-transit autonomous system. it is unlikely the BGP prefix would ever be able to become synchronized. The command to disable this feature is: router bgp xxxxx no synchronization Note that synchronization is off by default in recent IOS releases. Cisco recommends that BGP synchronization be disabled. Benefits The BGP synchronization feature is only beneficial in very specific environments. in which two requirements must be met. Guidelines It is common practice in virtually every BGP network to disable this feature. Synchronization Rules Background BGP synchronization feature refers specifically to prefix synchronization between BGP and the IGP. The purpose of the synchronization feature is to prevent traffic from being black-holed in networks that have non-contiguous BGP speakers. but in more recent versions. BGP Best Practice Version 3. • BGP prefix routing information is being redistributed into the local IGP. The default setting for BGP synchronization is to be enabled on older versions of IOS. BGP speakers will not advertise routes learned from an iBGP peer unless the destination described in the route is also reachable through the local IGP. it is disabled. The concept of a synchronized prefix is only relevant to iBGP learned prefixes.

It can also be achieved by increasing the number of links to the service providers and hence load share the traffic. This is commonly done through the use of different providers to provide a more diverse selection of paths to reach destinations. one of the following multi- homing scenarios can be used to provide Internet connectivity to an enterprise network. There would be logical redundancy but not physical. Multi-homing Background Many enterprises require continuous connectivity to the global Internet to provide email. The use of email and the web have become tightly integrated into the world economy and therefore are critical to the way business is done. but highly redundant connectivity. Customers usually prefer paths to different providers for improved redundancy reasons. connecting to the Internet. and business to business applications support. Keep in mind that multi-homing does not always provide physical path redundancy. More bandwidth does not necessarily mean increasing the pipe. BGP Best Practice Version 3. • Increased Bandwidth requirements As the traffic requirements grow. Multi-homing Scenarios Depending on the criticality of the connectivity and bandwidth requirements. It is in this capacity. web browsing. or multiple paths to different providers. This is highlighted in the case of two Service Providers that may be using the same physical equipment to deliver a circuit to the Enterprise. will provide the redundancy needed to ensure reliable service delivery. The term multi-homing has become quite common. So what does it mean to be multi-homed with respect to Internet connectivity? A network is multi-homed when it has more than one path to reach the Internet. • Optimal Routing The performance of connectivity to the Internet connectivity can be enhanced through multi-homing. many enterprises have turned to multi-homing. so does the need for more bandwidth from an enterprise to the outside world. that BGP is most commonly seen in enterprise environments. Multi-homing (when done correctly).0 45 . The tight integration of day-to-day business communication with the Internet has made Internet connectivity a mission critical service. Benefits There are a few primary reasons for multi-homing: • Reliability Internet connectivity has become a mission critical service in many environments. In order to support this continuous connectivity. This is commonly done through the use of different providers to provide a more diverse selection of paths to reach destinations. The performance of the Internet connectivity can be enhanced through multi-homing. This may be accomplished via multiple paths to a single provider. The requirement for Internet connectivity is not just for connectivity.

if the Enterprise CPE or service provider Access Router fails. the enterprise should typically receive a default route to the service provider. multiple static routes are used. • Some form of route filtering should be used to prevent the single router within the stub autonomous system from becoming a transit path for the peering AS. At most partial routes can be requested if needed. Risks and Limitations This setup provides protection against a link failure and covers for partial loss of connectivity in the service provider network. This is commonly achieved by using an as- path access list. Stub Network Multi-homed: Single Border Router This is a scenario where the enterprise connects to the Internet through a single service provider and (preferably) through different physical infrastructures. This is due to the single point of failure associated with this design strategy. • In the stub environment. • A single-homed stub network does not require the use of BGP. • Private AS numbers can be used since the enterprise is connecting to a single service provider. The enterprise uses a single router to achieve this connectivity. This type of multi-homing may be done via BGP Best Practice Version 3. The use of multiple connections in this design assumes they are all connected to the same routers. Receiving the full Internet routing table is not necessary in a stub environment. Guidelines BGP should be used for better traffic control in both directions thus achieving better load sharing across both links. • If multiple circuits are used between the provider router and customer router.Stub Network Single-homed Guidelines A single-homed stub network is one that connects to the Internet using a single connection to the service provider. connectivity to the Internet will be completely lost. Risks and Limitations Router failure will result in full loss of connectivity. • The provider will configure a static route for the customers prefix. However. The enterprise will configure a static default route. Stub Network Multi-homed: Multiple Border Routers This is a scenario where an enterprise network connects to the Internet using multiple border routers via a single service provider to achieve load sharing and resilience.0 46 .

Risks and Limitations In a rare event if service provider network is down completely due to a disaster. the traffic will not be black holed or looped. even if the transit routers do not have partial or full BGP routes. which should be investigated thoroughly before deciding to implement this solution) • The enterprise should then originate a default route from each border router.( Please note that there may be other disadvantages to using Tunnels on specific routers such as performance. or can be received from BGP and redistributed into the IGP. or a full routing table from the service provider. the default origination should be conditional on the circuit being up and active. partial routes. This is commonly achieved by using an as-path access list. Guidelines • BGP should be used for better traffic control in both directions thus achieving better load sharing across both links. Guidelines This design scenario requires the enterprise to obtain their own ASN from their regional registry. different POPs of the same ISP and preferably through two circuits which do not share the same physical infrastructure (CO. This way. fiber. Standard Multi-Homed Network: Single border router / Multiple ISP’s This is a scenario where an enterprise network connects to the Internet using a single border router and uses multiple links to multiple service providers. • Each of the customer border routers should eBGP peer with the service provider. Option 1 –Static Defaults The customer will use a static default route to each service provider. fragmentation etc. • Private AS numbers can be used because the enterprise is connecting to a single service provider.. Internet connectivity is lost. BGP Best Practice Version 3. • Some form of route filtering should be configured to prevent the customer’s autonomous system from becoming a transit path for the service provider. Depending on the resources available one of the following options can be considered. In order to prevent traffic from following a default route to a border router with a failed upstream circuit. do not redistribute this into any IGP process running on the border routers. depending on the level of outbound traffic control the customer would like to achieve. If additional prefix information is received from the upstream provider. The enterprise will also need a block of address space that is large enough to pass standard peering filters. The customer may accept a default route. This conditional advertisement can be based on a static default pointed at the interface. If this is not feasible. or other elements). all border routers should build iBGP adjacencies with each other through direct L2 link or through virtual interfaces such as GRE Tunnel.0 47 . • All customer border routers and all transit routers interconnecting the border routers should speak BGP and establish an iBGP full mesh between each other.

The customer should configure some form of route filtering to prevent the customer’s autonomous system from becoming a transit path between the two service providers. This can be resolved by asking the provider to send default + partial routes. This option has the advantage that out going traffic from enterprise destined for the connected providers locally originated networks and their directly connected networks gets optimally routed. Option 2 –ISP Provided Defaults The customer will use the default routes advertised by each service provider. A default route directed at each provider would still be required to reach any destinations that are beyond the immediate upstream providers and their customers. This will allow the enterprise border router to send traffic to the upstream provider that is logically closest to the destination. This logical distance is derived from the AS_PATH.This option allows for a very even sharing of outbound traffic. Typically. If the AS_PATH is the same length. the traffic will be sent to the upstream provider whose path had the lowest ROUTER_ID. This method results in the greatest resource requirements.0 48 . This is commonly achieved by using an as-path access list. The customer should request a default route and a set of partial routes from the two service providers. This traffic can be balanced between the two if eBGP Multi-Path is used. The enterprise can request the upstream providers send only their locally originated prefixes and those prefixes for their customers. Traffic destined for other networks may still take sub optimal routing because the router uses the default route for these networks. BGP Best Practice Version 3. Risks and Limitations Accepting full routing tables requires more resources (memory / cpu) on the border router. Option 3 – Receive Partial Routing The next option is to receive partial routing information. But it leads to sub-optimal routing because traffic destined for provider A or its directly connected customers may go through the other service providers. Refrain from implementing unfiltered redistribution of BGP routes into the enterprise IGP. Page: 48 Again. This is preferred over static default routes. Option 4 – Full BGP Routes The enterprise can also receive full tables from both providers. The customer should configure some form of route filtering to prevent the customer’s autonomous system from becoming a transit path between the two service providers. you want to advertise a default via your IGP. this design still poses a single-point-of-failure scenario whereby if the enterprise border router fails Internet connectivity is lost. Consideration has to taken with regard to how to advertise the routes within the enterprise AS. The following options require eBGP peering with the service providers. This will result in the enterprise being able to correctly route traffic destined to either provider.

the enterprise network connects to the Internet using multiple border routers and uses multiple service providers for resilience and load sharing.0 49 . Depending on the resources available one of the following options can be considered. Preferably. o The customer should be aware that if the service provider from which they have obtained an address block from aggregates the supplied addresses into a longer block. Option 2 Accept full routing updates from each upstream service provider. If required. Static defaults can also be used in conjunction with full routes if needed depending on the eBGP connectivity. • The enterprise will also need a block of address space that is large enough to pass standard peering filters as mentioned above. Risks and Limitations This option involves more complex configurations depending on the amount of level load sharing desired.Standard Multi-Homed Network: Multiple border routers In this scenario. This option requires additional default routes on each of the border routers to reach all other destinations. o The customer should be aware that some service providers will not advertise routes with a prefix length longer than some local policy. This will prevent a layer 2 failure from impacting both paths to the global Internet. BGP Best Practice Version 3. the two links to the two service providers will not share the same physical infrastructure. the attributes of the incoming updates can be altered to achieve required load sharing if the AS- PATH is the same. Option 1 Request upstream service providers to send partial routes (which will include their local prefixes and their directly connected customer networks). • eBGP should be configured with each of the service providers. Any address block obtained should fit into the local policy for longest routable prefix length for both service providers. Guidelines • This design scenario requires the enterprise to obtain their own ASN from their regional registry. • All the enterprise border routers and the routers in transit between border routers should build a full iBGP mesh. or work with the two service providers to obtain a block from one of the two service providers which the other service provider will agree to advertise. This is commonly achieved by using an as-path access list. traffic will be directed to their network in an asymmetric way. • The customer should obtain a publicly assigned block of address from the national registry. The customer should configure a route filter of some type which prevents the customer’s autonomous system from becoming a transit network between the two service providers.

When accepting full routes. Figure 2 Single-homed Enterprise Stub Network with Border Router This situation does not require the use of BGP. who owns the customer address space. sub optimal routing can occur or the enterprise may end up using only one service provider for the incoming traffic. It is important to check utilization of resources in such cases since memory usage increases a lot. Multi-homing Scenario Examples The following examples demonstrate the techniques described in the above section. This situation requires that provider A also advertise a specific route along with the summary. Load-balancing scenarios can typically be achieved when routes come from the same ISP. These filters are used to allow only prefixes of certain length or less. Service providers usually summarize the address space they assign to different connected customers and advertise only a summary to their customers and their upstream providers. This can lead to sub optimal inbound traffic when multi-homing to different service providers.0 50 . If the prefix-length for the customer’s address space is not large enough to pass these filters. BGP Best Practice Version 3. may summarize the address space to a smaller prefix length and other providers may advertise the actual prefix (larger prefix length than the summary). Default static routes out each link to the service provider will do the job. In such case most of the inbound traffic to enterprise prefers provider B. single homed Let us assume an enterprise network with a single BGP router that connects to the Internet using a single SP. The illustration begins with a simple sub network with single ISP connection and builds over this setup to achieve more complex scenarios. The concerns in “Stub Network Multi-homed: Multiple Border Routers” for how to advertise the eBGP learned routes to the rest of the AS still apply. The service provider will also have static routes to the address space of enterprise and advertises this network to their upstream providers and its customers. Scenario 1: Enterprise stub network. Provider A. let us assume that there are two links from the border router to the SP router. additional resources on the devices are required. In this scenario. Almost all of the service providers use filters at their public or private peering points.

There is increasing trend to use of BFD is on Ethernet circuits between a data center and SP. BGP Best Practice Version 3. Traffic is load balanced across both links. Bidirectional Forwarding Detection (BFD) on the circuits between the enterprise and SP may be required to cater to the needs of quicker link failure detection and BGP fallover to alternate path. The caveat of this design is that the single border router becomes a single point of failure. BFD support on the SP and Customer equipment needs to be checked.0 51 . This meets the additional bandwidth requirements and also protects against single link failure. Use of aggressive BGP timers instead f BFD is not recommended due to the problems witnessed by multiple customers and service providers. BGP should be configured to achieve load sharing and to achieve control of inbound and outbound traffic patterns. Scenario 2: Stub network multi-homed to single SP with single border router Figure 3 Stub Network Multi-homed to Single SP and Border Router This scenario assumes that the enterprise has a single border router with two connections to the same SP. By default the link failure detection can be delayed up to 3 minutes. The connections are terminated at the service provider in different locations.

All the enterprise border routers including the ones in the transit path should run BGP and should be fully meshed. The enterprise border routers should originate a default and this default should be generated on the condition that the link to the SP is operational to avoid black holing and looping of traffic. BGP Best Practice Version 3. BGP should be configured to achieve traffic control and load sharing. Subnets of the enterprise space can be advertised from each border router to achieve inbound traffic load sharing.Scenario 3: Stub network multi-homed to single SP with multiple border routers Figure 4 Stub Network Multi-homed to Single SP with Multiple Border Routers This scenario assumes that the enterprise has multiple border routers that connect to the same SP at different points of presence.0 52 . Use of BFD may again be a requirement in this scenario as well subject to need and support both on SP side and enterprise side equipment. Private ASN numbers can be used because of single SP used.

Use of BFD may again be a requirement at critical enterprise sites such as data centers as explained under scenario 2. This recommendation should be made subject to need and support both on SP side and enterprise side equipment. However. etc. for external internet connectivity.0 53 . MPLS and Internet SP’s can be same SP as well. Private IP addresses would suffice for internal enterprise connectivity between datacenter. static defaults may be required to achieve complete connectivity. and remote branches. The enterprise can obtain partial internet routes or full internet routing table depending on the extent of load sharing that needs to be achieved and cost of equipment requirement. BGP Best Practice Version 3. The more number of router. For this purpose. Additionally. higher the memory and processing power is needed at the enterprise router. The large enterprise is doing multi-homing to the same SP as well as multiple SPs depending upon its geography as well as depth of network redundancy needs. enterprise should obtain an address block from the Internet SP and use NAT at its internet gateways unless enterprise can afford to own large enough internet addressing space so that every device in the enterprise network has public IP address. head office.Scenario 4: Standard Network Multi-homed to Multiple ISPs with Multiple Border Routers Figure 5 Standard Network Multi-homed to Multiple SPs with Multiple Border Routers Inter-AS between AS100 and AS200 MPLS SP A Internet SP MPLS SP B AS 100 AS 500 AS 200 NAT NAT Enterprise AS 65001 This scenario is the most complex scenario in terms of connectivity. Enterprise should obtain its own AS number from the local registry but it can use private AS number if it is only connecting over an MPLS network to connect to remote branch offices with its datacenter and head office. Customer has multiple border routers connecting to multiple SPs. BGP should be definitely used to achieve load sharing and redundancy at least at critical hub sites such as data centers.

BGP Best Practice Version 3. • If the enterprise is multi-homing to different service providers. • Another way to influence inbound load balancing is to use an advertise-map. Inbound Load Balancing • If the enterprise is multi-homing to the same service provider. In reality. AS-PATH can be pre-pended while advertising the address space to the service provider where heavy utilization is observed. When load sharing is discussed. BGP will advertise a configured network to the service provider. but this is still no guarantee of 50/50 load balancing. With an advertise-map. customer can monitor a prefix and if the prefix vanishes. To achieve redundancy. • Another option is to use communities to tag the enterprise address space prefixes in accordance with the service provider’s policies so that the provider can take further action based on their values while advertising them to their customers and upstream providers. inbound traffic and outbound traffic should be treated separately. Outbound Load Balancing Scenario 1 If the enterprise is multi-homed to a single service provider. Guidelines While optimizing traffic flows. Perfect load balancing with any routing protocol is almost always impossible. as long as there are many distinct flows in the traffic mix. this does not provide adequate level of redundancy. the network only controls the load sharing of outbound network traffic. The goal is to achieve load sharing as close to equal distribution of traffic as possible. these smaller subnets can be advertised with different AS-PATH lengths to the ISP via different links. the address space can be broken down into smaller subnets. what is really meant is managing configuration to attain optimal traffic patterns. simple default routes pointing out each of the links may achieve the goal. Load Balancing Background Load Balancing is the ability to divide the traffic over multiple links to achieve even distribution.0 54 . The strategies for load sharing inbound traffic and for load sharing outbound traffic are separate and often work independent of each other. However. and BGP is no exception. Each subnet can then be advertised out separate links to achieve the load balancing goal. Inbound load sharing can be achieved by manipulation of certain BGP metrics.

For multiple paths to the same destination to be considered as multipaths. The best paths or multi-paths are then installed in the IP routing table of the router. A static route to the remote loopback is configured for each interface. BGP Best Practice Version 3. . local preference. One is to accept partial routes from each of the providers and use defaults for other traffic. Multi Exit Discriminator (MED). The eBGP sessions are tied directly to the interface addresses. The other option is to accept full routing updates from each of the service providers and filter routes to receive mutually exclusive partial routes from each of the providers. it is generally a good practice to have default route along with more specific routes in order to achieve complete connectivity Scenario 3 If multiple links are used to create multiple eBGP sessions between the enterprise router and the service provider router for additional bandwidth and/or redundancy. iBGP multipath provides for load balancing iBGP traffic between iBGP neighbors. Scenario 4 (iBGP Multipath) Similar to the eBGP multi-path feature. . The iBGP multipath load sharing feature enables the BGP speaking router to select multiple iBGP paths as best paths to a destination. Option 1: eBGP Multi-hop feature eBGP multi-hop feature uses a single eBGP session between the two routers. The attributes include weight.0 55 . One of the following options can be used to achieve outbound load balancing in such a scenario with option (1) as a preferred solution. which are identical except for neighbor address from which the path was received. with the eBGP session being sourced from a loopback instead of a physical interface. one for each link. there is a potential for using only one link because eBGP sessions are tied to the physical interface address of the link to the provider. The eBGP multipath feature will allow the router to install all paths up to the maximum-paths value configured. This provides the next-hop resolution and load balancing through recursive routing to the next-hop. .Scenario 2 If the enterprise is multi-homed to different providers. you need trial and error to decide this distribution) In each of the above cases. AS-PATH filtering can be used to achieve this. An eBGP session is configured between the two routers for each link. and Interior Gateway Protocol (IGP) distance. there are several options. Option 2: eBGP Multipath feature The eBGP multipath feature provides another solution to this problem. Local-pref based on some prefix-distribution (usually. autonomous system path (entire attribute and not just length). All attributes must be the same. The result is both routers receive multiple paths. the following criteria must be met: . This may achieve the required load sharing in some cases. origin code.

Over time.2S and IOS XR. the BGP speaking router will still designate one of the multipaths as the best path and advertise only this best path to its neighbors. Even if the criteria are met and multiple paths are installed. **** Scenario 5 (Link Bandwidth) The BGP Link Bandwidth feature is used to advertise the bandwidth of an autonomous system exit link as an extended community. When a router receives a route from a directly connected external neighbor and advertises this route to iBGP neighbors. The link bandwidth extended community attribute may be propagated to all iBGP peers and used with the BGP multi-path features to configure unequal cost load balancing. Hence the distribution of traffic occurs in proportion to the egress bandwidth. two paths to two next-hops.e. E. leading to a black-hole. Load balancing almost always brings some amount of asymmetric flows with traffic for a flow exiting a particular link while traffic belonging to the same flow entering the other link so this should be accounted for while designing networks. The fix suggested is to use "send-label" with BGP.CEF/LFIB picking an incorrect LDP label for the BGP next-hop.0S. with two BGP next hops and two core links CEF is not taking all four paths (i. traffic flows may change due to changing applications and application connectivity requirements in the enterprise. This problem is not present in 12. which is not supported in 12. BGP load sharing over an MPLS core does not work properly in 12.0 56 . The link bandwidth extended community indicates the preference of an autonomous system exit link in terms of bandwidth. the router may advertise the bandwidth of that link. At that time the whole exercise may need to be repeated.g. each via two outgoing interfaces as this requires in CEF two levels of recursion. Also be aware of CSCsb52253 . BGP Best Practice Version 3. The next hop router for each multipath must be different. Risks and Limitations While achieving load balancing by multi-homing to multiple service providers. . The BGP Link Bandwidth feature is supported by the internal BGP (iBGP) and external BGP (eBGP) multi-path features. the process may need multiple iterations to achieve the desired load sharing.0S.

. customer can monitor a prefix and if the prefix vanishes.15. Using this feature. BGP will advertise a configured network to the service provider.7. The disadvantage of AS-PATH pre-pending is that it doubles the prefixes in the upstream providers’ BGP tables.0. The other way of doing this is to use AS-PATH pre-pending. the enterprise router can be configured to advertise the address space of the other ISP2 to ISP1. When ISP2 link fails.7. BGP Best Practice Version 3. With advertise-map.0 57 .Details Using Advertise Map for Inbound Load Balancing A way to influence inbound load balancing is to use advertise-map. Example router bgp 500 neighbor <R1> advertise-map am non-exist-map bb ! access-list 1 permit 10..0/24 also to ISP1.15.15. 1.0 is not learnt through ISP1. similar configuration can be done to advertise one ISP’s address space to the other and vice versa in case of a failure of either of the links.0.0 !Advertise this when.15.0.0/24 is advertised to ISP2.0 !… this disappears route-map am permit 10 match ip address 1 route-map bb permit match ip address 2 This scenario achieves the purpose of inbound load sharing and in case of a failure the address space is still reachable via the other ISP. Make sure that 10. In situations of multi-homing with two ISPs.10. access-list 2 permit 10.15. it advertises 10.7.0/16 is advertised to ISP1 and 10. This technique is also called conditional advertisement Figure 6 Use of advertise-map to influence inbound traffic When all links are working fine. advertise-map monitors the prefix of the link between the R2 and ISP2 and when the prefix vanishes from the routing table.

This requires that all other attributes for both the paths be the same.1.Using EBGP Multipath for Outbound Load Balancing EBGP multipath can be used to load balance traffic across two links from an enterprise to the same ISP.0.18.1.1. BGP can install up to eight equal paths in the IP routing table. BGP multipath when enabled installs both the routes in the routing tables and hence load balancing can be achieved.6 remote-as 100 neighbor 10. When enabled.0 neighbor 10.1. Figure 7 Using EBGP Multipath for Outbound Load Balancing Sample Configuration Enterprise Router router bgp 65100 no synchronization bgp log-neighbor-changes network 172.1.0 58 . Typically BGP chooses only one of the paths and the tie breaker is the router-id in case all other attributes happen to be the same. To overcome this issue.10 remote-as 100 maximum-paths 3 no auto-summary ! ISP Router router bgp 100 no synchronization bgp log-neighbor-changes BGP Best Practice Version 3.1.2 remote-as 100 neighbor 10.

Link Bandwidth is an extended community of type 0x0004. Link bandwidth is propagated to IBGP peers only. Figure 8 Using BGP Link Bandwidth for Outbound Load Balancing Sample Configuration R1# router bgp 100 bgp dmzlink-bw maximum-paths ibgp 6 BGP Best Practice Version 3. The BGP Link Bandwidth feature is supported by the internal BGP (iBGP) and external BGP (eBGP) multi-path features. When used along with EBGP multi-path the bandwidth that is advertised to IBGP peers is the sum of DMZ link bandwidths of all EBGP multi-paths.1.0.1.5 remote-as 65100 neighbor 10.19.0 neighbor 10.2.1. The attribute is stripped while sending to EBGP peers. The link bandwidth extended community indicates the preference of an autonomous system exit link in terms of bandwidth. When a router receives a route from a directly connected external neighbor and advertises this route to iBGP neighbors.1. the router may advertise the bandwidth of that link.1 remote-as 65100 neighbor 10. network 172. The link bandwidth extended community attribute may be propagated to all iBGP peers and used with the BGP multi-path features to configure unequal cost load balancing.0 59 . Unequal cost load balancing with BGP using link bandwidth and BGP multi-path is supported in 12.1.1.9 remote-as 65100 maximum-paths 3 no auto-summary ! Using BGP Link Bandwidth for Outbound Load Balancing The BGP Link Bandwidth feature is used to advertise the bandwidth of an autonomous system exit link as an extended community.

5.6.1 dmzlink-bw Click here for Additional Details on BGP Link Bandwidth.10.1 dmzlink-bw R3# router bgp 100 neighbor 20.1.1 send-community extended neighbor 172. BGP Best Practice Version 3.10.5.0 60 .1.1 send-community extended neighbor 172.6.4.4.R2# router bgp 100 bgp dmzlink-bw maximum-paths 6 neighbor 10.1 dmzlink-bw neighbor 172.

However. Once set. Benefits Aggregation reduces the number of prefixes resulting in memory utilization savings since the size of the BGP and IP routing table is reduced. the router will fill in the AGGREGATOR attribute of the route with its own AS number and router ID. For aggregates. An AS-SET includes AS-PATH information from all more specific routes that contributed to the aggregate. When a route is generated as an aggregate route using aggregate-address. it causes the router to generate an AS-SET within the AS-PATH. they may choose to use the aggregate-address in order to include AS-SET and COMMUNITY summarization information in the aggregate route. There is no need for the Internet to know about more specific routes in a particular network or AS. the AS path is still needed for loop detection. so the aggregating router. you can track down the AS and router that is doing it. When the as-set keyword is included. If you use a network statement to generate your summary routes. an address block with a lower mask is advertised in order to summarize the various sub-prefixes used in the autonomous system. Typically. if the as-set keyword is not used. This can be useful for troubleshooting. on the other hand is less stable. Aggregation of routing information also increases stability—aggregate stays even if specifics come and go. AS-PATH information from the more-specific routes is lost. inserts all AS that BGP Best Practice Version 3. • When using the network command. the AS sequence information associated with the contributing routes is meaningless. However. The great advantage of using aggregate command is that the aggregate is dynamic (it may/may not exist and the attributes may change). you also need a static to null0. this attribute should never be removed by other routers that forward the route throughout the Internet. The command includes both net and mask. This minimizes flapping of routes and eases the mandate of route dampening. Route Aggregation Background Aggregation is the process of combining the characteristics of several different routes in such a way that a single route can be advertised. • ISPs can also use the network command to generate these summary routes. the route sets the ATOMIC- AGGREGATE attribute in the aggregate BGP route. Guidelines • An enterprise network’s address space should allow aggregation of its addresses to reduce the amount of extra information to be carried by each upstream provider.0 61 . • When using the aggregate-address command. Aggregation requires generating a stable summary (or supernet) route that covers all of the subnets in the network. The network command will not do this. the router will also include BGP community information from all of the more specific routes in the community attribute of the aggregate route. the atomic aggregate is not set. If you find someone is mistakenly advertising your address space as part of an aggregate. • The first step in configuring route aggregation with BGP is to specify the aggregate-address command to define the aggregate prefix to be generated. To signify this. Besides generating an AS-SET. Service Provider assigned address space aggregates routes from customers in order to minimize the size of the Internet route table.

Different methods of aggregation can be used depending on how the summaries and specifics need to be propagated. rather. • Use a route-map to adjust other BGP attributes. the upstream provider must ensure that all the components of the aggregate are contained within its administrative domain. When a route is permitted through the suppress map. If the route is not permitted (denied). BGP Best Practice Version 3. and contributed to by AS 10 and 20 (since 10 and 20 are in the AS set). the aggregate address command does not filter out the more specific BGP routes. Also. IP access lists and autonomous system path access lists match clauses are supported. the summary-only keyword needs to be used. can be redistributed from an IGP. You can use the match clauses of route maps to selectively suppress some more-specific routes of the aggregate and leave others unsuppressed. into the AS SEQUENCE .0 62 . it prevents it from being suppressed. For example. To do this. the provider may have to leak the "holes" of the non-provider /24’s back into it’s AS and to all its customers to ensure reach ability. Details By default. This is applicable in a scenario where the network is multi-homed to a single provider at different locations or multi-homed to different providers when the enterprise has provider independent address space. if a network connects to ISPs in two different locations. The more specific route can be injected in the BGP routing table by incoming updates from other AS areas. This situation calls for specialized configurations to ensure connectivity to all the components of the /16 inside and outside the provider AS. also known as proxy aggregation. and then puts it’s own AS number. say 4. Note that the deny logic here does not prevent the route from being advertised. if a route has an AS PATH of 60 50 {10 20}. the summary-only keyword needs to be used. aggregate was generated by AS 50. Risks and Limitations BGP will automatically summarize on classful boundaries. contribute to the aggregate route into the (unordered) AS SET say. To do this. the route is suppressed. Specifically. When aggregating. and generates two aggregates. The way aggregates are formed and advertised and whether they carry with them more specific routes will influence traffic patterns and sizes of BGP routing tables. • For the aggregate address to enter the BGP table there must be at least one specific route in the BGP table. Advertising summarized information to an upstream provider could result in sub-optimal routing decisions for inbound traffic through the provider into the network. • In certain instances the provider will do the aggregation for the network. and traffic to the other will come in through the other link (with both links providing a backup to the other). BGP aggregation applies to routes that exist in the BGP table. As mentioned earlier. at least one valid sub-prefix must be present in the routing table for the aggregate to be considered valid. An aggregate can be sent if at least one more specific route of that aggregate exists in the BGP table.4 {1 2 3}. or can be established by the network router configuration command. • More specific routes than the aggregate should be filtered using a prefix-list. the MED attribute can be set so that traffic for one aggregate will come in through one link. • By default. For example. but this default setting should be turned off using no auto-summary in networks relying on CIDR and VLSM since it takes control of summarization out of the hands of the engineer. the route is not suppressed--that is. Using the suppress-map keyword creates the aggregate route but suppresses advertisement of specified routes. {1 2 3}. The suppress-map is another form of route-map that can be used to indicate the more specific routes to be suppressed or the more specific routes to be allowed. the aggregate address command does not filter out the more specific BGP routes. allowed. it means that it was learned from AS 60.

16.0 63 . A high value for the administrative distance (254) can be optionally used so that the static route gets used only as a last resort within the router.0 255. It is important to note that the aggregate is created only if components are learned. the router will always prefer a more specific route first.16.0 summary-only The above configuration uses the aggregate-address command to aggregate all the more specific routes of 172. Note that the static entry is pointing to null0 (bit bucket).0 mask 255.255.50.16.0/16 into a single address. as long as more specific routes are available.255.0 mask 255.0.0 command allows a more specific route to be originated by the router so that aggregation can work.0/16 in the routing table.16. Aggregation using the aggregate-address command As explained.16.0 aggregate-address 172.255.255.0.0.255.0. the aggregate-address command provides more control over propagation of summaries and specifics.0 null0 254 The preceding configuration places a static instance of 172.0. Additional details on Route Aggregation BGP Best Practice Version 3.Configuration Example Aggregation using the network command router bgp 65000 network 172.0 ip route 172.50.16.255.0.16. The network 172.255.0. The summary-only argument at the end of the aggregate-address command tells the router to advertise the aggregate only.0 mask 255.0.0 255. However. Example: router bgp 65000 network 172.

The following formula shows this behavior: network/length < ge ge-length < le le-length <= 32 A prefix list is configured with a name and/or sequence number. An implicit deny is applied to traffic that does match any prefix-list entry. Prefix lists are configured to match an exact prefix length or a prefix range. The IP address can be a classful network. providing more flexible configuration than can be configured with just the network/length argument. or a single host route. if the first configured sequence number is 3. If only the ge value is entered. 5. Prefix-list counters can be reset by entering the clear ip prefix-list command.0. One or the other must be entered when configuring this command. If the ge. the range is the value entered for the ge ge-length argument to a full 32-bit length. Compared to access-lists prefix lists provide for a significant performance improvement as it requires fewer processing cycles. If only the le value is entered. 13.0. To deny the default route 0. The bit mask is entered as a number from 1 to 32.0. If a sequence number is entered for the first prefix list entry but not subsequent entries. Prefix list information and counters are displayed in the output of the show ip prefix-list command.0 64 . 18. 10. the range falls between the values used for the ge-length and le- length arguments. le-length keywords and arguments are entered. The ge and le keywords are used to specify a range of the prefix lengths to match. then the subsequent entries will also be incremented by 5 (For example. the range is from value entered for the network/length argument to the le le-length argument. and onwards). a subnet. Other Miscellaneous Areas Prefix Lists Prefix lists are used to filter IP prefixes and can match both the prefix number and the prefix length (mask). Default sequence numbers can be suppressed by entering the no form of this command with the seq keyword. ge-length and le. 15. Background The ip prefix-list command is used to configure IP prefix filtering. Once a match is made that covers the network the permit or deny statement is applied to that network and the rest of the list is not evaluated The prefix list is applied to inbound or outbound updates for specific peer by entering the neighbor prefix- list command.0/0 BGP Best Practice Version 3. and onwards). A prefix list consists of an IP address and a bit mask. If a sequence number is not entered a default sequence number of 5 is applied to the prefix list and subsequent prefix list entries will increment by 5 (for example. It also provides greater flexibility and supports incremental list updates. Prefix lists are evaluated starting with the lowest sequence number and continues down the list until a match is made.0/0: ip prefix-list abc deny 0.0. Examples The following examples show how a prefix list can be used. Prefix lists are configured with permit or deny keywords to either permit or deny the prefix based on the matching condition. then subsequent entries will be 8. The prefix list is processed using an exact match when neither the ge nor le keyword is entered.

0.1/24: ip prefix-list abc deny 192.0.0. http://www.168. Only those prefixes that are equal to or more specific than the original prefix may be injected. which are injected into the BGP routing table only if the configured conditions are met.0/8 The following examples show how to specify a group of prefixes.0/0 ge 8 le 24 To deny mask lengths greater than 25 bits in all address space: ip prefix-list abc deny 0.0.0.0/0 ge 25 To deny all routes with a prefix of 10/8: ip prefix-list abc deny 10.0.0/16 le 24 To deny mask lengths greater than 25 bits in routes with the prefix 192/16: ip prefix-list abc deny 192.168. The BGP Conditional Route Injection feature is enabled with the bgp inject-map exist-map command. To accept a mask length of up to 24 bits in routes with the prefix 192/16: ip prefix-list abc permit 192.0/8: ip prefix-list abc permit 10.0.0.0.0.168.168.html#wp1001470 Conditional Route Injection Background BGP uses conditional route injection to inject more specific prefixes into a BGP routing table over less specific prefixes that were selected through normal route aggregation.0/0 le 32 More information on Prefix-list configuration can be found at.0.0 65 .1.0/16 ge 25 To permit mask lengths from 8 to 24 bits in all address space: ip prefix-list abc permit 0. These more specific prefixes can be used to provide a finer granularity of traffic engineering or administrative control than is possible with aggregated routes.com/en/US/docs/ios/12_2/ip/configuration/guide/1cfbgp. This command uses two route BGP Best Practice Version 3. To permit the prefix10.0/24 ge 25 To permit all routes with a prefix of 0/0: ip prefix-list abc permit 0. This feature allows more specific routes to be generated based on administrative policy or traffic engineering information in order to provide more specific control over the forwarding of packets to these more specific routes. Enabling this feature will improve the accuracy of common route aggregation by conditionally injecting or replacing less specific prefixes with more specific prefixes.0.0. The BGP Conditional Route Injection feature allows originating a prefix into a BGP routing table without the corresponding match.cisco.0.0/8 le 32 To deny all masks with a length greater than 25 bits routes with a prefix of 192.

verify that the aggregate prefix is being received from the correct neighbor and the prefix list identifying that neighbor is a /32 match. Route-injection is not neighbor centric. advertising certain networks that are already in the BGP table to a given EBGP neighbor based on the existence of another particular network in the BGP table. Benefits The BGP Conditional Route Injection feature allows injecting more specific prefixes into a BGP routing table over less specific prefixes that were selected through normal route aggregation. maps (inject-map and exist-map) to install one (or more) more specific prefix into a BGP routing table. The exist-map specifies the prefixes that the BGP speaker will track. Ensure that the inject route map is configured with the set I address command and not the match ip address command. There are fundamental differences between the two.1. Guidelines The BGP Conditional Route Injection feature is based on the injection of a more specific prefix into the BGP routing table when a less specific prefix is present.1. If conditional route injection is not working properly. Conditional Route-Injection creates new routes in the BGP table if a particular route does exist in the BGP table. The inject-map defines the prefixes that will be created and installed into the local BGP table. These more specific prefixes can be used to provide a finer granularity of traffic engineering or administrative control than is possible with aggregated routes.0 66 . . check the following: . . Configuration Example This following configuration example configures conditional route injection for the inject-map named ORIGINATE and the exist-map named LEARNED_PATH. Conditional Advertisement is neighbor centric.0/24 BGP Best Practice Version 3. If the aggregate prefix exists but conditional route injection does not occur. router bgp 109 bgp inject-map ORIGINATE exist-map LEARNED_PATH ! route-map LEARNED_PATH permit 10 match ip address prefix-list ROUTE match ip route-source prefix-list ROUTE_SOURCE ! route-map ORIGINATE permit 10 set ip address prefix-list ORIGINATED_ROUTES set community 14616:555 additive ! ip prefix-list ROUTE permit 10. Verify that the prefix that is being injected is not outside of the scope of the aggregate prefix. BGP Conditional Route Injection should not be confused with the existing Conditional Advertisement feature.

2. removes any dependencies of MED-based path decisions. For this diagram R1 has a loopback (BGP ID of 1. ! ip prefix-list ORIGINATED_ROUTES permit 10. MED is an optional non-transitive attribute that is a four octet non-negative integer. For the topology above let's look at the routing BGP Best Practice Version 3. It ensures that MED comparison is made across all routes received from the same AS and the best path is chosen to reach the routers in that AS.1. The value of this attribute may be used by a BGP speaker's decision process to discriminate among multiple exit points to a neighboring autonomous system.1.2).0/25 ip prefix-list ORIGINATED_ROUTES permit 10.2. Benefits Using an example will illustrate the benefits of bgp deterministic-med.1).128/25 ! ip prefix-list ROUTE_SOURCE permit 10.1. R2 has a loopback (BGP ID of 2.1.2.1/32 BGP Deterministic-med Background As stated in RFC 1771.1. and so on. Enabling the bgp deterministic-med command ensures the comparison of the MED variable when choosing routes advertised by different peers in the same autonomous system. Figure 9 BGP Deterministic-med Example In some circumstances it is possible for a router to choose different paths as the "best" path depending on the order that the route advertisements were received in. Enabling bgp deterministic-med command.0 67 .1.1.

3. MED is not compared.4.3.3. valid.3.3.2 is better than the path from 3.2) Origin IGP.3 4.0.0/8 advertisement. version 4 Paths: (3 available. external 300 400 4.4) Origin IGP.4 from 4. localpref 100. Note that because the next hop AS is not the same for these two paths. internal 200 400 3.3 (3.0.3. metric 20.4 as best based on the following: .3.3.3) Origin IGP.4.4 neighbor so that the order the routes were received is changed: R1#clear ip bgp 4.2 from 2. localpref 100.3 is internal (Step 7).0.3.0. The path from 2.3 (3.0.0 68 .4. Now let's clear our 4.4.2.4 (4.2.3 because the route through 2.3. valid.4.4.4.4) Origin IGP. valid.4.2.4 300 400 4.4.3. metric 0.4. .4. R1# show ip bgp 70.3.4.4.2. table Default-IP-Routing-Table) Advertised to non peer-group peers: 2.4 (4.3. best R1# With the routes in this order R1 selects the route from 4. best #3.2 is external and the route through 3. The route from 4. localpref 100.0/8.3.2. Now the path from 4. metric 0.4 R1# show ip bgp 70.3.0 BGP routing table entry for 70. MED is compared in this case because the two paths share the same next hop AS. localpref 100.table from R1 for network 70.0.2.3 from 3.2.4. metric 30. internal 300 400 BGP Best Practice Version 3.4. localpref 100.4.3.3) Origin IGP. best #3. internal 300 400 2.4. metric 20.2. valid.0/8.0/8.2.0.4.2 will be compared.2 (2.4.0. table Default-IP-Routing-Table) Flag: 0x240 Advertised to non peer-group peers: 3.4.4. valid.3.0.4 will win because it has a lower MED value (Step 6).4 and 2.2.2.0.2.3 from 3.4.4. internal.4 from 4.0. Note that the MED values indicated on the diagram are only applied to the 70. version 5 Paths: (3 available.2 200 400 3.0.2.2.0 BGP routing table entry for 70.3.4.

3 (3. metric 0. metric 30. valid. order the entries within that sub-group based on MED. external.2.3.2. Let's look at the BGP table in R1 after configuring deterministic-med: R1# show ip bgp 70. .2.3 because 2.3 from 3. This inconsistency in selecting the best route can lead to routing loops in the network. best 300 400 4. Group the entries in the BGP table into sub-groups based on the next hop AS. internal 300 400 2.4.3. Let's take a look at why the 2.2 from 2. best #1.2) Origin IGP. localpref 100.4.2) BGP Best Practice Version 3.2.0.2.4.3.0 69 . The path from 2.2 route is now selected as best.2.4. . MED is not compared because they are from different AS.2.3.2.2. select the best route.2.2 200 400 3. valid.2. best R1# This is a problem because the 2.3. localpref 100.3. In order for R1 to consistently choose the same best path we need to configure deterministic-med as shown below: R1# ! router bgp 100 bgp deterministic-med ! Deterministic-med forces BGP to do the following: .4) Origin IGP.2 from 2.4.4 from 4.0.2. .3.2.3. metric 20.2 is better than the path from 3.0.0.4.4. 2. Compare the best route from each sub-group in order to select the best route.4 (4.3.2.2.3.2 is external and the route through 3. localpref 100.4 because 3.2 (2. table Default-IP-Routing-Table) Advertised to non peer-group peers: 2. The path from 3.3.2.2 (2. MED is not compared because they are from different AS.3) Origin IGP.3 is better than the path from 4.3.3 has a lower peer address (Step 13).0 BGP routing table entry for 70.2.3. version 4 Paths: (3 available.2 route was chosen as best: .2.3.2.0/8. . For each sub-group.2.3 is internal (Step 7).4. valid. For each sub-group.2.2. internal.

3 entry becomes the best path due to the following: .4.2.3 path to the 4.3.4. localpref 100.2 from 2.2.2. Guidelines BGP deterministic-med is disabled by default and it is recommended to ALWAYS enable BGP deterministic-med. metric 20.0.2.2.0.4.3. external R1# The path from 3. Configuration is shown below: router bgp 100 bgp deterministic-med BGP Best Practice Version 3. This path beats the 2. valid.4. best 300 400 4. The 3.3.4 has a lower MED (Step 6). The 3.3. table Default-IP-Routing-Table) Advertised to non peer-group peers: 2. Origin IGP. external R1# The 3.4 from 4.4.3.2.2.2.3.4.4) Origin IGP.3.3. The 4. Another requirement when running deterministic-med is that every BGP router in the AS must also have deterministic-med configured. internal.0. metric 30.0.2 200 400 3. version 4 Paths: (3 available.2.3. valid.4.3.3. internal 300 400 2. If we clear our 2.0 70 .4.2.3.3 path wins because it has a lower peer ID (Step 13).3 from 3. localpref 100.4. which would normally cause a change in what is considered the best path.3 is still the best path: R1#clear ip bgp 2. it does sometimes happen.3.2.3 path is the best for all routes with 200 as the first hop AS. .3. valid.0/8.2. localpref 100. metric 0. because the paths are ordered the same way each time.3.4. valid. localpref 100. we can see that 3.3 (3.3) Origin IGP.3.4 (4.2) Origin IGP.2. best #1. This should be enough proof to convince you and your customers to ALWAYS run deterministic-med.0 BGP routing table entry for 70.2 path because 4. . metric 30.2 peer.4.3 is now considered the best every time.2 R1#show ip bgp 70.4 path.3.3. Although the above situation is rare. Now we compare the 3.2 (2.2.4 path is the best for all routes with 300 as the first hop AS.4.

BGP ‘ORIGINATOR_ID’ and ‘CLUSTER_LIST’ path attributes are set to the BGP Router Identifier by default. . BGP Log Neighbor Changes Background BGP Log Neighbor Changes enables the logging of neighbor adjacency changes and/or resets.0 71 .) . Risks and Limitations Changing the BGP Router ID on an active BGP peer will force the session to be reset. . BGP synchronization requires that BGP and OSPF router identifiers match when redistributing routes between protocols. If the BGP router id is not manually set. BGP Best Practice Version 3.BGP Router Identifier Background The BGP Router Identifier is a 4-byte field in the BGP Open message used to identify a BGP speaker. When two equal cost BGP routes are considered by the BGP Best Path Algorithm one selection criteria is BGP Router Identifier. changing a routers IP address can cause a BGP router id change and reset the routers BGP peers. A router running only IPv6 requires a BGP Router Identifier be manually set. . Benefits Deterministically setting the BGP Router Identifier is beneficial when considering the following scenarios. (Route with lowest BGP RID is selected. Messages are written locally to the routers logging buffer or remotely to a Syslog server. Guidelines The BGP Router Identifier is set to the highest IP address assigned to a routers loopback interface. The ‘bgp router-id <ip address>’ command should be used to deterministically set the routers BGP Router ID to an IP address configured on a loopback interface. . If no loopback interface is configured the highest IP address configured on a physical interface is used. Benefits Logging changes in BGP neighbor state is useful for identifying and troubleshooting BGP peering issues.

0 72 . BGP Best Practice Version 3. Logging of neighbor adjacency state changes should be enabled using the ‘bgp log-neighbor-changes’ command.Guidelines Logging of BGP neighbor adjacency changes is disabled by default.