CyberWarfare – Defending the Electronic Frontier in the 21st Century and Beyond

During the reign of the Roman Empire, it was said that all roads led to Rome.  While these roads facilitated free-trade and were essential to the expansive growth of the Roman Empire, they also introduced a double-edged sword by creating convenient new avenues that could easily be maneuvered by the Empire’s enemies against its best interests.

It could be said that similar corollaries and conclusions could be drawn to that of the Internet, a project initially developed by the United States through the Advanced Research Projects Agency (ARPA or DARPA) in the late 20th century.  As we continue to move into the 21st century, the Internet is emerging as the new battlefield on the International stage.  As Internet connectivity becomes commoditized and the barrier to entry is further reduced, it becomes easier for potential adversaries of the United States to wield the power of the Internet to launch attacks against the US infrastructure and its National interests, disrupting the flow of information and leaving destruction in its wake.  It is becoming increasingly possible for our adversaries to not only cause extraordinary economic havoc, but also loss of life as critical services such as E911 and other emergency services become more dependent upon the Internet.  At the same time a perfect storm is brewing because the resources currently allocated to preparing ourselves for this advancing threat are stretched to the limit and largely focused on obsolete technologies that are considerably out of date.

On August 10th, 2010, in McLean, VA, a series of panelists will discuss this ever-increasing reality and the potential that wars in the future won’t be fought so much on the traditional battlefield but rather electronically targeting critical components of a national infrastructure.  The panelists will discuss what has been done by the US Government to secure certain core components of our national infrastructure, what remains to be done, and will also serve as a “call to arms” to better secure our national cyber landscape.  The members of this panel have all been intimately involved within the Networking Security industry for many years and are engaged in the implementation of their solutions at the ground level.  Their insights will provide invaluable viewpoints regarding this very real and emerging threat and will provide invaluable experience to the attendees of the presentation.

For more information, please take a look at the Event details at http://cyberwarfare.eventbrite.com/.  This event will be hosted by the Capital Technology Management Hub and will be moderated by Stefan Fouant.

What’s the BFD with BFD?

Many networks today are striving for “five nines” high availability and beyond. What this means is that network operators must configure the network to detect and respond to network failures as quickly as possible, preferably on the order of milliseconds. This is in contrast to the failure detection inherent in most routing protocols, which is typically on the order of several seconds or more. For example, the default hold-time for BGP in JUNOS is 90 seconds, which means that in certain scenarios BGP will have to wait for upwards of 90 seconds before a failure is detected, during which time a large percentage of traffic may be blackholed. It is only after the failure is detected that BGP can reconverge on a new best path.

Another example is OSPF which has a default dead interval of 40 seconds, or IS-IS which has a default hold-time of 9 seconds (for DIS routers), and 27 seconds (for non-DIS routers). For many environments which support mission-critical data, or those supporting Voice/Video or any real-time applications, any type of failure which isn’t detected in the sub-millisecond range is too long.

While it is possible to lower timers in OSPF or IS-IS to such an extent that a failure between two neighbors can be detected rather quickly (~1 second), it comes at a cost of increased protocol state and considerable burden on the Routing Engine’s CPU.  As an example, let us consider the situation in which a router has several hundred neighbors. Maintaining subsecond Hello messages for all of these neighbors will dramatically increase the amount of work that the Routing Engine must perform. Therefore, it is a widely accepted view that a reduction in IGP timers is not the overall best solution to solve the problem of fast failure detection.

Another reason that adjusting protocol timers is not the best solution is that there are many protocols which don’t support a reduction of timers to such an extent that fast failure detection can be realized. For example, the minimum BGP holdtime is 20 seconds, which means that the best an operator can hope for is a bare minimum of 20 seconds for failure detection.

Notwithstanding, this does nothing about situations in which there is no protocol at all, for example, Ethernet environments in which two nodes are connected via a switch as can be seen in the figure below.  In this type of environment, R1 has no idea that R2 is not reachable, since R1’s local Ethernet segment connected to the switch remains up.  Therefore, R1 can’t rely on an ‘Interface Down’ event to trigger reconvergence on a new path and instead must wait for higher layer protocol timers to age out before determining that the neighbor is not reachable.  (Note to the astute reader: Yes, Ethernet OAM is certainly one way to deal with this situation, but that is a discussion which is beyond the scope of this article).

L2-Connectivity

Essentially, at the root of the problem is either a lack of suitable protocols for fast failure detection of lower layers, or worse, no protocol at all.  The solution to this was the development of Bidirectional Forwarding Detection, or BFD, developed jointly by Cisco and Juniper.  It has been widely deployed and is continuing to gain widespread acceptance, with more and more protocols being adapted to use BFD for fast failure detection. 

So what is the Big Freaking Deal with Bidirectional Forwarding Detection anyway and why are so many operators implementing it in their networks?  BFD is a simplistic hello protocol with the express purpose of rapidly detecting failures at lower layers.  The developers wanted to create a low overhead mechanism for exchanging hellos between two neighbors without all the nonessential bits which are typical in an IGP hello or BGP Keepalives.  Furthermore, the method developed had to be able to quickly detect faults in the Bidirectional path between two neighbors in the forwarding plane.  Originally, BFD was developed to provide a simple mechanism to be used on Ethernet links, as in the example above, prior to the development of Ethernet OAM capabilities.  Hence, BFD was developed with this express purpose in mind with the intent of providing fault identification in an end-to-end path between two neighbors.

Once BFD was developed, the protocol designers quickly found that it could be used for numerous applications beyond simply Ethernet.  In fact, one of the main benefits of BFD is that it provides a common method to provide for failure detection for a large number of protocols, allowing a singular, centralized method which can be reused.  In other words, let routing protocols do what they do best – exchange routing information and recalculate routing tables as necessary, but not perform identification of faults at lower layers.  An offshoot of this is that it allows network operators to actually configure higher protocol timer values for their IGPs, further reducing the burden placed on the Routing Engine.

BFD timers can be tuned such that failure detection can be realized in just a few milliseconds, allowing for failure and reconvergence to take place in similar timeframes to that of SONET Automatic Protection Switching.  A word of caution – while BFD can dramatically decrease the time it takes to detect a failure, operators should be careful when setting the intervals too low.  Very aggressive BFD timers could cause a link to be declared down even when there is only a slight variance in the link quality, which could cause flapping and other disastrous behavior to ensue.  The best current practice with regards to BFD timers is to set a transmit and receive interval of 300ms and a multiplier of 3, which equates to 900ms for failure detection.  This is generally considered fine for most environments, and only the most stringent of environments should need to set their timers more aggressive than this.

One question that is commonly asked is how is it that BFD can send hello packets in the millisecond range without becoming a burden on the router.  The answer to this question lies in the fact that BFD was intended to be lightweight and run in the forwarding plane, as opposed to the control plane (as is the case with routing protocols).  It is true that while early implementations of BFD ran on the control plane, most of the newer implementations run in the forwarding plane, taking advantage of the dedicated processors built into the forwarding plane and alleviating the burden which would otherwise be place on the RE.  In JUNOS releases prior to JUNOS 9.4, BFD Hello packets were generated via RPD running on the RE.  In order to enable BFD to operate in the PFE in JUNOS versions prior to JUNOS 9.4, the Periodic Packet Management Daemon (PPMD) had to be enabled, using the command ‘set routing-options ppm delegate processing’.  In JUNOS 9.4 and higher this is the default behavior and BFD Hello packets are automatically handled by PPMD operating within the PFE.

Facilitating Firewall Filter Configuration in JUNOS using ‘apply-path’

Undoubtedly, one of the coolest features in JUNOS is the apply-path statement. Using apply-path, an operator can configure a prefix-list which comprises IP prefixes linked to a defined path within JUNOS. This facilitates tasks like configuring firewall filters to allow traffic from configured BGP neighbors, making them highly dynamic.

Continue reading “Facilitating Firewall Filter Configuration in JUNOS using ‘apply-path’”

Preparation Tips for the JNCIP-M/T and JNCIE-M/T Exams

So it’s been a while since I’ve posted, and I’m just now getting to a point where I can start writing a bit more, not to mention feeling quite a bit more well rested after a grueling month of December. I was quite busy during the last month as I was doing last minute preparations for my JNCIE-M/T exam scheduled for Dec. 17th, not to mention trying to wrap up several projects before years end. I’m delighted to say that I passed the exam and am now the proud recipient of the highly sought-after JNCIE-M/T designation!  I’d like to take a few moments to share about my experiences with the rest of you who may also decide to pursue this certification.

Continue reading “Preparation Tips for the JNCIP-M/T and JNCIE-M/T Exams”

Implementing Provider-Provisioned VPNs using Route Reflectors

MPLS/BGP Provider-Provisioned VPNs, such as those proposed in RFC 4364 (formerly RFC 2547) or draft-kompella variants, suffer from some scalability issues due to the fact that all PE routers are required to have a full iBGP mesh in order to exchange VPN-IPv4 NLRI and associated VPN label information.  In a modern network consisting of a large number of PE devices, it becomes readily apparent that this requirement can quickly become unmanageable.

The formula to compute the number of sessions for an iBGP full mesh is n * (n-1)/2.  10 PE devices would only require a total of 45 iBGP sessions (10 * (9)/2 = 45).  However, by simply adding 5 additional PEs into this environment your total number of sessions increases exponentially to 105.  Scalability issues arise because maintaining this number of iBGP sessions on each PE is an operational nightmare; similarly control plane resources are quickly exhausted.

An alternative to this that has gained widespread adoption is to utilize Route Reflectors to reflect the VPN-IPv4 NLRI and associated VPN label between PE devices.  However, several issues arise when using Route Reflectors in such an environment.  In a normal environment without the use of Route Reflectors, MPLS tunnels exist between each PE router such that when the VPN-IPv4 NLRI and associated VPN label are received, a PE router can recurse through its routing table to find the underlying MPLS tunnel used to reach the remote BGP next-hop within the VPN-IPv4 NLRI.  In the Route Reflection model, the Route Reflector typically doesn’t have an MPLS tunnel to each PE for which it is receiving VPN-IPv4 NLRI.  Therefore, these routes never become active and are therefore not candidates for reflection back to other client and non-client peers.

A few methods have been developed which circumvent this issue.  One method is to simply define MPLS tunnels from the Route Reflector to each PE.  This solves the problem by allowing the Route Reflector to find a recursive match (i.e. MPLS tunnel) in order to reach the remote PE.  However, this approach suffers from the drawback in that it requires a whole bunch of MPLS tunnels to be configured which only serve to allow the received VPN-IPv4 NLRI to be considered active.  Remember, these tunnels are completely useless in that they will never be used for the actual forwarding of data, they are only used within the control plane to instantiate routes.

An alternative and much more graceful solution to this problem is to configure the Route Reflector with a static discard route within the routing table which is used to reference BGP next-hops in MPLS environments (inet.3 for example in JUNOS).  This static discard route only serves to function as a recursive match when incoming VPN-IPv4 NLRI are received for the express purpose of making these routes active and therefore candidates for redistribution.  In JUNOS, one can accomplish this using the following configuration:

routing-options {
    rib inet.3 {
        static {
            route 0.0.0.0/0 discard;
        }
    }
}

With the above, any VPN-IPv4 NLRI received from a PE router is immediately made active due to the fact that a static route has been created in inet.3 which is the routing table used in JUNOS to recurse for BGP next-hops in MPLS environments.

An excellent whitepaper entitled “BGP Route Reflection in Layer 3 VPN Networks” expands upon this and describes the benefits of using Route Reflection in such environments. It also builds the case for using a distributed Route Reflection design to further enhance scalability and redundancy.

One thing to keep in mind is that with the Route Reflector approach, we have merely moved the problem set from that of the PE device to that of the Route Reflector.  Although it minimizes the number of iBGP sessions required on PE devices, the Route Reflector must be capable of supporting a large number of iBGP sessions and in addition, must be able to store all of the VPN-IPv4 NLRI for all of the VPNs for which it is servicing.  It is highly recommended that adequate amounts of memory are in place on the Route Reflector in order to store this large amount of routing information.

Finally, while using Route Reflectors is an acceptable solution in the interim to addressing scaling concerns with Provider-Provisioned VPNs, it is not clear if this approach is sufficient for the long term.  There are several other options being examined, with some of them outlined in a presentation entitled 2547 L3VPN Control Plane Scaling” given at APRICOT in Kyoto, Japan in 2005.

Book Review :: IS-IS: Deployment in IP Networks

IS-IS_DeploymentIS-IS: Deployment in IP Networks
by Russ White, Alvaro Retana
Hardcover: 320 pages
Publisher: Pearson Education
ISBN-13: 978-0201657722

2starsBetter off choosing an alternative selection

As IS-IS is one of the more esoteric protocols, understood only by a few people in large scale ISP environments, I thought this book would be a welcome addition to my library as there isn’t much else on the market covering this protocol. There are of course ISO 10589 and RFC 1195 which covers these protocols, but seeing as this is a short book I thought it might be able to shed some light on an otherwise complex protocol.

In reviewing this book I’ve come up disappointed in general. There are certainly a few golden nuggets and I give the book a couple of stars just for attempting to bridge the gap between the purely theoretical and the purely vendor specific. However, the book comes up short on most other points. Often times I found myself wanting to scrap this book in favor of some of the other selections on the market, but since I have respect for these authors I read the whole book hoping that they might be able to redeem themselves by the time I finished.

Obviously the authors have a great deal of knowledge about the subject, and I don’t fault them entirely. The quality of the editing is poor with many grammatical and syntactical errors littered throughout the text. There are abundant instances throughout the book where the diagrams used do not match the text describing them. I was rather disappointed because I usually find that Addison-Wesley publishes some of the best texts on the market.

All in all, I thought this book could have been a lot better than it was. After all, these authors have several other titles under their belt, most notably “Advanced IP Network Design”. But in this case, I would say that you are better off looking for other similar titles available on the market, such as Jeff Doyle’s “Routing TCP/IP Volume 1” or “The Complete IS-IS Routing Protocol” by Hannes Gredler and Walter Goralski.

Book Review :: MPLS-Enabled Applications: Emerging Developments and New Technologies

MPLS Enabled Applications

MPLS-Enabled Applications: Emerging Developments and New Technologies
by Ina Minei, Julian Lucek
Paperback: 526 pages
Publisher: Wiley
ISBN-13: 978-0470986448

5starsExcellent coverage of VPLS, and Multicast over Layer 3 VPNs

Recently I had to work on a project which involved demonstrating Multicast over Layer 3 VPN interoperability between Cisco and Juniper. I spent several days reading through all the RFCs and working-group drafts which pertained to this subject matter, after which I still had many unanswered questions. In order to round out my understanding, I decided to order the Second Edition of ‘MPLS-Enabled Applications’. Looking back, I wish I had read this book instead of wasting my time reading the various RFCs and working-group drafts. This book answered all of my questions and went above and beyond to give me a solid understanding of the concepts and their application. As other reviewers have pointed out, often one needs to read a book to understand the technology basics, and then refer to RFCs or working-group drafts in order to keep abreast of the latest changes. Not so with this book… In fact, this book is so current that reading the working-group drafts is largely unnecessary. It is incredibly comprehensive, concise, and gives the reader a thorough understanding of the business drivers. Furthermore, it illustrates the various ways in which MPLS services can be offered and outlines the pros and cons of each approach so that the network designer can make intelligent decisions with regards to implementation.

In addition to the great coverage that was provided by the First Edition, the Second Edition has updated the text to reflect newer trends and applications such as the transport of IPv6 over an IPv4 MPLS core, and detailed coverage of end-to-end and local protection schemes in MPLS networks. Likewise, the chapter previously called “Point-to-Multipoint LSPs” has now been renamed to “MPLS Multicast”, with much more detailed coverage of the P2MP hierarchy and the forwarding-plane and control-plane operation. The biggest value for me was the addition of a completely new chapter on “Multicast over Layer 3 VPNs” which provided comprehensive coverage of this emerging technology and fully illustrates the full gamut of operation of either the PIM/GRE approach, or the NG-VPN approach utilizing BGP and P2MP LSPs. Finally, the addition of a chapter on “MPLS in Access Networks” was well deserved seeing as Ethernet is quickly becoming the access technology of choice and MPLS will likely be utilized as an overlay in order to realize the full potential of Ethernet in these environments.

This book has earned a spot on my bookshelf as one of my most coveted resources, and I refer to it quite often to refresh my memory on the myriad workings of various functions within MPLS. I wish I could give this book a rating higher than five stars! I can’t overemphasize how exceptional this book is. If you are in the market for a book covering MPLS and emerging applications offered on MPLS networks, this single book should be at the top of your list!