What's lurking in your network? Find out by decrypting SSL

Organizations have spent vast sums of money on security systems and, when deployed and operated correctly, they play a key role in safeguarding the organization. However, most systems have one critical dependency: The traffic flowing through must be readable. If the traffic is encrypted, many systems are almost completely useless, giving the system owner a false sense of security.

Exactly how much of a problem is this? A recent report published by Palo Alto Networks sheds some light. According to the company's Application and Usage Risk Report, 7th Edition, 36% bandwidth on corporate networks is encrypted. That's a 36 in 100 chance your network-based information security systems will miss the bad stuff. And in reality, the chance is greater than 36%, because the bad guys know where to hide the bad stuff so your tools can't see it. Furthermore, the percentage of traffic that is encrypted is increasing as more applications and websites use encrypt-by-default policies.

CLEAR CHOICE TEST: SonicWall stands tall in SSL decryption testing

MORE: SSL decryption may be needed for security reasons, but employees are likely to 'freak out'

So what can be done? Clearly, blocking all encrypted traffic at the enterprise edge is not feasible. The answer lies with a technological capability that allows us to peek inside the encrypted traffic: on-the-fly decryption. The remainder of this article is dedicated to explaining how this can be done. I won't be referring to any one vendor's implementation, but rather will attempt to stick to the basics and explain how the technology works.

Contrary to what you may be thinking, you do not need a team of mathematicians or NSA-grade supercomputers for the task. On the contrary, it's actually quite simple once you understand the basics.

Encryption 101

When you open the browser on your computer (or smartphone or tablet) and go to a secure website such as your bank, you notice the URL begins with HTTPS (notice the "S"). This indicates that all data being exchanged with the remote Web server is being encrypted by an encryption scheme called PKI (public key infrastructure). It works like this:

  • The Web server has a secret encryption key called a private key, which is just a long, seemingly random string of characters stored in a computer file. Only the Web server has access to the private key. It also has a public certificate (which is also just a computer file) that contains another encryption key, called the public key, that is different from the secret key.
  • The private key and the public key are mathematically related such that anything encrypted by the public key can only be decrypted by the private key. In other words, the encryption operation cannot be reversed using the public key. (Exactly how the mind-bending math works is beyond both the scope of this article and, frankly, my intelligence).

When your browser wants to communicate with an encrypted Web server, the following sequence of events occurs (depicted graphically in Figure 1 for those who like pictures).

Make sense? The key to understanding PKI encryption is the relationship between the public and private keys; the public key is used to encrypt, and the private key is used to decrypt. And since the only entity in the whole world that has access to the private key is the server, anything encrypted by the public key can only be decrypted by the Web server.

Now that we have a basic understanding of PKI, let's get back to the subject at hand. To decrypt traffic so your security tools can examine it we have to get in the middle of the session. How we do this depends on the function or type of traffic you are trying to decrypt. There are two categories:

Let's take a look at each of these and explore how to decrypt each.

Decrypting inbound traffic

Let's say your company has a website hosted on a Web server in your data center. The website uses SSL, so all traffic to and from the website is encrypted. When a client on the Internet accesses the site using a computer, smartphone or tablet, an end-to-end SSL-encrypted connection is established between the client's browser and the Web server, making the connection completely invisible to your organization's network security tools.

Decrypting this traffic to make it visible to your security tools requires two steps:

The first step is easy, but the second step can be accomplished in several ways (depicted in Figure 2):

Each of the above methods has its strengths and weaknesses, and which is used in a given architecture depends on many factors. However, they all share one key point: Unencrypted data never leaves the device. As a result, end-to-end data encryption is maintained.

Decrypting outbound traffic

With outbound traffic, the vast majority originates from employees browsing the Internet, checking their email, posting on Facebook, etc. This traffic is potentially damaging to your organization in numerous ways: Users may send proprietary company information over web-based email, post confidential data on social network sites, etc. Every user in your organization who accesses an encrypted website is a potential point of entry into your network, or point of exit for confidential information.

Decrypting outbound traffic is a little trickier than decrypting inbound traffic. As we just discussed, when decrypting inbound traffic we load the private key for the server onto the decryption device, giving it the ability to decrypt traffic to or from the server. But we can't load the private key for every single Web server on the Internet onto your security device, so another strategy is necessary.

The strategy for decrypting outbound traffic requires a somewhat more detailed understanding of PKI. Look back again at the Encryption 101 section above. I intentionally skipped an important detail right after Step 2 to keep it simple. What really happens after the server sends its certificate to the browser, before going any further, the browser decides whether or not it trusts the certificate. It makes this decision based on who signed the certificate. An entity that signs certificates is called a certificate authority, and computer browsers come pre-loaded with a list of trusted certificate authorities. Any website certificate signed by a certificate authority that the browser trusts will also be trusted. We can exploit this behavior to decrypt outbound traffic like this:

  • Make the decryption device a certificate authority, giving it the ability to sign certificates
  • Configure the users' browser to trust the new certificate authority
  • Place the decryption device inline between the users and the Internet

MOXIE MARLINSPIKE WEIGHS IN: The SSL certificate industry can and should be replaced

Do you see where we are going with this? When a user browses to an encrypted website, the encryption device intercepts the request, generates a new certificate on the fly pretending to be the Web server, signs it, and sends it to the user. And because the user's browser is configured to trust certificates signed by the decryption device certificate authority, it will have no idea that it had the wool pulled over its eyes and continue establishing the encrypted connection. The decryption device then establishes its own connection to the actual Web server and transparently proxies all requests between the user and the server.

Not all of the decryption methods described above are appropriate for every scenario. You'll have to analyze your architecture to determine which solution works for your environment. Many vendors produce decryption-capable systems and I recommend you take a look at the strengths and weaknesses of several before deciding which to deploy. Be sure you understand the limitations of each and test in a lab or pilot environment before a production deployment.

With the right tools in the right place, you can take a peek inside your traffic and see what's lurking inside.

Heder, CCIE No. 24788, is a network architect with NES Associates in Alexandria, Va., specializing in large-scale enterprise and data center network design. Heder holds a master's degree with a concentration in network architecture and design, and has a patent filed for an IPv6 technology. He can be reached at brian.heder@gmail.com.

Read more about wide area network in Network World's Wide Area Network section.

Tags Certificate Authoritiesinbound traffic decryptionoutbound traffic decryptionSSLencrypted trafficpkiLAN & WANpublic key infrastructureSSL decryptionnetwork securitysonicwallsecurity

Comments

Comments are now closed

CSO Corporate Partners
  • f5
  • Webroot
  • Trend Micro
  • NetIQ
rhs_login_lockGet exclusive access to CSO, invitation only events, reports & analysis.
CSO Directory

Enterprise Virtualisation Security

Deep Security provides a comprehensive Server Security Platform giving organisations advanced protection for Physical, Virtual, and Cloud Servers.

Security Awareness Tip
Security ABC Guides

Warning: Tips for secure mobile holiday shopping

I’m dating myself, but I remember when holiday shopping involved pouring through ads in the Sunday paper, placing actual phone calls from tethered land lines to research product stock and availability, and actually driving places to pick things up. Now, holiday shoppers can do all of that from a smartphone or tablet in a few seconds, but there are some security pitfalls to be aware of.