Legal compliance challenges of Big Data: Seeing the forest for the trees

With the advent of big data, businesses today are managing databases of unprecedented size and complexity. With that size and complexity comes myriad legal and compliance challenges.

Foremost among them, the almost insurmountable task of complying with an alphabet soup of privacy and data security laws and regulations. In addition to local, state, national, and, even, international laws and regulations, there are many other potentially applicable standards and guidances.

[ The 5 worst Big Data privacy risks (and how to guard against them) ]

In the financial services and healthcare industries, there are many non-binding, but strongly recommended, guidances from a variety of regulators. There are also contractual standards, such as the Payment Card Industry Data Security Standard ("PCI DSS"), which governs cardholder information in credit card transaction. Finally, there are various industry standards for information security published by organizations like the Computer Emergency Response Team ("CERT") at Carnegie Mellon and the families of standards from the International Standards Organization ("ISO").

Reconciling all of these laws, regulations, standards, and guidances can be, at best, a full-time job and, at worst, the subject of fines, penalties, lawsuits, and, frequently, very adverse publicity and loss of business. In many instances, these obligations are vague and ambiguous, with little specific guidance as to compliance. Worse yet, the laws of different jurisdictions may be, and frequently are, conflicting. One state or country may require security measures that are entirely different from those of another state or country. Finally, the creation and use of the extremely large databases constituting "big data" is a relatively new phenomenon that has not yet been fully tested in the courts, particularly with regard to privacy and security issues.

The challenges of compliance with this ever increasing morass of laws, regulations, standards, and contractual obligations can be overwhelming. Even if no personally identifiable information is at risk, businesses have obligations to implement appropriate security measures to protect other highly sensitive information relating to, for example, their trade secrets, marketing efforts, business partner interactions, etc. All too often, businesses become fixated on a single tree or branch in the forest of laws, regulations, standards, and guidances and fail to appreciate, or even see, other nearby trees and their relationship and, certainly, seldom step back a sufficient distance to gain an overall view of the compliance forest.

We have sifted through various privacy and security laws, regulations, and standards to identify three common, relatively straightforward "threads" that run through many of them. By understanding these common threads, businesses can better understand their overall information security and compliance obligations with regard to big data. With this understanding, businesses may more readily address not only their current obligations, but have a framework for assessing new laws, regulations, and standards that may arise in the future.

Common misconceptions about information security compliance

There is much confusion and many misconceptions when it comes to information security and compliance with regard to big data. The two biggest misconceptions are that "it's all about the data" and "it's all about confidentiality." While data and confidentiality are certainly of critical importance, a more holistic approach is required. A business must be concerned about its data, but it must be equally concerned about the systems on which the data resides. In addition, confidentiality is only one of three key protections required for true security. Those three protections are frequently referred to by the well-known acronym "CIA," standing for Confidentiality, Integrity, and Availability. For data to be truly secure, each of these three elements must be satisfied.

"Confidentiality" is the most obvious of the three elements in CIA. It means the data is protected from unauthorized access and disclosure.

"Integrity" means the data can be relied upon as accurate and that it has not been subject to unauthorized alteration. Data integrity is likely the least obvious of the elements necessary for achieving good information security. Consider the importance of the integrity element in the context of a medical information system used in a hospital. If the data in a patient record cannot be relied upon (e.g., to identify a drug allergy, recent medical treatments, results of blood tests, etc.) because certain elements may have been altered, the entire database is rendered suspect.

Finally, "Availability" means data is available for access and use when required. It does no good to have data that is confidential and for which integrity is maintained if that data is not actually available when a user requires it. Consider, again, the example of the healthcare information system. If a patient record is unavailable because of a system failure when a patient comes into the emergency room in critical condition, it is useless. Hackers understand the substantial impact unavailability may have on a business, particularly online businesses. Denial-of-service attacks are frequent. In these attacks, hackers inundate a target business' services with fake requests in an effort to overwhelm them, preventing real users from accessing and using the systems.

The importance of CIA cannot be overstated. It is not just a well-worn concept in information security treatises. Lawmakers have directly incorporated that very language into certain information security and privacy laws and regulations. Businesses that fail to achieve CIA with regard to their data, may be found in violation of those laws.

A final misconception about information security and privacy laws is that they require perfection (i.e., any breach, regardless of how diligent the business has been, will create liability). This is not true. The laws and regulations in this area are directed at having businesses do what is reasonable and appropriate. If the business achieves that standard and a breach nonetheless occurs, it will generally not have a compliance problem. Liability will turn on whether the business has thoughtfully attempted to address the security of its data.

Finding common threads in compliance laws and regulations

The sheer number and variety of laws, regulations, and other standards governing the handling of sensitive information can be daunting, if not overwhelming. The problem escalates exponential when extremely large databases are involved -- databases that may contain data from individuals residing in dozens of jurisdiction around the world. In some instances, it may be almost impossible for even a large, sophisticated organization to identify all applicable requirements, reconcile inconsistencies, and then implement a compliance program.

In this section, the goal is not to discuss any specific laws, regulations, or standards, but to identify three common threads that run through many of them. By understanding those common threads, businesses can more easily understand their baseline compliance obligations and obtain at least a glimpse of the compliance forest.

As mentioned, there are three common threads to consider. These threads run not only through laws and regulations, but also contractual standards such as the PCI DSS and, even, common industry standards for information security published by organizations like CERT at Carnegie Mellon and the families of standards furnished by ISO. Embracing these common threads in designing and implementing an overall approach to information security will greatly increase a business' ability to achieve overall compliance with the laws, regulations, and other requirements applicable to it.

Confidentiality, Integrity, and Availability ("CIA"). As discussed, the well-established, foundational concept of CIA found in every handbook on information security has now been codified into many laws and regulations. The three prongs of this concept address the most fundamental goals of information security: the data/information must be maintained in confidence, it must be protected against unauthorized modification, and it must be available for use when needed. The lack of any of the foregoing protections, would materially impact compliance and the value of the information.

Acting "Reasonably" or taking "Appropriate" or "Necessary" measures. The concept of acting "reasonably" is used in many state and federal laws in the United States, Australia, and many other countries. The related concept of acting so as to take "appropriate'' or ''necessary'' measures is used in the European Union and many other areas. Together, they form the heart of almost every information security and data privacy law. A business must act reasonably or do what is necessary or appropriate to protect its data. Note that this does not require perfection. Rather, as discussed below, the business must take into account the risk presented and do what is reasonable or necessary to mitigate that risk. If a breach, nonetheless, occurs, provided the business has established this basic requirement, it will not be generally found in violation of the applicable law or regulation.

Scaling security measures to reflect the nature of the data and threat. A concept that is closely related to acting reasonably or doing what is appropriate is the idea of scaling security measures to reflect the nature of the threat and sensitivity of the data. That is, a business need not spend its entire security budget to address a low risk threat. But, if the risk is substantial, particularly in light of the volume and/or sensitivity of the data, the level of effort and expenditure by the business to address that risk must increase. A database with only names and physical addresses may not require as much security as a database of names, addresses and Social Security numbers. To better understand this concept, the following are excerpts from two laws that incorporate and define the concept of ''scaling'':

First example: The Massachusetts Data Security Law: ". . . safeguards that are appropriate to (a) the size, scope and type of business of the person obligated to safeguard the personal information under such comprehensive information security program; (b) the amount of resources available to such person; (c) the amount of stored data; and (d) the need for security and confidentiality of both consumer and employee information."

Second example: HIPAA Security Rule: Factors to consider:

(i) The size, complexity, and capabilities of the Covered Entity.

(ii) The Covered Entity's technical infrastructure, hardware, and software security capabilities.

(ii) The costs of security measures.

(iv) The probability and criticality of potential risks to ePHI.


While the number and complexity of privacy and information security related laws, regulations, and other standards is ever increasing, businesses should look for and appreciate common threads running through them. In this article, three of the most common and most important threads are presented. By understanding current law does not require perfection, but only due care, reasonableness, and scaling measures to reflect the sensitivity of the data being placed at risk, businesses can go a long way to achieving compliance. This same framework can be used to understand and assess laws, regulations, and standards implemented in the future.

Michael R. Overly is a partner in the Technology Transactions & Outsourcing and Privacy, Security & Information Management practices in the Los Angeles office of Foley & Lardner LLP. He counsels clients on matters including technology licensing and information security and has written numerous articles and books on these subjects, including the recently released book, "Big Data: A Business and Legal Guide." He can be reached at

Join the CSO newsletter!

Error: Please check your email address.

Tags Big Data Securitysecuritycarnegie mellondata protection

More about Computer Emergency Response TeamInternational Standards OrganizationISOMellonTechnologyTechnology Transactions

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Michael R. Overly

Latest Videos

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

  • 150x50

    IDG Live Webinar:The right collaboration strategy will help your business take flight

    Speakers - Mike Harris, Engineering Services Manager, Jetstar - Christopher Johnson, IT Director APAC, 20th Century Fox - Brent Maxwell, Director of Information Systems, THE ICONIC - IDG MC/Moderator Anthony Caruana

    Play Video

More videos

Blog Posts