​Data 61 finds way to collaborate on big data analysis while maintaining privacy

Big data analytics has been on top of everyone’s buzzword bingo cards for some time now. And while many organisations have their analytics house in order internally, solving the challenge of combining their data with that of other parties, while maintaining privacy has remained elusive.

Data 61 is part of the CSIRO, with a specific focus on digital technology. Stephen Hardy is the co-founder of Data61’s N1Analytics platform. This system uses homomorphic encryption which allows data to be analysed and manipulated without the need to decrypt it.

Hardy and his team initially started looking at the analysis of personal genome data. This data is a highly sensitive and can be used to determine an individual’s current and future health as well as revealing other personal information.

“The challenge was how could you analyse that data without disclosing the person;’s genome and still get interesting medical insights from it”.

But they saw wider applications as their platform can be used with any large dataset when the project started three years ago. After speaking with a number of businesses they found this was a common problem – how do you analyse data you can’t actually see?

The technology at the heart of solving this problem is homomorphic encryption. This is employed in mathematics to carry out calculations on encrypted numbers. The concept is not new – it’s been used for about 40 years says Hardy. But making it scalable and usable for business is a new field.

“We’re trying to build a practical, well engineered solution that lets you solve the problem of two organisations doping analysis on a problem jointly without putting it in the one spot or revealing the data to each other”.

Traditionally, two or more datasets would be joined into one table with everyone having access to the data. This new approach maintains the privacy of data as everything remains encrypted.

As the amount of data that is brought together increases, says Hardy, the more likely it becomes that a single individual can be identified separately from the data. Even if you remove identifiers such as names and addresses, once enough attributes are collected it becomes possible to isolate individuals.

Hardy has already been able to show this new technology to companies who have used it on their own internal data. The next step is to expand its application to external datasets – something that he says is being worked on.

A significant element of the application’s next phase is getting regulatory and legal clarity on how this works against the Australian Privacy Principles and other applicable laws. Hardy says this is expected to happen but that the regulatory side of things isn’t moving as quickly as the technology – something we’ve seen in many areas of technology. However, the N1Analytics platform is designed specifically to deal with the data privacy of individuals so he expects these issues to be overcome.

“We think that what we do is well supported by the Australian Privacy Principles and companies can use it safely and securely from an information point of view and data privacy point of view. It removes the risk of re-identification because the data is encrypted and fully de-identified in that way”.

Although N1Analytics started with applications in health, Hardy says there is broader application in finance and telecommunications, where datasets from different sources can be brought together for analysis without compromising the privacy of individuals.

For example, companies could potentially perform real-time credit analysis, without looking at actual personally identifiable data from financial institutions. For example, when a customer looks to enter a new contract for a mobile phone, the telco wouldn't see the financial data and the bank wouldn't see the telco data.

As well as the benefits for companies trying to keep data they see as corporately valuable private, there are benefits for private citizens. In the past, multi-organisational analysis required data to be centralised. This created a risk as a single breach could bring threats actors a rich reward. But by keeping data fragmented in encrypted repositories during analysis, personal data is better protected.

Hardy and the N1Analytics team from Data61 are still in the early stages of commercialising their platform. But they are working with companies to get the technology right and with the OAIC and other agencies to navigate the regulatory and legal challenges.

Join the CSO newsletter!

Error: Please check your email address.

Tags N1Analytics platformpersonal information.health carebig data analyticshomomorphic encryption​Data 61privacyCSO Australia

More about CSIRO

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Anthony Caruana

Latest Videos

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

  • 150x50

    IDG Live Webinar:The right collaboration strategy will help your business take flight

    Speakers - Mike Harris, Engineering Services Manager, Jetstar - Christopher Johnson, IT Director APAC, 20th Century Fox - Brent Maxwell, Director of Information Systems, THE ICONIC - IDG MC/Moderator Anthony Caruana

    Play Video

More videos

Blog Posts

Market Place