The Sorcerer's Apprentice: AI as an amplifier of human failings

The Sorcerer’s Apprentice was written by Goethe over 200 years ago and it was influenced by the Industrial Revolution. At the time, hand weavers were railing against the introduction of machines that would replace them. It’s against that backdrop that Cosive’s Jayne Naughton discussed machine learning on the final morning of AusCERT 2016.

Naughton’s rapid-fire tour of the world of AI and machine learning began by looking at neural networks where inputs are processed to deliver an output.

"What a machine learning algorithm does is try to find a line of best fit,” he says. By taking different variables, a computer system tries to work out all the possible outcomes and then determine the one that best answers the original question.

For example, if the temperature is a particular value and the wind is blowing at a particular speed, should I play golf?

Naughton says learning systems can either be supervised or unsupervised. Supervised systems place limits on how the system calculates its outcomes. And these limits can influence the outcomes. In other words, our inherent biases can significantly influence the answers these systems give.

The problem, says Naughton, is that we apply human models on machine-based outcomes. For example, we apply causality to some correlations. For example, Naughton showed a correlation between boat drownings and the marriage attrition rate in Kentucky. And we anthropomorphise systems, assigning them human characteristics, when hey don’t really apply.

“There’s no human thought process going on,” says Naughton. "You never know where machine learning is stacking the odds”.

Naughton noted that facial recognition systems are designed using sample sets of face to train the systems. As a result, the systems struggle to recognise some ethnicities because they are under-represented in the data used when programming the artificial intelligence.

What are we worried about?

Many people think "Terminator is the standout AI threat,” says Naughton.

But Naughton says the problem is we don’t need systems to become intelligent; they just need to rapidly try enough options so that they eventually make a correct decision.

And such systems can make some significant errors. For example, one system identified someone from news agency Al Jazeera as an operative for two opposing terrorist organisations.

In simple terms, artificial intelligence and machine learning are only as effective as their programming.

When AI goes wrong?

There are many applications from military drones, to self driving trucks and understanding language. The Microsoft AI bot, TayTweets, highlighted what can happen when language analysis AI gets it wrong. And this was despite TayTweets having human supervisors.

And Naughton says many of the AI initiatives coming from Silicon Valley are actually powered by lowly paid people who answer questions in a uniform way.

There’s also a push to use AI in healthcare, such as IBM’s Watson system being used in cancer detection. However, there are some risks as governments give private companies access to our personal health information.

Furthermore, Naughton says the use of AI to support infosec efforts should not be seen as a silver bullet.

“If you do machine learning over all your security data, you’re doomed,” he says.

In simple terms, the error rates mean adversaries will still be able to breach your systems.

AI could make things better

There are many positive applications of AI says Naughton from detecting rigged elections to assisting automated cropdusting.

Superintelligence is also a potential future, but Naughton says we often get the perspective wrong. Super intelligent systems will be used to understand the human brain and not the other way around. And AI systems are being used to evaluate other AI systems and then design other AI systems.

Where are we now?

"AI is an army of barely competent lab assistants,” says Naughton. AI systems are only as good as the data and algorithms they are built on.

Join the CSO newsletter!

Error: Please check your email address.

Tags AusCERT conferencecybersecurityAlMicrosoftprogrammingAusCERT2016microsoft al bot

More about Microsoft

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Anthony Caruana

Latest Videos

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

  • 150x50

    IDG Live Webinar:The right collaboration strategy will help your business take flight

    Speakers - Mike Harris, Engineering Services Manager, Jetstar - Christopher Johnson, IT Director APAC, 20th Century Fox - Brent Maxwell, Director of Information Systems, THE ICONIC - IDG MC/Moderator Anthony Caruana

    Play Video

More videos

Blog Posts