The website that was built for Google to destroy

'Unindexed,' an art-technology project, explored the permanence of the web

Matthew Rothenberg built a website that would erase itself when indexed by Google, an art-technology project exploring the permanence of the web.

Matthew Rothenberg built a website that would erase itself when indexed by Google, an art-technology project exploring the permanence of the web.

When Matthew Rothenberg created a new website in early February, he let about two dozen people know about it through an unlikely medium: postcards.

The unorthodox method was fitting for an unorthodox website called Unindexed. It was the latest project from Rothenberg, a 35-year-old based in Brooklyn, who has created a portfolio of interactive web installations and performance art projects around technology.

Unindexed is no more. The website was coded to erase itself once Google added it to its search index. It lasted a little over three weeks, disappearing forever on Feb. 24.

Rothenberg has done stints as head of product for Flickr and Bitly but for the last couple of years has focused on consulting and his art-technology side projects. His goal for Unindexed was to create a site where people could post comments safe in the knowledge that no record of those posts would ever exist again. It was also coded to prevent Google from caching it.

It was an experiment exploring the web's dichotomy: how it at times it can feel permanent, and at other times ephemeral.

"We've learned to be very careful and to mediate ourselves in the way we communicate online, and people don't really feel free," he said in a phone interview Monday. "I was curious, if I created a space where people could speak without feeling they were on a permanent record, what would happen."

In some ways, the web is still a fairly young medium, and people are still coming to grips with how permanent or ephemeral information they post there can be. Some things endure online for a very long time, while some services that seem stable can disappear.

In 2009, Yahoo shut down GeoCities, a large online community that let people create simple web pages and hosted other features such as bulletin boards and chat. It felt dated after being online for 15 years, but people reacted strongly when Yahoo scuttled it, and the Internet Archive launched a project to archive its content.

Rothenberg was unsure if Unindexed would live even for a day. If someone posted a link to it on a social network, that could bring it to Google's attention and the site would destroy itself.

There's certainly interest in applications that don't retain records. The popular Snapchat app allows people to send photos that disappear from recipients' devices. Wickr, a multimedia messaging app, lets messages self-destruct after a certain time.

Unindexed was even more fragile, and open on the web rather than a private messaging service. Its survival depended on a quality that seems rare these days: its users' self-restraint. The action sought after by so many app developers -- that people who discover a service share it with others -- could also kill it.

Those who received Rothenberg's postcards were warned that sharing information about the site online would lead to its demise. But he didn't advise them against it.

Rothenberg posted a short essay on the site outlining his thoughts. Writing doesn't come easy to him, he said, and he often agonizes about what he posts online. But in this case, "I didn't feel the same degree of pressure because I knew it wasn't permanent. It felt like somewhat of a safer space for me to write," he said.

The site had a few hundred visitors and was apparently shared discreetly. While the site stayed under Google's radar for more than three weeks, it was quickly visited by spam bots that posted random sentences.

Unindexed was "completely out there in the dark," a random domain name among millions promoted only modestly through post cards, "yet there are these bots out there that are searching for somewhere they can shout into," he said.

He's unsure how Google eventually latched onto the site, though it was only a matter of time. Unindexed was coded to constantly search for itself on Google to see if it had been spotted it. Its disappearance was something of a relief.

"I feel very much the kind of pressure of maintaining things forever," he said. "Every time I build a project and put it on the web, it's kind of expected that people are going to be able to see it in a year or five years. The reality is that software is very, very brittle. It breaks all the time. It needs constant attention. It's a very needy child.

"The more and more projects that I do, the more and more time I have to spend in the past just maintaining things. For me, there was something a little cathartic about knowing it's going to be gone."

Anyone wanting to repeat the experiment can find Unindexed's code at Github, and Rothenberg's explanation of the experiment, and other projects, on his website.

Send news tips and comments to Follow me on Twitter: @jeremy_kirk

Join the CSO newsletter!

Error: Please check your email address.

Tags securityinternetprivacy

More about GeoCitiesGoogleYahoo

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Jeremy Kirk

Latest Videos

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

  • 150x50

    IDG Live Webinar:The right collaboration strategy will help your business take flight

    Speakers - Mike Harris, Engineering Services Manager, Jetstar - Christopher Johnson, IT Director APAC, 20th Century Fox - Brent Maxwell, Director of Information Systems, THE ICONIC - IDG MC/Moderator Anthony Caruana

    Play Video

More videos

Blog Posts