Features

Sonic Cyborgs

By Max Kutner '11

During the spring semester, four artists and lecturers visited Vassar to explore breaking down the boundaries between humans and machines—and art and science

Neil Harbisson is likely the first person ever to listen to the stained glass window in the Vassar library. He hears it the moment he enters. “Very noisy,” he says, approaching the window as students study nearby. “Not too suitable for a library,” he jokes. Moving his head in a circular motion, as if watching laundry tumble in a dryer, he recites its colors as an antenna, implanted in his skull, hangs forward like a reading light, sensing them. “Very musical,” he says.

The press has called him “the world’s first cyborg.” Born with the visual disorder achromatopsia, he cannot see color—only shades of gray. In 2003, while studying music at England’s Dartington College of Arts, he and a cybernetics expert began creating a wearable device that could detect colors and convert their light frequencies to sound tones. He named the device the Eyeborg. The prototype transmitted the sound to headphones. He later convinced a surgeon to implant the antenna so the sounds would vibrate inside his head.

Neil Harbisson
Neil Harbisson

"I felt like there was a lot of information that I was missing becasue color is used as a code and to describe everyday things: I was reminded constantly that I was missing this layer of life. In some cases it made me feel socially exclused because I didn't understand much of the things that people were saying. If someone asked me if I had seen a man with blue eyes and brown hair, I wouldn't know. It made me feel disconnected from everyone else."

“I don’t feel that I’m wearing technology. I feel that I am technology,” Harbisson, who grew up in Catalonia, Spain, has said. He considers the antenna a body part and refers to it as an “organ.” Through sound, he can now perceive all 360 degrees of the color wheel, each corresponding to a microtone on the musical scale, plus ultraviolet and infrared rays. Harbisson, an artist, uses the Eyeborg to create paintings, musical performances, and other projects by turning colors into melodies or sounds into colors. People have streamed his TED Talk more than two million times.

Michael Chorost
Michael Chorost

"When [the cochlear implant] was first activated, it was one of the most shocking days of my life. I was expecting to hear badly; I was not expecting for everything to be incomprehensible. I learned how to hear all over again by starting with the Winnie the Pooh books on tape. So, I would read the book in my hands while listening to Charles Kuralt read the book aloud on tape. That allowed me to match up what I was getting from the implant with what I knew I was reading. I learned this is what an 's' sounds like now. This is what a 'th' sounds like now."

“People are starting to see that you don’t need a medical reason to become a biological cyborg,” he says, meaning someone whose body functions with the help of advanced technology. In a few years, he predicts, “we’ll see a boom in new organs and new senses.”

Harbisson is one of four “sonic cyborgs” that Assistant Professor of Physics and Science, Technology, and Society Jose Perillan and Adjunct Assistant Professor of Music Justin Patch invited to campus spring semester as part of a Vassar Creative Arts Across Disciplines (CAAD) residency titled “Future Sounds: The Emergence of Cyborgs.” They each lectured and met with students.

Other guests were: Michael Chorost, a technology theorist who has written about his cochlear implants; Trevor Pinch, a Cornell professor and expert on how technology impacts society; and Marco Donnarumma, a performance artist who created a wearable biosensor that captures the sounds of muscle movements. (He performed with it in the Vassar Chapel.)

Researchers coined the term cyborg—short for cybernetic organism—in 1960. Citing their work that year, the New York Times described the cyborg as “a man-machine system in which the control mechanisms of the human portion are modified externally by drugs or regulatory devices so that the being can live in an environment different from the normal one,” such as outer space.

Today’s humans are glued to technology in ways those researchers likely never dreamed. Wearable tech, hacking, and government surveillance are in the news. Cyborgs exist, though perhaps not in the way most people think (The Terminator) or those researchers anticipated (spacemen).

Trevor Pinch
Trevor Pinch

"I don't have a good explanation for the fact that it's the margin of society which produces innovations. Maybe because of the nature of capitalism, cultural production always needs something new and by definition new stuff comes from the margin...I actually love poeple who are designing at the margins. Circuit bending is making and adapting a standard piece of musical technology, a sort of opening up the back of a technology, and just adding something to make strange sounds, which the designers had not intended. There is a circuit bending scene, and one festival in New York, and I think this is really interesting, because it is something similar to the hacking culture—it involves people on the edge, who don't have very much money. It's quite possible that stuff on the margins, like that produced by circuit bending, will be the most interesting things to look at in the future."

“People have a very science-fictional view of bionics,” says Chorost. In movies, he says, doctors outfit someone with a bionic body part “and 10 minutes later they’re ready to go with superpowers.” In real life, however, “this does mean cutting into the body, it means blood, it means sawing into bone, it means scars, it means pain.”

Chorost wore hearing aids most of his life until he went deaf in 2001 at age 36. He decided to get cochlear implants. The technology, first developed in the 1970s, involves a headpiece containing a microphone that goes on the outside of the head and sticks magnetically to computer chips implanted inside the skull. The headpiece converts sound to electrical data and transmits the data through the skin to the chips. Then the chips deliver the information through a wire to the cochlea, part of the inner ear, which processes the information and relays it to the brain. There are some 324,000 registered cochlear implants worldwide.

The implant makes Chorost a cyborg, he has said, because the technology is fused with his body, and not only provides information, but also exerts some control. His hearing remains limited compared to most people’s, but he considers the technology miraculous. He can even stream music to his brain from his iPhone through Bluetooth. “It still blows me away,” he says.

Harbisson thinks more people should use cyborg technology, regardless of medical necessity. In 2010 he cofounded the Cyborg Foundation “to promote cyborgism” and “defend cyborg rights,” he says. He also cofounded Cyborg Nest, a company that develops “organs and senses.” This year it plans to release an implant that senses which direction is north.

Marco Donnatumma
Marco Donnatumma

"The body vexed. I take a very thin body, some ropes, two concerte blocks of about 50kg. Make that body pull the blocks for twenty minutes in a circle. Use wearable microphone sensors to capture the sound of blood flow pulsations, muscle vibrations and bones crackling. Diffuse those visceral sounds through 8 subwoofers and 8 loudspeakers. Make a computer record all those sounds and play them back on top of each other. By the time the action is ended the vexed body has exploded in a wall of acoustic vibrations that breach into the real world in the form of diffused sound."

“We are all cyborgs,” says Donnarumma, the performance artist. “I don’t think there is a need to have something specifically embedded in your body to be a cyborg.”

Starting in 2010, while studying sound design at the University of Edinburgh, Donnarumma developed a sensor called the Xth Sense. Using a special microphone, the sensor picks up the low-frequency sounds of the human body and sends them to a computer. Donnarumma amplifies and manipulates the sounds during performances; others have used his technology for game design, virtual reality, and animation. “It’s like connecting a guitar to an effect pedal; in this case you connect your body to a computer,” he likes to say. The Georgia Tech Center for Music Technology once named his sensor the “world’s most innovative new musical instrument.”

Pinch, the Cornell professor who spoke at Vassar about the history of the Moog synthesizer, says he’s pleased to see sound technology breaking new ground. “Sound is, I think, the most important sense,” he says. “You’re in the middle of it, you’re surrounded by sound.” To witness technology shaping art, and vice versa, he says, is “incredibly inspiring.”

Hearing these “cyborgs” speak could be like “being in that dorm room with the Google guys at Stanford,” says Perillan, “seeing something that’s on the horizon that has not yet hit in a wide way, but will.”

“There’s something very powerful about realizing that it’s ordinary people that make the future,” adds Patch. “You realize how human it is.” Well, mostly human.

Max Kutner '11 is a staff writer at Newsweek, where he covers higher education, museums, and other topics.