Smell, Taste and Hear the Data

Some of our contemporary computational and graphics tools make data visualization an exciting, rapidly advancing field. Sometimes scientists presenting data are not visual thinkers and may not be communicating their results transparently. Sometimes graphic designers put aesthetics ahead of the essence of what we could be communicating with a given dataset. But, when we get it right there is some real innovation occurring. Today I want to show you some 'visualizations' which are not in fact visual at all; I want to show some playful and intriguing forays into expressing data for senses other than vision.

Listen to wikipedia is a wonderful multimedia audiovisual exploration of an unexpected dataset: the recent changes to wikipedia feed. It is in fact surprisingly musical. Data are shown as circles in white (for edits by registered users), green (for unregistered users) and purple (for automated bots) on the Payne's grey field, like in the screenshot above. Text labels appear briefly to show which articles are being edited, the frequency of edits (84 edits per minute at the bottom left) and to post notices when new users register. Data make different sounds: bells indicate additions and subtractions sound like plucked strings. Pitch is proportional to the size of the edit; larger edits result in lower notes. While visualizations can communicate a lot of specific information compactly, this audio display of information certainly communicates a great deal while still allowing an observer to do something else; I am listening to it as I write. It is quite easy to pick up on the frequency of edits, their nature (additions or subtractions) and with a little more attention to the pitch and hence size of edits. Listen to Wikipedia was written by Stephen LaPorte and Mahmoud Hashemi, and is open-source. It was inspired by Listen to Bitcoin which plays tones with pitch scaled to the value of Bitcoin transactions.

BevLab image via Spark blog

You might recall magpie&whiskeyjack wrote about Kate McLean's SensoryMaps which plot scents on imaginative maps of places like 'Auld Reekie', Edinburgh itself. But smell and taste themselves can be used to express datasets. The CBC radio show Spark episode 227 profiled i & j ideations and their BevLab project which translates data into different flavoured beverages and invited the public to 'taste the data' of real-time tweets about food. The words they recorded were mapped onto flavour profiles which controlled proportions of ingredients (for instance blueberry juice for sweet, lemon juice for sour, and ginger juice for spicy, standing in for salty) in a beverage. A beverage can be produced at any given moment which is thus intended to represent the food words posted on twitter at that given moment. I am not certain of our ability to identify this three dimensional sweet/sour/spicy beverage and comprehend how the data is changing in time, by drinking a series of beverages, but the idea is playful and intriguing.



Artists are also playing with the use of data in their art - and these projects too sometimes involve more than just sight. Artist Charlotte Jarvis collaborated with the Netherlands Proteomics Centre on a fascinating project called 'Blighted By Kenning', described in the video below. The Universal Declaration of Human Rights was encoded* into DNA of bio-engineered bacteria which was sprayed onto the surface of apples grown near the International Court of Justice at The Hague. The concept was that The Hague was 'contaminated' with the message of the Universal Declaration of Human Rights on these contemporary 'forbidden fruit'.  Then they sent the apples were sent to Genomics laboratories across the world, which were asked to sequence the DNA and to find the message hidden within. Finally, the scientists who sought the hidden message were also invited to eat the fruit. So this is a very different way one can taste data.



*There is an established means of encoding letters in DNA by mapping each letter onto a codon, a tri-nucleotide unit consisting of a specific combination of Adenine (A), Thymine (T), Guanine (G) and Cytosine (C). Read more on Charlotte Jarvis' Blighted by Kenning site.

MODEL OF THE UNIVERSAL DECLARATION OF HUMAN RIGHTS EXPRESSED AS A PROTEIN

Blighted by Kenning installation (Photo by James Read)

 

Alan Turing Centenary

'Alan Turing'
Artist: Stephen Kettle
Title: Alan Turing
Material: stacked slate
photo: Leo Reynolds

Bletchley Park National Codes Centre
Bletchley Park, Milton Keynes, Buckinghamshire, England, UK

Alan Mathison Turing, OBE, FRS (23 June 1912 – 7 June 1954), mathematician, cryptanalyst, computer scientist, prophet and hero, would have been 100 today. He is someone I would like to portray, but I have been stumped. My scientists are shown with images of something quintessential to their science, or the reason they are famous (or should be), but Turing had so many accomplishments, it isn't obvious what to portray or how. You might recall his portrayal in Neal Stephenson's Cryptonomicon. I was introduced to him many year ago by Douglas Hofstadter's  Gödel, Escher, Bach (which I can't recommend highly enough... go read it now). You may be familiar with the Turing Test or at least its portrayal in Blade Runner. Turing foresaw not only that machines might quite likely develop the capacity to think (after all, our brains are only made of matter, and complex systems of neurons, which either fire or not, much like an electronic switch), but that we needed an objective, double-blind test to determine whether something/someone was able to think, as early as 1950, when most people were only dimly aware of the existence of any sort of computer. But Turing quite literally defined what we now mean by computation itself (with his concept of Turing Machines) back in 1936. During the WWII he worked as a codebreaker and invented the device which was finally able to crack the notorious German crypotographic Enigma machine! His work undoubtedly saved many lives, and today we recognize him as a genius and a hero. During his all too short life, he also made important contributions to mathematical biology and explaining morphogenesis (the biological process that causes an organism to develop its shape) and the existence of Fibonacci numbers in biology. 

Tragically, he lived in a time even more biased and bigoted than our own. Rather than recognizing the magnitude of his contributions to society during his lifetime, he was prosecuted for his homosexuality (still illegal in Britain in 1952) and forced to undergo chemical castration. He died two years later, after eating a cyanide-poisoned apple (determined by the coroner to be a suicide). It is truly abominable they way he was treated; while we can't address the past injustice we can remember, recognize and celebrate his remarkable achievements today.

Check out this Turing Machine built from LEGO, in honour of the 2012 Turing Year!

(via Brain Pickings, which also has an interesting article about Turing) 

This beautiful paper Turing Machine, by The Real M Davey employs a microcontroller and some electronics, but it is really the paper itself which is the computer (as the "tape" is described in Turing's original paper:

My goal in building this project was to create a machine that embodied the classic look and feel of the machine presented in Turing’s paper. I wanted to build a machine that would be immediately recognizable as a Turing machine to someone familiar with Turing’s work. Although this Turing machine is controlled by a Parallax Propeller microcontroller, its operation while running is based only on a set of state transformations loaded from an SD card and what is written to and read from the tape. While it may seem as if the tape is merely the input and output of the machine, it is not! Nor is the tape just the memory of the machine. In a way the tape is the computer. As the symbols on the tape are manipulated by simple rules, the computing happens. The output is really more of an artifact of the machine using the tape as the computer.

(↬ Adafruit blog)

You can also learn how to make a papercraft Enigma machine on MAKE.