Smell, Taste and Hear the Data

Some of our contemporary computational and graphics tools make data visualization an exciting, rapidly advancing field. Sometimes scientists presenting data are not visual thinkers and may not be communicating their results transparently. Sometimes graphic designers put aesthetics ahead of the essence of what we could be communicating with a given dataset. But, when we get it right there is some real innovation occurring. Today I want to show you some 'visualizations' which are not in fact visual at all; I want to show some playful and intriguing forays into expressing data for senses other than vision.

Listen to wikipedia is a wonderful multimedia audiovisual exploration of an unexpected dataset: the recent changes to wikipedia feed. It is in fact surprisingly musical. Data are shown as circles in white (for edits by registered users), green (for unregistered users) and purple (for automated bots) on the Payne's grey field, like in the screenshot above. Text labels appear briefly to show which articles are being edited, the frequency of edits (84 edits per minute at the bottom left) and to post notices when new users register. Data make different sounds: bells indicate additions and subtractions sound like plucked strings. Pitch is proportional to the size of the edit; larger edits result in lower notes. While visualizations can communicate a lot of specific information compactly, this audio display of information certainly communicates a great deal while still allowing an observer to do something else; I am listening to it as I write. It is quite easy to pick up on the frequency of edits, their nature (additions or subtractions) and with a little more attention to the pitch and hence size of edits. Listen to Wikipedia was written by Stephen LaPorte and Mahmoud Hashemi, and is open-source. It was inspired by Listen to Bitcoin which plays tones with pitch scaled to the value of Bitcoin transactions.

BevLab image via Spark blog

You might recall magpie&whiskeyjack wrote about Kate McLean's SensoryMaps which plot scents on imaginative maps of places like 'Auld Reekie', Edinburgh itself. But smell and taste themselves can be used to express datasets. The CBC radio show Spark episode 227 profiled i & j ideations and their BevLab project which translates data into different flavoured beverages and invited the public to 'taste the data' of real-time tweets about food. The words they recorded were mapped onto flavour profiles which controlled proportions of ingredients (for instance blueberry juice for sweet, lemon juice for sour, and ginger juice for spicy, standing in for salty) in a beverage. A beverage can be produced at any given moment which is thus intended to represent the food words posted on twitter at that given moment. I am not certain of our ability to identify this three dimensional sweet/sour/spicy beverage and comprehend how the data is changing in time, by drinking a series of beverages, but the idea is playful and intriguing.



Artists are also playing with the use of data in their art - and these projects too sometimes involve more than just sight. Artist Charlotte Jarvis collaborated with the Netherlands Proteomics Centre on a fascinating project called 'Blighted By Kenning', described in the video below. The Universal Declaration of Human Rights was encoded* into DNA of bio-engineered bacteria which was sprayed onto the surface of apples grown near the International Court of Justice at The Hague. The concept was that The Hague was 'contaminated' with the message of the Universal Declaration of Human Rights on these contemporary 'forbidden fruit'.  Then they sent the apples were sent to Genomics laboratories across the world, which were asked to sequence the DNA and to find the message hidden within. Finally, the scientists who sought the hidden message were also invited to eat the fruit. So this is a very different way one can taste data.



*There is an established means of encoding letters in DNA by mapping each letter onto a codon, a tri-nucleotide unit consisting of a specific combination of Adenine (A), Thymine (T), Guanine (G) and Cytosine (C). Read more on Charlotte Jarvis' Blighted by Kenning site.

MODEL OF THE UNIVERSAL DECLARATION OF HUMAN RIGHTS EXPRESSED AS A PROTEIN

Blighted by Kenning installation (Photo by James Read)