Mapping The Imagination: Visualizing Sensory Words
Design Study, Machine Learning
Mapping The Imagination
Related Link (Parsons MFADT Data Visualization Course Blog.)
Project Timeline: Mar 2016 - June 2016
Categories: Design Experiement
Tools: InDesign, D3.js, Handcraft
Human Imagination VS Machine Learning
As a data visualization designer, I have had a strong interest in Machine Learning as a new field of study.
I explored the domains in Machine Perception which is the capability of a computer system to interpret data in a manner that is similar to the way humans use their senses to relate to the world around them, and wondered how the future of A.I. would interpret complex, abstract concepts that only human could have understood.
Through deep learning, a machine can process large amounts of data, recognize complex patterns, and provide more detailed insights. In this context, I raise a new speculative question: what if the A.I. truly be creative and use their imagination? What if the computer can read literature?
This project is a poetic design experiment that I contemplated about the question, and it resulted in a photo viz collection.
Would the computer can be imaginative with sensory data?
Human brains are complex. Our cognitive process use recognition memory to recognize certain concepts. When we read a novel, we recall our previous memory to imagine those words in our brains. Recognition memory is a subcategory of declarative memory. Recognition memory can be subdivided into two component processes: recollection and familiarity, somtimes referred to as "remembering" and "knowing", respectively. In the age of emerging AI and Machine Learning, how the A.I. can figure abstract imagery that needs complex cognitive process which called human imagination?
Proust's In Search of Lost Time is known as world's the most imaginative modern novel that follows the narrator's recollections of childhood and experiences.
Let's imagine the A.I. reads a Proust's novel:
In Search of Lost Time.
Photo Data: The Images Collected by Google Search
"Our brain see words as pictures. We recall our previous memories or experiences when read a book."
Data Gathering & Design Process
The data collecting from Google image search result was really unique in terms of getting data one by one in a way of what machines do. I experimented on how Artificial Intelligence recognizes and matches words with images by representing the process of searching and choosing images on Google. Some proper nouns and common nouns with concrete images showed up right away when searching for them, on the other I had to arbitrarily choose some imaginative words with my own imagination.