Jing Yu
neutrality.gif

Aura

 

AURA

An installation that brings emotions to life as colorful ‘Auras.’ aura uses machine learning and projection mapping to visualize relationships derived from physical interactions and body language.

 
 
whiteaura.png
 
 

A SIMPLE CONCEPT

AURA uses Wekinator, a machine learning platform, to translate body-position data from the XBOX Kinect into projections mapped to the participant. Wekinator interprets the relationship of two people within the installation space and illustrates them using projection mapping.

 
 
auraprojection.png
 
 
Neutral

Neutral

Aggression

Aggression

Affection

Affection

 

PROCESS

Our team wanted to examine and reveal human relationships through the lens of machine learning. We wanted our output to be some kind of interactive installation that was inviting but not intrusive.

Concept Ideation Session – working to visual human relationships

Concept Ideation Session – working to visual human relationships

After the group concept ideation, I took on the role of coding the processing sketch that would receive data from the kinect, push the data to Wekinator and then project ‘auras’ according to the classification.

The first step was understanding the kind of information we could pull from the xbox kinect.

ezgif.com-video-to-gif.gif

Then we tested mapping joint data and motion to simple shapes and then more humanoid shapes.

Simple Shapes at midnight

Simple Shapes at midnight

The visual and auditory aesthetics of each class were designed by Juliana and Surojit. Once those aesthetics were translated to code we could then train Wekinator to learn to “see” behaviors, such as aggression.

trainingday.gif

The sketch was then calibrated to the dimensions of the surface so that the projections would mirror the corresponding participant.

auras5-1024x768.jpg

The Aura installation was shown at CIID in May 2018. The code can be reviewed here.

aurass-1024x762.jpg
 
 
 

 

aura was designed in collaboration with juliana lewis and Surojit dey under the guidance of gene kogan and andreas refsgaard.