Ghost – An Interactive Installation

Schermata 06-2456472 alle 13.19.45

Ghost was created at the end of Thomas Eberwein’s time as an interaction designer at the EPFL+ECAL Lab in Lausanne, Switzerland. It was created as a commission for the “Give Me More” exhibition by the EPFL+ECAL Lab at the Eyebeam gallery in NYC in Spring 2013. Ghost is a collaboration with Tim Gfrerer, Music by Freefarm.

Synopsis

A snow storm is raging within an abandoned, barren landscape. Within this storm the visitor can make out a procession of human forms which seemingly try to find a way out. These bodies are remnants of the installation’s previous visitors, their ghosts, forever trapped in this hostile environment. Ghost records short sequences of every visitor and adds them to the landscape. Ghost attempts to trap viewers; as they are looking into the landscape, they become part of it.

 

Detailed Description

Scene

Up to 50-100 recorded ghosts are set along a path in a long queue within a mountainous, vast landscape. These ghosts move along a trail, like a rope team in mountaineering.

Experience

The scene is seen through a camera roaming over the landscape, with the environment transitioniong through a set of pre-defined moods. Landscape moods influence the intensity of the storm, the music, the fog colour and the sound effects. The scene may transition from intense, loud and aggressive storms to gentle, quiet snow flurries.

When a visitor is detected, the camera moves towards them, choosing from a set of close up camera positions. If no visitors interact with the piece, the camera switches into wide shots of the landscape framing the captured ghosts within.

Visitor Interaction

Every time a visitor is detected, a new 3d ghost is added to the scene. The camera focuses on the new ghost, starting to record their movements for 30 seconds; then the new Ghost is left in the landscape and attention shifts towards the next visitors, their ghosts being placed next in line, until a never-ending procession of ghosts begin to form.

Ghost plays with the default behaviour of visitors towards interactive installations, the urge to wave in order to find out how an installation is supposed to react. Visitors might wave towards their ghosts, while other previous ghosts wave back, creating the possibility of endless feedback loops of ghosts waving at each other, trying to draw attention to each other.

Non-interactive approach

As much as Ghost is interactive, the installation has an allure as a passive, almost cinematic experience, inviting visitors to observe an ever-changing environment while standing outside of the detection zone. Landscape, storm and music together have an intrinsic beauty, reminiscent of classical landscape painting.

Music and Sound Effects

Ghost’s sound scape is composed from a range of musical building blocks, each based on a central musical theme. Several sound-beds, each with different intensities, matching a current mood of the landscape, from quiet flurries to raging storms.

Simon Pyke additionally created a large number of sound effects to be combined with the sound-beds, creating a rich and varied sound scape. These sound effects range from the more abstract, as in ‘foggy’ or ‘ghosts mumbling’ to the more concrete, like ‘gusts of wind’ or ‘radio glitches’.

Referencing the natural environment, large parts of the music were recorded using analog instruments, and then combined with glitches and digital effects. The sound track emphasizes the contrast between a natural stormy landscape and the sometimes broken movements of the ghosts.

Technology

The piece was made with openFrameworks, which is a community-driven, open-source framework for creative coding, and mainly written in C++.

While the landscape and the character models were designed in 3D modelling software, the bone-based character animations and the particle systems were created in code and largely written from scratch.

The visitor’s skeletal movements are recorded using the open source OpenNI framework, and fed to the installation’s main app via OSC, where the movements are then blended, attenuated, constrained and committed to memory as an animated ghost.

As soon as this new ghost is placed in the landscape, the scene camera picks up the cue, transitioning to one of its predefined positions, while looking at the new ghost. A mood sequencer decides whether to change the overall atmosphere of the piece by transitioning to another preset.

If a mood-change is requested, the general atmosphere –colour, wind speed, cloud, fog and snow density– will blend towards new presets upon the next musical beat. These presets were created & tweaked on-site so that the projection quality could be taken into account.

To achieve the performance necessary to animate up to 100 ghost characters simultaneously, OpenGL instanced rendering is used, which is a technique that allows the GPU to recycle character mesh vertex data, whilst animating each character instance individually. Custom GLSL shaders take care of bone-based character animation skinning and blending in real time.

An animation file format was defined to record and store skeletal movement. During the first run of the installation at Eyebeam / New York in Spring 2013, about 400 MB of skeletal animation data were collected, which will be re-used in future projects and iterations of Ghost.

Credits

Created by Thomas Eberwein / Thomas Traum and Tim Gfrerer
Music by Simon Pyke / Freefarm
Commissioned by the EPFL+ECAL Lab for the ‘Give Me More’ exhibition