app_demo_s

Production of interactive documentary in the latin area (III)

In All posts, Research Articles by Arnau Gifreu

 

The content of this series focuses on an introduction and theoretical approach to the emerging genre of interactive documentary in the Spanish and Latin area. Several examples are cited and three significant projects produced in recent years in the Spanish area are discussed in depth: Guernica, pintura de guerra (CCRTVi and Haiku Media, 2007), Las Voces de la Memoria (RTVE, Association of Relatives of Alzheimer’s Disease Patients of Valencia and Barret Films, 2010), and Metamentaldoc (Universitat de Vic and Pintao, 2011).

0

Figure 1. Audiovisual documentary inside the i-doc

En un xip multicolor [In a multi-color chip] (2010)

This project is part of a transmedia project which incorporates an audiovisual documentary, broken down by theme and videos that explain unfamiliar concepts about the figure of Neil Harbisson, a sonocromatic cyborg, an interactive doc and a mobile application (Eyeborg APP). In the interactive space, the user can experience the feeling of creating their own sound picture, with the possibility of sharing in social networks and post it on the online gallery.

1

Figure 2. Home

The idea is to create an interactive application and an interactive documentary. Both the application and the documentary are on Neil Harbisson disease condition known as achromatopsia or monochromatic (genetic disease, congenital non progressive that affects vision as a result of this are visible only black and white). The documentary addresses the theme of man and the use of technology to improve their living conditions and more specifically it uses the term cyborg, because the character in which the documentary is based on is considered as such. It is for this reason that one of the possible metaphors when designing the web interface is to create a system that allows the user to understand this idea of man and machine.

2

Figure 3. Interactive play

3

Figure 4. Online gallery

In the case of “En un xip multicolor”, the theme and its interface have been attractive for different institutions and centers that treat vision problems. They were interested in participating in the project, and this has enabled the Foundation Cyborg – an international organization dedicated to helping humans become cyborgs – to be better known and expand their partnerships: in 2010, Neil Harbisson and Moon Ribas created the Cyborg Foundation, an international organization to help humans become cyborgs.

eyeborgapp logo

 

Eyeborg App

 

The Eyeborg is an electronic device that allows colours to be perceived as sounds. It was developed by Neil Harbisson and Adam Montandon in 2004.It is a small sensor placed at eye level, held in place with a cable that transports colour information to a computer or chip. The electromagnetic light waves are turned into sound frequencies that are heard as musical notes.To learn the different combinations of colour and sound, Neil memorised the frequencies associated with each colour: high frequency colours have high notes and low frequency colours have low notes.

eyeborg_2

Figure 5. Neil Harbisson using the Eyeborg App

The foundation was created in response to the many letters and emails received from people interested in becoming a cyborg. The main objectives of the foundation are to extend the senses and human capacity by creating and applying cybernetic body extensions, promote the use of cybernetics in cultural events and defend the rights of cyborgs.

eyeborgapp4

Figure 6. Sonor picture generated with the Eyeborg App

The original interactive documentary has expanded to a transmedia project which integrates different parts and platforms to recreate the difficult situation of having visin problems.

app_demo_s

Figure 7.  Eyeborg App

eyeborgapp3

Figure 8. Examples showed at interDocsBarcelona Exhibition 2013

Las voces de la memoria [The voices of memory](2011)

Las voces de la memoria is a project that consists of a documentary about a choir composed of Alzheimer patients, from its creation to its premiere at the Palau de la Música in Valencia, and through many months of rehearsals. The documentary shows us, through its characters, the different states of Alzheimer’s disease, and the strategies and experiences of families and caregivers, focusing especially on the relationship between music and the brain and the power music has to recover memories, communication, emotions and the identity of the people who are losing these things due to the disease. This project is co-produced with the Association of Relatives of Alzheimer’s Disease Patients in Valencia and subsidized by the ICAA in its call for works using new technologies, as it is an ambitious transmedia project composed of a linear 54-minute television documentary and an interactive website that expands and complements the linear version with different tools.

Las voces de la memoria is a documentary about Alzheimer’s disease and the therapeutic power of music, which follows the progress of a musical group composed of thirty people suffering from this disease. The documentary tells the story of the choir from its beginnings to its performance at the Palau de la Música in Valencia on the World Alzheimer’s Day in front of hundreds of people. It is the story of the huge effort made by sick people suffering from a dementia that gradually destroys their memory, and cognitive and social skills, including the ability to read and speak, who perform harmoniously together in a concert in front of a large audience. This concert was held on September 26, 2010, when the audience at the Palau de la Música in Valencia witnessed a unique concert: La veu de la memòria, a choir made up of 30 Alzheimer patients who performed a repertoire including folk songs, gospel hymns and complex polyphonic exercises. During the course of this story, the viewer participates actively in the protagonists’ personal avatars – patients, caregivers, familiars, etc.. The proposal extends the information on the disease and music therapy, and is witness to the evolution of the characters, their perseverance to preserve their identity and their ability to enjoy and engage with music as an ally.

The audiovisual documentary, shot from the formation of the choir and over months of hard work, shows how it was possible to create a harmonious choir. This is the account of the unprecedented efforts made by patients, their families and caregivers, the music therapist, and the choir director. Spectators could attend rehearsals and preliminary concerts to the music therapy sessions and daily activities of the centre of the Association of Relatives of Alzheimer’s Disease Patients in Valencia, attended by the choir members. Information on the characters, showing their family life and how their caregivers live with the disease, was also given. In the interactive counterpart, the viewer decides how to experience the documentary thanks to an interface that offers a multitude of possible routes. By accessing the site, the interactor can find the home screen organized around the history of the choir as eight displays arranged chronologically: Music therapy session: Rhythm; First tests; Creating the choir music therapy session; Harmony, first concert; Music therapy session: Melody; Second concert: Music Day; Final testing, Final concert at the Palau de la Música (World Alzheimer’s Day).

4

Figure 9. Home

Each screen starts with an interactive music therapy session (karaoke, percussion, tuning, choir address, etc.) that places into context and introduces the corresponding video section on the history of the choir. Each video allows the viewer to connect with several of the choir members to access more videos that narrate their experiences, and those of their families and caregivers. These stories help us understand the different symptoms of the disease, its phases and the therapies that can be applied, stories that the viewer decides whether to keep or discard, stories that shape alternative routes based on decisions, stories that intertwine to build a documentary tailored to the viewer.

These stories, in turn, contain metadata that enable us to access medical and technical information on the subject of the sequence. So, when we’re watching a video on one of the characters, an icon of the brain lights up and indicates that some of what was said is labelled and therefore there’s information that can be consulted. Clicking on this icon will access a new interface that stores all multimedia, texts, graphics, interviews with experts, etc..
The user can navigate in this new interface independently and is part of a graphic animation illustrating the brain and the damage the disease causes in different regions of the brain as it progresses. Both interfaces are accessible at any time and at logoff the viewer’s computer stores the configuration, so that they can resume the story at the point where they left off or go back to the documentary through another narrative thread.

5

Figure 10. A song – a memory. Create your own music section

The “A song – a memory” allows complementary linear and nonlinear versions to expand the viewing experience. The website provides a more immersive viewing experience that illustrates the relationship between music and the brain, which is based on the use of music therapy as a tool for combatting Alzheimer’s. In the future there will be programmed music and video games, and several mechanisms have been designed taking advantage of applications like Spotify. The vastness of the song library and the Spotify social network makes it the ideal place to develop numerous interactions, such as sharing the musical experience of the film (the Spotify social app for generating and sharing playlists, which could be tailored by choosing among the issues of the choir and the many versions), as well as personally experiencing the evocative power of music to develop a list of personal life issues that is completely analogous to the history of music that is used in music therapy, going on a journey through songs linked to our memories, and making music: singing with lyrics and remembering lyrics. This shows that songs are simple games that give us an enormously entertaining and immersive experience that can be developed from the Spotify application. These games are based on the viewers’ choices and are perfectly suited to their interests and tastes.

6

Figure 11. Youtube channel

The documentary is broadcast using a multiplatform access strategy from the many websites and portals of the organizations working in the field of Alzheimer’s, as well as documentary media broadcast portals and local and national portals. It also has an innovative viral dissemination strategy based on the traces that viewers’ leave when they navigate in the documentary on the social networks Facebook and Twitter. Users can access the interactive documentary from their identities, and therefore provide their social networking profile news on their navigation in the webdocumentary.

 

Arnau Gifreu

Researcher, lecturer and producer

References

Barret Films; RTVE; Asociación de Familiares del Alzheimer de Valencia. Las voces de la memoria. Spain (spanish), 2011.  http://vocesdelamemoria.rtve.es/

Blasco, I.; Campos, G.; Figueira, M.; Gifreu A.; Molinos, M. Metamental-i-doc. Spain (català i espanyol), 2010). Facultat d’Empresa i Comunicació. Universitat de Vic. http://www.metamentaldoc.com

CCRTV Interactiva; Haiku Media. Guernika, pintura de guerra. Spain (catalan), 2007. http://www.tv3.cat/30minuts/guernica/home/home.htm

Costa, A.; GIFREU, A.; Martinez, I.; Parés, J.; Soldevila, R. En un xip multicolor. Facultat d’Empresa i Comunicació. Universitat de Vic. Spain (catalan, spanish and english), 2011.  http://www.xipmulticolor.com