Data as a Narrative Device – ‘A Means to an End and not the End itself’

An interview with Janine Steele and Loc Dao on data-driven story-telling and emotional engagement

During his stay as a Fulbright Scholar in the Digital Studios of the NFB in Vancouver, Lomax Boyd, a scientist and documentary producer at the Howard Hughes Medical Institute, had the opportunity to work with Janine Steele and Loc Dao of NFB/Interactive.

As Chief Digital Officer at National Film Board of Canada, Loc Dao is responsible for the planning, development, management and operation of the digital platforms of NFB Digital Studio – a laboratory working at the intersection of story, form, platform and technology. Together with Janine Steele, Operations Manager at the NFB, they have been part of the team behind award winning interactive media projects including  Circa 1948, Bear 71 and Welcome to Pine Point.

For both Loc and Janine, finding a balance between story-telling and scientific accuracy is one of the key issues when creating interactive factuals. In this conversation, they share insights from their professional experience on how narrative can be used as an emotional tool to go beyond the traditional use of infographics – and they address the crucial question of how to capture and measure emotional engagement in interactive documentary configurations.

One of the most recent NFB productions they address is Seven Deadly Digital Sins – a collaboration with The Guardian. This interactive documentary self-reflectively revisits its own medium. 25 years after the invention of the world wide web,  interview ‘confessions’ and info-graphics invite the user to examine how the information revolution has affected us personally, socially and morally.

A second paradigmatic project on how interactive configuration can bring data alive is The Last Hunt. In the form of an interactive photo essay, this work combines animated info-graphics, inspiring creative sketches and photographs. The user is taken on a journey with photographer Alexi Hobbs who sets off with his grandfather on his ‘last hunt’. This expedition into the Canadian wilderness becomes an occasion to relive a moving family history and to ponder on questions such as the strength of family ties, the spirituality within the act of hunting and the connection between hunting animals and shooting photos.

This post is one of a series of conversations conducted by Lomax Boyd with creatives and producers at the NFB on the complex interplay between (scientific) data, the representation of uncertainty, story and emotion in data-driven interactive factuals.  This series will offer insights for technologists, documentary artists and science communicators in imagining how data and other representations of scientific process might be used in the creative process.

“… to be continued…” – For more on this topic, stay with i-docs and follow the discussion on facebook!

---

LB: Do you think data can be used as a narrative tool beyond the infographic, as an emotive tool? Has it been intention not to use data in the traditional sense for NFB projects?

LD: I would say that it has been intentional. Although I personally do enjoy reading infographics and go to them when they are on site or in magazines, I reserve them for the realm of the geek or the scientifically inclined, which I consider myself. In terms of our audience, however, I see data and infographics as a tool or a means and not as the end product. When we used infographics in Seven Deadly Digital Sins, they are really a way to convey what we call ‘third layer’ – the larger question that we want people to ask themselves around their behavior. It was a way to expose other people’s behaviors as opposed to the end point. The end point that we wanted in that project was for people to contemplate how they treated others and whether that has changed and whether they are proud of it or not. I do think that the infographic is a good barometer, a way to look over time as we tell more stories with data. It’s a way to look over time, but whether it conveys story or not, I guess it depends on what you classify as data. Do you for example consider wikipedia as data or a form of storytelling?

LB: There are a lot of environmental themes in the studio’s first interactive documentaries, but ‘hard science’ was not the focus. Can you elaborate on the balance between science and story?

LD: In my eyes, there is science: the science is in the story, in the interviews. And there are elements of science that come out in the chapters, like when we look at the glycerides or the effects on amphibians. That’s using science to tell a story. This is an example on how science informs the story-telling as opposed to it being the end result.

LB: What possibilities do you envision for the future of web documentaries that utilize the firehouse of information for dynamic, generative stories?

LD: One paradigmatic case here is certainly what happened after the earthquake of Fukushima: think of the crowd sourced radiation readings! People bought their own kits in various locations and shared the data that they collected. It was interesting to follow that if you cared, and we cared. I monitored for a while but I could not attach any stories to the monitoring. Thus, for me, the challenge is: Do the two go together? There is such an opportunity there.

JS: Maybe the key is that data should be both story and project specific. There’s so much data available in the world and it is there for being used. It is how we as storytellers decide to interpret that and put a human face and condition on it. At the end of the day, the data, while it might be interesting or cool at the moment, does not give us an emotional response on its own. So the question is: How do we connect data to individual human beings so that people engage with them emotionally. That is generally what has the impact.

The Counted is an excellent example. It looks at police brutality across the US. It’s basically a running log of who has been killed in US by the police. And it gives you all the information that is publicly available on those individuals, including where they are from, why they were arrested, etc. – anything from the police reports that the system can glean. The producers basically put it up in order to act as a mirror back – and to ask the question: “Are you okay, as a society, with this volume of police violence?” They don’t give any answers, but it does put a very human face and condition on what is essentially a very large list of data.

LB: To pin down what we have been discussing so far: at the end, it’s all about emotional engagement. But how can this be measured? With regard to issues of data analytics in web documentaries and your interest in developing a data API framework for future projects – what is in your eyes missing from standard evaluation metrics (clicks, page views, etcetera)?

LD: We are juggling between the ethics of what we can do and Canada’s very strict privacy laws. Within these boundaries we would like to know a lot more about how users feel in story, whether it’s watching something linear or interacting with our work. It would be great to use this data to influence the story or experience. More information on the user action and engagement could enhance interactive projects at two different levels: It could be used in the story itself if the concept fits for a story/project, but it can also be used in our platforms overall. That is nothing new, we are far behind the tech companies who are actively developing and using those technologies.

LB: How do you measure how someone feels?

JS: Facebook can help to capture different responses. Now you can assign an emotional reaction to the information or stories that you are sharing on Facebook because people are giving you feedback. The same is possible if you convert all of their comments into text files that you then can parse through, you can start to get at emotion.

Of course, this isn’t something that we can apply on a large scale, but when Anagram did Door into the Dark at Tribeca, they hired the Harmony Institute, a social impact research company in the United States, and they used biometrics to capture reactions: they had for example heart rate monitors on all of the people that went through the experience. As Door into the Dark is a linear audio story, they could then correlate the participants’ heart rate to what they were hearing or seeing. In theory, you can parse through that data and see at what point in the story did people react or not. If you were doing that in an iterative, prototyping stage then you could adjust your story.

LD: We played with this method, not at the same level as the tech companies, but at the level of natural language processing and by observing what people are writing. Then we tried to build profiles of them, to come up with assumptions and to play with those assumptions. We haven’t released a project that is using that yet. In our Montreal studio, we are trying to see what we can do with this interview project. What is the role of AI in a VR project? How can we have a character react to you, or understand that person well enough to have an AI engine control how they react to you?

LB: Thinking about UX within a simple narrative space like The Last Hunt, where scrolling is the major interaction, how do you envision transferring analytics from projects with widely divergent forms and structures? Or is it only applicable within serialized content?

JS: Right now in analytics we use events – we want to know what the user is doing at this point – and we’ll flag the analytics system at that time, it’s about expanding in a project specific way. There are certain ones where we want to know how someone comes to the page, how long they stay, how many pages do they go through before they leave, do they share it. Those are universal analytics that pertain to all of projects.

LD: We need to baseline our projects.

JS: Those events become very unique and then you need an analyst that is their sole job. Importantly, what do we consider success for each project? Is it how long they stay on the page, whether they’ve made it to the end of the story, or all of those things.