Towards the end of last year, I convened a conversation between multimedia artist Oscar Raby and Pervasive Media Studio residents Pete Bennett, Amy Rose and May Abdalla.
To give some context, Oscar was at the Studio showing his VR documentary Assent and we thought it would be interesting to change up the interview format and allow for more of a discussion about virtual reality and storytelling. I invited Amy and May who together are Anagram, an award-winning creative digital company whose recent immersive documentary Door Into The Dark, won the Storyscapes prize for best interactive work at Tribeca Film Festival, 2015.
We were also joined by Pete Bennett, a researcher in human-computer interaction at the Bristol Interaction & Graphics Lab. His research is driven by the aim of joining the virtual world of digital bits with the real world of objects, and in doing so allowing people to physically interact with computers.
Jess Linington: Okay, Oscar do you want to start by briefly telling us about your work?
Oscar Raby: All right. Well my name is Oscar Raby, I’m a multimedia artist. I am working with virtual reality to create factual content. Experiences that use virtual reality to show you and put you in the place of someone else, or maybe somewhere else or some other place in time, where you could not naturally be. Perhaps challenging the medium to see what it can come up with, in terms of what it’s grammar can do, given that it is not yet developed. The project that I’m showing here in Bristol is called ‘Assent’ and it’s a virtual reality documentary about my father. It works as a portrait of the relationship I have, and I’ve had, with my father, himself being an army officer, in part of the military dictatorship in Chile, where I come from.
Pete Bennett: So, I’m Pete Bennett from the University of Bristol interactive graphics lab and we have been dabbling with some virtual reality in the project on called, ‘Tangible Memories’.
‘Tangible Memories’ is a project looking at putting stories within objects in care homes for the elderly to bring them together and create a space for them to talk and share stories.
We are actually using virtual reality as an opposite; this is an objectless way of telling stories. So rather than holding an object and telling a story around the object, you are taking them to a place that is entirely virtual.
“We’ve all got an idea of films and all sorts of popular culture saying ‘Virtual reality is amazing, it will take you out of your body’”Pete Bennett
JL: That leads in to my next question, so I’ll ask you first; What are you excited about – in terms of the technology and what it could offer a project like Tangible Memories?
PB: Well, in a way, the one thing that we’ve found is that over 80s don’t really have an idea of what virtual reality is. So we’ve all got an idea of films and all sorts of popular culture saying, “Virtual reality is amazing, it will take you out of your body.” Whereas, when you put it on a 90 year old, it’s just like they are seeing a really big TV screen. So there was actually a really indifferent response “Ah, hey. That’s nice.” So then you tell them to look round and rather than “look, wow it’s moving!”, they are like, “Oh, yes, yes, you can see over there too,”
It’s interesting because we don’t get the magical response, as such. It’s different, it’s enjoyable but it’s not wrapped up in ideas of what it could be. So it’s actually a really interesting base for people to try on.
A participant in the Tangible Memories VR experiments
OR: I actually wonder if that has to do with the transition they’ve gone through from have no TV previous to the ‘50s, to our times. Whereas us, we’re used to having screens around and we are used to managing the language of the frame. In that regard for me, there’s a strong interest in understanding how. It’s not so much about screens but about triggering responses. What your body already knows.
In that regard I connect strongly as well with the work that Amy and May here do, in terms of putting you in a place, in a space. In activating your body in the presence of a story.
“I don’t want to get intoxicated by this idea that I could just be transported somewhere else.”Amy Rose
JL: Yes I was going to say – Amy and May, you recently attended the XO Labs workshop in Liverpool on hacking the Oculus Rift – is it something you can see yourself pursuing?
Anagram (Amy Rose): I can definitely see us working on projects that use virtual reality. But, for me, I find the immediacy of this magic, how Pete just mentioned, a bit of red herring. I don’t want to get intoxicated by this idea that I could just be transported somewhere else. That would be like this fantastic thing.
I think it’s really important to hold on to your practice as an artist, or a maker, of some kind. For me, what we are interested in is empathy and being close to a story and making people feel that their participation is meaningful, that is has relevance to their life. And just because you are in virtual reality doesn’t mean that that’s going to happen.
So, I’m kind of interested in the relationship between haptic technology and virtual reality, and how we can use the body alongside these new zones of vision.
Although I do think that the effect it has on you and your feelings about, like putting yourself in a world; I think that’s really, really, interesting, and that could be taken in all sorts of directions.
“I have immersed myself in ulterior planets, without virtual reality, numerous times, through books..”May Abdalla
Anagram (May Abdalla): I guess, similarly, it’s interesting that you would say that the older people who haven’t been part of the dialogue of redemption through virtual technology reality, aren’t necessarily falling for it. If you have been waiting for the Star Trek Holodeck then you might think we’re getting closer.
It’s those kind of references that you feel like you are achieving something that you wanted to achieve as a child. On the other hand, I have immersed myself in ulterior planets, without virtual reality, numerous times, through books.
Anagram AR: And I think even the idea of feeling like this story is happening to you is something that film achieves over and over again. So it’s nice that people are talking about this because this is the kind of thing that I want people to be talking about. I’m excited by the way people feel they have to talk about virtual reality, as in putting the user first, and imagining this new possibility for storytelling, so that’s good.
Otherwise, I think if the question is; do I see myself using it? Possibly. It’s just a matter of experience first and I think the fun will be while there is still this kind of confused euphoria about what will happen.
It’s the early days of video art where anything that you make gets people exited. It’s a really good place to just play with ideas and get a response to that.
JL: Virtual Reality is currently very much a single person experience, that might change in the future, but at the moment it’s a single person thing. How do you think that affects the experiences and do you think that’s a limiting factor?
OR: It is limiting indeed, but that’s not a bad thing. I think it’s a clear response – at least I see it very clearly – to the social media feeling of the stories.
You know, you get your stories by this multitude of voices and there is a constant influx of them, and at some point you’d like to shut down. You would like to walk away from Facebook from Twitter, from whichever is the platform that you are using to get your information.
You want to be on your own, but still in touch with that digital world.
There is nothing necessarily intrinsically evil in the digital, whatever medium or language or tools that you are using. It is just having enough time to reflect back on what you already know or what you want to get in touch with.
These might be your feelings, your stories, the stories of people around you and not necessarily big corporate messages. I think having a mask on you, although it’s kind of still a box in your face, gives you that time where you can perhaps meditate in another form.
Assent exhibiting at Sundance Film Festival 2015
“There is nothing necessarily intrinsically evil in the digital, whatever medium or language or tools that you are using”Oscar Raby
PB: I’d also argue, in my experience, it’s always been actually quite social as well. So you have the rift on and you have a mirroring on your screen and then you have five people stood around going, “Wow, you’re just going to fall off the edge!”
So in my personal experience I haven’t actually ever sat at home yet and just popped it on, or seen anyone do that. It’s always been in a “Hey guys, check this out” setting. There is a whole observer element to this as well that we haven’t picked up on.
OR: There’s a performative side to it in terms of what happens to when you are in the room – you get to go through these experiences in a collective environment.
There are two things thing that I’ve observed: One, is that you get totally isolated, so you get to be with yourself, on your own, in front of a story. The other thing is, once past the funny moment of, “Oh, look at what he’s doing, he’s acting like, he’s moving his body in a crazy postures and gestures.”
Then there this is this, kind of, wrapping around that person to keep that person safe. Like, “Let’s not let that person fall. Let’s not let that person get dizzy.” There is some sort of taking care of that person when it happens, which is 99% of the time.
JL: Thinking back to Door in the Dark – what are your positives from having a single person experience?
Anagram: Well, for us it was a combination of wanting to make something in which you were not performing – with immersive theatre often the extroverts win. It’s really annoying if you are not one of those people, and you’re like, “I’m in the wrong place, and I’m doing the wrong thing. There’s these really funny people over there, and all the actors are talking to them I’m just like the boring person in the corner.” So we wanted to remove all of that competition.
May and Amy with their Door into the Dark at Tribeca Storyscapes 2015
But I also think that it was a question of our background. Film is a one-on-one experience, in a way, even though you can experience it as collective.
When you watch a film, the story takes you somewhere, so it is no surprise that we made something that was like that, because we have made films and it was quite hard to leap out. You don’t leave your roots immediately.
JL: What about scalability – how do you make a single person experience for the masses?
Anagram MA: It is a hilarious problem. Door into the Dark is like the worse business idea, ever. It’s really difficult. Where as something like the Rift is great, you can just build something. Just like a film, you’ve made something that can go everywhere and that’s brilliant. That’s good for the audience then more people can see it. And just because, Door into the Dark delivers this intense solitary experience, like it takes a huge amount of effort to make that happen. That’s a challenge, for us that is difficult.
Anagram AR: Luckily it wasn’t a business idea. Just to clarify, we weren’t horribly surprised when Nickelodeon didn’t call us up.
JL: How about you Oscar, in terms of funding and commercial scalability – is this something that occupies your thinking?
OR: Oh, totally, of course. But firstly in that regard I have to say something about Door into the Dark and immersive theatre. Especially, an experience like that, that is not aimed to reach out collectively, but more intensively to the individual experience.
There’s a politically engaged artistic stance in that. Having the biggest effort for the lowest of returns; you are saying something there, which funnily enough, is the opposite of what virtual reality in its current incarnation is doing.
I am quite fortunate to be producing stories for this platform, because this platform is growing itself. I don’t need to do that much effort. I just need to keep paddling until the wave crashes, which is, as you know, when the commercial launch happens. It all seems to be going that way. So I’m positive about that, but that doesn’t mean that the market will be overcrowded in no time.
JL: Do you think that currently, it’s still a relatively small audience that’s interested in this work.
OR: Today, yes. 2014, the only audience is us. People interested in how a technology is reaching this niche in the storytelling part of the world that we are interested in; which are factual, interactive and engaging stories. It’s not, you know, the headsets are not out there.
Anagram: Yes. It’s because of access, though, isn’t it?
JL: Yes.
Anagram MA: My mum hadn’t heard of it, she didn’t know what a Oculus Rift was.
Anagram AR: Nobody. Everyone was like, “What are you doing in Liverpool?” And I say three words and they don’t know what any of them meant: “Oculus Rift Hackathon” “What?”.
“The community is excited because somewhere there’s a golden egg”Anagram
JL: I think there’s always an interesting thing with that as well. When you are so deeply embedded within these communities and everyone is talking that language and is talking about these sorts of things as though they are the norm. Everyone’s aware and you are in this echo chamber of ideas.
At the same time it ends up making you feel like the industry, the community is a lot bigger than it actually is and there’s a lot more people engaged in it. But when it actually comes down to it, it’s still the same people talking about this work, the same people viewing this work.
Anagram: I was just going to say I think fundamentally for the audience, it’s beyond the people who want to make things for a platform. People are interested in it by way of what they will do with it; for the story.
Anagram: And, you know, the community is excited because somewhere there’s a golden egg. Even if people don’t think they are going to make it, they really want to start placing their bets on how it’s going to turn out.
PB: So, one thing I’m interested in – I can hear this in all this conversations – is the historical nature of the art.
At the moment the conversation is ‘Oculus Rift’, but in my mind, because I did Cybernetics as an undergraduate, we were doing VR modules in 2001, 2000. And it was deeply uncool. So I was really interested. Everyone was like, “Virtual Reality – that’s so late ‘80s, early ‘90s.” I was like, “No, it’s going to be big again.”
So I’m interested in how the artistic community engages with its past. People have been trying this stuff, storytelling in VR for 25 years, I’d say. Is that something you look back and observe, or is it all a future? It’s a really awkward question, I’m just throwing it in…
“We were doing VR modules in 2001, 2000. And it was deeply uncool.”Pete Bennett
OR: Totally. Media archaeology is that thing that allows your practice to have sense. It’s rooted in something else other than the time you spend in your studio. So, when we are using, when I’m using, the virtual reality platform, it’s to update a physical performance. That’s one of the disciplines that I am updating in my own practice. The other one would be the digital art, which is a newer discipline.
So yes, there is a tradition that you are bringing up to speed to what’s going on around you in the world. That’s what I meant by the political stance or the artistic statement that you are giving by using certain platforms but not the other, and then five years down the track you use another one. My previous practice was all physical performance.
Anagram: On that, I think there’s this thing about seeing the Oculus that has an impact on your experience. You are like, “Look at this black plastic spaceship-looking thing I’m going to put on my head.”
I want to know when it’s going to look different, or how we can use it to stage the experience in a different way. Can you put it inside a massive ball that you put on your head? Or, how do you theatricalise it?
Because Door into the Dark you have no idea that there is all this tech running. That these things called iBeacons are firing and that there’s an App, and it’s just totally invisible, and that was hard to make that work.
But it was really important because otherwise the audience is just like, “Oh, I wonder what the hell that’s working.” And they loose engagement with the story. It is different because in Oculus you put it on and then you’re in it and then maybe you do forget that there’s this techy thing.
I don’t know, we shouldn’t just make work about tech. We should make work that is about all sorts of other things and transcend that.
Door Into The Dark trailer from may abdalla on Vimeo.
“I want to know when it’s going to look different, or how we can use it to stage the experience in a different way”Anagram
Anagram: Yes, like a window. Or a swim. You’d go for a swim and see some stuff.
OR: Totally. Totally. Yes, you don’t think of the swimming pool as a platform, right? But you use it as a support for the swimming experience.
Anagram AR: Especially if you are going diving, when you go, and you put on some gear and you see other worlds. You know.
Anagram MA: You should probably mention your breathing experience.
Anagram AR: Yes, so, we met this guy in Amsterdam who’s this Irish dude who is a massive gamer who also builds games. And as a little side project him and this guy who he met on the internet are making this piece called ‘Deep’.
They’ve made a strip that goes around your diaphragm and it measures how deeply you are breathing in, by the movement of your rib cage. You put on the Rift and you are in this, kind of, undersea world.
You move by breathing, it basically is asking you to breathe very deeply. He has another interest in meditation and in mindfulness, well, deep breathing basically. So, the mechanic of the game is totally to do with that. It was really beautiful. I loved it.
___
JL: Oscar, how would you compare your work with the sort of ‘immersive journalism’ stories that Nonny de la Peña is producing?
OR: The context for the two pieces or for the two approaches; mine is one work, it happens to be autobiographical, so perhaps I can naturally claim it. That allows me to go for it. It comes from an artistic practice, whereas Nonny’s is a journalistic practice.
She’s a journalist. So she will be putting herself on the spot by doing that, and she’s been doing it for over seven years now. That’s the risk of every journalist, how and why could you claim to approach someone else’s story, other than your own.
I have no idea how to answer that, that’s just a matter or spending time with that story, and somehow making it work for you. And, through you, make it work for someone else, hopefully.
“My deepest drive is to think of myself when I was 12 and I would have loved to see this story”Oscar Raby
And then again, when I started doing this particular work, my first drive – and has remained my deepest drive – is to think of myself when I was 12 and I would have loved to see this story, to know that I wasn’t the only kid going through that.
Because somehow, in my head that kid was, you know, living, a happy life, and suddenly, grew up, had his coming of age moment and realising that he was not living on the bright side the historical coin. Once it flipped he realised, you know what, it wasn’t like that. So, you’d better check, and double check what you were thinking. So it’s kind of harsh.
I would love to reach out to that kid, whoever that is, and tell him, “You’re not the only one.” I’m not sure if your life is right or wrong, but you’re not the only one.
Assent – Trailer from Oscar Raby on Vimeo.
So was the reason why you chose to do this work in virtual reality, rather than making a traditional, linear documentary?
OR: You know what, the way to get there had to do with the irony. Again this has to do with the artistic practice and the archaeology, the lineage, of your artistic practice.
When I started, when I studied art, irony was the thing, and we know that. That was the previous thing, just yesterday. I think that I am walking, decidedly, away from irony and towards brutal honesty; updating that tone of postmodernism.
So yes, still playing with that, still feeling the whiff of irony, I thought, you know I’m going to do a twist on first person shooter game. That’s where I’m going. So I was cooking my story playing with first person shooters to see what kind of twist I could give them. I found the Oculus Rift, and realised virtual reality, and the story were a natural marriage. Yes. That’s how I got here.
“There is a relationship between work that makes you feel like you’re participating, or bearing witness, and that wider societal context, that I think needs to be explored.”Amy Rose
JL: Cool. Does anyone else have any comments?
Anagram AR: Yes – I think there is a theme between Assent and what Nonny does. That there is this thing of bearing witness and the experience of bearing witness is tricky, in our society at the moment, because we are an apathetic society.
We don’t really participate much, and there is a relationship between work that makes you feel like you’re participating, or bearing witness, and that wider societal context, that I think needs to be explored. Because, making a piece like Oscar’s, the thing that it made me feel was that I was experiencing this thing in which somebody felt impotent, in some way or unable to act.
Just like him, or like his father, or the story itself, I was experiencing the same thing. That I felt, you think that you can wander around this virtual zone, but in actuality you have no effect. And, there is something going on there and I don’t really know where it’s going.
With Nonny’s work, I haven’t done it, but everything I’ve read about it, and heard her speak, that she is passionately engaged with this thing of bearing witness, but the experience of hearing a Syrian street and bombs fall is ethically really tricky. You saw it though, didn’t you May?
Anagram MA: Yes I did.
Anagram AR: But yes, it’s a grey area. What do you think?
Anagram MA: On the subject of ethics, I think that conversation about whether it is good for us to see violence as opposed to cocoon ourselves into a bubble that has no relation to the experience of the rest of the world. I think, I can only give a personal response as opposed to something that I think is a hard and fast, should be enforced in law.
I know lots of people who would complain, but I think it is important. And I think it’s uncomfortable. The question then is: in whose hands do you leave those tools? And, obviously I respect Nonny and Oscar and I think they are allowed to make decisions about that.
Obviously there will be the gratuitous version in the way that there have been gratuitous films depicting violence, and those conversations will happen as and when we come to them.
But, in a way, perhaps it’s more like a freedom of expression idea, because otherwise… Sorry, I’m not making myself very clear but I think fundamentally the same questions exist. I don’t know if they are new. You can always look away. I guess it would be different if you forced people to see.
Often you don’t know what’s going to happen. I mean there is a warning in Assent that you will see, and it is shocking, and it’s visceral and it’s contextualised perfectly.
There is no way that that is used against me as the viewer and not everybody is going to have the sensitivity to do that right, and we won’t even be able to stop them doing it before they do it wrong, but we should definitely talk about it.
JL: Thanks. We have to wrap soon, so let me finish by asking Oscar one more question – Can you see yourself working in virtual reality more in the future and have you currently got any plans for projects?
OR: Definitely. Yes. I set out some three years to keep working on this project before trying to wrap it up and condense it into what I’ve learnt through my stint with virtual reality.
As you’ve said before, and I’m pretty clear about it, this is what I’m using to tell this story because today this is what is going on and I’m hoping that in 10 years we are going to meet again and we will be playing with pills that give you one part of a story, and another button in your eye that takes you to another part of a story.
And I’m not talking fiction, I’m pretty sure we are going there. It’s just a matter of moving from a screen outside to the device attached to your body, wearables and the like, or the Rift to hopefully in less than 10 years things that are bringing the story inside you, or at least, as part of you, chemically bonded there.
I’m currently working with a factual content production company in Melbourne, Australia where I live, and we are doing a big project for World War I in which you are war correspondent, and that war correspondent is in charge of finding what was going on with the Australian forces.
But, this time you are given the opportunity not to focus on the regular and usual white male soldier but all the other characters that were also there, that were part of the war effort. Not in a necessarily patriotic manner, but exploring the rest of the frame while people were there, willingly or not.
I’m also doing a VR expedition, I’m going to places and meeting people and seeing how and where they live, and trying to capture their presence; how I saw them, how I learned that they were alive during that brief moment where I met them. Again I’m working with 3D scanning and audio recording put inside a virtual reality environment. I’m hoping to collect something like 10 or 12 really short pieces over the whole year and package them together in some sort of interactive manner.
From Oscar’s current ‘Islands’ project
JL: Is it still very much stuck in a CGI world or will you be looking to create work using 360 photos or video?
OR: I’m actually trying to go the other way. Because when people that I’m working with have that approach, mostly they’ve been informed by filmmaking techniques, film production, or TV production procedures. And, again, talking about the grammar of VR there are things that you can get with this sense of enveloping environment that video can give you.
It feels like a dome projection of being inside a planetarium but when you can traverse that space there is something else. It’s another camera movement that gives you a gate into the story that’s different than the dome projection felling. That sense of depth, and that you can enter the frame, it adds a different tool to what you can use to tell the story. So I’m trying to do 3D modelling, 3D reconstruction, basically, CGI. And try to discover how much expressive capacity it has by letting it glitch; letting it express itself, letting it drip, like painting, and see what the blob and the splatter looks like and feels. To see what it can provide to you as an author.
JL: Yes. I think compared to where we are with video games now, the graphics in VR can seem quite crude and unrealistic, but I don’t think that it takes anything away from the experience.
OR: Yes, that’s not the point; the thing is having a mirror. Those mirrors can be more or less distorted and you are trying to trigger some sort of reflection on that mirror. It doesn’t have to be exactly the way your eyes see it. They have to put you in some sort of mood and place and space and setting.
Anagram: But I think the thing is that a lot of the aesthetics that are currently available have a lot of association. So you see them and you’re like, “I’m in Grand Theft Auto”. And that is a problem. Either you work with it or you do something about it, because…
Like you said you were kind of riffing, on the idea of a first person shooter, that’s a fun thing to do with those aesthetics, but if you’re not doing that you’re in trouble…
You can see more of Oscar’s work here, Anagram’s work here and Pete Bennett’s Tangible Memories project here.