Virtual Reality for Social Good, with Jeremy Bailenson

JOANNE MYERS: Welcome to this podcast, which is coming to you from the Carnegie Council in New York City. I am Joanne Myers, director of Public Affairs programs here at the Council.

Today I have the pleasure of speaking with Jeremy Bailenson. He is the author of a very informative and special book entitled Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do, all questions that we will be addressing in this conversation about the fantastic made real.

Jeremy is professor of communications and founding director of Stanford University's Virtual Human Interaction Lab. Professor Bailenson, thank you for joining us.

JEREMY BAILENSON: Thank you for having me. I am delighted to be here.

JOANNE MYERS: Since virtual reality (VR) seems poised to become the next mass-medium phenomenon, the question is, where will it take us? Will it be a tool for escapism, violence, and propaganda, or will it be used for social good, to foster empathy, and be a powerful new medium for learning?

But before we discuss these later uses, I think it would be helpful if you could just spend a few minutes talking about what virtual reality is.

JEREMY BAILENSON: Virtual reality is an experience where, I like to describe it as instead of watching a movie, you are inside the movie.

Two things make it different from a technological standpoint: the first is that it responds to your body. In the physical world for hundreds of thousands of years, when humans have stepped backward, things in their field of view have gotten smaller, when humans have put their right arms up in front of them, they actually see their arm in front of their eyes. In virtual reality we track your body movements, so the scene actually adjusts to how you move, and this naturalistic embodiment of "I control the scene by moving around through it" is really critical to the definition of virtual reality.

The second thing is that it is perceptually surrounding. So in virtual reality, when you turn your head to the right, you see stuff; when you walk toward a sound, it gets louder; when you touch something, you get what is called haptic feedback, which is a little bit of resistance or vibration so that you feel it. By having this be multisensory and surrounding, it becomes psychologically extremely immersive, and we call that "psychological presence."

JOANNE MYERS: How does it affect the brain?

JEREMY BAILENSON: I have been studying virtual reality since the late 1990s, and a lot of our experiments have examined whether people behave in virtual reality in a way that is similar to how they would in the real world, and in general what our findings say is the answer to that is yes, that when you meet an avatar, which is another human in virtual reality, you will respect that avatar's personal space, or you will not gaze into the avatar's eyes for too long. When we measure your physiology, you get aroused by virtual scenes in a way you would predict for physical ones.

We have some done brain science looking at putting people in an MRI machine, but the challenge to actually measuring brain activity in VR is if you have ever done an MRI, which I am sure a lot of our listeners have, you know that you are not allowed to move. You have to stand perfectly still like a statue. Virtual reality by definition requires movement. If you are not moving around, turning your head and walking, and these very intense and often jarring movements, then you are really not doing good VR. In terms of the brain itself, we just do not know all that much about the actual patterns of activation that occur inside of virtual reality.

JOANNE MYERS: That can lead to dangerous effects, I guess, if we do not have a sense of how long it lasts or what are the long-term effects.

JEREMY BAILENSON: In writing this book it was a journey that evolved. My thinking actually has evolved a lot since the book came out a few weeks ago, just in seeing and hearing the public's response.

My general guideline for thinking about VR is that VR is great for things that you cannot do in the real world: go to Mars, go to the bottom of the ocean, try wearing a different body, all these things that just are literally impossible to do in the real world.

But virtual reality is not ideal for things that you would not do in the real world. So my advice to parents and to consumers and people who are trying out this technology is that if there is some activity that if you did it in the real world, you would feel terrible about yourself, you would not be able to look at yourself in a mirror, you would not be able to have a good conversation with your partner that evening, then you probably should not do it in VR. In other words, because we know the brain tends to treat it as real, you should not be doing things that are going to make you feel bad about yourself, things that don't meet your personal code of ethics, because in general those experiences stay with you. They are not free.

JOANNE MYERS: I think that is a very good rule to try to live by.

You mentioned the word "ethics," so what ethical considerations should guide its use? Are there practical ones that really matter the most?

JEREMY BAILENSON: I have a ton of practical guidelines, for example, how to stay safe in VR, and I am happy to talk about those. These are things like—this is going to sound obvious, but clear sharp objects out of your space because once you go in VR, you cannot really see things, and you are going to step on that cactus. I say that as a joke, but actually there was a man in Moscow in December who was doing a virtual reality game, and he fell through a plate glass table and he bled to death. That was the first virtual reality death that I know of, and of course that is tragic.

JOANNE MYERS: And we hope it is the last one for a while.

JEREMY BAILENSON: We hope the last one, but Joanne, if there is any guide, in the United States we have many deaths each year due to smartphones, distracted drivers, people walking into traffic. There are statistics you can read about on government websites here, and when I talk to the virtual reality companies, which I do often, my plea for them is let's try to imagine you could go back and when we were first designing smartphones you could solve some of these problems that we have. Please, can we start thinking about VR hardware and software and cultural norms so that people are not putting on head-mounted displays and virtual reality goggles while they are driving?

I recently gave a talk at one of the biggest tech companies in the world, and I begged them—there were 80 engineers in the audience—I said, "Can you please make it so that your virtual reality hardware cannot work in a moving car? Just embed that into the hardware somehow."

It just cannot run in a car because I know you would never imagine, listeners, that somebody would put on virtual reality goggles inside driving a car, but we have some data on this. There was a video game called Pokémon Go, which was an augmented-reality game. That is different than wearing the goggles; you hold up a phone and you see video from the real world, and there are these Pokémons, these characters in a game that are superimposed or augmented onto the image of the real world, and there are many cases of people having car accidents because they were playing this augmented reality game while driving.

We can get to ethical concerns, but there are some really pragmatic and practical ones, which is you should not be using virtual reality in driving.

JOANNE MYERS: Good point. Many of us have knowledge about the virtual worlds, like you mentioned Pokémon, and the entertainment value. But in the time remaining I would like to have you talk about the ways virtual reality can be used for social good. For example, Stanford I know is one of the leading universities studying climate change. What role might virtual reality play in reversing the damage we are doing to the environment?

JEREMY BAILENSON: I have dedicated the last—about half of my work week goes to thinking about climate change and how VR can be used to help, and I can talk about this all day. I will try to be concise.

I think the best way to describe it is to give an example. I am going to talk about Palau. A lot of our listeners may not have heard of Palau. Palau is a small network of islands in Micronesia. Palau geographically is about the size of San Jose, California, but due to nautical law it owns a chunk of ocean about the size of France. So you have this very small country land-wise that owns this huge chunk of ocean.

The ocean is known for two things: (1) it has amazing coral, the richest, most gorgeous coral reefs perhaps anywhere; and (2) due to climate change, the country is going to be among the first portion at least that is really impacted by climate change, both in terms of the country going away due to sea level rise, but also their economy is based on tourism.

There are 20,000 [people living] on Palau, and about 100,000 tourists come there each year. The tourists come to go scuba diving, they come to go fishing, and they come to go snorkeling for the most part. There is not that much to do from a tourism standpoint on land.

So I flew to Palau, and my team—Elise Ogle and Tobin Asher—got to be with the great Rob Dunbar and the great Bob Richmond, both of whom are climate change scientists. We spent two weeks with them on a boat, and we went to different points on the island where we could see the impact climate change has had on Palau in terms of coral bleaching, in terms of ocean acidification, in terms of extreme weather events.

Remember, Palau, because there are only 20,000 citizens, there is nothing they can do to reduce the world's CO2 carbon footprint, right? They cannot control that. They can reduce their carbon footprint, but that is not going to change climate change. All they can do is adapt. The way they can adapt is in three ways: they can reduce overfishing by having more marine protected areas, they can train tourists to be better to the reefs, and they can stop cutting down mangroves, which actually prevent the sediment from going into the water from commercial farming, and that damages the reefs.

I am getting to VR, thanks for your patience on my longwinded explanation here. We took all this amazing underwater 360-degree video of the before and after of climate change, and we are in the process of actually producing a seven-minute learning journey that is going to premiere at the Tribeca Film Festival in April that we are going to give away free to the world to learn about Palau, because most people have not heard about Palau, and to understand the plight. Interestingly, we did something that was a little bit different while we were in Palau. We had the privilege of going to speak to the lawmakers of Palau. There are about a dozen senators and about twice as many house delegates, and we had most of those lawmakers in the same room.

Tobin and Elise stayed up all night one night after one of our dives, and they put together a before-and-after reel that showed what an amazing pristine reef looked like in Palau and then what one looked like after it had been damaged by tourists who did not know how to swim and were kicking the soft coral with their fins and destroying it.

Palau has a very interesting culture. Most of them do not scuba dive or snorkel. They go fishing, and they will wade in the water, but it is not part of what they do. A lot of them are actually afraid of sharks. So we had about a dozen of these lawmakers go through and experience their reefs, experience what it is like underwater.

Then we showed them what it was like after it was damaged by tourists, and Joanne, the response we got ranged from people yelling to people demanding that we put this in schools—and when I say people, I mean the lawmakers—suggesting that this can be part of a snorkel training and that tourists have to do this.

At the end of that day, the lawmakers signed into law a bill saying they are going to work more with climate change scientists. Since then there has been some movement on a way to get tourists better informed from the legislature so that they can reduce this.

From an ethical standpoint, climate change it is very abstract. Most of us, unless you live, say, in Florida or other places, or California where we have droughts, not many people have got direct experience of climate change. In VR, experiencing disaster is free, and you can take this very abstract "climate change is going to destroy my country or my city," and you can feel it now so that we can be motivated to change our behavior.

JOANNE MYERS: I could not agree with you more. I have often said that unless you have experienced some type of climate change—you have lived through a hurricane, a tsunami, or an earthquake—you have no idea, nor do you really care. So the idea is to get people, I think, to really care, and the best way, as you said, is to experience it.

What would be the distribution? Schools, I guess, are the first place one could go, and training the tourists, requiring that they see this. I guess those would be two ways to begin the process.

JEREMY BAILENSON: Yes. Again, there is trying to reach lawmakers. In the previous administration, we worked with the United States Agency for International Development (USAID), and we were trying to build simulations so that leaders of developing nations would be better motivated to embrace climate change adaptation now.

As you know, climate change has begun. At this point, we cannot stop it. We can try to mitigate it. I am not saying we should not pull out all the stops to reduce our carbon footprint, but we should also begin adapting, because it is coming.

There is kind of a focused effort that we were trying to come up with simulations to help leaders of developing nations understand that they need to embrace adaptation measures now. Again in Palau the example would be regulating tourists or trying to stop cutting down the mangroves for commercial farming.

But then there is this general gestalt: Can we use VR to get everybody to care more about these causes and to actually understand them? To that end, there are probably about 15 million high-end VR systems distributed throughout the United States right now. So the hardware is there, and we can kind of let that trickle down.

What my lab is doing is two things: One, we are building installations in museums and aquariums where people get to come through for ten hours a day and experience these simulations. The other thing we are doing is that we are building these educational modules, and we are putting them up for free on websites like Steam. Steam is a place where you go to buy video games, and we are giving away our educational content on places like Steam or Viveport or Oculus Share, where people get to download this for free and actually experience this in their homes or at their friends' houses or in their schools.

JOANNE MYERS: It is so exciting. The other thing that struck me when reading your book besides all these wonderful things you can do about climate change and making people aware is making people aware of the humanitarian situation, such as the Syrian refugees. Could you talk a little bit about that project?

JEREMY BAILENSON: There are a number of groups, including the United Nations, and Chris Milk is a famous VR director, and these people are building simulations to kind of let you know what is going on with the refugee crisis. We read these numbers, and they are so large and mind-boggling that we cannot even fathom then. Then you put the goggles on, and you are in one of these camps, and you get a sense of just how epically large this problem is.

There is some nice data now showing that when you experience a day in the life of these people that you are more likely to support the cause to help them. It is a neat way to walk a mile in someone else's shoes and to really get a sense of what their lives are like.

JOANNE MYERS: Have you done any videos of the economic migrants coming from Africa and the ordeal they go through on the boats?

JEREMY BAILENSON: We are just starting some conversations with an amazing group at the World Bank that is doing a lot of work in Africa, and we are thinking about a number of domains there. But Joanne, there are so many amazing causes to think about and applications where again, what you said, if you experience something—let me tell you the story of one of my academic heroes, Dr. Jane Lubchenco. She was the head of the National Oceanic and Atmospheric Administration under President Obama for four years.

I interview her in the book, and she has this quote, which is, "When you experience something, you see it in a different light." She talks about a lawmaker, who I will not name, and this lawmaker was a very vocal climate change denier. When a natural disaster hit this person's district, all of a sudden this person became a believer. The exact quote is: "I have seen the light. I have become a believer." This intense experience changed this person.

Now we cannot make everybody fly to Africa.

JOANNE MYERS: That was a tornado that hit his city?

JEREMY BAILENSON: That's right. That was the tornado that hit his city. And we cannot create tornadoes on the fly, and we cannot fly everybody to Africa, and that would not be a great use of resources, especially given fossil fuel burning, but we can bring events to people.

JOANNE MYERS: Right. Actually, you spoke a lot about how bringing people together would save transportation costs. There are so many connections that people just do not really think of. Could you talk about bringing people together and how that will eliminate some of the gasoline and fossil fuels polluting the environment? That was another good point.

JEREMY BAILENSON: One of the things I have been working on nonstop for 20 years is can we create what is called "networked VR," social VR, such that the intimacy, the "social presence," which is the technical term for it, is good enough that you feel like you are in the room with somebody, that it is not like—it is better than video conferencing, which is still pretty good, but not enough that you are not going to go to that meeting to close a deal for work or to meet a friend of a friend for coffee to talk about something that is not that important to you.

What I have been doing is trying to build the algorithms that can do things like eye contact and to test for the cues that are important to measure. Do you have to show somebody the twitch of a cheek, or is it enough just to show a smile without having to bridge all of the different facial muscles because we cannot track everything? So from a technology standpoint to figure which are the nonverbal cues that are sufficient to create this special sauce of social interaction, and then testing to see if those work.

To be clear, I am not trying to replace seeing your loved ones or going outside. If you think about the commute, we all get in these boxes and spend an hour, sometimes two hours or more each day going back and forth, following all these other boxes, and when we get to work, what are we doing? We are just clacking away on a keyboard and talking on the phone. If we could reduce work commuting by half, we have just put a huge dent in climate change.

JOANNE MYERS: You are at the forefront of such a wonderful change in our environment. As you said, we are at a moment in time where the best way to use virtual reality responsibly is to be educated about what it is capable of, to know how to use it, and to know how it can enhance our connection with the real world and with other people.

Thank you for taking the time to emphasize the positive applications and experiences that can change us and the world for the better.

JEREMY BAILENSON: Thanks for the work that you guys do as well. I appreciate it.