Design and Architecture

Design’s impact on experience

As with any interactive experience, as users enter they will begin trying to figure out what they can do. They learn the visual language as they do in traditional user interfaces, but in VR they also check to see where the content will live in space. As content creators, we placed targets around the edges of a view or choosing backgrounds which drew the eye so that users will see that there is more to explore. We taught participants early on that there is content outside what they see in front of them. This encouraged them to explore the edges as they move through the experience.

Understanding use cases is crucial when designing for VR. A content creator must consider many physical elements that are less of a factor to consider when designing for a desktop or mobile experience. Examples include hardware such as headsets where interaction is initiated through focus vs. headset where interaction is initiated with hand controllers, and the physical environment in which users will be participating in the experience.

For example:

• If users will be sitting on a couch or a stationary chair, constricting design to a more frontal experience will be optimal.

• If users will be standing, consider any obstructions in the environment or potential for dizziness.

• If a cardboard headset is being used where the focus reticle is used to initiate interactions, provide visual feedback to the user that they have activated an interaction. This will alleviate the confusion and surprise of an interaction when their actual intent was to focus on another object within the scene.

• If the users will be in a crowd of people, they may be self conscious about their physical movements.

Iconography

Information icons/boxes

Not surprisingly, information boxes were universally understood. Both the “i” and the “x” are familiar iconography and posed no problems for users.

Footprints icon on doorways

This icon was clear moving from room to room when it was on a door/doorway. Doorways have an expectation that aligns with the action, whereas computer screens left users feeling uncertain (see below).

Footprints on the screen

Some participants were confused when the footprints icon was used on the computer screen. Reactions ranged from confusion and uncertainty, to guesses that took the imagery quite literally. They assumed they would be seeing the inside of the computer if they clicked on the screen. Five of the participants applied the logic they’d experienced previously and assumed they would leave the room, but were still unclear as to where they would go.

Participants who were familiar with Four Kitchens being a distributed team came more easily to the conclusion that they would jump to another employee’s office. This prediction demonstrated how background and context play a role in understanding an interface.

“So since you guys are Four Kitchens and you’re always online with each other, it makes sense that you can travel through your computer and get to somebody else’s physical office. If I didn’t know this was online company, then it would have (been) a little weird.”

“It makes sense now that I think about it because that's how they communicate with each other through Slack. Their computer monitors are portals to one another like a door would be.”

When asked for suggestions as to what would make it more clear what would happen, participants noted that while using footprints on the computer screen as a transition point was not ideal, they did not have better ideas themselves.

Preview icon (Version B)

After seeing the preview option (vs footprints) preferences landed in both camps. This had to do with the mindset the user was in Those in the explorer mindset preferred the surprise and didn’t really care that much about where they were going. Where as those in the learner/understander mindset was longing for context and a heads up about where they had been and where they were going.

“This is exploratory, so I don't care that it says Mike's workshop.” vs. “I like it because it indicates where I'm going to...like, ‘Here's an overall view of different places that you can go to.’ Having that in context.”

Two participants also noted that their preference/mindset would depend on the content itself.

“Yes, if it was an application, and people talk about desktops and VR and stuff, sure.”

“I guess depending on the point, if I needed to know that much information before I went there or if I should just go. Kind of like if you're playing Mario and you know not all the tunnels go down but then this one tunnel does and you memorize where it ends up going, kind of like that. I don't think I would need so much information except to know I could go somewhere.”

Two participants noted that the preview images were difficult to decipher. Both liked the idea of it, but the images would need to be clearer in order for it to be helpful. One participant mentioned that he’d prefer a text label rather than a photo.

Footprints inside information box

Three participants didn’t think this was clear and would prefer not to have the footprint navigational link nested within the information box. Two of those users noted that they knew what the footprints would do once they open the information box, but they would not be looking for them there and did notice the inconsistency of this. The other felt as though it added an unnecessary step.

“Not as big of a fan of that, because I feel like there should be two separate icons to indicate if you get information or you can walk over there. … It's not intuitive to me to lump those two together. t should be two separate things in my mind.”

“I didn't know that I could go back there, and then when I click on the little “i” then it has the feet. It says ‘welcome to the shed quarters, go inside.’ So I clicked the feet there. … It felt a little hidden. I wouldn't have known had I not clicked on it.”

Two participants noted that the footprints being inside the info box did provide some context that they were missing before. One mentioned specifically how the “go inside” text prompt was helpful.

“It was a nice little change. I like it because, you're so into the interior of the room, you don't really think about what kind of space you're actually in, so it's kind of interesting to see oh that's...I was in *that*. I was virtually in that structure. And it’s pretty easy to go right back out.”

It is preferential to surface all primary navigation and find other ways to provide the context.

Sound Icon

This icon was universally understood with no issues at all. Issues came up in regards to the functionality and the sounds themselves, as outlined in the hand controller interaction section.

Display boxes

Legibility

Participants were able to communicate specific preferences when it came to the information display boxes. Text was universally legible, however position, scale and focus garnered suggestions.

Position

When it came to position, participants were nearly evenly split as to whether it was just right or needed improvement. The suggestion as to what could make it better was to lower the box a bit. This preference for things being lower rather than above also came up in regards to targets. Participants liked the novelty of seeing things on the ceiling but didn’t want the info box to pop-up there. They would rather click above and then move their head to a neutral position to read. While looking up, one participant placed his hand on the back of his head. Commented that some of the targets were too high and annoying to have to look up in order to interact (Xs to close display boxes).

[referencing the “i” icon up high in the tree house] “It's neck strain trying to focus on that “i’ up there. Make the icons a little bigger when they're higher up, and--because it's hard to focus on that, my neck back like that.”

Scale

For scale, responses were more weighted in the ‘could be better’ camp, wishing the information boxes were a tiny bit smaller. They said things like it felt very close to their faces. Some went so far as to gesture, pushing their hands away from their faces when they described it, others moved their bodies back to illustrate wanting to back away from it. Things that are “big” (in scale) feel close — like they are invading your space a bit (close talking).

“Reading in VR is cumbersome. I have to move my head around a lot.”

However, to make the text big enough to read, it needs to be fairly large. To this participant, it felt zoomed in, which caused the head movements he described. Since, in our experience using 360° photos, there was no way for the participants to move along Z axis, we had to find that sweet spot between big enough and not too big.

Clarity/Focus

Regarding clarity/focus, most participants commented that while the text was clear, the graphics did not feel like they were in focus—that they seemed “pixelated.” Headset positioning, in particular with glasses wearers, also impacted focus. If glasses-wearing participants were looking above their glasses, images became blurry. If the headset wasn’t positioned properly on the face this also led to clarity issues. A slight repositioning generally resolved the issue of blurriness around edges (blurriness that didn’t correspond to image quality/pixelation).

Interaction feedback

Users of the Google Cardboard were unsure of when they initiated interaction with a target. We noticed participants making exaggerated head movements to avoid accidental clicks. Providing better visual feedback, perhaps outlining/highlighting or enlarging the interactive target, would inform the users when they have initiated an interaction and free them from worries of accidental “clicks”.

There is also the option to provide audio feedback when users are hovering over an active area. This can be a tricky choice as it would need to be obvious enough for users to realize what is happening without becoming annoying with repetition.

Photo changes

There needs to be alignment between the preview icon and the photo that it links to. The content within the preview needs to be distinct and memorable for the user to understand and recognize the location, but once the new location is loaded the initial view should also align with the user expectations - confirming that they arrived at the expected location.

"The fact that I was coming from the home screen where I was clicking on something that said Mike's office, then I immediately saw something that said Suzy's office threw me off."

Teleportation

For our experience, we established links between photographs. When a participant fused on a link, it would display the next photograph of the new location. Our participants were nearly evenly divided about their thoughts on how this transition method of teleportation from photo to photo.

Five of the participants commented that it was “OK”, standard, or was not disorienting. However, for four other participants it did not meet with their expectations. The general consensus of the group was that more steps were needed in the transition. One participant mentioned wanting to be able to see where she came from by leaving the door open.

"I thought the door was going to actually slide open or something, and there was going to be some kind of-I didn't know I was just going to pop into another room. That was kind of surprising."

"I was expecting this door would be open so I can see where I went from to. Just to get a feeling that, ‘oh, I was over there, but now I'm over here.’ As opposed to ‘you're in a completely different room, but it's not necessarily related to that room over there.’ It would be helpful if the door was left open or something in the picture to show that's where you came from. Give you context of how do I get back there? I like to have context."

User’s point of view

Keep the user’s point of view consistent from photo to photo. If the buttons are on the bottom of the screen, then there should be something there for the user to look at when they enter the next photo. It would be disorienting to the user to change the point of view by re-orienting to center.

Wayfinding

It became clear that an area that needed work was orienting the participant as they moved from photo to photo. A certain amount of mystery and surprise can be part of the fun of an immersive experience, but in many cases, participants ventured into confusion and disorientation. When asked what might help them understand where they are and where they could go, we received the following suggestions:

• Seeing the room from the outside before entering

• Seeing where they were (i.e. open door indicating what you just came out of)

  • These two work well when the scenes are adjacent to each other.

• Labeling the current location -- providing user with a sense of “where I am”

• Preview image for the next location target (Version B)

• Persistent map or navigation tool with all of the options—suggested that it would only work for a small number of options

• Starting at the home screen (Version B)

  • The home screen provides the user with a sense of the whole and reassurances that they can always start over.

For our experience, we switched from one employee’s home to another’s home, teleporting between homes. Additional research needs to be done on these suggestions to best provide context between photos when they are not physically next to each other.

Home Screen

Version B of the WebVR experience starts at a home screen that presented the participant with the four location options to choose from, Mike’s studio, Mike’s office, Suzy’s office, and Suzy’s backyard.

For Version B, we had comments such as:

"Four choices to explore, which I'm guessing I can just push a button, on any four."

"I have home screen that have four different panes and so I either go to Suzy's backyard. … I get the where four different locations that you can go to."

"It looks like you can just click on a square and it goes. It goes to where it says…."

"This is a nice welcome home page. Like--this is like your desktop, … it's a comforting picture, and then you decide where you want to go. It makes me feel like a home page. and I decide where I want to go from here. I like that. That's cool. It's better than just starting off in Mike's workshop immediately. I can kind of choose your own adventure--that's one of those books."

The participants in Version B were able to anticipate what the experience would entail, perhaps how long it would take to view, and decide which location they wanted to visit first. Having the home screen gave the participants a sense of control for their experience.

Preferences

Three of our participants preferred Version A. Five preferred Version B with the home screen. The reasons for their preferences were the enlightening.

Those participants the preferred Version A seemed to like the unknown and unexpected:

"I liked the first one. I like the minimal directions and you had to feel your way around and figure out what everything does rather than things being labeled and told exactly what is going to happen. I liked the mystery of it."

"I think I prefer the first. I think just the element of surprise of like oh I'm going to walk here, and I don't know what it's going to be, versus like oh it's Mike's workshop."

Participants that preferred Version B liked the context that it provided them and the ability to quickly get back to the home screen:

"I liked starting out in the home screen. It's more intuitive. I think even most systems have some kind of general, "Here's where you're at. Here are some places you can go. Here's home base."

"I think the second because it was more clear where you were going to go if you were going to go out of - Yes, the main difficulty I had was with the computer screens and taking you to a different house. And you didn't know it was a different house that you explore more."

Layout

When designing the home screen be sure to have enough blank space between the location options for Google Cardboard users to “rest” the focal reticle. We had one participant that accidentally fused with locations when reading the names for locations. He compensated by keeping the focal reticle between the location targets and just moving his eyes within the headset to read the different options.

"In order to look at these, I guess, I'd have to focus on the middle, between them, and then look right or left without moving my head, which feels a little unnatural, because I don't want to hover on my--because that changed way to fast."

The background of the home screen should have a fairly clean space to place the location options. Keep this in mind when you are taking a 360° photo to use as the background of the home screen. You could also apply print design techniques that subdue the background in order to make the location options stand out. Lastly consider the depth of the location options against your background image. For our experience, we placed the location options against the sky of a landscape photo. The difference in depth between the location options and the sky background upset one of our participants.

"The images are right in your face compared to the background or what it is. ... It just took a while to get used to the square being so much closer than the scene."

If photos are used for the location options, those photos should be taken with a standard, non-360° camera with compositions designed for that purpose. Our participants found our location photos too dark, too small, and lacking the detail. It was difficult to distinguish what they were looking at. Many participants relied on memory from Version A to discern which location they were looking at. We also used text labels, which is how four participants made the connection that they were looking at two different rooms from two different Four Kitchens employees.

"So, since I was just there I recognize the green jacket and the posture. I know where that's going. But maybe if it were a bigger Mike had here or something. Or a picture of something in a workshop, because the big picture of the workshop here is pretty low res."

Home button

During our research on VR interface design, we saw in experiences and about read in articles to place the button to return to the home screen towards the bottom of the photo. We questioned if this was a natural expectation and if this would become the convention or best practice for navigating back to the home screen. In Version B, we set up a button to return home on each photo towards the bottom of the scene in order to test this pattern. Seven of the nine participants understood that the home icon would take them to the home screen, and remarked that they were surprised when they found it. Five participants commented that the ground was an out of the way place for the home button (particularly for 360° photos which tend to be busy photos), but that they might miss it because they might not look that far down. Two participants didn’t like having the home button on the ground, but were unsure where they they would put the home button. One participant suggested using the minus button on the controller as a way to return home. In general there was more negative association with having the home button on the ground. A design pattern still yet needs to emerge for this.

"I wasn't expecting it. And I just looked down earlier and totally missed it. … If I had done some virtual reality and that was the normal thing? Then I'd be, "OK, I'm done here. I wanna go see where the home part is."

Head tilt for menu

One participant suggested that instead of using a home screen for navigational options, place the menu towards the bottom of each photo especially when the number of options is minimal. In thinking about this while analyzing the results of the study, we asked more questions. If we use a menu, would it block points of interest for the user within the 360° photo? Where should be put the menu? How far down should the user need to look to see it? What if they are looking at a different area of the 360° photo and miss it? For our experience we had the target for the home screen anchored to a single place for each photo. One participant asked if there was a home target in each photo - for she did not notice all of them. Having the home target in only one place meant that the user needed to search for it. It might be more accessible to have the home target appear whenever the user tilted their head down below a certain angle no matter where in the photo they are looking. Additional research should be done to see if this is intuitive for users to find the menu or if users will have difficulties making the menu visible.

"I can imagine a-maybe a permanent home area down here, with those four areas. Because when you only have four items, it seems kind of a lot to have to go back to a screen, to navigate out of them. Because if I am one of four, why can't I just go to one of the other three?"

360° photos

During the background interview, we asked participants about their prior VR experience. One participant mentioned viewing 360° videos prior to this experience. He mentioned that he could look all around himself but the movement of the video was predetermined when it was recorded. Another participant mentioned that she viewed an experience that either had 360° photography or realistic illustrations. Other than these similar experiences, 360° photos were new for our participants.

Seven out of the nine participants brought up the quality of the 360° photos during their experience. Only two participants from the youngest age group, who also did not need any eye correction, commented that the photos were focused. One even commented that they were able to see enough detail to determine that an aircraft on the top bookshelf was made out of Legos.

But for those participants who needed a form of eye correction or were in any of the age groups of 35 years or older commented that the photos were not as in focus or crisp as they would like them to be.

“The text [in the informational displays that were added on top of the photographs] looks good but everything else has a low resolution look to it.” or “There's a bookshelf. This is a little blurry. I couldn't tell you what books are on the shelf. It looks like the photos could have been focused in a little more. There's nothing in particular; it's just the whole experience is sort of like walking around without glasses on. I guess is the best way that I could describe it.”

Items that are important for the content and story of the experience should be within 3 feet of the camera. This ensures the item will be in focus and have greater clarity and detail for the user. In most situations having the object slightly below lens height will feel like the user is viewing an object in front of them.

Lighting

One participant commented about the uneven lighting in the room, in which a window was blowing out that area and making it very bright to look at. Getting even lighting within a 360° photo is extremely hard. Usually the space doesn’t have equal light source on all sides, including outdoors. Using additional lighting sources to brighten interior shots is tricky, for the lighting equipment will be within the photo producing a hotspot.

It is best to use natural light as much as possible for both outdoor and indoor photos. You might need to use screens to filter hotspots and/or reflectors to move light into darker areas. Areas or objects that are important for the content or story of the experience could be supplemented with additional lighting. This will tell where the user should focus most of their attention. It will also brighten those objects so that the user is able to see details about the object.

Height of camera

One participant noticed that the camera height seem too high. She commented:

“At first, it seems like, why am I on the ceiling looking down? ... Feels like I’m taller than human height.”

Just like regular photography, it is better to have the lens of the camera at chest-height for when you are taking full length photos. When you view photos taken at eye-height there is a shortening of the perspective for everything that is below waist level. The results can make individuals feel as if they are taller.

Moving within photos

Two participants wanted to be able to move within a photo. For example, one participant questioned if he would be able to use the trackpad area of the Daydream controller to “zoom into” a portion of the photo in order to see more detail and/or read smaller type. Another participant expressed wanting to get closer to read the titles of books and games on the shelves in Mike’s office.

Take multiple photos of each scene, as if participants are walking through the space. Designers can connect these photos with targets allowing the user to move up to objects.

Allow the user to click on an image for a zoomed in view. In order to do this, take high-detailed, well-lit, focused photos (non-360° photos) of objects within the scene. You can use these to showcase objects within the photo, by allowing the user to view it closer and/or provide additional information about the object. It is hard to predict what objects will pique the interest of the users, but if you have these resources on hand you have provide more options for what the users are able to explore in more detail. Through showcasing certain objects in the scene with more detailed photos and additional information, you can tell a story or informed the user about the location or a character within the experience. As one participant explained:

“I was just playing this video game the other day called Uncharted, and at the beginning of the game, you start at the main character's house, and you explore his whole house, and it does a great job of explaining and giving depth to your character without giving any dialogue, narrative, or anything. You just go to a bookshelf and you find some piece from the past, and you look at it, and inspect it, and make your own assumptions about what it means. And it was the coolest way of learning about someone.”

Storytelling

Conveying a storyline using only photography is difficult. For the experience that we built there was an assumption that the viewer would connect that each employee showcased two photos from their home office. Mike's photos included his studio and office, whereas Suzy's photos included her office and backyard. One participant mentioned that they didn’t realize that the studio and office belonged to Mike until they saw Version B which the photos are labeled on the home screen. It is important to provide the user with the context to understand the connections and storyline of the experience. This is also a good reason to get feedback from people who aren't intimately involved with the project. When we are working closely with a project we take things for granted, overlooking these obvious things.

Photographing people

Only one participant brought up viewing people in 360° experiences, though he had several thoughts on the topic. He compared the photo of Suzy’s backyard with six people in the photo. Some of the people are interacting with each other, while others are looking at the camera. But those that are looking at the camera are more than 3 feet from the lens. These people do not feel to close or invading the personal space of the viewer. It felt like a more natural distance you would keep when you are in a public space with people you don’t know.

Whereas the photo of Mike’s studio, Mike is standing directly opposite of the participants on the other side of the table and looking directly at them. Is posture has him leaning on the table and forwards to the camera. This posture feels a bit overbearing and intruding into the participants' personal space. (This is contradictory to Mike’s own personality, perhaps he was camera shy.) Below are the participant’s comments about both photos.

"I think it's good. I thought you guys got pretty natural--I was--to get six people in Susie's backyard, just taking control of ones at different positions, and Mike pointing at the beard guy with no hair on his head is hard to set up and not have people look awkward and unnatural, but everyone looked pretty natural in there."

"Mike was a little too close for my comfort in his workshop. He's just staring right at me the whole time. No one really does that in real life. I feel like I'd feel more comfortable if he's just doing his own thing, just focusing on his work and not staring at me. That's why I feel like Susie's backyard is a lot more--less upfront, the subjects in it. [Additional comment later on in the session.] I'd rather [Mike] be just looking at his work as he's doing it. It would make me feel like I'm more here, just hanging out with Mike."

Another suggestion is to have the people in the photo interacting with objects and act as if they are not aware of the camera. Instead of a snapshot style where people are looking at the camera and smiling, catch a moment of their private lives - perhaps they are concentrating on working with an object, reading a book, or typing on the computer. Nothing too personal, but present in the space with a sense that the person in the photo could be interrupted to have a conversation.

The same individual commented that when he visited Mike’s and Suzy’s offices he felt that he was stepping into a day in the life of this person. Since the offices were photographed without any people within them, it was an opportunity to be nosy in their offices and look around without getting caught. This trade off of photos without people, contributed to the participant’s ability to relax in the experience. It allow the viewers an opportunity to explore without being self-conscious.

"The offices were cool. There's no one in there, so I felt like I was them. I'm sure they cleaned them up a little bit and put things in perfect little order, and make it look Hollywood-like, like a set. But that's fine."