Elena Xinyue Peng

AR Creative Technologist

Elena Xinyue Peng is a creative technologist and AR developer based in China and New York. She has interests in immersive installation, AR/VR/MR, illustration and feminism. She enjoys challenges and solving coding problems. She values inclusive and equitable design.

Thesis Faculty

Melanie Crean,Ben Norskov,Ernesto Klar,Xin Xin,Or Zubalsky,Charles Huang

Monster Zone

An Augmented Reality Two-Player Collaborative Game

Monster Zone is a AR two-player collaborative iOS mobile game that uses hand gesture control. It explores an intuitive control in AR gameplay and seeks the key moments in human relationship. The unseen combination of hand gesture, AR gameplay and two-player collaboration fosters an unexpected fun experience for both players and even the viewers.

As multimedia technology advances in Mixed Reality, there is an increasing demand for creative explorations in user experience and human-computer interaction. The intent for this project is to challenge people with new ways of interacting with the Mixed Reality world and bring people together in a fun two-player game experience.

I believe the most intuitive way for people to interact with the world is using their hands . Using hands to interact with objects can break the boundaries between screen and reality. There is opportunity to create a more natural interaction in AR experience because the interaction in most current AR applications stays on screen.

Even though some MR products like Oculus could allow the user to do hand gesture control, those functions are bound to the product itself. I want to develop a Mixed Reality experience with a reasonable user interface that does not require any specialized devices. I also want to study the impact of collaborative and competitive gameplay on interpersonal relationships in the media of mixed reality as I want to find out how people can restore or strengthen their current relationship through gameplay.

Game Background

Everyone has potential to connect with the Monster Zone. However, only few can connect with their inner Monsters. Everyone’s Monster is unique with different colors and different forms. Each Monster is alone in its Monster Zone. The only way to connect with other’s inner Monsters is through the magical device in your hand, Monster Zone Portal.

Enter your name and see if the Monster Zone has chosen you!

In order to connect with other’s Monsters, the first step is to see if you are qualified to enter the Monster World. As I said, not everyone has the ability to connect.

Congrats! You are permitted to enter the Monster World. You can either create your own unique Zone or you can enter other’s Zone to connect.

In order for two Monsters to connect, you will need to collaborate and reach the end of the road. There, you can enter each other’s space and get a unique color from each other.

Symbolism

The Monster Zone symbolize each person’s inner world. The Monsters with unique color combinations and shapes represent individual’s thoughts and values. The gameplay that two player go through is the process of trying to understanding each other and discovering new viewpoints. This process might be fun, frustrating, exciting and sometimes stressful. However, at the end of this process, two players will form a stronger bond and exchange a piece of their inner world as a gift from this journey. The unique and precious gameplay experience that player spend with the other individual will become part of you, stored in your memory.

Design Iterations

Early iteration of gesture control system

According to my user testings, many people will like to keep running while rotating. This behavior will cause a problem: when the phone is slightly tilting, usually the pointing gesture will be partially cut in the camera. ManoMotion will need a full image of the hand to detect the gesture. This causes a lot of unexpected behavior for the character. The outcome of this round of user testing shows that using a phone as a controller is not an ideal way for directional control because the phone already has the most essential role, the viewport.

Final gesture control system : Grab, Open, Fist

Final direction control : Rotate left and right using index finger

Inspired by oculus gesture control, I decided to use the pointing gesture to rotate the monsters. Open hand gestures under traffic context already have a stop meaning. I set its opposite gesture, close hand, as the movement gesture. Many of my users like this new system much better and they commented that this is easier to control.