The Chinatown Intention

An AR experience of a digital artifact about the NYC Chinatown ideas and aesthetics

Abstract

The project is about an augmented reality installation project which illustrates my interpretation of the status of New York City Chinatown: Chinatown is suffering from culture deforestation. The installation is a digital sculpture viewed through a mobile phone with the AR app I build. The digital contents of the sculpture are replicas of real-world objects and environments. LiDAR scan and photogrammetry technologies are used here to help achieve the building of all the digital assets. The use of AR technology to create this mixed reality experience is a deconstruction and reconstruction of the current Chinatown environment, but also to provoke people to think about the future of Chinatown community.

From my research and design outcomes, the project aims to push local NYC residents, especially the Chinese community start thinking about their community and culture, and what place Chinatown ideas and aesthetics could have in a future Metaverse.

Introduction

This project is based on my study about the NYC Chinatown community, and how the Metaverse technology being used to transfer the idea into a digital structure based on my interpretation. We are living in an era that is progressively developing from a real society to a virtual society. People are seeking to create their own worlds and unique experiences in virtual environments. Metaverse and virtual society will bring new ways of experiencing. Virtual reality technology will also provide new methods of illustration ideas. The diagram below shows the research and design logic of this project:

I chose Chinatown as my cultural research subject. I defined the New York City Chinatown’s status through research, site visiting, and analysis. I also investigated the concept of the Metaverse and identified its core feature – how the concept of decentralization is a way of creating personalized experiences that can be shared with the virtual community. In my following design, I chose to use augmented reality technology to present my vision of the Chinatown community as a 3D sculpture. The core idea of the final work is how the real-world Chinatown ideas and aesthetics are preserved in the form of digital sculpture through my unique interpretation.

Chinatown Impression

There are many similarities in people’s memories of a particular scene or environment. These similar feelings come from the sensory intention that the spatial environment brings to people. The concept of the urban intention comes from Kevin Lynch, which refers to the formal characteristics of the urban environment in the eyes of the public. In other words, whether a city is lined with tall buildings or blended with mountains and rivers, whether the urban buildings are full of regional characteristics or with a modern style… These are the intuitive impressions left by the city. Everyone has different impressions of the same city. For example, a white-collar worker working in Manhattan will think that there is a symbol of vividness, power, and greatness from the skyline, but homeless people might think it feels oppressive and crowded. Therefore, the impression of a community, and further of a city, must be a superposition of many individual impressions, and it is the public’s collective impression of the environment.

To learn about the current Chinatown is of course the major thing in this research since the project is to present the the core of the NYC Chinatown and how its cultural history make it achieved its current form. I conducted several site visiting in the Chinatown area, observing the environment and people there. My goal was to have a general understanding about what the Manhattan Chinatown is like today as how is it like, what do people do there and what is the relationship of it and NYC.

Diagram below is a collage I made about some interesting elements and objects that I observed in Chinatown. I explained each of the objects in the mapping about where they come from and how does it relate to Chinese culture. These visual elements are recognizable in Chinatown culture. They have a long history, and never changed or being dumped in the culture of Chinatown. They are part of the Chinatown.

Other than space and visual environment, human behavior is another important part in Chinatown culture. What people do, how do they live, their certain habits also take a big part in how others feel about the group. In this case, I also conducted site visits focusing on observing human behaviors. I made a behavior mapping based on my observation and analysis:

I found that the best assets to present Chinatown are the materials that exist in Chinatown. I can distill the main concepts of Chinatown and use these most intuitive visual elements to present my perspective of Chinatown. This idea is the bridge of how I combine Chinatown culture ideas with the concept of metaverse creation.

A Future Towards Metaverse

The so-called virtual world of “The Second Life”, especially the virtual world in a three-dimensional environment, is the most attractive concept created by digital Internet technology. The emergence and development of various information technologies in the new era have made people increasingly look forward to and committed to building a fully digital virtual world. Film, literature, and entertainment works in the field of metaverse are also emerging people aspire to travel in the Oasis in Ready Player One, want to explore the game world like the Sword Art Online, hope to become a member of the cyberpunk world and consider the possibilities of The Matrix again and again.

The charm and temptation of the virtual world are endless, because only in it can we satisfy all the inability of reality. The pursuits of human beings could be glimpsed in the creation of virtual worlds because we can become the creators of the world. The concept of Metaverse has also been pushed to the forefront with technological innovation and human hedonistic consciousness as the game world gradually strengthens.

Figure below is a timeline diagram I made about how the idea of Metaverse has been developed from the first multiplayer game till today, with the last two as my prediction of where it is heading. The application of Metaverse originated from virtual games but will eventually surpass games. Since the birth of the virtual world, people are no longer satisfied with the game in reality, but dream of entering the splendid virtual world.

The development of Metaverse will our society and lifestyle. However, Metaverse technologies bring new ways of creation to us. My goal is to use mixed reality technology to illustrate my vision of the Chinatown society.

Design Exploration

I started building my actual design work since I have done decent enough research for both of the main concepts of my project. I first need to build a collection of digital model database of the replicas from the real Chinatown environment and use them as the assets to establish my sculpture. Then I need to transfer this digital interpretation in to an AR scene so that it can be view from mobile in different location.

LiDAR Scan

I mainly use LiDAR scan to capture the objects and environment scenes from the real world. Polycam, Meta Scan, Scanniverse and Trnio are the 4 software I use to scan. I then export the scans from the apps as obj formats and use Blender to editing the meshes.

Photogrammetry

Photogrammetry can build really great quality meshes. It is a technology that get 3D information from photos. It processes the overlapping photos of objects, and convert them into 3D models.

For my digital assets, I mainly worked with LiDAR scan since I need to capture most of the scenes and objects in Chinatown. Photogrammetry needs very strict lighting and setups in order to get good quality models.

I bought a buddha head from Chinatown and made a simple setup home to take picture around it. Then I upload the images into Reality Capture, a software for photogrammetry, to build the mesh of the head.

Figure below is the working interface in Reality Capture. I first aligned the photos that I uploaded, and then run the matching function to let the software build meshes based on analyzing the photos. After the mesh is built out, I got the sketch model and could do simple editing such as reduce faces and close holes so the model can be ready to use.

Design Sketch

After I build my digital assets database, I started to brainstorm the form of the sculptures and did some sketch about my ideas based on the models I have.

I also iterate some of my ideas with watercolor drawings.

Digital Artifact Prototype

Based on my sketches, I use my model assets to build the digital sculpture structures. Below are some of the close-up meshes of my prototype: