Hanul-Computer Interaction (HCI)

An Open Relationship With My Machine

Machine Learning + Visual Image Data + Human Pose Estimation

I. Concept

WHAT
Hanul-Computer Interaction (HCI) constructs a digital-physical experience with movement-based interactivity that utilizes machine learning library for human pose detection technology. The web and mobile based sketches let users draw real-time graphic reflections of their bodies on screen.
“Draw With Your Own Bodies!”

WHY
First, this process-oriented work contemplates the collaborative creative practices between the designer and the machine. The designer a.k.a. Hanul and the machine a.k.a. Computer developed an interdependent and intimate partnership while working on multiple machine learning projects. The real-time interaction between the user and the graphic represents the interrelationship between the designer and the machine.
Second, the constructed experience expresses a sense of melancholia from human species’ collective becoming of digital data due to the rapidly advancing facial recognition and body detection technologies. It critiques how such digitized human data are surveilled, censored, and owned by actively rejecting data collection and individual profiling.

HOW
The product utilizes PoseNet tool from ml5.js library to capture real-time body movements of users, based on human body data trained with tensorflow.js. The blurry graphic style is intended for thorough anonymization of participants. The intentional abstraction resists individual profiling and datafication of human image. By playing with the interactive interface, the users organically participate in the critical conversation.
“Let My Critique Be Fun!”

II. Process

Approach
This is a project focused more on process than production. If this project was a painting, its primary purpose would be to study a subject rather than to draw an object. I am not only the creator but more importantly the curator, and my curation becomes a pivotal element of the project.

Research
I reflect on my collaborative working experience with my computer in the light of “Actor-Network Theory” based on autoethnographic and digital-ethnographic methods. The final product is a visual and computational expression of my investigation and contemplation. I am particularly interested in the ideas of agency & authorship that emerge in my intimate relationship with my machine.

III. Product

Design
The interactive interface reflects the designer’s experience of working with the machine. The experience offers the participants with the opportunity to embody such collaborative relations while playing with the product. The users experience and reflect on the direct co-creative interrelationship with the machine through their physical motions. The questions regarding materiality and virtuality arise. How do physical bodies interact with software interfaces, and how are the material movements manifested in the virtual realm?

Technology
The final product utilizes a machine learning model called PoseNet that enables real-time human pose estimation in the browser. The model is created by ml5.js, which is a powerful and accessible resource powered by TensorFlow.js and programmed in p5.js for creative sketching. When the product loads on the web browser, it immediately reads the visual data of users captured on the camera and estimates movements. The ml5.js library is completely open source. It’s been developed by a group of open-minded engineers, educators, and artists who shared the common interest in making machine learning technology accessible for public creativity.

PROJECT WEBSITE