Chef, Shark and Melody

Interactive System with Real-Time Gesture Recognition

Role

Interactive Systems Designer

Tool

Unity, Processing, WebSocket, Teachable Machine

Team

Becky Wang, Sijia Ma

Project Overview

Chef, Shark and Melody is an interactive system that uses gesture recognition powered by machine learning to create real-time, responsive experiences. Built for playful, practical scenarios, it explores how human-computer interaction can feel intuitive, connected, and responsive.

Harvard SCI 6338 Introduction to Computational Design

#interactive_design #gesture_recognition #real_time_connectivity #machine_learning #human_computer_interaction #practical_applications #computational_design

Sep 2024 - Dec 2024

The Possibility of Gestures

Inspired by James J. Gibson’s theory of affordance—“The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill”—this project explores the potential of intangible interaction. By training hand gestures through Teachable Machine and applying them in Unity, users can trigger diverse interactive scenarios. Each gesture serves as a rethinking of conventional input, encouraging users to discover new meanings and functions behind familiar postures within digital environments.