SenseBox

Interaction demo — how the space reads body position, posture and expression, and responds.

A Multimodal Computation Framework for Behavioral Recognition and Interaction for Mixed Reality Workspace

A multilevel computing framework for behavior recognition and interaction in mixed reality workspaces Paper

Simulating Interactive Architecture: A Behaviour-Based Interactive Design Platform Paper

Construction

Framework Design: The workspace box (3m x 3m x 2.5m) features a spiral tri-ring design with a core steel pipe framework. Secondary lines are added and secured with bundling straps, then rotated 180 degrees to create a symmetrical structure, forming the final plan and renderings.

Bundling and Node Development: Various bundling methods were explored, resulting in specific bundled nodes for the final design. Each pipe was bent, numbered, and bundled in sequence to form six main components.

Construction and Assembly: The assembly process involved bundling parts together, installing depth cameras, and attaching acrylic seat and table surfaces, completing the Sense Box construction.

Interaction

Utilized mixed reality technology to assist in bending steel pipes for constructing a 3x3x3m installation space prototype.

Acquired human data using ZED camera and headset data, with body landmark detection and emotion recognition conducted.

Analyzed human data using qualitative and quantitative methods, and formulated rules based on distances, orientations, postures, head states, and expressions to derive interaction, task, attention, and emotional patterns.

Implemented real-time spatial changes based on human data, affecting spatial enclosure, height, openness, regularity, and color, using Grasshopper, Unity, and Python. Integrated with physical spaces to create dynamic virtual environments that adapt to human work states.

Full walkthrough

Construction and interaction from start to finish.

Related — Nebula

Follow-up pavilion built in 2025, where I took part as teaching assistant.