Spatial Computing UX Research and Design

Researched interaction models and established principles

Before defining new spatial interactions, we looked at existing models from desktop, mobile, human-computer interaction, and related interface research to understand which principles might still apply in three-dimensional environments.

This included examining established interaction rules such as Fitts's Law and considering how factors like target size, distance, movement, visibility, and physical effort might change when interaction moves away from a flat screen and into physical space.

The goal was not to assume that familiar interface rules would transfer directly. We used them as a framework for identifying which principles remained useful, which needed reinterpretation, and where spatial computing introduced different constraints.

Surveyed existing AR and VR environments

We conducted a broad review of existing spatial computing environments across multiple AR and VR platforms.

We gathered screenshots, notes, interaction examples, and observations to compare how different systems handled application management, object selection, menus, navigation, positioning, visibility, and other spatial behaviors.

This helped us identify recurring patterns, inconsistencies between platforms, and areas where the industry had not yet converged on a clear interaction model.

Interviewed subject matter experts

We interviewed people working in AR and VR to understand current challenges, likely platform directions, and unresolved UX problems.

These conversations helped ground the work in real technical and interaction constraints rather than treating spatial computing as a purely speculative design space.

Adapted the research approach for remote testing

The original plan involved in-person testing of interactive VR prototypes. When COVID-19 made that impractical, we redesigned the research approach rather than abandoning prototype evaluation.

We used video calls to stream the live video output from the headset while I interacted with the prototype in real time. Participants could watch the spatial experience unfold on screen, ask questions, and react to proposed interactions without needing to be physically present or use the headset themselves.

We also created recorded prototype videos with voiceover so concepts could be reviewed asynchronously by people who were not available for live sessions.

This approach preserved enough spatial and interaction context to continue discussing and evaluating the concepts remotely, while also making the work accessible to participants without direct access to VR hardware.

Explored physical and spatial interaction

We sketched interactions, discussed them as a team, and often acted them out physically.

This helped reveal which gestures or movements might feel natural, awkward, tiring, difficult to repeat, or hard to understand in a three-dimensional environment.

It also helped connect the earlier interaction-model research to the physical realities of using spatial interfaces rather than treating interaction principles as abstract rules.

Created live streamed and recorded VR prototype walkthroughs

I used tools including Microsoft Maquette and Unity to create prototype scenes that could be experienced, demonstrated, and recorded in VR.

During remote sessions, I streamed the headset's live video output through video calls while directly interacting with the prototype. This allowed participants to see proposed application-management behaviors in context rather than relying on static screens or verbal descriptions alone.

The live walkthroughs turned interactions that were difficult to explain into observable spatial behaviors. Participants could watch the prototype respond in real time, ask questions, and react to the interaction model as it was demonstrated.

Recorded video prototypes extended that work beyond scheduled sessions, making the concepts easier to review asynchronously and giving the team a reusable way to communicate spatial behavior.