Abstract ideation to immersive, multi-medium creation/co-creation

Imagine designing a service robot meant to comfort elderly residents, but all you have are paper sketches and static storyboards. How can users truly judge whether the robot feels trustworthy or intrusive? What about its subtle interactive behaviors or hidden attributes?

Such challenges highlight a core difficulty in design—especially in its early stages—where capturing dynamic, emotional, and sensory experiences remains nearly impossible through traditional means. As a result, meaningful user involvement is often delayed until functional prototypes are built, limiting opportunities to influence the design when it is still flexible and evolving. In typical product development cycles, users are invited mainly to validate ideas after key decisions have already been made. This late-stage involvement restricts creativity, narrows feedback to superficial observations, and risks overlooking deeper insights that only surface through lived experience. 

Among several technological advances, immersive technologies—Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), collectively known as XR—are offering a new way forward. With recent improvements in accessibility and usability, XR technologies are now increasingly finding their place in early-stage design, opening the door for users to experience, influence, and shape ideas far earlier than before. Rather than imagining hypothetical experiences, users can step inside evolving designs virtually, experiencing interactions firsthand.

Until recently, user involvement was largely about capturing the "voice of the customer" through surveys, interviews, or the occasional feedback session on static prototypes. Early VR-based approaches mainly extended these practices, offering passive immersion without real agency. Interaction was often observational—interesting but ultimately limited. Today, thanks to easier XR integration, users can engage directly with evolving concepts, influence interactive behaviors, and refine experiences dynamically. Remote participation has also become seamless, enabling geographically dispersed users to contribute meaningfully across different design stages. Embedding co-creation across these stages means involving users at every critical point: not just visualizing ideas for them, but inviting them to help build, live, and refine them. What follows is a look at how XR is reshaping user involvement at different stages of the design journey.

Experiencing Possibilities Early

Experience prototyping has long played a crucial role in early-stage design, relying on sketches, mockups, and rough prototypes to help users imagine how a product might behave [1]. Yet envisioning multi-sensory, dynamic interactions through static visuals and mockups is inherently limiting. This is where VR steps in. In relatively simple use cases, such as product design, users can interact with virtual prototypes—table lamps that glow differently based on proximity, or alarm clocks that respond to touch or gestures. In more complex scenarios like architectural design, users can walk through virtual spaces, adjusting layouts, experiencing scale, and sensing emotional resonance in real time. By experiencing possibilities rather than imagining them, users move from abstract ideation to visceral engagement. They don't just suggest features; they react emotionally, physically, and intuitively—leading to richer and more grounded design directions.

Co-creating Design and even Behaviors: Directly and Passively

Traditionally, designing how a product behaves—how it moves, reacts, responds—has been the domain of late-stage, expert-driven prototyping. XR, especially MR, is shifting that balance. In mixed-reality environments, users can co-create behaviors in two key ways:

• Active involvement, where users directly modify product interactions virtually, adjust response timings, and build dynamic behaviors using intuitive modules or catalogs (like "shake," "vibrate," "light up") [2].

• Passive collaboration, where users work alongside designers—experiencing prototypes live and offering real-time feedback or suggestions during design sessions [3].

Even small degrees of control—like allowing users to select from pre-prepared interaction building blocks—can significantly enhance immersion, agency, and creative contribution. Here, users evolve from idea testers to behavior architects, shaping how products feel, respond, and connect with people on a deeper level.

Refining Designs Through Embodied Feedback

Traditional revision cycles often rely on notes, surveys, or interviews—second hand impressions of an experience. XR closes that distance. By immersing users directly inside evolving designs, XR allows feedback to emerge from lived, felt experience. Users can adjust behaviors, fine-tune emotional expressions, and even simulate real-world lifecycles—how a design might behave over time, under different contexts, or across environmental changes. Revision becomes more than critique; it becomes evolution. Instead of discarding ideas wholesale, users help iterate designs with greater nuance, sensitivity, and authenticity. XR also makes iteration faster and more fluid—an experiential feedback loop that encourages exploration and discovery, not just correction.

Final Thoughts: Toward a More Collaborative Future

XR is doing more than offering designers better visualization tools. It’s changing the ethos of design itself. By embedding co-creation across early ideation, behavior development, and iterative refinement, XR gives users agency not just to comment on designs but to help shape them. It invites users to be creative partners in envisioning what products can feel like, not merely verifying what has already been decided. As XR technologies become increasingly accessible—particularly with the spread of consumer-grade VR headsets, AR apps, and cloud-based collaboration—the real opportunity lies ahead, expanding participation beyond the technically skilled or privileged few. The future of design will belong to those who create with users, not just for them—experientially, dynamically, and continuously. In this evolving landscape, immersive technologies offer a bridge to a more human-centered, collaborative design future.