As education moves into immersive environments, new opportunities for enhancing student-instructor communication open up. In the past few years, my research team has been experimenting with an exciting new technology called “immersive gaze sharing”. To explain what it is and how it can support design education, I’ll need to first explain the general idea of “gaze sharing”. Here goes.

Gaze Sharing Explained in One Minute

Gaze sharing enables us to know where other people are looking, in real-time. With gaze sharing, communication is simpler, more effective, and often more enjoyable. Naturally, it can be extremely helpful in educational scenarios.Gaze sharing is possible thanks to advances in eye tracking technology, which resulted in wearable eye trackers that look like a standard pair of glasses. By collecting data about a person’s focus and sending it to another person, we can enable gaze sharing. 

The following image shows an example of gaze sharing in a one-on-one design critique. In this case, the instructor can see where the student is looking, by glancing at a tablet device. The student’s focus is captured by a wearable eye tracker and then sent to a tablet device which the instructor is holding. Whenever the student focuses on something in the room a red circle appears on the instructor’s tablet’s screen, providing him with useful information that can be used for teaching. For instance, the instructor can know if the student is following his explanation or not. How amazing is this?

From Gaze Sharing to Immersive Gaze Sharing

Yes, gaze shearing is indeed amazing. But you may have noticed that there is one major drawback in the technology that we just showed - to know where the student is looking, the instructor must move their eyes to glance at the tablet device. This is quite distracting and hinders the natural flow of conversation. A potential solution to this issue is to get rid of the tablet device altogether, but how do we do that? 

Obviously, we would need to visualize the focus of the student somewhere else. Luckily, Augmented Reality (AR) technology provides an effective solution. Wearable AR glasses enable us to add a virtual layer of information to the world in front of us. Since we can place this information anywhere in space, we can use AR to “mark” the focus of the student using a hologram, which will appear right in front of the instructor in the right location. In this case, the tablet device becomes unnecessary altogether. 

Taking this opportunity, our team developed a next-gen system for sharing gaze information in an immersive environment. Unsurprisingly, this technology is called “immersive gaze sharing”. The example below shows how 

What’s next?

As you can imagine, this new technology is extremely promising as a tool for training our future generation of designers. In our investigation, we discovered that to use it in practical scenarios we must consider the issue of timing - sharing our gaze makes our thoughts transparent, to some extent. Naturally, we do not always want to disclose our focus or interest to the other party, for various reasons. For example, in one of our experiments, an instructor reported that, while using the system, they found it hard to relax during the conversation, since the student was constantly aware of their focus. In other words, future systems must enable instructors and students to determine when and how much of their gaze information should be shared. Controlling this manually sounds inconvenient at best and impractical at worst. This means that we need to find other ways to detect the moment when a person is interested in sharing their gaze. Intriguingly, a possible solution would be to read signals from the instructor or student’s brain, using technologies such as EEG, and interpret them with artificial intelligence. This form of “mind reading” may enable us to put this exciting technology to proper use, without compromising the privacy of its users.