Researchers have found a way to use these subtle light reflections on the human eye to see and reconstruct the scene that the person is observing. The study is out of the University of Maryland, and it shows how the team responsible for the research captured multiple views of a scene by imaging the eyes of a moving person.
However, the process is complex and it isn’t without its challenges. One of these is cornea localization; identifying the precise location and orientation of the cornea can be a noisy and inaccurate process, potentially hindering the efficacy of the 3D reconstruction.
Another challenge is related to the complexity of the iris textures. The iris of the eye has a unique texture that could make interpreting the reflections complicated. Lastly, the researchers noted that the reflections captured in each image tend to be low resolution, which could be another obstacle to detailed reconstruction.
To address these issues, the researchers proposed several novel approaches. The first of these is cornea pose optimization, which is a process that corrects for the noise in identifying the cornea’s position and orientation. The second approach is iris texture decomposition. This could be a method that simplifies or interprets the complex textures of the iris, making it easier to understand the reflections. Finally, they introduced a radial texture regularization loss, which appears to be an error metric that aids their model in better understanding the textures of the human eye iris.
One of the most significant advantages of this method is that it doesn’t require a moving camera to reconstruct the 3D scene. Instead, it relies on the motion of the person whose eye reflections are being used. The researchers have indicated that their method has been effective with real-world data.
The researchers aim to create new possibilities in 3D scene reconstruction by using accidental or unexpected visual signals from the reflections in a person’s eyes. They believe their research has the potential to broaden the field of 3D reconstruction, and they hope their work will inspire more exploration in this area.
Alzayer, H., Zhang, K., Feng, B., Metzler, C., & Huang, J.-B. (2023). Seeing the world through your eyes. https://doi.org/10.48550/arXiv.2306.09348