HOME >> English >> HySAR: Hybrid Material Rendering by Projection Mapping and OST-HMD

HySAR: Hybrid Material Rendering by Projection Mapping and OST-HMD

Spatial augmented reality (SAR) pursues realism in rendering materials and objects. To advance this goal, we propose a hybrid SAR (HySAR) that combines a projector with optical see-through head-mounted displays (OST-HMD). In an ordinary SAR scenario with co-located viewers, the viewers perceive the same virtual material on physical surfaces. In general, the material consists of two components: a view-independent (VI) component such as diffuse reflection, and a view-dependent (VD) component such as specular reflection. The VI component is static over viewpoints, whereas the VD should change for each viewpoint even if a projector can simulate only one viewpoint at one time. In HySAR, a projector only renders the static VI components. In addition, the OST-HMD renders the dynamic VD components according to the viewer’s current viewpoint. Unlike conventional SAR, the HySAR concept theoretically allows an unlimited number of co-located viewers to see the correct material over different viewpoints. Furthermore, the combination enhances the total dynamic range, the maximum intensity, and the resolution of perceived materials. With proof-of-concept systems, we demonstrate HySAR both qualitatively and quantitatively with real objects. First, we demonstrate HySAR by rendering synthetic material properties on a real object from different viewpoints. Our quantitative evaluation shows that our system increases the dynamic range by 2.24 times and the maximum intensity by 2.12 times compared to an ordinary SAR system. Second, we replicate the material properties of a real object by SAR and HySAR, and show that HySAR outperforms SAR in rendering VD specular components.

Overview of the hardware setup of the real material experiment. Studiolights are set for a VI component, and flashlights are set for a VD component. An HMD and a camera are fixed with aluminum frames to capture images from a user-perspective viewpoint. We prepare a piece of pottery as a real reference object to extract textures and another similar piece of pottery with white matte paint to be projected as a diffuse object.
Captured images from two viewpoints in different setups.

Publications

  • Takumi Hamasaki, Yuta Itoh, Yuichi Hiroi, Daisuke Iwai, and Maki Sugimoto, "HySAR: Hybrid Material Rendering by an Optical See-Through Head-Mounted Display with Spatial Augmented Reality Projection," IEEE Transactions on Visualization and Computer Graphics (Proceedings of IEEE Conference on Virtual Reality and 3D User Interfaces), Vol. 24, No. 4, pp. 1457-1466, 2018. [pdf]
  • Yuichi Hiroi, Yuta Itoh, Takumi Hamasaki, Daisuke Iwai, and Maki Sugimoto, "HySAR: Hybrid Material Rendering by an Optical See-Through Head-Mounted Display with Spatial Augmented Reality Projection," In Proceedings of IEEE Virtual Reality, pp. 209-210, 2017.