HOME >> English >> DeforMe: Projection-based Nonrigid Deformation Visualization

DeforMe: Projection-based Nonrigid Deformation Visualization

We propose a projection-based mixed reality system that visualizes the tangential deformation of a nonrigid surface by superimposing graphics directly onto the surface by projected imagery. The superimposed graphics is deformed according to the surface deformation. To achieve this goal, we develop a computer vision technique that estimates the tangential deformation by measuring the frame-by-frame movement of an infrared (IR) texture on the surface. IR ink, which can be captured by an IR camera under IR light but is invisible to the human eye, is used to provide surface texture. Consequently, the textures do not degrade the image quality of the augmented graphics. The proposed technique measures surface motion between two successive frames individually. Therefore, it does not suffer from occlusions caused by interactions and allows various interactions (e.g., touching, pushing, pulling, and pinching). The moving least squares (MLS) technique interpolates the measured result to estimate denser surface deformation. The proposed method relies only on the apparent motion measurement; thus, it is not limited to a specific deformation characteristic, but is flexible for multiple deformable materials, such as viscoelastic and elastic materials. Experiments confirm that the proposed method can visualize surface deformation of various materials by projected illumination even when a user's hand occludes the surface from the camera.

Publications

  • Parinya Punpongsanon, Daisuke Iwai, and Kosuke Sato, "Projection-based Visualization of Tangential Deformation of Nonrigid Surface by Deformation Estimation Using Infrared Texture," Virtual Reality, Vol. 19, No. 1, pp. 45-56, 2015.
  • Parinya Punpongsanon, Daisuke Iwai, and Kosuke Sato, "DeforMe: Projection-based Visualization of Deformable Surfaces using Invisible Textures," In Proceedings of SIGGRAPH Asia Emerging Technologies, pp. 8:1-8:3, 2013.
  • Parinya Punpongsanon, Daisuke Iwai, and Kosuke Sato, "Projection-based Mixed Reality for Deformable Surfaces," Korea-Japan Workshop on Mixed Reality, 2013. (No publication) (Best Presentation Award)
  • Parinya Punpongsanon, Daisuke Iwai, and Kosuke Sato, "Projection-based Visualization of Viscoelastic Surfaces Deformation using Infrared Vision Technique," In Proceeding of Thailand-Japan International Academic Conference, pp.172-173, 2013.