Research

Color Recovery of IR Image for Tele-operated robot

  • Suthawan Jirandorn and Kosuke Sato,“Color Recovery of IR Image for Tele-operated robot“, In proceeding of FrontierLab@Osaka University Program, 2012.

Color Recovery of IR Image for Tele-operated robot   We proposed a novel framework for tele-operated robot to augment visibilty of grayscale image when the robot operates under a dark condition. We extend existing colorization method which colorize images using manually-given color cues. In our method, for automatic acquisition of color cues we use a SLAM algorithm which can detect environmental landmarks from video frames and localize the camera position from the landmarks in real-time. The robot firstly operates in a bright condition, for pre-acquisition of environmental landmarks and their colors as prior knowledge. Then in a dark condition, the feature points corresponding to the landmarks are detected in acquired frame and color marks were given to the frame with the landmarks’ colors. After that, by using colorization algorithm, a fully colorized image was produced. The result proves that the colorized image is similar to the one in reality and the colors are close to the ground truth. This result indicates SLAM algorithms can be employed to give color marks automatically for the colorization method.
  

Return To The Top Page

Real-Time Multiview Recognition of Human Gestures by Distributed Image Processing

  • Toshiyuki Kirishima, Yoshitsugu Manabe, Kosuke Sato, and Kunihiro Chihara, Real-Time Multiview Recognition of Human Gestures by Distributed Image Processing, EURASIP Journal on Image and Video Processing, vol. 2010, Article ID 517861, 13 pages (2010).

Real-Time Multiview Recognition of Human Gestures by Distributed Image Processing
  Since a gesture involves a dynamic and complex motion, multiview observation and recognition are desirable. For the betterrepresentation of gestures, one needs to know, in the first place, from which views a gesture should be observed. Furthermore, itbecomes increasingly important how the recognition results are integrated when larger numbers of camera views are considered.To investigate these problems, we propose a framework under which multiview recognition is carried out, and an integrationscheme by which the recognition results are integrated online and in realtime. For performance evaluation, we use the ViHASi(Virtual Human Action Silhouette) public image database as a benchmark and our Japanese sign language (JSL) image databasethat contains 18 kinds of hand signs. By examining the recognition rates of each gesture for each view, we found gestures thatexhibit view dependency and the gestures that do not. Also, we found that the view dependency itself could vary depending onthe target gesture sets. By integrating the recognition results of different views, our swarm-based integration provides more robustand better recognition performance than individual fixed-view recognition agents.
  

Return To The Top Page

Uncalibrated Synthetic Aperture for Defocus Control

  • Natsumi Kusumoto, Shinsaku Hiura and Kosuke Sato,Uncalibrated Synthetic Aperture for Defocus Control,Proc. CVPR2009, pp. 2552-2559, 2009.

Uncalibrated Synthetic Aperture for Defocus Control
  Exaggerated defocus can not be created with an ordinarycompact digital camera because of its tiny sensor size,so it is hard to take pictures that attract a viewer to themain subject. On the other hand, there are many methodsfor controlling focus and defocus of previously taken pictures.However, most of these methods require purpose-builtequipment such as a camera array to take pictures. Therefore,in this paper, we propose a method to create imagesfocused at any depth with arbitrarily blurred backgroundfrom the set of images taken by a handheld compact digitalcamera moved randomly. Using our method, it is possible toproduce various aesthetic blurs by changing the size, shapeor density of the blur kernel. In addition, we confirm thepotential of our method through a subjective evaluation ofblurred images created by our system.
  

Return To The Top Page

Bokode: Imperceptible Visual Tags for Camera-based Interaction from a Distance

  • Ankit Mohan, Grace Woo, Shinsaku Hiura, Quinn Smithwick and Ramesh Raskar, "Bokode: Imperceptible Visual Tags for Camera-based Interaction from a Distance", ACM Transactions on Graphics, Vol.28, No.3, pp.98:1-98:8 (2009). (Proceedings of SIGGRAPH)

Bokode: Imperceptible Visual Tags for Camera-based Interaction from a Distance
  We show a new camera based interaction solution where an ordinarycamera can detect small optical tags from a relatively large distance.Current optical tags, such as barcodes, must be read within ashort range and the codes occupy valuable physical space on products.We present a new low-cost optical design so that the tags canbe shrunk to 3mm visible diameter, and unmodified ordinary camerasseveral meters away can be set up to decode the identity plusthe relative distance and angle. The design exploits the bokeh effectof ordinary cameras lenses, which maps rays exiting from an out offocus scene point into a disk like blur on the camera sensor. Thisbokeh-code or Bokode is a barcode design with a simple lensletover the pattern. We show that a code with 15m features can beread using an off-the-shelf camera from distances of up to 2 meters.We use intelligent binary coding to estimate the relative distanceand angle to the camera, and show potential for applications in augmentedreality and motion capture. We analyze the constraints andperformance of the optical system, and discuss several plausible applicationscenarios.
  

Return To The Top Page

Robust Estimation of Light Directions and Diffuse Reflectance of Known Shape Object

  • Takehiro Tachikawa, Shinsaku Hiura and Kosuke Sato, "Robust Estimation of Light Directions and Diffuse Reflectance of Known Shape Object", In Proceedings of Vision, Modeling, and Visualization Workshop,pp.37-44, 2009.

Robust Estimation of Light Directions and Diffuse Reflectance of Known Shape Object
  We propose a method to determine the light directionand diffuse reflectance property from two imageswith different light conditions. In our method,it is assumed that the shape of the target object isgiven. Using the relationships between light directionand diffuse reflectance, we can estimate bothof them simultaneously from more than five pointson two images. While speculars and shadows affectthe estimation as outlier and cause errors, wecan avoid these outliers by robust estimation usingRANSAC. The good nature of our method in termsof feasibility and stability is shown in experimentalresults with both simulation and real images.
  

Return To The Top Page

Uncalibrated Synthetic Aperture Photography for Defocus Control

  • Natsumi Kusumoto, Shinsaku Hiura and Kosuke Sato,Uncalibrated Synthetic Aperture Photography for Defocus Control,Journal of the Institute of Image Information and Television Engineers,Vol.63, No.6, pp. 857-865 (June 2009)

Uncalibrated Synthetic Aperture Photography for Defocus Control
  Exaggerated defocus cannot be achieved with an ordinary compact digital camera because of its tiny sensorsize, so taking pictures that draw the attention of a viewer to the subject is hard. Many methods are available for controllingthe focus and defocus of previously taken pictures. However, most of these methods require custom-built equipment such as acamera array to take pictures. Therefore, in this paper, we describe a method for creating images focused at any depth withan arbitrarily blurred background from a set of images taken by a handheld compact digital camera that is moved at random.Our method can produce various aesthetic blurs by changing the size, shape, or density of the blur kernel. In addition, wedemonstrate the potential of our method through a subjective evaluation of blurred images created by our system.
  

Return To The Top Page

Krill-eye : Superposition Compound Eye for Wide-Angle Imaging via GRIN Lenses

  • Shinsaku Hiura, Ankit Mohan and Ramesh Raskar, "Krill-eye : Superposition Compound Eye for Wide-Angle Imaging via GRIN Lenses", OMNIVIS2009, ICCV Workshops, pp. 2204-2211, 2009.

Krill-eye : Superposition Compound Eye for Wide-Angle Imaging via GRIN Lenses
  We propose a novel wide angle imaging system inspiredby compound eyes of animals. Instead of using a single lens,well compensated for aberration, we used a number of simplelenses to form a compound eye which produces practicallydistortion-free, uniform images with angular variation.The images formed by the multiple lenses are superposedon a single surface for increased light efficiency.We use GRIN (gradient refractive index) lenses to createsharply focused images without the artifacts seen when usingreflection based methods for X-ray astronomy. We showthe theoretical constraints for forming a blur-free image onthe image sensor, and derive a continuum between 1 : 1 flatoptics for document scanners and curved sensors focusedat infinity. Finally, we show a practical application of theproposed optics in a beacon to measure the relative rotationangle between the light source and the camera with IDinformation.
  

Return To The Top Page

3-D Shape Measurement Method with Modulated Slit Light Robust for Interreflection and Subsurface Scattering

  • Tatsuhiko Furuse, Shinsaku Hiura and Kosuke Sato,3-D Shape Measurement Method with Modulated Slit Light Robust for Interreflection and Subsurface Scattering,Proc. PROCAMS2009, pp. 1-2, 2009.

3-D Shape Measurement Method with Modulated Slit Light Robust for Interreflection and Subsurface Scattering
  We propose a method to accurately measure the shapeof objects by suppressing the effect of indirect reflectioncaused by the interreflection and subsurface scattering. Weuse a M-sequence pattern shifted along the line of the slitlight, and the sequence of captured images is analyzed usingsynchronous demodulator. This method utilizes two propertiesof indirect reflection; one is the transfer characteristicsof higher spatial frequency components, and the other isgeometric constraint between the projector and the camera.Prior to the measurement, epipolar constraint is obtainedthrough calibration, and then the phase consistency is evaluatedto suppress the interreflection. The cross-correlationvalue is also used to suppress the dilation of the slit lightcaused by the subsurface scattering.
  

Return To The Top Page

Touch Sensing by Image Analysis of Fingernail

  • Naoki Sugita, Daisuke Iwai, Kosuke Sato, "Touch Sensing by Image Analysis of Fingernail", In Proceedings of SICE Annual Conference 2008, pp.1520-1525, 2008.

Touch Sensing by Image Analysis of Fingernail
  In this paper, we propose a new method to recognize finger contact by image analysis, which does not disturbuser’s tactile feeling. When a finger touches a desktop surface, the color pattern of its nail changes from pink to white.The appearance change of the nail can be used to distinguish the finger contact onto a desktop surface by image matchingbetween a camera image and a fingertip image database. Due to high dimensionality of the database, we apply EigenNailmethod based on principal component analysis (PCA) to realize the quick image matching.
  

Return To The Top Page

A Rapid Anomalous Region Extraction Method by Iterative Projection onto Kernel Eigenspace

  • Satoshi Kawabata, Shinsaku Hiura and Kosuke Sato, "An Rapid Anomalous Region Extraction Method by Iterative Projection onto Kernel Eigenspace", Proc. ICPR2008 CD-ROM (Dec. 2008 at Tampa)

A Rapid Anomalous Region Extraction Method by Iterative Projection onto Kernel Eigenspace
  In computer vision, background subtraction methodis widely used to extract a changing region in a scene.However, it is difficult to simply apply this method to ascene with moving background object, because such objectmay be extracted as a changing region. Therefore,a method has been proposed to estimate both currentbackground image and occluding object region simultaneouslyby using eigenspace-based background representation.On the other hand, image completion methodusing eigenspace have been extended to non-linear subspaceusing kernel trick, however, such existing methodtakes large computational cost. Therefore, in this paper,we propose a method for rapid simultaneous estimationof a background image and occluded regionin non-linear space, using the kernel trick and iterativeprojection.
  

Return To The Top Page

Real-Time Detection of Anomalous Objects in Dynamic Scene

  • Satoshi Kawabata, Shinsaku Hiura and Kosuke Sato : Real-Time Detection of Anomalous Objects in Dynamic Scene, Proc. of 18th ICPR, pp.1171-1174 (Aug. 2006)

Real-Time Detection of Anomalous Objects in Dynamic Scene
  There are many methods to extract moving objects in a scene using background subtraction. However, most methods assume that there are no moving objects except intruders in the observing space. In this paper, we propose the iterative optimal projection method to estimate a varied background in real time from a dynamic scene with intruders. At ャ〉st, background images are collected for a while, because we assume that the motion of background is well known. Then, the background images are compressed using eigenspace method to form a database. While monitoring the scene, new image is taken by a camera, and the image is projected onto the eigenspace to estimate the background. But however, the estimated image is much affected by the intruders, so the intruder region is calculated by using background subtraction with former estimated background to exclude the region from the projection. Thus the image whose intruder region is replaced by the former background is projected to eigenspace and we have updated background. We proved that the cycle converges to a correct background image and we conャ〉med we can calculate the right region of the object through some experiments.
  

Return To The Top Page

Unknown Object Detection based on Trained Background Database

  • Satoshi Kawabata, Shinsaku Hiura, Kosuke Sato : Unknown Object Detection based on Trained Background Database, SICE2005 CD-ROM (Aug. 2005)

Unknown Object Detection based on Trained Background Database
  We have proposed an intrusion detection system which calculates the shape of intersection of intruders on sensitive planes using silhouette-based shape reconstruction. However, it is assumed that there are no moving objects except intruders in the observing space. In this paper, we propose the iterative optimal projection method to estimate a varied background in real time from a dynamic scene with intruders. At first, captured background sequence is compressed using eigenspace method to form a database. While the monitoring of the scene, estimation of background image and extraction of the region of intruder are executed simultaneously. We proved that the cycle converges to a correct background image and we confirmed we can calculate the right region of the object through some experiments.
  

Return To The Top Page

Autonomous Scene Monitoring System based on Multi Viewpoint Images

  • Takayuki Moritani, Shinsaku Hiura and Kosuke Sato : Object Tracking by Comparing Multiple Viewpoint Images to CG Images, Systems and Computers in Japan, Vol.37, No.13, pp. 28-39 (Oct. 2006)

Autonomous Scene Monitoring System based on Multi Viewpoint Images
  We propose an imprementation of automated monitoring system using multiplecameras embedded in the environment. At first, the object intuding to a sensitive area is detectedusing planar view-volume intersection method. It assumes no prior knowledge and stably detectsany object crossing into the monitored area. Then the whole shape of the object is measured usingmultiple stereo cameras. The region of object is extracted using improved background subtractionmethod using disparity images, and measured range images are aligned using common coordinatesystem. Finally, the motion of the object is tracked using generated CG images.
  

Return To The Top Page

Visual object tracking based on multi-viewpoint 3-D gradient method

  • Takayuki Moritani, Shinsaku Hiura, Kosuke Sato, Visual Object Tracking Based on Multi-Viewpoint 3-D Gradient Method, 1st International Symposium on Systems & Human Science (SSR2003), pp. 257-274 (2003)

Visual object tracking based on multi-viewpoint 3-D gradient method
  A Method for 3-D object motion tracking without feature extraction is necessary for monitoring human action inan usual civil-life scene. To realize a fast and robust object tracking method, we propose a model-based method using intensityimages taken with a multiple viewpoint camera connected to a PC cluster system. At first, the whole 3-D shape and reflectance ofthe object are prepared using several rangefinders. Each rangefiner is constructed with a camera, projector and PC, and all PCsare connected via network each other. For tracking the object, several CG images with varied object pose and position aregenerated in each PC using the object model, and then compared to the input intensity image in parallel. The result of thecomparison is transferred to a master PC, and the pose and position of the object are estimated by minimizing the residual of theCG and input images. We made a special CG generator which is a precise simulator of the real camera to generate a CG imageidentical to the input image. We confirmed the ability of our method and achieved a 3DOFs real-time object tracking system inthe rate of 4.5frames/sec, and 6DOFs tracking using four cameras in 2frames/sec.
  

Return To The Top Page

Object Tracking by Comparing CG Images with Multiple Viewpoint Images

  • Takayuki Moritani, Shinsaku Hiura, Seiji Inokuchi, Object Tracking by Comparing CG Images with Multiple Viewpoint Images, IEEE Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI2003), pp. 241-246 (2003)

Object Tracking by Comparing CG Images with Multiple Viewpoint Images
  A Method for 3-D object motion tracking without feature extraction is necessary for monitoring human action in a usual civil-life scene. To realize a fast and robust object tracking method, we propose a model-based method using intensity images taken with a multiple viewpoint camera connected to a PC cluster system. At first, the whole 3-D shape and reflectance of the object are prepared using several rangefinders. Each rangefiner is constructed with a camera, projector and PC, and all PCs are connected via network each other. For tracking the object, several CG images with varied object pose and position are generated in each PC using the object model, and then compared to the input intensity image in parallel. The result of the comparison is transferred to a master PC, and the pose and position of the object are estimated by minimizing the residual of the CG and input images. We made a special CG generator which is a precise simulator of the real camera to generate a CG image identical to the input image. We confirmed the ability of our method and achieved a 3DOFs real-time object tracking system in 4.5frames/sec speed.
  

Return To The Top Page

Intrusion Detection System using Contour-based Multi-Planar Visual Hull Method

  • Satoshi Kawabata, Shinsaku Hiura, Seiji Inokuchi, Intrusion Detection System using Contour-based Multi-Planar Visual Hull Method, IEEE Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI2003), pp. 293-298 (2003)

Intrusion Detection System using Contour-based Multi-Planar Visual Hull Method
  In this paper we propose a practical intrusion detection system based on a contour-based multi-planar visual hull method. The system consists of several sets of a camera and a PC connected via network. Prior to the monitoring, a system mamager will set up several sensitive planes in the world using a GUI. To detect the intruding objects, images taken by the cameras are compared with a maintained background image using PISC method. Then the contour of the object region on a image is traced and transformed to the coordinate of sensitive plane. The contours are collected to a single PC and existence of a common region is judged. If the intruding object exceeds the sensitive plane, the contours from each PC would have some common regions. If the intruder occludes the critical point on the sensitive planes from the all cameras to avoid observation, the system also alerts to the user. This system does not reconstruct the shape of the object, but the system is able to handle 3-D protection volumes because multiple sensitive planes can be set arbitrary. This system does not require us to calibrate the camera because this system is based on 2-D homography.
  

Return To The Top Page