Fig. 1 3D Motion Mimesis from 2D Images
Fig. 2 (top) Demonstrator’s 2D motion
(bottom) Imitated 3D motion model (blue) is compared with real model (magenta)
Imitation skills for humanoids have received a great deal of attention because the imitation function is the most primitive and fundamental factor of intelligence and imitation learning provides automatic programming without extensive trials. Inspired by the discovery of mirror neuron in neuroscience, YNL developed HMM-based mimesis model, which is a corresponding forms of mirror neuron.
Although the mimesis model is a nice mathematical framework, there are critical issues for applications in the real world: dealing with partial observation and various forms of sensory data. Therefore, we developed "complete motion imitation from incomplete observation" by statistical inference based on symbolized motion database. The follows are solved with the proposed method.
- 3D motion imitation from 2D images
- Whole body imitation from occluded observation
- The method is extended into following applications.
- Motion Capturing from Monocular Vision
- Dongheui Lee and Yoshihiko Nakamura, Mimesis Scheme using a Monocular Vision System on a Humanoid, in Proc. 2007 IEEE int'l. Conf. on Robotics and Automation (ICRA'07), pp. 2162-2168, Rome, Italy, Apr 10-14, 2007
- Dongheui Lee and Yoshihiko Nakamura, Stochastic Model of Imitating a New Observed Motion Based on the Aquired Motion Primitives, In Proc. 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'06), pp. 4994-5000, Beijing, China, Oct. 10-13, 2006
- Dongheui Lee and Yoshihiko Nakamura, Mimesis from Partial Observations, In Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'05), pp. 1911-1916, Aug. 2005
- Dongheui Lee and Yoshihiko Nakamura, Probabilistic Model of Whole-body Motion Imitation from Partial Observations, In Proc. of The 12th International Conference on Advanced Robotics (ICAR'05), pp.337 ? 343, Seattle, July 18-20, 2005
- Dongheui Lee and Yoshihiko Nakamura, Human Motion Understanding from 2D Partial Observation of Vector Field based on Particle Filter (written in Japanese), Proceedings of the 2007 JSME Conference on Robotics and Mechatronics (ROBOMEC'2007) , Akita, Japan, May 10-12, 2007
- Dongheui Lee and Yoshihiko Nakamura, Stochastic Theory for Motion Capturing from Onboard Monocular Vision of Humanoid Robots, in Proc. 12th Robotics Symposia, 4B2, pp.424-429, Nagaoka, JAPAN, Mar 15-16, 2007
- Dongheui Lee and Yoshihiko Nakamura, Motion Recognition with a Monocular Vision System Based on Mimesis Scheme (written in Japanese), Proceedings of the 24th Annual Conference of the Robotics Society of Japan (RSJ'2006), 1M23, Okayama,Japan, September 14-16, 2006
- Dongheui Lee and Yoshihiko Nakamura, Relaxed Viterbi Training: Fast and Reliable Symbol Acquisition of Mimesis Model, Proceedings of the 24th Annual Conference of the Robotics Society of Japan (RSJ'2006), 2D22, Okayama,Japan, September 14-16, 2006
- Dongheui Lee and Yoshihiko Nakamura, Probabilistic Mimesis Model: Recognition and Generation from Partial Observations (written in Japanese), Proceedings of the 2006 JSME Conference on Robotics and Mechatronics (ROBOMEC'2006) , 1P1-E11, Waseda, Tokyo, Japan, May 26-28, 2006
DongHeui Lee, Hirotoshi Kunori
(Your opinions and questions about Partial Observation are welcome.E-mail at
)