Imitation learning | Mark Edmonds

Captured the complex hand forces humans use to open seven different medicine bottles with a tactile glove covered in IMUs (inertial measurement units) and force sensors.
Constructed a robot action planner using a haptic network, an And-Or Graph, and the generalized Earley parser.
Result: questioned the common structure humans see in the procedure to open any medicine bottle versus the widely varying forces and action sequences the robot used for each bottle. This prompted an investigation of abstraction and generalization in causal learning.

Engineering contributions

Neural network training for action planning and embodiment mapping between a human demonstrator and a robot.
Localization using SLAM, IMU, and wheel odometry, combined with Kalman filtering, using a Microsoft Kinect and a Velodyne VLP-16.
ROS navigation stack, including a dynamic footprint based on the current position of the arms.

Collaborators: Feng Gao, Hangxin Liu, Xu Xie, Matt Millar, Siyuan Qi, Yixin Zhu, Brandon Rothrock (NASA JPL), Song-Chun Zhu

@inproceedings{edmonds2017feeling, title = {Feeling the Force: Integrating Force and Pose for Fluent Discovery through Imitation Learning to Open Medicine Bottles}, author = {Edmonds, M. and Gao, F. and Xie, X. and Liu, H. and Qi, S. and Zhu, Y. and Rothrock, B. and Zhu, S.-C.}, booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, year = {2017} }

@inproceedings{xie2018unsupervised, title = {Unsupervised Learning of Hierarchical Models for Hand-Object Interactions}, author = {Xie, X. and Liu, H. and Edmonds, M. and Gao, F. and Qi, S. and Zhu, Y. and Rothrock, B. and Zhu, S.-C.}, booktitle = {IEEE International Conference on Robotics and Automation (ICRA)}, year = {2018} }

@inproceedings{liu2017glove, title = {A Glove-based System for Studying Hand-Object Manipulation via Pose and Force Sensing}, author = {Liu, H. and Xie, X. and Millar, M. and Edmonds, M. and Gao, F. and Zhu, Y. and Santos, V. and Rothrock, B. and Zhu, S.-C.}, booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, year = {2017} }

Publications

Feeling the Force: Integrating Force and Pose for Fluent Discovery through Imitation Learning to Open Medicine Bottles

Unsupervised Learning of Hierarchical Models for Hand-Object Interactions

A Glove-based System for Studying Hand-Object Manipulation via Pose and Force Sensing