Testing results in simulation

[0xcuriousapple]

Hey @j96w
Awesome paper
I am trying to imitate my own choice of tasks
I am done with data collection and training, and now I want to emulate it in simulation before trying it out on actual leap hand.
Any thoughts on how one can test DexCap results in simulated environment (something like Issac, Gazebo..)?

Thanks

[0xcuriousapple]

Hey,

Just to give you an update,
I went ahead with using Webots.
I'm done with the Webots and controller setup, and now looking to attach a ROS node for inference.

my_project_3.mp4

(ps: this is just normal pick and place, there is no AI or imitation learning involved as of now)

If you have any suggestions, please let me know :)

Thanks

[j96w]

Hi @0xcuriousapple, thanks for sharing these! A simulator is definitely helpful. We also considered it earlier, but since DexCap collects human data in the real world, we would need an additional process to build a "digital twin" in the sim to replay the collected human data. How do you plan to handle the real-to-sim gap? Or are you just looking for a teleoperation setup in the sim? DexCap is designed to collect data without a robot (so it's not directly designed for teleop).