Dobb-E for Web Apps

Dobb-E is an open-source framework aimed at enhancing household robots through imitation learning. It utilizes an affordable tool known as the Stick, which is made from a Reacher-grabber stick, 3D printed parts, and an iPhone. This setup allows for the collection of demonstration data, essential for training robots. The framework is built on a dataset called Homes of New York (HoNY), which includes 13 hours of interaction data from 22 homes, featuring RGB and depth videos along with detailed action annotations.

Central to Dobb-E's functionality is its training of a model named Home Pretrained Representations (HPR), based on the ResNet-34 architecture. This model uses self-supervised learning to teach robots how to perform tasks in new environments. With an average success rate of 81% in completing novel tasks within 15 minutes, Dobb-E offers access to pre-trained models, source code, and extensive documentation on GitHub, in addition to a research paper outlining its methodology.