Robotic perception and manipulation of deformable linear objects

Deformable objects are pervasive in the everyday life environment, in the form of clothes, cables, wires, ropes and many other objects. Despite their importance and widespread diffusion, there still exist many limitations when it comes to deploying robotic systems for interacting with deformable objects. This thesis presents a comprehensive exploration of research activities geared towards enhancing the perception and manipulation capabilities of a robotic system when dealing with deformable linear objects. The activities are organized into two main research aspects, namely perception and manipulation. In the first part of this thesis, the focus is on developing perception solutions for deformable linear objects, primarily relying on visual data and exploiting deep learning techniques. Consequently, innovative methods are developed to address the dataset generation challenge with minimal to no human intervention. Furthermore, novel approaches are applied to tackle the instance segmentation task by combining deep learning techniques with graph-based representations of the object's configuration. The 3D reconstruction task is also addressed through a multi-view stereo reconstruction approach. The second aspect of the research concentrated on the manipulation problem, specifically in predicting how robot actions affect the deformable linear object's configuration. This is achieved by employing a differentiable model of the object's dynamics that is used for planning the optimal manipulation action for achieving a target configuration. The same model is also used for estimating model parameters, thereby improving the prediction accuracy and consequently enhancing the robotic system's manipulation capabilities. Finally, the perception methods developed in this thesis are extended to encompass the perception of deformable multi-linear objects, such as wire harnesses. To this end, a learning-based topological representation is conceived and applied in the context of a dual-arm disentangling manipulation task.