Learning to Group

We address the problem of discovering 3D parts for objects in unseen categories. Being able to learn the geometry prior of parts and transfer this prior to unseen categories pose fundamental challenges on data-driven shape segmentation approaches. Formulated as a contextual bandit problem, we propose a learning-based agglomerative clustering framework which learns a grouping policy to progressively group small part proposals into bigger ones in a bottom-up fashion.

Figure 1: Overview of the proposed approach.

At the core of our approach is to restrict the local context for extracting part-level features, which encourages the generalizability to unseen categories. On the large-scale fine-grained 3D part dataset, PartNet, we demonstrate that our method can transfer knowledge of parts learned from 3 training categories to 21 unseen testing categories without seeing any annotated samples. Quantitative comparisons against four shape segmentation baselines shows that our approach achieve the state-of-the-art performance.

Figure 2: Qualitative results on unseen categories. The tested model are trained on Chair, Table, and Furniture data.

Figure 3: Grouping tree: intermediate sub-parts during the grouping process. Please click Visualization for seeing the trajectories in the browser.

• Learning to Group in [PyTorch (Official)].
• Used 3D data with part annotations can be downloaded in [Google Drive (.zip)].
• Pre-trained models are provided in [Github repo].