NePu

Neural Puppeteer is an efficient neural rendering pipeline for articulated shapes. Through inverse rendering, it predicts 3D keypoints from multi-view 2D silhouettes without needing texture information. Additionally, this model can predict 3D keypoints for the same class of shapes using a single trained model. It also generalizes effectively from synthetic data training, as demonstrated by its success in zero-shot synthetic to real-world experiments. Furthermore, the neural rendering pipeline learns a global texture embedding which can be used in a downstream task to identify individuals.