Bryan Willimon, Steven Hickson, Stan Birchfield, and Ian Walker
We propose an algorithm that uses energy minimization to estimate the current configuration of a nonrigid object. Our approach utilizes an RGBD image to calculate corresponding SURF features, depth, and boundary information. We do not use predetermined features, thus enabling our system to operate on unmodified objects. Our approach relies on a 3D nonlinear energy minimization framework to solve for the configuration using a semiimplicit scheme. Results show various scenarios of dynamic posters and shirts in different configurations to illustrate the performance of the method. In particular, we show that our method is able to estimate the configuration of a textureless nonrigid object with no correspondences available.
Our algorithm consists of minimizing an energy equation with four terms:
Our goal is to locate the mesh that best explains the data while adhering to the smoothness prior. To find the configuration of minimum energy, we compute the partial derivative of the energy with respect to the vectors X_{t}, Y_{t}, and Z_{t}, and set the result to zero. In order to calculate the minimal solution, we employ the semiimplicit scheme used by Kass et al. (IJCV 1988). The smoothness term is treated explicitly, while the data terms are treated implicitly. Let V_{t} represent the mesh at iteration t and V_{t  1} the mesh at the previous iteration, t  1. Then the terms are given in the following table.
¡¡
Smoothness Term Correspondence Term Depth Term Boundary Term
Below are the three equations for X_{t}, Y_{t}, and Z_{t} that we set to zero and minimize.
We captured RGBD video sequences of shirts and posters to test our proposed method¡¯s ability to handle different nonrigid objects in a variety of scenarios. We also verified the contributions made by the novel depth and boundary energy terms to the accuracy of the estimated object configuration. For our experiments, the weights were set according to λ_{C} = 1.3, λ_{B} = 0.8, and λ_{D} = 0.6.
Contribution of Each Energy Term to Improve the Final Mesh 
The improvement resulting from the various terms. From left to right: 
Textured Shirt Experiment 
Four frames of a sequence in which a shirt partially occludes itself. Top: The estimated mesh (V) overlaid on the RGB color image. Bottom: a 45° pan view of the 3D mesh

Untextured Shirt Experiment 
Five frames from a sequence in which a completely textureless shirt with no pattern is rotated out of the image plane, while the algorithm estimates the configuration of the shirt as a triangular mesh estimate. Even without data correspondences, the proposed energy function is able to accurately estimate the configuration of a dynamic nonrigid object.

Bryan Willimon, Steven Hickson, Stan Birchfield, and Ian Walker, An Energy Minimization Approach to 3D NonRigid Deformable Surface Estimation Using RGBD Data, Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS), 2012.
This research was supported by the U.S. National Science Foundation under grants IIS1017007, and IIS0904116.