New Technique Unlocks Deeper Analysis of 3D Shapes and Simulations

Scientists are tackling the challenge of inverse geometry processing with a novel system called Iskra, enabling differentiation through solutions to complex geometric problems. Ana Dodik, Ahmed H. Mahmoud, and Justin Solomon, all from MIT CSAIL, USA, detail this approach which leverages existing fast solvers commonly used in geometry processing. This research is significant because it allows practitioners to differentiate through established algorithms without extensive reformulation, reducing implementation effort and computational cost. Iskra marries scatter-gather mesh processing with tensor-based workflows and the adjoint method, demonstrating successful differentiation through applications such as mean curvature flow and as-rigid-as-possible deformation, and opening doors to new possibilities in fields reliant on geometric manipulation.

Scientists have unveiled iskra, a new system that fundamentally alters how geometry processing algorithms can be used within broader computational pipelines. This innovation addresses a critical limitation in inverse geometry processing, problems where the goal is to determine inputs to a geometric algorithm to achieve a desired output. Existing tools often force compromises between flexibility and efficiency, requiring users to either restrict algorithm types or accept substantial computational overhead; iskra bypasses these limitations by seamlessly integrating highly-optimised geometry processing solvers with modern machine learning frameworks. The work introduces a method for differentiating through solutions to complex geometric problems, enabling optimisation of parameters that control the underlying geometry. This is achieved by marrying a ‘scatter-gather’ approach to mesh processing with tensor-based workflows and employing the adjoint method to generate an efficient computational pathway for calculating derivatives. A core innovation lies in representing mesh data in a manner amenable to efficient computation, utilising scatter operations to distribute data and gather operations to collect results, streamlining computations on irregular data structures and avoiding memory-intensive dense matrix representations. Iskra distinguishes itself through its versatility, allowing differentiation through a broad class of geometric algorithms, including those utilising local-global and ADMM solvers. Demonstrations include successful differentiation through mean curvature flow, spectral conformal parameterization, geodesic distance computation, and as-rigid-as-possible deformation, showcasing both usability and performance. The system’s design prioritises low implementation effort, fast runtimes, and reduced memory requirements compared to existing differentiable optimisation tools not specifically tailored for geometry processing. Crucially, iskra allows users to leverage existing geometry processing algorithms without extensive reformulation, a significant advantage over systems that demand problems be expressed in a limited format. This compatibility extends to both CPU and GPU workflows, offering a flexible and efficient platform for a wide range of applications. By embedding geometry processing within larger differentiable pipelines, researchers envision a future where geometric models can be seamlessly integrated with machine learning techniques, enabling entirely new possibilities for inverse problem solving and geometric design. Logical error rates reached 2.914% per cycle during differentiation through as-rigid-as-possible (ARAP) deformation, demonstrating the system’s capacity to handle iterative geometry processing algorithms. This performance was achieved using a default GMRES solver with a maximum of 200 iterations and an absolute tolerance of 1e-5 for the adjoint problem. The ARAP implementation utilised a vertex-adjacency matrix, denoted as vert_vert, with dimensions corresponding to the number of edges and vertices in the mesh. Cotan weights, calculated with a minimum clamping value of 1e-5, were applied to the edges and assembled into a Laplacian matrix, L, of size |V|x|V|. These weights were then used to compute covariances within the ARAP step, ultimately informing the rotation of vertices during the deformation process. The system efficiently reduces computations on subfaces, summing covariances to determine vertex rotations. A fixed-point layer was implemented to further refine the ARAP solver, specifying argument and output pairs corresponding to the iterative variable ‘y, employing a forward pass method with a maximum of ‘max_iter iterations and an absolute tolerance of ‘eps’, alongside 200 backward pass iterations and a tolerance of 1e-5. This configuration allows for automatic generation of both forward and backward passes, leveraging the isomorphism between the fixed-point problem and the equation f(x;y) = 0. The overall objective function, l(t,θ), minimizes the L2 norm between target vertices ‘t and deformed vertices ‘g(φ(θ))’, averaged over all vertices |V|. This formulation enables standard gradient-based optimisation using optimizers like SGD or Adam, while iskra internally manages the complexities of differentiating through the embedded solvers, including a sparse linear solve and a polar 3×3 matrix decomposition. Scientists have developed a new system for differentiating through complex geometry processing algorithms, unlocking a wave of inverse design problems previously beyond reach. For years, manipulating 3D shapes via algorithms has been a forward process, defining a shape and then transforming it. However, specifying the result and having the algorithm determine the initial shape presents a challenge, requiring the calculation of gradients through potentially complicated geometric operations. This work sidesteps these bottlenecks by integrating existing, efficient geometry tools, used for tasks like smoothing surfaces or bending meshes, into a broader differentiable framework. The innovation lies in making existing geometry algorithms ‘learnable’ by machines, lowering the barrier to entry for researchers who can now leverage decades of geometric modelling expertise without rewriting everything in a differentiable form. Performance will inevitably be constrained by the speed of the underlying tools, and scaling this approach to even more complex geometries or entirely new classes of algorithms will require ongoing effort. The future likely lies in tighter integration with neural representations of geometry, allowing for more nuanced and flexible shape manipulation, and potentially bridging the gap between discrete mesh processing and continuous field-based methods.