A Review of Differentiable Simulators

Authors: Rhys Newbury, Jack Collins, Kerry He, Jiahe Pan, Ingmar Posner, David Howard, Akansel Cosgun

Abstract: Differentiable simulators continue to push the state of the art across a range of domains including computational physics, robotics, and machine learning. Their main value is the ability to compute gradients of physical processes, which allows differentiable simulators to be readily integrated into commonly employed gradient-based optimization schemes. To achieve this, a number of design decisions need to be considered representing trade-offs in versatility, computational speed, and accuracy of the gradients obtained. This paper presents an in-depth review of the evolving landscape of differentiable physics simulators. We introduce the foundations and core components of differentiable simulators alongside common design choices. This is followed by a practical guide and overview of open-source differentiable simulators that have been used across past research. Finally, we review and contextualize prominent applications of differentiable simulation. By offering a comprehensive review of the current state-of-the-art in differentiable simulation, this work aims to serve as a resource for researchers and practitioners looking to understand and integrate differentiable physics within their research. We conclude by highlighting current limitations as well as providing insights into future directions for the field.

If you find our review useful, please cite us.

@ARTICLE{newbury2024Review,
  author={Newbury, Rhys and Collins, Jack and He, Kerry and Pan, Jiahe and Posner, Ingmar and Howard, David and Cosgun, Akansel},
  journal={IEEE Access}, 
  title={A Review of Differentiable Simulators}, 
  year={2024},
  volume={},
  number={},
  pages={1-1},
  doi={10.1109/ACCESS.2024.3425448}}

Publication Accepted to IEEE Access

Preprint: arXiv

Work	Authors	Engine Name	Gradient Method	Dynamical Model	Soft Body?	Contact Model	Integrator
End-to-end differentiable physics for learning and control, (2018)	de Avila Belbute-Peres et al.		Auto Diff	Newton	-	LCP	Explicit
A Differentiable Physics Engine for Deep Learning in Robotics, (2019)	Degrave et al.		Auto Diff	Newton	-	Complementarity	Semi-Implicit
Interactive Differentiable Simulation, (2019)	Heiden et al.	TDS	Auto Diff	Newton	-	Compliant Model	Semi-Implicit
Real2Sim Transfer using Differentiable Physics, (2019)	Heiden et al.		Auto Diff	Newton	-	None	Semi-Implicit
Chainqueen: A real-time differentiable physical simulator for soft robotics, (2019)	Hu et al.		Analytical and Symbolic	Continuum Mechanics	✅	MLS-MPM	Explicit
Differentiable cloth simulation for inverse problems, (2019)	Liang et al.		Implicit Diff	Newton	✅	Position Based	Implicit
ADD: Analytically Differentiable Dynamics for Multi-Body Systems with Frictional Contact, (2020)	Geilinger et al.		Adjoint Method	Newton	✅	NCP	Implicit
phiflow: A differentiable pde solving framework for deep learning via physical simulations, (2020)	Holl et al.		Adjoint Method	Fluid Simulation	-	None	Explicit
phiflow: A differentiable pde solving framework for deep learning via physical simulations, (2020)	Holl et al.	PhiFlow	Auto Diff	Fluid Simulation	-	Explicit	None
Scalable Differentiable Physics for Learning and Control, (2020)	Qiao et al.		Implicit Diff	Newton	-	Position Based	Implicit
Learning to Slide Unknown Objects with Differentiable Physics Simulations, (2020)	Song & Boularias		Analytical	Newton	-	None	Explicit
Dynamic visual reasoning by learning differentiable physics models from video and language, (2021)	Ding et al.		Auto Diff	Newton	-	Impulse Based	Implicit
DiffPD: Differentiable projective dynamics, (2021)	Du et al.		Adjoint Method	Projective Dynamics	✅	Complementarity	Implicit
Underwater Soft Robot Modeling and Control With Differentiable Simulation, (2021)	Du et al.		Auto Diff	Projective Dynamics	✅	Complementarity	Implicit
Brax - A Differentiable Physics Engine for Large Scale Rigid Body Simulation, (2021)	Freeman et al.	Brax	Newton	XPBD	-	Position Based	Semi-Implicit
NeuralSim: Augmenting Differentiable Simulators with Neural Networks, (2021)	Heiden et al.		Auto Diff	Newton	-	NCP + Compliant	Semi-Implicit
PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics, (2021)	Huang et al.		Auto Diff	Continuum Mechanics	✅	MLS-MPM	Explicit
gradSim: Differentiable simulation for system identification and visuomotor control, (2021)	Jatavallabhula et al.	GradSim	Auto Diff	Newton	✅	Compliant Model	Semi-Implicit
Differentiable simulation for physical system identification, (2021)	Le Lidec et al.		Implicit Diff	Lagrangian	-	Complementarity	Implicit
Differentiable physics models for real-world offline model-based reinforcement learning, (2021)	Lutter et al.		Auto Diff	Newton	-	None	Not Specified
PODS: Policy Optimization via Differentiable Simulation, (2021)	Mora et al.		Adjoint Method	Newton	-	None	Implicit
Efficient Differentiable Simulation of Articulated Bodies, (2021)	Qiao et al.		Adjoint Method	Newton	-	LCP	Explicit
Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulation, (2021)	Ścibior et al.		Auto Diff	Newton	-	None	Not Specified
Sim2Sim Evaluation of a Novel Data-Efficient Differentiable Physics Engine for Tensegrity Robots, (2021)	Wang et al.		Auto Diff	Newton	-	Impulse Based	Semi-Implicit
Fast and Feature-Complete Differentiable Physics for Articulated Rigid Bodies with Contact, (2021)	Werling et al.	Nimble	Symbolic	Lagrangian	-	LCP	Explicit
Accelerated Policy Learning with Parallel Differentiable Simulation, (2021)	Xu et al.		Auto Diff	Newton	-	Compliant Model	Semi-Implicit
Extending Lagrangian and Hamiltonian Neural Networks with Differentiable Contact Models, (2021)	Zhong et al.		Implicit Diff	Lagrangian	-	Convex Optimization	Explicit
Differentiable dynamics for articulated 3d human motion reconstruction, (2022)	Gärtner et al.		Auto Diff	Newton	-	LCP	Semi-Implicit
Fine-grained Differentiable Physics: A Yarn-level Model for Fabrics, (2022)	Gong et al.		Implicit Diff	Lagrangian	✅	Compliant Model	Implicit
Model Identification and Control of a Low-cost Mobile Robot with Omnidirectional Wheels using Differentiable Physics, (2022)	Granados et al.		Auto Diff	Newton	-	None	Explicit
Dojo: A Differentiable Simulator for Robotics, (2022)	Howell et al.	Dojo	Implicit Diff	Lagrangian		Complementarity	Implicit
Fast Aquatic Swimmer Optimization with Differentiable Projective Dynamics and Neural Network Hydrodynamic Models, (2022)	Nava et al.		Auto Diff	Fluid Simulation and Continuum Mechanics	✅	None	Implicit
Learning Object Manipulation Skills from Video via Approximate Differentiable Physics, (2022)	Petrı́k Vladimı́r et al.		Auto Diff	Newton	-	Impulse Based	Explicit
Grasp’d: Differentiable contact-rich grasp synthesis for multi-fingered hands, (2022)	Turpin et al.		Auto Diff	Newton	-	Compliant Model	Semi-Implicit
Differentiable Simulation of Inertial Musculotendons, (2022)	Wang et al.		Auto Diff	Newton	-	None	Implicit
A Recurrent Differentiable Engine for Modeling Tensegrity Robots Trainable with Low-Frequency Data, (2022)	Wang et al.		Auto Diff	Newton	-	Impulse Based	Implicit or Semi-Implicit
Automatic Co-Design of Aerial Robots Using a Graph Grammar, (2022)	Zhao et al.		Auto Diff	Newton and Fluid Simulation	-	None	Explicit
Graph Grammar-Based Automatic Design for Heterogeneous Fleets of Underwater Robots, (2022)	Zhao et al.		Analytical	Newton and Fluid Simulation	-	None	Not Specified
JAX-Fluids: A fully-differentiable high-order computational fluid dynamics solver for compressible two-phase flows, (2023)	Bezgin et al.	JAX-Flows	Auto Diff	Fluid Simulation		Explicit	None
DaXBench: Benchmarking Deformable Object Manipulation with Differentiable Physics, (2023)	Chen et al.	daX	Auto Diff	Continuum Mechanics	✅	MLS-MPM	Explicit
DiffXPBD: Differentiable Position-Based Simulation of Compliant Constraint Dynamics, (2023)	Stuyck & Chen		Analytical	XPBD	✅	Implicit	Compliant Model
Differentiable physics simulation of dynamics-augmented neural objects, (2023)	Le Cleac’h et al.		Auto Diff	Newton	-	Compliant Model	Implicit
Robotic manipulation of deformable rope-like objects using differentiable compliant position-based dynamics, (2023)	Liu et al.		Auto Diff	XPDB	✅	Position Based	Explicit
Advanced soft robot modeling in ChainQueen, (2023)	Spielberg et al.		Auto Diff	Continuum Mechanics	✅	MLS-MPM	Explicit
Fast-Grasp’D: Dexterous Multi-finger Grasp Generation Through Differentiable Simulation, (2023)	Turpin et al.		Auto Diff	Newton	-	Position Based	Semi-Implicit
SoftZoo: A Soft Robot Co-design Benchmark For Locomotion In Diverse Environments, (2023)	Wang et al.		Auto Diff	Continuum Mechanics	✅	MLS-MPM	Semi-Implicit
Training Efficient Controllers via Analytic Policy Gradient, (2023)	Wiedemann et al.		Auto Diff	Newton	-	None	Explicit
FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation, (2023)	Xian et al.	FluidLab	Analytical	Continuum Mechanics and Fluid Simulation	-	MLS-MPM	Explicit

A Review of Differentiable Simulators

Acknowledgments