Data-driven Control
Classical control approaches are based on physical dynamic models, which are required to describe the true underlying system behaviour in a sufficiently accurate fashion. For complex dynamical systems, however, such descriptions are often extremely hard to obtain or even nonexistent, hence data-driven approaches have to be employed. Data-driven models are based on observations and measurements of the true system and only require a minimum amount of prior knowledge of the system. However, they require new control approaches since classic analysis and synthesis tools are not suitable for models of probabilistic nature. Our work focuses on the Gaussian process model, which is very generally applicable and has shown to be successful in many control scenarios. We develop new control algorithms, which not only improve the overall performance but also guarantee the stability of the closed-loop system. Finally, the approaches are tested and validated in robotic experiments.
Try Gaussian Process Regression
Identification and Control with Gaussian Processes
Researcher: Thomas Beckers, Alexandre Capone, Jonas Umlauft
Motivation
Data-driven approaches from machine learning provide powerful tools to identify dynamical systems with limited prior knowledge of the model structure. These are, on the one side, very flexible to model a large variety of systems, but, on the other side, also bring new challenges: Classical control approaches need to be adapted to work successfully on data-based models, certain desired properties on convergence are difficult to prove and multiple ways exist to exploit the prior knowledge available.
Research Questions
- How to enforce stability in data-driven models?
- Which control laws allow formal guarantees in closed-loop systems?
- Can knowledge on model fidelity be used in the control design?
Approach
On the system identification side, we focus on Gaussian processes to model unknown systems. We use approaches from robust and adaptive control in the design and analysis of the controller to handle imprecision in the identified model. Since data-driven models are often of probabilistic nature, tools for stochastic differential equations are required. For the future, we are planning to apply stochastic optimal control and scenario-based model predictive control in this setting.
Key results
- Different methods for the identification of systems which are a priori known to be stable have been developed.
- A closed-loop identification of control-affine systems using Gaussian processes was proposed and applied in a feedback linearization setting.
- Developing of a GPR-based control law for Lagrangian systems which guarantees a bounded tracking error of the closed-loop system.
- Stability properties of Gaussian Process State Space Models for different kernel functions.
Optimal Learning Control based on Gaussian Processes
Researcher: Armin Lederer
Motivation
Model predictive control is a modern control technique that has been applied to a wide variety of systems. Its success stems from its capability to explicitly handle constraints on states and control inputs as well as simple implementation of tracking control. However, it requires a precise model of the controlled system, which is often not available because of high system complexity or inherent system uncertainty. Gaussian processes offer a solution to this issue by allowing to learn system models from data of the system dynamics. Nevertheless, using learned models in model predictive control raises questions at the intersection between machine learning in control theory.
Research Questions
- Can Gaussian processes be modified to allow on-line learning while providing theoretical learning error bounds?
- Is it possible to guarantee stability of a system controlled by model predictive control if only samples of the system's dynamics are known?
- How can model uncertainty be exploited for control as well as learning in closed-loop?
- Do Gaussian processes exist which facilitate the design of optimal control?
Approach
For on-line learning we focus on local Gaussian processes and Gaussian processes with compactly supported kernels allowing exact inference. By combining these approaches with methods from computational geometry they can be implemented efficiently. On the control side we apply sampling based approaches for stability verification and parameter optimization of model predictive control. Furthermore, we develop robust control strategies tailored to the setting provided by Gaussian process models. In the future we plan to investigate the effect of on-line learning on closed-loop stability and to develop control schemes that allow optimal learning in closed-loop.
Selected publications
- Safe learning-based trajectory tracking for underactuated vehicles with partially unknown dynamics. IEEE Control Systems Letters (submitted), 2020 more… BibTeX
- How Training Data Impacts Performance in Learning-based Control. IEEE Control Systems Letters, 2020, 1-1 more… BibTeX
- Feedback Linearization based on Gaussian Processes with event-triggered Online Learning. IEEE Transactions on Automatic Control, 2020 more… BibTeX
- Visual Pursuit Control with Target Motion Learning via Gaussian Process. Proceedings of the Conference of the Society of Instrument and Control Engineers of Japan, 2020 more… BibTeX
- Confidence Regions for Simulations with Learned Probabilistic Models. Proceedings of the American Control Conference (ACC), 2020 more… BibTeX
- Data Selection for Multi-Task Learning Under Dynamic Constraints. IEEE Control Systems Letters 5 (3), 2020, 959-964 more… BibTeX
- Prediction with Gaussian Process Dynamical Models. IEEE Transactions on Automatic Control (submitted), 2020 more… BibTeX
- Learning Stochastically Stable Gaussian Process State-Space Models. IFAC Journal of Systems and Control 12, 2020 more… BibTeX
- GP3: A Sampling-based Analysis Framework for Gaussian Processes. Proceedings of the 21st IFAC World Congress , 2020 more… BibTeX
- Learning Stable Nonparametric Dynamical Systems with Gaussian Process Regression. Proceedings of the 21st IFAC World Congress , 2020 more… BibTeX
- Localized active learning of Gaussian process state space models. Learning for Dynamics & Control, 2020 more… BibTeX
- Parameter Optimization for Learning-based Control of Control-Affine Systems. Learning for Dynamics & Control, 2020 more… BibTeX
- Smart Forgetting for Safe Online Learning with Gaussian Processes. Learning for Dynamics & Control, 2020 more… BibTeX
- Posterior Variance Analysis of Gaussian Processes with Application to Average Learning Curves. arXiv preprint: arXiv:1906.01404, 2019 more… BibTeX
- Stable Gaussian Process based Tracking Control of Euler-Lagrange Systems. Automatica (103), 2019, 390-397 more… BibTeX
- Backstepping for Partially Unknown Nonlinear Systems Using Gaussian Processes. IEEE Control Systems Letters 3 (2), 2019, 416 - 421 more… BibTeX
- Local Asymptotic Stability Analysis and Region of Attraction Estimation with Gaussian Processes. Proceedings of the 58th Conference on Decision and Control (CDC) , 2019 more… BibTeX
- Closed-loop Model Selection for Kernel-based Models Using Bayesian Optimization. Proceedings of the 58th Conference on Decision and Control (CDC), 2019 more… BibTeX
- Keep soft robots soft - a data-driven based trade-off between feed-forward and feedback control. Workshop on Robust autonomy: tools for safety in real-world uncertain environments (RSS 2019), 2019 more… BibTeX
- Uniform Error Bounds for Gaussian Process Regression with Application to Safe Control. Conference on Neural Information Processing Systems (NeurIPS), 2019 more… BibTeX
- An Uncertainty-Based Control Lyapunov Approach for Control-Affine Systems Modeled by Gaussian Process. IEEE Control Systems Letters 2 (3), 2018, 483-488 more… BibTeX
- A Scenario-based Optimal Control Approach for Gaussian Process State Space Models. Proceedings of the European Control Conference (ECC), 2018 more… BibTeX
- Mean Square Prediction Error of Misspecified Gaussian Process Models. Proceedings of the 57th Conference on Decision and Control (CDC), 2018 more… BibTeX
- Gaussian Process based Passivation of a Class of Nonlinear Systems with Unknown Dynamics. 2018 European Control Conference (ECC), IEEE, 2018 more… BibTeX
- Stable Model-based Control with Gaussian Process Regression for Robot Manipulators. Proceedings of the 20th IFAC World Congress, 2017 more… BibTeX
- Feedback Linearization using Gaussian Processes. Proceedings of the Conference on Decision and Control (CDC), IEEE, 2017 more… BibTeX
- Bayesian Uncertainty Modeling for Programming by Demonstration. International Conference on Robotics and Automation (ICRA), IEEE, 2017 more… BibTeX
- Learning Stable Gaussian Process State Space Models. American Control Conference (ACC), IEEE, 2017 more… BibTeX
- Stable Gaussian Process based Tracking Control of Lagrangian Systems. Proceedings of the 56th Conference on Decision and Control (CDC), 2017 more… BibTeX
- Learning Stable Stochastic Nonlinear Dynamical Systems. International Conference on Machine Learning (ICML), 2017 more… BibTeX
- Equilibrium distributions and stability analysis of Gaussian Process State Space Models. Proceedings of the 55th Conference on Decision and Control (CDC), 2016 more… BibTeX
- Stability of Gaussian Process State Space Models. Proceedings of the European Control Conference (ECC), 2016 more… BibTeX