Safety and Robustness

We are interested in formal synthesis, automatic abstraction, guarantees, testing, simulation, code analysis, chaos engineering, set-based methods, and model checking. We use them to improve the safety and robustness of all sorts of applications, for instance, autonomous driving, robotics, healthcare systems, and java programs. While in our monthly seminars, we discuss novel tools and latest developments, reproducibility, verification, modeling, and methodologies.

Slack Channel: #ctc_safety_and_robustness_of_autonomous_systems
Github: https://github.com/wasp-sras
Gather Town: https://gather.town/app/eNJcKaW5tbxHsi6x/wasp-sras
To join us, please contact the cluster leader with your email address, github username and affiliation.

Cluster activities

Monthly reunions at Gather Town
We are currently exploring two topics in separate focus groups
- Safe autonomous driving
- Safety despite uncertainty

Contact all Members

Mailing List: CTC_SafetyRobustness@wasp-sweden.se

Cluster Leader

Ricardo Caldas

Cluster leader of Safety and Robustness of Autonomous Systems

Send Email

Active Members (12)

Hannes Eriksson

Affiliation: PhD Student @ Zenseact AB / Chalmers, Department of Computer Science
Supervisor: Christos Dimitrakakis
Research Abstract: My main research is into sequential decision-making problems for autonomous driving. In particular, we are looking at how the decision-maker can adhere to certain constraints which form the risk-sensitive decision-making problem. To do this, we use Reinforcement Learning and place importance on different uncertainties. For example, we consider the uncertainty that comes about due to us not knowing the problem parameters (epistemic risk) and the inherent uncertainty in a model (aleatory risk). By leveraging methods that take these uncertainties into account, we can arrive at a decision-making agent that in some instances more accurately reflect the behavior we would want such an agent to have.
Research Keywords: Reinforcement Learning, Risk-sensitive decision-making, AI Safety
Email: hannese@chalmers.se

Jonas Krook

Affiliation: Zenseact, Chalmers
Supervisor: Martin Fabian, Sahar Mohajerani, Zhennan Fei
Research Abstract: My research is focused on synthesizing correct-by-construction decision logic for autonomous vehicles. The main driver is that it is difficult to ensure correctness of manually implemented decision logic operating in adversarial environments. One way this synthesis can be performed is to employ formal synthesis methods; they provide guarantees that the synthesized controllers are correct with respect to a formal specification in a formal model of the environment. The main research problems are how to do these formalizations of the safety-critical requirements and the environment of the autonomous vehicles.
Research Keywords: Autonomous driving, safe decision making, formal synthesis
Email: jonas.krook@zenseact.com

Jens Henriksson

Affiliation: Semcon & Chalmers University of Technology
Supervisors: Stig Ursing (Semcon), Christian Berger (Chalmers)
Abstract: Testing of ML applications and ensuring they are operating correctly is an existing challenge, especially when it comes to safety critical applications. Ensurance functionality requires both offline tests and online techniques. One techniques that can act both offline and online is adapting outlier detection to act as a safety measure. This research focus on evaluating and improving outlier detection techniques and shows how to properly use them on various scenarios.
Research Keywords: Machine learning safety; out-of-distribution detection;
Email: jens.henriksson@semcon.com

Long Zhang

Affiliation: KTH Royal Institute of Technology
Supervisor: Martin Monperrus
Research Abstract: I’m mainly focusing on software resilience problems. It’s impossible to predict every failure or unanticipated situation of a system, especially when it is deployed into production. So it’s important to improve a system’s resilience, enabling it to bear and self-heal the perturbations. For example, I’m using chaos engineering to address such problems. Chaos engineering is the practice of experimenting on a distributed system in order to build confidence in the system’s capability to withstand unexpected conditions in production. In other words, breaking things on purpose.
Research Keywords: chaos engineering, fault injection, reliability, antifragile
Email: longz@kth.se
Linkedin: https://www.linkedin.com/in/gluckzhang/
Github: @gluckzhang

Magnus Gyllenhammar

Affiliation: Zenseact/KTH
Supervisor: Gabriel Rodrigues de Campos (Zenseact), Martin Törngren (KTH), co-supervisor Jonas Fredriksson (Chalmers)
Research Abstract: In my research I focus on the challenge of providing a safety argumentation for automated driving systems (ADS), specifically by trying to answer the research question: What are efficient strategies for safety assurance of ADSs? They way I intend on approach the problem is by analysing the traffic challenges by ways of statistical models. These models can then tell us more precisely what the ADS (1) needs to handle, but also (2) what it does not need to handle. This approach thus has the opportunity to provide a more efficient approach to achieve system safety compared to other more rigid models for safety (e.g. formal methods such as Mobileye’s RSS).
Research Keywords: Safety, Automated driving systems, Modeling, functional safety, safety assurance, scenario modeling
Email: magnus.gyllenhammar@zenseact.com
Linkedin: https://www.linkedin.com/in/maggyl/

Masoud Bahraini

Affiliation: PhD student at Chalmers, Electrical Engineering Department, Mechatronics Group
Supervisors: Paolo Falcone and Henk Wymeersch
Research Abstract: My main research is focused on joint control and communication schedule for networked control systems. In particular, I study uncertain systems with hard state and input constraints. These constraints can represent physical limitations of the systems or could be imposed due to safety requirements. The design goal is to robustly preserve the constraints’ satisfaction for all systems in presence of communication-link imperfections, such as bandwidth limitation and packet loss. Typical tools used in my research include reachability analysis, model predictive control, and Windows Scheduling Problem among other things.
Research Keywords: Networked Control Systems, Model Predictive Control, Robust Invariance, Communication Scheduling
Email: masoudb@chalmers.se

Matteo Tadiello

Affiliation: Theoretical Computer Science at KTH Royal Institute of Technology
Supervisors: Elena Troubitsyna
Research Abstract: My research is about safety and reliability of intelligent autonomous systems.
Systems like autonomous vehicles need to operate in complex environments with uncertainty in the sensors and with multiple other systems. Machine learning techniques are essential to generalize and operate in these environments. However, proving the safety of these models or train models that are intrinsically safe is often difficult to obtain. Trying to solve these problems is the focus of my research.

In particular, I’m focusing on the decision-making process and the development of Deep Reinforcement Learning techniques that generate safe agents. My main application domain is Autonomous Vehicles.
Research Keywords: Reinforcement Learning, Deep Learning, Safety and Reliability, Autonomous Vehicles
Email: tadiello@kth.se

Qunying Song

Affiliation: PhD Student @ Lund University, Department of Computer Science
Main supervisor: Per Runeson
Research Abstract: My research generally focuses on approaches and techniques for software testing of autonomous systems. Specifically, we are now working with autonomous driving systems by partnering with Volvo Cars and are experimenting as well as exploring the worst-case test scenario identification approach. In detail, the approach starts by analyzing the system under test and identify the relevant parameters as well as criticality objectives; then a first initial set of scenarios will be generated and executed in simulation; lastly, the optimization application is used to automatically explore and identify the most critical scenarios to facilitate testing in the naturalistic traffic.
Research Keywords: Software Testing, Scenario Optimization
Email: qunying.song@cs.lth.se

Ricardo Caldas

Affiliation: PhD Student @ Chalmers, Department of Computer Science
Supervisor: Patrizio Pelliccione and Thorsten Berger
Research Abstract: My main research interest is assurance provision for systems that should self-adapt at run time. So far, we explored have (i) synthesis of goal-oriented controllers to mitigate uncertainty, (ii) apply AI to optimize performance in the synthesis of adaptation engines, (iii) synthesis of runtime monitors for artificially immune systems, and we are currently exploring (iv) how to apply control theory properties to software verification. All of these were applied to a system in the healthcare domain. Apart from that, I am also interested in testing for self-driving vehicles and specification language engineering for multi-robot systems.
Research Keywords: Self-Adaptation, Control Theory, Software Engineering
Email: ricardo.caldas@chalmers.se

Shamisa Shoja

Affiliation: Linköping University
Supervisor: Daniel Axehill
Research Abstract: My research is in the area of optimization control and my main subject is the complexity certification of Mixed-Integer Linear Programming (MILP) and Mixed Integer Quadratic Programming (MIQP). An important application of such problems is on designing Model Predictive Control (MPC) for hybrid systems in which both continuous-time and logical decision variables interact in the system. My objective is to certify these mixed-integer problems by finding a method to analyze how many and exactly which sequence of sub-problems are solved to compute the optimal solution at each parameter of interest. By knowing this sequence of subproblems, a worst-case bound on iterations a solver requires to converge could be determined.
Research Keywords: mixed-integer programming, complexity certification, hybrid MPC
Email: shamisa.shoja@liu.se

Truls Nyberg

Affiliations: Division of Robotics, Perception and Learning, KTH Royal Institute of Technology and Autonomous Systems, Scania CV AB
Supervisor: Jana Tumova, Patric Jensfelt and Christian Pek
Research Abstract: My research is within situational awareness of autonomous vehicles (AVs). When driving, the task is to efficiently make progress towards some goal, without jeopardizing anyone’s safety or comfort. In my research, I use formal methods to enhance AVs’ abilities to plan safe trajectories and make risk-aware decisions.
Research Keywords: autonomous vehicles, autonomous driving, situational awareness, risk assessment, risk measures, decision making, motion planning, optimal control
Email: trulsny@kth.se, truls.nyberg@scania.com

Vandana Narri

Affiliations: Division of Decision and Control Systems, KTH
ATS Pre-Development & Research, Scania CV AB
Supervisor: Karl Henrik Johansson and Jonas Mårtensson
Research Abstract: The focus of my research is “shared situational awareness using V2X (vehicle to everything communication) for heavy-duty autonomous vehicles”. In most of the urban scenarios, the understanding of the environment is limited to information perceived by the local sensor which are mounted of the ego-vehicle and this information might be severely affected by occlusion caused by fixed obstacles or other vehicles along the road. My project focuses on how to automate efficient and safe operations in collaborative and dynamically changing situations. I have started exploring set-based methods for estimating states of the other road users present in the scenario. And also to analyze how to handle varying uncertainties in the sensor observations.
Research Keywords: Set-based Methods, estimation methods, situational awareness, guarantees for state estimations and safety (robustness of the framework)
Email: vandana.narri@scania.com, narri@kth.se