Virtual and Mixed Reality Training Environments for First Responders - coupling CFD and ABM to VR/MR

AUGGMED (AUtomated serious Game scenario Generator for MixED reality training) is part of the European Commission’s Horizon 2020 programme (AUGGMED Project reference: 653590).

The aim of AUGGMED was to develop a serious game platform to enable single and team-based training of end-users (e.g. firefighters, police, counter-terrorism, medics, etc) with different level of expertise from different organisations responding to terrorist and organised crime threats within crowded places. The AUGGMED platform automatically generates non-linear scenarios tailored to suit the needs of individual trainees with learning outcomes that will improve the acquisition of emotional management, analytical thinking, problem solving and decision making skills. The game scenarios include advanced simulations of operational environments, agents, telecommunications and threats, and will be delivered through Virtual Reality (VR) and Mixed Reality (MR) environments with multimodal interfaces.

The environment couples advanced simulations of crowds, produced by the FSEG software buildingEXODUS, hazardous fire environments generated using the FSEG CFD fire simulation software SMARTFIRE and explosion modelling capabilities. The users are effectively immersed within the modelling environment produced by the agent based and CFD models, and can interact with the agents, the agents respond to their actions.

The environment has three levels:

Level 1: Interactive VR. Trainee uses mouse + keyboard to move around and interact with VW, views interaction on computer screen. This capability is demonstrated using a terrorist attack on an airport, with armed tactical response personnel attempting to find the marauding armed terrorist. The terrorist has detonated an improvised explosive device which starts a fire within the airport. Two police play the role of the armed tactical response unit while another police officer plays the role of the terrorist.

Level 2: Immersive Interactive VR. Trainee uses immersed VR head mounted display and hand controllers. Limited mobility. Tactile feedback possible via optional haptic vest. This capability is demonstrated using a medical first responder example. An explosive device has been detonated within an underground station and two paramedics respond and must perform triage on the scene.

Level 3: MR environment, training on site, full mobility with tactile feedback via optional haptic vest. Can have mixed VR/MR trainees. This capability is demonstrated using a terrorist attack on a passenger port facility (a real facility). Armed special forces enter the port facility and attempt to locate and neutralise the armed terrorist. Actual special forces personnel test the facility within a real passenger port facility.

All three levels are described in the presentation.

In addition, towards the end of the presentation, three new FSEG modelling capabilities are described. These are:
(1) matEXODUS: this is a version of the EXODUS software that can simulate the impact of marauding armed terrorists in crowded places.
(2) SMARTFIRE-pathogen: this is a version of SMARTFIRE that has been extended to simulate the dispersion of respiratory aerosols within ventilated spaces. The water mist model within SMARTFIRE has been modified to represent respired droplets and aerosols. To represent the wake produced by moving people within SMARTFIRE a new capability to simulate moving obstacles, using the Immersed Boundary Method has been developed. The software can model the dispersal of aerosols either as a scalar or by using the droplet model.
(3) Physical distancing within buildingEXODUS: The behaviour rules within buildingEXODUS have been modified to enable agents to maintain a Specified Distance Apart (SDA). To achieve this, agents are prepared to divert their path slightly. Using this approach, we can explore the impact of imposing SDA (e.g. 2.0m, 1.5m, 1.0m) and other physical separation methods on people flows, throughput and operational efficiency. The severity and duration of proximity breaches can be measured using a new ‘dose concept’ providing a means for evaluating the effectiveness of implemented mitigation strategies. The water mist model within SMARTFIRE has been modified to represent respired droplets and aerosols. To represent the wake produced by moving people within SMARTFIRE requires a new capability to simulate moving obstacles. This is being developed using the Immersed Boundary Method.