Data Collection, Analysis and Replay

In partnership with Ibeo, an urban vehicle was retrofitted with state of the art 360 degrees perception and communication technology. This vehicle perception and localisation capabilities were demonstrated at the ITS world congress in Melbourne (October 2016). This vehicle will collect a significant amount of naturalistic driving information and provide reference information for our fundamental research work in situational awareness, safety, machine learning and driverless vehicles. Large amounts of data are generated by the sensors and algorithms used in intelligent connected vehicles. In order to improve system performance and to facilitate work on more advanced algorithms these data must be collected and stored for later access. The manner in which this is done should facilitate meaningful future use of the data. We have made a number of our datasets from existing projects available for use by other researchers..


A probabilistic estimate of the pose and dynamic properties of a vehicle can be obtained by fusing information from GPS/GLONASS receivers and local reference sensors such as gyroscopes, accelerometers, wheel encoders and vehicle model constraints. One of the focuses in our group is the use of affordable sensors for implementation in vehicles, in particular retrofit installations.

Our localisation efforts include:

  • Sensors for acceleration, angular velocity, magnetic flux, etc
  • Global Navigation Satellite System (GPS, GLONASS, Galileo)
  • Augmentation with perception/maps

Localisation can be improved using map information. Our group has significant experience in the formulation and learning of map properties from collected position information. We are also looking to incorporate the available map databases (such as OpenStreetMap) as an additional source of map geometry and relationship data.

Intrinsic Detection

This term refers to the strategy of detecting nearby threats using sensors mounted to the ego vehicle such as cameras, lasers and radars. This form of threat detection requires sensing and the processing and interpretation of the sensor data to estimate the pose and dynamic properties of vehicles within sensor range. The ACFR has significant experience in the design and use of multimodal perception using all most type of standard sensors and development of new sensing in the area of Millimetre Wave Radar Technology. These developments have been applied to various industries such as defence, agriculture and mining among others.

Extrinsic Detection

This term describes the detection of nearby threats using a strategy of ego-localisation and broadcast of V2V and V2I messages of the vehicle state. This allows vehicles with compatible communication equipment to receive regular updates regarding potential threats in the nearby environment.

This strategy has several benefits, including transmitting information about the vehicle and driver that are not possible to observe using intrinsic detection. The vehicle control inputs such as indicators, accelerator and brake position can be broadcast, allowing algorithms to predict driver intentions.

In addition, the V2V strategy has the advantage of operating without a direct line of sight with the target vehicles, which is generally not possible with any intrinsic sensor. The disadvantage is that detection is only possible with other vehicles that have compatible equipment that is fully functional.

Our group has commercial experience with the design and implementation of V2V communications in an industrial environment. Recent work has focused on using a standard V2V message set (SAE J2735) which allows for interoperability between different OEMs. The management of vehicle state information of varying quality is an open research question.

Road and Area Learning

High quality road maps are desirable for localisation and trajectory estimation. Many parameters can be encoded into the map, such as velocity profiles, acceleration profiles, expected behaviour and other similar information. This can be used in the threat estimation through time to collision and anomaly detection by examining the range of typical behaviour given the location. We have significant experience in the formulation and use of maps based on collected information with several contributions published in this area.

Behaviour Learning and Analysis

There are several components to behaviour learning and analysis relating to both perception and comprehension components of situation awareness. Behaviour analysis is considered in two main categories - local driver behaviours and location specific behaviours.

Local driver behaviour involves research into modelling vehicle motion and other available information to estimate future driver behaviour. This includes work in segmentation of motion sensor data with machine learning to understand the high level motion patterns of the vehicle.

Location specific behaviours involve considering map information and adding location specific contextual information to learn the expected behaviour of a driver given the type of area in which they operate. This is important because what is considered safe rules of operation is highly dependent on the type of environment being considered (i.e. different rules for parking lot compared to a highway).

This can be used to evaluate the performance of drivers, and metrics and indicators of safety.

Threat Assessment

Threat assessment involves the estimation of the threat caused by vehicles and other road users in the local vicinity. We consider threat assessment in two categories: proximal safety indicators and detecting threats through behaviour analysis and the detection of anomalies.

Proximal safety indicators

Estimating the time to interaction between vehicles can be used as part of threat assessment. The potential motion of the vehicles is defined in an environmental model. The location of each vehicle on the map is determined based on the position and heading. The distance via the road network is then calculated based on the shortest path search over the map. This allows the estimation of time until vehicle interaction, and the more well known time to collision estimate based on velocity models for each road.

Behaviour analysis

Threats can be inferred by detecting anomalous situations by comparing expected behaviour, derived from predictive models, against actual behaviour. This give an an analysis of risk based on all available information, given physical limitations (road maps) and predefined road rules.

System Failure Characterisation

Vehicle systems operate under real world conditions and are prone to partial or complete failure. In order to identify and account for these failures we require an understanding of the failure modes and a way of predicting the reliability of ego information and information available from other sources.

Human Machine Interfaces

Improvements in driver situation awareness can only occur if the outputs of intelligent connected vehicle systems can be conveyed to the driver. The provision of timely, accurate, useful and appropriate information is a key concern in the design and implementation of human machine interfaces. Our group has significant expertise with development and deployment of safety systems in industrial and urban environments.


The development and deployment of autonomous systems required the introduction of the concept of integrity. The ACFR has been responsible for the development of high integrity navigation systems implemented in some of the largest autonomous operations in field robotics. The concept of integrity has been developed and applied to other areas such as perception, communication and positioning.

Automation and Augmentation

The implementation of autonomy in urban environments presents significant challenges. As the quality of the situation awareness information becomes more reliable/robust, the next stage is to evaluate the capability of the technology to transfer control of the vehicle from the driver. To remove control of the vehicle will require a comprehensive understanding of the position and sensor uncertainty, and an estimation of the full state of the hardware (such as sensors, antennas, processing units, etc). The concept of integrity is currently investigated for vehicle automation with successful demonstration of these capabilities in international collaborative projects, such as our collaboration with Renault.