Worst Case Analysis: A Video Analysis Approach to Injury Potential in Traffic

Introduction

Safe System (also known as Vision Zero in Sweden and Sustainable Safety in the Netherlands) is regarded as best-practice within traffic safety management. The appeal of the approach is its strong theoretical and ethical foundation, and substantial empirical evidence has been accumulated proving that, if properly implemented, it is very efficient in reducing fatalities and injuries from crashes on the road (Edvardsson Björnberg et al., 2023; Elvik, 2023; Elvik & Nævestad, 2023; Khan & Das, 2024). The fundamental principles of Safe System can be summarised as follows (Corben et al., 2022; Edvardsson Björnberg et al., 2023; SWOV, 2018; Tingvall & Haworth, 1999; Wegman et al., 2023; Weijermars & Wegman, 2011):

• Human factors. Humans make mistakes, and have physical and mental limitations, which the traffic system design needs to take into account.

• Shared responsibility. Responsibility for safety should not lie exclusively on the individual road users; all stakeholders, such as road authorities, vehicle manufacturers, enforcement agencies, etc. work together to achieve a safe traffic system.

• Systematic and proactive. A holistic and evidence-based approach should be applied to identify and address safety risks before they result in crashes and injuries.

• Forgiving and protective. Severe injuries and fatalities in traffic are prevented by reducing the kinetic energy to which the road users are exposed, by limiting speeds and providing adequate protection for all types of road users.

• Aspirational and ethical. The operational goals for traffic safety must be ambitious and motivating, and in line with the values of the society; the long-term goal is a total elimination of fatalities and serious injuries in traffic (Vision Zero).

Aim

Translation of these theoretical principles into practical guidelines is not always straightforward (Green et al., 2022). This paper focuses on one aspect of Safe System, limitation of exposure to excessive kinetic energy, and proposes a trajectory-based method for infrastructure safety assessment of road user interactions. Specially, this paper aims to develop a method that uses microscopic trajectory data from processed video-data to estimate the exposure to excessive kinetic energy in typical traffic conditions at almost any location with road user interactions. Note that the processing of video-data, including video recording, video calibration, object detection and tracking will not be covered in the paper but are necessary prerequisites to use the method. Also note that the proposed method will only include an analysis of interactions between road users and will not include crashes involving only one road user.

Theoretical background

Traffic casualties ( C ) can, on a conceptual level, be seen as a product of three components—exposure, i.e. amount of opportunities for a crash ( E ), crash risk per exposure unit ( A / E ), and the risk of a casualty given a crash ( C / A) (Nilsson, 2004; Rumar, 1999):

$$C = E \cdot \frac{A}{E} \cdot \frac{C}{A}$$

To reach zero casualties, at least one of these components must be zero. For interacting road users, total separation of road users is not a totally unplausible approach (e.g., pedestrians and cyclists are prohibited on motorways), but in many cases a certain degree of interaction between road users is unavoidable. As long as such interactions are allowed, there will also be a chance of a mistake in an interaction (see human factors principle of the Safe System), and thus a non-zero risk of crashes. Therefore, a way to reach zero casualties, from interactions, is to ensure that crashes do not lead to severe outcomes in the form of injuries and fatalities. Again, this is in line with the Safe System principle of not exposing road users to external forces that exceeds the limits of biomechanical tolerance and lead to injuries (SWOV, 2018).

The traditional methods in infrastructure safety assessments primarily focus on measuring the frequency of crashes, injuries, or their surrogates (e.g., traffic conflicts (Johnsson et al., 2018)). As the transport system becomes safer, black spot analysis becomes more and more difficult and costly. Already in countries like Sweden or Netherlands, elaborate crash modelling techniques are rarely used for the simple reason that the incident counts at any single location are too low. The necessary dataset size is either no longer feasible due to the data collection costs, or are not available, even when the entire country road network is considered.

Therefore, and more in line with the Safe System thinking, the question needs to change from ‘how many injuries per time unit could we expect here?’ to ‘what is the potential for an injury in the long run?’.

Video analysis and road safety

Using video to analyse road safety is not new. and has been made relatively common by recent improvements to techniques used to acquire trajectory data (Abdel-Aty et al., 2023). This type of video analysis focuses on the application of microscopic trajectory data. Trajectory data consist of a sequence of points or bounding boxes (2D or 3D rectangles that enclose the entire road user) that represent the position of vehicles at a different time throughout a video recording. Through a sequence of processing steps, see Abdel-Aty et al. (2023), it is possible to estimate the position, speed, acceleration, and type of road users from the video-data. Note that the accuracy of the trajectory data is important and depends on several factors, including the position of the camera, lighting condition, vehicles with unusual shapes and adverse weather (Abdel-Aty et al., 2023; Li et al., 2020).

The simplest form of this type of analysis is traffic counting and speed measurements (Buch et al., 2011; Dahl & Javadi, 2020). These measurements are critical to any road safety analysis and are highly relevant when evaluating the safety of any road section. Another commonly used method to evaluate traffic safety is incident-based, otherwise called surrogate measures of safety or traffic conflicts. This method is based on the idea that it is possible to identify safety-critical events, such as near-crashes, in traffic and use these dangerous interactions as surrogates for crashes. The reviews by Johnsson et al. (2018) and Arun et al. (2021) provide comprehensive overviews of the many aspects of this methodology. Finally, it is also possible to study other behavioural observations in traffic and use these to evaluate road users’ interactions with the infrastructure. The review by van Haperen et al. (2019) found many such behavioural parameters, such as yielding behaviour, turn indicators, overtaking behaviour and violations, etc.

Looking back at the guiding principle of this paper, all of these methods have limitations. The speed measurements do not consider differences in speed or directions of different traffic streams, the traffic conflict approach focuses on rare critical events and does not consider the general patterns of traffic conditions, nor the speed of any vehicle not involved in a conflict. Furthermore, the conflict approach typically attempts to directly estimate crash frequency which is different from the potential safety. The many other behavioural observations focus on specific types of behaviours which relate to safety in different ways but typically not related to the approach proposed in this paper. Note that all these methods are useful in their own way and are valuable tools for understanding and studying road safety, but they lack the direct ability to capture the potential for safety as described by SWOV (2018).

Safe speeds and injury models

Safe speeds are the travel speeds that minimise the risk and severity of crashes, considering the road environment, the vehicle characteristics, and the human factors. Many studies have analysed crashes and developed various models to predict the outcome of crashes based on their mass, direction, and speed (Doecke et al., 2021; Gabauer & Gabler, 2006; Jeppsson & Lubbe, 2020; Jurewicz et al., 2016; Lubbe et al., 2022; Rosén et al., 2011; Shannon et al., 2020). Injuries are typically defined using the Abbreviated Injury Scale (AIS) (Greenspan et al., 1985), or its maximum value (MAIS), which represents the threat to life associated with the injury rather than the comprehensive assessment of the severity of the injury. A MAIS value of two indicates at least a moderate injury and a value of three indicates at least a serious injury.

There are at least two commonly used metrics in these types of injury models, closing speed and Delta-V. Closing speed, or relative speed, is the vector sum of the speed of two vehicles. For example, two cars both travelling at 50 km/h colliding head-on have a closing speed of 100 km/h while the same two cars traveling in the exact same direction will have a closing speed of 0 km/h. Delta-V refers to the change of velocity experienced by a road user during a crash. Both indicators can estimate the severity of an impact and the potential for injury following a crash using different injury prediction models. These metrics have also been used in traffic conflict studies to develop indicators which can consider this aspect of safety-critical events (Shelby, 2011).

Injury risk curves, such as those developed by Lubbe et al. (2022), can estimate the potential for injury based on the closing speed of two vehicles, with regard to the type of road user (motor vehicles, bicyclists, pedestrians, etc.). These models can be used to estimate the potential safety of any location by studying the movement of traffic at a location and referring to an injury model to estimate the potential risk of injury should a crash occur.

On a large scale, this is the approach suggested by the Safe System framework proposed by Porter et al. (2021) and the method proposed by Jurewicz and Sobhani (2016). Porter et al. (2021) incorporated injury risk curves based on Delta-V, combined with conflict points and speed measurements to provide a safety assessment of intersections based on the design. The main difference between these methods and the method proposed in this paper is the use of video observational data which allows for an analysis of not only the designed elements but also the actual movements of road users which can reveal information about unplanned and undesired movements.

Methodology

By combining microscopic trajectory data from video analysis with existing injury risk curves, it is possible to estimate the potential safety of all interactions between road users at any road section. The basic idea is to calculate the potential for injuries (MAIS3+) between road users, which tracks have at least one intersection point with another road user track, regardless of when they appeared in the video footage. We can then estimate the potential safety of a specific location by finding the highest injury within the vicinity of the location of interest. This value will then provide an estimate of the worst-case scenario (i.e., the potential), at that specific location considering the road users’ differences in speed, direction, and type of road user, depending on the characteristics of the injury model used in the analysis.

The approach consists of the following general steps:

1. Acquire trajectory data of road users from the studied locations

2. Identify all intersection points between all road user trajectories in the dataset and calculate the indicators used in the injury risk model between each combination of trajectories

3. Calculate the potential for safety using the injury risk curves with regards to the involved class of road users (motor vehicle, bicyclists, pedestrians, etc.)

4. Divide the location into a grid and identify the highest injury risk calculated in each cell of the grid

5. Visualise the grid in a heat map to provide an assessment of the potential safety of the studied area

For this study, intersection points have been identified as all paths within 1 metre of each other and the grid used to create the heat map also has a resolution of 1x1 metre. All calculations have been done in the MATLAB software. No consideration has been taken to the manoeuvres of individual road users, meaning that road users travelling on similar paths will generate many intersection points. However, they typically have a lower closing speed as their direction and speed are similar.

Note that the method itself does not require the trajectory data to be acquired by a specific method. Any traffic camera, drone, or other surveillance camera could be used to record the video data. There are also many different computer vision approaches to identify and track road users in videos (Abdel-Aty et al., 2023).

Showcase locations

To illustrate the proposed method and discuss its potential as a safety assessment tool, this paper uses trajectory data from the InD dataset, which is an open dataset of naturalistic vehicle trajectories recorded at German intersections using a drone (Bock et al., 2020). The dataset contains motor vehicles (cars, trucks, buses) along with both bicyclists and pedestrians. Figure 1 below shows the camera view from the two locations included in this paper (Location 1 and 3 in the open dataset are Location 1 and 2 in this paper). The dataset for Location 1 contains a total of 2,347 road users (2,098 motor vehicles, 86 bicyclists and 163 pedestrians). The dataset for Location 2 contains a total of 6,235 road users (2,436 motor vehicles, 1,700 bicyclists and 2,099 pedestrians). The data provide trajectories at 25Hz with a positional error less than 0.1 metres (Bock et al., 2020).

According to Bock et al. (2020), all intersections are located in the city of Aachen, Germany, are unsignalised and have a speed limit of 50 km/h. They state that Location 1 (Neuköllner Strasse, Aachen) is in an industrial area next to a bus depot. The location has a lot of transit traffic and many buses. There are also a significant number of pedestrians walking to the two bus stops. Location 2 (Frankenburg) is near the city centre close to several parking lots. The traffic is characterised by many bicyclists and pedestrians together with vehicles moving in and out of parking spaces.

Figure 1.Camera view from the two locations. Neuköllner Strasse, Aachen (left), Frankenburg, Aachen (right)

Injury risk curves

The injury risk curves used in this paper are the ones developed by Lubbe et al. (2022). However, other injury risk models could easily be used instead depending on location, environment, road users, etc. The curves are logistic regression models which calculate a probability of MAIS3+ injury based on the closing speed of the road users combined with the age of the road user (set to 40 years in all calculations). The models relate to impacts with the front of a passenger car. The model does not consider gender differences and is not intended to model injury risk for children. Equations 1, 2 and 3 show the models used for motor vehicles, bicyclists and pedestrians and Figure 2 shows the probability of injury based on closing speed for the three models.

$$P_{MV} = \frac{1}{1 + e^{7.654 - 0.041 \ast CS - 0.021 \ast 40}}\tag{1}$$

$$P_{BIC} = \frac{1}{1 + e^{7.467 - 0.079 \ast CS - 0.047 \ast 40}}\tag{2}$$

$$P_{PED} = \frac{1}{1 + e^{6.190 - 0.078 \ast CS - 0.038 \ast 40}}\tag{3}$$

Figure 2.MAIS3+ injury risk curves for motor vehicles, bicyclists and pedestrians for road users at 40 years of age (Lubbe et al., 2022)

Results

The results consist mainly of three different heat maps (one per class of road user) showing the injury potential as a value between 0 and 1 throughout the studied location. Only intersection points involving at least one motor vehicle have been included as the injury model does not include any crashes between vulnerable road users. Heavy vehicles, such as buses and trucks, have been considered as cars for the calculation. A high value indicates that the speed of the road users is high enough to create the potential for injuries at that location. The sections below show these heat maps in conjunction with a visualisation of the unprocessed trajectory data. Note that only injury potential values higher than 0.1 are visualised to make the heatmaps less cluttered.

Location 1

The dataset for Location 1 contains a total of 2,347 road users (2,098 motor vehicles, 86 bicyclists and 163 pedestrians). Figure 3 shows the resulting heat maps from Location 1 in conjunction with the unprocessed trajectory data, divided by the class of road users. The result indicates that the potential for injuries is very low for motor vehicle occupants but is increased in some areas for both bicyclists and pedestrians. The result for pedestrians is especially clear with the crossings on the lower road showing a significant potential for injuries. This is especially relevant given the proximity of Location 1 to the bus depot which seemed to generate pedestrian traffic that does not use the nearby crossing. The result for bicyclists is quite scattered, probably because they are mixed with the motor vehicles on the road. The area with the highest risk seems to be located where the bicyclists turning left intersect with the motor vehicles travelling in the opposite direction.

Figure 3.Results from Location 1

Left: raw trajectory data divided by the class of road user (top: motor vehicles, middle: bicyclists, bottom: pedestrians)
Right: heat map visualising the potential safety throughout the location (divided by class)
Note: values below 0.1 are not displayed

Location 2

The dataset for Location 2 contains a total of 6,235 road users (2,436 motor vehicles, 1,700 bicyclists and 2,099 pedestrians). Figure 4 shows that the injury potential at this location is more spread out compared to Location 1. This does not necessarily mean that Location 2 is “worse” than Location 1, only that the travel path of the road users overlaps across a larger area. In comparison, the maximum values seem to be somewhat lower compared to the other location, possibly due to lower motor vehicle speeds. Interestingly, the two areas with a higher potential for both bicyclists and pedestrians are on the left side of the intersection and within the parking area to the right and not within the intersection itself. One potential explanation could be that both pedestrians and bicyclists in these areas take shortcuts through the lane for cars travelling in their opposite direction which would create higher closing speeds. Within the intersection itself, we could expect that the pedestrians and bicyclists do not deviate from the expected paths, which results in them being separated from traffic in the opposite direction.

Figure 4.Results from Location 2

Left: raw trajectory data divided by the class of road user (top: motor vehicles, middle: bicyclists, bottom: pedestrians).
Right: heat map visualising the potential safety throughout the location (divided by class).
Note: values below 0.1 are not displayed

Discussion

Potential safety analysis, as described in this paper, can provide a comparatively quick analysis of the potential for injuries in an area based on observed traffic. The analysis is not complicated as it is basically a combined speed analysis which considers different types of road users. Indeed, it is possible to do a similar analysis by combining speed maps with information about road user movement. However, the proposed method simplifies the analysis into a single figure which is simpler to view and understand. The analysis identifies the areas where there is some potential for injuries. These areas might then require further analysis to properly evaluate all dimensions of road safety. However, the most relevant parts of the analysis are the areas which have a low potential for injuries. These areas can be considered safe, at least during the observational period, and might be of limited interest for more safety-related analysis or measures to improve safety. Note that the size of the areas only reflects where the trajectories of road users overlap with enough kinetic energy. A larger area does not necessarily reflect a more dangerous location, only that the potential for injuries are less concentrated.

The ability to evaluate areas as safe is the most useful part of the methodology and can be used to evaluate infrastructure alone or as a first step in which further analysis focuses on the areas with higher potential for injuries. For example, any additional safety-related analysis or infrastructure at Location 1 should focus on the three areas where pedestrians pass the bottom road and not cross at the top road. The result of such analysis can be used also in the decision-making phase for the implementation of small safety treatments. Especially in parts of the road infrastructure where there are no evident issues highlighted by road casualties but have potential for improvement.

Exposure, Risk and Consequence

The resulting images do not show safety, only potential safety (i.e., injury risk in the worst-case scenario). The figures showing the raw trajectory data clearly show how the potential safety analysis does not consider the amount of traffic travelling throughout a specific area of the locations. For example, the crossing towards the top of Location 1 has a low potential for injury when compared to the crossing to the right. However, the number of pedestrians passing the top crossing is considerably higher than the right one; this could mean that even though the potential is higher on the right crossing, we might still expect more actual injuries at the top crossing, simply due to the difference in exposure.

As mentioned in the introduction of this paper, the risk dimension was ignored in the analysis. The safety level at a location also depends on the behaviour of the road users and other infrastructure elements that affect the risk that an encounter between two road users will result in a crash. For example, the mixed bicycle traffic at the studied locations increases the potential for injuries due to the higher relative speeds between the bicyclists and the motor vehicles. However, to gain a complete understanding of the safety of these roads, the behaviour of the road users when they interact must be considered. Note that the principle of sustainable safety and safe speed works in the same way. They focus less on exposure and risks and more on limiting the potential of any injuries that could feasibly occur at the location. Therefore, the limitation of the method to be used to judge safety does not diminish the value of the results, rather it is a consequence of choosing to focus on the potential safety. Any analysis of potential safety should consider this limitation, and the other dimensions of safety are important to provide a complete analysis.

One further note related to exposure is that the analysis can only provide insight into the potential safety concerns at the moments recorded in the video footage. If the observational period is too short, infrequent movement patterns might be missed. It is also possible that changes in exposure levels might affect the movement of the road users, such as pedestrians choosing to cross in different locations depending on the level of traffic. One possible way to counteract this could be to do a separate analysis for different levels of exposure or different time periods. Finally, it should also be noted that the method cannot analyse crashes involving only one road user and can only be used to study interactions between road users.

Additional analyses

The results following the basic methodology presented in this paper are several heat maps showing areas of interest at the studied locations. The results could be expanded with more spatial and temporal information. For example, information about the manoeuvres of the various road users could be added to better understand how the potential is divided. Temporal information could also be added to generate different images in various conditions. For example, specific heat maps for time periods close to the arrival and departure of buses could be created at Location 1 to investigate how buses might interact with the potential safety of the pedestrians.

Finally, the injury risk curves used could be replaced by other models based on different indicators, such as Delta-V, with different characteristics and other limitations which might be more suitable for specific scenarios. The main benefit of using the injury model is that they provide a robust way of estimating if speeds are too high depending on the characteristics of the model being used. It would also be possible to generate different results by different road user characteristics (e.g., age) or other characteristics included in an injury model. Another slight alteration would be to not use the absolute highest value from each cell and instead use the 95th or 99th percentile values of injury risk; this would remove any outliers from the observations and would also help to alleviate tracking errors from the video analysis. Since the method relies on the highest values, the quality and accuracy of the trajectory data become important. We recommend having an accuracy like that of the InD dataset (0.1 metre). In the same way, it would also be possible to remove any cell data that contained very few observations. However, any outliers detected during the observational period might be common enough to include in the analysis depending on the length of observation.

Conclusion

This paper presents a comparably simple method for analysing the potential for injuries in any traffic scenario involving interactions using trajectory data from typical traffic conditions. The method focuses explicitly on the worst possible consequence of the observed traffic and does therefore not consider the dimensions of risk and exposure. While this is a limitation in the sense that the proposed method cannot provide a complete analysis of traffic safety, it can provide an excellent starting point for any traffic safety analysis. It is especially useful for its ability to identify critical areas for further study. The method can also be used to identify the effectiveness of safety interventions, by applying it to the after-condition of a treatment and identify any possible “migration” of hazardous areas. It is also quite flexible in that the underlying injury model can be replaced by models more suitable for any specific study without changing the way the method operates.

Acknowledgements

Thank you to both Carmelo D’agostino and Hampus Norén for helpful discussions and input.

Author contributions

Carl Johnsson: conception, design, execution, analyses, interpretation, writing
Aliaksei Laureshyn: conception, writing, interpretation

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability statement

The data used for the paper comes from the publicly available inD-dataset available for non-commercial use online at https://levelxdata.com/ind-dataset/

Conflicts of interest

The authors declare that there are no conflicts of interest