Summary
The aim of the research project, which is being carried out at FHNW in collaboration with SNCF Réseau, is to use artificial intelligence (AI) and an integration of multimodal data (multispectral satellite time series, aerial images, agricultural and climatological information, as well as data from terrestrial sensors) to model the residence probabilities of wildlife in the vicinity of the track infrastructure of the SNCF network.
The resulting predictive habitat models will help to better prevent incidents, for example through appropriate construction measures in specific risk areas.
In this Big Data project, the latest Machine Learning technologies are to be applied in order to process the extremely exhaustive and demanding data sources and establish profitable correlations from the processing.
The project is based on many years of recognized experience at the FHNW Institute of Geomatics in AI-based approaches to detection, classification tasks and modeling in the field of spatially and temporally referenced geodata.
Introduction
In a collaborative effort the researchers Adrian Meyer (Geodata Scientist and Animal Biologist) and Denis Jordan (Professor for Applied Mathematics) launched a project in 2023 together with Pascal Baran at SNCF Réseau, the infrastructure branch of France’s national railway society. The team is harnessing the power of machine learning to tackle a growing problem on the tracks: Wildlife-related accidents.
Accidents involving wild animals, particularly boars and deer, pose a significant risk to the extensive French railway network. They not only cause ecological, safety and property concerns but also contribute to significant financial costs, estimated at up to €100’000 per incident on the high-speed lines, while totaling over 200’000 minutes of schedule delays each year. Traditional methods for avoiding such accidents, such as fencing, wildlife crossings, and scaring devices are resource-intensive and not always effective, as it’s challenging to predict wildlife hotspots.
However, the team aims to revolutionize this process by creating a comprehensive wildlife accident risk prediction system. Utilizing a range of data sources, they are developing a dynamic predictive model that uses novel machine learning algorithms to assess the risk of animal presence in specific areas. This information will allow for more precise, cost-effective interventions, potentially reducing accidents and boosting network reliability and safety.
The models will integrate imagery from aerial and satellite remote sensing datasets, track-based mobile mapping photography, and infrared imagery from drones and camera traps. This data will be used to train a predictive multimodal fusion model that generates localized risk scores, highlighting potential wildlife hotspots. When converted into maps, these risk scores will provide a vital tool for planning mitigation actions.
This project is a response to the recent sharp rise in wildlife incidents on France’s railways, which comprise around 30’000 km of lines. The country-wide scale calls for modern geospatial database structures employing effective Big Data strategies. With around 1’500 regularity incidents linked to wildlife encounters impacting roughly 8,000 trains as of 2020, the SNCF has recognized the need for an innovative solution.
The ultimate goal is a flexible software prototype capable of predicting animal presence across large portions of the SNCF network. It will calculate a collision risk factor for small scale sections on each track, providing a multitemporal localized information basis for processes like barrier constructions, implementation of wildlife crossings, or interventions such as scaring devices and population management.
The results generated in this project will mark a significant step forward in the intersection of technology and wildlife management, while enhancing the safety and reliability of the French railway operations.
Fig. 1: Camera traps are a proven tool for monitoring wildlife abundance.
Fig. 2: Muddy sediments deposits serve as wildlife crossing indicators.
Fig. 3: Drone surveys enable terrain models to be collected in sensitive areas and wildlife to be observed directly using thermal imaging cameras.
Fig. 4: Mobile Mapping images (Imajnet) can be used to identify wildlife crossings particularly frequented by gravel coloration.
Fig. 5: The freely available high-resolution multispectral data from ESA’s Sentinel-2 satellite program provide an opportunity every six days to calculate a classifiable landscape model for the whole of France.
Fig. 6: Schematic overveiw of multimodal AI prototype (French version, status: June 2023)
Method
Our system accepts spatio-temporal data to generate risk forecasts over several time horizons (Fig. 6). Its robustness and high level of automation enable the production of a prototype for identifying critical incident points. Key steps include data compilation, pre-processing, selection of classification systems and evaluation of algorithmic approaches, with particular attention to ease of use and scalability.
To predict risk levels associated with wildlife accidents on the rail network, we couple automated classifications of terrestrial imagery (Fig. 4) and remote sensing (Fig. 5) with Machine Learning models. In the presence of fluctuating variance, convolutional neural network models – e.g. new architectures with short-term memory (CNN-LSTM) – outperform traditional regression models.
We will study early, late and hybrid fusion approaches to identify the most effective risk prediction model. Early fusion simplifies the training process by using a single model, but the results may lack interpretability. Late fusion provides modality-specific decisions and handles missing data well, but requires extensive feature engineering and multiple models. Hybrid fusion combines early and late fusion, delivering comprehensive results at the cost of increased complexity.
Expertise in the local rail network and animal habitats (Fig. 7) is essential to validate the accuracy of predictive results and the relevance of reference input data. We use mud marks on railroad tracks (Fig. 2) as indirect clues to wild boar crossing, validated by photographic traps (Fig. 1).
Drones equipped with infrared thermal imaging and zoom cameras (Fig. 3) are used to monitor wildlife behavior and population, either by counting animals in flight, or by analyzing images with computer vision neural networks.
Fig. 7: Early prognostic heat map for the evolution of incident intensity forecast for 2024. Autoregressive integrated moving average (ARIMA) based on 2015-2023 incident data resampled to 20x20km resolution.
Color code: Blue – Fewer incidents expected; White – No change / No data; Yellow – More incidents expected; Red – Strong intensification expected.
Results
The project will provide robust, accurate and timely forecasts of the risk of accidents involving wild animals (particularly wild boar) on the French rail network. This information, derived from sophisticated machine learning models and multimodal data fusion, will enable SNCF to better understand the hotspots of incidents. This could lead to significant cost savings by reducing the number and severity of wildlife incidents, as well as improving operational efficiency.
By facilitating more informed decision-making, the system will help SNCF to better allocate resources for infrastructure construction projects, such as fencing and wildlife crossings. In addition, the model’s atlas of predictions can serve as an essential input for wildlife management strategies, helping SNCF to maintain a harmonious relationship with the surrounding environment.
In the future, this project could serve as a model for new target species or other frequently encountered environmental challenges. As the system matures and more data is collected, it will continually learn and refine its predictions, further increasing its value to SNCF.