2305/2305.03001.md

Title: Open Sensor Data for Rail 2023 This work was supported by the DZSF. This is not an official statement, guideline or directive of the German Federal Railway Authority. Human faces and license plates are sufficiently blurred in the images for anonymity. ©2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. DOI: 10.1109/ICRAE59816.2023.10458449

URL Source: https://arxiv.org/html/2305.03001

Markdown Content: Rustam Tagiew* Pavel Klasek Roman Tilly German Centre for Rail Traffic Research

Federal Railway Authority

Dresden, Germany

forschung@dzsf.bund.de

*Corresponding author Martin Köppel Patrick Denzler Philipp Neumaier Tobias Klockau Martin Boekhoff Digitale Schiene Deutschland

DB Netz AG

Berlin, Germany

{martin.koeppel, patrick.denzler,

philipp.neumaier}@deutschebahn.com

Abstract

To achieve a driverless train operation on mainline railways, actual and potential obstacles for the train’s driveway must be detected automatically by appropriate sensor systems. Machine learning algorithms have proven to be powerful tools for this task during the last years. However, these algorithms require large amounts of high-quality annotated data containing railway-specific objects as training data. Unfortunately, all of the publicly available datasets that tackle this requirement are restricted in some way. Therefore, this paper presents OSDaR23, a multi-sensor dataset of 45 subsequences acquired in Hamburg, Germany, in September 2021, that was created to foster driverless train operation on mainline railways. The sensor setup consists of multiple calibrated and synchronized infrared (IR) and visual (RGB) cameras, lidars, a radar, and position and acceleration sensors mounted on the front of a rail vehicle. In addition to the raw data, the dataset contains 204 091 polyline, polygonal, rectangle, and cuboid annotations in total for 20 different object classes. It is the first publicly available multi-sensor dataset annotated with a variety of object classes that are relevant for the railway context. OSDaR23, available at data.fid-move.de/dataset/osdar23, can also be used for tasks beyond collision prediction, which are listed in this paper.

Index Terms:

robotics, computer vision

I INTRODUCTION

In automatic train operation (ATO), technical systems take over tasks that had previously been performed by the operating staff. ATO includes different grades of automation (GoA), up to GoA4 in which the train is fully automated with no staff on board (Tab.I).

Figure 1: Captured data in a railway environment.

One of the main tasks of the train driver is to monitor the train’s driveway (also known as train’s path [1]) to predict collisions and act accordingly. This task is rated the most challenging according to a lately conducted rail industry survey [2] and impedes an automation upgrade to GoA3 and GoA4, where no train driver is needed. In this regard, ATO of mainline trains differs from metros. Fully automated metros, such as the Nuremberg U-Bahn, are closed systems meaning that traffic runs in isolated and mostly enclosed environments such as tunnels. Such conditions are also known as “horizontal lift”. As a result, no object detection is needed and consequently, no computer vision (CV) systems are required on-board [3]. In contrast, open rail systems are subject to environmental interferences, e.g. at level-crossings or on unfenced tracks. Therefore, various classes of objects may protude in the train’s driveway and need to be detected. Hence, replacing human vision with CV systems is a growing area of research for mainline railway.

The usage of machine learning in developing state-of-art CV systems grows [4], and therefore large amounts of annotated sensor data. Currently, only a few datasets with objects from the railway environment are available and these datasets are often limited. In order to narrow this gap, this publication presents the “Open Sensor Data for Rail 2023” (OSDaR23) [5, 6], a freely accessible annotated multi-sensor dataset containing objects that are relevant to CV systems in the context of the mainline railway and beyond.

TABLE I: Simplified description of GoA from DIN EN 62290-1.

While the development of CV-based collision prediction for ATO is the main purpose of OSDaR23, the dataset may also be used for related tasks and can also serve as a basis for extending open datasets created by others. This dataset can enable developers of CV systems to transfer their algorithms to the railway domain, thus fostering research in this area.

OSDaR23 was created in the project “Aufbereitung von Datensätzen für Anwendungen des automatisierten Fahrens im Eisenbahnbetrieb” (English: “Development of Datasets for Applications of Automated Driving in Railway Operations”) conducted by the German Centre for Rail Traffic Research at the Federal Railway Authority (DZSF) using sensor data (cf. Fig.1) provided by DB Netz AG within the sector initiative of Digitale Schiene Deutschland (DSD). The annotations of this dataset were created by FusionSystems GmbH. The dataset is published together with a research report as well as a labeling guide that specifies how the annotations were created and how future datasets can be annotated similarly. These documents can be obtained from dzsf.bund.de.

II Dataset Requirements for OSDaR23

Fully functional CV systems for collision prediction need to include algorithms for the following three sub tasks: obstacle detection, distance estimation, and track detection [1]. Given these groups, several requirements for datasets for the development of CV systems can be derived.

First, datasets must contain potential obstacles endogenous to the railway system, such as rail cars and buffer stops, as well as exogenous obstacles, such as pedestrians, road cars, animals, trees, rocks, misplaced drag shoes, fires and many others.

Second, objects are only considered to be obstacles when they are in the train’s driveway. While object detection in the railway context usually refers to objects placed on the tracks, obstacles can belong to rather unusual object types or occur in unusual places as reported cases of bicycles hanging from the catenary illustrate [7, 8]. Therefore, in addition to the existance of objects, their position relative to the driveway needs to be determined. The region of interest (ROI) for object detection includes the tracks as well as the 3D tubular space formed by the predicted train’s driveway and minimum clearance profile. To determine the ROI, the datasets must include tracks and other rail infrastructure objects such as switches and transitions, which allow the prediction of the train’s driveway.

Third, compared to road vehicles, braking distances of rail vehicles are about five times larger, and whistling and emergency braking are the only available reactions to obstacles, since evasion is obviously not an option. Therefore, datasets should include data from sensors that include objects at distances over several hundred meters.

Fourth, object detection performance doesn’t only depend on the distance of an object, but also on several other factors, such as the size of the object, the visual contrast of the object compared to the background, the speed of the ego-vehicle, the weather and time of the day, the occlusion of the object by other objects and other visual conditions. As a reference, Tab.II presents median distances for human train driver’s performance in detecting objects, which might serve as a benchmark in safety argumentation [8]. Therefore, another requirement is that datasets need to contain data collected under these different circumstances.

TABLE II: Human detection of objects on railway [4, 8].

Object Median distance (area or size)of detection in m ≥\qty⁢0.4⁢m 2 absent\qty 0.4 superscript m 2\geq\qty{0.4}{\text{m}^{2}}≥ 0.4 m start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT, \qty⁢30⁢%\qty 30%\qty{30}{\text{%}}30 % (visual) contrast>750 absent 750>750> 750 \qty⁢2⁢m 2\qty 2 superscript m 2\qty{2}{\text{m}^{2}}2 m start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT, \qty⁢8⁢%\qty 8%\qty{8}{\text{%}}8 % contrast 500 500 500 500 \qty⁢0.4⁢m 2\qty 0.4 superscript m 2\qty{0.4}{\text{m}^{2}}0.4 m start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT, \qty⁢8⁢%\qty 8%\qty{8}{\text{%}}8 % contrast 240 240 240 240 \qty⁢2⁢m 2\qty 2 superscript m 2\qty{2}{\text{m}^{2}}2 m start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT, \qty⁢30⁢%\qty 30%\qty{30}{\text{%}}30 % contrast, at night 180 180 180 180 \qty⁢0.4⁢m 2\qty 0.4 superscript m 2\qty{0.4}{\text{m}^{2}}0.4 m start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT, \qty⁢30⁢%\qty 30%\qty{30}{\text{%}}30 % contrast, at night 60 60 60 60 ≤\qty⁢2⁢m 2 absent\qty 2 superscript m 2\leq\qty{2}{\text{m}^{2}}≤ 2 m start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT, \qty⁢8⁢%\qty 8%\qty{8}{\text{%}}8 % contrast, at night<60 absent 60<60< 60 [9] 40×40×40 40 40 40 40\times 40\times 40 40 × 40 × 40 cm 250 250 250 250 20×20×20 20 20 20 20\times 20\times 20 20 × 20 × 20 cm 175 175 175 175 10×10×10 10 10 10 10\times 10\times 10 10 × 10 × 10 cm 50 50 50 50 5×5×5 5 5 5 5\times 5\times 5 5 × 5 × 5 cm<25 absent 25<25< 25 fluorescent objects at night, \qty⁢60⁢km/h\qty 60 km h\qty{60}{\nicefrac{{\text{km}}}{{\text{h}}}}60 / start_ARG km end_ARG start_ARG h end_ARG[10] person in safety jacket 400 400 400 400 passenger car 300 300 300 300 person 240 240 240 240 passenger car at night, person with<60 absent 60<60< 60 and without safety jacket at night [11] tree, 50 50 50 50-\qty⁢70⁢km/h\qty 70 km h\qty{70}{\nicefrac{{\text{km}}}{{\text{h}}}}70 / start_ARG km end_ARG start_ARG h end_ARG 60 60 60 60 fallen rock, 20 20 20 20-\qty⁢120⁢km/h\qty 120 km h\qty{120}{\nicefrac{{\text{km}}}{{\text{h}}}}120 / start_ARG km end_ARG start_ARG h end_ARG 30 30 30 30 Japanese accident statistics [12]

Fifth, depending on GoA and the technical implementation, the task of detecting train signals and their meaning is either conducted by a human, a CV system, or by cab signaling (GoA2). In case of CV systems, the datasets must contain annotated signals. It should also be taken into account that railway signals significantly differ from street signs in shape, colour and placement as well as among railway systems of different countries.

Lastly, it is necessary to predict the future trajectory of objects in order to determine whether a collision is likely to happen or not. This requires tracking objects over time. Therefore, the datasets need to contain tracking identifiers that allow mapping different annotations to the same real-world object.

III Existing Datasets for CV in Railway

TABLE III: Existing Public Single-Sensor Datasets for CV in Railway

To the best of the authors’ knowledge, datasets listed in Tab.III are the only CV datasets recorded by frontal on-board sensors of a railway train including object annotations that have explicitly been published for free use by the research community. They contain annotated single RGB-camera frames from video sequences. RailSem19 contains frames of railway and tram scenes from 38 38 38 38 countries. It contains annotations in form of geometric shapes and dense pixel-wise semantic segmentation for trains, switches, switch states, platforms, buffer stops, rail traffic signs and railway signals. RailSet is similar in its annotation to RailSem19 and contains annotated railway scenes, both under regular and defective conditions, such as rails discontinuity and hole anomalies. Rail-DB contains annotated recordings of high-speed rail infrastructure.

FRSign is a dataset of frames annotated with boxes of French railway signals in different signal states and GERALD a dataset of frames containing German signals. RAWPED contains frames with box annotations for pedestrians. In addition to open datasets, there are also datasets that have been used in research but have not (yet) been published, such as, the RAILOD dataset of 4 651 4651\numprint{4651}4 651 manually annotated single RGB-camera frames from 6 6 6 6 scenes [19].

IV Motivation for OSDaR23

OSDaR23[5] is a manually annotated open dataset with a multi-sensor setup for the railway environment. The multi-sensor setup includes cameras, lidars, a radar as well as position and acceleration sensors. In contrast, available open datasets for railway focus mainly on camera images. The goal of creating OSDaR23 was to advance the development of AI algorithms for the higher GoA of mainline railways, in terms of object detection. Another related goal is making developers of AI systems more familiar with the railway context and to support standardization activities for the safety approval process [20].

V Multi-Sensor System

Figure 2: The utilized vehicle, with the mounted sensor setup [6].

OSDaR23 was recorded on several data collection runs in September 2021 in Hamburg, Germany, by DB Netz AG. The data was then selected in a way that it covers many different situations in regular train operation. Additionally, some special situations and objects, e.g. flames and smoke, were staged.

A DZSF research project on sensor technology for driverless trains has shown that multi-sensor systems can guarantee a complete environmental perception [2]. Although the optimal sensor setup is still a subject of research, lidar, radar, RGB and IR cameras have proven to be particularly useful. Due to the long range requirements resulting from long stopping distances, cameras with high resolutions and narrow fields of view (FoV) should be used. Narrow FoVs and the presence of curved tracks require multiple cameras positioned at different angles relative to the vehicle.

The vehicle utilized for data collection was a track working vehicle with attached profiles for the multi-sensor system. Fig.2 shows the sensors at the front of the vehicle mounted on the profiles. The sensor configuration included a total of six RGB cameras, three IR cameras, six lidar sensors with different ranges and field of view (FoV) coverage, a 2D radar sensor, and position and acceleration sensors (GNSS+IMU). The camera, lidar, and radar data are shown in Fig.1. The cameras are divided into multiple sets of 3 3 3 3 cameras each, aligned like a trident, diagonally left, center and diagonally right in the direction of travel. The main technical specifications of the sensors are listed in Tab.IV. The sensors were calibrated according to the reference coordinate system shown in Fig.3. The FoV areas of the sensors covered the area in front and partially to the side of the vehicle and overlap each other.

The sensors were synchronized at a frame rate of \qty⁢10⁢Hz\qty 10 Hz\qty{10}{\text{Hz}}10 Hz based on the acquisition time of each sensor. The sensors used the Precision Time Protocol (PTP) in order to synchronize their clocks. As the radar has a capturing rate of \qty⁢4⁢Hz\qty 4 Hz\qty{4}{\text{Hz}}4 Hz only, some radar images are duplicated. The union of frames from all sensors at a given point in time forms a multi-sensor frame (m-frame). For the dataset, the point clouds of the six lidars were ego-motion compensated and merged into a single point cloud per m-frame.

Figure 3: The utilized reference coordinate system.

TABLE IV: Multi-sensor system

VI Annotation Specification

In addition to the raw sensor data, the annotations are the core part of OSDaR23. A labeling guide forms the basis for the manual creation of annotations by the annotators. This guide refers to the described sensor data from IR/RGB cameras, the radar and point cloud data from the lidars.

The choice of relevant objects for annotation is motivated by the tasks of a contactless perception system replacing a human driver as described in Sec.II, whereby no specific operational design domain is chosen. The task of obstacle detection requires the annotation of endangered objects such as people and animals as well as endangering objects such as road and rail vehicles, drag shoes etc. The ROI detection task requires the annotation of infrastructure. Signal detection requires the annotation of signals.

In the camera and radar frames, axis-parallel and rotated rectangles, polylines and polygons are used as two-dimensional (2D) annotation geometries. The rectangles and polygons enclose the annotated objects in the sensor images as accurately as possible. The polylines are used to annotate tracks and transitions – they follow the outer rail edges. In the lidar point clouds, three-dimensional (3D) cuboids and polylines are used, as well as semantic segmentations. Cuboids enclose objects and polylines mark contours analogous to the 2D annotations. In semantic segmentations, individual lidar points are assigned to an object.

Using these geometries, objects of 20 20 20 20 different classes from four different categories were annotated. The classes for dynamic objects are person, crowd, train, wagons, bicycle, group of bicycles, motorcycle, road vehicle, animal, group of animals, wheelchair, and drag shoe. The railway related objects are track, switch, and transition. Static objects are catenary pole, signal pole, signal, signal bridge, and buffer stop. Special classes are flame and smoke. As the sensor data were recorded in Germany, only German railway signals are part of the dataset. All object annotations have tracking IDs by which they can be assigned to the same physical object in the real world in all m-frames of a sequence.

The annotations are provided in JSON-files that follow the RailLabel JSON schema. The RailLabel schema is a sub-schema of the universally usable ASAM OpenLABEL standard developed by the Association for Standardization of Automation and Measuring systems [21]. The RailLabel scheme specifies options that were kept open in the standard ASAM OpenLABEL format. The development software kit for this dataset is online available on github.com/DSD-DBS/raillabel.

Details on class attributes, annotation rules and the storage format are presented in the project report and in the separately published labeling guide. Fig.4 shows two examples for the sensor data and its associated annotations. In the left column a recording of the main station in Hamburg is presented. In the right column a recording in the proximity to station Hamburg-Ohlsdorf is shown. The first row shows the images from the center camera. In the second row the ego-motion compensated and merged point clouds of the lidars are shown. The third row presents the data of the central IR camera. In the last row the outcome of the radar is pictured. Please note, that the radar images are zoomed in due to the long viewing range of the radar.

Figure 4: Examples for annotated sensor data from OSDaR23. Left column: Sequence inside Hamburg Main Station. Right column: Sequence at Station Ohlsdorf. First Row: RGB center camera. Second Row: Merged lidar point cloud. Third row: IR center camera. Last Row: Radar (zoomed). In the sensor frames different annotations are visualized: Rectangles and cuboids for persons (dark blue), switches (blue), signals (red), signal poles (dark brown), polylines for rails (pink) and polygons for trains beige).

VII OSDaR23 Statistics

The annotated dataset comprises 21 21 21 21 sequences, divided into 45 45 45 45 subsequences, with a total of 1 534 1534\numprint{1534}1 534 annotated m-frames and 204 091 204091\numprint{204091}204 091 annotation objects. Since an m-frame is composed of nine camera frames, one radar frame and one lidar frame, the total number of individual sensor frames is 1 534⋅11=16 874⋅1534 11 16874\numprint{1534}\cdot 11=\numprint{16874}1 534 ⋅ 11 = 16 874. An annotation object refers to the annotation of a physical real-world object captured in a sensor at a point in time. A real-world object is typically captured by multiple sensors and over multiple points in time and is therefore represented in multiple annotation objects. Additionally, for each annotation, the values for several class-specific attributes are given.

To cover a wide variety of object classes and environments while enabling the development of object tracking, the dataset contains several shorter sequences of different locations and situations with ten m-frames each as well as some longer sequences of 40 40 40 40 to 100 100 100 100 m-frames. The most common annotation objects are persons such as passengers or staff. These are followed by static objects such as signals, catenary poles, tracks, signal poles, buffer stops, and dynamic objects such as road vehicles and trains. Tab.V gives an overview of the distribution of the annotation objects.

OSDaR23 can be divided into training, test and validation sets. Our suggested division is set in a way, that ∼70%similar-to absent percent 70\sim 70%∼ 70 % of the m-frames can be used for training and ∼15%similar-to absent percent 15\sim 15%∼ 15 % for test and validation. In the following the division is outlined:

Training dataset: 1_calibration_1.2, 3_fire_site_3.1, 3_fire_site_3.3, 4_station_pedestrian_bridge_4.3, 5_station_bergedorf_5.1, 6_station_klein_flottbek_6.2, 8_station_altona_8.1, 8_station_altona_8.2, 9_station_ruebenkamp_9.1, 12_vegetation_steady_12.1, 14_signals_station_14.1, 15_construction_vehicle_15.1, 20_vegetation_squirrel_20.1, 21_station_wedel_21.1, 21_station_wedel_21.2,

Test dataset: 1_calibration_1.1, 3_fire_site_3.2, 4_station_pedestrian_bridge_4.1, 4_station_pedestrian_bridge_4.4, 5_station_bergedorf_5.2, 7_approach_underground_station_7.1, 7_approach_underground_station_7.3, 8_station_altona_8.3, 9_station_ruebenkamp_9.2, 9_station_ruebenkamp_9.6, 10_station_suelldorf_10.1, 13_station_ohlsdorf_13.1, 16_under_bridge_16.1, 17_signal_bridge_17.1, 19_vegetation_curve_19.1

Validation dataset: 2_station_berliner_tor_2.1, 3_fire_site_3.4, 4_station_pedestrian_bridge_4.2, 4_station_pedestrian_bridge_4.5 6_station_klein_flottbek_6.1 7_approach_underground_station_7.2 9_station_ruebenkamp_9.3, 9_station_ruebenkamp_9.4, 9_station_ruebenkamp_9.5, 9_station_ruebenkamp_9.7, 11_main_station_11.1, 14_signals_station_14.2, 14_signals_station_14.3, 18_vegetation_switch_18.1, 21_station_wedel_21.3

TABLE V: Number of annotations per object class

VIII Limitations

In the following, the limitations of OSDaR23 are the explained.

First, the dataset does not include an associated digital map. Nonetheless, a digital map can be created from the annotated data using visual odometry or self-localization.

Second, while signals are annotated, they are not associated with the track to which they belong. Since exceptions to the rules for signal placement exist, these associations can not be easily derived [22]. Further, OSDaR23 does not provide the classification of signal states.

Third, automation of shunting operation in shunting yards requires recordings and annotations of special signals such as fouling point indicators at the railway switches, which OSDaR23 does not provide.

Lastly, the visual inspection of the infrastructure and vehicle is an important task to prevent accidents additionally to predictive maintenance. Damaged infrastructure and slipping load of crossing trains must be detected to induce emergency braking. However, such cases are not covered by OSDaR23.

IX Conclusion and Future Work

The open multi-sensor dataset OSDaR23[5] for driverless operation in the railway environment is presented, probably the first of its kind. Multiple calibrated synchronized IR/RGB cameras, lidars, a radar and GNSS+IMU sensors are mounted on the front of a track work vehicle during data acquisition. The dataset contains 204 091 204091\numprint{204091}204 091 annotations for 20 20 20 20 different object classes. Developers can now use an industrial-grade dataset to create AI models for object detection, ROI determination, and distance estimation required for collision prediction by state-of-the-art perception systems. The dataset can be extended with user-specific complementary data like objects intruding into the railways.

Today, there are more than 200 200 200 200 known methods for pedestrian motion prediction [23], which can be tested on OSDaR23. Detecting pedestrians is a very important task as trespassing railways is a life-threatening criminal offense that also obstructs train operation.

Further datasets beyond OSDaR23 will be required to develop CV systems fulfilling high safety requirements of ATO. They should include different sensor configurations, a quantitative increase of data, qualitative expansion of the object classes, their attributes, geometries, environments, and situations as well as recordings of unusual, (anticipated) critical and incident events. The DZSF continues the activities in processing and publishing annotated sensor data with the support of DB Netz AG.

OSDaR23 will be integrated in the DSD Data-Factory [24]. The Data-Factory is a platform for the systematic provision and processing of sensor data for the development of AI functions and for the simulation of photorealistic scenarios including reasonable trajectories of the relevant objects as well as artificial sensor data.

OSDaR23 and its annotation specification can serve as a reference as well as a basis for extensions, which will fill the gaps described in Sec.VIII. Developers of CV systems that work in related fields of research like visual infrastructure inspection [25], automated security surveillance [26], train door operation, digital railway mapping etc. performed on-board and off-board might take advantage of these extensions and contribute to the development of further datasets. Similarly, developers of multi-sensor data generation systems for railways [27] might also improve the performance of their system by using OSDaR23. All stakeholders in the rail sector and beyond are invited to participate in this effort and, if possible, publicate new datasets to achieve a broad research and development community.

ACKNOWLEDGMENT

The authors thank their colleagues Dr. Kai Hofmann (DZSF), Florian Reiniger, Markus-Franz Ziegler, Volker Eiselein, Arne Jacobs (all DB Netz AG) and Michael Scheithauer, Kai-Uwe Kaden and Kevin Förster (all FusionSystems GmbH) for their support in the project, as well as the Vicomtech Research Foundation (vicomtech.org) for providing the WebLabel Player which can be used to visualize the dataset.

References

• [1] D.Ristić-Durrant, M.Franke, and K.Michels, “A review of vision-based on-board obstacle detection and distance estimation in railways,” Sensors, vol.21, no.10, p. 3452, 2021.
• [2] D.Leinhos, H.Flatt, and S.Witte, “Sensorik als technische Voraussetzung für ATO-Funktionen,” DZSF, Tech. Rep., 2022.
• [3] R.Tagiew, T.Buder, K.Hofmann, C.Klotz, and R.Tilly, “Towards Nucleation of GoA3+ Approval Process,” in HPCCT, 2021, pp. 41–47.
• [4] R.Tagiew, T.Buder, and M.Fietze, “Überblick zur fahrzeugseitigen Kollisionsvorhersage für Eisenbahnen,” ZEVrail, Magazine, vol.4, 2023.
• [5] R.Tilly, P.Neumaier, K.Schwalbe, P.Klasek, R.Tagiew, P.Denzler, T.Klockau, M.Boekhoff, and M.Köppel, “Open Sensor Data for Rail 2023 (OSDaR23),” TIB – Leibnitz Information Centre for Science and Technology, University Library, 2023.
• [6] R.Tilly, P.Neumaier, K.Schwalbe, P.Klasek, R.Tagiew, P.Denzler, M.Köppel, and T.Klockau, “Offener Multisensordatensatz für die Entwicklung der Umfeldwahrnehmung beim vollautomatischen Fahren,” ETR, Magazine, vol.4, 2023.
• [7] “Gegenstände in Oberleitung: Züge erreichen Landshut nicht,” Fränkische Landeszeitung, 15.07.2022.
• [8] R.Tagiew, “Mainline automatic train horn and brake performance metric,” CoRR, vol. 2307.02586, 2023.
• [9] J.Polz, P.Mihm, and W.Czurda, “Automatisches Fahren (AF) - Lastenheft ”Hinderniserkennung Fahrweg”,” BMBF, Tech. Rep., 2003. [Online]. Available: www.dzsf.bund.de/SharedDocs/Downloads/DZSF/Veroeffentlichungen/BMBF_Lastenheft_Hinderniserkennung_Fahrweg.html
• [10] K.Itoh, M.Seki, and T.Hoshino, “A Simulation-based Study on Night Train Operators’s Track Inspection Performance by Use of Cognitive-Perceptual Model,” DGLR Bericht, 2001.
• [11] S.Mockel, F.Scherer, and P.Schuster, “Multi-sensor obstacle detection on railway tracks,” in IEEE IV, 2003, pp. 42–46.
• [12] R.Nakasone, N.Nagamine, M.Ukai, H.Mukojima, D.Deguchi, and H.Murase, “Frontal obstacle detection using background subtraction and frame registration,” Quarterly Report of RTRI, vol.58, no.4, pp. 298–302, 2017.
• [13] O.Zendel, M.Murschitz, M.Zeilinger, D.Steininger, S.Abbasi, and C.Beleznai, “RailSem19: A Dataset for Semantic Rail Scene Understanding,” in CVPRW, 2019, pp. 1221–1229.
• [14] J.Harb, N.Rébéna, R.Chosidow, G.Roblin, R.Potarusov, and H.Hajri, “FRSign: A Large-Scale Traffic Light Dataset for Autonomous Trains,” CoRR, vol. 2002.05665, 2020.
• [15] T.Toprak, B.Belenlioglu, B.Aydın, C.Guzelis, and M.A. Selver, “Conditional Weighted Ensemble of Transferred Models for Camera Based Onboard Pedestrian Detection in Railway Driver Support Systems,” IEEE TVT, vol.69, no.5, pp. 5041–5054, 2020.
• [16] X.Li and X.Peng, “Rail detection: An efficient row-based network and a new benchmark,” in Proceedings of the 30th ACM International Conference on Multimedia.Association for Computing Machinery, 2022, p. 6455–6463.
• [17] A.Zouaoui, A.Mahtani, M.A. Hadded, S.Ambellouis, J.Boonaert, and H.Wannous, “Railset: A unique dataset for railway anomaly detection,” in IEEE IPAS, vol. Five, 2022, pp. 1–6.
• [18] P.Leibner, F.Hampel, and C.Schindler, “GERALD: A novel dataset for the detection of German mainline railway signals,” Journal of Rail and Rapid Transit, 2023.
• [19] L.Guan, L.Jia, Z.Xie, and C.Yin, “A Lightweight Framework for Obstacle Detection in the Railway Image based on Fast Region Proposal and Improved YOLO-tiny Network,” IEEE Transactions on Instrumentation and Measurement, vol.71, pp. 1–16, 2022.
• [20] R.Tagiew, T.Buder, R.Tilly, K.Hofmann, and C.Klotz, “Datensätze für das autonome Fahren als Grundlage für GoA3+,” ETR, Magazine, vol.9, 2021.
• [21] “Association for Standardization of Automation and Measuring Systems,” 2021, ASAM OpenLABEL. [Online]. Available: asam.net/standards/detail/openlabel/
• [22] A.D. Petrović, M.Banić, M.Simonović, D.Stamenković, A.Miltenović, G.Adamović, and D.Rangelov, “Integration of Computer Vision and Convolutional Neural Networks in the System for Detection of Rail Track and Signals on the Railway,” Applied Sciences, vol.12, no.12, p. 6045, 2022.
• [23] A.Rudenko, L.Palmieri, M.Herman, K.M. Kitani, D.M. Gavrila, and K.O. Arras, “Human motion trajectory prediction: A survey,” The International Journal of Robotics Research, vol.39, no.8, pp. 895–935, 2020.
• [24] P.Neumaier, “Die Data Factory – “Datenproduktion” für das Training von KI-Software,” Digitale Schiene Deutschland, 2022. [Online]. Available: digitale-schiene-deutschland.de/aktuelles/Data-Factory
• [25] S.Liu, Q.Wang, and Y.Luo, “A review of applications of visual inspection technology based on image processing in the railway industry,” Transportation Safety and Environment, vol.1, no.3, pp. 185–204, 2019.
• [26] A.Zaman, B.Ren, and X.Liu, “Artificial intelligence-aided automated detection of railroad trespassing,” Transportation research record, vol. 2673, no.7, pp. 25–37, 2019.
• [27] G.D’Amico, M.Marinoni, F.Nesti, G.Rossolini, G.Buttazzo, S.Sabina, and G.Lauro, “TrainSim: A Railway Simulation Framework for LiDAR and Camera Dataset Generation,” CoRR, vol. 2302.14486, 2023.