Fraunhofer IOSB - Experts & Thought Leaders

Up-to-date and accurate maps of the waterways are a prerequisite for safe and efficient shipping in Germany. Authorities such as the Federal Maritime and Hydrographic Agency, but also private harbour operators, are obliged to provide these in an up-to-date form at all times. The main reason for this is to avoid accidents caused by an incorrect or outdated card. This includes the classification of federal waterways, their kilometres, existing locks, lifting and barrages, other surrounding waters, places and borders. Currently, mapping is carried out by conventional ships with trained personnel, which is time-consuming and expensive. This can result in long intervals between measurements at a particular location. 3D map of the surroundings Bank areas and surface forms are monitored by an optical plan with two integrated cameras (Semi-)automated monitoring of waterways by autonomous platforms with obstacle avoidance and traffic detection could optimise this process and provide maps in less time. To this end, the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB has developed an autonomous surface vehicle capable of autonomously surveying bodies of water as part of a three-year in-house research project.  For this purpose, the river bed is recorded using sonar. Furthermore, the bank areas and surface structures are monitored by an optical system with two integrated industrial cameras from IDS Imaging Development Systems GmbH. The information obtained in this way is then merged and transferred to a 3D map of the surroundings. Application The precise surveying of water bodies is a demanding task. TAPS, the Fraunhofer IOSB's semi-automatic direction-finding system for rivers and lakes is able to perform this task both under and above water. The vehicle, which measures approximately 2 x 1.5 x 1 metres, travels along the relevant waterway and automatically avoids obstacles, whether stationary or moving. At a speed of two knots - which corresponds to around 3.7 kilometres per hour - such a mission can last up to 20 hours. For visual mapping of the shore areas, TAPS has two cameras attached to a mast, each pointing to starboard or port and whose fields of view do not overlap. Due to their high resolution, they allow a visual inspection of relevant infrastructure, such as quay walls, as well as 3D modelling of the shore area based on the recorded image data. Orientation of the TAPS platform GNSS and IMU data serve as the basis for picking the position and exposure of the TAPS platform "We use an automated system for intelligent image capture. As soon as one or both cameras are focussed on a predefined area of interest, image recording is started. The vehicle's own movement is also used to store only image data that is recorded from different angles and therefore offers added value in terms of content," explains Boitumelo Ruf, expert in photogrammetry in the Autonomous Robot Systems research group at Fraunhofer IOSB. GNSS (Global Navigation Satellite Systems, such as GPS) and IMU (Inertial Measurement Unit for position determination) data serve as the basis for determining the position and orientation of the TAPS platform. "When the images are captured, they are enriched with the current GNSS position data. The latter are required later for the assignment of precise coordinates," Ruf continues. COLMAP photogrammetric toolbox After data acquisition, the recorded images are transmitted together with the GNSS data to a ground control station, where a photogrammetric reconstruction is carried out. Boitumelo Ruf describes the procedure in more detail: "Among other things, we use the COLMAP photogrammetric toolbox. It uses outstanding image features to first compare the input images, calculate their relative positions and create a true-to-life 3D model of the environment." "We then use the tool to match the images pixel by pixel, i.e. we search for corresponding pixels and merge them exactly. The result is a dense 3D point cloud that is georeferenced using the GNSS positions, i.e. provided with the corresponding current coordinates." The 3D model can then be used for other tasks, such as visual inspections or bank monitoring. Robust industrial cameras defy wind and weather On the camera side, the Fraunhofer Institute relies on two uEye FA industrial cameras from IDS. The robust and resilient models with PoE are ideal for demanding environments. Camera housings, lens tubes and the screwable connectors meet the requirements of IP code IP65/67 and are thus optimally protected against dirt, dust and splash water. The robust and resilient models with PoE are ideal for demanding environments The model used has the large-format 1.1" CMOS sensor IMX304 from Sony and delivers very clear, noise-free images with a resolution of 4096 x 3000 pixels. Thanks to its remarkable dynamic range and very high sensitivity, the global shutter sensor of the second pixel generation of the Pregius series is particularly suitable for metrology applications, among others. It was precisely these characteristics that were decisive for the Fraunhofer IOSB's choice of camera. In addition to robustness and weather protection, important requirements for the cameras included a compact form factor and high image resolution to enable detailed visual inspections. The sensor should also offer a high dynamic range in order to capture images with a high level of detail in both sunny and shady areas. Integration via the standard GigE Vision interface enables the development and use of one driver for several systems with different cameras. Outlook The shortage of skilled labour is also a major issue in the surveying of waterways. Automated mapping of the waterways could not only reduce costs but also the time and, above all, the labour required. The aim of the TAPS project was to develop a prototype for this task and to demonstrate its potential in relevant scenarios. As a research organisation, Fraunhofer typically develops technologies up to a level at which the functionality and suitability of the system have been proven. The next step is now cooperation with industrial partners. Dr. Janko Petereit, head of the Autonomous Robot Systems research group at Fraunhofer IOSB, explains: "We are actively looking for partners to take this final step towards commercialisation together. In view of the positive results of our initial tests and the increasing demand for accurate, up-to-date waterway maps, we expect strong demand for our technology in the coming years. The knowledge gained can be used in a variety of ways. Autonomous watercraft can be used to transport people and goods by sea or inland waterways. The excavation of waterways and the autonomous surveying of fairways are also conceivable. “ The project demonstrates the high potential of the interaction between robot systems and digital image processing. The cards could be reshuffled in the future, particularly with regard to visual environment detection and scene interpretation.

Radiflow, globally renowned provider of cybersecurity solutions for industrial automation networks, and the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation (Fraunhofer IOSB), a prominent research institute for applied science in Germany, today announced the launch of a joint research project for applying advanced machine learning and artificial intelligence to cyber security for industrial automation networks. Artificial intelligence techniques For this research project, Radiflow and Fraunhofer IOSB will collaborate on developing machine learning methods and artificial intelligence techniques for allowing the autonomous detection of non-compliant and anomalous behaviors on industrial automation networks. This applied research will involve evaluating graph-based and semantic approaches for event correlation and context awareness in order to develop these new machine learning and artificial intelligence capabilities. The outcome of this research will be the development of a prototype for an Autonomous Industrial Cyber security Assistance System (AICAS) that expands on existing approaches for detecting deviations and anomalies to a baseline of network behaviors on OT networks. This prototype will be designed to self-learn the underlying behaviors of industrial automation networks and the functions of the connected assets in order to dynamically detect new and unknown cyber-threats. Applying AI to the industrial cyber security Radiflow incorporates the new capabilities of this AICAS prototype into its iSID industrial threat detection system “The question of how AI can enhance industrial cyber security to better respond to changing OT environments and new attack techniques is timely and essential,” said Dr. Christian Haas, Group Manager at Fraunhofer IOSB. “Radiflow and its extensive experience working with industrial enterprises and critical infrastructure operators make the company the ideal research partner for applying AI to the industrial cyber security domain.” The funding for this research project, which is scheduled to last two years, was granted by the Innovation Authority in Israel and the Federal Ministry of Education and Research in Germany. At the conclusion of this research project, Radiflow intends to incorporate the new capabilities of this AICAS prototype into its iSID industrial threat detection system. Securing an OT network “Determining if abnormal behaviour has been caused by normal operational activities or by cyber-attackers is critical for understanding and securing an OT network,” explained Yehonatan Kfir, CTO of Radiflow. “AI holds the potential to improve the situational awareness of OT networks by efficiently distinguishing between abnormal behavior that was caused by normal operations and abnormal behavior that is connected to a cyber-attack.” “We are excited to partner with Fraunhofer IOSB on this innovative research project,” added Kfir. “We expect that the outcome of this research will expand the cyber-monitoring capacities for our customers and MSSP partners with new capabilities that require less analyst input to highlight the most critical events on dynamically changing OT networks.” The funding for this applied research project was coordinated by the Variance Ascola, a globally renowned financial and economic advisory firm in Israel.