The decommissioning of nuclear facilities poses major challenges for operators. Whether it is decommissioning or safe containment, the amount of nuclear waste to be disposed of is growing at an overwhelming rate worldwide.
Automation to handle nuclear waste
Automation is increasingly required to handle nuclear waste, but the nuclear industry is reluctant of fully autonomous robotic control methods for safety reasons, and remote-controlled industrial robots are preferred in hazardous environments.
However, such complex tasks as remote-controlled gripping or cutting of unknown objects, with the help of joysticks and video surveillance cameras, are difficult to control and sometimes even impossible.
Robot assistance system
The robot assistance system developed there enables ‘shared’ control to perform complex manipulation tasks
To simplify this process, the National Centre for Nuclear Robotics (NCNR), led by Extreme Robotics Lab, at the University of Birmingham, in the United Kingdom, is researching automated handling options for nuclear waste.
The robot assistance system developed there enables ‘shared’ control to perform complex manipulation tasks, by means of haptic feedback and vision information, provided by Ensenso 3D camera. The operator, who is always present in the loop, can retain control over the robot’s automated actions, in case of system failures.
Application
Anyone who has ever tried out a fairground grab machine can confirm that manual control of grab arms is anything but trivial. As harmless as it is to fail, when trying to grab a stuffed bunny, failed attempts can be as dramatic, when handling radioactive waste.
To avoid damage with serious consequences for humans and the environment, the robot must be able to detect the radioactive objects in the scene extremely accurately and act with precision. The operator literally has it in his hands, as it is up to him to identify the correct gripping positions.
At the same time, the operator must correctly assess the inverse kinematics (backward transformation) and correctly determine the joint angles of the robot’s arm elements, in order to position it correctly and avoid collisions.
Standard industrial robot with Ensenso N35 3D camera
The system autonomously scans unknown waste objects and creates a 3D model of them
The assistance system developed by the British researchers simplifies and speeds up this task immensely, with a standard industrial robot equipped with a parallel jaw gripper and an Ensenso N35 3D camera.
The system autonomously scans unknown waste objects and creates a 3D model of them, in the form of a point cloud. This is extremely precise, because Ensenso 3D cameras work according to the principle of spatial vision (stereo vision), which is modelled on human vision. Two cameras view the object from different positions.
Ensenso software
Although the image content of both camera images appears identical, they show differences in the position of the objects viewed. Since the distance and viewing angle of the cameras, as well as the focal length of the optics are known, the Ensenso software can determine the 3D coordination of the object point, for each individual image pixel.
In this case, the scene is captured using different scanning positions of the camera and combined to get a complete 3D surface from all viewing angles.
Ensenso’s calibration routines
Ensenso’s calibration routines help transform the individual point clouds into a global coordinate system, which improves the complete virtual image. The resulting point cloud thus contains all the spatial object information needed to communicate the correct gripping or cutting position to the robot.
With the help of the software, the Enseno 3D camera takes over the perception and evaluation of the depth information for the operator, whose cognitive load is considerably reduced as a result. The assistance system combines the haptic features of the object to be gripped, with a special gripping algorithm.
Hypothesis ranking algorithm
The scene cloud is used by our system to automatically generate several stable gripping positions"
Dr. Naresh Marturi, the Senior Research Scientist at the National Centre for Nuclear Robotics (NCNR), said “The scene cloud is used by our system to automatically generate several stable gripping positions. Since the point clouds captured by the 3D camera are high resolution and dense, it is possible to generate very precise gripping positions, for each object in the scene. Based on this, our ‘hypothesis ranking algorithm’ determines the next object to pick up, based on the robot's current position.”
The determined path guidance enables the robot to navigate smoothly and evenly, along a desired path, to the target gripping position. Like a navigation system, the system supports the operator in guiding the robot arm to the safe grasp, if necessary, also past other unknown and dangerous objects. The system calculates a safe corridor for this and helps the operator not to leave the corridor through haptic feedback.
Operator retains manual control over the robot
The system maps the operator’s natural hand movements exactly and reliably in real time, to the corresponding movements of the robot. The operator, thus, always retains manual control and is able to take over in the event of component failure. He can simply turnoff AI (Artificial Intelligence) and move back to human intelligence, by turning off the ‘force feedback mode’.
In accordance with the principle of shared control between man and machine, the system, thus, remains under control at all times, essential in an environment with the highest level of danger.
Ensenso N35 model 3D cameras with blue LEDs
Dr. Naresh Marturi said, “For all our autonomous grasp planning, remote control and visual object tracking tasks, we use Ensenso N35 model 3D cameras with blue LEDs (465nm) mounted on the end effector of the robots, along with other tools.”
He adds, “Most of the systems from the Extreme Robotic Lab have so far been equipped with a single 3D camera. However, recently to speed-up the process of 3D model building, we have upgraded our systems to use additional three scene mounted Ensenso 3D cameras, along with the one on-board the robot.”
Space-saving stationary or mobile use on a robot arm
The Ensenso N series is pre-destined for this task. It was specially designed for use in harsh environmental conditions
The Ensenso N series is pre-destined for this task. It was specially designed for use in harsh environmental conditions. Thanks to its compact design, the N series is equally suitable for the space-saving stationary or mobile use on a robot arm, for the 3D detection of moving and static objects.
Even in difficult lighting conditions, the integrated projector projects a high-contrast texture onto the object to be imaged, by means of a pattern mask with a random dot pattern, thus supplementing the structures that are not or only weakly present on its surface.
Aluminium housing ensures optimal heat dissipation
The aluminium housing of the N30 models ensures optimal heat dissipation of the electronic components and thus stable light output even under extreme ambient conditions. This ensures the consistently high quality and robustness of the 3D data.
Even in difficult lighting conditions, the integrated projector projects a high-contrast texture onto the object to be imaged by means of a pattern mask with a random dot pattern, thus supplementing the structures that are not or only weakly present on its surface.
Easy to set up and operate cameras
Cameras of the Ensenso N camera family are easy to set up and operate, via the Ensenso SDK. It offers GPU-based image processing, for even faster 3D data processing and enables the output of a single 3D point cloud of all cameras used in multi-camera operation, which is required in this case, as well as the live composition of the 3D point clouds, from multiple viewing directions.
For the assistance system, the researchers have developed their own software in C++ to process the 3D point clouds captured by the cameras.
Software using Ensenso SDK and calibration routines
Our software uses the Ensenso SDK (multi-threaded) and its calibration routines"
Dr. Naresh Martur said, “Our software uses the Ensenso SDK (multi-threaded) and its calibration routines, to overlay texture on the high-resolution point clouds and then transform these textured point clouds into a world coordinate system. Ensenso SDK is fairly easy to integrate with our C++ software. It offers a variety of straightforward functions and methods to capture and handle point clouds, as well as camera images.”
He adds, “Moreover, with CUDA support, the SDK routines enable us to register multiple high-resolution point clouds to generate high-quality scene clouds in global frame. This is very much important for us, especially to generate precise grasp hypothesis.”
Main advantages of the system
- Operators do not need to worry about scene depth or how to reach the object, or where to grasp it. The system can figure out everything in the background and helps the operator to move exactly to the place, where the robot can best grasp the object.
- With haptic feedback, operators can feel the robot in their hand, even when the robot is not present in front of them.
- By combining haptics and grasp planning, operators can move the objects in a remote scene very easily and very quickly, having very low cognitive load. This saves time and money, avoids errors and increases safety.
Outlook
Researchers at the Extreme Robotic Lab in Birmingham are currently developing an extension of the method, to allow the use of a multi-fingered hand, instead of a parallel jaw gripper. This should increase flexibility and reliability, when gripping complex objects.
In the future, the operator will also be able to feel the forces to which the fingers of the remote-controlled robot are exposed, when gripping an object.
Fully autonomous gripping methods
Fully autonomous gripping methods are also being developed, in which the robot arm is controlled by an AI and guided by an automatic vision system. The team is also working on visualisation tools to improve human-robot collaboration, to control remote robots via a ‘shared control’ system.
This is a promising approach for the safety and health of all of us. The handling of hazardous objects, such as nuclear waste is ultimately a matter of concern to us all. By reliably capturing the relevant object information, Ensenso 3D cameras are making an important contribution to this globally prevalent task of increasing urgency.
Discover how AI, biometrics, and analytics are transforming casino security
- Network / IP
- Remote surveillance
- Remote video surveillance
- IP video surveillance
- Electronic security systems
- Application security
- Industrial security
- Remote security
- Commercial security
- Perimeter security
- Private sector security
- Public sector security
- Security management
- Security policy
- Security devices
- Security installation
- Security cameras
- Covert cameras
- Security camera systems
- Security monitoring system
- Network monitoring
- Asset tracking
- Remote video monitoring
- Intrusion detection
- Building security
- Explosives detection
- Facility security
- Public security
- Industrial security systems
- Network cameras
- Security software
- IP Surveillance
- Security service
- Industrial surveillance
- Outdoor video surveillance
- Indoor surveillance
- IP security solutions
- Covert Surveillance
- Security communication
- Integration software
- Perimeter protection
- Cyber security
- Internet of Things (IoT)
- Outdoor Security
- Indoor Security
- Data Security
- Intruder Lights
- Incident Management
- Security Assessments
- Cloud security
- Robotics
- Drones
- Artificial intelligence (AI)
- Video surveillance
- Weapon detection
- Machine Learning
