Tracking the Cracks with US Robots

AI-guided robot assistants could identify and inspect damage to structures and buildings to prevent failures, according to the developers of a new system.

Augmenting visual inspection technologies with a new machine learning approach, the team from Drexel University in Pennsylvania created a system that they believe could efficiently find and assess ‘problem areas’.

“Our built environment is aging and failing faster than we can maintain it,” the researchers said. “Recent building collapses and structural failures of roads and bridges are indicators of a problem that’s likely to get worse, according to experts, because it’s just not possible to inspect every crack, creak and crumble to parse dangerous signs of failure from normal wear and tear.”

The new multi-scale system combines computer vision with a deep learning algorithm to pinpoint problem areas, before directing a series of laser scans of the regions to create a digital twin that can be used to assess and monitor the damage.

The process could “significantly reduce” the inspection workload and enable focused work to prevent structural failures, the researchers said.

“Cracks can be regarded as a patient’s medical symptoms that should be screened in the early stages,” said Arvin Ebrahimkhanlou and Ali Ghadimzadeh Alamdari, authors of the new research.

“Early and accurate detection and measurement of cracks are essential for timely diagnosis, maintenance, and repair efforts, preventing further deterioration and mitigating potential hazards.”

The American Society of Civil Engineers has estimated a backlog of $786bn in repairs to roads and bridges in the US. There is also a growing shortage of skilled infrastructure workers, including inspectors and repair workers.

Instead of a physical measurement, the new system sends a high-resolution stereo-depth camera feed of the structure into a deep learning program called a convolutional neural network. These programs, which are being used for facial recognition, drug development and deep fake detection, are gaining attention for their ability to spot the finest of patterns and discrepancies in massive volumes of data.

“The neural network has been trained on a dataset of sample cracks, and it can identify crack-like patterns in the images that the robotic system collects from the surface of a concrete structure. We call regions containing such patterns ‘regions of interest’,” said Ebrahimkhanlou.

Once the region of interest is identified, the program directs a robotic arm to scan over it with a laser line scanner, which creates a three-dimensional image of the damaged area. At the same time a Lidar camera scans the structure surrounding the crack. Stitching both plots together creates a digital model of the area that shows the width and dimensions of the crack, and allows workers to track changes between inspections.

“Tracking crack growth is one of the advantages of producing a digital twin model,” Alamdari said. “In addition, it allows bridge owners to have a better understanding of the condition of their bridge, and plan maintenance and repair.”

The team tested the system in the lab on a concrete slab with a variety of cracks and deterioration. In a test of its ability to detect and measure small cracks, the system was sensitive enough to pinpoint and accurately size the smallest of fissures – less than a hundredth of a millimetre wide – outperforming top-of-the-line cameras, scanners and fibreoptic sensors by what the researchers called “a respectable margin”.

While human inspectors would still make the final call on when and how to repair the damages, the robotic assistants could greatly reduce their workload. An automated inspection process could reduce oversights and subjective judgement errors.

“This approach significantly reduces unnecessary data collection from areas that are in good structural condition, while still providing comprehensive and reliable data necessary for condition assessment,” the researchers said.

The team envisions incorporating the multi-scale monitoring system as part of a larger autonomous monitoring framework including drones and other autonomous vehicles.

“We aim to integrate this work with an unmanned ground vehicle, enhancing the system’s ability to autonomously detect, analyse, and monitor cracks,” Alamdari said.

“The goal is to create a more comprehensive, intelligent and efficient system for maintaining structural integrity across various types of infrastructure. Additionally, real-world testing and collaboration with industry and regulatory bodies will be critical for practical application and continuous improvement of the technology.”