Submitted by jb733 on Fri, 05/12/2025 - 17:09
Congratulations to Percy Lam, a FIBE2 cohort 3 student, who was awarded a prize at the DIFA: Deep Learning-based Information Fusion and Its Applications Workshop, affiliated to the 36th British Machine Vision Conference.
The paper, co-authored with Dr Weiwei Chen, Prof Lavindra de Silva and Prof Ioannis Brilakis, investigates modern technological processes for investigating road degradation that leads to the notorious pothole. The greater speed of detection and repair of roads has the potential to greatly reduce both costs and down-time to industry and the wider public.
Abstract
Having potholes detected and repaired is essential for road maintenance and uninterrupted transport. Generations of researchers and engineers have been innovating on using computer vision and sensors to detect potholes, however, gaps in knowledge remain in the lack of robustness of detectors trained with RGB images in varied environments and the inability to harness depth in detecting potholes with monocular images. This research proposes a solution by fusing predictions made by RGB-trained detectors and monocular depth estimation. The solution first orthorectifies the pavement from perspective RGB images. It then receives predictions from RGB trained detectors and a depth detector enabled by DINOv2 independently. These predictions are subsequently fused by weighted bounding box fusion and have masks predicted by Segment Anything. Whereas RGB-trained detectors perform well in test sets within the training context, they show a drastic loss of performance in out-of-context situations, such as in images taken on different roads in more challenging environmental conditions. Fusing predictions with depth enhances F1 scores by 16% to 81% in out-of-context situations, reinforcing detection robustness. This solution paves the way for further research on expanding detection to motorways and overcoming shadows in images.