Divakarla, U.Bhat, R.Madagaonkar, S.B.Pranav, D.V.Shyam, C.Chandrashekar, K.2026-02-062023Lecture Notes in Networks and Systems, 2023, Vol.615 LNNS, , p. 683-69423673370https://doi.org/10.1007/978-981-19-9304-6_61https://idr.nitk.ac.in/handle/123456789/29561Recently, autonomous vehicles (namely self-driving cars) are becoming increasingly common in developed urban areas. It is of utmost importance for real-time systems such as robots and automatic vehicles (AVs) to understand visual data, make inferences and predict events in the near future. The ability to perceive RGB values (and other visual data such as thermal, LiDAR), and segment each pixel into objects is called semantic segmentation. It is the first step toward any sort of automated machinery. Some existing models use deep learning methods for 3D object detection in RGB images but are not completely efficient when they are fused with thermal imagery as well. In this paper, we summarize many of these architectures starting from those that are applicable to general segmentation and then those that are specifically designed for autonomous vehicles. We also cover open challenges and questions for further research. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.Autonomous vehiclesMultimodal learningReal-time inferencesSemantic segmentation