Master's Thesis

Integrating information from radar sensors into a semantic data model of autonomous vehicles for navigation in complex terrain environments

Final Thesis 8.97 MB Appendix 1.16 MB

Author of thesis: Ing. José Manuel Melero Anaya

Acad. year: 2025/2026

Supervisor: prof. Ing. Aleš Prokeš, Ph.D.

Reviewer: Ing. Jan Král, Ph.D.

Abstract:

This master’s thesis presents a multimodal perception framework for autonomous vehicles operating in challenging terrain, focusing on the integration of radar and camera sensors into a unified semantic embedding space for navigation and localization.
The proposed approach introduces an intermodal fusion architecture based on convolutional encoders, cross-attention mechanisms, and feature integration with a gate. The system is trained using contrastive learning (InfoNCE), which aligns radar and visual representations in a shared latent space.
To enhance robustness and spatial consistency, a GPS-constrained batch sampling strategy is introduced, ensuring geographically diverse training batches. Additionally, a modality dropout mechanism is employed to simulate sensor degradation and missing modalities.
The system is evaluated on a multimodal dataset containing synchronized camera images, radar scans, and GPS trajectories. Experimental results show that the proposed fusion model consistently outperforms monomodal baseline models in search-based localization tasks. Furthermore, the radar modality significantly improves robustness under adverse visual conditions, such as noise, blurring, and lighting changes.
The learned embedding space exhibits strong intermodal alignment and meaningful geometric structure, confirming the effectiveness of the proposed approach for learning multimodal semantic representations.

Keywords:

Radar-Camera Fusion, Unmanned Ground Vehicles, Autonomous Driving, Multimodal Learning, Contrastive Learning, InfoNCE loss, Cross-Attention, Semantic Embeddings, Localization, Off-Road Navigation, Sensor Fusion

Date of defence

10.06.2026

Result of the defence

Defended (thesis was successfully defended)

znamkaAznamka

Grading

A

Process of defence

At first, the student presented the results of his diploma thesis. The opinions were then read and student agreed with both of opinions without any comments. Then came the questions: 1) Student presented his answers for 2 questions from oppononet opinion. 2) prof. Prokes: what you expect when you use more comprehensive dataset include building, people, more cars etc.? -student answered and then explained that he used only simple dataset on one road without the traffic

Language of thesis

English

Faculty

Department

Study programme

Automotive Electronics and Electromobility (MPA-AEE)

Composition of Committee

prof. Ing. Aleš Prokeš, Ph.D. (předseda)
doc. Ing. Jaroslav Láčík, Ph.D. (místopředseda)
Ing. Michal Kubíček, Ph.D. (člen)
doc. Ing. Pavel Vorel, Ph.D. (člen)
Ing. Zdeněk Kincl, Ph.D. (člen)

Supervisor’s report
prof. Ing. Aleš Prokeš, Ph.D.

The student explored a highly topical area focusing on integrating data from radar sensors into a semantic data model for the navigation of autonomous vehicles in challenging terrain. The objectives of the thesis were fulfilled within the required scope. The student conducted a comprehensive review of the current state of the art in radar perception, multimodal sensor fusion, and semantic environment representations, and based on this review proposed an original solution employing modern machine-learning techniques. The thesis includes the design of a radar-camera fusion architecture, its implementation, and an experimental evaluation of the achieved results.

I consider the selected approach appropriate and well aligned with current research trends in autonomous systems. The student demonstrated the ability to independently study a broad and interdisciplinary topic, develop a suitable methodology, and critically interpret the obtained results.

The thesis also exhibits several minor shortcomings. The experimental validation was performed on a limited dataset, and some aspects of the proposed semantic model could have been elaborated in greater detail, particularly with respect to practical deployment in complex autonomous systems. However, these limitations are largely attributable to the demanding scope of the topic and the limited time available for implementation and more extensive experimental validation of the proposed approach.

Throughout the course of the thesis, the student worked diligently, independently, and with a genuine interest in the subject. He regularly discussed intermediate steps and achieved results, actively sought new knowledge, and demonstrated the ability to carry out independent and creative engineering work.

The work was supervised primarily by Associate Professor Jan Mazal, Ph.D., whose review is attached. Points proposed by supervisor: 91

Grade proposed by supervisor: A

File inserted by supervisor Size
Posudek vedoucího práce [.pdf] 41,91 kB

Reviewer’s report
Ing. Jan Král, Ph.D.

The master’s thesis addresses the integration of radar sensing into a semantic data model for autonomous vehicle navigation. The topic is highly relevant, technically demanding, and situated at the intersection of robotics, multimodal machine learning, and autonomous systems. The thesis is structured as a research driven project combining theoretical grounding, system design, and experimental validation. Formally, the thesis is well organized and clearly written. The author defines the motivation, research questions, and objectives with precision and follows them consistently throughout the work. The text is stylistically mature, technically accurate, and supported by informative figures illustrating the architecture and experimental results.

The main contribution lies in the design of the multimodal system FusionNet, which fuses radar and camera data using convolutional encoders, a cross attention mechanism, and an adaptive gated fusion module. The author further proposes several original components—most notably the GPS constrained batch sampling strategy and the modality dropout mechanism—which demonstrably improve robustness. The theoretical part is rigorous and shows a strong understanding of contrastive learning and latent space representations.

The experimental section is extensive and carefully executed. The author analyzes the embedding space, compares modality specific performance, and evaluates robustness under image degradation. The results convincingly demonstrate that multimodal fusion significantly outperforms single modality baselines. The discussion of failure cases and practical implications for autonomous navigation is also commendable.

However, the thesis does exhibit several limitations that deserve stronger emphasis. The evaluation is conducted on a single dataset, which restricts the generalizability of the conclusions; testing on additional, more diverse datasets would strengthen the claims considerably. Furthermore, the work focuses exclusively on frame based processing and does not address temporal modeling, which is a critical aspect of real world navigation.

Overall, this is a high quality master’s thesis that meets and in several aspects exceeds the expectations for this level of study. It demonstrates strong technical competence, originality, and the ability to conduct independent research. Despite the noted limitations, the work represents a meaningful contribution to multimodal perception for autonomous systems. Topics for thesis defence:
  1. How would the proposed FusionNet system perform when deployed on significantly different datasets or in other geographical conditions, and what architectural modifications would be necessary to ensure better generalization?
  2. Why did you decide not to include temporal modeling (e.g., LSTM, Transformer-based temporal fusion), and what specific benefits or risks do you believe extending the system to include sequential data processing would entail?
Points proposed by reviewer: 88

Grade proposed by reviewer: B

Responsibility: Mgr. et Mgr. Hana Odstrčilová