Bachelor's Thesis

Scene Annotation Using Augmented Reality and AI

Final Thesis 3.79 MB

Author of thesis: Bc. Kirill Kurakov

Acad. year: 2025/2026

Supervisor: doc. Ing. Vítězslav Beran, Ph.D.

Reviewer: Ing. Marek Vaško

Abstract:

This work presents a mobile AR system that enables annotation of 3D objects by guiding the user through a structured scanning workflow using a consumer smartphone. The system combines real-time object detection and instance segmentation on a lightweight server, isolates target objects within the AR point cloud across multiple viewpoints, monitors scan quality and exports the annotated objects as PLY files uploaded to a server. The result is an accessible, equipment-free pipeline for 3D scene annotation suitable for robotics, AR training data collection, and spatial computing.

Keywords:

Augmented Reality, 3D Point Cloud, Object Annotation, Instance Segmentation, Object Detection, YOLO, FastSAM, ARKit, AR Foundation, Unity, FastAPI, Mobile Computing, Dataset Creation, LiDAR, Scan Quality Metrics

Date of defence

19.06.2026

Result of the defence

Defended (thesis was successfully defended)

znamkaBznamka

Grading

B

Process of defence

Student nejprve prezentoval výsledky, kterých dosáhl v rámci své práce. Komise se poté seznámila s hodnocením vedoucího a posudkem oponenta práce. Student následně odpověděl na otázky oponenta a na další otázky přítomných. Komise se na základě posudku oponenta, hodnocení vedoucího, přednesené prezentace a odpovědí studenta na položené otázky rozhodla práci hodnotit stupněm B.

Topics for thesis defence

  1. What is the difference between the instance segmentation masks produced by YOLO and the refined masks generated by FastSAM?
  2. Why do you not use the internal representation of the registered scene provided by ARKit for creating the 3D object?
  3. What is the variation of segmentation masks across different frames?
  4. How would you improve the current system? Where do you see its main current limitation?
  5. How often does your application send an image to a server?

Language of thesis

English

Faculty

Department

Study programme

Information Technology (BIT)

Composition of Committee

prof. Dr. Ing. Jan Černocký (předseda)
doc. Ing. Vojtěch Mrázek, Ph.D. (místopředseda)
Ing. Aleš Smrčka, Ph.D. (člen)
Ing. Matěj Grégr, Ph.D. (člen)
Ing. Tomáš Milet, Ph.D. (člen)

Kirill Kurakov dedicated himself to the project conscientiously, showing an excellent capacity for independent work. He significantly expanded his solution by integrating CNN-based detection and segmentation models as automatic features to ease manual annotation. Furthermore, he went beyond the initial assignment by developing metrics to evaluate the quality of the scanned 3D object's coverage. The student successfully proposed solid partial solutions, effectively resolved complex technical challenges, and delivered an high-quality project.

Evaluation criteria Verbal classification
Information about assignment

The bachelor's thesis focuses on developing a mobile application for scanning a 3D object and annotating it, resulting in a 3D point cloud of the object along with a set of 2D images tracking the camera position. The difficulty of the assignment lies in working with 3D graphics and data structures, including interactive manipulation of 3D data and its visualisation using AR technology. The student successfully completed the assignment by selecting suitable models, tools, and libraries to create a functional prototype and by interpreting the evaluation experiments. The project is self-contained and does not build on previous work.

Work with literature

The student actively searched for technical resources and materials regarding various aspects of the solution. He utilised several essential relevant sources, while less methodical references, such as documentation and software manuals for the development frameworks, are appropriately cited in the footnotes.

Activity during solution, consultations, communication

Kirill Kurakov was highly active and demonstrated a deep interest in the topic. He attended consultations regularly and well-prepared. In the initial phase, the student focused heavily on the functionality of individual components and their integration into the core pipeline. It took a significant amount of time before he achieved a deeper understanding of the core challenges and began approaching the problems more methodically, which eventually allowed him to shift his focus to specific subtasks with high technical value.

Activity during completion

The work on designing the system and developing the prototype progressed continuously and on schedule, allowing the thesis to be completed well ahead of the deadline. After implementing the basic functional version of the entire system, the student methodically elaborated on individual components and finalised the user interaction process. The final content and evaluation experiments were fully consulted, and the supervisor's recommendations were incorporated.

Publication activity, awards

The project was presented at the Excel@FIT 2026 student conference.

Points proposed by supervisor: 95

Grade proposed by supervisor: A

Reviewer’s report
Ing. Marek Vaško

Overall, the thesis presents an interesting approach to semantic segmentation of a 3D scene based on 2D segmentation models. The author implemented a functional application and demonstrated its usability through a user study. I evaluate the handling of literature somewhat negatively, and I note that only a single final processing pipeline was proposed without an analysis of alternative approaches. Overall, I grade the thesis as C.

Evaluation criteria Verbal classification Points
The difficulty of the assignment

Evaluation level: more difficult assignment

The thesis requires integrating several existing systems to accomplish individual tasks, including working with computer vision models, using augmented reality systems, and implementing both the application and its supporting information system. Due to the complexity of the required system, the assignment is more demanding than typical.

Presentation level of the technical report

The work is logically structured and generally well written. A minor issue is the repetition of information between the introduction and Section 3.1, where the work's motivation is explained again. In addition, Section 3.1 serves more as a review of existing solutions than as a design of the author's own solution.

85
Formal preparation of a technical report

The thesis is easy to read, with only minor formal shortcomings.

My primary criticism concerns the use of certain terminology; for example, Chapter 3 claims that it “motivates and defines” the system presented in the thesis.

85
Realisation output

The work was not demonstrated to me personally, so I can evaluate it only based on the submitted video and the code documentation, since the overall app has narrow system requirements.

The pipeline is a reasonable prototype demonstrating the concept from a user-experience perspective. However, the geometric quality is significantly limited by the LiDAR point cloud's density. More generally, quality appears to be influenced by the author's iterative point cloud addition approach, which is prone to error accumulation.

The metrics used for quality evaluation appear appropriate. On the other hand, metrics for the overall structural accuracy of the surfaces appear to be absent. The abstraction and implementation of the application, as well as the separation of detection logic, are good.

The application shown in the submitted video appears functional. I also positively evaluate the user testing included in the penultimate chapter.

The evaluation lacks discussion of alternative approaches and proposals for improving the overall system.

85
Usability of results

Overall, the application appears functional. However, the quality of the generated scans is somewhat inferior to that of other solutions; for example, the Heges Scanner (and other applications) on the iPhone could be considered. In its current form, I do not see strong publication potential.

The extent to which the requirements of the assignment have been met

Evaluation level: assignment fulfilled

Extent of the technical report

Evaluation level: is within the usual extent

Work with literature

The bibliography contains only nine sources.

However, the theoretical part appears to draw on a greater number of sources, including documentation for the individual frameworks. In principle, these sources should also be included in the bibliography, especially when information presented in the theoretical section is based on them. In the thesis, however, they are referenced through page footnotes.

Footnotes also contain citations to articles, which significantly deviates from standard citation practices.

Regarding the quality of the cited sources, the references to the models used are the most relevant. However, an overview of other possible solutions is missing. Such a review would help assess whether the selected models are appropriate for the task and whether another path is more suitable.

55
Topics for thesis defence:
  1. How would you improve the current system? Where do you see its main current limitation?
  2. What is the difference between the instance segmentation masks produced by YOLO and the refined masks generated by FastSAM?
  3. Why do you not use the internal representation of the registered scene provided by ARKit for creating the 3D object?
  4. What is the variation of segmentation masks across different frames?
Points proposed by reviewer: 75

Grade proposed by reviewer: C

Responsibility: Mgr. et Mgr. Hana Odstrčilová