Přístupnostní navigace
E-application
Search Search Close
Bachelor's Thesis
Author of thesis: Denys Chernenko
Acad. year: 2025/2026
Supervisor: Ing. Martin Kostelník
Reviewer: Ing. Michal Hradiš, Ph.D.
Large language models (LLMs) have achieved significant success in natural language processing tasks. However, their training and adaptation require substantial computational resources. Parameter-efficient fine-tuning techniques, such as LoRA (Low-Rank Adaptation) and QLoRA (Quantized Low-Rank Adaptation), aim to reduce these requirements while maintaining competitive performance. This thesis experimentally compares three approaches to adapting large language models: full fine-tuning, LoRA, and QLoRA. The methods are applied to two models of different size, TinyLlama-1.1B and Llama 3.1 8B, on an instruction-following task, with varying LoRA rank and numerical precision. The methods are evaluated with respect to memory consumption, training time, and the quality of generated outputs measured by BERTScore and ROUGE-L. The results show that LoRA and QLoRA achieve quality close to full fine-tuning while requiring significantly less memory and fewer trainable parameters, with QLoRA providing the largest memory savings on the larger model.
LLM, Fine-tuning, Parameter-Efficient Fine-Tuning, LoRA, Quantization, QLoRA
Date of defence
16.06.2026
Result of the defence
Defended (thesis was successfully defended)
Grading
B
Process of defence
Student nejprve prezentoval výsledky, kterých dosáhl v rámci své práce. Komise se poté seznámila s hodnocením vedoucího a posudkem oponenta práce. Student následně odpověděl na otázky oponenta a na další otázky přítomných. Komise se na základě posudku oponenta, hodnocení vedoucího, přednesené prezentace a odpovědí studenta na položené otázky rozhodla práci hodnotit stupněm B.
Topics for thesis defence
Language of thesis
English
Faculty
Fakulta informačních technologií
Department
Department of Computer Graphics and Multimedia
Study programme
Information Technology (BIT)
Composition of Committee
doc. Ing. Lukáš Burget, Ph.D. (předseda) doc. Mgr. Adam Rogalewicz, Ph.D. (místopředseda) Ing. Libor Polčák, Ph.D. (člen) Ing. Michal Hradiš, Ph.D. (člen) Ing. Martin Žádník, Ph.D. (člen)
Supervisor’s reportIng. Martin Kostelník
The student worked actively throughout the entire academic year. He independently completed the tasks agreed upon during consultations and demonstrated the ability to quickly acquire the necessary knowledge and tools. He independently learned to work with the MetaCentrum computing infrastructure, studied modern approaches to training and fine-tuning large language models, and conducted a systematic set of experiments from which he formulated appropriate conclusions.
I see certain shortcomings mainly in the structure of the chapter devoted to experiments, which could have been clearer. Furthermore, it would be appropriate to supplement the evaluation of the results with confidence intervals and a more detailed statistical analysis.
Overall, however, I evaluate the thesis positively and propose a grade of A.
The thesis partially deviated from the original assignment toward the experimental evaluation of parameter-efficient fine-tuning methods for large language models in chatbot applications, specifically instruction fine-tuning using LoRA and QLoRA. However, this focus thematically corresponds to the area of chatbots based on deep neural networks and at the same time goes beyond the knowledge acquired during bachelor studies.
The student had to independently study the principles of large language models, transformer architectures, and modern techniques for efficient model fine-tuning. The thesis also included a systematic experimental evaluation of the individual approaches in terms of both achieved quality and hardware requirements. The student demonstrated the ability to navigate the current state of the field and achieved reasonable results. Therefore, I consider the assignment fulfilled.
The student worked with literature recommended by the supervisor and independently searched for additional sources.
The student was active throughout the entire academic year and regularly consulted his progress. He was prepared for consultations, and the work progressed continuously and at a reasonable pace.
The thesis was prepared well in advance. The first parts of the text were submitted for review already at the beginning of April. The student continuously incorporated feedback, and the completion of the thesis proceeded without time-related complications. The final content was sufficiently consulted.
Grade proposed by supervisor: A
Reviewer’s reportIng. Michal Hradiš, Ph.D.
Although the student deviated from the original thesis topic, the result is a coherent and systematic work. The experiments are rather basic, and the selected models and the dataset may already be outdated.
Evaluation level: průměrně obtížné zadání
The experiments are more computationally demanding, but the thesis is limited to standard methods and approaches.
The text is concise, easy to understand, and follows a clear logical structure. Just the presentation of results could be less verbose and results of experiments should be presented in a single table or graph when the text compares them. I would prefer if chapter and sections did not start with "This chapter defines ...".
The thesis is generally well written, and the typography is also satisfactory. Tables and graphs are consistent and well formatted. I have only one typographical reservation: tables and figures should be placed at the top of the page, their placement in the middle of the page interrupts the flow of the text. A minor additional point is that some equations are wrongly followed by a paragraph break.
Although the experiments are rather basic, they are methodical and informative. However, the selected dataset and models may already be outdated. The source code is well structured, documented, and configurable. Considering the very small differences in the results, some statistical analysis would be appropriate, at least in the form of confidence intervals.
Experiments provide some information and the code could be used for further experiments.
Evaluation level: student se odůvodněně odchýlil od zadání
The student deviated significantly from the assignment and focused on fine-tuning language models. In this form, the thesis is consistent and meaningful.
Evaluation level: je v obvyklém rozmezí
The thesis references 22 relevant, high-quality sources, which are appropriate for the topic and used appropriately throughout the text. The related work could be extended to cover existing models, datasets, and evaluation metrics.
Grade proposed by reviewer: C
Responsibility: Mgr. et Mgr. Hana Odstrčilová