Přístupnostní navigace
E-application
Search Search Close
Master's Thesis
Author of thesis: Ing. Sathvik Udupa
Acad. year: 2025/2026
Supervisor: Ing. Petr Schwarz, Ph.D.
Reviewer: Ing. Karel Veselý, Ph.D.
Cascaded spoken dialogue systems are bottlenecked by sequential processing latency and unreliable generation on domain-specific queries. This thesis addresses both limitations within the Unmute full-duplex framework. We propose Endpoint Anticipation (EPA), a fixed-horizon binary classification task that forecasts turn completion from acoustic features without dependence on ASR output. A multi-horizon model (EPA-M) is integrated into Unmute via speculative execution, reducing average response latency from 1195\,ms to 808\,ms at an Expected Redundant Computation of 28.4\%. Evaluation against a Voice Activity Projection baseline demonstrates consistent improvements across four proposed metrics quantifying the trade-off between realised latency savings and wasted speculative computation. We further propose a latency-aware RAG integration combining speculative retrieval with context-aware chunking, improving retrieval hit rates by up to 54 percentage points on structured categories. The combined system reduces end-to-end response latency from 1520\,ms to 770\,ms relative to a retrieval-augmented baseline while maintaining answer correctness. Together, the contributions demonstrate that cascaded architectures can support complex real-time processing without sacrificing modularity or controllability.
full-duplex speech model, spoken dialogue system, turn-taking, retrieval-augmented-generation, multimodal interaction models
Date of defence
25.06.2026
Result of the defence
Defended (thesis was successfully defended)
Grading
A
Process of defence
The student first presented the results he had achieved in the thesis. The committee then reviewed the supervisor’s evaluation and the reviewer’s assessment. The student then answered questions from the reviewer and additional questions from the people present at the defense. Based on the reviewer’s assessment, supervisor’s evaluation, the presentation and the the student's answers to the asked questions, the committee has evaluated the thesis with the grade A.
Topics for thesis defence
Language of thesis
English
Faculty
Fakulta informačních technologií
Department
Department of Computer Graphics and Multimedia
Study programme
Master of Information Technology (MIT-EN)
Composition of Committee
prof. Ing. Adam Herout, Ph.D. (předseda) prof. Dr. Ing. Pavel Zemčík, dr. h. c. (místopředseda) doc. RNDr. Milan Češka, Ph.D. (člen) Ing. David Bařina, Ph.D. (člen) doc. Ing. Vítězslav Beran, Ph.D. (člen) Ing. Tomáš Milet, Ph.D. (člen)
Supervisor’s reportIng. Petr Schwarz, Ph.D.
The assignment was fulfilled, and the work was significantly expanded by the Retrieval Augmented Generation (RAG) module, which accesses information from documents through speculative processing to reduce system latency. The output has practical applications in projects carried out by the Speech@FIT research group. The work was accepted for publication at the prestigious InterSpeech conference.
The work was challenging. The student had to study the issues of voice dialogue systems, understand the current shortcomings of state-of-the-art systems and respond to them. The work included research and implementation. The student fulfilled the assignment and, in some areas, went beyond the assignment. The work was based on the needs of research projects being solved at the faculty.
The work was completed well in advance and its definitive content was duly consulted.
The work was presented at Excel@FIT. It was also accepted for publication at the prestigious conference InterSpeech.
The student actively approached obtaining and using study materials, primarily articles from research conferences, journals, models, and source codes from HuggingFace.
The student actively and continuously consulted on the work and was always properly prepared for the consultations.
Grade proposed by supervisor: A
Reviewer’s reportIng. Karel Veselý, Ph.D.
Well done! This work has a very high level, proposing to post it to the competition IT SPY.
Evaluation level: assignment fulfilled, and the work contains significant extensions
Extra work, not in assignment:- Implementation of Retrieval Augmented Generation (RAG) for feeding relevant context to LLM responses
Evaluation level: is within the usual extent
the report is well structured, it reads well and can be used as introduction to the topic area
the language used is professional, in some places the text could be more condensed
the manuscript is richly citing research works highly relevant to the topic
Source code is opensourced on GitHub:
https://github.com/bloodraven66/Low_latency_Spoken_Dialogue_System_RAGThe code is documented, building on top of existing tool "Unmute". It is relatively readable, although the architecture is complex. The python server is using asyncio to achieve the full-duplex streaming.The source code also contains training code for the endpoint anticipation and service code for building the RAG knowledge base from faculty website. The code contribution is large.
Evaluation level: very difficult assignment
The topic is research oriented. Outcome of the work was published in prestigious conference ASRU 2025 (Article: "Streaming Endpointer for Spoken Dialogue using Neural Audio Codecs and Label-Delayed Training", where Sathvick is the 1st author)
Grade proposed by reviewer: A
Responsibility: Mgr. et Mgr. Hana Odstrčilová