Publication result detail

Factors affecting the in-context learning abilities of LLMs for dialogue state tracking

HEGDE, P.; KESIRAJU, S.; ŠVEC, J.; SEDLÁČEK, Š.; YUSUF, B.; PLCHOT, O.; DEEPAK, K.; ČERNOCKÝ, J.

Original Title

Factors affecting the in-context learning abilities of LLMs for dialogue state tracking

English Title

Factors affecting the in-context learning abilities of LLMs for dialogue state tracking

Type

Paper in proceedings (conference paper)

Original Abstract

This study explores the application of in-context learning (ICL) to the dialogue state tracking (DST) problem and investigates the factors that influence its effectiveness. We use a sentence embedding based k-nearest neighbour method to retrieve the suitable demonstrations for ICL. The selected demonstrations, along with the test samples, are structured within a template as input to the LLM. We then conduct a systematic study to analyse the impact of factors related to demonstration selection and prompt context on DST performance. This work is conducted using the MultiWoZ2.4 dataset and focuses primarily on the OLMo-7B-instruct, Mistral-7B-Instruct-v0.3, and Llama3.2-3B-Instruct models. Our findings provide several useful insights on in-context learning abilities of LLMs for dialogue state tracking.

English abstract

This study explores the application of in-context learning (ICL) to the dialogue state tracking (DST) problem and investigates the factors that influence its effectiveness. We use a sentence embedding based k-nearest neighbour method to retrieve the suitable demonstrations for ICL. The selected demonstrations, along with the test samples, are structured within a template as input to the LLM. We then conduct a systematic study to analyse the impact of factors related to demonstration selection and prompt context on DST performance. This work is conducted using the MultiWoZ2.4 dataset and focuses primarily on the OLMo-7B-instruct, Mistral-7B-Instruct-v0.3, and Llama3.2-3B-Instruct models. Our findings provide several useful insights on in-context learning abilities of LLMs for dialogue state tracking.

Keywords

dialog state tracking | in-context learning

Key words in English

dialog state tracking | in-context learning

Authors

HEGDE, P.; KESIRAJU, S.; ŠVEC, J.; SEDLÁČEK, Š.; YUSUF, B.; PLCHOT, O.; DEEPAK, K.; ČERNOCKÝ, J.

Released

17.08.2025

Publisher

International Speech Communication Association

Location

Rotterdam, The Netherlands

Book

Proceedings of the Annual Conference of the International Speech Communication Association Interspeech

Periodical

Interspeech

State

Kingdom of the Netherlands

Pages from

4818

Pages to

4822

Pages count

5

URL

https://www.isca-archive.org/interspeech_2025/hegde25_interspeech.pdf

BibTex

@inproceedings{BUT199388,
  author="Pradyoth {Hegde} and Santosh {Kesiraju} and Ján {Švec} and Šimon {Sedláček} and Bolaji {Yusuf} and Oldřich {Plchot} and  {} and Jan {Černocký}",
  title="Factors affecting the in-context learning abilities of LLMs for dialogue state tracking",
  booktitle="Proceedings of the Annual Conference of the International Speech Communication Association Interspeech",
  year="2025",
  journal="Interspeech",
  pages="4818--4822",
  publisher="International Speech Communication Association",
  address="Rotterdam, The Netherlands",
  doi="10.21437/Interspeech.2025-2071",
  url="https://www.isca-archive.org/interspeech_2025/hegde25_interspeech.pdf"
}

Documents

hegde_interspeech_2025