Detail publikačního výsledku

Factors affecting the in-context learning abilities of LLMs for dialogue state tracking

HEGDE, P.; KESIRAJU, S.; ŠVEC, J.; SEDLÁČEK, Š.; YUSUF, B.; PLCHOT, O.; DEEPAK, K.; ČERNOCKÝ, J.

Originální název

Factors affecting the in-context learning abilities of LLMs for dialogue state tracking

Anglický název

Factors affecting the in-context learning abilities of LLMs for dialogue state tracking

Druh

Stať ve sborníku v databázi WoS či Scopus

Originální abstrakt

This study explores the application of in-context learning (ICL) to the dialogue state tracking (DST) problem and investigates the factors that influence its effectiveness. We use a sentence embedding based k-nearest neighbour method to retrieve the suitable demonstrations for ICL. The selected demonstrations, along with the test samples, are structured within a template as input to the LLM. We then conduct a systematic study to analyse the impact of factors related to demonstration selection and prompt context on DST performance. This work is conducted using the MultiWoZ2.4 dataset and focuses primarily on the OLMo-7B-instruct, Mistral-7B-Instruct-v0.3, and Llama3.2-3B-Instruct models. Our findings provide several useful insights on in-context learning abilities of LLMs for dialogue state tracking.

Anglický abstrakt

This study explores the application of in-context learning (ICL) to the dialogue state tracking (DST) problem and investigates the factors that influence its effectiveness. We use a sentence embedding based k-nearest neighbour method to retrieve the suitable demonstrations for ICL. The selected demonstrations, along with the test samples, are structured within a template as input to the LLM. We then conduct a systematic study to analyse the impact of factors related to demonstration selection and prompt context on DST performance. This work is conducted using the MultiWoZ2.4 dataset and focuses primarily on the OLMo-7B-instruct, Mistral-7B-Instruct-v0.3, and Llama3.2-3B-Instruct models. Our findings provide several useful insights on in-context learning abilities of LLMs for dialogue state tracking.

Klíčová slova

dialog state tracking | in-context learning

Klíčová slova v angličtině

dialog state tracking | in-context learning

Autoři

HEGDE, P.; KESIRAJU, S.; ŠVEC, J.; SEDLÁČEK, Š.; YUSUF, B.; PLCHOT, O.; DEEPAK, K.; ČERNOCKÝ, J.

Rok RIV

2026

Vydáno

17.08.2025

Nakladatel

International Speech Communication Association

Místo

Rotterdam, The Netherlands

Kniha

Proceedings of the Annual Conference of the International Speech Communication Association Interspeech

Periodikum

Interspeech

Stát

Francouzská republika

Strany od

4818

Strany do

4822

Strany počet

5

URL

https://www.isca-archive.org/interspeech_2025/hegde25_interspeech.pdf

BibTex

@inproceedings{BUT199388,
  author="Pradyoth {Hegde} and Santosh {Kesiraju} and Ján {Švec} and Šimon {Sedláček} and Bolaji {Yusuf} and Oldřich {Plchot} and  {} and Jan {Černocký}",
  title="Factors affecting the in-context learning abilities of LLMs for dialogue state tracking",
  booktitle="Proceedings of the Annual Conference of the International Speech Communication Association Interspeech",
  year="2025",
  journal="Interspeech",
  pages="4818--4822",
  publisher="International Speech Communication Association",
  address="Rotterdam, The Netherlands",
  doi="10.21437/Interspeech.2025-2071",
  url="https://www.isca-archive.org/interspeech_2025/hegde25_interspeech.pdf"
}

Dokumenty

hegde_interspeech_2025