Symbiotic Local Search for Small Decision Tree Policies in MDPs

Symbiotic Local Search for Small Decision Tree Policies in MDPs

Stať ve sborníku v databázi WoS či Scopus

We study decision making policies in Markov decision processes (MDPs). Two key performance indicators of such policies are their value and their interpretability. On the one hand, policies that optimize value can be efficiently computed via a plethora of standard methods. However, the representation of these policies may prevent their interpretability. On the other hand, policies with good interpretability, such as policies represented by a small decision tree, are computationally hard to obtain. This paper contributes a local search approach to find policies with good value, represented by small decision trees. Our local search symbiotically combines learning decision trees from value-optimal policies with symbolic approaches that optimize the size of the decision tree within a constrained neighborhood. Our empirical evaluation shows that this combination provides drastically smaller decision trees for MDPs that are significantly larger than what can be handled by optimal decision tree learners.

We study decision making policies in Markov decision processes (MDPs). Two key performance indicators of such policies are their value and their interpretability. On the one hand, policies that optimize value can be efficiently computed via a plethora of standard methods. However, the representation of these policies may prevent their interpretability. On the other hand, policies with good interpretability, such as policies represented by a small decision tree, are computationally hard to obtain. This paper contributes a local search approach to find policies with good value, represented by small decision trees. Our local search symbiotically combines learning decision trees from value-optimal policies with symbolic approaches that optimize the size of the decision tree within a constrained neighborhood. Our empirical evaluation shows that this combination provides drastically smaller decision trees for MDPs that are significantly larger than what can be handled by optimal decision tree learners.

Markov Decision Processes; Decision trees; Local search

Markov Decision Processes; Decision trees; Local search

ANDRIUSHCHENKO, R.; ČEŠKA, M.; CHAKRABORTY, D.; JUNGES, S.; KRETINSKY, J.; MACÁK, F.

21.08.2025

ML Research Press

Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence

Proceedings of Machine Learning Research

Spojené státy americké

132

142

10

https://proceedings.mlr.press/v286/andriushchenko25a.html