Detail publikačního výsledku

PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark

BELANEC, R.; PECHER, B.; SRBA, I.; BIELIKOVÁ, M.

Originální název

PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark

Anglický název

PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark

Druh

Stať ve sborníku mimo WoS a Scopus

Originální abstrakt

Despite the state-of-the-art performance of Large Language Models (LLMs) achieved on many tasks, their massive scale often leads to high computational and environmental costs, limiting their accessibility. Parameter-Efficient Fine-Tuning (PEFT) methods address this challenge by reducing the number of trainable parameters while maintaining strong downstream performance. Despite the advances in PEFT methods, current evaluations remain limited (in terms of evaluated models and datasets) and difficult to reproduce. To bridge this gap, we introduce PEFT-Bench, a unified end-to-end benchmark for evaluating diverse PEFT methods on autoregressive LLMs. We demonstrate its usage across 27 NLP datasets and 7 PEFT methods. To account for different PEFT training and inference factors, we also introduce the PEFT Soft Cost Penalties (PSCP) metric, which takes trainable parameters, inference speed, and training memory usage into account.

Anglický abstrakt

Despite the state-of-the-art performance of Large Language Models (LLMs) achieved on many tasks, their massive scale often leads to high computational and environmental costs, limiting their accessibility. Parameter-Efficient Fine-Tuning (PEFT) methods address this challenge by reducing the number of trainable parameters while maintaining strong downstream performance. Despite the advances in PEFT methods, current evaluations remain limited (in terms of evaluated models and datasets) and difficult to reproduce. To bridge this gap, we introduce PEFT-Bench, a unified end-to-end benchmark for evaluating diverse PEFT methods on autoregressive LLMs. We demonstrate its usage across 27 NLP datasets and 7 PEFT methods. To account for different PEFT training and inference factors, we also introduce the PEFT Soft Cost Penalties (PSCP) metric, which takes trainable parameters, inference speed, and training memory usage into account.

Klíčová slova

parameter-efficient-training, LLM Efficiency, NLP in resource-constrained settings

Klíčová slova v angličtině

parameter-efficient-training, LLM Efficiency, NLP in resource-constrained settings

Autoři

BELANEC, R.; PECHER, B.; SRBA, I.; BIELIKOVÁ, M.

Vydáno

24.03.2026

Nakladatel

Association for Computational Linguistics

Místo

Morocco

ISBN

979-8-89176-380-7

Kniha

Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)

Strany od

3035

Strany do

3054

URL

https://aclanthology.org/2026.eacl-long.140/

BibTex

@inproceedings{BUT200142,
  author="Róbert {Belanec} and Branislav {Pecher} and Ivan {Srba} and Mária {Bieliková}",
  title="PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark",
  booktitle="Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)",
  year="2026",
  pages="3035--3054",
  publisher="Association for Computational Linguistics",
  address="Morocco",
  doi="10.18653/v1/2026.eacl-long.140",
  isbn="979-8-89176-380-7",
  url="https://aclanthology.org/2026.eacl-long.140/"
}