Detail publikačního výsledku

MediaPipe and Its Suitability for Sign Language Recognition

ŠNAJDER, J.; KREJSA, J.

Originální název

MediaPipe and Its Suitability for Sign Language Recognition

Anglický název

MediaPipe and Its Suitability for Sign Language Recognition

Druh

Stať ve sborníku mimo WoS a Scopus

Originální abstrakt

The paper presents the framework MediaPipe as a tool to extract pose features for the task of word-level isolated sign language recognition. It tests the framework’s suitability on the state-of-the-art sign language dataset AUTSL. Extracted sequences of pose features are classified by the Long Short-Term Memory recurrent neural network constructed with the TensorFlow computational library. The paper describes the proposed method flow, preprocessing of the extracted features, and training. Obtained results are then validated on test datasets, and the impact of face landmarks is evaluated. The top-1 accuracy with face landmarks is 49.89 %, while 53.21 % without them.

Anglický abstrakt

The paper presents the framework MediaPipe as a tool to extract pose features for the task of word-level isolated sign language recognition. It tests the framework’s suitability on the state-of-the-art sign language dataset AUTSL. Extracted sequences of pose features are classified by the Long Short-Term Memory recurrent neural network constructed with the TensorFlow computational library. The paper describes the proposed method flow, preprocessing of the extracted features, and training. Obtained results are then validated on test datasets, and the impact of face landmarks is evaluated. The top-1 accuracy with face landmarks is 49.89 %, while 53.21 % without them.

Klíčová slova

Sign language recognition; MediaPipe; Long Short-Term Memory; neural network; classification

Klíčová slova v angličtině

Sign language recognition; MediaPipe; Long Short-Term Memory; neural network; classification

Autoři

ŠNAJDER, J.; KREJSA, J.

Rok RIV

2024

Vydáno

10.05.2023

Nakladatel

Institute of Thermomechanics of the Czech Academy of Sciences

Místo

Prague

ISBN

ISBN 978-80-87012-84

Kniha

ENGINEERING MECHANICS 2023

Edice

First edition

ISSN

1805-8256

Periodikum

Engineering Mechanics ....

Stát

Česká republika

Strany od

251

Strany do

254

Strany počet

4

URL

https://www.engmech.cz/improc/2023/251.pdf

BibTex

@inproceedings{BUT184379,
  author="Jan {Šnajder} and Jiří {Krejsa}",
  title="MediaPipe and Its Suitability for Sign Language Recognition",
  booktitle="ENGINEERING MECHANICS 2023",
  year="2023",
  series="First edition",
  journal="Engineering Mechanics ....",
  number="1",
  pages="251--254",
  publisher="Institute of Thermomechanics of the Czech Academy of Sciences",
  address="Prague",
  isbn="ISBN 978-80-87012-84",
  issn="1805-8256",
  url="https://www.engmech.cz/improc/2023/251.pdf"
}