Publication detail

MediaPipe and Its Suitability for Sign Language Recognition

ŠNAJDER, J. KREJSA, J.

Original Title

MediaPipe and Its Suitability for Sign Language Recognition

Type

article in a collection out of WoS and Scopus

Language

English

Original Abstract

The paper presents the framework MediaPipe as a tool to extract pose features for the task of word-level isolated sign language recognition. It tests the framework’s suitability on the state-of-the-art sign language dataset AUTSL. Extracted sequences of pose features are classified by the Long Short-Term Memory recurrent neural network constructed with the TensorFlow computational library. The paper describes the proposed method flow, preprocessing of the extracted features, and training. Obtained results are then validated on test datasets, and the impact of face landmarks is evaluated. The top-1 accuracy with face landmarks is 49.89 %, while 53.21 % without them.

Keywords

Sign language recognition; MediaPipe; Long Short-Term Memory; neural network; classification

Authors

ŠNAJDER, J.; KREJSA, J.

Released

10. 5. 2023

Publisher

Institute of Thermomechanics of the Czech Academy of Sciences

Location

Prague

ISBN

ISBN 978-80-87012-84

Book

ENGINEERING MECHANICS 2023

Edition

First edition

Edition number

1

ISBN

1805-8256

Periodical

Engineering Mechanics ....

State

Czech Republic

Pages from

251

Pages to

254

Pages count

4

URL

https://www.engmech.cz/improc/2023/251.pdf

BibTex

@inproceedings{BUT184379,
  author="Jan {Šnajder} and Jiří {Krejsa}",
  title="MediaPipe and Its Suitability for Sign Language Recognition",
  booktitle="ENGINEERING MECHANICS 2023",
  year="2023",
  series="First edition",
  journal="Engineering Mechanics ....",
  number="1",
  pages="251--254",
  publisher="Institute of Thermomechanics of the Czech Academy of Sciences",
  address="Prague",
  isbn="ISBN 978-80-87012-84",
  issn="1805-8256",
  url="https://www.engmech.cz/improc/2023/251.pdf"
}