Course detail

Introduction to Neural Models for AI

FIT-INAaAcad. year: 2026/2027

The course allows students to understand the fundamental building blocks of current generation AI models that are based on large language models and chat bots.

The course starts from fundamental concepts starting from linear models to word embeddings to neural language models like transformers with their applications to specific tasks in natural language processing.

Advanced topics such as instruction tuning, chain-of-thought and reinforcement learning are covered in the final few lectures.

The course builds theoretical concepts with practical sessions (jupyter notebooks, labs) incrementally leading to final tangible projects.


Language of instruction

English

Number of ECTS credits

5

Mode of study

Not applicable.

Guarantor

prof. Dr. Ing. Jan Černocký

Department

Department of Computer Graphics and Multimedia (UPGM)

Entry knowledge

Basic of Programming (Python, C).

Fundamentals of Probability Theory, Statistics, and Mathematical Analysis.

The course also covers basic concepts from the field of probability theory, statistics, and mathematical analysis, but some prior knowledge in these areas will be an advantage.

Rules for evaluation and completion of the course

  1. The final exam will be in written form if there are too many students. Oral exam if the number of students is around 30. (52 points)
  2. Project (24 points) - evaluation method will be decided based on the number of students and groups.
  3. A couple of the class tests will be considered as mid-terms. These will be 30 minute long multiple-choice-question (MCQ) based exams with penalties (~30 min). (16 points).
  4. Non-midterm class tests (MCQ format) will contribute to half the points as mid-term because they will be shorter in duration (~15 min) (8 points).

Aims

Allow students to understand the fundamental building blocks of current generation AI models that are based on large language models and chat bots. The course builds theoretical concepts with practical sessions (jupyter notebooks, labs) incrementally leading to final tangible projects.

Study aids

Not applicable.

Prerequisites and corequisites

Not applicable.

Basic literature

Deep Learning - Foundations and Concepts. Springer https://link.springer.com/book/10.1007/978-3-031-45468-4
Deep learning. MIT Press. https://www.deeplearningbook.org/

Recommended reading

Machine Learning with PyTorch and Scikit-Learn. Packt Publishing Ltd. ISBN-10: 1801819319 ISBN-13: 978-1801819312. https://github.com/rasbt/machine-learning-book
Natural Language Processing with Transformers. O'Reilly https://github.com/nlp-with-transformers

Classification of course in study plans

  • Programme BIT Bachelor's 2 year of study, summer semester, elective
  • Programme BIT Bachelor's 2 year of study, summer semester, elective

Type of course unit

 

Lecture

26 hours, optionally

Teacher / Lecturer

Santosh Kesiraju, Ph.D.
Ing. Martin Fajčík, Ph.D.
prof. Dr. Ing. Jan Černocký

Syllabus

  1. Introduction to present-day AI
  2. Perceptron model 
  3. Word embeddings: purpose, training, interpretation
  4. Multi-layer perceptron (MLPs)
  5. Feedforward NN for Language Modelling
  6. Optimization in MLPs
  7. Recurrent NN for LM
  8. Transformer LM
  9. Fine-tuning pre-trained LMs
  10. Instruction tuning and Chain of thought
  11. Fine-tuning with human preferences
  12. Reinforcement learning - Agentic systems
  13. Poster session - project demonstrations (depending on number of students) or Guest lecture

Seminar

13 hours, optionally

Teacher / Lecturer

prof. Dr. Ing. Jan Černocký

Project

12 hours, optionally

Teacher / Lecturer

prof. Dr. Ing. Jan Černocký

Individual preparation for a lecture

26 hours, optionally

Teacher / Lecturer

prof. Dr. Ing. Jan Černocký

Individual preparation for excercises

13 hours, optionally

Teacher / Lecturer

prof. Dr. Ing. Jan Černocký

Individual preparation for a project work

24 hours, optionally

Teacher / Lecturer

prof. Dr. Ing. Jan Černocký

Individual preparation for a final exam

24 hours, optionally

Teacher / Lecturer

prof. Dr. Ing. Jan Černocký