Bachelor's Thesis

New Models for Automatic Detection of Performance Degradation

Final Thesis 2.14 MB

Author of thesis: Ing. Šimon Stupinský

Acad. year: 2018/2019

Supervisor: doc. Mgr. Adam Rogalewicz, Ph.D.

Reviewer: doc. RNDr. Milan Češka, Ph.D.

Abstract:

Performance testing is a critical factor in the optimisation of programs during its development, but it is still not so well developed in comparison to functional testing. A framework Perun provides full automation of performance management, thereby contributing to the development of this area. We have introduced three non-parametric approaches to performance data modelling: regressogram, moving average and kernel regression, which were integrated within this framework. We try to achieve appropriate approximations of performance data using these techniques, without the assumption of dependence between two variables, which represents the main advantage in comparison to parametric techniques. Further, we have proposed and implemented two methods for automatic detection of performance changes, which works with all kinds of models within Perun. We have demonstrated our solutions on the real project (Vim), and on the set of the experimental cases, in which we compared proposed solutions with existing. We have achieved decreased time processing about two-thirds and an almost triple improvement in the fitness of data modelling with new modelling approaches. The proposed detection methods detected performance degradation of three specific functions in comparison of two different versions of Vim, where was present a known performance issue.

Keywords:

performance changes, automatic detection, regressogram, moving average, kernel regression

Date of defence

10.06.2019

Result of the defence

Defended (thesis was successfully defended)

znamkaAznamka

Grading

A

Process of defence

Student nejprve prezentoval výsledky, kterých dosáhl v rámci své práce. Komise se poté seznámila s hodnocením vedoucího a posudkem oponenta práce. Student následně odpověděl na otázky oponenta a na další otázky přítomných. Komise se na základě posudku oponenta, hodnocení vedoucího, přednesené prezentace a odpovědí studenta na položené otázky rozhodla práci hodnotit stupněm "A".

Otázky u obhajoby:
1) Můžete demonstrovat fungování metod detekce degradace na konkrétních modelech výkonosti (tj. ukázat modely pro novou a starou verzi programu a demonstrovat detekci degradace)? Tato část mi v BP chybí.
2) Můžete se lépe vymezit proti existujícím (statistickým i jiným) metodám detekce degradace při vývoji SW?

Language of thesis

Czech

Faculty

Department

Study programme

Information Technology (IT-BC-3)

Field of study

Information Technology (BIT)

Composition of Committee

prof. Ing. Tomáš Vojnar, Ph.D. (předseda)
prof. Ing. Jan M. Honzík, CSc. (místopředseda)
Ing. František Grézl, Ph.D. (člen)
doc. Ing. Jan Kořenek, Ph.D. (člen)
Ing. Aleš Smrčka, Ph.D. (člen)

Supervisor’s report
doc. Mgr. Adam Rogalewicz, Ph.D.

Grade proposed by supervisor: A

File inserted by supervisor Size
Hodnocení vedoucího [.pdf] 86,06 kB

Grade proposed by reviewer: A

File inserted by the reviewer Size
Posudek oponenta [.pdf] 88,33 kB

Responsibility: Mgr. et Mgr. Hana Odstrčilová