Erratic server behavior detection using machine learning on streams of monitoring data

0538411 - FZÚ 2021 RIV FR eng C - Conference Paper (international conference)
Adam, Martin - Magnoni, L. - Pilát, M. - Adamová, Dagmar
Erratic server behavior detection using machine learning on streams of monitoring data.
EPJ Web of Conferences. Vol. 245. Les Ulis: EDP Sciences, 2020 - (Doglioni, C.; Jackson, P.; Kamleh, W.; Kim, D.; Silvestris, L.; Stewart, G.), s. 1-8, č. článku 07002. ISSN 2100-014X.
[International Conference on Computing in High Energy and Nuclear Physics (CHEP 2019) /24./. Adelaide (AU), 04.11.2019-08.11.2019]
R&D Projects: GA MŠMT LM2015058; GA MŠMT EF16_013/0001404; GA MŠMT(CZ) LM2018104
Grant - others:OP VVV - CERN-C(XE) CZ.02.1.01/0.0/0.0/16_013/0001404
Institutional support: RVO:68378271 ; RVO:61389005
Keywords : machine learning * monitoring
OECD category: Automation and control systems; Particles and field physics (UJF-V)

With the explosion of the number of distributed applications, a new dynamic server environment emerged grouping servers into clusters, utilization of which depends on the current demand for the application. To provide reliable and smooth services it is crucial to detect and fix possible erratic behavior of individual servers in these clusters. Use of standard techniques for this purpose requires manual work and delivers sub-optimal results. Using only application agnostic monitoring metrics our machine learning based method analyzes the recent performance of the inspected server as well as the state of the rest of the cluster, thus checking not only the behavior of the single server, but the load on the whole distributed application as well. We have implemented our method in a Spark job running in the CERN MONIT infrastructure. In this contribution we present results of testing multiple machine learning algorithms and pre-processing techniques to identify the servers erratic behavior.
Permanent Link: http://hdl.handle.net/11104/0316216