Towards on-line tuning of adaptive-agent’s multivariate meta-parameter

0543581 - ÚTIA 2022 RIV DE eng J - Journal Article
Kárný, Miroslav
Towards on-line tuning of adaptive-agent’s multivariate meta-parameter.
International Journal of Machine Learning and Cybernetics. Roč. 12, č. 9 (2021), s. 2717-2731. ISSN 1868-8071. E-ISSN 1868-808X
R&D Projects: GA MŠMT(CZ) LTC18075
Grant - others:The European Cooperation in Science and Technology (COST)(XE) CA16228
Institutional support: RVO:67985556
Keywords : Bayesian learning * Adaptive agent * Meta-parameter tuning * Fully probabilistic design * Kullback–Leibler divergence * Dynamic decision making
OECD category: Automation and control systems
Impact factor: 4.377, year: 2021
Method of publishing: Limited access
http://library.utia.cas.cz/separaty/2021/AS/karny-0543581.pdf https://link.springer.com/article/10.1007/s13042-021-01358-w

A decision-making (DM) agent models its environment and quantifes its DM preferences. An adaptive agent models them locally nearby the realisation of the behaviour of the closed DM loop. Due to this, a simple tool set often sufces for solving complex dynamic DM tasks. The inspected Bayesian agent relies on a unifed learning and optimisation framework, which works well when tailored by making a range of case-specifc options. Many of them can be made of-line. These options concern the sets of involved variables, the knowledge and preference elicitation, structure estimation, etc. Still, some metaparameters need an on-line choice. This concerns, for instance, a weight balancing exploration with exploitation, a weight refecting agent’s willingness to cooperate, a discounting factor, etc. Such options infuence, often vitally, DM quality and their adaptive tuning is needed. Specifc ways exist, for instance, a data-dependent choice of a forgetting factor serving to tracking of parameter changes. A general methodology is, however, missing. The paper opens a pathway to it. The solution uses a hierarchical feedback exploiting a generic, DM-related, observable, mismodelling indicator. The paper presents and justifes the theoretical concept, outlines and illustrates its use.
Permanent Link: http://hdl.handle.net/11104/0320766