Abstract
Bayesian decision making (DM) quantifies information by the probability density (pd) of treated variables. Gradual accumulation of information during acting increases the DM quality reachable by an agent exploiting it. The inspected accumulation way uses a parametric model forecasting observable DM outcomes and updates the posterior pd of its unknown parameter. In the thought multi-agent case, a neighbouring agent, moreover, provides a privately-designed pd forecasting the same observation. This pd may notably enrich the information of the focal agent. Bayes’ rule is a unique deductive tool for a lossless compression of the information brought by the observations. It does not suit to processing of the forecasting pd. The paper extends solutions of this case. It: \(\triangleright\) refines the Bayes’-rule-like use of the neighbour’s forecasting pd \(\triangleright\) deductively complements former solutions so that the learnable neighbour’s reliability can be taken into account \(\triangleright\) specialises the result to the exponential family, which shows the high potential of this information processing \(\triangleright\) cares about exploiting population statistics.
Similar content being viewed by others
Availability of data and material
Not applicable.
Notes
The existence of regular probability densities of inspected probabilistic measures with respect to Lebesgue’s or counting measures is assumed [39].
Our manipulations assume discrete-valued modelled variables. The uncertain pds acting on them are probabilistic vectors and their distributions are then modelled without technicalities of the measure theory. The found solution is valid without this assumption.
The work [43] calls the same functional “cross-entropy”. The use of this term is often challenged so we stay with the name “Kulback–Leibler divergence”.
The agreed implicit conditioning on the agent’s action, \(a_{\mathfrak {a}}\), and its regressor, \(r_{\mathfrak {a}}\), applies.
It uses the implicit conditioning \(\mathsf {F}_{\mathfrak {n}}(o_{\mathfrak {a}})=\mathsf {F}_{\mathfrak {n}}(o_{\mathfrak {a}}|a_{\mathfrak {a}},r_{\mathfrak {a}})\), \(\mathsf {M}_{\mathfrak {a}}(o_{\mathfrak {a}}|p)=\mathsf {M}_{\mathfrak {a}}(o_{\mathfrak {a}}|p,a_{\mathfrak {a}},r_{\mathfrak {a}})\). The assumed neighbour, see Sect. 2, implies the relevance of \(a_{\mathfrak {a}},r_{\mathfrak {a}}\) in the forecasting pd \(\mathsf {F}_{\mathfrak {n}}\).
Let us stress that the neighbour, \(\mathfrak {n}\), is generally unaware of the model, \(\mathsf {M}_{\mathfrak {a}}\), and its parameter, p.
It uses the KLDs of posterior pds not the KLDs of joint pds.
References
Antoniak C (1974) Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann Stat 2(6):1152–1174
Åström K (1970) Introduction to stochastic control. Academic Press, New York
Bader K, Lussier B, Schon W (2017) A fault tolerant architecture for data fusion: a real application of Kalman filters for mobile robot localization. Robot Auton Syst 88:11–23
Bar-Shalom Y, Li X, Kirubarajan T (2003) Estimation with applications to tracking and navigation. Wiley, Hoboken
Barndorff-Nielsen O (1978) Information and exponential families in statistical theory. Wiley, New York
Berger J (1985) Statistical decision theory and Bayesian analysis. Springer, Berlin
Bernardo J (1979) Expected information as expected utility. Ann Stat 7:686–690
Bogdan P, Pedram M (2018) Toward enabling automated cognition and decision-making in complex cyber-physical systems. In: 2018 IEEE ISCAS, pp 1–4
Foley C, Quinn A (2018) Fully probabilistic design for knowledge transfer in a pair of Kalman filters. IEEE Signal Proc Lett 25(4):487–490
Galeano P, Pena D (2019) Data science, big data and statistics. Test 28:289–325
Genest C, Zidek J (1986) Combining probability distributions: a critique and annotated bibliography. Stat Sci 1(1):114–148
Hall D, Llinas J (1997) An introduction to multisensor data fusion. Proc IEEE 85(1):6–23
Hlaváčková-Schindler K, Naumova V, Pereverzyev S (2016) Granger causality for ill-posed problems: ideas, methods, and application in life sciences. In: Wiedermann W, von Eye A (eds) Statistics and causality: methods for applied empirical research, Wiley, pp 249–276
Hoshino T, Igari R (2017) Quasi-Bayesian inference for latent variable models with external information: application to generalized linear mixed models for biased data. In: Keio-IES Discussion Paper Series 2017–014, Institute for Economics Studies, Keio University
Jazwinski A (1970) Stochastic processes and filtering theory. The Press of Atlantic City, Pleasantville
Jensen F (2001) Bayesian networks and decision graphs. Springer, New York
Kárný M, Bodini A, Guy T, Kracík J, Nedoma P, Ruggeri F (2014) Fully probabilistic knowledge expression and incorporation. Stat Interface 7(4):503–515
Kárný M, Böhm J, Guy T, Jirsa L, Nagy I, Nedoma P, Tesař L (2006) Optimized bayesian dynamic advising: theory and algorithms. Springer, London
Kárný M, Guy T (2012) On support of imperfect Bayesian participants. In: Guy T et al (eds) Decision making with imperfect decision makers, vol 28. Springer, Berlin, pp 29–56 Int. Syst. Ref. Lib
Kárný M, Herzallah R (2017) Scalable harmonization of complex networks with local adaptive controllers. IEEE Trans SMC Syst 47(3):394–404
Kasabov N, Hu Y (2010) Integrated optimisation method for personalised modelling and case studies for medical decision support. Int J Funct Inform Person Med 3(3):236–256
Kern-Isberner G, Lukasiewicz T (2017) Special issue on challenges for reasoning under uncertainty, inconsistency, vagueness, and preferences. Künstl Intell 31:5–8. https://doi.org/10.1007/s13218-016-0479-z
Koopman R (1936) On distributions admitting a sufficient statistic. Trans Am Math Soc 39:399
Kracík J, Kárný M (2005) Merging of data knowledge in Bayesian estimation. In: Filipe J et al (eds) Proceedings of the 2nd International Conference on Informatics in Control, Automation and Robotics, Barcelona, pp 229–232
Kuhn H, Tucker A (1951) Nonlinear programming. In: Proceedings of the 2nd Berkeley Symposium, University of California Press, pp 481–492
Kulhavý R, Zarrop MB (1993) On a general concept of forgetting. Int J Control 58(4):905–924
Kullback S, Leibler R (1951) On information and sufficiency. Ann Math Stat 22:79–87
van Laere J (2009) Challenges for IF performance evaluation in practice. In: 12th Intern. Conf. on Information Fusion, IEEE, Seattle, pp 866–873
Lee H, Lee B, Park K, Elmasri R (2010) Fusion techniques for reliable information: a survey. Intern J Digit Cont Technol Appl 4(2):74–88
Leevy J, Khoshgoftaar T, Bauder R, Seliya N (2018) A survey on addressing high-class imbalance in big data. J Big Data 5(42):1–30
Meng T, Jing X, Yan Z, Pedrycz W (2019) A survey on machine learning for data fusion. Inform Fus. https://doi.org/10.1016/j.inffus.2019.12.001
Mine H, Osaki S (1970) Markovian decision processes. Elsevier, Amsterdam
Nelsen R (1999) An introduction to copulas. Springer, New York
O’Hagan A, et al. (2006) Uncertain judgement: eliciting experts probabilities. J Wiley
Pearl J (1988) Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufman, Burlington
Peterka V (1981) Bayesian system identification. In: Eykhoff P (ed) Trends and progress in system identification, Pergamon Press, Oxford, pp 239–304
Quinn A, Ettler P, Jirsa L, Nagy I, Nedoma P (2003) Probabilistic advisory systems for data-intensive applications. Int J Adapt Control Signal Proc 17(2):133–148
Quinn A, Kárný M, Guy T (2017) Optimal design of priors constrained by external predictors. Int J Approx Reason 84:150–158
Rao M (1987) Measure Theory and Integration. J Wiley
Sassani B, Alahmadi A, Sharifzadeh H (2019) A cluster based collaborative filtering method for improving the performance of recommender systems in e-commerce. In: K. Arai et al (eds) Proceedings of the Future Technologies Conference (FTC) 2018, Advances in Intelligent Systems and Computing, vol 881. Springer, Cham
Savage L (1954) Foundations of statistics. Wiley, Hoboken
Scanagatta M et al (2019) A survey on Bayesian network structure learning from data. Progress AI 8:425–439
Shore J, Johnson R (1980) Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE Trans Inform Theory 26(1):26–37
Smith A, Makov U (1978) A quasi-Bayes sequential procedures for mixtures. J R Stat Soc 40(1):106–112
Tsai C, Lai C, Chiang M, Yang L (2014) Data mining for internet of things: a survey. IEEE Commun Surv Tutor 16(1):77–95
Wang P, Yang L, Li J, Chen J, Hu S (2019) Data fusion in cyber-physical-social systems: State-of-the-art and perspectives. Inform Fus 51:42–57
Xu Z, He Y, Wang X (2019) An overview of probabilistic-based expressions for qualitative decision-making: techniques, comparisons and developments. Int J Mach Learn Cybern 1513–1528:10
Zadeh L (1976) A fuzzy-algorithmic approach to the definition of complex or imprecise concepts. Syst Theory Soc Sci 8:202–282
Acknowledgements
The paper was notably influenced by discussions with Dr. T.V. Guy.
Funding
The reported research has been supported by MŠMT ČR LTC18075 and EU-COST Action CA16228.
Author information
Authors and Affiliations
Contributions
Both authors tightly cooperated on the paper. MK dominated in writing the text and FH in experiments.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no affiliation with any organization with a direct or indirect financial interest in the subject matter discussed in the manuscript. This manuscript has not been submitted to, nor is under review at, another journal or other publishing venue.
Code availability
The code of examples is available at https://gitlab.com/hula-phd/bks.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kárný, M., Hůla, F. Fusion of probabilistic unreliable indirect information into estimation serving to decision making. Int. J. Mach. Learn. & Cyber. 12, 3367–3378 (2021). https://doi.org/10.1007/s13042-021-01359-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-021-01359-9