Skip to main content
SpringerLink
Log in

Probabilistic morphisms and Bayesian nonparametrics

  • Regular Article
  • Published:
The European Physical Journal Plus Aims and scope Submit manuscript

Abstract

In this paper we develop a functorial language of probabilistic morphisms and apply it to some basic problems in Bayesian nonparametrics. First we extend and unify the Kleisli category of probabilistic morphisms proposed by Lawvere and Giry with the category of statistical models proposed by Chentsov and Morse–Sacksteder. Then we introduce the notion of a Bayesian statistical model that formalizes the notion of a parameter space with a given prior distribution in Bayesian statistics. We revisit the existence of a posterior distribution, using probabilistic morphisms. In particular, we give an explicit formula for posterior distributions of the Bayesian statistical model, assuming that the underlying parameter space is a Souslin space and the sample space is a subset in a complete connected finite dimensional Riemannian manifold. Then we give a new proof of the existence of Dirichlet measures over any measurable space using a functorial property of the Dirichlet map constructed by Sethuraman.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. In [15, p. 74] Chentsov used the notation \(L({{\mathcal {X}}}, \Sigma _{{\mathcal {X}}})\) which is equivalent to our notation, and in [9, p. 371, vol. 2] Bogachev used the notation \({{\mathcal {L}}}^\infty _{\Sigma _{{\mathcal {X}}}}\) instead of our notation \(L({{\mathcal {X}}})\).

  2. Kallenberg in [30, p.1] defined \(\Sigma _w\) on the space of all locally finite measures on \({{\mathcal {X}}}\) as in [33].

  3. If \({{\mathcal {X}}}\) is infinite, then \(({{\mathcal {S}}}({{\mathcal {X}}}), \tau _w)\) is non-metrizable [11, p. 102] (warning: Bogachev’s \({{\mathcal {M}}}({{\mathcal {X}}})\) is our \({{\mathcal {S}}}({{\mathcal {X}}})\)).

  4. Chentsov called the induced transformation \(P_*(T)\) (see Remark 4(2)) of a probabilistic morphism T a Markov morphism [14].

References

  1. L. Arnold, Random dynamical systems (Springer Monographs in Mathematics. Springer-Verlag, Berlin, 1998)

    Book  Google Scholar 

  2. R.J. Aumann, Borel structures for function spaces. Illinois J. Math. 5(4), 614–630 (1961)

    Article  MathSciNet  Google Scholar 

  3. N. Ay, J. Jost, H.V. Lê, L. Schwachhöfer, Information geometry and sufficient statistics. Probab. Theor. Relat. Fields 162(1), 327–364 (2015)

    Article  MathSciNet  Google Scholar 

  4. N. Ay, J. Jost, H. V. Lê, L. Schwachhöfer, Information geometry, vol. 64 of Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge. A Series of Modern Surveys in Mathematics [Results in Mathematics and Related Areas. 3rd Series. A Series of Modern Surveys in Mathematics]. Springer, Cham, (2017)

  5. N. Ay, J. Jost, H.V. Lê, L. Schwachhöfer, Parametrized measure models. Bernoulli 24(3), 1692–1725 (2018)

    Article  MathSciNet  Google Scholar 

  6. F. Baudoin, Diffusion processes and stochastic calculus (EMS Textbooks in Mathematics, European Mathematical Society (EMS), Zürich, 2014)

    Book  Google Scholar 

  7. J. O. Berger, Statistical decision theory and Bayesian analysis. Springer Series in Statistics. Springer-Verlag, New York, 1993. Corrected reprint of the second (1985) edition

  8. D. Blackwell, Equivalent comparisons of experiments. Ann. Math. Statist. 24(2), 265–272 (1953)

    Article  MathSciNet  Google Scholar 

  9. V.I. Bogachev, Measure Theory, vol. I (II. Springer-Verlag, Berlin, 2007)

  10. V.I. Bogachev, Differentiable Measures and the Malliavin Calculus, Mathematical Surveys and Monographs, vol. 164. (American Mathematical Society, Providence, RI, 2010)

  11. V.I. Bogachev, Weak Convergence of Measures, Mathematical Surveys and Monographs, vol. 234. (American Mathematical Society, Providence, RI, 2018)

  12. L.L. Cam, Sufficiency and approximate sufficiency. Ann. Math. Statist. 35(4), 1419–1455 (1964)

    Article  MathSciNet  Google Scholar 

  13. J.T. Chang, D. Pollard, Conditioning as disintegration. Stat. Neerlandica 51(3), 287–317 (1997)

    Article  MathSciNet  Google Scholar 

  14. N. Chentsov, Categories of mathematical statistics. Doklady of Acad.U.S.S.R. 165, 165 (1965), 511–514

  15. N. N. Chentsov, Statistical decision rules and optimal inference. Izdat. “Nauka”, Moscow, (1972)

  16. K. Cho, B. Jacobs, Disintegration and Bayesian inversion via string diagrams. Math. Struct. Comput. Sci. 29(7), 938–971 (2019)

    Article  MathSciNet  Google Scholar 

  17. C. Dellacherie, P.-A. Meyer, Probabilities and Potential (North Holland, 1978)

  18. H. Federer , Geometric measure theory. Die Grundlehren der mathematischen Wissenschaften, Band. 153, Springer, (1969)

  19. T.S. Ferguson, A Bayesian analysis of some nonparametric problems. Ann. Statist. 1(2), 209–230 (1973)

    Article  MathSciNet  Google Scholar 

  20. T. Fritz, A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics. Adv. Math. 370, 107239 (2020)

    Article  MathSciNet  Google Scholar 

  21. T. Fritz, P. Perrone, Bimonoidal structure of probability monads. Electronic notes in theoretical computer science 341 (2018), 121–149. Proceedings of the thirty-fourth conference on the mathematical foundations of programming semantics (MFPS XXXIV)

  22. M. Gaudard, D. Hadwin, Sigma-algebras on spaces of probability measures. Scand. J. Stat. 16(2), 169–175 (1989)

    MathSciNet  MATH  Google Scholar 

  23. S. Ghosal, A. van der Vaart, Fundamentals of nonparametric Bayesian inference, Cambridge Series in Statistical and Probabilistic Mathematics, vol. 44. (Cambridge University Press, Cambridge, 2017)

  24. J.K. Ghosh, R.V. Ramamoorthi, Bayesian nonparametrics (Springer Series in Statistics. Springer-Verlag, New York, 2003)

    MATH  Google Scholar 

  25. M.A. Giry, categorical approach to probability theory. In Categorical aspects of topology and analysis (Ottawa, Ont., , vol. 915 of Lecture Notes in Math. Springer, Berlin-New York 1982, 68–85 (1980)

  26. P. V. Golubtsov, Monoidal Kleisli category as a background for information transformers theory (in russian). Information Theory and Information Processing 2, 1 (2002), 62–84. Translated from Russian. jip.ru/2002/GOLU1.pdf

  27. J. Jost, Postmodern Analysis, 3rd edn. (Universitext. Springer-Verlag, Berlin, 2005)

  28. J. Jost, H. V. Lê, T. D. Tran, Differentiation of measures on complete Riemannian manifolds. arXiv:2008.13252 (2020)

  29. J. Jost, R. Matveev, J. Portegies, C. Rodrigues, On the regular representation of measures. Commun. Anal. Geom. 27(8), 1799–1823 (2019)

    Article  MathSciNet  Google Scholar 

  30. O. Kallenberg, Random Measures, Theory and Applications, Probability Theory and Stochastic Modelling, vol. 77. (Springer, Cham, 2017)

  31. Y. Kifer, Ergodic Theory of Random Transformations, Progress in Probability and Statistics, vol. 10. (Birkhäuser Boston Inc, Boston, MA, 1986)

  32. Y. Kifer, Random Perturbations of Dynamical Systems, Progress in Probability and Statistics, vol. 16. (Birkhäuser Boston Inc, Boston, MA, 1988)

  33. W. F. Lawvere, The category of probabilistic mappings. Available at https://ncatlab.org/nlab/files/lawvereprobability1962.pdf. (1962)

  34. H. V. Lê, Bayesian statistical models and Bayesian machine learning, preprint, (2019)

  35. H.V. Lê, Diffeological statistical models, the Fisher metric and probabilistic mappings. Mathematics 8(2), 167 (2020)

    Article  MathSciNet  Google Scholar 

  36. S. Mac Lane, Categories for the working mathematician, 6th corrected printing ed., vol. 5 of Graduate Texts in Mathematics. Springer-Verlag, New York, (1994)

  37. N. Morse, R. Sacksteder, Statistical isomorphism. Ann. Math. Statist. 37(1), 203–214 (1966)

    Article  MathSciNet  Google Scholar 

  38. J. Neyman, Sur un teorema concernente le considette statistische sufficienti. G. Ist. Ital. Attuari 6, 320–334 (1935)

    MATH  Google Scholar 

  39. P. Panangaden, The category of markov kernels. Electronic Notes in Theoretical Computer Science 22 (1999), 171–187. PROBMIV’98, First International Workshop on Probabilistic Methods in Verification

  40. P. Panangaden, Labelled Markov Processes (Imperial College Press, London, 2009)

    Book  Google Scholar 

  41. K. R. Parthasarathy, Probability measures on metric spaces. Probability and Mathematical Statistics, No. 3. Academic Press, Inc., New York-London, (1967)

  42. A. J. Parzygnat, Inverses, disintegrations, and bayesian inversion in quantum markov categories. arXiv:2001.08375 (2020)

  43. J. Pfanzagl, Mathematical statistics. Springer Series in Statistics. Springer-Verlag, Berlin, (2017). Essays on history and methodology, Springer Series in Statistics. Perspectives in Statistics

  44. R. Sacksteder, A note on statistical equivalence. The Ann. Math. Stat. 38(3), 787–794 (1967)

    Article  MathSciNet  Google Scholar 

  45. M.J. Schervish, Theory of statistics (Springer Series in Statistics. Springer-Verlag, New York, 1995)

    Book  Google Scholar 

  46. J. Sethuraman, A constructive definition of dirichlet priors. Stat. Sin. 4(2), 639–650 (1994)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Tobias Fritz for discussions on probability monads, Markov category and many useful comments on an early version of this paper, Lorenz Schwachhöfer, Juan Pablo Vigneaux for beneficial comments on different versions of this paper, and Ulrich Menne for alerting us on the relation between [28] and [18, 2.8.9]. We appreciate Duc Hoang Luu for Example 1 (3) and helpful suggestions. We are grateful to the anonymous referee for critical comments and valuable suggestions. HVL would like to thank Domenico Fiorenza and XuanLong Nguyen for helpful comments on the first version of this paper [34]. She warmly thanks JJ, TDT and Duc Hoang Luu for hospitality during her visit to MPI MIS Leipzig in 2019 where a part of this paper was discussed.

Author information

Authors and Affiliations

  1. Max-Planck-Institute for Mathematics in the Sciences, Inselstrasse 22, 04103, Leipzig, Germany

    Jürgen Jost & Tat Dat Tran

  2. Institute of Mathematics of the Czech Academey of Sciences, Zitna 25, 11567, Praha 1, Czech Republic

    Hông Vân Lê

  3. Mathematisches Institut, Universität Leipzig, Augustusplatz 10, 04109, Leipzig, Germany

    Tat Dat Tran

Authors
  1. Jürgen Jost
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hông Vân Lê
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tat Dat Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hông Vân Lê.

Additional information

Research of HVL was supported by GAČR-project 18-01953J and RVO: 67985840.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jost, J., Lê, H.V. & Tran, T.D. Probabilistic morphisms and Bayesian nonparametrics. Eur. Phys. J. Plus 136, 441 (2021). https://doi.org/10.1140/epjp/s13360-021-01427-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1140/epjp/s13360-021-01427-7

Search

Navigation