12

What's the deal. How does this work, or can you point me to some references? I tried $\mu(A|B) = \mu(A \cap B) / \mu(B)$ and got stuck on $\mu(B) = 0$.

Edit: Sorry for being lazy. My background is the basics of measure theory (working on it): measurable spaces, measurable functions, Lebesgue integral, that's about it so far. I haven't yet learned much about measure theory and probability. I am mainly just curious if there is a "formula" for Bayes' rule in measure theory? And interested in anything relevant.

One motivation is we often model a game in economics by have a finite set of states of the world with a prior distribution, then we learn that the true state is in some subset and update based on Bayes rule. I haven't seen how to model this with an infinite state space (I can only think of special cases where it would work).

Thanks!

SBF
  • 36,041
usul
  • 3,724
  • You can't condition on measure zero sets, it just doesn't make sense. Related question – icurays1 Jan 25 '13 at 21:38
  • I don't get it? You want a measure theoretc treatment of Bayesian statistics? – Michael Greinecker Jan 25 '13 at 21:44
  • Also, did you mean to say $\mu(A\vert B)=\mu(A\cap B)/\mu(B)$? The equation you have isn't correct. – icurays1 Jan 25 '13 at 21:44
  • @icurays1: Yes, of course, sorry. I have fixed it. Unfortunately, it seems no-one in your link has pointed to any rigorous treatment of the problem... – usul Jan 25 '13 at 22:22
  • @MichaelGreinecker: Yes, exactly. Well, I'm more interested in probability than statistics, but sure. – usul Jan 25 '13 at 22:23
  • 3
    @usul I fyour interest is probabilitisc conditioning, the most comprehensive book is probably Conditional Measures and Applications by M.M. Rao. A treatment of Bayesian statistics can be found here. None of these is for the beginner though. – Michael Greinecker Jan 25 '13 at 22:40
  • @Michael - thanks, I'll maybe try to check out the book. I don't have the category theory for most of the paper but it helps...it seems that they are only concerned with showing existence (/uniqueness) of the inference function, not with providing a formula for it, or else I am missing it? – usul Jan 25 '13 at 23:46
  • @MichaelGreinecker also that is a valid answer since this is mainly a reference request (didn't think to add the tag earlier, it's there now) – usul Jan 26 '13 at 00:52
  • 2
    @MichaelGreinecker To send somebody to that reference for "a treatment of Bayesian statistics"? Honestly, this is rather flabbergasting. – Did Jan 26 '13 at 03:20
  • 1
    @usul Can you specfy what your background is and what you want to learn this for? Then I might give you a more tailormade reference. – Michael Greinecker Jan 26 '13 at 10:13
  • @MichaelGreinecker yes, thanks! I have edited the post. – usul Jan 26 '13 at 15:21
  • There's also an intro-level videolecture by P.Orbanz on measure theory and Bayesian inference: http://videolectures.net/mlss09uk_orbanz_fnbm/ – ocramz Aug 09 '14 at 13:43

2 Answers2

9

A classic book on Bayesian statistics, that makes (modest) use of measure theory is Optimal Statistical Decisions by Morris DeGroot.

Of course, Bayes rule holds even in the framework of measure theoretic probability. But for more general treatments of probabilistic conditioning, there is the very abstract framework for conditional expectations due to Kolmogorov based on the Radon-Nikodym theorem. Since the probability of an event equals the expectation of its indicator function, one can use this framework to treat conditional probabilities. More concrete, but less general, is working with regular conditional probabilities. Something these approaches are not going to help you with is conditioning on probability zero events, they do not matter in classical probability theory and they do not matter in Bayesian statistics.

Probability zero events do matter a lot in game theory, where the show up as off-equilibrium-beliefs. But the theory of refinements for Bayesian games with infinite type spaces is not yet satisfactory. For simple Bayesian equilibrium, the ability to build expectations ex ante is enough. The classical treatment of the issue can be found in the 1985 paper Distributional strategies for games with incomplete information by Milgrom and Roberts (in Mathematics of OR). The paper makes great use of the theory of weak convergence, which is quite important in mathematical game theory.

SBF
  • 36,041
Michael Greinecker
  • 32,841
  • 6
  • 80
  • 137
2

The Bayes formula in measure theory is studied on Nonparametric Bayesian. I suggest you to see the slide 11 on

http://stat.columbia.edu/~porbanz/talks/MLSS12_2.pdf

where it is read that "NPB models do not generally satisfy Bayes’ theorem".

Raúl Machado
  • 21