Concept
How can Bayes’ Theorem identify the likely author of a document?
Stephen Davies, Ph.D. Version 2.2.2 Through Discrete Mathematics A Cool Brisk Walk / Chapter 1
"Anyway, all the stuff about diseases and tests is a side note. The main point is that Bayes’ Theorem allows us to recast a search for Pr( X | Y ) into a search for Pr( Y | X ), which is often far easier to find numbers for. One of many computer science applications of Bayes’ Theorem is in text mining. In this field, we computationally analyze the words in documents in order to automatically classify them or form summaries or conclusions about their contents. One goal might be to identify the true author of a document, given samples of the writing of various suspected authors. Consider the Federalist Papers , the group of highly influential 18th century essays that argued for ratifying the Constitution. These essays were jointly authored by Alexander Hamilton, James Madison, and John Jay, but it was uncertain for many years which of these authors wrote which specific essays. Suppose we’re interested in determining which of these three Founding Fathers actually wrote essay #84 in the collection. To do this, the logical approach is to find Pr(Hamilton | essay84), Pr(Madison | essay84), and Pr(Jay | essay84), and then choose the author with the highest probability. But how can we possibly find out Pr(Hamilton | essay84)? “Given that essay #84 has these words in this order, what’s the probability that Hamilton wrote it?” Impossible to know. But with Bayes’ Theorem, we can restructure this in terms of Pr(essay84 | Hamilton) instead. That’s a horse of a different color. We have lots of known samples of Hamilton’s writing (and Madison’s, and Jay’s), so we can ask, “given that Hamilton wrote an essay, what’s the probability that he would have chosen the words that appear in essay #84?” Perhaps essay #84 has a turn of phrase that is very characteristic of Hamilton, and contains certain vocabulary words that Madison never used elsewhere, and has fewer sentences per paragraph than is typical of Jay’s writing. If we can identify the relevant features of the essay and compare them to the writing styles of the candidates, we can use Bayes’ Theorem to estimate the relative probabilities that each of them would have produced that kind of essay. I’m glossing over a lot of details here, but this trick of exchanging one conditional probability for the other is the backbone of this whole technique."
Related Ideas
- How does Bayes’ Theorem make difficult probabilities easier to estimate?Stephen Davies, Ph.D. Version 2.2.2 Through Discrete Mathematics A Cool Brisk Walk · Chapter 1
- What is the probability that someone coming out of a voting booth has a Twitter account?Stephen Davies, Ph.D. Version 2.2.2 Through Discrete Mathematics A Cool Brisk Walk · Chapter 1
- Why is conditional probability not commutative?Stephen Davies, Ph.D. Version 2.2.2 Through Discrete Mathematics A Cool Brisk Walk · Chapter 1
- How does conditional probability revise an estimate using background knowledge?Stephen Davies, Ph.D. Version 2.2.2 Through Discrete Mathematics A Cool Brisk Walk · Chapter 1
- How do Bayesians interpret probability?Stephen Davies, Ph.D. Version 2.2.2 Through Discrete Mathematics A Cool Brisk Walk · Chapter 1
- What is the formula for the Law of Total Probability?Stephen Davies, Ph.D. Version 2.2.2 Through Discrete Mathematics A Cool Brisk Walk · Chapter 1
- How can background knowledge drive a conditional probability to zero or one?Stephen Davies, Ph.D. Version 2.2.2 Through Discrete Mathematics A Cool Brisk Walk · Chapter 1
- How do frequentists define probability?Stephen Davies, Ph.D. Version 2.2.2 Through Discrete Mathematics A Cool Brisk Walk · Chapter 1