Bifalgorithm post · Brisk · Chapter 1

How can Bayes’ Theorem identify the likely author of a document?

How can Bayes’ Theorem identify the likely author of a document? (Brisk · Chapter 1) — "Anyway, all the stuff about diseases and tests is a side note. The main…

Ideas in this flowchart

  • Bayes’ theorem exchanges conditional probabilities to make difficult inferences tractable.
  • The theorem recasts a search for Pr( X | Y ) as a search for Pr( Y | X ), which is often easier to quantify.
  • A computer science application of Bayes’ theorem is text mining.
  • Text mining analyzes words in documents to classify them or summarize and infer their contents.
  • A specific goal is identifying the true author of a document from samples of suspected authors’ writing.
  • The Federalist Papers were jointly authored by Hamilton, Madison, and Jay, but the authorship of specific essays was uncertain.
  • The task is to determine which of the three authors wrote essay #84.
  • The direct approach compares Pr(Hamilton | essay84), Pr(Madison | essay84), and Pr(Jay | essay84), choosing the highest probability.
  • Directly finding probabilities such as Pr(Hamilton | essay84) is impossible to know from the essay alone.
  • Bayes’ theorem reformulates the question as Pr(essay84 | Hamilton), and analogously for the other candidates.
  • Known samples of each author’s writing make these likelihoods accessible.
  • Relevant features include characteristic phrases, vocabulary usage, and the number of sentences per paragraph.
  • Comparing these features with the candidates’ styles allows estimation of relative authorship probabilities; this exchange is the technique’s backbone.
Feed/

How can Bayes’ Theorem identify the likely author of a document?

Channel

Brisk

1 page

"Anyway, all the stuff about diseases and tests is a side note. The main point is that Bayes’ Theorem allows us to recast a search for Pr( X | Y ) into a search for Pr( Y | X ), which is often far easier to find numbers for. One of many computer science applications of Bayes’ Theorem is in text mining. In this field, we computationally analyze the words in documents in order to automatically classify them or form summaries or conclusions about their contents. One goal might be to identify the true author of a document, given samples of the writing of various suspected authors. Consider the Federalist Papers , the group of highly influential 18th century essays that argued for ratifying the Constitution. These essays were jointly authored by Alexander Hamilton, James Madison, and John Jay, but it was uncertain for many years which of these authors wrote which specific essays. Suppose we’re interested in determining which of these three Founding Fathers actually wrote essay #84 in the collection. To do this, the logical approach is to find Pr(Hamilton | essay84), Pr(Madison | essay84), and Pr(Jay | essay84), and then choose the author with the highest probability. But how can we possibly find out Pr(Hamilton | essay84)? “Given that essay #84 has these words in this order, what’s the probability that Hamilton wrote it?” Impossible to know. But with Bayes’ Theorem, we can restructure this in terms of Pr(essay84 | Hamilton) instead. That’s a horse of a different color. We have lots of known samples of Hamilton’s writing (and Madison’s, and Jay’s), so we can ask, “given that Hamilton wrote an essay, what’s the probability that he would have chosen the words that appear in essay #84?” Perhaps essay #84 has a turn of phrase that is very characteristic of Hamilton, and contains certain vocabulary words that Madison never used elsewhere, and has fewer sentences per paragraph than is typical of Jay’s writing. If we can identify the relevant features of the essay and compare them to the writing styles of the candidates, we can use Bayes’ Theorem to estimate the relative probabilities that each of them would have produced that kind of essay. I’m glossing over a lot of details here, but this trick of exchanging one conditional probability for the other is the backbone of this whole technique."

Flowchart preview
Series cover
👍
0
0 comments0 shares

Sign in to view and post comments.