Essay · The setup

Chapter 3 of 9

How We Asked the Question

What the project did, in plain English. Why we picked the corpus we did, what counts as ‘Shakespearean influence’, and how we kept ourselves honest.

The hardest part of a question like “how Shakespearean is the Founders’ writing?” is not collecting the data. It’s deciding what would even count as an answer. Does it count if Adams uses the word methinks a few times in his diary? Does it count if Hamilton uses multitudinous (a word Shakespeare popularised) in Federalist 75? Does it count if Washington signs a letter to a soldier as a member of a “band of brothers”, knowing or not knowing that the phrase is from Henry V? Does Franklin’s habit of writing ‘tis across sixty years of letters count, even if he never names Shakespeare?

Different answers to those questions produce wildly different stories. The project’s methodology is the set of choices it made about which to count, how, and what to do when one method finds something another doesn’t.


What we’re working with

The Founders’ corpus is six men’s collected writings: John Adams, Benjamin Franklin, Alexander Hamilton, Thomas Jefferson, James Madison, and George Washington. Letters, diaries, drafts, debates, essays, political papers. Sourced primarily from the Founders Online archive run by the National Archives. Supplementary texts from Project Gutenberg (the Federalist Papers, Franklin’s autobiography, Jefferson’s Notes on the State of Virginia, Madison’s debates of the Constitutional Convention, Washington’s Farewell Address, Hamilton’s Report on Manufactures) were originally ingested too, but a source-level audit found ingestion errors and double-counted material in them, and methodology v3 excludes them from the statistical corpora. Total: 68,287 documents, 23.7 million words.

The Shakespeare side is the Project Gutenberg edition of his complete works: 36 plays plus the Sonnets and A Lover’s Complaint. 38 documents, 891,092 words, about one twenty-seventh the size of the Founders’ corpus.

That size mismatch matters. A measure like “how many different words does the writer use?” will always favour the writer with the bigger corpus, because they had more chances to use unusual words. So whenever a measure depends on corpus size, we have to either correct for it statistically or sample each Founder down to Shakespeare’s size before comparing. The project does both, depending on the measure.


Asking the right question

The first methodological choice is the most important. There are two ways to compare these corpora, and they give different stories.

The first is symmetric: what makes the Founders’ writing different from Shakespeare’s? This question is easy to answer. The corpora differ in genre (correspondence vs drama), period (180-year gap), purpose (political vs literary), and subject (a constitutional republic that didn’t exist in Shakespeare’s lifetime). They differ in vocabulary, sentence rhythm, address conventions, and a hundred other things. You don’t need a study to know that the Federalist Papers don’t sound like Hamlet.

The second framing is asymmetric: which features of Shakespeare’s English persist in the Founders’ writing, and in whom? Shakespeare wrote in 1590-1614. The Founders wrote in 1750-1820. Influence can travel only forward in time, from Shakespeare to the Founders, not the other way around. The relevant question isn’t how the corpora differ; it’s what survives the journey.

Every measurement in the project is built around the asymmetric framing. We first find what’s Shakespeare-distinctive against the Founders, then measure how much of that distinctive material survives in each Founder’s writing. That’s the influence question, operationalised.


Two big kinds of evidence

Shakespearean inheritance leaves two very different kinds of trace in a corpus, and they require different methods to find.

Passage-level evidence is the kind a literary scholar would look for. Did Adams write “there is a tide in the affairs of men” in a letter? Yes, on five separate occasions. Did Washington write “band of brothers” in the General Orders at Valley Forge? Yes, on 6 April 1778. These are specific passages that can be located, dated, and verified. The project catalogues them by scanning every Founder document for any run of five or more consecutive words, with enough substantive content words to count, that also appears in Shakespeare. Each match is then tiered by confidence: runs of seven or more words make the top tier; shorter matches are medium; everything else falls below the threshold for the published catalogue.

Pattern-level evidence is what a corpus linguist looks for, and it’s invisible to passage-level scans. Does the Founder use old-fashioned English forms (thou, hath, ‘tis) at high rates? Are their preferred metaphors for political institutions the same kinds Shakespeare reaches for? Do the words that hang around honour in their writing resemble the words that hang around honour in Shakespeare’s? Do their pronoun-distributions, sentence rhythms, and emotional registers overlap with his? These are statistical questions, answered by counting features across the whole corpus rather than locating specific passages.

Neither kind of evidence is sufficient on its own. A pure passage-level scan would tell you Adams is overwhelmingly the most Shakespearean Founder and Franklin barely Shakespearean at all; it would miss Franklin’s entire absorbed Shakespearean register. A pure pattern-level scan would clearly see Franklin’s register and might dismiss Adams’s specific quotations as too rare to be a real signal; it would miss Adams’s deliberate engagement. The project runs both kinds of analysis and compares what each finds.

Between those two extremes sits a third, intermediate tier: below the strict catalogue, above the pattern-level signal. These are short verbatim matches (four or five consecutive words containing at least one distinctively Shakespearean content word) and thematic invocations of Shakespearean characters as types. The strict catalogue would filter them out; the pattern-level statistics smooth them away. Read on their own they recover phrases like a pound of flesh (Jefferson 1790), full of sound and fury (Adams 1758 and again in 1813), and another Sir John Falstaff (Adams 1776). Most short matches between any two large English corpora are coincidence, so the middle tier is published with explicit caveats; the candidate echoes and thematic allusions explorers let you judge each case yourself.


Eleven independent ways of measuring it

Pattern-level questions have many possible answers. To avoid getting fooled by any single statistical artefact, the project runs eleven independent analyses on eleven different aspects of the corpus. Each one ranks the six Founders from most Shakespearean to least.

Briefly, the eleven are:

  1. Overall ranking: the composite of the ten base methods below.
  2. Pronoun distribution: the relative rates of I, you, he, she, lord, sir, friend, etc.
  3. Archaic forms: the survival of old-fashioned English (thou, thee, hath, doth, ‘tis) that was vanishing by 1750.
  4. Metaphor patterns: what kinds of source-domain metaphors (body, ship, fire, building, plant) each writer reaches for when describing political institutions.
  5. Statistical style: sentence-level features like sentence length, punctuation density, and emotional charge, and which features the Founder shares with Shakespeare.
  6. Use of popular Shakespeare-coined phrases: twenty-four phrases that compilations like Crystal’s Think on My Words attribute to Shakespeare, and whether the Founder uses them.
  7. Shakespearean vocabulary: how many words distinctive of Shakespeare survive in the Founder’s writing.
  8. Shakespearean context patterns: the specific neighbours Shakespeare’s favourite abstract nouns travel with, and whether the Founder uses any of those neighbours too.
  9. Verified Shakespeare references per million words: the strict catalogue of hand-verified references divided by each Founder’s corpus size.
  10. Thematic character invocations per million words: recognised character-as-type passages per million words.
  11. Candidate-echo density per million words: MEDIUM-or-HIGH-confidence short verbatim matches per million words.

The first seven measure stylistic and statistical patterns; the last three count passage-level evidence directly. How they agree, and where they disagree, is the substance of the Eleven Ways of Looking essay, which shouldn’t be skipped if you want to see the joint reading they produce.


The composite score: how the ranking is built

The single number that appears on every Founder’s profile and powers the ranking (Adams 0.87, Franklin 0.78, Jefferson 0.65, Washington 0.34, Hamilton 0.25, Madison 0.11) is not a probability, not a similarity coefficient, and not a raw count. It is the inverse of average rank position across ten base methods, rescaled to a 0–1 range.

Here is how it’s built. Ten separate measures of Shakespearean inheritance are computed independently for each Founder against the full corpus. Seven of them are statistical or stylistic:

  1. Pronoun-distribution similarity. How closely each Founder’s pronoun and address-term usage matches Shakespeare’s.
  2. Old-fashioned word survival. How many archaic Shakespearean forms (hath, doth, thou, methinks, prithee, whilst, amongst) each Founder still uses.
  3. Metaphor pattern similarity. Distance between each Founder’s metaphor-family distribution (body, edifice, ship, path, fire) and Shakespeare’s.
  4. Statistical-style overlap. Which feature-bin combinations the Founder’s writing shares with Shakespeare’s under Configural Frequency Analysis.
  5. Use of Shakespeare-coined phrases. How often each Founder reaches for the well-known Shakespeare-attributed idioms (band of brothers, pound of flesh, flesh and blood).
  6. Shakespearean vocabulary. Weighted absorption of Shakespeare’s vocabulary items.
  7. Shakespearean context patterns. Collocational absorption: how many of Shakespeare’s bigram and trigram patterns the Founder reproduces.

The remaining three methods are evidence-based and were added after the project expanded its data layers:

  1. Verified Shakespeare references per million words. The 137 hand-verified catalogue references (61 direct quotations + 76 by-name, after the source-level audit) divided by each Founder’s corpus size.
  2. Thematic character invocations per million words. Character-as-type passages per million words. The Shakespeare-only characters (Falstaff, Shylock, Hotspur, Lady Macbeth) are recorded separately from the Roman-ambiguous ones.
  3. Candidate-echo density per million words. MEDIUM-or-HIGH-confidence short verbatim matches per million words of corpus.

For each of those ten measures, the six Founders are ranked from 1 to 6. The overall composite is the inverse of the average rank, rescaled so higher means more Shakespearean. Adams averages rank 1.65 across the ten methods and scores 0.87; Madison averages 5.45 and scores 0.11.

That construction has three properties worth flagging. First, the composite is relative. A Founder’s score depends on the other five he’s being compared to; it is not a fixed similarity to Shakespeare. If we added a seventh Founder, or dropped one, the numbers would shift. Second, the ten measures are equally weighted. Vocabulary breadth counts as much as the verified-references column. We picked equal weighting as the most defensible default. Third, the composite captures both citation and absorbed style. The original seven statistical measures favoured Franklin’s absorbed register; the three evidence-based measures favour Adams’s citational reach. Adams leads because he ranks first or near-first on most measures of both kinds.

The composite is reported alongside each of the ten underlying methods on the Ranking explorer. Where the ten rankings agree, the composite is confirmed. Where they disagree, the disagreement itself is the substantive story (Adams dominates on the citational measures; Franklin dominates on the statistical ones). The Eleven Ways of Looking essay walks through that pattern.

The composite generation lives in the research repository: scripts/export_site_data.py (function export_composite). For each of the ten base methods it produces a per-Founder rank with tied positions averaged; the composite is the inverse of the resulting average rank, rescaled to a 0–1 range. The function writes data/composite.json directly on the site, alongside a data/manifest.json that records the methodology version, generation timestamp, and a SHA-256 hash of each source CSV consumed.


Keeping the project honest

Two practical principles run through everything the project does.

The first is traceability. Every claim on this site corresponds to a row in a CSV file produced by a specific Python script run on the corpus database. Every figure is generated from data, not drawn. Every quotation in the case studies has been pulled directly from the database and verified against the source document. When the Two Modes essay reports that Franklin uses ‘tis at two and a half times the rate of any other Founder, there’s a CSV row and a Python script behind that number, both available in the research repository.

The second is effect size over significance. At the scale of this corpus (roughly 24 million words) almost any comparison between the Founders and Shakespeare produces a statistically “significant” result. That doesn’t mean the difference matters. The right question is: how big is the effect? When the project reports that “Shakespeare’s vocabulary is more diverse than the Founders’ under fair comparison”, the underlying effect size is very weak: statistically present but substantively trivial. The text says so when this happens. Statistical significance is reported but never treated as the headline.


What the project can’t say

Some questions a methodologically honest project should identify up front as outside its reach.

  • The project cannot fully separate Shakespeare- distinctive language from general 18th-century English. To do that cleanly we’d need a third reference corpus (a representative sample of what ordinary educated English looked like in the period) that we don’t have. Some phrases popularly attributed to Shakespeare were already general English by the Founders’ day; the project flags this where it matters.
  • Some Shakespearean material in a Founder’s text may be quotation of someone else who quoted Shakespeare. Madison’s only frequent Shakespearean trace is the word ay, and it surfaces almost entirely in his transcripts of debates: other people’s speech, not his composition.
  • Adams’s diary may inflate his Shakespeare score. A large share of his most-Shakespearean passages are from an intensive cluster of 1758 diary entries in which he was, by his own description, studying Shakespeare. A future split-by-genre analysis (diary vs essays vs correspondence) would let us see whether his Shakespeareanism travels with the genre or with him.
  • The project measures writing, not reading or knowledge. Hamilton owned editions of the plays and, by some accounts, could quote Shakespeare in conversation. What the data tells us is what appears on the page, sentence by sentence. The silence in Hamilton’s writing isn’t silence in his mind.

The scholarly paper walks through twelve such limits and caveats in detail, including a few more technical ones that don’t change the substantive story but should be on the record.

For the full statistical methodology (G log-likelihood tests, Bonferroni correction, sample-size-matched lexical richness, Configural Frequency Analysis on stylometric features) see the scholarly paper or the analysis scripts in the research repository linked from the credits page.