The Most Connected State Pair Is a PDF Setting

State legislatures copy each other. They copy from each other’s bills, and they copy from the interest groups that write model legislation and shop it around statehouses. This is not a fringe phenomenon nor is it a partisan one: when somebody built a tool to measure it, a conservative umbrella group and a liberal one had almost identical hit rates (~9%). It is simply how a lot of policy travels in America.

Measuring it requires finding bills that share text. Political scientists do this to estimate how policy diffuses between states; journalists have done it to identify model bills moving between statehouses. A method everyone in this lineage uses is the one biologists use to align gene sequences (Smith-Waterman local alignment): take two bills, find the longest run of matching words, score it for similarity.

Because this is unsupervised, you’ll need a cutoff. Score everything against everything and you have hundreds of millions of bill pairs with a number attached; somewhere on that scale is the line between coincidence and copying. Where you put that line is where every downstream finding comes from, because every count of how much two states share is computed over the pairs above it.

Here is the scale, and here is a line:

The line sits on the steep slope rather than out in the quiet part of the distribution, so moving it a little moves the population a lot: 53 million pairs sit in the bucket immediately below it.

Data and a Threshold

This data is from a CC0 corpus of state bills. The accompanying paper’s conclusion advises future scholars to work in the upper quantiles, and says that for searching out bill pairs “this can be done through thresholding on high alignment scores (e.g., 50+) to indicate the presence of highly similar policy proposals in legislation.” That said, they acknowledge that such thresholding “may discard informative variation,” and assert that high-score thresholding “would focus the analysis on bill pairs that reliably contain very similar policy enactments, and pose little risk of a false positive.”

That is a balanced statement about high-score thresholding in general, not necessarily a recommendation of 50. Read strictly, that advice is about searching out bill pairs: producing candidates worth a human’s attention. Anybody who opens a candidate and finds a PostScript block will throw it away, which is a fair defence of the sentence as written.

Unfortunately, it stops working at scale. The cutoff selects 2,968,117 pairs, which is not a set anybody reads, and the point of a scored corpus is that the score stands in for the reading. So the question is what the cutoff hands you when nobody looks.

I classified the aligned text of every one of those 2,968,117 pairs. 785,330 of them, 26.46%, are not legislative text at all.

What Is Actually Up There

They are page furniture: the line numbers printed down a page margin, the rule that separates a bill’s summary from its body, and a block of PDF printer settings that was never meant to be read by anyone. Text extraction turned them into words. Two bills produced by the same software then “share” those words, and the aligner scores it like any other match.

That is the part I did not expect. The artifact is not sprinkled through the corpus and then swept up incidentally by the cutoff. It is concentrated almost entirely in the region the cutoff selects. A block of printer settings shared between two documents is the most perfect match in the whole corpus.

It Manufactures Edges

If the artifact were spread evenly it would inflate a denominator and leave the ranking of state pairs alone. It is not spread evenly. A quirk of drafting or scanning software belongs to a jurisdiction, so the damage lands on the pairs that jurisdiction belongs to.

Ranked by how many above-threshold pairs they share, the most connected pair of states in this data is Arizona and Nevada, by a factor of nine over the runner-up. 99.9% of those pairs are two PDFs agreeing about their printer settings.

Strip the artifacts and Arizona-Nevada falls from rank 1 to rank 754 of 1,247. Nine of the top twenty dyads leave the top 100.

Two caveats on that table:

• Ranking by raw pair count rewards states with many bills and good text coverage, and coverage in this corpus varies by 100 percentage points across states. This means the ranking is volume-sensitive before any artifact enters it.
• The removal column only subtracts, so it cannot show a dyad crossing into the top 100 from underneath.

And one thing I am not claiming. This ranking is mine. The paper’s own dyadic validation regresses an aggregate alignment score on diffusion ties across 1,225 state pairs; it does not rank dyads by above-threshold count, and I have not re-estimated any published result. I would expect the contamination to survive a change of statistic, since 460,447 pairs at roughly 440 apiece is a great deal of mass in any sum over az-nv, but I have not run that version and it is not what the census shows.

What the census shows is narrower and needs no model: threshold this deposit at a number its own paper offers, and the highest-degree edge in the resulting network is an extraction artifact.

Where It Comes From

The blocks are not spread across the corpus. Scanning all 571,554 bills finds them in 3,551 Arizona bills, 3,178 Nevada bills, one Delaware bill, and nowhere else. So I fetched the original PDFs from both legislatures and re-extracted them.

These PDFs carry an embedded file attachment holding the Acrobat Distiller job options used to produce them, Flate-compressed inside a /Type /EmbeddedFile stream. For one Arizona bill, decompressing that attachment gives 14,581 bytes against the 14,580 characters sitting in the corpus for that same bill. Opening <<, closing >> setpagedevice, indentation and all 220 newlines included. The two are identical once the attachment’s trailing newline comes off.

The corpus’s text extraction emitted PDF file attachments as document text. Not a font problem, not an encoding problem, and not a property of the source documents. pdftotext walks page content and ignores attachments, which is why it reproduces the block 0 times in 92 from documents that demonstrably contain it.

Two details make this worse than a scoring problem. Smith-Waterman is not a cheap algorithm to run. So to reduce the number of bills one runs this alignment scoring for, resesarchers first perform a coarse retrieval stage to pre-filter comparisons to those with the highest similarity as determined by bag-of-words methods (like BM-25). The block also wins the retrieval: a bill carrying one spends 15.00% of its candidate slots on other bills carrying one, against a 1.36% base rate, and a candidate that retrieval never returned was never scored at all.

The paper anticipated the general worry. Bills are compared against the 500 nearest neighbours Elasticsearch returns, and the authors checked that cap by re-running at 1,000, concluding it introduces no selection bias “because the set of bills that result in modest or large text reuse scores is always smaller than 500.” That reasoning holds when the top 500 is filled by candidates competing on legislative language. What the crowding number shows is a population where the premise is doubtful: across the 6,700 queries carrying a distiller block, an elevenfold enrichment means a large share of those slots is won on a string that is not language at all. I have not re-run the authors’ check, so this is a reason to think it deserves re-running on the artifact-carrying subset, not a finding that it fails.

And the bias cannot be signed. A distiller block is appended after the page text, so excising it can only lower a score. A line-number run sits between two passages of prose, so excising it joins them and the score rises, by up to 63 points. The artifact invents reuse in the pairs it creates and conceals reuse in the pairs it interrupts.

The Problem No Cleaning Fixes

Everything above is a defect. A better extractor and a screening pass fix all of it, and anything built on stripped text is recoverable.

The pairs that survive every screen have a separate problem, and this one is a property of the measure rather than of this corpus. It survives perfect text.

Two states implementing the same federal statute share text word for word, and neither one copied the other. The alignment is exact. The reuse is real. It just does not run between the two states in the edge.

Who Did the “Reading”

I tried a language model first. MiniMax M3 labelled every prose pair in my sample, three times over, and on the 477 where all three passes parsed, its verdicts flipped on 33.5%.

Where it broke is the useful part. The instability was worst exactly at the bar, around 57% stable in the 50 to 99 band against 100% above 500, which left two readings the run could not separate: either the model was unreliable on hard cases, or pairs near the bar are genuinely undecidable from the text.

So I read 80 of the pairs again on a blinded page, stratified toward the 50 to 99 band where the instability lived, with score, bill ids and jurisdictions hidden. And so did Claude Opus, on the identical page. Whereas M3 had been the arms-length instrument, prompted and run in batch, Claude was the model I had been working the audit with in Claude Code. It knew my hypotheses and had already seen some of these spans.

The agreement number below is not an inter-rater statistic. It is a person and a model that shared a hypothesis, labelling the same page. I am reporting it because the structure of the disagreement turned out to matter more than its size, and because the answer to the question the M3 run could not settle came out of it: agreement held at 77.4% in the 50 to 99 band, roughly flat where M3’s own stability had collapsed. Pairs near the bar are decidable. The unreliable instrument was the model that kept changing its mind.

No reading makes this class mostly boilerplate. The stricter reading still puts two thirds of it on unambiguous shared source. The 11% both call generic is what you would guess: enacting clauses, appropriation formulae, quorum boilerplate, bill titles on unrelated subjects. The pairs that survive the artifact screen are, in the main, real reuse.

The third row is the finding and the weakest row in the table. Every pair I called shared source, Claude did too. The split is not about where a line was drawn. Instead, the model could name an upstream source and I could not. Text you cannot place reasonably looks like it might be generic.

The Names Are the Result, Not the Labels

A name is checkable in a way a label is not.

Claude named 37 upstream sources across the 80 pairs. I checked eight of them against published law, choosing the ones the finding leans on hardest:

Seven verbatim, one partial, none refuted, and all eight sit in the disputed column. That is the result. Text that a careful reader calls generic formula is sometimes a named federal act reproduced word for word, and unlike every similarity score in this corpus, that is a claim a stranger can destroy with a search engine.

The twenty-nine unchecked names are not evidence, and the reason is on the record. On a control pair reading this act shall be known and may be cited as the school protection act, M3 confidently named an ALEC model act, the Liberty Preservation Act, which has nothing to do with it. A model asked to name a source will produce a plausible one whether or not it exists. The eight are worth something precisely because they were checked; the rest are leads.

The Part That Survives a Clean Corpus

A threshold on text similarity is selecting two different phenomena and cannot tell them apart. States adopting a private model act is policy diffusion, which is what the measure was built to find. States independently implementing the same federal statute is mandated uniformity, which is a different causal story with the same textual signature. The alignment is equally exact in both cases, which is precisely why no score separates them. A higher threshold does not help. A better aligner does not help. Cleaning the text does not help, because there is nothing wrong with the text.

And the error has structure, which is what makes it bite. States differ in how much federal law they implement and how fast, so a diffusion network built on this measure counts federal conformity as interstate influence, and it does so unevenly across exactly the dyads people draw conclusions from.

None of which is news to the diffusion literature. Vertical influence, federal mandates and common exposure are long-standing confounds in that field, and there are methods built to handle them. The narrow claim is the one worth making, and it is about the instrument: a text-reuse score offers no purchase on the distinction at all, because the two cases have an identical textual signature. Whatever a researcher does about common exposure, they have to do it from outside this measure, and a network drawn straight from alignment scores carries the confound with no way to see it.

The fix is a labelled source: to know whether copying ran between the two jurisdictions in your edge, you have to know what was copied.

How Far That Goes, and How Far It Does Not

The sample is not the population. The 80 pairs were drawn to over-represent the 50 to 99 band, so the 23% is a rate within a deliberately skewed draw and I would not quote it as a corpus rate. Read it as context for the eight verified names rather than as the finding.

There is one human in this design. Me, not blind, holding the hypothesis while labelling. Everything else in the reading pipeline was a model. What that design can establish is bounded, and what it did establish is the checkable half: eight names, seven exact.

What the 80 pairs support is that the distinction exists, that it is not rare, and that where it has been tested it holds. How large it is corpus-wide is unmeasured, and a second human annotator who had never read this audit is the thing that would change it. That is not a run I can do, so the blinded page and both sets of labels ship with the code, and somebody else can.

So, Would a Language Model Have Helped?

I came at this wanting to know whether a language model was useful for legislative text reuse. Having spent the time, the answer is yes, and not where I expected.

The contamination needed no model at all. It is a defect: a few dozen lines of pattern matching find every class, removal is lossless because each one is self-delimiting, and anything built on stripped text is recoverable.

The uniformity confound is the opposite. It is not removable at any threshold and a better similarity score will not touch it. What would separate it is knowing what was copied, which is a provenance question, and provenance is the kind of question a language model is actually good at.

It was the one thing both models did well here. Asked to name upstream sources, M3 produced the Uniform Controlled Substances Act, the Streamlined Sales and Use Tax Agreement, the Uniform Vehicle Code, 18 U.S.C. 2510, and those names were more durable than its own verdicts, agreeing across passes 77% of the time against 66.5% for the labels. None of that batch was checked against published law. The eight that were checked came from the blinded run and seven were exact.

With the condition from the section above attached, and it is not a small one: the behaviour that produces a correct name also produces the ALEC misfire, and nothing in the output tells you which one you are holding. Only a check does.

That is the whole thing in miniature:

A similarity label is an opinion. A named source is a claim. Only the second kind can be refuted.

A wrong name announces itself the moment somebody checks it. A wrong similarity score is invisible forever. The second failure mode is the one you want.

So the honest next artifact here is not a better detector. It is a labelled source corpus, built by something that can propose a name and verified by somebody who checks it.

If You Use This Deposit

Strip all three classes before thresholding. Each is self-delimiting, so removal is exact rather than heuristic.

Three things that are easy to get wrong. Do not just exclude Arizona and Nevada, because the other two classes cover more dyads than the distiller block does. Treat Hawaii separately, because its text is OCR-damaged underneath the artifact, that damage is what its source documents actually say, and neither removal nor re-extraction fixes it. And before removing a run, check that it ascends by one, because a run that ascends on every second position is a statutory rate schedule with a counter threaded through it, and excising it deletes enacted numbers.

Do not trust a high threshold to be safe either. The most reassuring number here, that the band above 500 is 99.1% clean, is an accident: the distiller blocks stop scoring at 442 only because Arizona ran Distiller 8.2.5 and Nevada ran 6.0, so the two job-options files agree only through their shared prefix. Two jurisdictions on the same version would share a byte-identical 14,500-character block, which aligns into the thousands.

The Full Audit

The writeup behind all of this is a technical report: the four independent instruments, the removal experiment and its label-blind control, the scrape-era census that explains why the damage partitions by legislative session rather than by state, the limitations including a cluster bootstrap that widens my own headline interval by a factor of five, and an appendix of the four things earlier drafts got wrong. It is at REPORT.md in the page-furniture repo, alongside the screen itself and the dated results file behind every number here.

The 80-pair adjudication is in there too. The per-pair record has every pair, both labels, the upstream act Claude named where it named one, and which eight I checked against published law. The 29 unchecked names are published as well, because a name nobody has checked is a lead, and I would rather ship it as one than quietly drop it.

Attribution

The corpus was collected by Open States and assembled by Burgess et al. It is CC0; US legislative text is uncopyrightable at origin under Georgia v. Public.Resource.Org (2020).

Burgess, Matthew, Eugenia Giraudy, Julian Katz-Samuels, Joe Walsh, Derek Willis, Lauren Haynes, and Rayid Ghani. “The Legislative Influence Detector: Finding Text Reuse in State Legislation.” Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 57-66, 2016, doi:10.1145/2939672.2939697.

The deposit measured here is Linder, Fridolin J., “Replication Data for: Measuring Policy Similarity Through Bill Text Reuse”, Harvard Dataverse, doi:10.7910/DVN/CZ25GF, accompanying Linder, Desmarais, Burgess and Giraudy, “Text as Policy: Measuring Policy Similarity through Bill Text Reuse”, Policy Studies Journal, doi:10.1111/psj.12257. The prior art is Citron, Daniel T. and Paul Ginsparg, “Patterns of text reuse in a scientific corpus”, PNAS 112(1):25-30, 2015, doi:10.1073/pnas.1415135111.