How Does Congress Actually Work? Data from 15K Bills

Introduction

Understanding how Congress works requires looking at data, not just following news coverage. I scraped data from Congress.gov to analyze what actually happens to the thousands of bills introduced each session.

This analysis focuses on the 117th Congress (2021-2023), examining 15,000+ bills to understand basic patterns: Which bills get introduced? How many receive votes? What factors influence success?

While there are larger goals like geographic analysis and predictive modeling, this post covers the foundational exploratory analysis and data collection process.

Data Collection

My primary source is Congress.gov, maintained by the Library of Congress. I focused on the 117th Congress (2021-2023), collecting data on resolutions and joint resolutions while omitting amendments and concurrent resolutions.

Data collected:

Technical Implementation

The web crawler used standard Python libraries to handle Congress.gov’s structure. The site loads content dynamically, requiring both static and dynamic scraping approaches.

Implementation details:

• Python for core logic
• Selenium for dynamic page loading
• BeautifulSoup for HTML parsing
• Regex for text processing

I added 5-second delays between requests to avoid overloading the server, resulting in a 3-day collection period. The crawler and processed data are available on GitHub.

For each bill, I queried two pages:

• All info page: https://www.congress.gov/bill/117th-congress/{bill_type}/{bill_id}/all-info
• Text page: https://www.congress.gov/bill/117th-congress/{bill_type}/{bill_id}/text?format=txt

The parsing process involved targeting specific HTML elements and implementing basic caching to avoid redundant requests.

Key Findings

The analysis reveals clear patterns in congressional activity. Most bills never receive votes, and success rates vary significantly by party and policy area.

Legislative Outcomes

The fundamental question: what happens to bills after introduction?

Each bill has a tracker status indicating its position in the legislative process. The eight possible statuses can be grouped into three meaningful categories:

• Introduced: Bills introduced but never voted on
• Stalled: Bills that saw votes but didn’t become law (since the 117th Congress ended, these effectively died)
• Law: Bills signed by the President

Key insights:

• Only 7% of introduced bills ever receive a vote
• Of bills that receive votes, 36% become law
• Overall, just 2% of introduced bills become law

Sponsor Analysis

The bill sponsor—the primary member who introduces legislation—provides insights into party and geographic patterns.

Party Breakdown

Party comparison:

• Democrats: 7.5% of bills moved beyond introduction; 2.6% became law
• Republicans: 5.5% of bills moved beyond introduction; 2.1% became law
• When bills do advance, Republicans have a slightly higher success rate (38% vs 35%)

Geographic Distribution

Top 10 states by bills introduced:

Per-representative normalization reveals different patterns:

Top Individual Sponsors

Most prolific legislators by bills introduced:

Effectiveness score (laws enacted / total bills):

$$ \text{effectiveness} = \frac{\text{bills that became law}}{\text{total bills introduced}} $$

Policy Focus Areas

Each bill is assigned a primary policy area. Here are the most active areas by legislative outcome:

Notable patterns: Health dominates introductions but has lower success rates, while government operations and armed forces bills are more likely to become law.

Next Steps

This analysis establishes baseline patterns: most bills fail, party affiliation affects success rates, and certain policy areas perform better than others.

Future work could explore:

• Committee dynamics and voting patterns
• Geographic analysis of state-level interests
• Bill text analysis using NLP techniques
• Predictive modeling for bill outcomes

Update: I’ve since applied machine learning to this type of data in Congressional Bill Policy Area Classification, using 48K+ bills from three Congresses to automatically categorize bills by policy area.

The complete dataset and code are publicly available to support further research into legislative transparency.