rlauxe

Corla2020 Using votedatabase for Cvrs

9/24/26

We obtained the cvr data from https://votedatabase.com for the Colorado 2020 General elections, and used it to run a real audit. This is for testing purposes only: the data is not official.

votedatabase

missing San Juan county
there are two counties (Monroe and Roosevelt) from some other state
Baca has a copy of Huerfano’s cvrs instead of Baca’s cvrs.
Garfield has an older or ad-hoc export format (added a workaround)
Las Animas only has 106 cards out of ~8000
Las Platas was missing the last 5 contest headers (fixed in our copy of votedatabase)

imprintedId mistakenly converted to date

The CVR imprintedId field has sometimes been munged into a date format, eg “2-2-86” has been converted (presumably by a spreadsheet like Excel) to “02/02/1986”. We are catching that and demunging. Perhaps was changed in the votedatabase processing?
Does not happen in the 2026 primary cvr data (4 counties)
Does not happen in any of the Boulder data

2026-09-24T05:27:31 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Boulder/cvr.csv has 27429 munged imprintedIds out of 205845
2026-09-24T05:27:44 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Cheyenne/cvr.csv has 750 munged imprintedIds out of 1146
2026-09-24T05:27:45 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Clear Creek/cvr.csv has 1456 munged imprintedIds out of 6608
2026-09-24T05:27:45 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Conejos/cvr.csv has 503 munged imprintedIds out of 4404
2026-09-24T05:27:45 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Crowley/cvr.csv has 1327 munged imprintedIds out of 1769
2026-09-24T05:27:45 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Custer/cvr.csv has 706 munged imprintedIds out of 3670
2026-09-24T05:28:39 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Dolores/cvr.csv has 706 munged imprintedIds out of 1501
2026-09-24T05:29:29 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Jackson/cvr.csv has 706 munged imprintedIds out of 886
2026-09-24T05:29:37 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Jefferson/cvr.csv has 23686 munged imprintedIds out of 381840
2026-09-24T05:29:57 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Lake/cvr.csv has 282 munged imprintedIds out of 4010
2026-09-24T05:30:01 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Larimer/cvr.csv has 25772 munged imprintedIds out of 226901
2026-09-24T05:30:22 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Phillips/cvr.csv has 775 munged imprintedIds out of 2562
2026-09-24T05:30:28 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Rio Grande/cvr.csv has 706 munged imprintedIds out of 6451
2026-09-24T05:30:31 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Summit/cvr.csv has 2909 munged imprintedIds out of 18957
2026-09-24T05:30:44 WARN  CorlaRawCvrs: /home/stormy/datadrive/votedatabase/cvr/Colorado/Yuma/cvr.csv has 620 munged imprintedIds out of 5029

auditcenter

Gunnison, San Juan are missing from tabulate.cvs and tabulate_county.csv
Perhaps can use CVR data to substitute for Gunnison (TODO)?. Note github/nealmcb/auditcenter/2020/general/gunnison/GunnisonAuditReport.pdf
Grand,Town of Granby Board of Trustees, “Chris Michalowski, Natascha O’Flaherty, Kristie DeLay, Mary (Cathy) Tindle, Rebecca Quesada” but Natascha O’Flaherty is not listed in tabulate_county.csv. On purpose or accidental ? According to CVRs she has 496 votes.
countyTabs.csv has inconsistent candidate naming

Final Report (from auditcenter)

Each county has a final report in pdf that contains:

Total Ballot Cards In Manifest
Total CVRs in CVR Export File
Total Ballot Cards Audited
Number of Audit Rounds

Round Summary (for each round)
Ballot Cards Audited
Discrepancies (Audited Contests)
Discrepancies (Non‐Audited Contests)
Disagreements (Audited Contests)
Disagreements (Non‐Audited Contests)

Audited Contests 
    Choice Votes Margin Diluted Margin %

List of Discrepencies (for each round)
    Discrepancies Recorded
    No Disagreements Recorded
    Ballot Cards Selected Imprinted ID

The Discrepencies show which ballots, but not which contests

Spot check (need to parse pdf to do this systematically) Total Ballot Cards In Manifest == Total CVRs in CVR Export File Total Ballot Cards In Manifest == county population (total cards) from round.ballotCardCount

TODO most of this data is also in github/nealmcb/auditcenter/2020/general/round_last/stateReport.xlsx

Compare votedatabase CVRS to auditcenter Manifest

Arapahoe

Batch 3-260 record 1-100 has no match in the CVRs.

Assume lost CVRs, turn into phantoms? or redactions?

Boulder

redacted ncards = 2634 need 15 more redacted cards

2649 mvrs with no match in the CVRs

Douglas

from manifest: Douglas,1,Gen-2026,30,295 should be capitalized

Garfield

has a different cvr file format than usual. see Garfield20cvrs.kt
has a different imprintedId than usual see GarfieldManifest.kt.

El Paso

from https://ballotverifier.clerkandrecorder.elpasoco.com/search?e[0]=8

383,204 Cast Vote Records

field	value	description
county	El Paso	county name
population	382583	county population from round.ballotCardCount
manifestCount	382583	number of entries in the manifest
cvrUnredacted	383160	count of unredacted Cvrs
cvrRedacted	44	count of redacted Cvrs
missing	-621	manifestCount - (cvrUnredacted + cvrRedacted)
cvrInManifest	382424	count of Cvrs that match entries in the Manifest
cvrNoManifest	736	count of Cvrs that dont match entries in the Manifest
manifestNoCvr	159	count of Manifest entries that dont match cvrs
minCardsForVote	8	minimum cards needed for missing votes
ngroups	0	number of redacted groups

found 383204 - 382583 = 621 manifest entries.

So votedatabase and new cvrs probably agree, and auditcenter manifest must have been updated.

old (auditcenter) manifest:

tabulator 10
- Manifest tabulator 10 has entries up to batch 888
- Cvrs have 608 sequential entries from 10-889-1 to 10-907-1
tabulator 82
- Manifest tabulator 82 has entries up to batch 195
- Cvrs have 128 sequential entries from 82-196-1 to 82-198-11
possible mixup of batch 82-164 with 82-197 ?
- manifest has batch 82-164 (records 1-115) not in the CVRs
- Cvrs has batch 82-197 (records 1-115) not in the manifest

So 608 + 128 - 115 = 621 more CVRS than manifest

Redactions

Now catching redactions. See Redaction2020.

Do votedatabase and auditcenter agree ?

The votedatabase CVR data very closely match the auditcenter contest vote counts, when we have both. All but 15 contests have cvrs within 1% of the auditcenter, and most are below .1%. It seems likely most differences are due to redacted ballots. See Vote Differences for details.

Summary of data (with 4 missing counties)

missing counties are “Baca”, “Gunnison”, “Las Animas”, “San Juan”, which are left out of the audit
total cards = 4,103,490
total contests = 651
3 contests are below recountMargin of .005
30 contests have only 5 cards
auditable contests = 526 (after removing NoLoser, MinMargin, and MinSize contests)

Uniform vs Style Based Sampling

The Colorado 2020 General Election audit used a risk limit of 4%.

Here are several scenarios that differ in which contests are chosen for the style-based audit. The corla column is what was actually done in the audit, and does not vary across scenarios.

only targeted contests

rlauxe nmvrs = 4153
corla nmvrs = 8245
contests under maxRisk (rlauxe) = 230 / 526 = 43%
contests under maxRisk (corla) = 364 / 526 = 69%

	rlauxe	corla
under maxRisk	230	364
under 5%	234	366
under 10%	259	381
under 20%	297	393
under 30%	321	397

targeted contests and contests with estMvrs <= maxMvrs

maxMvrs = 120
rlauxe nmvrs = 8223
corla nmvrs = 8245
contests under maxRisk (rlauxe) = 482 / 526 = 91%
contests under maxRisk (corla) = 364 / 526 = 69%

	rlauxe	corla
under maxRisk	482	364
under 5%	482	366
under 10%	483	381
under 20%	487	393
under 30%	490	397

targeted plus important contests

targeted
multicounty contests
contestName.startsWith(“Representative to the”)
contestName.startsWith(“State”)

rlauxe nmvrs = 7680 corla nmvrs = 8245 contests under maxRisk (rlauxe) = 379 / 526 = 72% contests under maxRisk (corla) = 364 / 526 = 69%

	rlauxe	corla
under maxRisk	379	364
under 5%	384	366
under 10%	391	381
under 20%	402	393
under 30%	413	397

all contests

rlauxe nmvrs = 19404
corla nmvrs = 8245
contests under maxRisk (rlauxe) = 526 / 526 = 100%
contests under maxRisk (corla) = 364 / 526 = 69%

	rlauxe	corla
under maxRisk	526	364
under 5%	526	366
under 10%	526	381
under 20%	526	393
under 30%	526	397

Experiments with relaxed risk limits

all contests with relaxed risks

if estMvrs > 250, use 20% risk limit
if estMvrs > 150, use 10% risk limit
if estMvrs in [50, 150), use 5% risk kimit
if estMvrs < 50, use audit risk limit

rlauxe nmvrs = 14056
corla nmvrs = 8245
contests under maxRisk (rlauxe) = 526 / 526 = 100%
contests under maxRisk (corla) = 367 / 526 = 69%

	rlauxe	corla
under maxRisk	526	367
under 5%	488	366
under 10%	511	381
under 20%	526	393
under 30%	526	397

all contests > 500 with relaxed risks

if estMvrs > 500, do not include
if estMvrs > 250, use 20% risk limit
if estMvrs > 150, use 10% risk limit
if estMvrs in [50, 150), use 5% risk kimit
if estMvrs < 50, use audit risk limit

rlauxe nmvrs = 11273
corla nmvrs = 8245
contests under maxRisk (rlauxe) = 519 / 526 = 98%
contests under maxRisk (corla) = 367 / 526 = 69%

	rlauxe	corla
under maxRisk	519	367
under 5%	485	366
under 10%	501	381
under 20%	519	393
under 30%	520	397

TODO DONE

contact Philip about uniform algorithm DONE
read Garfield DONE
remove small contests from being audited DONE
remove Baca and Las Animas from tabulation DONE
- could use Baca and Las Animas tabulate_county.csv entries to synthesize cvrs
- could use Gunnison cvrs to synthesize its entry in tabulate_county.csv
add Boulder redacted ballots back in (by simulation). DONE
compare votedatabase and auditcenter votes DONE
compare uniform and style sampling DONE
look for problems and bugs in uniform vs style comparison DONE
in separate election, generate synthetic cvrs for all counties in 2020 and compare to real ones DONE
improve algorithm for generating synthetic cvrs DONE
generate 2024 general election with synthetic cvrs DONE
show plot of incremental cost of adding the low margin contests: what do the lowest n contests cost ? DONE

Appendix A Votes differences between auditcenter and votedatabase

Colorado 2020 Vote Differences Table

(Use the rlauxe viewer to see these interactively)

Appendix B track down bug in countyTabulation

Senator

                                      V contest = {0=1719784, 1=1419004, 2=9730, 3=1376, 4=55861, 5=55, 6=78, 7=4} sumVotes=3205892     Nc=3279180          Nu=73288
  ac =[0=1719784, 1=1419004, 2=9730, 3=8917, 4=55861, 5=55, 6=78, 7=4],  nvotes=3213433 ncards=3268188, undervotes=54755    cvrs = [0=1719445, 1=1418651, 2=9724, 3=8913, 4=55873, 5=55, 6=78, 7=4],  nvotes=3212743 ncards=3268188, undervotes=55445

0 ‘John W. Hickenlooper’: votes=1719784 (winner) 1 ‘Cory Gardner’: votes=1419004 2 ‘Daniel Doyle’: votes=9730 3 ‘Stephan “Seku” Evans’: votes=1376 <——- 4 ‘Raymon Anthony Doane’: votes=55861 5 ‘Danny Skelly’: votes=55 6 ‘Bruce Lohmiller’: votes=78 7 ‘Michael Sanchez’: votes=4

Presidential Electors

                                                      V                                V                                                                      V co={0=1792789, 1=1354251, 2=5024, 3=2716, 4=8922, 5=353,  6=4870, 7=2499, 8=1990, 9=562,  10=60, 11=375, 12=491, 13=350, 14=195, 15=755, 16=1032, 17=609, 18=567,  19=16, 20=8033, 21=0, 22=3, 23=21} ac=[0=1792789, 1=1354251, 2=5024, 3=2716, 4=8922, 5=353, 6=52104, 7=2499, 8=1990, 9=562, 10=635, 11=375, 12=491, 13=350, 14=195, 15=755, 16=1032, 17=609, 18=567, 19=175, 20=8033, 21=0, 22=3, 23=21], cv=[0=1792457, 1=1353905, 2=5021, 3=2717, 4=8923, 5=353, 6=52112, 7=2500, 8=1993, 9=562, 10=635, 11=375, 12=491, 13=350, 14=195, 15=755, 16=1032, 17=609, 18=567, 19=175, 20=8037, 22=3,       23=21], 

all the discrepency is where we have a candidate mismatch, must be in CountyContestBuilder

0 ‘Joseph R. Biden / Kamala D. Harris’: votes=1792789 (winner) 1 ‘Donald J. Trump / Michael R. Pence’: votes=1354251 2 ‘Don Blankenship / William Mohr’: votes=5024 3 ‘Bill Hammons / Eric Bodenstab’: votes=2716 4 ‘Howie Hawkins / Angela Nicole Walker’: votes=8922 5 ‘Blake Huber / Frank Atwood’: votes=353 6 ‘Jo Jorgensen / Jeremy “Spike” Cohen’: votes=4870 <——- 7 ‘Brian Carroll / Amar Patel’: votes=2499 8 ‘Mark Charles / Adrian Wallace’: votes=1990 9 ‘Phil Collins / Billy Joe Parker’: votes=562 10 ‘Roque “Rocky” De La Fuente / Darcy G. Richardson’: votes=60 <——- 11 ‘Dario Hunter / Dawn Neptune Adams’: votes=375 12 ‘Princess Khadijah Maryam Jacob-Fambro / Khadijah Maryam Jacob Sr.’: votes=491 13 ‘Alyson Kennedy / Malcolm Jarrett’: votes=350 14 ‘Joseph Kishore / Norissa Santa Cruz’: votes=195 15 ‘Kyle Kenley Kopitke / Nathan Re Vo Sorenson’: votes=755 16 ‘Gloria La Riva / Sunil Freeman’: votes=1032 17 ‘Joe McHugh / Elizabeth Storm’: votes=609 18 ‘Brock Pierce / Karla Ballard’: votes=567 19 ‘Jordan “Cancer” Scott / Jennifer Tepool’: votes=16 <——- 20 ‘Kanye West / Michelle Tidball’: votes=8033 21 ‘Kasey Wells / Rachel Wells’: votes=0 22 ‘Todd Cella / Timothy Bryan Cella’: votes=3 23 ‘Tom Hoefling / Andy Prior’: votes=21

ac=nvotes=3234451 ncards=3268190, undervotes=33739, cv=nvotes=3233788 ncards=3268190, undervotes=34402,

county=Denver csvfile = /home/stormy/datadrive/votedatabase/cvr/Colorado/Denver/cvr.csv no matches for original candidate ‘Jo Jorgensen / Jeremy ‘‘Spike’’ Cohen’ for contest ‘Presidential Electors’ county ‘Denver’ no matches for original candidate ‘Roque ‘‘Rocky’’ De La Fuente / Darcy G. Richardson’ for contest ‘Presidential Electors’ county ‘Denver’ no matches for original candidate ‘Jordan ‘‘Cancer’’ Scott / Jennifer Tepool’ for contest ‘Presidential Electors’ county ‘Denver’ no matches for original candidate ‘Stephan ‘‘Seku’’ Evans’ for contest ‘United States Senator’ county ‘Denver’

county=Garfield csvfile = /home/stormy/datadrive/votedatabase/cvr/Colorado/Garfield/cvr.csv no matches for original candidate ‘Jo Jorgensen / Jeremy ““Spike”” Cohen’ for contest ‘Presidential Electors’ county ‘Garfield’ no matches for original candidate ‘Roque ““Rocky”” De La Fuente / Darcy G. Richardson’ for contest ‘Presidential Electors’ county ‘Garfield’ no matches for original candidate ‘Princess Khadijah Maryam Jacob-Fambro/Khadijah Maryam Jacob’ for contest ‘Presidential Electors’ county ‘Garfield’ no matches for original candidate ‘Jordan ““Cancer”” Scott / Jennifer Tepool’ for contest ‘Presidential Electors’ county ‘Garfield’ no matches for original candidate ‘Stephan ““Seku”” Evans’ for contest ‘United States Senator’ county ‘Garfield’

county=La Plata csvfile = /home/stormy/datadrive/votedatabase/cvr/Colorado/La Plata/cvr.csv no matches for original candidate ‘Jo Jorgensen / Jeremy Spike Cohen’ for contest ‘Presidential Electors’ county ‘La Plata’ no matches for original candidate ‘Roque Rocky De La Fuente / Darcy G. Richardson’ for contest ‘Presidential Electors’ county ‘La Plata’ no matches for original candidate ‘Jordan Cancer Scott / Jennifer Tepool’ for contest ‘Presidential Electors’ county ‘La Plata’ no matches for original candidate ‘Stephan Seku Evans’ for contest ‘United States Senator’ county ‘La Plata’

3 ‘Stephan “Seku” Evans’: votes=1376 <——-

Contests by County (from auditcenter)

has county contest ids:

county_id,county_name,contest_name,contest_id
1,Adams,Presidential Electors,3739728
1,Adams,United States Senator,3739730
1,Adams,Representative to the 117th United States Congress - District 4,3739732
...
2,Alamosa,Presidential Electors,880210
2,Alamosa,United States Senator,880212
2,Alamosa,Representative to the 117th United States Congress - District 3,880214

that dont seem to relate to anything.