last update: 8/17/2026
| SHANGRLA (nealmcb/SHANGRLA) | rlauxe (JohnLCaron/rlauxe) | |
|---|---|---|
| Language | Python | Kotlin (JVM) |
| Status | Research reference implementation | “Work in Progress” production-oriented library |
| Build system | requirements.txt / no formal build | Gradle (multi-module) |
| License | AGPL-3.0 | (uses AGPL via RAIRE-Java dependency) |
port/ test package that cross-validates against SHANGRLA).verify module for third-party audit checking.nsimTrials simulated audits to estimate batch sizes at a target percentile).
Can also use Uniform Sampling for comparison with Corla.dominion_tools.py), SUITE format (via suite_tools.py).Both support phantom CVRs and MVRs with appropriate assorter treatment. rlauxe has deeper analysis of phantom effects on sample size (documented in ClcaErrors.md).
Assertion, Assorter, CVR, TestNonnegMean) are clean, well-documented with docstrings.assertion_audit_utils.py; new_sample_size() references an undefined prng variable (a latent bug); some methods have # [FIX ME] placeholders.find_margins, find_p_values, summarize_status) with printing mixed into logic — not ideal for library use.audit, betting, cli, core, dhondt, estimate, irv, oneaudit, persist, strata, util, verify, workflow.cli package) for command-line use.test_* functions at the bottom of assertion_audit_utils.py, run via if __name__ == "__main__".test_assorter_mean() has a pass # [FIX ME] — it does nothing.audit, betting, cli, core, corla, dhondt, estimate, irv, oneaudit, persist, port, shangrla, strata, util, verify, workflow).testFixtures/ directory for shared test helpers.shangrla/ test package that explicitly cross-validates rlauxe results against SHANGRLA’s Python implementation — important for correctness validation.port/ test package appears to contain ports of SHANGRLA’s original tests.core/src/test/python) for cross-language validation.core/src/test/data and core/src/test/resources..github_disable directory), but the test infrastructure is there.Neither repository has formal code coverage reporting (no coverage.py, no JaCoCo configuration visible).
assertion_audit_utils.py has tests; suite_tools.py (~932 lines), dominion_tools.py, and IRVVisualisationUtils.py have no associated automated tests. The test functions themselves skip some cases (test_assorter_mean is empty).AlphaMart.md, BettingRiskFunctions.md, GeneralizedAdaptiveBetting.md), audit spec (RlauxeSpec.md, 45 KB), case studies (CaseStudies.md), error analysis (ClcaErrors.md), D’Hondt (Dhondt.md, 34 KB), OneAudit use cases, sample populations, verification spec, attacks, developer guide, audit record format.| Dimension | SHANGRLA | rlauxe |
|---|---|---|
| Audit types | CLCA, Polling | CLCA, OneAudit, Polling |
| Contest types | Plurality, Supermajority, IRV | Plurality, IRV, D’Hondt, Runoff |
| Statistical methods | Kaplan family, Wald SPRT | AlphaMart, BettingMart (newer/more optimal) |
| Workflow completeness | Primitives only | End-to-end incl. verification |
| Consistent sampling | No | Yes |
| Multi-round estimation | Basic | Simulation-based |
| Tests | Ad-hoc functions, not automated | Full JUnit suite with cross-validation |
| Coverage | Low (partial, informal) | Better structured (internal IntelliJ) |
| Documentation | Good for research use | Extensive for implementors |
| Code size | ~3,200 lines | 10-15K LOC |
| Production readiness | Research prototype | More mature, though still WIP |
Bottom line: SHANGRLA is the authoritative Python research prototype from Stark et al., excellent for understanding the mathematical foundations and for reproducing specific pilots. rlauxe is a more ambitious, production-oriented Kotlin reimplementation that adds OneAudit, D’Hondt, a complete audit workflow, and the newer betting-based risk functions — at the cost of being a much larger, more complex, still-evolving codebase. The two are complementary: rlauxe explicitly cross-validates against SHANGRLA in its test suite, treating SHANGRLA as the mathematical ground truth.
An RLA’s statistical guarantee only holds if the evidence being tested was committed to before the random seed was drawn. If an election authority could rearrange CVRs, adjust vote totals, or swap which ballots appear in a sample after seeing the seed, they could manipulate the audit to pass even when the election was wrong. Commitment is the mechanism that locks evidence in place and makes the process verifiable by anyone.
SHANGRLA has no commitment model at all. It’s purely a math library — you pass it data and it returns p-values. It trusts that whoever is calling it has provided honest inputs. This is fine for a research tool and for pilots where the auditors are themselves trustworthy, but it doesn’t give the public any independent assurance.
rlauxe formalizes a two-stage commit sequence:
Stage 1 — Election Commitment (before seed is drawn): The election authority publishes and digitally signs:
cardManifest.csv — the full list of physical ballot cards with unique identifiers; defines the entire population that can be sampledcontests.json — the contest descriptions, reported vote totals, and Nc (the trusted bound on valid cards per contest)electionInfo.json — election-level metadatacardPools.csv / cardStyles.json — OneAudit pool descriptions and ballot style maps, if applicableOnce signed and published, these cannot be changed. Specifically, the CardManifest can’t be altered to add, remove, or swap ballot identifiers. The reported vote totals can’t be adjusted. Any verifier can check these against their own copy.
Stage 2 — Audit Commitment (immediately after seed is drawn):
sortedCards.csvThe key point: the authority cannot choose which specific ballots appear in the sample. They can only choose how many ballots per contest to request in each round. The ballots themselves are determined entirely by the committed PRNG order — the first N cards in the sorted manifest that contain contest C.
Each audit round adds three sub-commits:
auditRoundConfigX.json (which contests, how many samples) + samplePrnsX.json (the specific PRNs chosen) — locked before anyone touches paper.sampleMvrsX.csv (the human-read ballots) + sampleCardsX.csv (the matched manifest cards) — locked before the audit algorithm runs.auditStateX.json (the p-values and which contests passed) — the public result.The repository includes a complete worked example of a CLCA election commitment in
core/src/test/data/testRunCli/clca/audit/.
The Stage 1 files are:
electionInfo.json — election name, audit type (CLCA), total card count, number of contestscontests.json — full contest descriptions with candidate names, reported vote counts, and Nc boundscardManifest.csv — every physical ballot card’s unique ID, location, and index (before the seed is drawn, prn column is 0)After the seed is drawn, auditCreationConfig.json records the published seed:
{
"auditType": "CLCA",
"riskLimit": 0.05,
"seed": "-8405a348931706"
}
The sortedCards.csv file then contains every card re-sorted by its PRNG-derived PRN. A round-1 example shows the three sub-commits in round1/: auditRoundConfig1.json, samplePrns1.json (listing the hex PRNs of the sampled cards), sampleCards1.csv, sampleMvrs1.csv, and auditState1.json (final p-values).
rlauxe’s commitment integrity relies on PRNG recomputation rather than external cryptographic file signing. The PRNG is implemented in Prng.kt using HmacSHA256
keyed on the published seed:
PRN(i) = HmacSHA256(seed_bytes, index_bytes)[0..8] // truncated to Long, then absolute value
Any verifier can independently reproduce every PRN by running this deterministic function on the published seed, then confirm that sortedCards.csv lists cards in ascending PRN order and that samplePrnsX.json for each contest contains the lexicographically smallest PRNs for cards in that contest.
The verification logic is implemented in VerifyAuditCommitment.kt, which checks:
sortedCards.csv are in ascending PRN order.Npop and Nphantoms counts agree between contests.json and the manifest.The AuditRecord.md spec states that the Stage 1 files should be “digitally signed and published publicly” before the seed is drawn. In the current implementation the integrity guarantee is provided by: (a) public publication of all files before sampling begins, (b) the deterministic PRNG check described above, and (c) re-computation of p-values from committed MVRs. External PKI-based file signing (e.g., GPG) is the intended future enhancement.
Because every file is published (and can be signed externally), anyone (opposition parties, journalists, academic researchers) can:
sortedCards.csvsamplePrnsX.json contains the smallest PRNs for each contest (i.e., no cherry-picking)The VerifierSpec.md formalizes exactly this: the verifier confirms (a) ballot IDs are unique, (b) PRNs match the PRNG output from the seed, (c) sampled ballots are the ones with the smallest PRNs for each contest, (d) p-value calculations are reproducible.
SHANGRLA lacks all of this. It’s up to whoever runs the audit to establish commitment externally (e.g., by publishing files manually and having them hash-verified). Several real-world deployments have had to build this scaffolding outside SHANGRLA.
In a real audit, you can’t sample one ballot at a time: auditors need to physically locate ballots in storage boxes, transport them, hand them to adjudicators, enter results. That’s expensive per-trip. So audits run in rounds — you commit to a batch size, go get those ballots, audit them all, then decide whether you’re done or need more.
The estimation problem is: how big should the batch be? Underestimate and you need more rounds (more logistics). Overestimate and you hand-audit more ballots than needed (more cost, more time, more auditor fatigue). The “right” answer depends on the true error rate between CVRs and hand-read ballots, which you don’t know before the audit — you only learn it as you go.
Logistics and scheduling: Audit rounds have fixed overhead. In Colorado, batches of ballots must be pulled from county vaults, transported to a central location, and auditors must be scheduled days in advance. An underestimated round means a second trip to the vault. An overestimated round means paying auditors for ballot-reading they didn’t need to do. The cost difference between 1 round vs. 3 rounds for a close contest can be tens of thousands of dollars.
Close-margin contests are highly sensitive: For a 1% margin contest (which is common in down-ballot races), rlauxe’s data shows that the 1-sigma sample-size range might span from ~500 to ~1,200 ballots. If you commit to 500 and the random sample happened to hit slightly more errors than average, you’ll need a second round. If you plan for 1,200 to “be safe,” you audit 700 extra ballots for no benefit. The estimation algorithm attempts to hit the 50th percentile on round 1 (accept ~50% chance of needing another round) and the 80th on subsequent rounds.
Multi-contest elections: When 10–30 contests span a single ballot, different contests reach their risk limits at different times. Some contests with wide margins might satisfy risk after 50 ballots; a tight IRV contest might need 5,000. The estimation must handle each contest separately but consistent sampling means you’re pulling one set of physical ballots that serves all contests simultaneously. Getting the batch size wrong for any single contest cascades into the physical logistics for all the others.
Updating error rates between rounds: After round 1, you have measured data on the actual CVR-vs-MVR error rate. The rlauxe strategy uses round-1 errors to re-estimate for round 2, but (counterintuitively, from their empirical data) using 0% simulated errors for round 1 is actually optimal for minimizing total extra ballots — because the variance is so large that you’re better off starting conservative, measuring actual errors, then recalculating. The practical implication: auditors should not be told “enter your expected error rate” on round 1; they should just use the optimistic default.
Percentile choice is a policy decision: The estimation asks: what percentile of the simulated sample-size distribution should we use? The 50th percentile means the audit will need a second round about half the time. The 90th percentile means it almost always finishes in one round, but uses more ballots. This is not a statistical question — it’s a resource and schedule management decision that election officials need to make based on their operational constraints (how expensive are additional rounds vs. extra ballot-reading?). rlauxe makes this explicit and configurable; SHANGRLA has no such machinery.
Legal/political time pressure: Many jurisdictions have certification deadlines. If the audit of a close contest isn’t complete by the deadline, there can be real consequences — the audit is abandoned, or the result is certified without conclusive audit evidence. Accurate multi-round estimation lets the canvassing board make an informed decision early: “based on round 1 errors, we need approximately 2 more rounds; our deadline is in 8 days; here is whether that’s feasible.”
new_sample_size() in assertion_audit_utils.py has a rudimentary extrapolation: given the current sequence of observed overstatement values, it bootstraps by sampling from the already-observed values to project how many more would be needed. This is useful for understanding how an ongoing audit is tracking, but it: (a) references an undefined prng variable (a bug), (b) has no simulation infrastructure, (c) doesn’t account for multi-contest consistent sampling, and (d) produces no percentile-based planning estimates.