By Vanessa Teague and Andrew Conway.

We recomputed the New South Wales local government elections of 2012.

First, we observed that randomness in preference distributions affects results. One candidate wins with 97% probability in our count, but did not win a seat in the election.

Second, and more significantly, we found an error in the official counting software, which caused a mistake in the count in the council of Griffith - in south central NSW - where candidate Rina Mercuri narrowly missed out on a seat.  

We believe the software error incorrectly decreased Mercuri's winning probability to about 10%. According to our count she should have won with 91% probability.  

None of this implies that there was fraud, or that the elected councillors did not have significant support, or that the official outcome could not have occurred with some (small) probability.  

The NSW Electoral Commission (NSWEC) corrected their code when we pointed out the error, and will make their own announcement today.  

We are not certain that our code is perfect, or that our interpretation of the legislation is correct; we invite interested readers to inspect our code, check our complete results or read our tech report.

Source code for election software should be openly available for public scrutiny

Our discovery, and hence the correction, of this problem, was made possible by NSWEC’s decision to publish full preference data and the functional specification of the algorithm (with code fragments), and to share with us their experiments on the probabilities of different outcomes.  

Similar stories have emerged from other states. When the ACT Electoral Commission released their counting code, researchers at ANU found three bugs that were quickly fixed before any had affected an election outcome. 

When the Victorian Electoral Commission made their electronic voting protocol available to researchers in 2010, we identified a protocol-level security weakness which was rectified before the election.  

If full source code was available there would be even more opportunity to examine the system to find mistakes before, rather than after, the election.  

This is important for NSW and even more urgent for the Australian Senate count.  Finding and fixing bugs is good for democracy and good for the electoral commissions.

The role of randomness in NSW

When a candidate is elected in NSW, their preferences are distributed by randomly selecting excess ballots.  Hence there was a known risk that different random samples might produce different election winners.  

In 2012 there were candidates with a high probability of winning a seat, who didn’t win.  Anne Connon wins 97% of our runs of the Mosman election, but didn’t win a seat in the official count.  Nor did Geoff Peterson (83%, Goulburn Mulwaree), Paul Magee (66%, Boorowa) or Peter Cohilj (67%, Willoughby Sailors Bay).  

This is not the fault of the NSW Electoral Commission– randomness is specified by legislation.

However, neither the source code for the count nor the method of choosing the randomness are observable by the public.  Without a transparent process showing that the randomness is fairly generated, the outcome could be accidentally or deliberately biased.  There is no evidence of bias, but also no evidence that the random choices were fair.  

The “last parcel” computation error and the effect in Griffith

There’s a detail about how the count is actually done in practice that almost nobody thinks is important, but actually in the NSW local council election this detail did make a difference.

This detail is called the “last parcel”. Basically, it’s the last pile of votes that was passed to the candidate in the process of running the count.  When a candidate is elected, only votes in the “last parcel” are distributed to the next preference.

Our program said that Rina Mercuri should get elected about 91% of the time in Griffith - she was not.  Her probability of winning hinges on the calculation of the “last parcel”.  The NSWEC interpretation of “last parcel” is complicated, and their code contained a subtle error that miscounted the “last parcel” in this case.  NSWEC has now corrected the error in time for the 2016 local government elections.

NSWEC has lots of software on which an accurate election outcome depends.  Only this software was fully testable by outsiders.  We find that its output differs from our recount.

Suggestions for improving NSWEC electronic voting and counting

The software error that caused the mistake in Griffith was easy to correct once discovered.

Dealing with randomness is harder.  We support the removal of randomness from the count altogether.  

If this is not feasible before the next election, the NSWEC should use a lotto machine or a public dice roll to generate the randomness.  If the counting code was publicly available, anyone could then recompute the outcome.  This would be good evidence that the process was fair.  

A good process is a double benefit.  It gives candidates and the public confidence that the process was fair; it also gives the Electoral Commission a defence against a candidate who claims the count was biased.

There are two other software stages in NSW elections:

1.The software to enter the paper ballots into the computer

2.The iVote Internet voting software

For the first, scrutineers should be able to audit the paper evidence against the electronic data.

However, vote privacy prevents anyone having access to iVote’s inputs. 

There was some limited private testing. We can’t check it. If there were an error or if a hacker changed the results, there may not be any way to detect the problem.  

Conclusion and opinion on the implications

Our research supports the conclusion of similar studies in Australia and overseas: certified code may contain undetected software errors that impact election results.  

Software certification in NSW has failed to detect both the iVote security vulnerability and the counting error described here.  This is hardly surprising, since almost any complex piece of software is overwhelmingly likely to contain errors or security holes.  

It would be good for democracy, and good for the Electoral Commissions, to make election-related source code public before an election.

That doesn’t guarantee that the software is correct or secure, but it raises the likelihood that errors will be identified and corrected before an election.

Federal counting code

The Australian Electoral Commission (AEC) has successfully defended the secrecy of the code used to count Australian Senate votes, despite both a freedom-of-information request and a Senate motion requesting publication.  

The AEC cited the code’s commercial value, a claim we find weak given that it took us three person-days of work to write a similar program. Of course testing takes a long time… but many interested members of the public would do this for free if they could.  

The code may need swift updating for the new federal voting rules and imminent double dissolution.  It could only help the AEC to allow the public to identify errors and resolve ambiguities before the election, rather than leaving it until afterwards.

Authors: Andrew Conway; Michelle Blom (Computing and Information Systems, University of Melbourne), Rajeev Goré (Australian National University, Research School of Information Sciences and Engineering), Ekaterina Ledbedeva (ANU RSISE), Lee Naish (CIS UoM); Vanessa Teague (CIS UoM)

Image credit: Kjetil Korslien/Flickr