Commission on Electronic Voting: summary and conclusions
4.1 General observations On the basis of its investigations and its review of expert reports, submissions received and other relevant information, the Commission has noted the following in relation to the chosen system:
the physical layout of the system is straightforward, contributing to ease of use by both voters and election officials;
the system eliminates many inadvertent voter errors as well as the need for subjective judgment by returning officers;
the system was deployed in pilot tests at previous elections and a referendum in Ireland;
a system designed and manufactured by the same suppliers is in use at elections in the Netherlands and Germany;
testing carried out by experts retained by the Commission on a significant sample of the voting machines deployed to returning officers confirms that the system can accurately and consistently record voter preferences;
testing of the counting software carried out by experts retained by the Commission using voting information from pilot tests during previous elections in Ireland confirms that it accurately counted the votes recorded at these elections;
parallel testing of the counting software programme carried out by experts retained by the Commission using a large number of sample data sets and a similar counting programme developed for the Commission confirms that it can accurately count votes in most situations, including unusual or difficult electoral situations;
miniature end-to-end testing of the system carried out by experts retained by the Commission confirms that it can accurately record and count the votes in the context of multiple simultaneous elections;
election results can be calculated and published quickly; use of the system may secure future reductions in election costs.
4.2 Testing, accuracy and secrecy
However, and within the timeframe of this report, the Commission has not been able to satisfy itself sufficiently as to the accuracy and secrecy of the chosen system. The concerns of the Commission in this regard relate to the testing of the system as it would actually be deployed in June 2004.
Testing
The principal issues identified by the Commission in relation to the testing of the system are as follows:
the software has been updated many times since the pilot elections in 2002 and since the full desk review of the source code:
- the original desk reviews of earlier versions of the software continue to be relied upon as the baseline for evaluating the ongoing changes to the system that give rise to new versions,
- there have been a large number of new versions of the software since the original desk reviews and tests took place,
- as changes are made to the system, each new software version needs to be reviewed and tested in full before it can be relied upon for use in real elections,
- it has not been possible for the Commission to review the impact of the changes made in successive versions of the software in time for inclusion in this report,
- the fact that new versions of the software continue to be issued in the run-up to the June elections is unsatisfactory,
- there is not sufficient time before the June elections for full testing of the final version of the software which would be essential before the software could be run in these elections;
it has not been possible for the Commission to obtain access to the full source code of the system: - it has therefore not been possible to carry out the preliminary review of the full source code that might have been possible within the timeframe of this report,
- a comprehensive review of the full source code of the system is necessary to establish its trustworthiness to a level compatible with the critical importance of voting at elections: such a comprehensive code review is outside the timeframe of this report,
- there is not sufficient time before the June elections to allow a full code review of the final version of the software that would actually run in these elections;
some components of the system have not been tested, in particular those at the interface between tested components;
the tests of the system carried out to date are insufficient to establish its reliability for use at elections in Ireland in June:
- there has been very limited "end-to-end" testing of the full system in its entirety as it would run in June, and none has been carried out independently: significant in this context is that the system as a whole will be required to register, combine, disaggregate, mix and count votes for up to four different polls being held at the same time,
- there has been no parallel testing of the system in a real election, either against the traditional manual system of voting or against an alternative electronic means; such parallel testing is very important for such a critical system as voting at elections: although the system was deployed on a pilot basis in 2002, these elections were not run in parallel with a paper ballot, and the software has been modified many times since then;
the system has not been tested as a whole or certified as being suitable for use in an Irish electoral context by an accredited testing and certification authority.
Accuracy
The principal issues identified by the Commission in relation to the accuracy of the proposed system largely follow from the Commission's concerns about testing:
as the software version proposed for use at the forthcoming elections is not as yet finalised, it is impossible for anyone to certify its accuracy;
the issues set out above in relation to the testing of the system make it impossible to determine its accuracy in the context of this report;
certain of the tests performed at the request of the Commission identified an error in the count software which could lead to incorrect distributions of surpluses; there is a possibility that further testing will uncover further software errors;
while eliminating the possibility of certain types of inadvertent voter error, the chosen system introduces the possibility of new types of error in the use by electors of the voting machine, particularly in the context of a number of simultaneous polls;
there is a possibility of interference with the voting machine, ballot module and hardened PC:
- in particular, experts retained by the Commission found it very easy to bypass electronic security measures and gain complete control of the "hardened" PC, overwrite the software, and thereby in theory to gain complete control over the count in a given constituency;
- the examinations carried out by the Commission's experts suggests that these "hardened" PCs are the weakest link in the security of the proposed system and it is significant that there appears to have been no systematic testing and certification of the "hardening" of the PCs notwithstanding their susceptibility to either inadvertent error or deliberate manipulation by those with access to them;
the system allows the inadvertent use of outdated versions of the software, as well as the overwriting of the software with replacement software;
attention is required to procedural issues and controls regarding the storage, handling, deployment and use of the equipment by election personnel as contained in the documentation issued to returning officers.
Furthermore, in the context of the June elections, in which each elector would be asked to use the same voting machine to vote simultaneously on a number of different matters, it is important to note that accuracy in the translocation and counting of votes critically involves the system for the aggregation of votes from many different polling machines, followed by their subsequent disaggregation, then separate mixing and counting in local and European elections, as well as the proposed referendum.
Secrecy
The principal issues identified by the Commission in relation to the secrecy of the system are as follows:
the voting machine "beeps" as preferences are being selected, and to signal voter errors; this allows limited inferences to be drawn by those outside the polling booth about the number of preferences cast: in particular a voter voting for a single candidate would be easy to identify by those in the vicinity of the machine;
there is reduced voting secrecy for persons with certain disabilities (although this is not a legal issue in the sense that, in McMahon v Attorney General the Court held that the right to secrecy is not an absolute one) as well as for persons who are unfamiliar with technology and who may need third-party assistance in using the machine;
publication of ballot results in full is a valuable aid in checking the accuracy of the results but this can in theory reveal deliberate voter "signatures" of low-preference votes which could allow voters to identify themselves in a context of corruption or intimidation;
it may be possible for an insider to overcome the randomness of the method used for the storage of votes in the ballot module.
4.3 Overall conclusion
The Commission accordingly concludes that, having regard to the issues of secrecy, accuracy and testing as set out in its terms of reference, it is unable to recommend the use of the proposed system at the local and European elections and, by extension, at the referendum due to be held on June 11th.
The Commission wishes to emphasise that its conclusion is not based on any finding that the system will not work, but on the finding that it has not been proven at this time to the satisfaction of the Commission that it will work.
In addition, the Commission recognises that the threshold of proof required to support its recommendation against the use of the proposed system is much lower than that which would be required to recommend in its favour.
It is for this reason that, although its work is incomplete, the Commission is in a position to make its recommendation within the timeframe of this report.
4.4 Other Issues
The Commission also makes the following observations in relation to the chosen system which, although not falling strictly within its terms of reference, have a bearing on the successful implementation of the system at elections in Ireland:
under the system, voters who wish to register an abstention by voting for no candidate cannot do so in secrecy;
the system does not have a voter-verifiable audit trail (VVAT), argued by many to:
- reassure voters that their vote has been correctly recorded,
- create a disincentive to the manipulation of the system by providing an external check on accuracy,
- enable recovery from a serious system failure;
the absence of a VVAT significantly raises the standards and quality of other system testing that is required;
the proposed system focuses a large number of new responsibilities on returning officers: it has been argued that an explicit and carefully specified "segregation of duties" between different election officials would increase safeguards against errors being made or improper manipulation by a single person operating parts of the system away from public scrutiny;
although it has the potential to be able to carry out calculations to a higher degree of perfection than the hand counting method, the system has been designed to replicate, but in a consistent manner, the inaccuracy inherent in the current vote counting rules as regards the transfer of surpluses.
Furthermore, one consequence of retaining the current counting rules as regards the initial mixing of the votes and for the random selection of votes on the transfer of a surplus is that if a manual recount of an election were required (as in the case of an election petition) it would not be possible to achieve the same result in a hand count as in the original electronic count, in view of the different random selections that would be made in each case.
In short, retaining the random element in surplus distribution makes it inherently more difficult to check the accuracy of the proposed system using a manual recount and this therefore has a bearing on the value of VVAT in the context of the chosen system.
This could only be avoided by a change in the electoral law to dispense with random selection in favour of a counting method such as "the Gregory rules". This, in turn, would enhance electronic voting by allowing computer systems to be used to their full capacity and would, more importantly, be more democratic in that every preference would be taken into account.
In making these observations the Commission is not advocating any particular view on the issues raised but is including them in its report in the interests of completeness.
4.5 Recommendations for action
The following additional work will be required for the Commission to be in a position to satisfy itself as to the secrecy and accuracy of the system:
there needs to be a final definitive version of the software and all related hardware and software components to be used at elections in Ireland;
there then needs to be a full independent review and testing of the source code of the final system to be used: any subsequent software modification will necessitate a further full system re-test;
there should be independent parallel testing of the system, including where possible in a live electoral context;
there should be independent end-to-end testing of the system; there should be testing and certification by a single accredited body of the suitability of each new version of the entire system for use at elections in Ireland.