Witnesses:

Professor Michael Alverez, Cal Tech/MIT Voting Project
David Jefferson, California Internet Task Force
Professor Susan Roth, Ohio State University

Senator Gorton: I think, Dr. Alvarez, we will start with you.

Dr. Michael Alvarez: Thank you for inviting me. I am Michael Alvarez, a professor of political science at the California Institute of Technology. Unfortunately, Senator Boschwitz has left because what I was going to point out was that the thing that got me excited in politics originally was being involved in some of his campaigns early in the 1980s. Now he is leaving.

Senator Gorton: You can repeat it.

Dr. Alvarez: That was what got me excited in politics. I grew up in Minnesota, and then what happened subsequently to that was I went to a college in Minnesota, Carleton College, which this Center is familiar with as well. Probably to his chagrin, I took classes from a gentleman named Paul Wellstone, and my politics have changed considerably since then.

That said, since I have been here in California, I have been involved in studying voting behavior, campaigns and elections, and public opinion. More recently, I have been involved in lots of election reform-types of efforts. In particular, in the last couple of years, I was involved in the California Internet voting task force. I was more recently involved with the NSF group on Internet voting. Their report, I think, was released just a month or two ago. And, of course, now I am on the Caltech-MIT Voting Technology Project.

What I want to talk about briefly is what our project is about, then talk a little bit about what we have found so far, and leave you with just a few parting thoughts about elections and voting technologies. The Carnegie Corporation and our two institutions currently fund our project.

What we are doing now is what we are calling phase one of the project, which is essentially a data collection, or information-gathering, part of the project. We are examining the current systems that are in place and trying to evaluate their reliability and accuracy. We are going to attempt from there to think seriously about the types of attributes that voting systems ought to have and to establish some sort of quantitative guidelines for performance and reliability of voting systems. We are going to attempt to put together some uniform guidelines and requirements for reliable voting systems. What we are trying to do right now-in the phase one that is currently ongoing-we are going to attempt to produce a report sometime towards the end of June or thereabouts. We are getting to the end of our information-gathering phase, and we are, right now, starting to write specific parts of our report.

I wanted to mention that by voting systems, or election systems-terms that have been bantered about a bit here today-we are very inclusive when we talk about voting systems. We mean registration processes, we mean polling-place voting, but we also mean systems of remote or early voting. We are trying to be as inclusive as possible.

When we conclude with this first phase of our project, we will then move to the second phase, which is what I think our presidents at MIT and Caltech were originally excited about, that is trying to develop new technologies. We felt that it was very important that we study the existing technologies and try and see what is out there and how it works before we make any effort to think about improving those technologies or developing new types of technologies. So the phase two of the project will begin sometime this summer, and on into the fall, and go on into the indefinite future.

What have we found so far? The biggest part of our project to date has been gathering data. We have used a data set from Election Data Services. We have used that to produce a preliminary report, which the Florida Task Force requested. I have provided every member of the Commission with an updated copy that was recently released. We have tried, in that report, to study the existing systems using historic data going back to 1988 and look at the different technologies and the accuracy rates using those technologies.

I would point people�s attention to Tables 2 and 3 of this report where we break down the residual vote rate, which is, in presidential elections, the difference between the number of ballots that were cast and counted for the presidential race in any particular county in our study. That is, in some ways, a very problematic measure. It is, in some ways, a very good measure. The best attribute of this measure from a statistical point of view is that it is one that we have. It is also one that we have very reliable data on, across counties and across time. I would like to point out that we are gathering many other types of measures of machine-accuracy rates, including a survey that we are conducting now of counties and their over-vote rates as well as data that we have collected from various secretaries of state and counties on spoilage and voided ballot rates. We are going to be doing as widespread a study as we can.

The problem that we found-which surprised us dramatically and surprised a number of people in the election industry-is that when we looked at the historic data, we found that the electronic machines seemed to have a relatively high residual vote rate. In other words, they have a relatively high rate of under-votes. Because in these electronic machines, as you have probably seen, you cannot produce an over-vote. That puzzled us. A number of people have been puzzled by that. We are currently trying to figure out exactly what is causing that problem. Why is it that the electronic machines seem to have residual vote rates, at least in the historic data, that look about as bad as punch card systems?

Now in Table 3, which is a new table we have recently put together, you will see that, over time, it looks as if the electronic machines are getting better. We take that as a hopeful sign that the new technologies are improving or perhaps that voters are gaining experience with these technologies. We are working very hard to try and look at these different machines and try to parcel out exactly what it is about these machines that seem to be causing these high under-vote rates. In fact, we are actually working with both Mischelle Townsend and Conny McCormack, who are really out at the forefront of implementing these systems.

We think there are some issues with the electronic systems. In this report we outline five issues. A lot of these systems have very peculiar interfaces and very peculiar interface designs, which again, if you go over and look at these machines, look very different in many ways from the traditional ballots that voters face. There may be a technology learning curve, and that technology learning curve may exist in three different fronts. It may exist in the industry where the industry is learning how to develop and implement these technologies. It probably exists also in the local election official offices, where they are learning how to use these technologies. And it also, I am sure, exists in the voting place and with the voters as they learn to use these technologies.

There are also some questions that we have-and especially my mechanical engineering colleagues have-about the reliability of these complex machines, especially as they are stored and used repeatedly over time in polling places. And the last question we have is about voter acceptability of these kinds of systems. There have been some people who have suggested that perhaps these electronic machines simply just facilitate higher under-voting rates for various reasons by voters.

Now we are again producing the phase one report, and as soon as that is available, we will certainly provide it to this Commission and anyone else who is interested. We recently had a conference. A number of people in this room were at that conference. All of the materials from that conference are available on our website, which is www.vote.caltech.edu.

Before I conclude and turn it over to Dr. Jefferson, I would like to give three different warnings about technology. I, of course, am a Southern Californian and you cannot separate me from my cell phone and my palm pilot and my laptop computer. But I am quite concerned that the pace of machine development is outstripping the development of appropriate interfaces for voters to use. There is a serious problem here, and we will hear more about it in a few minutes-about the interface between human and the machine.

The second warning that I would provide the Commission is that I also fear that with the rapid adoption and development of new technologies for elections that we are leaving some voters, and particularly some classes of voters, behind. In the last panel we heard questions about the digital divide. That is a problem. It will increasingly be a problem as more new technologies are developed and implemented for voting.

The third thing is something we heard about in the last panel as well. We do have lots of exciting new technologies that can be used, but there are a lot of important low-technology solutions that need to be put on the table and discussed. We need better polling-place training. We need better poll-worker training. We need more information in the polling places. We need to better educate voters. There is a whole series of, what I would call low-tech issues that really need to be considered before we move rapidly down a high-tech path. Thank you.

Dr. David Jefferson: Thank you, members of the Commission, for inviting me here. I am David Jefferson. Two years ago, the Secretary of State of California, Bill Jones, who testified earlier today, appointed a task force to study the subject of Internet voting. I was the technical committee chair for that task force. Our report is on-line. I would encourage you, as Secretary of State Jones did, to refer to it. Then, again last year, in 2000, the National Science Foundation, as you just heard, also had a panel to, again, study the subject of Internet voting. I was a member of that panel as well. Its report is also on-line. The two reports agree substantially, so I am going to combine my discussion of the two of them.

The panels were composed of election officials and Internet security experts, political scientists, citizen and voter advocacy groups, and election system vendors. We considered the full range of privacy, security, reliability, audibility, and operational and public policy issues regarding technologies in the Internet voting world. I think item one of what these two panels accomplished was a rational classification of Internet voting systems, which you must understand before further discussion makes a lot of sense.

Roughly speaking, you can divide systems into four levels. Level zero would be, such as represented over here with the DRE systems, these are standalone computers used as voting machines but not connected to any network or the Internet at all.

Level one would be what we would call poll-site Internet voting systems. These are computers used as voting systems at traditional precinct or other voting places. They collect the votes and also transmit the votes back to the counties when possible. Otherwise, they just act as voting machines, as DRE machines, if for some reason communication is disrupted. The key thing about them is that the election officials are in charge of configuring poll-site Internet voting machines as to the hardware, the software, the networking, and the physical environment surrounding them. The voters must come to them. Voters also identify themselves to election officials in the traditional way, rather than authenticate themselves to the machine. That is a poll-site Internet voting system.

The next level, two, is what we called kiosk voting systems, which are similar in many ways to poll-site Internet voting systems except that they do not have to be manned by election officials. The idea is that the voter brings something and authenticates him or herself to the machine, which is located in a public place like a library or a shopping center or some other place convenient to voters. It is suitable for early voting and distance voting, but it requires the solution of the additional security problem of how to authenticate a voter without the presence of election officials.

The third state-the last, most ambitious stage-is what has come to be called remote Internet voting. Here, we are talking about voting from home or office or school or some place not under the control of the election officials. The configuration of that machine and its software and its networking is under the control of either the voter who owns it or the employer, if the voter is voting from the employer�s place, or from a hotel or something like that. The ideal of any platform, anywhere voting is perhaps sometimes the way this is summarized.

Now, the two panels came up with some recommendations about these, which I will compress for you here. As far as poll-site Internet voting is concerned, there are, of course, a large number of security and privacy and reliability problems. But it was our opinion that these were understandable and manageable and that it was appropriate to proceed with experimentation leading to certification even of such systems.

For kiosk voting systems where the voter identifies or authenticates him or herself to the machine and not to election officials, there isn�t, any yet, any consensus over exactly how to do that for the general American population. So until there is consensus about how to do that, the deployment of kiosk Internet voting systems will have to wait. But progress is being made, and presumably, consensus can be achieved over time in some groups if not in the general population. That is, some groups may use this technology earlier than the general population: for example, military voting. So we could proceed with certification of such systems as that consensus develops.

With regard to remote Internet voting from private platforms, home and office computers, however, the original California committee started out coming in with bias enthusiastic and in favor of that. But we left with the opposite perspective. We found profound security problems from voting in that type of situation. I will just tick them off, and you can talk to me about them later if there are further questions.

There were Trojan horse and virus attacks, attacks through remote management software, so-called spoofing attacks where you put up a fake voting site and lure people into voting at the fake site rather than the real site, denial of service attacks, automated vote-selling schemes. These have the power to allow large numbers of votes to be changed or recorded or discarded or sold irreversibly, undetectably. Such attacks can be launched by anyone, anywhere in the world. They are not resolvable by any simple mechanisms today. No software upgrade is going to fix these. It is going to take a lot more than that. So we came away recommending against-for the time being, until these problems are solved-extending remote Internet voting to the general U.S. population. If that were not enough, there are other problems that are not security related: the problem of on-screen advertising and electioneering while you are voting, for which there is no good technical solution; the vast and constantly changing array of combinations of platforms that would have to be supported, and they change every six months; the digital divide problem mentioned earlier; there is also the problem of potential coercion in potential employment or institute voting situations, which we could not measure and could not control. So we feel that, except in special situations, we are a long way from recommending remote Internet voting. But I repeat that these long lists of problems are not problems with poll-site Internet voting or kiosk Internet voting, properly implemented.

We also made a recommendation against, again for the time being, consideration of any all-electronic, non-paper-based voter registration system. We don�t have a strong way of authenticating through the Internet alone that a person is an American citizen, over 18, living where he or she says she lives. The fear is that automated attacks to allow registration of a large number of phony voters are possible, and we just did not want that to be possible. The problem of course with automating any system, such as voting, is that you also permit automated modes of attack. Our bottom line was concerned with security and reliability, to ensure that it simply would not be possible to conduct a credible automated attack on these systems or we would not be able to live with ourselves.

Other remarks I want to make echo some things that Mike just said. All electronic modes of voting-from DRE to Internet voting systems, poll site and on up-are going to be changing rapidly. We are never again going to see a situation, I think, where you procure a system and live with it for twenty years. They will be changing every election cycle, at least software upgrades, but also hardware upgrades. These will be mandatory. They are not going to be particularly voluntary. The pace of change will be so rapid that it will put pressure on the election systems in several dimensions. One is the certification process. I think we are going to move from a certify once and then freeze the system for twenty years system to a continuous certification and decertification system. The analogy I make is to the FAA, who is continuously certifying the airworthiness, or the lack thereof, of planes and has the authority to ground individual planes or various kinds of fleets of them. I think something like that may be necessary for voting systems as well.

There is currently a serious bottleneck at the federal level, for example, in the certification process. There is a four to six month backlog today. If something is not done about this we may lose the opportunity to introduce new voting technologies for the 2002 general election, even if there is not any other problem. We also should be adding computer and network security experts to certification panels in the future for electronic voting systems.

The last degree of pressure, I think, is on the business model of voting. Historically, counties have treated the purchase of voting systems as a single capital purchase, then amortized it over twenty years, and they have a strong relationship end-to-end with a single provider. Since I don�t think the hardware dimension, and certainly not the software dimension of future systems, is going to last twenty years; I think that type of financing is going to break down, and we are going to need a more creative set of business models for this. There is a lot of discussion about this. I think this is a solvable problem, but the world is going to change as we move to electronic voting.

There are just a few other issues that I just want to mention that the world of electronic voting will bring up that really has little or no analog in the worlds of traditional voting. One has to do with open versus closed specifications for voting systems. Traditionally, voting systems have been proprietary, and the mechanisms have been largely secret or open only to election officials and to certifiers. I think when there is more and more software involved in the handling of votes from the voter to the final tally, the need to have openness, at least in the specifications, becomes paramount. Open specification is essentially the interchangeable-parts rule for software. You can go a step further and talk about public source to all of the software involved in elections. This is essentially the requirement that there shouldn�t be any secret mechanisms in the software of voting systems. Today, as I said, voting systems traditionally have been totally proprietary. I would like this Commission to look at the issue of whether that is appropriate in a software world.

The last issue that I want to mention is that there are cryptographic protocols, that are so-called universal verification protocols, that permit any citizen, once these protocols have been implemented and are in place, to verify virtually end to end that the entire election has been conducted fairly and equitably and properly-i.e. without votes being changed, phony votes being inserted, votes being removed or anything like that. It has, in effect, the power to eliminate most modes of insider fraud. In case anybody loses confidence in elections, because of necessity, electronic elections are a little less transparent in their mechanics to the public than paper ballots, which people can see. This is a type of technology that I think we should strongly consider. Thank you.

Senator Gorton: Ms. Roth.

Ms. Susan Roth: I would like to thank the Commission for inviting me to participate. I am very pleased to see a truly bipartisan effort to reform and improve the election process in the U.S. I am an associate professor and chairperson of the Department of Industrial, Interior, and Visual Communication Design at the Ohio State University. I am also associated faculty with the John Glenn Institute for Public Service and Public Policy.

Since 1993, I have been conducting research on the usability and design of voting systems with a focus on information design and access to the voting process. Initially, I was interested in providing greater access to older voters and to voters with different education levels. I have examined the interaction between the voter and voting systems in the context of the election situation. This provides a glimpse into the voting booth and identifies some of the problems that voters encounter as well as how improved design can solve some of these problems and possibly reduce disenfranchisement.

Research conducted through statistical analyses of election results, such as that done through the Caltech/MIT project, can identify broad patterns and average percentages, for such factors as under-voting and over-voting, but doesn�t necessarily identify the root causes-the symptoms but not the cause. Small-scale studies that focus on voter activities and the perceptions of the voters can help to explain why some of these occur. To understand the context for the use of voting systems, I examined the technical and structural aspects of systems and election administration at the polls. I became trained on new equipment as an election officer. I interviewed vendors and election administrators about some of the problems that voters encounter, as well as how design standards could be improved to solve these problems.

Simulated elections were then set up with the cooperation of county officials in Ohio so that existing equipment and official ballots could be used for the simulated election. A diverse group of subjects of all ages and educational levels and all ethnic backgrounds was assembled, and they were videotaped, interviewed, and engaged in group discussions following the activities so that we could learn more about their opinions of the systems and their experiences in the various voting booths that they had tested.

Their feedback was very helpful in identifying areas that needed to be improved. For example, type size on the ballot was too small or displayed at a height that made it difficult for them to read. This was especially true for the older voters who have reduced visual acuity that comes with natural aging. Ballot format was ambiguous in one case so that subjects were not sure which electronic button was associated with their choice of presidential candidate. This was from the 1992 presidential race. And I do want to add that this was something that applies not only to the older systems, but also to the new systems such as the DREs that we have on display.

Ballot language in the area of ballot issues was confusing and written in such a way that, in the words of one subject, "it contained a lot of negatives so a vote for or against became less clear." They were not sure how to vote for the issue. The ballot language is quite complex, especially for users with lower-education levels and users for whom English is not their first language.

Subjects disliked punch cards more than any other system tested. Aside from obvious tabulation problems that were highlighted in the media following the Florida election, these people had difficulty reading the small numbers printed on the card in order to punch the correct hole. This was in a provisional voting process. The ballots were displayed on the DRE or in a ballot book, and the punch cards were actually used to record the votes.

To address the issue from an earlier study that voters were not sure they had selected the candidate intended, I decided to experiment with this study. I gave them a list of candidates to vote for. Analysis of the punch card showed that many of them had voted other positions-including punching the hole below the number rather than the number above it-as they had been instructed to do. If you vote absentee on punch cards, you will see there are instructions printed on there that you should punch the correct hole. At the polling place, you won�t always find instructions, and you can�t always rely upon oral instructions being given to each voter to eliminate that kind of voter. In fact, there is probably no way to track that kind of error and see if there have been those mistakes made because we vote in secret.

Many of these problems go undetected just for that reason because we have protected the right to secrecy, and there is often no immediate and specific feedback or verification of votes as cast given to the voter. This is intended to prevent vote fraud, coercion, and intimidation, but it also means that many problems go undetected and votes may be lost-or worse, cast for the unintended candidate. And such problems-as well as those at a later stage during the tabulation of punch cards in central counting of optical scan or mark-sensed ballots, and other cases-mean that we don�t know how many votes are lost.

The media coverage of the Florida recount brought many of these problems to light, but they are not new. Election administrators and candidates involved in contested elections, vendors, and local media that have covered such stories have heard about these problems before. In fact, the Columbia Journalism Review did an article outlining some of these stories that have been printed over the years on just these topics and asked the question: "Why hasn�t something been done?" The public may not have realized the scope of these problems, but this year the incredible coverage that came through print and electronic media and broadcast media has raised the general level of awareness to the level that something really needs to be done. I hope the Commission is the first step in this process.

The FEC has requested a budget increase this year to revise the voting systems� standards document. If you have seen that, from 1990, it is quite a substantial document. There is quite a lot of specific information on technical standards. It needs to be updated badly. It is quite out-of-date, and they acknowledge this. They would like to improve that situation. It also needs to include such issues as ballot design guidelines. There is a way to provide examples of good and bad ballot design in terms of effectiveness and clarity of communication and so on. This could be produced as part of the standards. The standards could also be put on-line. Ballot guidelines could be on-line so that you would have very wide distribution down to the local level where a lot of these activities are being implemented. If there is no other concrete result of the many initiatives related to election reform, including this one, I believe this will be a major step toward improvement. Either the FEC or some other body that can have supervisory, but not control, responsibilities for what happens at the local level is what we need. I think the coordination would be very welcome.

However, vendors must be requested to meet the standards of performance, including usability and accuracy. The systems might be rated on their ability to minimize disenfranchisement. This was not done in the first standards or any other document. Such ratings should be widely and publicly distributed. To do less would be to work long and hard on developing standards and updating them and then see them applied to existing and new systems at the discretion of voting system vendors. New technologies may solve many problems with current systems, but unless they are tested with a representative section of the diverse voting population, you won�t find the problems in order to fix them before the systems are marketed.

I look forward to the exciting potential for new systems developed by groups around the USA, various consortia, including the Caltech-MIT initiative. However, technology alone does not solve the problems; we have to go back to the experience of the voter in the voting booth. Is the ballot communicating correctly? Is the information clear? Is the action that the voter needs to take not ambiguous? Do they have some sort of assurance that the way they are voting, their vote will be cast and their vote will be counted? All of these are low-tech issues that must be applied to the new voting systems. I look forward to hopefully working with all of you on improving the situation. Thanks.

Senator Gorton: Dr. Alvarez, let me start if I may. I am looking at Table 2, but any of the later tables would be as valid. Residual vote is a synonym of under-vote?

Dr. Alvarez: It is under-vote plus over-vote. It is both. It is the difference between, in the presidential race, the number of votes that were cast and the number of votes that were counted. So it is both under- and over-votes.

Senator Gorton: Are these figures, the residual vote figures, after the ballots have gone through whatever machine counting takes place or after both the machine counting and the human analysis of questionable ballots?

Dr. Alvarez: These, in general, are all before recounts. I don�t think these include recount data in any case that I know of. I am not entirely sure about that.

Senator Gorton: I think that is significant. I hope you can provide us with an answer.

Dr. Alvarez: I will have to consult with Stephen Solderberger at MIT, but my understanding is that the recount information that we have is going to be a separate analysis.

Senator Gorton: So, the actual number of votes counted-the figures of residual vote-after we have gone through a recount, or not so much a recount but a first count of ballots that did not register on the machine, the figures will all be lower than these numbers?

Dr. Alvarez: I would not speculate on that. I don�t really know.

Senator Gorton: I think that it would be valuable in connection with these alternative technologies to tell us how many ballots had to go through some kind of subjective human count or human determination with the smaller number obviously being a superior number.

Dr. Alvarez: That is an excellent suggestion, and we will go back and put our heads together and figure out how to do that.

Mr. Edley: If I could just interject, because I remember that the Civil Rights Commission took a swing at this issue, also probably using some of the same data. I thought this was based on the number certified, and what is certified can vary from jurisdiction to jurisdiction as to whether or not it includes the human inspection as well as the first-cut machine inspection. So there may not be a consistent answer to that.

Senator Gorton: I was surprised at Dr. Alvarez�s initial answer to me. I do think one of the goals that we have to have is to reduce the number of ballots that have to go through that subjective, human determination. It would be nice to know which system is the most accurate before you even get to that.

Dr. Alvarez: I think that is an excellent suggestion.

Mr. Edley: The other thing that I think we found out is that there are some places-this seemed totally weird to me-that there are some places that don�t actually count the number of folks who show up and sign in at the polling place. So you don�t actually have a comparison between the number of votes counted on the one hand and the number of people who showed up on Election Day. There are a few jurisdictions that do not give you that number.

Representative Panetta: If I could-I am going to have to leave-you just scared the hell out of me. I don�t think I will ever use the Internet again. If we can�t develop a secure system for voting, I don�t know what else is happening out there in terms of everything else we do on the Internet. But having said that, you mention something that I also thought was very interesting, which is that the technology is going to change so quickly in terms of this whole area. How can a community afford to invest in equipment at the present time knowing full well that the technology is going to change within a short period of time and that they will have to re-invest or spend additional expenditures within a short period of time?

Dr. Jefferson: First of all, I intended to scare you as far as Internet voting from home and office platforms is concerned because that is a real problem for which there are no good short-term solutions. So we just recommended that. As far as the technology changing rapidly, I believe that it is up to voting system vendors to find a way to make this technology affordable to jurisdictions. That means they are going to have to come up with new ways of financing it. They are going to have to pay for the certification processes. They are going to have to find ways to make the hardware, for example that is used in an election, to be transferable into the county for use for other purposes between elections perhaps. The business model for elections, in other words, is going to have to change. If vendors cannot find a way of making this affordable, then it is not going to happen.

Senator Gorton: Dr. Jefferson, let�s get really practical. Do you feel that Riverside County and Ms. Townsend acted improvidently?

Dr. Jefferson: No I don�t. In fact, I congratulate them on having the courage to attempt something that few other counties had at the time-meaning wholesale replacement rather than experimentation with electronic voting. But, that said, there are limitations to the system that they have, which I think will become apparent in a few years. Upgrade paths, for example, are limited. The equipment that they are using is special purpose-it is not based on the architecture of standard PCs-so they are not going to follow the same price curve that PCs have followed. Their systems are proprietary, so they are locked into a vendor. I think these issues, over time, will become real. But no, I would not second-guess Riverside County.

Mr. Seigenthaler: Could I just ask about the issue of certification. How do you envision it would work? We have heard that the present system relies only upon a signature.

Dr. Jefferson: The certification code is, of machines, that they meet the election code of the state and the federal government. Authentication is that you are a legitimate voter.

Mr. Seigenthaler: How does the certification of machines work? Is it all machines?

Dr. Jefferson: Well, traditionally, each state has its own rules. Most states require that voting systems-and it is whole systems, not individual machines-are certified by the Federal Election Commission as meeting their voluntary standards. States are not required to adopt the federal standards, but most do. In addition, many states impose their own requirements on top of the federal standards. These standards require tests of accuracy, tests of robustness-like if they fall on the floor, they should still work. In other words, there is a whole range of requirements that they are tested against.

I am arguing that those requirements are going to have to change rapidly as more of the election system becomes software and less of it becomes hardware. Certifying that software does what it is supposed to do is completely a different task and requires completely different expertise, for example, than certifying that hardware does what it purports to do. This is especially true when you are talking about security and cryptographic protocols. A different kind of expertise is called for, so I think a lot will have to change in the certification.

Mr. Seigenthaler: Could I just pursue the question of authentication for a moment? What is your vision that how that ultimately works? Must there be a code that can be verified at whatever the nerve center is?

Dr. Jefferson: What you want to make sure of is that the voter is who he or she says she is. If you are talking about not changing the voter registration system, then you somehow need to compare a signature of the voter who presents him or herself for voting against the signature on file at registration time. If you were talking about a newer type of voter registration system, perhaps not based on electronic signatures but based on digital signatures or something, it would be a different thing. The bedrock of the authentication problem is how do you authenticate voters against the voter registration system? We would have to talk about what voter registration system you envision before I could talk about authentication.

Mr. Seigenthaler: Let me just follow up for a moment, Mr. Chairman, if I might. If I, as a layperson, had never experienced the process of validating ballots cast in an election and had not heard it from so many officials, now, I would have serious doubts about having the faith they seem to have in a written signature. Just take that as a standard for a moment. If you were where they are-and maybe we are talking about apples and oranges here-would that be acceptable to you? And let me just ask beyond that, have the technology experts set the standard-set the bar so high-for the authentication of voters that it is ultimately going to be, even when it begins to move, the question that is going to be on everybody�s minds?

Dr. Jefferson: Well, let me talk about that-have we set the bar too high? I think that is an important question that I am asked frequently. Neither panel that I am representing here was terribly concerned to try and prevent every last vote error or every last one or two vote fraud. Our concern was that once you start automating election systems, automated means of vote fraud begins to appear. We wanted to absolutely close the door on those. So where we set the bar-how much security is enough-was there has to be enough to prevent any type of automated attack on these election systems. But we would be satisfied-the members of the panels, I am talking about-if the same level of fraud and error as are present in current systems were to remain later, not that anybody is for that, but we did not want to set the bar too high.

[Unidentified member of commission]: What we have just been through, I don�t feel to comfortable about that either.

Dr. Jefferson: I would support strengthening the standards that are in use today, certainly.

Dr. Alvarez: Being on these panels, just to elaborate, to put it in a slightly different way, when Dr. Jefferson is talking about automation, the other way to think about it is the scale at which fraud can exist and can be perpetrated on electronic system is vastly greater than it is in current precinct-based voting. Here in California, it would be relatively easy for me to walk into a precinct and impersonate another voter. I might be able to do that twice in the same precinct. But it is very unlikely that I am going to be able to do that more than a couple of times. But if we move to these kinds of electronic systems, the scale at which you can automate these kinds of fraud and attacks on the system increases exponentially.

Now, since I have the microphone, I wanted to say two other things. One was that Connie McCormack reminded me, and it is very important for me to point out, that both the systems that were in place in Riverside and in Los Angeles County, using touch screens, have dramatically lower error rates than we report on average in the research that we have done because they are using some of the newer and better systems. So I wanted to make sure our friends in Southern California were not being criticized.

The second is to reiterate something that Dr. Jefferson said. I personally feel that the current business model for election technologies and election systems is simply wrong. I am not going to criticize election officials for their purchase of equipment, but I think that in the future the business model is going to disassociate the hardware from the software. I hope that we are going to get into a situation where firms are going to start marketing software systems, or leasing software systems, and not hardware systems. So you are going to be in a situation where county election officials will be more in the business of acquiring and updating software and not in the business of acquiring, maintaining, and updating hardware. They are going to lease the hardware or obtain the hardware from some other provider. It might make their life more difficult, but I think that is a much more feasible business model.

Ms. McAndrews: We talked about that at lunch today in terms of the need to transfer the software from hardware to hardware as it becomes obsolete and having transferability. I am not a techie, so I don�t even begin to understand that objective. If you could comment on that and how soon in the future that might be.

And then secondly, you are very persuasive about how afraid we should be about this kind of automated attack and disruption of an election, but we had heard at an earlier hearing that I guess the military did some Internet voting in the last election.

Dr. Alvarez: We are under a non-disclosure agreement. We can�t talk about that.

Ms. McAndrews: I am curious if you could explain, did they have some kind of secret encryption or why are they relying upon it?

Dr. Jefferson: I will explain it. The Federal Voting Assistance Program for which Dr. Alvarez and I were both on the panel that evaluated it, so we do know. They had an experiment in which 85 votes nationwide, a total of 85 votes in four states, were cast and counted in the last presidential election.

Ms. McAndrews: Via the Internet?

Dr. Jefferson: Yes, sorry, they were remote voting systems of the kind that I have warned against, so one of my concerns is that the Federal Voting Assistance Program re-evaluate on that basis should they try and scale this up to 850,000 instead of 85 votes. I also had concerns because they are somewhat closed-mouth about the system. We did have to sign non-disclosure agreements in order to serve on this panel. I at least believe that voting systems have to be open and public. We should not be under that type of requirement. That said though, for a first shot at a project, at a very important problem, there are many barriers to overseas military voting, not the least of which that at any one location you have military personnel from 500 hundred different jurisdictions with 3,000 different ballot types all trying to vote in one location. That is an enormous problem; they have a huge, enormous problem. For a first shot at tackling the problem, it was time to try it. I assume they will learn from this experience and try again on a larger scale and with somewhat different architecture.

Senator Gorton: But there was no value in trying to screw up the system with 85 votes. There would be with 850,000.

Dr. Jefferson: I agree with you completely.

Ms. McAndrews: And were they using something that is not available to vendors or isn�t available?

Dr. Jefferson: They commissioned the building of a special system for this experiment. They hired a developer. In a sense they contracted.

Ms. McAndrews: Are they using bells and whistles that would not be available to the private sector?

Dr. Jefferson: No. There was no brand-new proprietary or military technology that they used, except they did base it on the military�s identification of individuals, which does not extend to the general population. But the military has ways of strongly identifying their own personnel or dependents. They used that, and it made a whole lot of sense.

Ms. McAndrews: Do you have any idea what the cost to vote was?

Dr. Alvarez: About $1,000 a vote. Yes, something like that.

Dr. Jefferson: We have to remember that there is a difference between development cost and at-scale cost.

Dr. Alvarez: A lot of that was a fixed development cost, but what the Federal Voting Assistance Program did in this particular experiment was two different things. One was focus on their traditional role of facilitating voter registration. And that part of the system, Dr. Jefferson, and I can speak for him, and a number of us thought that it was a very interesting prototype for thinking about moving registration on-line. The problem, of course, is that it is built on this foundation of the Defense Department�s public key infrastructure and how they identify Defense Department people on-line. Moving that to the general population involves a lot of problems.

The second part then was involving local election officials in, essentially, the absentee voting process online. That is the part that I think that we all got very scared about in terms of the security risks, especially in the case of scaling it up.

Dr. Philip Zelikow: Dr. Alvarez, you said that you were collecting data on voting systems including voter registration. There are a number of critical assertions that have been made that involve testable hypotheses in which there is data about the accuracy of voter lists and especially alternative ways of verifying the accuracy of voter lists. Let me mention three examples or three different hypotheses of ways to improve the accuracy of voter lists. One is the use of signature verification, which we were told was highly effective in the Oregon case, for instance, and in verifying the identity of voters who voted absentee. Second is cross-referencing voter registration lists against other lists maintained by government agencies. Within this category there are a number of different ways to configure this: DMV lists, Department of Corrections lists, postal service lists, and INS databases as well to verify citizenship. A third possibility that has been proposed is to try to improve the accuracy of voter lists by putting voter lists out on the Internet. Since they are public records, you put them out on the Internet with limited information, such as simply name and address in, the expectation that the free world in due course, will help you police that information. There are some privacy concerns that were registered. To which the response was made that the material was already a public record anyway. My question to you is, has anybody gathered any meaningful data or studied any of these basic assertions or reached any even tentative judgments about the value of different methods of maintaining or improving the accuracy of voter lists?

Dr. Alvarez: At our conference a few weeks in Pasadena, we had a number of election officials speak to these kinds of issues. It is hard from a quantitative social science point of view to try and study accuracy of voter registration lists. However, a number of election officials, including Conny McCormack, have done some of these very things, such as attempts to cross-verify lists. In fact in Los Angeles County, I should let her speak to this, but her job in Los Angeles County involves maintaining many of these very databases you are talking about. However, trying to integrate those databases that are running on different platforms, using different kinds of software, turns out to be very difficult. What we are doing is essentially talking to people in the field who are using many of these different kinds of means to update their databases and keep them as accurate as possible and to try and obtain more of a qualitative assessment of how some of these methods work.

Dr. Zelikow: Since you were at this conference which you helped sponsor or for at which you were at least one of the rapporteurs, did you hear anything at this conference that would be either promising or disturbing news that this Commission might benefit from hearing?

Dr. Alvarez: The promise, I think, is something where I believe that there was a representative from Compaq computer who engaged in a nice debate with Conny McCormack over how you go about using different types of software to try and integrate many of the databases that the government currently has. I think there are ways in which technology can do that. I think those will be important solutions to some of the problems with voter registration that we currently see.

There are important privacy problems, I think, with your one suggestion about putting voter registration databases openly on-line. There are some states that allow searching of registration databases on the Internet. Here in California, I don�t think that is legal. However, individuals can obtain the voter registration database for certain purposes, including academic research and political use. I do think there are ways that we can think about technology helping the registration process. In particular helping local election officials with their database comparisons and maintaining accurate databases.

Dr. Jefferson: We also made a suggestion in one of the panels that not original voter registration but perhaps changes to your voter registration, changes of addresses, might be securely completed over the Internet-but not original registrations.

Mr. Edley: Can I ask about the evolution in technology issue and the various propositions related to competition, innovation, standards, and decentralization of decisions and marketing. If any of you could give us your thoughts on to what extent that centralized-that is to say federal-performance standards would interfere with the evolution of technology or help the evolution of technology? To what extent do you think that highly decentralized decision-making is essential to the evolution of the technology?

Dr. Jefferson: I do believe that the federal government, the FEC in particular, makes a major contribution by creating careful voting standards-which are voluntary-but which are nonetheless extremely valuable to most of the states, and they just adopt them. I think that this is important and should be continued. To the extent that you want to beyond that and to try to influence states to adopt the standards more strongly than just voluntarily, say, be subsidizing those states that do, that is a different policy issue that I guess I don�t want to comment on.

There is a virtue to having a wide variety of different systems out in the world, a number of virtues, including economic competition. You don�t want a situation where there are only two or three vendors in the field-as, frankly, there are nationally today-with any market share. You would like to increase the number of systems. You would like public standards to be in place so that the systems could inter-operate. For example, if you buy regular voting machines from one vendor but that vendor doesn�t have a handicapped voting machine, you would like to be able to go to a different vendor who can read the same specifications and sell you the special handicapped voting machines that will inter-operate with it. You would like, for example, if a vendor goes bankrupt, to be able to buy a backend system or an extension to that backend system from another vendor that has access to the same standards. Standardization processes, I think, are becoming more important. I also personally feel that it is just not a good idea for a county to be in a position where it is complete beholden to the one vendor it made a business deal with three administrations and ten years ago. That is kind of the situation that we are in today. So, having open standards and multiple vendors implementing to those same standards, to the extent that they inter-operate, I think is a good idea and good public policy.

Mr. Edley: Can I just comment on something? This business model issue, I think, is obviously a very, very sound point. But I think that it is important to keep in mind that there are two sides of this. Part of it is perhaps that we want competition so that vendors, over time, will make these technologies affordable. But on the other hand, from the buyer�s point of view, we shouldn�t expect that we are going to get democracy on the cheap, either. Maybe the purchaser ought to be willing to put more money on the table at the same time that we are expecting the price curve to help us out. The other point that I want to make is about this danger of obsolescence and the like. We didn�t not move to automobiles and stick with the horse and buggy because we were afraid that automobile models were going to be changing over time. So there is a component in there about doing a correct appraisal of what additional benefits would we get by making the change in technology. Are we going to get more participation? Are we going to get more accuracy? Are we going to get the uses of multiple languages?

Dr. Jefferson: Many of the changes will be forced on you. Dial-up Internet access is going to disappear soon. People will have wired network connections. Wired network connections will disappear after that and people will go to wireless connections for low bandwidth applications like voting. So if you don�t have any precinct locations that are suitable for the old standard, you are going to be in trouble anyway.

It will be the same with software standards. New security protocols appear. Your old security protocols that are built into the machines are no longer considered reliable or have worse-known holes in them. You are forced to upgrade. You don�t have a lot of choice. But I do take it as positive. We shouldn�t avoid this technology because we have to upgrade them regularly any more than other modes of business avoid investing in new technology because they know they must replace it, or large parts of it, every couple of years. I think this is a fact of life about high technology. The world is going to change, and elections should be more conservative and will follow it, in time, with a lag. But I don�t think they will escape it.

Dr. Robert Pastor: I have two questions for Dr. Alvarez. I think your study-the study that MIT and Caltech is doing-is very, very important because you are the only one who is trying to go back and put a full database together. It seems to me that-in addition to the question whether one technology may be better or less bad than the other, in terms of under-votes and over-votes-I am wondering if you have enough data to distinguish between the ages of the machines. Because if you look at the discrepancies in the last election, particularly on punch card voting from one county to the other-and we saw this very clearly in Georgia but also in parts of Florida-it was not so much the discrepancy between machines but between the different ages of the machines.

The second question has to do with a closer analysis of the under-vote. It is clear that your database does not permit that, but we do have a new database from the state of Florida that, for the first time, can help us to really understand a lot more about what really went wrong. Of the 42,000 under-votes in the state of Florida, roughly half of those had some mark on them. The machine was not picking up that mark. I wonder if you wouldn�t mind going back and looking at the different technologies and answer the question as to which of those technologies might be adapted in some ways to permit a lower-error rate. This is a question that President Carter asked last time at one of the sessions, which was whether or not you could come up with a standard that would, in effect, reduce the under-votes better for one technology versus the other and whether one might be better for that.

Dr. Alvarez: My analogy of this project is that it sort of reminds me of an onion: every time I peel a layer, there are ten layers that grow inside of it. Because every time that we think that we have put together some data and are producing some analyses that are useful, people come up with all of these wonderful extra suggestions for us. On both of these points we have some information. In particular, from Colorado, we have been able to pull together a database that gives us the age of the machines in every county in Colorado for a couple of years. So we will be looking at those data quite closely. We don�t have those data nationally, but we are looking at it in some certain areas where we get it. I fully expect that your hypothesis about the impact of the age of the technology is probably correct.

Dr. Pastor: I think the implication may be a very powerful one. If the Commission, for example, is unable-like your study seems to imply-to decide on one technology over another, it may very well be that the principal recommendation may be that you have to replace your machines within a certain period of time, whatever machine it might be. That will reduce the error rate much more than picking one technology versus the other. But that will follow from your analysis.

b>

Dr. Alvarez: Exactly. That is exactly the sort of suggestion that we will be in a position to make after that kind of analysis, but I would warn you that we are not going to be in the business of recommending one technology over another. Because I do think-and this goes back to some earlier questioning-in my opinion, heterogeneity is a great thing. We need to allow county election officials to make these types of decisions because by allowing all these different technologies and different systems to operate in different places, we one produce lots of quasi-experimental data; I am a social scientist and a technologist. But it allows for innovation. We really do need for local election officials to have these kinds of technologies and to work with them and to innovate with them. One of the things that you can do with these DRE machines is county election officials can reprogram these interfaces once they have these machines. So, hopefully we are going to see more of these kinds of attempts in the future as local election officials obtain these technologies and try to develop them more suitably for their own locations.

On under-votes, there is lots of neat stuff we can do with the Florida data. I am really excited that New York is going to release their analysis. Additionally, there is one other interesting state, which is Nevada, which allows people to actually cast in federal elections a vote for none of the above. Of course, one of the problems with under-votes is that you don�t know whether someone is intentionally under-voting or not. So the Nevada data will, I think, help us try and answer the intentional versus the unintentional vote question.

Dr. Pastor: Puerto Rico has long done that; by the way, that data is all available.

Dr. Alvarez: There you go. The onion just expanded again.

Ms. Roth: Could I add something to that also? In terms of your error rate question, both under-votes and over-votes can also be caught at the polling place if the tabulation is done in the precinct as opposed to centrally tabulated. So there are even some low-tech solutions to existing systems that can help reduce under-votes and over-votes.

Senator Gorton: I want to thank you for a most interesting presentation from the three of you here today. It has been a great help to us. The members of Commission will need to meet briefly-or not so briefly-as soon as we are finished here. This hearing is adjourned. We will next meet in Austin, Texas on May 24^th.