Our research builds on two consecutive years of surveys sent to LEOs in 2018 and 2019. We used slightly different sampling procedures between the two years to try and control for the impact of jurisdiction size as measured by total number of registered voters.
The 2018 and 2019 LEO surveys were significant learning opportunities to develop best practices for sampling, surveying, and producing accurate statistical estimates across the “complex quilt” (Brown, Hale, and King 2020) of American elections and election administration. This page and other working papers (listed below) describe these best practices in more detail.
The 2018 Local Election Official Survey utilized a sampling frame built off of a comprehensive list of all local election officials in the country obtained from the US Vote Foundation. This was matched with registered voter totals from 2018 Election Administration and Voting Survey (EAVS) administered by the US Election Assistance Commission. We also used local contacts and websites when necessary for sub-county data. We drew a sample of 3,000 jurisdictions, using sampling proportional to the registered voter population – that is, a given small jurisdiction is less likely to be sampled than a given large jurisdiction.
For the 2019 Survey of Local Election Officials we developed a new sampling frame by building off of the 2018 EAVS and scraping data from Secretary of State websites or similar sites, state-by-state. We made a series of edits to this list of jurisdictions to create proper entries for each local jurisdiction that included a local administrator responsible for election administration. This resulted in a sampling frame with 7,834 local jurisdictions. We drew a sample of n = 3,000 from this list using the random systematic sampling method, with inclusion probabilities proportional to number of registered voters in each jurisdiction. This ensured that all of the largest jurisdictions (> 15,000 registered voters) were included in the sample, and we collected a representative sample of jurisdictions of smaller sizes.
Our sampling method was determined with two goals in mind. First, we wanted our sample to be representative of local election officials. Second, we wanted our sample to be nationally representative of service provision to voters, or put another way, we wanted to assure that we have sufficient coverage of local election officials serving a large and diverse American electorate. As pointed out by previous researchers, “[l]ess than 6% of the local election officials in the United States serve more than two-thirds of the voters in a national election” (Kimball and Baybeck 2013). Therefore, following past practice, we have sampled jurisdictions proportional to the number of registered voters they serve. In practice, what this means is that 100% of the larger jurisdictions (> 15,000 registered voters) fell into our sample.
A key challenge in sampling LEOs is the diversity of jurisdictions, and that most of the LEOs in the United States are concentrated in smaller jurisdictions when measured by number of registered voters. This is connected to state-specific election administration systems; several states in the Midwest and New England administer elections at the municipality or township level, as opposed to the county-level administration elsewhere. For example, the top five states in terms of number of LEOs represent 57% of the sampling frame. However, these five states combined only represent 8% of registered voters in our sampling frame, and for all but one of these states, the median number of registered voters in the jurisdiction is less than 1,800. A graphical representation of what we call the “75:8 problem” is shown below.
Our sample resulted in fewer of the very small jurisdictions than we would have in a purely random sample. This was done purposefully, but using “unequal probability sampling” means that we have to generate survey weights and weight our statistical estimates if we want to make inferences back to the overall LEO population.