NPI, Change Research discuss how to read and assess this year’s Seattle electoral polling

This past week, our team at the North­west Pro­gres­sive Insti­tute released the ini­tial results of our sec­ond 2021 sur­vey of the Seat­tle elec­torate, which was con­duct­ed for us by Change Research of Cal­i­for­nia. We found Bruce Har­rell six­teen points of Lore­na González for May­or, Ann Davi­son nine­teen points ahead of Nicole Thomas-Kennedy for City Attor­ney, Tere­sa Mosque­da eight points ahead of Ken­neth Wil­son for Coun­cil #8, and Sara Nel­son four points ahead of Nikki­ta Oliv­er for Coun­cil #9, with sig­nif­i­cant num­bers of vot­ers still unde­cid­ed. We also found huge majori­ties of vot­ers unde­cid­ed in the school board races.

As of today, one or more of our find­ings have been men­tioned dur­ing the evening news­casts of all four of Seat­tle’s local tele­vi­sion sta­tions and have also received exten­sive radio and online cov­er­age. The Seat­tle Times’ Daniel Beek­man wrote an excel­lent sto­ry sum­ma­riz­ing the find­ings that appeared in print and on Last­ly, our find­ings have received quite a bit of atten­tion and dis­cus­sion on social plat­forms like Face­book, Red­dit, and Twitter.

We’re heart­ened that so many peo­ple are inter­est­ed in our research.

We know that try­ing to fig­ure out how to read and assess poll data can be chal­leng­ing. Our sur­vey was one of only a few inde­pen­dent sur­veys to be con­duct­ed in Seat­tle dur­ing this gen­er­al elec­tion, and the only one to have field­ed this month (the oth­ers field­ed in Sep­tem­ber). Since oth­er recent polling in Seat­tle con­duct­ed for inter­est­ed par­ties has not been pub­licly released, our data is basi­cal­ly stand­ing alone on its own instead of being one sur­vey among many.

When you’ve got only one poll to look at in a giv­en time peri­od, you can­not make com­par­isons with oth­er polls to ascer­tain trends and com­mon­al­i­ties. All you can do is judge whether the one poll in front of you is cred­i­ble or not.

Today, to address some of the ques­tions and com­ments we’ve got­ten about our poll since Tues­day, we are delight­ed to wel­come Ben Green­field to NPI’s Cas­ca­dia Advo­cate. Ben is the Senior Sur­vey Data Ana­lyst at Change Research respon­si­ble for the field­ing of our sur­veys along with his col­league Ben Sullivan.

We hope you enjoy this Q&A and find it help­ful for putting our results in context.

Andrew Vil­leneuve, NPI: Ben, thanks for join­ing me to dis­cuss our work togeth­er this year! We’re delight­ed to have had the oppor­tu­ni­ty to launch our research polling part­ner­ship with Change Research in Seat­tle. It’s been a fas­ci­nat­ing elec­tion cycle and we’re not even to the end of it yet!

Ben Green­field, Change Research: Work­ing with you and the North­west Pro­gres­sive Insti­tute team has been a great plea­sure for us — and I can’t dis­agree with you on it being a fas­ci­nat­ing elec­tion cycle!

Andrew Vil­leneuve, NPI: Our Seat­tle polling this year con­sist­ed of two sur­veys: one in July that pre­ced­ed the Top Two elec­tion and one this month pre­ced­ing the Novem­ber gen­er­al elec­tion. We don’t know what the results of the gen­er­al elec­tion will be, but we know that our first sur­vey was able to antic­i­pate a lot of the dynam­ics we saw in the Top Two elec­tion, with sev­en of the eight can­di­dates who advanced to the runoff round hav­ing placed first or sec­ond in our polling.

Ben Green­field, Change Research: Indeed. While it’s always impor­tant to remem­ber that polling pro­vides a snap­shot of where vot­ers are at a giv­en moment before they cast their bal­lots — and not a pre­dic­tion of where they’ll go — we were pleased to see that the poll we con­duct­ed togeth­er accu­rate­ly cap­tured some of the key dynam­ics and vot­er pref­er­ences in the Top Two election.

Andrew Vil­leneuve, NPI: After we released our gen­er­al elec­tion poll find­ings this week, we start­ed get­ting inquiries and com­ments about our sur­vey’s method­ol­o­gy and sam­pling. One of those ques­tions per­tained to the diver­si­ty of the sam­ple: 82% of the sur­vey tak­ers iden­ti­fied as white, which is a high­er per­cent­age than in oth­er sur­veys of the Seat­tle elec­torate this year, like the Strate­gies 360/KOMO poll from last month. But dif­fer­ent sur­veys are mod­eled on dif­fer­ent uni­vers­es. We chose to poll like­ly vot­ers instead of reg­is­tered vot­ers, and con­se­quent­ly, our sam­ple is mod­eled on the last sim­i­lar elec­tion, which occurred in Novem­ber 2017. How is polling like­ly vot­ers dif­fer­ent from polling reg­is­tered vot­ers or even sur­vey­ing the pop­u­la­tion of a city like Seat­tle as a whole, and what ram­i­fi­ca­tions did our choice have for the sur­vey’s ethnic/racial composition?

Ben Green­field, Change Research: Poll­sters have dif­fer­ent meth­ods for pro­ject­ing who’s like­ly to turn out in a giv­en elec­tion and who’s less like­ly, and that can dif­fer from elec­tion to elec­tion. Our turnout mod­el took into account both past vot­ing pat­terns in Seat­tle munic­i­pal elec­tions and sur­vey respon­dents’ self-stat­ed like­li­hood of vot­ing. Both turnout his­to­ry and self-stat­ed like­li­hood of vot­ing are imper­fect pre­dic­tors of turnout, but both have some rela­tion­ship to actu­al turnout, and we believe our com­bi­na­tion of these fac­tors leaves us with a view of the elec­torate that approx­i­mates what we’ll actu­al­ly see.

As far as the ethnic/racial com­po­si­tion of the sur­vey, our pro­jec­tions are again based on each group’s his­tor­i­cal turnout rate. In munic­i­pal elec­tions, turnout has his­tor­i­cal­ly been high­er among white vot­ers than vot­ers of col­or, and as a con­se­quence of that, the his­tor­i­cal elec­torates, and our sur­vey, have been whiter than the entire pop­u­la­tion of Seattle.

Andrew Vil­leneuve, NPI: In our sur­vey, 7% of respon­dents iden­ti­fied as Asian or Pacif­ic Islander, 5% of respon­dents iden­ti­fied as His­pan­ic or Latino/a, 3% of respon­dents iden­ti­fied as Black or African Amer­i­can, and 1% of respon­dents iden­ti­fied as Amer­i­can Indi­an or Alas­ka Native. Since the like­ly elec­torate next month will be over­whelm­ing­ly white, you opt­ed to cre­ate a com­bined “peo­ple of col­or” sub­sam­ple. Can you explain why it isn’t fea­si­ble to break out each group of vot­ers sep­a­rate­ly, and why you chose this approach instead?

Ben Green­field, Change Research: Essen­tial­ly, the small­er a sam­ple is, the larg­er the mar­gin of error — mean­ing that if we sur­vey five ran­dom Seat­tle res­i­dents, we can be much less con­fi­dent that they rep­re­sent the entire pop­u­la­tion than if we sur­veyed 5,000. We nev­er pub­lish break­downs of respons­es from groups small­er than fifty, because the mar­gins of error are just so high. Since none of these indi­vid­ual racial/ethnic groups had at least fifty respons­es, we couldn’t pub­lish any of their break­downs indi­vid­u­al­ly. In order to ensure we were not just sin­gling out the views of white vot­ers, we cre­at­ed a break­down of all peo­ple of color.

Andrew Vil­leneuve, NPI: Seat­tle and oth­er cities like New York hold their city-lev­el elec­tions in odd-num­bered years. Turnout is typ­i­cal­ly much low­er in odd-num­bered years. In fact, in Novem­ber of 2017, the last time Seat­tleites elect­ed a may­or, few­er than half of the reg­is­tered vot­ers turned out. And that was high com­pared to oth­er cities. If Seat­tle were to switch to hold­ing its elec­tions in even-num­bered years, as NPI has been advo­cat­ing all cities in Wash­ing­ton do, do you agree we’d see a more diverse elec­torate vot­ing on these city positions?

Ben Green­field, Change Research: Yes, because we tend to see more diver­si­ty in even-num­bered years, not only along racial/ethnic lines, but also across dif­fer­ent socioe­co­nom­ic and age groups.

Andrew Vil­leneuve, NPI: Our gen­er­al elec­tion sur­vey con­sist­ed of 617 inter­views, the same as our Top Two sur­vey. This was improp­er­ly char­ac­ter­ized on Twit­ter by a cou­ple folks as a low sam­ple size. In fact, it’s the high­est sam­ple size of any of the sur­veys con­duct­ed in Seat­tle this cycle with pub­licly released results. Lore­na González’s poll­ster GQR uses sam­ple sizes of 400; Elway/Crosscut also use a sam­ple size of 400, and Strate­gies 360/KOMO had a sam­ple size of 450. Hav­ing a high­er sam­ple size does­n’t nec­es­sar­i­ly mean a sur­vey is more accu­rate, but it does mean that our sur­vey has a low­er mar­gin of error. For those unfa­mil­iar with accept­ed polling prac­tices, can you explain what a typ­i­cal sam­ple size is and why the com­po­si­tion of the sam­ple is far more impor­tant than the size?

Ben Green­field, Change Research: Sam­ple sizes can dif­fer by geog­ra­phy. For exam­ple, it’s not uncom­mon to see 1,000-person polls nation­wide, but you’d rarely see a sam­ple that large in a city like Seat­tle. Though a larg­er sam­ple size will result in small­er mar­gins of error, a poll is only as good as its sam­ple. For exam­ple, a sur­vey of 800 peo­ple who show up at a Trump ral­ly is not going to reflect the views of vot­ing Seat­tleites. But a 617 per­son poll with a rep­re­sen­ta­tive sam­ple of vot­ers across all areas and all back­grounds in the city can.

Andrew Vil­leneuve, NPI: As we have dis­closed through the pub­li­ca­tion of our sur­vey method­ol­o­gy, some of our sur­vey par­tic­i­pants were recruit­ed to par­tic­i­pate in part using ads placed on Insta­gram and Face­book in addi­tion to text mes­sage. It might seem illog­i­cal that a poll con­duct­ed online with respon­dents recruit­ed from Face­book and Insta­gram could be cred­i­ble or trust­wor­thy, but as we like to say, it’s the method that mat­ters, not the medi­um. Can you speak to how Change Research builds its sam­ples and ensures that they are representative?

Ben Green­field, Change Research: While many polling firms recruit their par­tic­i­pants by call­ing their phones, where response rates are incred­i­bly low and call screen­ing is high, we reach vot­ers where they are, and allow them to take sur­veys on their own time. Between social media tar­get­ing and SMS [SMS stands for Short Mes­sage Ser­vice] mes­sages to any­one with a cell phone on record, we are able to reach the vast major­i­ty of vot­ers, and ensure that we are receiv­ing a pro­por­tion­ate response rate from vot­ers of every age, gen­der, race or eth­nic­i­ty, polit­i­cal per­sua­sion, region of the city, and so on.

Andrew Vil­leneuve, NPI: Anoth­er com­ment we saw ques­tioned whether the gen­er­al elec­tion sur­vey’s results had any valid­i­ty giv­en that Coun­cilmem­ber Tere­sa Mosque­da received 39% in the poll after get­ting 59% in the Top Two elec­tion, with 26% unde­cid­ed. Still anoth­er com­menter argued that it was absurd that so many peo­ple could be unde­cid­ed this close to Elec­tion Day. But, in fact, it’s not  uncom­mon for sig­nif­i­cant per­cent­ages of vot­ers to be unde­cid­ed in “non­par­ti­san” local elec­tions like this in the days lead­ing up to an elec­tion, or for some of the sup­port a can­di­date pre­vi­ous­ly received to be tepid, is it?

Ben Green­field, Change Research: Not at all. Espe­cial­ly in munic­i­pal elec­tions, a large per­cent­age of vot­ers tend to make up their minds in the final days before they cast their vote. What’s more, par­tic­u­lar­ly in non­par­ti­san elec­tions, it’s com­mon for peo­ple to give the can­di­dates a fresh look in a gen­er­al elec­tion, and not always default to the can­di­date they vot­ed for initially.

Andrew Vil­leneuve, NPI: In our statewide polling, we con­sis­tent­ly see a high­er num­ber of unde­cid­ed vot­ers in races where no par­ty pref­er­ence is pro­vid­ed on the bal­lot, like State Supreme Court races. (Wash­ing­ton elects its jus­tices to six-year terms, unlike at the fed­er­al lev­el, where they are appoint­ed and serve for life.) Oth­er local­i­ties around the Unit­ed States have par­ti­san local elec­tions instead of “non­par­ti­san” elec­tions. Change Research does work all around the coun­try. Do you find that in par­ti­san local elec­tions, there’s typ­i­cal­ly few­er unde­cid­ed vot­ers than in “non­par­ti­san” local elec­tions like those Wash­ing­ton’s cities have?

Ben Green­field, Change Research: Yes. It’s very com­mon in par­ti­san local elec­tions to see vot­ers cer­tain about who they’ll vote for even if they indi­cate zero famil­iar­i­ty with the can­di­dates — they’ll just choose based on the candidate’s par­ty. Since this is obvi­ous­ly not pos­si­ble in non­par­ti­san races, there are often many more unde­cid­ed voters.

Andrew Vil­leneuve, NPI: Seems like it’s also worth not­ing that Tere­sa Mosqueda’s oppo­nent Ken­neth Wil­son did not poll any high­er than the per­cent­age all of her chal­lengers col­lec­tive­ly received in the Top Two elec­tion (he got 31%). Tere­sa Mosque­da could still end up with most or near­ly all of the unde­cid­ed vot­ers in the gen­er­al elec­tion. We have char­ac­ter­ized her as the favorite. But a vic­to­ry for Wil­son is also a pos­si­bil­i­ty. Polls such as our sur­vey can’t pre­dict the future, as you not­ed, but they can help us guess more effec­tive­ly what could hap­pen by pro­vid­ing evi­dence sup­port­ing one or more plau­si­ble outcomes.

It’s impor­tant to remem­ber that a whole week has already tran­spired since our sur­vey fin­ished field­ing. We can pre­sume the elec­toral dynam­ics have already changed a bit. It’s an elec­tion in progress. Last time, we saw two can­di­dates get big surges of sup­port after our last sur­vey field­ed. One was the can­di­date we’ve been dis­cussing, Ken­neth Wil­son, who polled at 1% and end­ed with over 16%. The oth­er was Sara Nel­son, who polled at 11% and end­ed with 39.47%.

Ben Green­field, Change Research: Exact­ly. So many dynam­ics can change in an instant: a can­di­date receives a key endorse­ment; a video goes viral, etc.

Andrew Vil­leneuve, NPI: Ben, thanks so much for this dis­cus­sion. As I said, we’ve enjoyed work­ing with you this year and look for­ward to con­tin­u­ing to do so. Any con­clud­ing thoughts for our read­ers as we approach the gen­er­al election?

Ben Green­field, Change Research: I’ll say what I said at the begin­ning: polls are just snap­shots, not pre­dic­tions. Any­thing could hap­pen between now and when the final bal­lots are cast, and none of these races have been won or lost.

So if you’re eli­gi­ble, vote!

Thanks again to Change Research for join­ing us to talk about the sci­ence behind our polling! If you have a ques­tion or con­cern we did­n’t answer here, you can leave a com­ment or reach out to us pri­vate­ly using our con­tact form.

And, as Ben said, remem­ber to return your bal­lot by Novem­ber 2nd at 20:00 (8:00 PM) if you’re a Wash­ing­ton State vot­er. We have guid­ance on how to vote on those “advi­so­ry votes” you’ll see at the top of the bal­lot at and there are many orga­ni­za­tion­al endorse­ment guides avail­able if you’d like to take your research beyond the voter’s pam­phlet statements.

Andrew Villeneuve

Recent Posts

Washington’s 2021 general election certified; turnout is the third-worst in state history

Fewer than four in ten voters voted for just the third time in a statewide…

3 days ago

New rules to bar oil/gas drilling in the vicinity of Chaco Culture National Historical Park

Situated in the remote Four Corners area in New Mexico and reached by a rough…

3 days ago

East Link from the air, part two: Get a bird’s eye view of station construction in Redmond

This second part of NPI's East Link from the air project focuses on NPI's hometown…

6 days ago

Last Week In Congress: How Cascadia’s U.S. lawmakers voted (November 15th-19th)

The week's major votes included House passage of the Build Back Better Act and Senate…

6 days ago

House of Representatives approves historic Build Back Better Act, sending it to the Senate

At 6:44 AM Pacific on November 19th, 2021, by a vote of 220-213, the House…

1 week ago

Supreme Court to Redistricting Commission: Tell us exactly what you did on Monday night

The Court sent a clear message to the commissioners with its order today: Explain yourselves.…

1 week ago