NPI's Cascadia Advocate

Offering commentary and analysis from Washington, Oregon, and Idaho, The Cascadia Advocate is the Northwest Progressive Institute's unconventional perspective on world, national, and local politics.

Friday, October 22nd, 2021

NPI, Change Research discuss how to read and assess this year’s Seattle electoral polling

This past week, our team at the North­west Pro­gres­sive Insti­tute released the ini­tial results of our sec­ond 2021 sur­vey of the Seat­tle elec­torate, which was con­duct­ed for us by Change Research of Cal­i­for­nia. We found Bruce Har­rell six­teen points of Lore­na González for May­or, Ann Davi­son nine­teen points ahead of Nicole Thomas-Kennedy for City Attor­ney, Tere­sa Mosque­da eight points ahead of Ken­neth Wil­son for Coun­cil #8, and Sara Nel­son four points ahead of Nikki­ta Oliv­er for Coun­cil #9, with sig­nif­i­cant num­bers of vot­ers still unde­cid­ed. We also found huge majori­ties of vot­ers unde­cid­ed in the school board races.

As of today, one or more of our find­ings have been men­tioned dur­ing the evening news­casts of all four of Seat­tle’s local tele­vi­sion sta­tions and have also received exten­sive radio and online cov­er­age. The Seat­tle Times’ Daniel Beek­man wrote an excel­lent sto­ry sum­ma­riz­ing the find­ings that appeared in print and on seattletimes.com. Last­ly, our find­ings have received quite a bit of atten­tion and dis­cus­sion on social plat­forms like Face­book, Red­dit, and Twitter.

We’re heart­ened that so many peo­ple are inter­est­ed in our research.

We know that try­ing to fig­ure out how to read and assess poll data can be chal­leng­ing. Our sur­vey was one of only a few inde­pen­dent sur­veys to be con­duct­ed in Seat­tle dur­ing this gen­er­al elec­tion, and the only one to have field­ed this month (the oth­ers field­ed in Sep­tem­ber). Since oth­er recent polling in Seat­tle con­duct­ed for inter­est­ed par­ties has not been pub­licly released, our data is basi­cal­ly stand­ing alone on its own instead of being one sur­vey among many.

When you’ve got only one poll to look at in a giv­en time peri­od, you can­not make com­par­isons with oth­er polls to ascer­tain trends and com­mon­al­i­ties. All you can do is judge whether the one poll in front of you is cred­i­ble or not.

Today, to address some of the ques­tions and com­ments we’ve got­ten about our poll since Tues­day, we are delight­ed to wel­come Ben Green­field to NPI’s Cas­ca­dia Advo­cate. Ben is the Senior Sur­vey Data Ana­lyst at Change Research respon­si­ble for the field­ing of our sur­veys along with his col­league Ben Sullivan.

We hope you enjoy this Q&A and find it help­ful for putting our results in context.


Andrew Vil­leneuve, NPI: Ben, thanks for join­ing me to dis­cuss our work togeth­er this year! We’re delight­ed to have had the oppor­tu­ni­ty to launch our research polling part­ner­ship with Change Research in Seat­tle. It’s been a fas­ci­nat­ing elec­tion cycle and we’re not even to the end of it yet!

Ben Green­field, Change Research: Work­ing with you and the North­west Pro­gres­sive Insti­tute team has been a great plea­sure for us — and I can’t dis­agree with you on it being a fas­ci­nat­ing elec­tion cycle!

Andrew Vil­leneuve, NPI: Our Seat­tle polling this year con­sist­ed of two sur­veys: one in July that pre­ced­ed the Top Two elec­tion and one this month pre­ced­ing the Novem­ber gen­er­al elec­tion. We don’t know what the results of the gen­er­al elec­tion will be, but we know that our first sur­vey was able to antic­i­pate a lot of the dynam­ics we saw in the Top Two elec­tion, with sev­en of the eight can­di­dates who advanced to the runoff round hav­ing placed first or sec­ond in our polling.

Ben Green­field, Change Research: Indeed. While it’s always impor­tant to remem­ber that polling pro­vides a snap­shot of where vot­ers are at a giv­en moment before they cast their bal­lots — and not a pre­dic­tion of where they’ll go — we were pleased to see that the poll we con­duct­ed togeth­er accu­rate­ly cap­tured some of the key dynam­ics and vot­er pref­er­ences in the Top Two election.

Andrew Vil­leneuve, NPI: After we released our gen­er­al elec­tion poll find­ings this week, we start­ed get­ting inquiries and com­ments about our sur­vey’s method­ol­o­gy and sam­pling. One of those ques­tions per­tained to the diver­si­ty of the sam­ple: 82% of the sur­vey tak­ers iden­ti­fied as white, which is a high­er per­cent­age than in oth­er sur­veys of the Seat­tle elec­torate this year, like the Strate­gies 360/KOMO poll from last month. But dif­fer­ent sur­veys are mod­eled on dif­fer­ent uni­vers­es. We chose to poll like­ly vot­ers instead of reg­is­tered vot­ers, and con­se­quent­ly, our sam­ple is mod­eled on the last sim­i­lar elec­tion, which occurred in Novem­ber 2017. How is polling like­ly vot­ers dif­fer­ent from polling reg­is­tered vot­ers or even sur­vey­ing the pop­u­la­tion of a city like Seat­tle as a whole, and what ram­i­fi­ca­tions did our choice have for the sur­vey’s ethnic/racial composition?

Ben Green­field, Change Research: Poll­sters have dif­fer­ent meth­ods for pro­ject­ing who’s like­ly to turn out in a giv­en elec­tion and who’s less like­ly, and that can dif­fer from elec­tion to elec­tion. Our turnout mod­el took into account both past vot­ing pat­terns in Seat­tle munic­i­pal elec­tions and sur­vey respon­dents’ self-stat­ed like­li­hood of vot­ing. Both turnout his­to­ry and self-stat­ed like­li­hood of vot­ing are imper­fect pre­dic­tors of turnout, but both have some rela­tion­ship to actu­al turnout, and we believe our com­bi­na­tion of these fac­tors leaves us with a view of the elec­torate that approx­i­mates what we’ll actu­al­ly see.

As far as the ethnic/racial com­po­si­tion of the sur­vey, our pro­jec­tions are again based on each group’s his­tor­i­cal turnout rate. In munic­i­pal elec­tions, turnout has his­tor­i­cal­ly been high­er among white vot­ers than vot­ers of col­or, and as a con­se­quence of that, the his­tor­i­cal elec­torates, and our sur­vey, have been whiter than the entire pop­u­la­tion of Seattle.

Andrew Vil­leneuve, NPI: In our sur­vey, 7% of respon­dents iden­ti­fied as Asian or Pacif­ic Islander, 5% of respon­dents iden­ti­fied as His­pan­ic or Latino/a, 3% of respon­dents iden­ti­fied as Black or African Amer­i­can, and 1% of respon­dents iden­ti­fied as Amer­i­can Indi­an or Alas­ka Native. Since the like­ly elec­torate next month will be over­whelm­ing­ly white, you opt­ed to cre­ate a com­bined “peo­ple of col­or” sub­sam­ple. Can you explain why it isn’t fea­si­ble to break out each group of vot­ers sep­a­rate­ly, and why you chose this approach instead?

Ben Green­field, Change Research: Essen­tial­ly, the small­er a sam­ple is, the larg­er the mar­gin of error — mean­ing that if we sur­vey five ran­dom Seat­tle res­i­dents, we can be much less con­fi­dent that they rep­re­sent the entire pop­u­la­tion than if we sur­veyed 5,000. We nev­er pub­lish break­downs of respons­es from groups small­er than fifty, because the mar­gins of error are just so high. Since none of these indi­vid­ual racial/ethnic groups had at least fifty respons­es, we couldn’t pub­lish any of their break­downs indi­vid­u­al­ly. In order to ensure we were not just sin­gling out the views of white vot­ers, we cre­at­ed a break­down of all peo­ple of color.

Andrew Vil­leneuve, NPI: Seat­tle and oth­er cities like New York hold their city-lev­el elec­tions in odd-num­bered years. Turnout is typ­i­cal­ly much low­er in odd-num­bered years. In fact, in Novem­ber of 2017, the last time Seat­tleites elect­ed a may­or, few­er than half of the reg­is­tered vot­ers turned out. And that was high com­pared to oth­er cities. If Seat­tle were to switch to hold­ing its elec­tions in even-num­bered years, as NPI has been advo­cat­ing all cities in Wash­ing­ton do, do you agree we’d see a more diverse elec­torate vot­ing on these city positions?

Ben Green­field, Change Research: Yes, because we tend to see more diver­si­ty in even-num­bered years, not only along racial/ethnic lines, but also across dif­fer­ent socioe­co­nom­ic and age groups.

Andrew Vil­leneuve, NPI: Our gen­er­al elec­tion sur­vey con­sist­ed of 617 inter­views, the same as our Top Two sur­vey. This was improp­er­ly char­ac­ter­ized on Twit­ter by a cou­ple folks as a low sam­ple size. In fact, it’s the high­est sam­ple size of any of the sur­veys con­duct­ed in Seat­tle this cycle with pub­licly released results. Lore­na González’s poll­ster GQR uses sam­ple sizes of 400; Elway/Crosscut also use a sam­ple size of 400, and Strate­gies 360/KOMO had a sam­ple size of 450. Hav­ing a high­er sam­ple size does­n’t nec­es­sar­i­ly mean a sur­vey is more accu­rate, but it does mean that our sur­vey has a low­er mar­gin of error. For those unfa­mil­iar with accept­ed polling prac­tices, can you explain what a typ­i­cal sam­ple size is and why the com­po­si­tion of the sam­ple is far more impor­tant than the size?

Ben Green­field, Change Research: Sam­ple sizes can dif­fer by geog­ra­phy. For exam­ple, it’s not uncom­mon to see 1,000-person polls nation­wide, but you’d rarely see a sam­ple that large in a city like Seat­tle. Though a larg­er sam­ple size will result in small­er mar­gins of error, a poll is only as good as its sam­ple. For exam­ple, a sur­vey of 800 peo­ple who show up at a Trump ral­ly is not going to reflect the views of vot­ing Seat­tleites. But a 617 per­son poll with a rep­re­sen­ta­tive sam­ple of vot­ers across all areas and all back­grounds in the city can.

Andrew Vil­leneuve, NPI: As we have dis­closed through the pub­li­ca­tion of our sur­vey method­ol­o­gy, some of our sur­vey par­tic­i­pants were recruit­ed to par­tic­i­pate in part using ads placed on Insta­gram and Face­book in addi­tion to text mes­sage. It might seem illog­i­cal that a poll con­duct­ed online with respon­dents recruit­ed from Face­book and Insta­gram could be cred­i­ble or trust­wor­thy, but as we like to say, it’s the method that mat­ters, not the medi­um. Can you speak to how Change Research builds its sam­ples and ensures that they are representative?

Ben Green­field, Change Research: While many polling firms recruit their par­tic­i­pants by call­ing their phones, where response rates are incred­i­bly low and call screen­ing is high, we reach vot­ers where they are, and allow them to take sur­veys on their own time. Between social media tar­get­ing and SMS [SMS stands for Short Mes­sage Ser­vice] mes­sages to any­one with a cell phone on record, we are able to reach the vast major­i­ty of vot­ers, and ensure that we are receiv­ing a pro­por­tion­ate response rate from vot­ers of every age, gen­der, race or eth­nic­i­ty, polit­i­cal per­sua­sion, region of the city, and so on.

Andrew Vil­leneuve, NPI: Anoth­er com­ment we saw ques­tioned whether the gen­er­al elec­tion sur­vey’s results had any valid­i­ty giv­en that Coun­cilmem­ber Tere­sa Mosque­da received 39% in the poll after get­ting 59% in the Top Two elec­tion, with 26% unde­cid­ed. Still anoth­er com­menter argued that it was absurd that so many peo­ple could be unde­cid­ed this close to Elec­tion Day. But, in fact, it’s not  uncom­mon for sig­nif­i­cant per­cent­ages of vot­ers to be unde­cid­ed in “non­par­ti­san” local elec­tions like this in the days lead­ing up to an elec­tion, or for some of the sup­port a can­di­date pre­vi­ous­ly received to be tepid, is it?

Ben Green­field, Change Research: Not at all. Espe­cial­ly in munic­i­pal elec­tions, a large per­cent­age of vot­ers tend to make up their minds in the final days before they cast their vote. What’s more, par­tic­u­lar­ly in non­par­ti­san elec­tions, it’s com­mon for peo­ple to give the can­di­dates a fresh look in a gen­er­al elec­tion, and not always default to the can­di­date they vot­ed for initially.

Andrew Vil­leneuve, NPI: In our statewide polling, we con­sis­tent­ly see a high­er num­ber of unde­cid­ed vot­ers in races where no par­ty pref­er­ence is pro­vid­ed on the bal­lot, like State Supreme Court races. (Wash­ing­ton elects its jus­tices to six-year terms, unlike at the fed­er­al lev­el, where they are appoint­ed and serve for life.) Oth­er local­i­ties around the Unit­ed States have par­ti­san local elec­tions instead of “non­par­ti­san” elec­tions. Change Research does work all around the coun­try. Do you find that in par­ti­san local elec­tions, there’s typ­i­cal­ly few­er unde­cid­ed vot­ers than in “non­par­ti­san” local elec­tions like those Wash­ing­ton’s cities have?

Ben Green­field, Change Research: Yes. It’s very com­mon in par­ti­san local elec­tions to see vot­ers cer­tain about who they’ll vote for even if they indi­cate zero famil­iar­i­ty with the can­di­dates — they’ll just choose based on the candidate’s par­ty. Since this is obvi­ous­ly not pos­si­ble in non­par­ti­san races, there are often many more unde­cid­ed voters.

Andrew Vil­leneuve, NPI: Seems like it’s also worth not­ing that Tere­sa Mosqueda’s oppo­nent Ken­neth Wil­son did not poll any high­er than the per­cent­age all of her chal­lengers col­lec­tive­ly received in the Top Two elec­tion (he got 31%). Tere­sa Mosque­da could still end up with most or near­ly all of the unde­cid­ed vot­ers in the gen­er­al elec­tion. We have char­ac­ter­ized her as the favorite. But a vic­to­ry for Wil­son is also a pos­si­bil­i­ty. Polls such as our sur­vey can’t pre­dict the future, as you not­ed, but they can help us guess more effec­tive­ly what could hap­pen by pro­vid­ing evi­dence sup­port­ing one or more plau­si­ble outcomes.

It’s impor­tant to remem­ber that a whole week has already tran­spired since our sur­vey fin­ished field­ing. We can pre­sume the elec­toral dynam­ics have already changed a bit. It’s an elec­tion in progress. Last time, we saw two can­di­dates get big surges of sup­port after our last sur­vey field­ed. One was the can­di­date we’ve been dis­cussing, Ken­neth Wil­son, who polled at 1% and end­ed with over 16%. The oth­er was Sara Nel­son, who polled at 11% and end­ed with 39.47%.

Ben Green­field, Change Research: Exact­ly. So many dynam­ics can change in an instant: a can­di­date receives a key endorse­ment; a video goes viral, etc.

Andrew Vil­leneuve, NPI: Ben, thanks so much for this dis­cus­sion. As I said, we’ve enjoyed work­ing with you this year and look for­ward to con­tin­u­ing to do so. Any con­clud­ing thoughts for our read­ers as we approach the gen­er­al election?

Ben Green­field, Change Research: I’ll say what I said at the begin­ning: polls are just snap­shots, not pre­dic­tions. Any­thing could hap­pen between now and when the final bal­lots are cast, and none of these races have been won or lost.

So if you’re eli­gi­ble, vote!

Thanks again to Change Research for join­ing us to talk about the sci­ence behind our polling! If you have a ques­tion or con­cern we did­n’t answer here, you can leave a com­ment or reach out to us pri­vate­ly using our con­tact form.

And, as Ben said, remem­ber to return your bal­lot by Novem­ber 2nd at 20:00 (8:00 PM) if you’re a Wash­ing­ton State vot­er. We have guid­ance on how to vote on those “advi­so­ry votes” you’ll see at the top of the bal­lot at VoteMaintained.org and there are many orga­ni­za­tion­al endorse­ment guides avail­able if you’d like to take your research beyond the voter’s pam­phlet statements.

Adjacent posts

  • Enjoyed what you just read? Make a donation


    Thank you for read­ing The Cas­ca­dia Advo­cate, the North­west Pro­gres­sive Insti­tute’s jour­nal of world, nation­al, and local politics.

    Found­ed in March of 2004, The Cas­ca­dia Advo­cate has been help­ing peo­ple through­out the Pacif­ic North­west and beyond make sense of cur­rent events with rig­or­ous analy­sis and thought-pro­vok­ing com­men­tary for more than fif­teen years. The Cas­ca­dia Advo­cate is fund­ed by read­ers like you and trust­ed spon­sors. We don’t run ads or pub­lish con­tent in exchange for money.

    Help us keep The Cas­ca­dia Advo­cate edi­to­ri­al­ly inde­pen­dent and freely avail­able to all by becom­ing a mem­ber of the North­west Pro­gres­sive Insti­tute today. Or make a dona­tion to sus­tain our essen­tial research and advo­ca­cy journalism.

    Your con­tri­bu­tion will allow us to con­tin­ue bring­ing you fea­tures like Last Week In Con­gress, live cov­er­age of events like Net­roots Nation or the Demo­c­ra­t­ic Nation­al Con­ven­tion, and reviews of books and doc­u­men­tary films.

    Become an NPI mem­ber Make a one-time donation

  • NPI’s essential research and advocacy is sponsored by: