A microscope
A microscope analyzing a sample (K. Kkolosov, reproduced under a free content license)

This week, the right wing group Future42 — a “capac­i­ty group” of the right wing polit­i­cal infra­struc­ture ini­tia­tive Project 42 formed not long ago by for­mer Wash­ing­ton Pol­i­cy Cen­ter CEO Dann Mead Smith — released the results of a statewide poll it com­mis­sioned from Ech­e­lon Insights on a range of cur­rent issues.

Right wing com­men­ta­tors have been eager­ly shar­ing screen­shots from this sur­vey on their Twit­ter accounts, while for­mer Q13 Fox per­son­al­i­ty Bran­di Kruse devot­ed a spe­cial episode of her pod­cast to high­light­ing a num­ber of the findings.

Often­times, when a sur­vey like this comes out from a right wing out­fit, the spon­sor sim­ply does­n’t pub­lish enough infor­ma­tion for our team to con­duct an in-depth eval­u­a­tion of the sur­vey. How­ev­er, to their cred­it, Future42 has pub­lished what appears to be their full script and com­plete topline results, along with a descrip­tion of their method­ol­o­gy. That impressed our team, so we decid­ed to take the time to go through the sur­vey and eval­u­ate their work.

Here are our impres­sions of their survey.

How we evaluate polls

First, a few words about how we eval­u­ate polls.

We believe that sub­jec­tive orga­ni­za­tions are per­fect­ly capa­ble of con­duct­ing objec­tive research, so we won’t reflex­ive­ly dis­miss data from a sur­vey mere­ly because it comes from right wing spon­sor. How­ev­er, we are stick­lers for the sci­en­tif­ic method, so we will look to see if that spon­sor and their poll­ster adhered to best prac­tices for obtain­ing sound data, or whether they cut corners.

The key to accu­rate, cred­i­ble polling is neu­tral ques­tions asked of a rep­re­sen­ta­tive sam­ple. You’ve got to have both, or your data is going to be skewed. You can’t find out what peo­ple think if you tell them what to think first, which is why neu­tral ques­tions are impor­tant. And your sam­ple needs to prop­er­ly resem­ble the elec­torate or the pop­u­la­tion of the area you’re sur­vey­ing, or the find­ings sim­ply aren’t going to accu­rate­ly reflect what pub­lic opin­ion is.

It’s extreme­ly dif­fi­cult to write neu­tral ques­tions and it can also be tough to build rep­re­sen­ta­tive sam­ples. But it can be done, as we’ve demon­strat­ed over the past decade in our work with Pub­lic Pol­i­cy Polling of North Car­oli­na, a well-regard­ed poll­ster, and more recent­ly also with Change Research of California.

About the Echelon/Future42 survey

Accord­ing to Ech­e­lon Insights, this was a sur­vey of 600 reg­is­tered vot­ers in the “Wash­ing­ton like­ly elec­torate” (mean­ing, it appears to be a poll of like­ly vot­ers, per­haps like­ly 2024 gen­er­al elec­tion vot­ers, though this is not stat­ed). 70% par­tic­i­pat­ed online (text-to-web) and 30% by land­line (live inter­views). The poll field­ed from June 5th — 7th, 2023, and it has a mar­gin of error of +/- 5.0%.

Ech­e­lon Insights has not released enough pub­lic polling to receive a let­ter grade from FiveThir­tyEight, one of the few enti­ties that rates poll­sters. Instead, it has a pro­vi­sion­al A/B rat­ing. You can see its pro­file on FiveThir­tyEight here.

The firm is based in Vir­ginia, and is Republican-aligned.

(As you can prob­a­bly guess, a Repub­li­can-aligned firm is par­ti­san and works main­ly with right wing clients. A Demo­c­ra­t­ic-aligned firm like either of our poll­sters is the equiv­a­lent, also par­ti­san and works with pro­gres­sive clients.)

A copy of the results file is below.

Results file for Ech­e­lon / Future 42’s spring 2023 survey

Evaluating the sample

On sev­er­al fronts, Ech­e­lon Insights and Future42 did alright with their statewide sam­ple. How­ev­er, in cru­cial areas, we think their sam­ple is skewed.

Here’s a score­card of their demographics:

Look­ing good / pass
  • Gen­der
  • 2020 pres­i­den­tial vote
  • Race and ethnicity
Prob­lem­at­ic / fail
  • Par­ty iden­ti­fi­ca­tion and ideology
  • Edu­ca­tion
  • Age groups

Let’s dive into the prob­lem­at­ic areas.

Ech­e­lon reports that 24% of their respon­dents ini­tial­ly iden­ti­fied as Repub­li­can, 33% Demo­c­ra­t­ic, 35% inde­pen­dent, 6% “some­thing else,” and 2% don’t know / refused / unsure (Ques­tion 27). The ini­tial per­cent­ages for Repub­li­cans and inde­pen­dents seem alright, but the per­cent­age of Demo­c­ra­t­ic vot­ers is too low.

Look­ing at the answers to the very next ques­tion (Ques­tion 28), we can see that the “Some­thing else / don’t know / and refused” vot­ers are more Repub­li­can than Demo­c­ra­t­ic, which means that the sam­ple skews Republican:

QUESTION: If inde­pen­dent, mem­ber of anoth­er par­ty, or unsure, ask: Which of the fol­low­ing state­ments best describes you?

[Ran­dom­ize scale]


Total votes for Repub­li­cans: 38%
Total votes for Democ­rats: 27%

1. I almost always vote for Repub­li­can can­di­dates: 15%
2. I vote for Repub­li­cans more often than I vote for Democ­rats: 23%
3. I vote for Democ­rats more often than I vote for Repub­li­cans: 17%
4. I almost always vote for Demo­c­ra­t­ic can­di­dates: 10%
5. I vote equal­ly for Democ­rats and Repub­li­cans: 28%
6. Don’t know/Refused/Unsure: 7%


QPar­ty + QPartyLean

1. Total Republican+Republican-leaning Inde­pen­dents: 40%
2. Total Democratic+Democratic-leaning Inde­pen­dents: 46%
3. Total Inde­pen­dent: 13%

Then, in the answers to the very next ques­tion, we get con­fir­ma­tion of the skew: 34% say they are very/somewhat con­ser­v­a­tive and 35% say very/somewhat lib­er­al. The num­ber who are con­ser­v­a­tive are sta­tis­ti­cal­ly the same as the num­ber who are lib­er­al? That sim­ply does­n’t reflect Wash­ing­ton State’s elec­toral profile.

As any­one famil­iar with our state’s pol­i­tics knows, Wash­ing­ton is a pre­dom­i­nant­ly Demo­c­ra­t­ic state. We haven’t vot­ed for a Repub­li­can for pres­i­dent since 1984, a Repub­li­can for gov­er­nor since 1980, or a Repub­li­can for U.S. Sen­ate since 1994… and so on. Democ­rats cur­rent­ly hold every statewide office, have large majori­ties in both cham­bers of the Leg­is­la­ture, and have eight of ten U.S. House seats.

Most vot­ers here are Demo­c­ra­t­ic or lean Demo­c­ra­t­ic. Under­rep­re­sent­ing them in a sur­vey is a car­di­nal sin if you’re doing pub­lic opin­ion research in Washington.

In the edu­ca­tion crosstabs, there’s also a skew.

Those with only a high school edu­ca­tion or less are slight­ly over-rep­re­sent­ed, while col­lege grad­u­ates are slight­ly under­rep­re­sent­ed. Here’s a breakdown:

QUESTION: What is the high­est lev­el of edu­ca­tion you have completed?


1. High school or less: 25% [Per­cent­age is too high]
2. Some col­lege: 22% [A lit­tle low]
3. Associate’s degree: 12% [A lit­tle low]
4. Bachelor’s degree: 25% [A lit­tle low]
5. Grad­u­ate degree: 15% [Per­cent­age OK]

Edu­ca­tion nowa­days has a strong cor­re­la­tion with par­ty — the more edu­cat­ed a vot­er is, the more like­ly they are to be a Demo­c­ra­t­ic vot­er ver­sus a Repub­li­can vot­er — so this is addi­tion­al con­fir­ma­tion that their sam­ple skews Republican.

On age, we see more younger vot­ers than we’d expect to see and few­er mid­dle-aged vot­ers. How­ev­er, this is more of a minor quib­ble com­pared to the oth­er two cat­e­gories and prob­a­bly does­n’t neg­a­tive­ly impact the toplines as much.

The bot­tom line? This unfor­tu­nate­ly isn’t a prop­er­ly rep­re­sen­ta­tive sam­ple of the Wash­ing­ton State elec­torate. It’s just too right wing.

The questions

The sur­vey con­tains twen­ty-two top­i­cal or issue ques­tions, grouped under sev­er­al cat­e­gories: Land­scape, The Leg­is­la­ture, Crime/Policing, Cost of Liv­ing and Tax­es, Ener­gy Costs, Housing/Homelessness, and Parental Rights.

Cri­tiquing every ques­tion would result in an extreme­ly lengthy post, so rather than do that, I’ll dis­cuss a few select­ed ques­tions. First, let’s con­sid­er a ques­tion that gets a pass­ing grade with respect to the neu­tral­i­ty of its wording:

QUESTION: If the elec­tion for the Wash­ing­ton state leg­is­la­ture in your dis­trict were held today, would you vote for the [ROTATE: Republican/Democratic] can­di­date or the [ROTATE: Democratic/Republican] candidate?

[PHONE: If Republican/Democratic: And would you DEFINITELY vote for the (Republican/Democratic) can­di­date or only PROBABLY vote for the (Republican/Democratic) candidate?]


The Repub­li­can can­di­date, definitely/probably: 44%
The Demo­c­ra­t­ic can­di­date, definitely/probably: 45%

1. The Repub­li­can can­di­date, def­i­nite­ly: 29%
2. The Repub­li­can can­di­date, prob­a­bly: 15%
3. The Demo­c­ra­t­ic can­di­date, prob­a­bly: 18%
4. The Demo­c­ra­t­ic can­di­date, def­i­nite­ly: 27%
5. Don’t know/Refused/Unsure: 11%

The word­ing here works because no attempt is being made to influ­ence peo­ple’s respons­es one way or the oth­er, which is extreme­ly important.

They also rotat­ed their answer options, which they did­n’t have to do, but which is very defen­si­ble. And unless some­thing was omit­ted from the script, they avoid­ed try­ing to stack the deck by putting neg­a­tive mes­sag­ing into pre­vi­ous ques­tions (e.g. by say­ing Demo­c­ra­t­ic leg­is­la­tors have been pass­ing bad policies).

How­ev­er, because their sam­ple skews Repub­li­can, the respons­es to this ques­tion don’t reflect the state’s actu­al elec­toral dynam­ics. We’d expect to see some­where in the range of like a five to ten point lead for the Demo­c­ra­t­ic Par­ty in a gener­ic bal­lot ques­tion like this. Instead, the Repub­li­cans are even with the Democrats.

This is an unsup­port­able finding.

Sim­i­lar­ly, in an adja­cent ques­tion, imme­di­ate­ly pri­or to this one, respon­dents were asked whether they have a favor­able or unfa­vor­able view of three peo­ple and two groups: Com­mis­sion­er of Pub­lic Lands Hilary Franz, Democ­rats in the Wash­ing­ton State Leg­is­la­ture, Attor­ney Gen­er­al Bob Fer­gu­son, Gov­er­nor Jay Inslee, and Repub­li­cans in the Wash­ing­ton State Legislature.

The ques­tion word­ing looks just fine.

But because of the sam­ple’s Repub­li­can skew, the favor­a­bil­i­ty rat­ings don’t line up with what we’d see in a cred­i­ble pub­lic opin­ion research survey.

For exam­ple, con­sid­er Gov­er­nor Inslee’s favor­a­bil­i­ty rating:

QUESTION: Do you have a favor­able or unfa­vor­able view of the fol­low­ing groups or indi­vid­u­als in the news late­ly? If you haven’t heard of this per­son, just say so. [RANDOMIZE OPTIONS] [PHONE: If favorable/unfavorable: And would you say you have a VERY (favorable/unfavorable) view or only a SOMEWHAT (favorable/unfavorable) view?


  • Favor­able (Very/somewhat): 40% 
    • Very favor­able: 20%
    • Some­what favor­able: 20%
  • Unfa­vor­able (Very/somewhat): 56% 
    • Some­what unfa­vor­able: 12%
    • Very unfa­vor­able: 44%
  • Heard of / no opin­ion: 3%
  • Haven’t heard of: 1%

We peri­od­i­cal­ly ask about Gov­er­nor Jay Inslee’s job per­for­mance, includ­ing as recent­ly as last month. That’s not the same thing as favor­a­bil­i­ty, but it is a sim­i­lar line of inquiry. Typ­i­cal­ly, we see close to half our sam­ple giv­ing Gov­er­nor Inslee pos­i­tive marks and close to half giv­ing him neg­a­tive marks.

That was true again about a month ago:

QUESTION (FROM NPI’S RESEARCH): Do you approve or dis­ap­prove of Gov­er­nor Jay Inslee’s job performance?


  • Approve: 48%
  • Dis­ap­prove: 47%
  • Not sure: 5%

Our sur­vey of 773 like­ly 2024 Wash­ing­ton State vot­ers was in the field from Wednes­day, June 7th through Thurs­day, June 8th, 2023.

The poll uti­lizes a blend­ed method­ol­o­gy, with auto­mat­ed phone calls to land­lines (41%) and online answers from cell phone only respon­dents (59%).

It was con­duct­ed by Pub­lic Pol­i­cy Polling (PPP) for the North­west Pro­gres­sive Insti­tute, and has a mar­gin of error of +/- 3.5% at the 95% con­fi­dence interval.

NPI and PPP have worked togeth­er for a decade and have a track record of excel­lence, as detailed in this 2022 elec­toral polling recap and this 2020 one.

You can see from oth­er pub­lic opin­ion research since the 2020 pres­i­den­tial and guber­na­to­r­i­al elec­tion that our find­ing is a typ­i­cal one, where­as Echelon/Future42’s is an out­lier. It is not sur­pris­ing that a poll that over-sam­ples Repub­li­cans would yield more neg­a­tive num­bers for Gov­er­nor Inslee, as well as Attor­ney Gen­er­al Bob Fer­gu­son, who’s now run­ning for governor.

Now let’s look at a ques­tion that utter­ly fails the neu­tral­i­ty test.

QUESTION: Under Fed­er­al law, cap­i­tal gains are con­sid­ered income, and the state con­sti­tu­tion of Wash­ing­ton pro­hibits an income tax. Recent­ly, the State Supreme Court ruled that a cap­i­tal gains tax is per­mis­si­ble if it is called an “excise tax”. Was the Supreme Court right or wrong to revise the def­i­n­i­tion of income to allow for this new tax?

(Answer options were ran­dom­ized)


  • Wrong: 62%
  • Right: 24%
  • Don’t know / refused / unsure: 14%

This is def­i­nite­ly a con­tender for the worst ques­tion in this poll: it’s prej­u­di­cial­ly word­ed and it con­tains inac­cu­rate infor­ma­tion. Let’s cor­rect the record:

  1. What fed­er­al law says about cap­i­tal gains is not rel­e­vant because peo­ple are being asked about a state law which levied a cap­i­tal gains tax only on the wealth­i­est Wash­ing­ton State fam­i­lies and households.
  2. The Wash­ing­ton State Con­sti­tu­tion does not pro­hib­it an income tax. This is a myth. Go ahead and read our plan of gov­ern­ment for your­self — nowhere will you find a pro­vi­sion say­ing that the state may not impose an income tax. There is an old Supreme Court deci­sion from the 1930s which held that income is con­sid­ered prop­er­ty, and there is a clause in the Con­sti­tu­tion (the “uni­for­mi­ty clause”) that says tax­es on prop­er­ty must be uni­form, but that’s not a pro­hi­bi­tion against hav­ing an income tax.
  3. The state Supreme Court’s recent rul­ing in Quinn is egre­gious­ly mis­char­ac­ter­ized here. The Court did not rule that a cap­i­tal gains tax is per­mis­si­ble if it is called an excise tax. Rather, it ruled the cap­i­tal gains tax is per­mis­si­ble because it is an excise tax. The Court exam­ined the tax in-depth and the law that cre­at­ed it, and emphat­i­cal­ly reject­ed the right wing’s argu­ment that it is not real­ly an excise tax.
  4. And final­ly, the Supreme Court did not “revise the def­i­n­i­tion of income to allow for this new tax.” Rather, the Court con­clud­ed that the Leg­is­la­ture had act­ed with­in its law­ful author­i­ty to cre­ate an excise tax on cap­i­tal gains. See for your­self by read­ing the major­i­ty opin­ion.

At no point in this ques­tion are respon­dents giv­en any pos­i­tive infor­ma­tion about the cap­i­tal gains tax on the wealthy. At no point are they told who pays it and what’s exempt. Instead, they’re told the tax is bad and then invit­ed to agree with the authors of the ques­tion that the Supreme Court erred by uphold­ing the tax.

As I men­tioned ear­li­er, you can’t find out what peo­ple think if you tell them what to think first. Future42 did not write this ques­tion in a way that would allow the respon­dents to bring their own opin­ions to the table. Instead, they wrote a loaded ques­tion which reads like a Tim Eyman style push poll. Bad ques­tions and a skewed sam­ple com­bine here to yield utter­ly worth­less figures.

Here is what a bal­anced, neu­tral­ly word­ed ques­tion looks like. This is one we’ve asked for sev­er­al con­sec­u­tive years now, includ­ing last month:

QUESTION (FROM NPI’S RESEARCH): Pro­po­nents say that Wash­ing­ton State’s new state cap­i­tal gains tax on the wealthy will raise about $500 mil­lion a year in cru­cial fund­ing for edu­ca­tion in Wash­ing­ton State, includ­ing ear­ly learn­ing and child­care, and will help bal­ance our upside-down tax code by requir­ing the wealth­i­est 8,000 indi­vid­u­als to step up and pay their fair share in dues to our state. Oppo­nents say that the new state cap­i­tal gains tax on the wealthy is an uncon­sti­tu­tion­al and ille­gal income tax that will hurt job cre­ation and put the state at a com­pet­i­tive dis­ad­van­tage, hurt­ing the whole econ­o­my while fail­ing to address regres­siv­i­ty. Both sides agree that the text of the cap­i­tal gains tax law ful­ly exempts retire­ment accounts, fam­i­ly farms, and all real estate. Hav­ing heard the argu­ments for and against, do you strong­ly sup­port, some­what sup­port, some­what oppose, or strong­ly oppose Washington’s new state cap­i­tal gains tax on the wealthy?


  • Sup­port: 57% 
    • Strong­ly sup­port: 44%
    • Some­what sup­port: 13%
  • Oppose: 37%
    • Some­what oppose: 9%
    • Strong­ly oppose: 28%
  • Not sure: 6%

Sup­port is about the same as when we asked last year (57% now ver­sus 56% last June.) Oppo­si­tion is the same, at 37%, with a tiny, sta­tis­ti­cal­ly insignif­i­cant bit of soft­en­ing. The num­ber of not sure vot­ers has also like­wise bare­ly changed.

You’ll notice that in our ques­tion, the uncon­sti­tu­tion­al and ille­gal income tax fram­ing is present, but it’s appro­pri­ate­ly attrib­uted to oppo­nents of the tax rather than being passed off as a fact. (If you’re curi­ous, the “con” phras­ing used came direct­ly from a group led by Rob McKen­na — Oppor­tu­ni­ty Washington.)

The qual­i­ty of the non-demo­graph­ic ques­tions real­ly var­ied in this poll.

Gen­er­al­ly speak­ing, the elec­toral ques­tions and favor­a­bil­i­ty ques­tions had accept­able word­ing, while the ques­tions about issues uti­lized right wing fram­ing and lacked neu­tral­i­ty to vary­ing degrees. Inter­est­ing­ly, Ech­e­lon gen­er­al­ly asked respon­dents whether some­thing was “a good idea” or “a bad idea” rather than ask­ing if they sup­port­ed or opposed it, which is the dichoto­my we gen­er­al­ly use. Their results file says the answer options were ran­dom­ized as a gen­er­al practice.

Concluding thoughts

Our assess­ment is that this sur­vey skews Repub­li­can by as much as ten points, which is coin­ci­den­tal­ly twice its stat­ed mar­gin of error. Accord­ing­ly, even the topline respons­es to the neu­tral­ly word­ed ques­tions are flawed.

That’s dis­ap­point­ing, because this sur­vey might oth­er­wise have yield­ed some use­ful find­ings and added to our state’s body of cred­i­ble pub­lic opin­ion research.

For their next sur­vey, we urge Echelon/Future42 to (1) fix their sam­pling to include a suf­fi­cient num­ber of Demo­c­ra­t­ic and col­lege-edu­cat­ed vot­ers and (2) sim­pli­fy their par­ty iden­ti­fi­ca­tion ques­tion, which also could help with (1). Those moves might at least yield usable data from their elec­toral ques­tions, even if their issue-relat­ed ques­tions remain poor­ly and prej­u­di­cial­ly written.

About the author

Andrew Villeneuve is the founder and executive director of the Northwest Progressive Institute, as well as the founder of NPI's sibling, the Northwest Progressive Foundation. He has worked to advance progressive causes for over two decades as a strategist, speaker, author, and organizer. Andrew is also a cybersecurity expert, a veteran facilitator, a delegate to the Washington State Democratic Central Committee, and a member of the Climate Reality Leadership Corps.

Adjacent posts