With 2022 drawing swiftly to a close, The New York Times today published an article with not one, not two, but three reporters’ bylines that examines the impact that garbage data from untrustworthy polls had on the 2022 midterms, including Washington State’s U.S. Senate race.
Titled “The ‘red wave’ washout: How skewed polls fed a false election narrative,” the story mainly focuses on Republican-aligned firms’ bad polls and how they landed, but makes an important admission in passing: newspapers like the New York Times bought wholesale into an electoral narrative that wasn’t real.
“Not for the first time, a warped understanding of the contours of a national election had come to dominate the views of political operatives, donors, journalists and, in some cases, the candidates themselves,” the story penned by the trio of Jim Rutenberg, Ken Bensinger, and Steve Eder notes after several introductory paragraphs dissecting the Republicans’ bad Evergreen State data.
A few paragraphs later came a more explicit admission — still in passing, but an important and necessary admission nonetheless — acknowledging that The New York Times’ own coverage was wrongly wedded to a “red wave” narrative:
The skewed red-wave surveys polluted polling averages, which are relied upon by campaigns, donors, voters and the news media. It fed the home-team boosterism of an expanding array of right-wing media outlets — from Steve Bannon’s “War Room” podcast and “The Charlie Kirk Show” to Fox News and its top-rated prime-time lineup. And it spilled over into coverage by mainstream [mainstream meaning big] news organizations, including The Times, that amplified the alarms being sounded about potential Democratic doom.
Highlighting is mine.
For an example, the trio and their editors picked a pretentious column by Blake Hounshell, published October 19th. Titled “Democrats’ Feared Red October Has Arrived,” it began with this declaration: “Here’s the thing about elections: When they break, they usually break in one direction. And right now, all the indicators on my political dashboard are blinking red — as in, toward Republicans.”
Hounshell’s column was a well-chosen example. But it was actually just one of a great many pieces that the paper published over the course of weeks and months favoring Republicans and playing up their chances. We documented how The Times was failing its readers with its slanted, narrative-driven coverage right here on the Cascadia Advocate. (If you haven’t read that piece, it’s worth your time.)
On Election Night (Tuesday, November 8th), it was evident that while Republicans might be on their way to a House majority, there was no “red wave.” Democrats were doing far better than many pundits and reporters had expected. The New York Times promptly pivoted away from the narrative it had helped to promote at nearly every opportunity as its editors and reporters processed the data.
If you browse the NYT’s archives, you can see that Election Night was a huge demarcation point. Until November 8th, the paper’s midterms coverage was narrative-driven; after November 8th, the paper’s midterms coverage became data and reality-driven thanks to the availability of the initial returns.
Here on the Cascadia Advocate, we tried to elevate in our preelection coverage the voices of those who were looking carefully at the data and warning of the folly of drawing conclusions based on narratives, such as the New Democrat Network’s Simon Rosenberg and TargetSmart’s Tom Bonier.
Appropriately, both Rosenberg and Bonier are mentioned in this NYT story.
Rosenberg’s tussle with FiveThirtyEight founder Nate Silver on Twitter over the garbage Republican polling indexed by FiveThirtyEight is also mentioned.
“Mr. Silver did not respond to repeated requests for comment,” the story says.
Silver, himself employed for a stretch by The New York Times, is the author of a book called The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t. On one of FiveThirtyEight’s podcasts, Silver had ridiculously responded to Rosenberg’s criticisms by characterizing Rosenberg as smoking “hopium.” Of course, as events showed, Rosenberg’s concerns were well-founded, and he was entirely justified in declaring that FiveThirtyEight’s data had become polluted.
As October drew to a close, our team at NPI were likewise shaking our heads at the appearance of more and more garbage polls in Washington’s U.S. Senate race. There were so many bad polls that the FiveThirtyEight index was no longer showing any credible ones above the fold on its contest-specific list:
All of the polls shown above-the-fold on FiveThirtyEight as of the end of the voting period were from firms aligned with Republicans that erroneously showed a close race. To see any credible polls, it was necessary to click or tap “Show more polls.” (Screenshot)
Our own polling had consistently shown over the course of eighteen months that Patty Murray was on track to be comfortably reelected, yet the contest between Murray and her Republican challenger Tiffany Smiley was increasingly and incorrectly being described as “tight”, “close”, and “volatile.”
In a story that aired the week before the election, KIRO7’s Graham Johnson looked at the body of polling, bad and good lumped together, and talked to three people about what it could mean: consultant Crystal Fincher, Republican operative Alex Hays, and myself. During my interview with Graham, I explained that not all polling is created equal, and it’s essential to separate the wheat from the chaff.
Credible polling, I explained, had Murray up and well ahead of Smiley, whereas the garbage polling that incorrectly and improperly suggested Smiley was close to Murray was being funded and produced by Republicans.
Only a snippet of my comments made it into the final story that ran on-air.
But, as you can see, Graham and the KIRO team chose well:
“I think we’re seeing volatility and I think we’re seeing some tightening, and that’s not unusual coming down close to the wire,” said Democratic strategist Crystal Fincher.
“I think we’re at 50–50, which is very surprising to me. I had no expectation of feeling that way even two weeks ago,” said Republican strategist Alex Hays.
Recent polls showing the race within a couple of points come from right-leaning polling firms.
“It’s not that close. The race is currently nine to 12 points, give or take,” said Andrew Villeneuve of the Northwest Progressive Institute. The institute commissioned a poll last month that showed Murray up 10 points.
He questions the methodology of the most recent polls, and says they’re timed to motivate Republicans to feel they’re on the cusp of victory.
“That is the strategy, it’s a self-fulfilling prophecy. It’s all a psychological game,” said Villeneuve.
The Republicans’ desperate polling gambit was a failure, and in no state are firms like Trafalgar more of a laughingstock than here in Washington.
Murray’s victory ultimately exceeded nine to twelve points, which came as no surprise to our team. We knew that those undecided voters had to go somewhere, and we figured there was a good chance they’d end up with Murray.
For today’s retrospective, the NYT published a chart plotting what it dubbed “nonpartisan” polling on one line and Republican-funded polling on another, showing that there was a significant divergence between the two.
I presume that the New York Times chose to exclude NPI and our statewide pollster’s research because it is considered “partisan” by FiveThirtyEight.
However, our work was just as credible and accurate as the “nonpartisan” polls. Only the Republican-funded polls were wildly off.
It’s a huge mistake to judge the credibility of a poll merely by the ideology (or lack thereof) of its sponsor. Supposedly “nonpartisan” pollsters can get it wrong, while subjective organizations are perfectly capable of producing objective research. It is essential to understand that the Republican-funded polls were not bad merely because they were produced by Republicans; they were bad because the firms that were putting them out weren’t following the scientific method.
It can be very difficult to discern which polls are credible and which are not, which is precisely why people turn to aggregators like FiveThirtyEight.
And, to be fair, FiveThirtyEight does grade pollsters based on their performance. It just hasn’t developed a good mechanism for screening garbage data out of its index, particularly from outfits like Trafalgar that have previously obtained “A” grades, but have no loyalty to the scientific method. FiveThirtyEight’s failure to respond in real time to Republicans’ data pollution practices is gravely concerning.
For the 2024 cycle, FiveThirtyEight and others need a better system in place for vetting polls and deciding what merits inclusion. Perhaps outliers and problematic polls can be hidden from the default view, but available in an expanded view. Such a practice could help eliminate right wing firms’ temptation to cheat, which would be good for everybody, but especially helpful to poll-attuned reporters.
Saturday, December 31st, 2022
NYT admits its midterms coverage was wrongly wedded to a “red wave” narrative
With 2022 drawing swiftly to a close, The New York Times today published an article with not one, not two, but three reporters’ bylines that examines the impact that garbage data from untrustworthy polls had on the 2022 midterms, including Washington State’s U.S. Senate race.
Titled “The ‘red wave’ washout: How skewed polls fed a false election narrative,” the story mainly focuses on Republican-aligned firms’ bad polls and how they landed, but makes an important admission in passing: newspapers like the New York Times bought wholesale into an electoral narrative that wasn’t real.
“Not for the first time, a warped understanding of the contours of a national election had come to dominate the views of political operatives, donors, journalists and, in some cases, the candidates themselves,” the story penned by the trio of Jim Rutenberg, Ken Bensinger, and Steve Eder notes after several introductory paragraphs dissecting the Republicans’ bad Evergreen State data.
A few paragraphs later came a more explicit admission — still in passing, but an important and necessary admission nonetheless — acknowledging that The New York Times’ own coverage was wrongly wedded to a “red wave” narrative:
Highlighting is mine.
For an example, the trio and their editors picked a pretentious column by Blake Hounshell, published October 19th. Titled “Democrats’ Feared Red October Has Arrived,” it began with this declaration: “Here’s the thing about elections: When they break, they usually break in one direction. And right now, all the indicators on my political dashboard are blinking red — as in, toward Republicans.”
Hounshell’s column was a well-chosen example. But it was actually just one of a great many pieces that the paper published over the course of weeks and months favoring Republicans and playing up their chances. We documented how The Times was failing its readers with its slanted, narrative-driven coverage right here on the Cascadia Advocate. (If you haven’t read that piece, it’s worth your time.)
On Election Night (Tuesday, November 8th), it was evident that while Republicans might be on their way to a House majority, there was no “red wave.” Democrats were doing far better than many pundits and reporters had expected. The New York Times promptly pivoted away from the narrative it had helped to promote at nearly every opportunity as its editors and reporters processed the data.
If you browse the NYT’s archives, you can see that Election Night was a huge demarcation point. Until November 8th, the paper’s midterms coverage was narrative-driven; after November 8th, the paper’s midterms coverage became data and reality-driven thanks to the availability of the initial returns.
Here on the Cascadia Advocate, we tried to elevate in our preelection coverage the voices of those who were looking carefully at the data and warning of the folly of drawing conclusions based on narratives, such as the New Democrat Network’s Simon Rosenberg and TargetSmart’s Tom Bonier.
Appropriately, both Rosenberg and Bonier are mentioned in this NYT story.
Rosenberg’s tussle with FiveThirtyEight founder Nate Silver on Twitter over the garbage Republican polling indexed by FiveThirtyEight is also mentioned.
“Mr. Silver did not respond to repeated requests for comment,” the story says.
Silver, himself employed for a stretch by The New York Times, is the author of a book called The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t. On one of FiveThirtyEight’s podcasts, Silver had ridiculously responded to Rosenberg’s criticisms by characterizing Rosenberg as smoking “hopium.” Of course, as events showed, Rosenberg’s concerns were well-founded, and he was entirely justified in declaring that FiveThirtyEight’s data had become polluted.
As October drew to a close, our team at NPI were likewise shaking our heads at the appearance of more and more garbage polls in Washington’s U.S. Senate race. There were so many bad polls that the FiveThirtyEight index was no longer showing any credible ones above the fold on its contest-specific list:
All of the polls shown above-the-fold on FiveThirtyEight as of the end of the voting period were from firms aligned with Republicans that erroneously showed a close race. To see any credible polls, it was necessary to click or tap “Show more polls.” (Screenshot)
Our own polling had consistently shown over the course of eighteen months that Patty Murray was on track to be comfortably reelected, yet the contest between Murray and her Republican challenger Tiffany Smiley was increasingly and incorrectly being described as “tight”, “close”, and “volatile.”
In a story that aired the week before the election, KIRO7’s Graham Johnson looked at the body of polling, bad and good lumped together, and talked to three people about what it could mean: consultant Crystal Fincher, Republican operative Alex Hays, and myself. During my interview with Graham, I explained that not all polling is created equal, and it’s essential to separate the wheat from the chaff.
Credible polling, I explained, had Murray up and well ahead of Smiley, whereas the garbage polling that incorrectly and improperly suggested Smiley was close to Murray was being funded and produced by Republicans.
Only a snippet of my comments made it into the final story that ran on-air.
But, as you can see, Graham and the KIRO team chose well:
The Republicans’ desperate polling gambit was a failure, and in no state are firms like Trafalgar more of a laughingstock than here in Washington.
Murray’s victory ultimately exceeded nine to twelve points, which came as no surprise to our team. We knew that those undecided voters had to go somewhere, and we figured there was a good chance they’d end up with Murray.
For today’s retrospective, the NYT published a chart plotting what it dubbed “nonpartisan” polling on one line and Republican-funded polling on another, showing that there was a significant divergence between the two.
I presume that the New York Times chose to exclude NPI and our statewide pollster’s research because it is considered “partisan” by FiveThirtyEight.
However, our work was just as credible and accurate as the “nonpartisan” polls. Only the Republican-funded polls were wildly off.
It’s a huge mistake to judge the credibility of a poll merely by the ideology (or lack thereof) of its sponsor. Supposedly “nonpartisan” pollsters can get it wrong, while subjective organizations are perfectly capable of producing objective research. It is essential to understand that the Republican-funded polls were not bad merely because they were produced by Republicans; they were bad because the firms that were putting them out weren’t following the scientific method.
It can be very difficult to discern which polls are credible and which are not, which is precisely why people turn to aggregators like FiveThirtyEight.
And, to be fair, FiveThirtyEight does grade pollsters based on their performance. It just hasn’t developed a good mechanism for screening garbage data out of its index, particularly from outfits like Trafalgar that have previously obtained “A” grades, but have no loyalty to the scientific method. FiveThirtyEight’s failure to respond in real time to Republicans’ data pollution practices is gravely concerning.
For the 2024 cycle, FiveThirtyEight and others need a better system in place for vetting polls and deciding what merits inclusion. Perhaps outliers and problematic polls can be hidden from the default view, but available in an expanded view. Such a practice could help eliminate right wing firms’ temptation to cheat, which would be good for everybody, but especially helpful to poll-attuned reporters.
# Written by Andrew Villeneuve :: 1:15 PM
Categories: Elections, Media & Culture
Tags: Media Criticism, Research Polling Retrospectives, WA-Sen
Comments and pings are currently closed.