It’s almost 2018. For some, that means looking forward to the new year — what are the big political storylines to watch in the coming year? But I can’t do that until we finish one last end-of-year tradition: remembering all the times I screwed up!
That’s right: It’s time for my annual mea culpa column. Happily, this year’s column isn’t nearly as painful as 2016’s, and certainly not 2015’s, the year I laughed off Donald Trump’s entrance into the presidential race. Still, there’s always stuff that I wish I had written better, written differently or not written at all.
First, I think I undersold the chances of Republicans passing a health care bill. Of course, they didn’t pass a health care bill. The various bills they considered were super unpopular and the GOP’s math in the Senate was always difficult, so we were skeptical that Republicans would pass anything. Probably a little too skeptical, however.
Take a look at a couple of these headlines: “The GOP Health Care Bill May Have Found A Better Way To Fail” and “Trump’s Health Care Bill Won Over The Freedom Caucus — But Risks Losing Everyone Else.” For one, it didn’t alienate everyone else — it passed the House. And later, it was just one John McCain thumb away from passing the Senate too. So, I don’t think the GOP health care bill was destined to fail. After all, only recently Republicans passed another unpopular bill.
Screw-up No. 2 came in my preview of Virginia’s gubernatorial primaries. That article included a “survey” from “CSP Polling.” It shouldn’t have. I had never heard of CSP Polling, which showed up in May. And, as I wrote later — in “Fake Polls Are A Real Problem” — I could never find any person willing to go on the record with his or her actual name to stand behind the work. That’s obviously a big problem. We tend to take an inclusive attitude toward pollsters here at FiveThirtyEight, but any legitimate pollster should have real, identifiable people behind it.
Including one stray suspect poll might seem minor, but it’s not. A lot of people come to FiveThirtyEight for guidance on polling. We let them down in this instance. In 2018, we need to be extra vigilant on this front. There are probably going to be a ton of competitive House races. And most of them are unlikely to be surveyed by “gold-standard” pollsters. In other words, it’ll be tempting to grasp at any poll that pops up in a lot of these districts. Unfortunately, many will probably be suspect or shoddy. I put together some advice on how not to fall for these polls, and I would do well to heed it.
OK, screw-up No. 3 is a little wonky but still important. In early January, I wrote “Registered Voters Who Stayed Home Probably Cost Clinton The Election.” The main point of this piece relied on a SurveyMonkey poll. That data was fine, and I don’t think anything in the article was wrong, really. But it was thin. I should have used a voter file too.
Some pollsters use random digit dialing to do surveys — call a random phone number and ask that person if they’re registered to vote. Other pollsters use voter files, lists of registered voters with a bunch of information about each one, to construct their samples. I think a mixed approach to surveying (e.g. combining random digit dialing and a registered voter list) is probably best. And that goes for a lot of analyses too. That article on how much registered voters who stayed home hurt Hillary Clinton should have used more than self-reported voting status, which can be unreliable at times. I wish I had supplemented it with voter file data.
One of the big questions for the 2018 midterm elections is what will turnout look like. The evidence so far suggests Democrats will do far better turning out voters than they did in the 2010 or 2014 elections. How much better? We don’t know. So let’s not make too many assumptions about who will turn out in 2018 based on previous midterm elections. Voter lists will help us understand just how different the turnout patterns are in 2018.
Screw-up No. 4 is an error of omission. In July, I wrote, “Red-State Senate Democrats Haven’t Drawn Strong Opponents — Yet.” That was true at the time. But it’s not true anymore, and it hasn’t been for some time. In fact, the GOP has a number of fairly good candidates running in red states with Democratic senators, including Indiana and Missouri. You wouldn’t know that, though, if you were just reading FiveThirtyEight. That article deserved a follow-up.
Often, it’s not the bad articles you write that get you in trouble. Instead, it’s the articles you don’t write.
So that’s that. I’m sure there were other things I messed up in 2017, but these are the errors that still gnaw at me. As is tradition, however, let’s close out this column on a more hopeful note. Here are a few things I think I got right:
- I never lost faith in the laws of political gravity. After Trump’s election, it became fashionable to say that “nothing matters,” that the normal rules of politics don’t apply anymore. I never thought that was the case. And, as we’ve seen in all types of elections in 2017, it’s not.
- I’m now 2-for-2 — or 3-for-3, depending on how you count — in pulling out the “normal polling error” card. The idea here is that many people treat polling as an exact science. It’s not. Instead, it’s perfectly normal for polls to miss the result by a lot. So, just before the 2016 election I pointed out that Trump was only a normal polling error behind Clinton. And just before the special Alabama Senate election, I wrote that Democrat Doug Jones was just a normal polling error behind Republican Roy Moore.
- Finally, here’s an article I’m proud of: “Fake Polls Are A Real Problem.” After a bunch of media outlets cited a Delphi Analytica poll of the 2018 Michigan Senate race showing Kid Rock winning, I spent weeks sifting through the survey’s data and trying to track down the people behind Delphi Analytica. The resulting article showed just how careful journalists need to be in citing pollsters they’ve never heard of.
Thanks for reading. I’ll see you all in 2018!